Shell script is to help programmers and system administrators to complete time-consuming and laborious boring work, is an effective way to interact with the computer and manage files and system operations. A few lines of code can get the computer close to doing what you want it to do.
We sorted out 23 examples, through 23 actual combat classic script examples, showed the shell scripting practical technology and common tool usage. All you need to do is generalize these common tasks and portable automation scripts to other similar problems that will solve those recurring problems.
Check file consistency in the specified directory on the two servers
#! / bin/bash # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # to detect two servers specified directory file consistency # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # Dir =/data/web b_ip=192.168.88.10 dir=/data/web b_ip=192.168.88.10 And written to the specified file find $dir -type f | xargs md5sum > / TMP/md5_a. TXT SSH $b_ip "find $dir -type f | xargs md5sum > / TMP/md5_b. TXT" For f in 'awk '{print 2} / TMP /md5_a. TXT' 'do # When b machine directly when there is no traverse object files in the output does not exist if the result of the grep -'ve "$f"/TMP/md5_b. TXT then md5_a = ` grep -w "$f"/TMP/md5_a. TXT | awk '{1} print ` Md5_b = ` grep -w "$f"/TMP/md5_b. TXT | awk '{1} print ` # when file exists, if inconsistent md5 value is the result of a change in the output file if [$md5_a! = $md5_b ] then echo "$f changed." fi else echo "$f deleted." fi doneCopy the codePeriodically empty file contents and record file sizes
#! / bin/bash # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # execute a script (task plan) per hour, when the time for zero or twelve o 'clock, Empty all files in the target directory, but do not delete files. At other times, only count the size of each file, one line per file, output to the file named with the date and time. Need to consider the target directory under the secondary and tertiary DengZi directory file # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # logfile = / TMP / ` H - date + % % F `. The log N = ` date + % H ` if [$n - eq 00] | | [$n - eq 12] then # by the for loop, as a condition of traverse to find command, For I in 'find /data/log/ -type f' do true > $I done else for I in 'find /data/log/ -type f' do du -sh $i >> $logfile done fiCopy the codeDetects network adapter traffic and records it in logs in a specified format
#! / bin/bash # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # detection of network traffic, and recorded in the log as a specified format # # once a minutes record log format prescribed as follows: #2019-08-12 20:40 #ens33 input: 1234bps #ens33 output: 1235bps ######################################################3 while : Do # set the language to English to ensure that the output is In English. LANG=en logfile=/ TMP/' date +%d '.log # exec >> $logfile date +"%F %H:%M" The unit of traffic statistics for SAR command is KB /s, and the log format is BPS. So to 59 * 1000 * 8 SAR - n DEV 1 | grep business | grep ens33 | awk '{print $2,"\t","input:","\t",$5*1000*8,"bps","\n",$2,"\t","output:","\t",$6*1000*8,"bps"}' echo "####################" Sleep done is not required because SAR command execution takes 59 secondsCopy the codeCount the number of digits in each line of the document and count the total number of digits in the entire document
#! / bin/bash # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # to calculate the number of digits in a document appear in a row, And calculate the total number of digital document # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # use awk only output lines (interception of the first paragraph) n = ` wc -l a.t xt | awk '{print $1}' 'sum=0 # for I in' seq 1 $n 'do # for I in' seq 1 $n 'do # for I in' seq 1 $n 'do # for I in' seq 1 $n 'do # for I in' seq 1 $n 'do # Statistics the longest length n_n = ` echo $line | sed s' / [^ 0-9] / / 'g | wc -l ` echo $n_n sum = $[$sum + $n_n] done echo "sum: $sum"Copy the codeKill all scripts
#! / bin/bash # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # has some scripts to join into the cron, existing script has not yet run out and new tasks need to be performed, # Cause the system load to increase, so you can write a script to filter out processes that affect the load and kill them all at once. # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # ps aux | grep specify the process name | grep -v grep | awk '{print $2}' | xargs kill -9Copy the codeDownload files from the FTP server
#! /bin/bash if [ $# -ne 1 ]; then echo "Usage: $0 filename" fi dir=$(dirname $1) file=$(basename $1) FTP -n -v << EOF # -n Automatic login Open 192.168.1.10 # FTP server user admin CD $dir get "$file" EOF $dir get "$file" EOFCopy the codeEnter 5 consecutive numbers up to 100, counting sum, minimum and maximum
#! /bin/bash COUNT=1 SUM=0 MIN=0 MAX=100 while [ $COUNT -le 5 ]; Do read -p "please enter 1-10 integers:" INT the if [[! $INT = ~ ^ [0-9] + $]]. Then echo "Input must be an integer! exit 1 elif [[ $INT -gt 100 ]]; Then echo "Input must be within 100! exit 1 fi SUM=$(($SUM+$INT)) [ $MIN -lt $INT ] && MIN=$INT [ $MAX -gt $INT ] && MAX=$INT let COUNT++ done echo "SUM: $SUM" echo "MIN: $MIN" echo "MAX: $MAX"Copy the codeThe user guesses the number
#! /bin/bash # Generates a random number within 100, prompting the user to guess the number. Based on the user's input, prompting the user to guess the number,# guess small or big, until the user guesses the number. ‐100 num=$[RANDOM%100+1]echo "$num" # use read to ask user to guess number # Use if ‐eq(equal to),‐ne(not equal to),‐gt(greater than),‐ge(greater than or equal to),# ‐lt(less than),‐le(less than or equal to)while :do read -p "The computer generates a random number of 1‐100, guess: $CAI -eq $num] then echo "exit elif [$CAI -gt $num] then echo "Oops fidoneCopy the codeMonitor Nginx access log 502 and act accordingly
Assume that the server environment is LNMP, and 502 occurs frequently in recent access, and the 502 error disappears after the PHP-FPM service is restarted. Therefore, a monitoring script needs to be written. Once 502 occurs, the phP-FPM service will be restarted automatically.
Log file access path: /data/log/access.log #2. The script checks in an infinite loop every 10 seconds. The number of logs in 10 seconds is 300. If the ratio of 502 is at least 10% (30 logs), you need to restart phP-fPM service #3. D/PHP -fpm restart #! / bin/bash # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # 502 monitoring Nginx access log book, And do the corresponding action # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # log = / data/log/access. To set threshold value while the log N = 30 # : Do # View the latest 300 entries in the access log, And statistics of 502 the number of times the err = ` tail - $300, log n | grep -c '502 "' ` if ge $n] [$err - then/etc/init. D/PHP - FPM restart 2 > / dev/null PHP - FPM service sleep 60 fi sleep 10 doneCopy the codeAssign the results to the variables separately
Application scenario: You want to assign execution results or positional parameters to variables for later use. For I in $(echo "4 5 6"); Do eval a$I =$I done echo $a4 $a5 $a6 do eval a$I =$I done echo $a4 $a5 $a6 Let num+=1 eval node${num}="$I "done echo $node1 $node2 $node3 # bash a.sh 192.168.1.1{1,2} 192.168.1.11 192.168.1.12 method 3: arr = (4 5 6) INDEX1 = $(echo ${arr [0]}) INDEX2 = $(echo ${arr [1]}) and INDEX3 = $(echo ${arr [2]})Copy the codeChanging file Names in Batches
Example: # touch article_{1.. 3}.html # ls article_1.html article_2.html article_3.html do mv $file bbs_${file#*_} # mv $file $(echo $file |sed -r 's/.*(_.*)/bbs\1/') # mv $file $(echo $file |echo bbs_$(cut -d_ -f2) done method 2: for file in $(find. -maxdepth 1-name "* HTML "); Do mv $file bbs_${file#*_} done method 3: # rename article BBS *.htmlCopy the codeDelete all letters in the first five lines of a document and all letters in lines 6 to 10
1) Prepare a test file named 2.txt
Line 1 1234567 without the letters Line 2 56789BBBBBB Line 3 67890CCCCCCCC Line 4 78ASDFDDDDDDDDDDD Line 5 123456EEEEEEEE Line 6 1234567ASDF line 7 56789ASDF Line 8 67890ASDF line 9 78asdfADSF line 10 123456AAAA Line 11 67890ASDF line 12 78asdfADSF line 13 123456AAAACopy the code2) The script is as follows:
#! / bin/bash # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # to a document contained in the letter of the five elements to delete, At the same time deleted from six to 10 lines contain all the letters # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # sed - n '1, 5 p 2. TXT | sed' / [a zA - Z] / 'd Sed - n '6, 10 p 2. TXT | sed s' / [a zA - Z] / /' g sed - n '$' 11, p 2. TXT # end result is printed on the screen as a result, if you want to change the file directly, the output can be written to a temporary file, and then replace 2. TXT or using the -i optionCopy the codeStatistics the total size of files ending in. HTML in the current directory
# find. -name "*.html" -exec du -k {} \; | awk 'END {sum + = $1} {print the sum}' method 2: for the size in $(ls - l *. HTML | awk '{print $5}); do sum=$(($sum+$size)) done echo $sumCopy the codeScan the status of host ports
#! /bin/bash HOST=$1 PORT="22 25 80 8080" for PORT in $PORT; do if echo &>/dev/null > /dev/tcp/$HOST/$PORT; then echo "$PORT open" else echo "$PORT close" fi doneCopy the codePrint the words with less than 6 letters in the sample statement with the shell
#Bash also interprets a number of multi-character options. #! / bin/bash # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # shell print sample statement letter number less than 6 words ############################################################## for s in Bash also interprets a number of multi-character options. do n=`echo $s|wc -c` if [ $n -lt 6 ] then echo $s fi doneCopy the codeEnter a number to run the corresponding command
#! / bin/bash # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # input digital operation commands accordingly ############################################################## echo "*cmd menu* 1-date 2-ls 3-who 4-pwd 0-exit " while : The do # capture user type value read -p "both please input the number:" n n1 = ` echo $n | sed s' / [0-9] / / g ` # empty input check if [-z "$n] then continue fi If [-n "$n1"] then exit 0 fi break done case $n in 1) date; 2) ls ;; 3) who ;; 4) pwd ;; 0) break ;; Echo "Please input number is [1-4]" esACCopy the codeExpect implements SSH interactive command execution
Expect is a tool for automatic interactive applications such as Telnet, FTP, passwd, and so on. You need to install the Expect software package. Method 1: EOF standard output as expect standard input #! /bin/bash USER=root PASS=123.com IP=192.168.1.120 expect << EOF set timeout 30 spawn SSH $USER@$IP expect {"(yes/no)" {send "yes\r"; exp_continue} "password:" {send "$PASS\r"} } expect "$USER@*" {send "$1\r"} expect "$USER@*" {send "exit\r"} expect eof EOF method 2: #! /bin/bash USER=root PASS=123.com IP=192.168.1.120 expect -c "spawn SSH $USER@$IP expect {\"(yes/no)\" {send \"yes\r\"; exp_continue} \"password:\" {send \"$PASS\r\"; exp_continue} \"$USER@*\" {send \"df -h\r exit\r\"; Exp_continue}}" method 3: separate expect script: # cat login.exp #! /usr/bin/expect set ip [lindex $argv 0] set user [lindex $argv 1] set passwd [lindex $argv 2] set cmd [lindex $argv 3] if { $argc ! = 4 } { puts "Usage: expect login.exp ip user passwd" exit 1 } set timeout 30 spawn ssh $user@$ip expect { "(yes/no)" {send "yes\r"; exp_continue} "password:" {send "$passwd\r"} } expect "$user@*" {send "$cmd\r"} expect "$user@*" {send "exit\r"} expect Eof execute command script: write a loop to batch operate multiple servers #! /bin/bash HOST_INFO=user_info.txt for ip in $(awk '{print $1}' $HOST_INFO) do user=$(awk -v I="$ip" 'I==$1{print $2}' $HOST_INFO) pass=$(awk -v I="$IP "'I==$1{print $3}' $HOST_INFO) expect login.exp $IP $user $pass $1 done # cat user_info.txt 192.168.1.120 root 123456Copy the codeCreate 10 users and set their passwords. The passwords must contain 10 characters of uppercase and lowercase letters and digits. Save the passwords of each user in a specified file
#! / bin/bash # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # to create 10 users, and set the password, The password must contain 10 characters of uppercase and lowercase letters and digits. The password of each user must be saved in a specified file. Install mkpasswd command # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # to generate the sequence of 10 users (00-09) for u in ` seq -w 0 09 ` do # # to create user useradd user_ $u generated password p = ` mkpasswd -s 0 to 10 ` # l read from standard input password modification (unsafe) echo $p | passwd -- stdin user_ $u # regular change passwords echo - e "$p $p" \ n | passwd user_ $# u will create the user and the corresponding password record to the log file echo "user_ $u $p > > / TMP/userpassword doneCopy the codeMonitor the number of HTTPD processes and handle them accordingly
#! /bin/bash # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # ####### # Required: #1. Monitor the number of HTTPD processes every 10 seconds. If the number of HTTPD processes is greater than or equal to 500, the Apache service automatically restarts and checks whether the service is restarted successfully. If the system fails to restart, it needs to restart again. If the system fails to restart for five times, it sends an alarm email to the administrator and exits detection #3. If yes, wait 1 minute and check the number of HTTPD processes again. If yes, check the number of HTTPD processes again every 10 seconds. Otherwise, abort the restart and send an alarm email to the administrator. And exit detection # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # check_service counter function () {j = 0 for I ` in seq # 1 5 ` do restart Apache command/usr/local/apache2 / bin/apachectl restart 2 > If [$? -eq 0] then break else j=$[$j+1] then if [$j-eq 5] then mail.py exit fi done } while : Do n = ` pgrep -l HTTPD | wc -l ` # whether service HTTPD process for more than 500 if [$n - gt 500] then/usr/local/apache2 / bin/apachectl restart if [$? - ne 0] then check_service else sleep (n = 60 ` pgrep -l HTTPD | wc -l ` # whether after the restart still more than 500 if [$n2 - gt 500] then mail. Py exit Check sleep 10 done every 10sCopy the codeChange server user passwords in batches
Linux host SSH connection information: old password # cat old_pass. TXT 192.168.18.217 root 123456 22 192.168.18.218 root 123456 22 IP User Password Port SSH remote modify Password script: new Password is randomly generated https://www.linuxprobe.com/books #! /bin/bash OLD_INFO=old_pass.txt NEW_INFO=new_pass.txt for IP in $(awk '/^[^#]/{print $1}' $OLD_INFO); do USER=$(awk -v I=$IP 'I==$1{print $2}' $OLD_INFO) PASS=$(awk -v I=$IP 'I==$1{print $3}' $OLD_INFO) PORT=$(awk -v I=$IP Echo "$IP $USER $NEW_PASS $PORT" >> $NEW_INFO expect -c "$IP $USER $NEW_PASS $PORT" >> $NEW_INFO expect -c" spawn ssh -p$PORT $USER@$IP set timeout 2 expect { \"(yes/no)\" {send \"yes\r\"; exp_continue} \"password:\" {send \"$PASS\r\"; exp_continue} \"$USER@*\" {send \"echo \'$NEW_PASS\' |passwd --stdin $USER\r exit\r\"; Exp_continue}}" done generate new password file: # cat new_pass. TXT 192.168.18.217 root n8wX3mU% 22 192.168.18.218 root c87; ZnnL 22Copy the codeIptables automatically blocks IP addresses that frequently visit websites
Scenario: Malicious access, security prevention 1) Mask IP addresses that access more than 200 IP addresses per minute method 1: Based on the access log (for example, Nginx) #! /bin/bash DATE=$(date +%d/%b/%Y:%H:%M) ABNORMAL_IP=$(tail -n5000 access.log |grep $DATE |awk '{a[$1]++}END{for(i in A)if(a[I]>100)print I}') # Awk cannot filter logs directly because they contain special characters. for IP in $ABNORMAL_IP; do if [ $(iptables -vnL |grep -c "$IP") -eq 0 ]; Then iptables -I INPUT -s $IP -j DROP fi done /bin/bash ABNORMAL_IP=$(netstat -an |awk '$4~/:80$/ && $6~/ESTABLISHED/{gsub(/:[0-9]+/,"",$5); {a[$5]++}}END{for(I in a)if(a[I]>100)print I}') #gsub do if [ $(iptables -vnL |grep -c "$IP") -eq 0 ]; Then iptables -i INPUT -s $IP -j DROP fi done /bin/bash DATE=$(DATE +"%a %b %e %H:%M") And % d display 07 ABNORMAL_IP = $(lastb | grep "$DATE" | awk '+ +} {a ($3) END {for (I a) in the if (a [I] > 10) print I}') for IP in $ABNORMAL_IP; do if [ $(iptables -vnL |grep -c "$IP") -eq 0 ]; Then iptables -I INPUT -s $IP -j DROP fi done /bin/bash DATE=$(date +"%b %d %H") ABNORMAL_IP="$(tail -n10000 /var/log/auth.log |grep "$DATE" |awk '/Failed/{a[$(NF-3)]++}END{for(i in a)if(a[i]>5)print i}')" for IP in $ABNORMAL_IP; do if [ $(iptables -vnL |grep -c "$IP") -eq 0 ]; then iptables -A INPUT -s $IP -j DROP echo "$(date +"%F %T") - iptables -A INPUT -s $IP -j DROP" >>~/ssh-login-limit.log fi doneCopy the codeAccording to web access logs, the IP address whose request volume is abnormal is blocked. If the IP address is restored half an hour later, the IP address will be unblocked
#! / bin/bash # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # according to the web access log, banned requested IP anomaly, If the IP address recovers after half an hour, Is lifted ban # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # logfile = / data/log/access. Log D1 = 'date -d "-1 minute" +%H%M' d2= 'date +%M' ipt=/sbin/iptables ips=/ TMP /ips.txt block() { # to a minute ago all log filtering and extracting IP and statistical visited grep '$d1:' $logfile | awk '{print $1}' | sort - n | uniq -c | sort - n > $ips For I in 'awk '$1>100 {print $2}' $ips' do $ipt -i INPUT -p TCP --dport 80 -s $i-j REJECT echo "' date +%F-%T '$I" >> / TMP /badip.log done} unblock() INPUT - line - Numbers | grep '0.0.0.0/0 | awk' $2 < 10} {print $1 '| sort - nr ` do INPUT $$ipt - D a done $ipt - Z} # when time at 00 and 30 to perform the unlock function if [$d2 - eq "00"] | | [$d2 - eq "30"] then # to solution first, then seal, because the number that, when just banned PKTS rarely unblock block the else block fiCopy the codeCheck whether the entered IP address is an IP address
Method 1: #! /bin/bash function check_ip(){ IP=$1 VALID_CHECK=$(echo $IP|awk -F. '$1< =255&&$2<=255&&$3<=255&&$4<=255{print "yes"}') If echo $IP | grep - E ^ "[0-9] {1, 3} \. [0-9] {1, 3} \. [0-9] {1, 3} \. [0-9] {1, 3} $" > / dev/null; then if [ $VALID_CHECK == "yes" ]; then echo "$IP available." else echo "$IP not available!" fi else echo "Format error!" Fi} check_ip 192.168.1.1 check_ip 256.1.1.1 / bin/bash function check_ip () {IP = $1 if [[$IP = ~ ^ [0-9] {1, 3} \. [0-9] {1, 3} \. [0-9] {1, 3} \. [0-9] {1, 3} $]]. then FIELD1=$(echo $IP|cut -d. -f1) FIELD2=$(echo $IP|cut -d. -f2) FIELD3=$(echo $IP|cut -d. -f3) FIELD4=$(echo $IP|cut -d. -f4) if [ $FIELD1 -le 255 -a $FIELD2 -le 255 -a $FIELD3 -le 255 -a $FIELD4 -le 255 ]; then echo "$IP available." else echo "$IP not available!" fi else echo "Format error!" Fi} check_ip 192.168.1.1 check_ip 256.1.1.1 added version: add a loop, if the IP is available to exit, not available to continue to enter, and use awK judge. #! /bin/bash function check_ip(){ local IP=$1 VALID_CHECK=$(echo $IP|awk -F. '$1< =255&&$2<=255&&$3<=255&&$4<=255{print If "yes"} ') echo $IP | grep -e "^ [0-9] {1, 3} \. [0-9] {1, 3} \. [0-9] {1, 3} \. [0-9] {1, 3} $" > / dev/null. then if [ $VALID_CHECK == "yes" ]; then return 0 else echo "$IP not available!" return 1 fi else echo "Format error! Please input again." return 1 fi } while true; do read -p "Please enter IP: " IP check_ip $IP [ $? -eq 0 ] && break || continue doneCopy the code