Implement Python automation in Linux environment

Today, the main introduction of Cron under Linux this service, skilled use of it can let you save most of the time at work, improve work efficiency

Install crON

Basically all Linux distributions have the cron tool preinstalled by default. Even if cron is not pre-installed, it is simple and can be manually installed by executing a few simple commands

Check whether cron Service cron Status is preinstalledCopy the code

The default is generally the following, indicating that cron is ready to use

root@vmi411292:~# servide cron status -bash: servide: Command not found root@vmi411292:~# service cron status ● cron. service-regular background program processing daemon Loaded: loaded (/lib/systemd/system/cron.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2020-07-01 04:36:42 CST; 5 months 2 days ago Docs: man:cron(8) Main PID: 487 (cron) Tasks: 1 Memory: 10.0m CPU: 25min 11.945s CGroup: / system. Slice/cron service └ ─ 487 / usr/sbin/cron - fCopy the code

If not, enter the following table

Installation: apt-get install cron Start /stop/restart: service cron start/stop/restart Query the current task: crontab -lCopy the code

2. cron

There are a few simple ways to use cron, and a case study will explain how to use it in detail

First, list the current user’s planned cron jobs:

crontab -l
Copy the code

To view cron jobs for other users:

Crontab - l - u usernameCopy the code

To remove a planned CRon job:

Crontab -r # Note that this command removes all the configured automation commands. To delete one, use crontab -e to edit itCopy the code

Now that you’re familiar with cron viewing, let’s look at the syntax

m h dom mon dow commmand
Copy the code

This is an example of a crontab scheduling job that can be used to set scheduled tasks. The syntax is as follows:

M h dom mon dow command * * * * * command -- -- -- -- -- - | | | | | | | | | | | - advance execution command | | | | -- -- -- -- -- Said week 0 ~ 7 (which can be used on Sunday said 0 or 7) | | | -- -- -- -- -- -- -- said in 1 ~ 12 | | -- -- -- -- -- -- -- -- -- said date 1 ~ 31 | -- -- -- -- -- -- -- -- -- -- -- said hours 1 ~ 23 (0, 0) -- -- -- -- -- -- -- -- -- -- -- -- -- Minutes 1 to 59 Minutes are expressed in * or */1Copy the code

A few simple application cases:

  • Perform tasks at 04:00 every day
0 4 * * * command
Copy the code
  • 5 and 17:00 daily
0 5,17 * * * command
Copy the code
  • A task is executed every 10 minutes
*/10 * * * * command
Copy the code
  • Operate at 17:00 on Sundays in certain months
0 17 * jan,may,aug sun command
Copy the code

In the example above, command represents the specific task you need to perform, such as printing a paragraph:

echo "Hello World"
Copy the code

Or output this paragraph to TXT:

echo "Hello World" >> /tmp/test.txt 
Copy the code

Or you need to execute a Python script:

python demo.py filepath
Copy the code

Three, regular task combat

Now that we’ve cleared up, we can get on with the big show of the day

First of all, we need to download the latest task data from the FTP server every day, download the data to the local computer, collect statistics through Python, and finally save the results in the database. If there is a problem in a certain link, we will send an alarm email.

1. Write Python scripts

First, you need to write a Python script to do the following:

  • Gets the latest data date from the database
  • Download the latest data from the FTP server to the local PC
  • Collect the latest local data
  • The statistical results are stored in the database for summary
  • Email notification

The pseudocode for the above process looks like this:

if __name__ == '__main__': "" latest_date = get_max_date() # create a folder with the name of the latest date download_dir = os.path.join(sys.argv[1], latest_date) if not os.path.exists(download_dir): Os.makedirs (download_dir) """ Download the latest data from FTP """ download_file(latest_date, "" process_data(latest_date, download_dir) """ Process_data (latest_date, download_dir)Copy the code

2. Write a cron schedule task. Open crontab, edit the following content to the last line, save and exit

Crontab automatically updates the task list in real time. You can also restart the cron service by running the restart command

05 8,18 * * * python /home/demo.py /home/source data for scheduled tasksCopy the code

Here’s a tip: Use absolute paths for all paths

If the Python code is fine, the task executes regularly.

It is recommended that you run your command separately in the console and write it in the cron task list when there is no problem.

Four,

Cron is very convenient if you have a lot of repetitive tasks in your daily work, such as collecting daily indicators, collecting statistics, automatically forwarding emails, etc

Once you can set up the logic in a script, automated tasks can do it. At most, you need to check your email every day to see if anything went wrong.

Five, digression

Sometimes if your server was hacked, you can also use cron – l see hackers have on your server to perform automated tasks regularly, because most hackers with injection timing to perform his script, use your computer to dig, and so on, small make up had suffered the hacker’s attack for several times, are interested in this topic you can talk to you next time, Interested friends after reading remember to click a like to go oh, code word is not easy haha