If you need Python code to monitor server resource information, such as content usage, CPU consumption, and alerts when resources are low, this article is for you.

Sweet sweet taste, have you tried? It's hot. Have you tried it? Have you ever heard the songs of Internet celebrities? Python system resource information retrieval tool, have you used? Real grilled gluten can be a kick! ~~~ let you eat real benefits! Different taste!!Copy the code

Hail the unyielding man (Gluten Brother Cheng Shulin)

Tool is introduced

Psutil (Process and System Utilities) is a cross-platform library, github, official documentation

We can use it to see the system running process and resource utilization. It is mainly used for system monitoring, process resource analysis and limitation, and operation process management. The main functional structure of PSUtil is shown below

It has three functional modules, namely System Related Function, Processes and Windows Service.

Psutil implements many of the features provided by UNIX command line tools, such as TOP, free, netstat, kill, and more, and is compatible with various operating systems:

The installation of psutil

We can read psutil’s official documentation while writing code so that we can learn faster.

To install in Python, use the PIP command directly:

pip install psutil
Copy the code

However, if you only want the current user to be available, rather than the entire system, you’ll need to adjust your commands:

pip install -user psutil
Copy the code

Wget, curl, etc. See the install section of the documentation for details, but generally the PIP install psutil command is enough.

System-related operations

It provides us with some common operation content of the system to refer to the documentation. This includes CPU, memory, disk, network, sensors, and processes.

CPU monitoring

You can use the CPU_times module to view CPU resources, for example:

>>> import psutil
>>> psutil.cpu_times()
scputimes(user=477.29, nice=0.0, system=262.86, idle=6074.83)
Copy the code

What does the parameter cpu_times mean? Specific definitions are also given in the document

It provides users with more detailed information monitoring:

Returns the system CPU time as a named tuple. Each attribute represents the number of seconds the CPU spent in a given mode. The availability of attributes varies by platform: user: the time taken by a normal process to execute in user mode; On Linux, this also includes visitor time System: time spent by a process executing in kernel mode Idle: time spent idle Platform-specific fields: Nice (UNIX) : time spent by a NICed (priority) process executing in user mode; On Linux, this also includes guest_nice time IOWAIT (Linux) : the time spent waiting for I/O to complete IRQ (Linux, BSD) : the time spent on service hardware interruption softirQ (Linux) : Steal (Linux 2.6.11+) : steal (Linux 2.6.24+) : Steal (Linux 2.6.24+) : Steal (Linux 2.6.24+) : Time spent running virtual CPUS for guest operating systems under the control of the Linux kernel guest_nice (Linux 3.2.0+) : Time spent running the Niced guest VIRTUAL machine (the virtual CPU of the guest operating system under the control of the Linux kernel) interrupt (Windows) : time taken to service hardware interrupts (similar to "IRQ" on UNIX) DPC (Windows) : The service delays the time taken by the procedure call (DPC); A DPC is an interrupt that runs at a lower priority than a standard interrupt.Copy the code

If you look down, you can also see other CPU monitoring modules, such as cpu_percent(interval=None, perCPU =False), which returns a floating point number representing the current system-wide CPU utilization percentage. Compare the system CPU time (blocking) before and after the interval when the interval is > 0.0. Returns immediately when interval is 0.0 or None comparing system CPU time since the last call or module import. This means that the first call to it will return a meaningless value of 0.0, which you should ignore. In this case, it is recommended that 0.1 call this function at least accurately between calls. Returns a list of floating point numbers representing utilization when percpu is True, expressed as a percentage percpu. The first element in the list is the first CPU, the second element is the second CPU, and so on. The order of the list is consistent between calls.

The number of CPU cores can be viewed with cpu_count(), which returns the number of cpus in the system (same as os.cpu_count).

My computer has an I3 CPU, but I got a 4 CPU (I should have gotten a 2 CPU), why?

Cpu_count () returns the logical number of cpus, plus the physical number of cpus (logical = True).

Small demo

If we wanted to get a usage ranking similar to the top command, what would we do? Given the cpu_percent we learned earlier, let’s try:

psutil.cpu_percent(interval=3, percpu=True)
Copy the code

Interval indicates the system CPU time before and after the comparison interval, that is, the CPU usage in the 3-second interval

If we want to implement ranking, then we need a lot of records, which means we can try to do this through the for loop (where the interval is set to 1) :

Of course, this is only output 10 times, if you want to rank, you need to do something else.

The figure above is a screenshot of the resource information of my computer. Combined with the screenshot, it can be seen that the system resource information obtained through the code is relatively reliable

For additional information about CPU retrieval, check out the CPU section of the psutil official documentation, and then learn about memory retrieval.

Obtaining memory Information

Virtual_memory () returns statistics about system memory usage as a named tuple, including the following fields in bytes. Main indicators:

Total: Total physical memory. Available: Memory that can be made available to a process immediately without the system entering the swap. This is calculated by summing up different memory values based on platform, and it should be used to monitor real memory usage in a cross-platform manner. Other indicators: Used: Used memory. It is calculated based on different platforms and is for reference only. Total - free does not necessarily match use. Free: Memory is not used (to zero) and is always available. Note that this does not reflect the actual memory available (use it instead). Total - Use does not necessarily match free. Active (UNIX) : Memory currently in use or recently used, so it is in RAM. Inactive (UNIX) : Memory marked as unused. Buffers (Linux, BSD) : Caching file system metadata, that sort of thing. Cached (Linux, BSD) : Cache various things. Shared (Linux, BSD) : Memory that can be accessed by multiple processes at the same time. Slab (Linux) : Kernel data structure cache. Wired (BSD, macOS) : Memory marked as always retained in RAM. It never moves to disk.Copy the code

Of course, we don’t care about detailed memory information, just the amount of content, current available, or current used.

Small demo of out of memory alert

Now that there is a requirement for code to warn when less than 300M of memory is monitored, we can simplify the requirement to: out of memory, alert

Of course, the above code doesn’t really warn us, because it just shows you what it can do. If you want to monitor, you need to write more code and design better processing logic.

Other modules for obtaining memory information are introduced

Swap_memory () returns system switched memory statistics as a named tuple:

Total: total swap memory (in bytes) Used: swap memory used in bytes Free: free swap memory in bytes Percent: calculated percentage usage (total - available)/total * 100 sin: Number of bytes exchanged from disks (total) SOUT: Number of bytes exchanged from disks (total)Copy the code

Normally, we don’t use it very much, because it doesn’t return information about all the memory resources on the system, but rather statistics about swapped memory.

Process information

Psutil.pids () returns a list of pids that are currently running, for example:

>>> psutil.pids()
[1, 2, 3, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 19, ..., 32498]
Copy the code

There is an interesting introduction in the documentation

We can filter process information by name. Why is it interesting?

For example, we can record Python process information, memory resource information, CPU resource information to analyze the resource usage of Python code on the server. We tried to filter out python-related Pids:

>>> [p.info for p in psutil.process_iter(attrs=['pid'.'name']) if 'python' in p.info['name']]

[{'name': 'python3.6'.'pid': 1447}] > > >Copy the code

This process, PID 1447, happens to be the Python process I started while demonstrating code in the console.

conclusion

Ok, so that’s the brief introduction to Psutil. Don’t you think you’ve learned anything? It feels like it’s been skimmed over, and it doesn’t leave a deep impression?

Because that’s what the author thinks, but that doesn’t stop us from thinking

Maybe next time, we’ll implement a server resource monitoring and matching tool via Psutil, something like this:

It might be perfect, but it might not be that versatile.

To learn more about Python, crawlers and deep learning, you can pay attention to the wechat official account [Coder].

Please call for Gluten brother Cheng Shulin in the comments section