Today, we will write functional modules and analyze them before the code starts. We will refine the module functions by drawing flow charts and UML class diagrams. Then, we will write a module step by step from building a skeleton.

【 I 】 Build distributed server monitoring platform with Flask + Mongo from 0
Summon division canyon adorable new

Module writing sequence

Consider: in many modules, which module should Meng Xin finish first?

Let’s rank the requirements, look at their respective weights and dependencies, and review the modules we need to write:

• Data processing and visualization
• Information monitoring
• The alarm module
• Data is stored
• The API view

Quinn draws a graph that shows the dependencies and roles of each module:

First of all, the resources of each server should be read, and when reading the resources, check whether the threshold is exceeded. Then the host obtains all the server resource information and stores it in the database at one time, and then calculates the data out of the database and processes it into a visual chart. So you can adjust the sequence of the above modules:

• Information monitoring
• Data is stored
• Data processing and visualization

As for the rest of the alert module and API view, which are used for debugging during the development of the first three modules, you can just leave the logical placeholder (also known as the Pass in Python) and wait until the first three modules are completed and then go back to replenish the logic of the alert module and API view.

Realization of information monitoring module

In the Python System Resource Information Retrieval tool, have you used it? We have learned how to get (non-Windows) server resource information, now we need to transform the previous knowledge into a functional module, that is, in the code through the class to achieve resource information acquisition, so that we can get the required data through class instantiation and method call.

Think: After we understand the requirements, the implications, can we start writing code?

If you’re an experienced developer, you probably already have a rough structure in mind. But as new kids, we’re not able to generate structures in our minds that quickly, so we still need to draw. Designing and drawing the code before it starts increases the chances of making fewer errors and changes to the code. Now that we’re writing classes, let’s draw a UML class diagram.

First of all, we should give the class want to a name, just call PresentMomentSystemResource, so an empty UML diagram can be drawn:

As for the method inside, let’s think about it:

• There should be an init method so that some class variables can be specified when the class is instantiated;
• Different methods should be used to obtain CPU, memory, hard disk, and process data.
• Given that the class may be instantiated multiple times and invoked in multiple places, it is possible to use the singleton pattern;

So change the UML diagram:

Quinn explains what this UML class diagram means:

• The magic method new completes the class singleton pattern
• The magic method init sets some class variables to use when instantiating
• We then define a method for each hardware information data and return the result as dict data type

Then we can start coding.

For this project, we will create a new folder Monitors, a Python package named Monitors, and a file named core.py within the project, as shown below:

And write the class infrastructure according to the UML class diagram:

Cui Qingcai | Jingmi, Wei Shidong | Quinn invites you to follow the wechat public number [attack Coder] and work together to make progress
class PresentMomentSystemResource:

    def __new__(cls, *args, **kwargs):
        # singleton
        pass

    def __init__(self):
        pass

    def memory_usage(self):
        """Current memory usage information"""
       pass

    def cpu_usage(self):
        """Current CPU usage information"""
        pass

    def disks_usage(self):
        """Current root disk usage information"""
        pass

    def processes_id(self):
        """Filter the list and number of Pids related to the current keyword"""
        pass

Copy the code

Once the skeleton of the class is laid out, it’s time to write the actual code for each method.

First, we use the magic method new to turn the class into a singleton, so the code for the new part of the method is changed to:

    def __new__(cls, *args, **kwargs):
        # singleton
        if not hasattr(cls, '_instance'):
            cls._instance = super(PresentMomentSystemResource, cls).__new__(cls, *args, **kwargs)
        return cls._instance
Copy the code

As an experienced driver, you should be able to start writing the next method as soon as you finish writing the code. But now we’re the new kids of Summoner Canyon, right? So we need to write the test code, as long as we are sure that there is only one instance object after the class is instantiated, then we can proceed to the next step.

So we also need to create a new testing.py file and write:

from monitors.core import PresentMomentSystemResource


if __name__ == "__main__":
    p1 = PresentMomentSystemResource()
    p2 = PresentMomentSystemResource()
    print(p1, p2)
Copy the code

Run the testing.py file and take a look at the console output:

<monitors.core.PresentMomentSystemResource object at 0x7fb0862a7128>
<monitors.core.PresentMomentSystemResource object at 0x7fb0862a7128>

Process finished with exit code 0
Copy the code

According to the output result, P1 and P2 are the same instance object, indicating that the new method is effective in implementing the singleton pattern.

So let’s write the next method. In the previous article, The Python System Resource Information Retrieval Tool, Have you used it? Psutil, which can fetch system resource information, knows the methods used to fetch CPU information, memory information, and disk information, so we can initialize these methods in init method:

import psutil

    def __init__(self):
        self.memory = psutil.virtual_memory()
        self.cpu = psutil.cpu_times()
        self.disk = psutil.disk_usage("/")
Copy the code

For the information about the disk part, we specify the “/” drive letter information, other mount disk information is not so important.

The unit of value obtained is k, but usually we use M or G. Here we need to set two more units:

from math import pow

    def __init__(self):
        self.memory = psutil.virtual_memory()
        self.cpu = psutil.cpu_times()
        self.disk = psutil.disk_usage("/")
        self.mb = pow(1024, 2)
        self.gb = pow(1024, 3)

Copy the code

The POW method is a method in python’s built-in library that computes powers of a number m to the NTH power. It can be interpreted as:

Pow of m, n is m to the n.

When it’s time to actually code each hardware resource information, let’s look at memory first. Memory requirements include the total amount of memory, used memory, remaining memory, and remaining percentage. Self. memory: self.memory: self.memory: self.memory: self.memory: self.memory: self.memory: self.memory

    def memory_usage(self):
        """Current memory usage information"""
        total = self.memory.tota
        used = self.memory.used
     
Copy the code

Psutil does not provide us with a direct access to the margin and percentage of the margin, so we can mathematically calculate the margin and percentage of the margin after we have calculated the numerical units. Here the memory_usage code is changed to:

    def memory_usage(self):
        """Current memory usage information"""
        total = self.memory.total/self.gb
        used = self.memory.used/self.gb
        free = total - used
        percent = round(free/total, 2)
        return total, used, free, percent
Copy the code

Then go to Testing.py and test it:

from monitors.core import PresentMomentSystemResource


if __name__ == "__main__":
    p1 = PresentMomentSystemResource()
    res = p1.memory_usage()
    print(res)
Copy the code

The output after running is as follows:

(7.676643371582031, 1.7717132568359375, 5.904930114746094, 0.77)
Copy the code

Compare it with the system resource monitoring system:

In general, it is consistent, indicating that there is no problem in the value and calculation of this method.

Then, following the previous article and this method, I will write code for several other hardware, and finally the entire core.py file will be written as follows:

import psutil
from math import pow
from functools import reduce


class PresentMomentSystemResource:

    def __new__(cls, *args, **kwargs):
        # singleton
        if not hasattr(cls, '_instance'):
            cls._instance = super(PresentMomentSystemResource, cls).__new__(cls, *args, **kwargs)
        return cls._instance

    def __init__(self):
        self.memory = psutil.virtual_memory()
        self.cpu = psutil.cpu_times()
        self.disk = psutil.disk_usage("/")
        self.mb = pow(1024, 2)
        self.gb = pow(1024, 3)

    @property
    def memory_usage(self):
        """Current memory usage information"""
        total = self.memory.total/self.gb
        used = self.memory.used/self.gb
        free = total - used
        percent = round(free/total, 2)
        buffers = self.memory.buffers/self.gb
        cached = self.memory.cached/self.gb
        total, used, free, buffers, cached = map(lambda x: round(x, 2), [total, used, free, buffers, cached])
        return {"total": total, "used": used, "free": free, "free_percent": percent, "buffers": buffers, "cached": cached}

    @property
    def cpu_usage(self):
        """Current CPU usage information"""
        count = psutil.cpu_count()
        logical_count = psutil.cpu_count(logical=True)
        percent = psutil.cpu_percent(interval=1)
        return {"count": count, "logical_count": logical_count, "percent": percent}

    @property
    def disks_usage(self):
        """Current root disk usage information"""
        total, used, free = map(lambda x: round(x/self.gb), self.disk[:3])
        percent = self.disk.percent
        return {"total": total, "used": used, "free": free, "free_percent": percent}

    def processes_id(self, keywords=['python') :"""Filter the list and number of Pids related to the current keyword"""
        attrs = psutil.process_iter(attrs=['pid'.'name'])
        pid = [[p.info for p in attrs if keyword in p.info['name']] for keyword in keywords]
        pid_number = reduce(lambda x, y: x+y, [len(p) for p in pid])
        return {"pid": pid, "type_number": len(pid), "pid_number": pid_number}

Copy the code

It’s important to note that since our monitoring is for crawler/server resource relationships, the processes_id method restricts the process ID retrieval to python-related processes only.

Flask view writing

So let’s see how this class can be used in flask.

First, we set up the flask in the init.py file for monitors

from flask import Flask
from flask.ext.restful import Resource, Api

app = Flask(__name__)
api = Api(app)
resource = Resource
Copy the code

Then create a new start.py file and assemble flask’s skeleton as in the previous article

(Install flask and Flask-restful in python on your PC) :

# start.py
from monitors import app, api, resource

class PresentMomentSystemResourceView(resource):
    """Current system Resource Usage View"""
    def __init__(self):
        pass

    def get(self):
        
        return {"status": "success"."message": "this is flask view"}

api.add_resource(PresentMomentSystemResourceView, '/')

if __name__ == '__main__':
    app.run(debug=False)

Copy the code

Then run the start.py file to get the output:

Flask.ext. restful import Resource, Api * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)Copy the code

Note We can access port 5000 of the local computer through the browser:

Flask’s view skeleton is set up. The next step is to associate the system resource information fetching class with the view class and change the code of start. py to:

# start.py
from monitors import app, api, resource
from monitors.core import PresentMomentSystemResource


class PresentMomentSystemResourceView(resource):
    """Current system Resource Usage View"""
    def __init__(self):
        self.sr = PresentMomentSystemResource()

    def get(self):
        memory = self.sr.memory_usage
        disks = self.sr.disks_usage
        cpu = self.sr.cpu_usage
        pid = self.sr.processes_id()
        return {"cpu": cpu, "memory": memory, "disk": disks, "pid": pid}

Copy the code

After running the start.py file, we refresh the previous browser’s page and get a string of data (formatted automatically in Firefox, but not so neatly in other browsers) :

This is the CPU, memory, disk, and process information data that we return in the view class.

At this point, we have written the server information retrieval module of demasia camp, and next time we will write the data storage and other modules.

---

summary

(The code, flow chart and UML class diagram used in the article, etc., can be downloaded by sending “Best Practice II” on wechat official account after following wechat official account.)

What you learned in Summoner Canyon today is not only module writing and flask’s basic code writing, but more importantly, how to analyze the composition of modules, refine the modules you wrote by drawing flow charts and UML class diagrams, and finally realize an independent module. As the darling of summoner Canyon, did you see what Quinn meant today?