Based on Rancher K8S deployment Prometheus monitoring swoole project core indicators of actual combat

keyword Rancher Prometheus swoole docker docker-compose

Prometheus

features

Prometheus is an open source monitoring alarm system and time sequence database (TSDB) developed by SoundCloud. Prometheus, developed using the Go language, is an open source version of Google’s BorgMon monitoring system and capable of supporting tens of thousands of clusters.

• Multidimensional data model
• Flexible query language
• Individual server nodes are independent of distributed storage
• Time series data is collected in PULL mode based on HTTP
• Temporal sequence data can be pushed through an intermediate gateway
• Discover target service objects through service discovery or static configuration
• Supports a wide variety of charts and interfaces, such as Grafana

The basic principle of

Prometheus uses HTTP to periodically fetch Metrics, sort out and clean Metrics data according to certain rules, and store them in a new time series. In this actual practice, the CORE indicators of PHP Swoole used by the business will be monitored and output through grafana graphics tools will be displayed.

Set up steps

Native orchestration of Prometheus Docker

For direct deployment on Rancher, we can use Docker-compose locally to conduct remote deployment of K8S after local arrangement experiment, and use Docker-compose locally to simulate K8S container arrangement, which is convenient for development experiment.

Start by creating a Monitor folder as the path for this project

$ mkdir monitor && cd monitor
Copy the code

Create a Prometheus folder to store our monitoring configuration files

$ mkdir prometheus && cd prometheus && touch prometheus.yml
Copy the code

Add the following to the prometheus.yml configuration file

global:
  scrape_interval:     30s # Global period
scrape_configs:
- job_name: 'foo' # job name
  scrape_interval: 30s # call cycle
  metrics_path: "/health/metrics" Call path
  static_configs:
    - targets: ['foo.com:80'] # remote service address
Copy the code

In the configuration file above we pointed out the foo the monitoring work, and specify the 30 s running period, each call to address to foo.com: 80 / health/metrics that interface address. How to output data for this interface will be discussed below.

Create the docker-comemage. yml file at the root of the monitor folder

$ touch docker-compose.yml
Copy the code

Add the following

version: '2'
services:
    prometheus:
        container_name: prometheus # container name
        image: prom/prometheus # mirror name
        networks: # Shared network
            - foo
        volumes:
            - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml # to local configuration files, mounted to the target container/etc/Prometheus/Prometheus yml position
        ports:
            - '9090:9090'
        command:
            - '--config.file=/etc/prometheus/prometheus.yml'
            - '--web.enable-admin-api'
            - '--web.enable-lifecycle' Support hot update of configuration files

    grafana:
        container_name: grafana
        image: Grafana/grafana: 6.4.3
        ports:
            - '3000:3000'
        depends_on:
            - prometheus # start dependent on Prometheus service
        volumes:
            - ./data:/var/lib/grafana
            - ./prometheus.yml:/etc/prometheus/prometheus.yml
        networks:
            - foo
        environment:
            GF_SECURITY_ADMIN_USER: ${GRAFANA_USERNAME} # grafana username (variable)
            GF_SECURITY_ADMIN_PASSWORD: ${GRAFANA_PASSWORD}

networks: Define an external network for easy access between services
    foo:
        external: true
Copy the code

The configuration writing problem in the Command option of the Prometheus service is highlighted here, because the PROM/Prometheus image already has program entry and is configured with –config.file and –web.enable-admin-api. In this case, we only need to append the web.enable-lifecycle command to enable the monitoring configuration of service hot update. If only the last parameter is written, the preceding two parameters will not be configured and the service will not be started due to the lack of configuration.

Swoole interface

In the Swoole project, write the interface /health/metrics/ as follows:

$server = new Swoole\Http\Server("127.0.0.1".80); // The $server object type is only shown here. The server object needs to be retrieved from the current project framework. Do not instantiate it again.
$metrics = $server->stats();
 foreach ($metrics as $k= >$v) {
     echo sprintf("%s %s\n".$k.$v);
 }
 echo "# coroutine state \n";

 $metrics = Swoole\Coroutine::stats();
 foreach ($metrics as $k= >$v) {
     echo sprintf("%s %s\n".$k.$v);
 }
Copy the code

http://127.0.0.1:80/health/metrics/ will have the following output

start_time 1573612504 connection_num 1 accept_count 7 close_count 6 worker_num 2 idle_worker_num 1 tasking_num 0 Request_count 10 worker_request_count 0 worker_dispatch_count 11 coroutine_num 1 # Coroutine status event_num 2 signal_listener_num 0 aio_task_num 0 c_stack_size 2097152 coroutine_num 1 coroutine_peak_num 1 coroutine_last_cid 3Copy the code

You can refer to the Swoole documentation for a detailed explanation of the parameters.

Enabling the Monitoring Service

Enter the monitor directory and run the docker-compose up -d command. A message may be displayed indicating that the external network does not exist. Create the external network as prompted. After the service is started, visit http://127.0.0.1:9090 to view the Prometheus Web page.

Go to http://127.0.0.1:9090/targets

The Prometheus service is properly monitored

Visit http://127.0.0.1:3000/ by clicking Add Data Source and selecting Prometheus as the data source

Enter http://prometheus:9090 in the HTTP URL field, because the local container uses the same network and can use the service name directly for service access.

Click Grafana Home, New Dashboard, Choose Visualization, and select Graph

Click Queries and select Query as Prometheus

Add Metrics in A. Use Legend to Add Metrics. Click Add Query to draw multiple index lines in A chart. The local deployment is now complete.

Rancher K8S deployment

Rancher deployment is the same, the only difference being that Rancher is used as a container choreographer. Open Rancher and click Deploy Service

Set the image to PROM/Prometheus, configure the data volume, and configure the YML file for the monitor to mount the container. Refer to the Rancher documentation for mounting and use here.

. In order to join – config file = / etc/Prometheus/Prometheus yml — web. Enable the admin – API – web. Enable – lifecycle

Click the start button at the bottom and configure load balancing in your workload

In this case, you can access grafana.google.com locally. If no public NETWORK DNS exists, you need to modify /etc/hosts locally.

Grafana can be configured in the same way according to the local build mode, which will not be repeated in this article.

Prometheus configuration file hot update

$ curl -X POST 'http://grafana.google.com/-/reload'
Copy the code