introduce

Prometheus is an open source monitoring alarm system and time sequence database (TSDB) developed by SoundCloud. Prometheus, developed in the Go language, is an open source version of Google’s BorgMon monitoring system. In 2016, Google launched the Linux Foundation’s Cloud Native Computing Foundation, which included Prometheus as its second open source project. Prometheus is currently active in the open source community. Prometheus and Heapster(Heapster is a subproject of K8S for obtaining cluster performance data.) Compared with the function more perfect, more comprehensive. Prometheus is also capable of supporting tens of thousands of clusters.

Features of Prometheus:

  1. Multidimensional data model.
  2. Flexible query language.
  3. Individual server nodes are independent of distributed storage.
  4. Time series data is collected in PULL mode based on HTTP.
  5. Temporal sequence data can be pushed through an intermediate gateway.
  6. Discover target service objects through service discovery or static configuration.
  7. Supports a wide variety of charts and interfaces, such as Grafana.

Architecture diagram

Prometheus service process:

  • Prometheus periodically captures metrics data from targets, and each fetching target exposes an HTTP service interface for the target to periodically capture. Prometheus supports configuration files, text files, Zookeeper, Consul, and DNS SRV Lookup to specify capture targets. Prometheus monitors in a PULL fashion, in which servers Push data directly through target PULL data or indirectly through an intermediate gateway.

  • Prometheus stores all captured data locally, cleans and organizes the data according to certain rules, and stores the results into new time series.

  • Prometheus visually presents the collected data through PromQL and other apis. Prometheus supports diagram visualization in many ways, such as Grafana, its own Promdash, and its own template engine. Prometheus also provides HTTP API queries to customize the required output.

  • PushGateway allows the Client to actively push metrics to PushGateway, whereas Prometheus periodically grabs data from the Gateway.

  • Alertmanager, an independent component of Prometheus, supports Prometheus’ query statements and provides flexible alarm methods.

  • Prometheus obtains Mertics data through SNMP. After configuring job, use SNMP_export to read device monitoring information.

Metric type

  • Counter Counter, counting from data 0. The ideal state will grow forever. Calculate the total number of requests
  • Evacuation instantaneous state values. The value can be arbitrarily changed, applicable to CPU usage and temperature
  • Histogram samples data over a period of time and sums all values together with statistical quantities and bars. A certain time for a certain metric, grouping, a period of time HTTP corresponding size, the time the request took.
  • Summary also generates multiple indicators, with suffixes _bucket(only histogram), _sum, and _count, respectively

Quantiles are available for both Histogram and Summary. Quantiles are obtained by means of Histogram index data collected from Prometheus and calculated by histogram_quantile() of Prometheus’s query function. Summary is the quantile computed directly in the application. The differences between summaries and summaries are explained in Histograms and summaries, in particular quantiles cannot be aggregated. Note that this failure to aggregate does not mean that it is functionally unsupported, but that it is generally meaningless to aggregate quantiles. LatencyTipOfTheDay: You can’t average Percentiles. Period specifies that “quantiles” cannot be averaged: quantiles themselves are used to slice data, and their averages do not have the same quantile effect.

We mainly use the top two types for monitoring. I have no contact with the following two types at present. The above text and introduction are quoted from LijiaocN

Install the Prometheus

Docker method was used for this construction. Two containers are needed to complete the construction. Do not configure alarm correlation, only monitoring data

The premise

  • Build location: /home/long/prometheus /

  • Environment: Please refer to the official documentation of Docker19.03.1 to specify the version.

  • System: centos7

The preparatory work

Yml We built at the root of the build location: Touch Prometheus. Yml added the test demo configuration to the profile

global:
  scrape_interval: 15s
  scrape_timeout: 10s
  evaluation_interval: 15s
scrape_configs:
- job_name: prometheus
  honor_timestamps: true
  scrape_interval: 5s
  scrape_timeout: 3s
  metrics_path: /metrics
  scheme: http
  static_configs:
  - targets:
    - localhost:9090
Copy the code

Notice The format of the configuration file is YAML. For details about the syntax, seehere.

Installation and operation

  • Start Prometheus from Docker.

    docker run -d -p 9090:9090 \
         -v /home/along/prometheus.yml:/etc/prometheus/prometheus.yml \
          --name prometheus \
          prom/prometheus \
          --config.file=/etc/prometheus/prometheus.yml \
          --web.enable-lifecycle
Copy the code

–web.enable-lifecycle Enable the remote hot loading configuration file curl-x POST http://IP:9090/-/reload

Note docker thermal load there is a problem here, to hang in the above documents for/home/along/Prometheus. If yml directly edit this file will change the file inodes, heat load will not succeed.

** Workaround: Instead of making changes hanging from the configuration file, make a copy and flush it down to Prometheus.yml **

Such as:

  1. cp /home/along/prometheus.yml /home/along/prom-edit.yml

  2. vi /home/along/prom-edit.ym

  3. cat /home/along/prom-edit.ym > /home/along/prometheus.yml

  4. Curl -x POST http://IP:9090/-/reload is loaded successfully.

The validation service

If you visit http://IP:9090, the simple webUI is displayed. This is the Web interface for Prometheus.

I saw some information and surveillance data. You can show charts.

Click Status->Target to see the monitored device information.

Visit http://IP:9090/metrics to see the monitoring data.

At this point,Prometheus has been installed and native data has been monitored.

Grafana installation

Grafana is an open source application for visualizing large measurement data, providing a powerful and elegant way to create, share, and browse data. The Dashboard displays data from your different metric sources. Grafana is most commonly used for Internet infrastructure and application analysis, but is also useful in other areas, such as industrial sensors, home automation, process control, and more. Grafana supports hot-swappable control panels and extensible data sources, and currently supports Graphite, InfluxDB, OpenTSDB, Elasticsearch, Prometheus, etc

Docker run -d -p 320:3000 — Name grafana grafana/grafana Docker run -d -p 320:3000 –name grafana grafana/grafana

Docker run -d -p 320:3000 –name grafana chenwl2016/grafana-chs:0.1.5 docker run -d -p 320:3000 –name grafana chenwl2016/grafana-chs:0.1.5

After the command is executed, access grafana through HTTP :IP:3000. The default account password is admin

After entering, there will be a guide to the home page. Add the data source and select Prometheus. You will see data on the default template.

Query template plug-ins on the official website. Import into the system. Custom template selection needed data to display, here I haven’t played 6, not to say more.

reference

Grafana.com/docs/ Prometheus. IO/docs/introd… www.hi-linux.com/posts/25047… www.lijiaocn.com/%E9%A1%B9%E…