First, the alertmanager

1.1 Creating an AlertManager Configuration File

vim /root/alertmanager/config.yml
Copy the code
global: resolve_timeout: 5m http_config: follow_redirects: true smtp_hello: localhost smtp_require_tls: true pagerduty_url:'https://events.pagerduty.com/v2/enqueue' opsgenie_api_url: 'https://api.opsgenie.com/' wechat_api_url: '[https://qyapi.weixin.qq.com/cgi-bin/ ' wechat_api_corp_id: Wxe11111111111ca id # enterprise victorops_api_url: 'https://alert.victorops.com/integrations/generic/20131114/alert/' route: Receiver: zhangsan // Corresponding to the name group_by: - groupLabel // classification field in the following receivers, which can be customized. Receiver: zhangsan // Corresponding to the alarm rules field continue: false group_wait: 30s group_interval: 3m repeat_interval: 3m receivers: - name: zhangsan wechat_configs: - send_resolved: True http_config: follow_redirects: true API_secret: <secret> // Password generated after applying for wechat applications corp_id: wxe111111111CA message: '{{ template "wechat.default.message" . }}' api_url: https://qyapi.weixin.qq.com/cgi-bin/ to_user: @all = to_party: '{{template "wechat. Default.to_party ".}}' to_tag: '{{template "wechat.default.to_tag".}}' agent_id: "1000211" // Application wechat id message_type: text templates: - / apps/SRV/alertmanager/templates / *. TMPL / / alarm templates pathCopy the code

1.2 Alarm Template

Sample test template, customized as required

{{define "sight.default. message"}} {{-if gt (len.alerts.Firing) 0 -}} @alarm [{{len.alerts.Firing}}】 {{range Annotations. Summary}} {{.Annotations. Description}} {{ (.StartsAt.Add 28800e9).Format "2006-01-02 15:04:05" }} </pre> {{ end }} {{ end -}} {{ end }}Copy the code

2 build alermanager

$ docker search alertmanager
$ docker pull docker.io/prom/alertmanager:latest
$ docker run -d -p 9093:9093 -v /root/alertmanager/config.yml:/etc/alertmanager/config.yml docker.io/prom/alertmanager:latest --config.file=/etc/alertmanager/config.yml
Copy the code

Port /#/alerts = port/#/alerts

2. Prometheus configuration modification

1.1 Modifying the Prometheus Configuration File

# Alertmanager configuration alerting: alertmanagers: - static_configs: - targets: - ip:9093 # Load rules once and periodically evaluate them according to the global 'evaluation_interval'. rule_files: - "rules/*.yml" # A scrape configuration containing exactly one endpoint to scrape: # Here it's Prometheus itself. scrape_configs: # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config. - job_name: 'AlertManager' # Specifies the alertManager static_configs: -targets: -ip :9093 # Specifies the IP address of the alertManager hostCopy the code

1.2 Adding Rules Alarm rule configuration

test_rules.yml

Groups: - name: passportHttpCode rules: - alert: XX service expr: code{service="1"}! = 200 for: 1m labels: type: httpCode object: overall title: XX service HTTP response code groupLabel: passportHttpCode Annotations: summary: Description: "The current number of abnormal response codes is {{printf \"%.2f\" $value}}\n Trend: http://XXX"Copy the code

Restart Prometheus for the configuration to take effect, and access IP :9090/graph