openresty定制prometheus metrics监控nginx(1)
标签:openresty,prometheus,nginx,grafana
之前总喜欢看服务器日志,看有多少google爬虫访问过我的博客。时间长了就觉得太low了,肉眼搜索google的user agent,还要看ip是不是google的.觉得low就撸点代码搞定
nginx-lua-prometheus
代码很简单,参考nginx-lua-prometheus就行了
http{
server{
init_by_lua_file /root/lua/init.lua;
location / {
log_by_lua_file /root/lua/stat.lua;
}
}
server{
#export port for prometheus pulling
listen 9145;
location /metrics {
content_by_lua '
prometheus:collect()
';
}
}
}
init.lua
prometheus = require("resty.prometheus").init("prometheus_metrics")
requests = prometheus:counter("nginx_http_requests_total", "Number of HTTP requests", {"host", "status"})
google_requests=prometheus:counter("nginx_http_google_requests_total", "Number of HTTP requests from google", {"status"})
stat.lua
local forwarded_for=ngx.var.http_x_forwarded_for
local cidr = require("resty.libcidr-ffi")
local is_from_google=cidr.contains(cidr.from_str('66.249.0.0/16'), cidr.from_str(forwarded_for))
ngx.log(ngx.ERR,'is_from_google: ',is_from_google,' ',forwarded_for)
local user_agent=ngx.var.http_user_agent
local is_google_bot=string.match(user_agent,'Googlebot/2.1;')
ngx.log(ngx.ERR,'is_google_bot: ',is_google_bot,' ',user_agent)
if is_from_google and is_google_bot then
google_requests:inc(1, {ngx.var.status})
end
requests:inc(1, {ngx.var.server_name, ngx.var.status})
说明
libcidr-ffi
安装可以参考openresty设置访问白名单66.249.0.0/16
范围有点大,google不会公布爬虫ip,只能自己观察- exporter没有用Nginx VTS exporter,因为想要可定制性更强的exporter
效果
监控提醒
监控好了,可是还是要主动看监控,才能知道有没有google爬虫访问.如果能'提醒'我就更好了,可以定义prometheus
的alerting rules
,提醒的内容是google爬虫访问的路径(path)
alerting rules
rule.yml
groups:
- name: google request
rules:
- alert: google request
expr: sum(increase(nginx_http_google_requests_total[2m])) by (path) >0
for: 5s
annotations:
description: "path:{{$labels.path}}"
{{$labels.path}}
可以获取对应的label中的path{{$value}}
可以获取提醒的时间点对应的值
当然前面定义的counter
也要改一下
init.lua
添加label-path
google_requests=prometheus:counter("nginx_http_google_requests_total", "Number of HTTP requests from google", {"path"})
stat.lua
对应的写入path
if is_from_google and is_google_bot then
local path=ngx.re.gsub(ngx.var.request_uri, "[.|/]html", "")
google_requests:inc(1, {path})
end
Alertmanager
global:
resolve_timeout: 5m
slack_api_url: 'https://hooks.slack.com/services/your_path'
route:
group_wait: 10s
group_interval: 10s
repeat_interval: 1h
receiver: 'slack-notifications'
receivers:
- name: 'slack-notifications'
slack_configs:
- title: 'google request'
text: "{{ .CommonAnnotations.description }}"
{{ .CommonAnnotations.description }}
获取前面rule.yml
定义的annotations description
最后配置prometheus.yml
alerting:
alertmanagers:
- static_configs:
- targets:
- 127.0.0.1:9093
rule_files:
- "rule.yml"
效果
然而这里有个问题,导致无法触发告警,参见下一篇
不定期更新