首页

    openresty定制prometheus metrics监控nginx(1)

    标签:openresty,prometheus,nginx,grafana

    之前总喜欢看服务器日志,看有多少google爬虫访问过我的博客。时间长了就觉得太low了,肉眼搜索google的user agent,还要看ip是不是google的.觉得low就撸点代码搞定

    nginx-lua-prometheus

    代码很简单,参考nginx-lua-prometheus就行了

    http{
        server{
            init_by_lua_file /root/lua/init.lua;
            location / {
                log_by_lua_file /root/lua/stat.lua;
            }
        }
        server{
            #export port for prometheus pulling
            listen 9145;
            location /metrics {
                content_by_lua '
                  prometheus:collect()
                ';
            }
        }
    }
    

    init.lua

    prometheus = require("resty.prometheus").init("prometheus_metrics")
    requests = prometheus:counter("nginx_http_requests_total", "Number of HTTP requests", {"host", "status"})
    google_requests=prometheus:counter("nginx_http_google_requests_total", "Number of HTTP requests from google", {"status"})
    

    stat.lua

    local forwarded_for=ngx.var.http_x_forwarded_for
    local cidr = require("resty.libcidr-ffi")
    local is_from_google=cidr.contains(cidr.from_str('66.249.0.0/16'), cidr.from_str(forwarded_for))
    ngx.log(ngx.ERR,'is_from_google: ',is_from_google,' ',forwarded_for)
    
    local user_agent=ngx.var.http_user_agent
    local is_google_bot=string.match(user_agent,'Googlebot/2.1;')
    ngx.log(ngx.ERR,'is_google_bot: ',is_google_bot,' ',user_agent)
    
    if is_from_google and is_google_bot then
      google_requests:inc(1, {ngx.var.status})
    end
    
    requests:inc(1, {ngx.var.server_name, ngx.var.status})
    

    说明

    效果

    监控提醒

    监控好了,可是还是要主动看监控,才能知道有没有google爬虫访问.如果能'提醒'我就更好了,可以定义prometheusalerting rules,提醒的内容是google爬虫访问的路径(path)

    alerting rules

    rule.yml

    groups:
      - name: google request
        rules:
        - alert: google request
          expr: sum(increase(nginx_http_google_requests_total[2m])) by (path) >0
          for: 5s
          annotations:
            description: "path:{{$labels.path}}"
    
    • {{$labels.path}}可以获取对应的label中的path
    • {{$value}}可以获取提醒的时间点对应的值

    当然前面定义的counter也要改一下

    init.lua添加label-path

    google_requests=prometheus:counter("nginx_http_google_requests_total", "Number of HTTP requests from google", {"path"})
    

    stat.lua对应的写入path

    if is_from_google and is_google_bot then
      local path=ngx.re.gsub(ngx.var.request_uri, "[.|/]html", "")
      google_requests:inc(1, {path})
    end
    

    Alertmanager

    global:
      resolve_timeout: 5m
      slack_api_url: 'https://hooks.slack.com/services/your_path'
    
    route:
      group_wait: 10s
      group_interval: 10s
      repeat_interval: 1h
      receiver: 'slack-notifications'
    receivers:
    - name: 'slack-notifications'
      slack_configs:
        - title: 'google request'
          text: "{{ .CommonAnnotations.description }}"
    

    {{ .CommonAnnotations.description }}获取前面rule.yml定义的annotations description

    最后配置prometheus.yml

    alerting:
      alertmanagers:
      - static_configs:
        - targets:
          - 127.0.0.1:9093
    
    rule_files:
      - "rule.yml"
    

    效果

    然而这里有个问题,导致无法触发告警,参见下一篇


    不定期更新