需求

• 抓取nginx-module-vts的数据
• 计算每个servername、upstream的每秒请求数
• 发送到监控平台zabbix和夜莺

文档

• nginx-moudle-vts
• zabbix-LLD
• zabbix自动发现
• n9e-push-api-metrics

设计

• 方案一（不合理）： 利用zabbix的LLD每次执行时发送请求抓数据,出现断点,机器负载高
• 方案二: 定期每秒抓取数据计算后写文件,zabbix的LLD每次执行读文件数据

方案一

• 源码: main.go

  # 编译
  go build -o nginx-vts-zbx
  mv nginx-vts-zbx ./bin
  
  # for zabbix LLD
  # 查看所有servernames
  bin/nginx-vts-zbx -s
  # 查看servername的req/s
  bin/nginx-vts-zbx -s -o "test.abc.com"
  
  # 查看所有upstreams
  bin/nginx-vts-zbx -u
  # 查看单个upstream的req/s
  bin/nginx-vts-zbx -u -o "test_upstream-10.11.100.79:9000"
  
  # for nightinagle
  # 一分钟上报一次, 默认endpoint为10.10.10.10
  # -t 开启nightingale,关闭zabbix LLD
  # -a 指定transfer
  # -p 上报endpoint
  bin/nginx-vts-zbx -t -a "http://10.51.1.31:5810/api/transfer/push" -p 10.10.10.111

  # 部署
  # for zabbix
  # zabbix client
  # cat /usr/local/zabbix/conf/zabbix_agentd/userparameter_ngx_vts.conf
  UserParameter=serverzones.discovery,/bin/nginx-vts-zbx -c "http://127.0.0.1/status/format/json" -s
  UserParameter=serverzone.reqs[*],/bin/nginx-vts-zbx -s -o $1
  UserParameter=upstreamzones.discovery,/bin/nginx-vts-zbx -u
  UserParameter=upstreamzone.reqs[*],/bin/nginx-vts-zbx -u -o $1
  
  # zabbix server 验证
  zabbix_get -s 10.201.0.11 -k serverzones.discovery
  zabbix_get -s 10.201.0.11 -k serverzone.reqs[s.abc.com]
  zabbix_get -s 10.201.0.11 -k upstreamzones.discovery
  zabbix_get -s 10.201.0.11 -k upstreamzone.reqs["weixin-10.21.4.157:8087"]

方案二

• fetch-nginx-vts: 每分钟获取nginx-vts的数据计算后写文件

• ngx-vts-zbx: 读文件提供给zabbix和夜莺

  ## fetch-nginx-vts
  go build -o fetch-nginx-vts
  # 指定nginx-vts源
  ./fetch-nginx-vts -u "http://screen.abc.com/ngx_status/format/json"
  # 指定写的文件
  ./fetch-nginx-vts -f "nginx-request.json"
  
  ## ngx-vts-zbx
  go build -o ngx-vts-zbx
  # zabbix的LLD  s: serverzone  u: upstream
  go run  main.go -f "nginx-request.json" -s
  go run  main.go -f "nginx-request.json" -u
  go run  main.go -f "nginx-request.json" -s -o screen.abc.com
  go run  main.go -f "nginx-request.json" -u -o test-10.11.100.79:9093
  # 夜莺
  go run  main.go -f "nginx-request.json" -n
  # 指定endpoint上报地址
  go run  main.go -f "nginx-request.json" -c "http://10.51.1.31:5810/api/transfer/push" -p 10.10.10.11 -n

  ## 部署
  ## for nightinagle : 用cron每分钟写入
  * * * * * /usr/local/monitor/nginx_vts/fetch-ngx-vts >/dev/null 2>&1
  * * * * * /usr/local/monitor/nginx_vts/ngx-vts-zbx -n -p 10.11.100.230 >/dev/null 2>&1
  
  ## for zabbix LLD
  /etc/zabbix/zabbix_agentd.d/scripts/ngx-vts-zbx
  # 不改zabbix用户自定义参数
  ln -s  /etc/zabbix/zabbix_agentd.d/scripts/ngx-vts-zbx /bin/nginx-vts-zbx

知识

nginx-moudle-vts

• install && config

  # check nginx install module
  nginx/sbin/nginx -t
  configure arguments: ... --add-module=../nginx-module-vts
  
  # install
  wget http://nginx.org/download/nginx-1.15.10.tar.gz
  tar xvf nginx-1.15.10.tar.gz -C /opt
  cd /opt/nginx-1.15.10
  git clone git://github.com/vozlt/nginx-module-vts.git
  ./configure --prefix=/usr/local/nginx --add-module=nginx-module-vts
  
  # config
  http {
      vhost_traffic_status_zone;
      ...
      server {
          ...
          location /status {
              vhost_traffic_status_display;
              vhost_traffic_status_display_format html;
          }
      }
  }
  
  # access
  http://127.0.0.1/status/format/json

zabbix LLD

• 理解：通过自动发现配置宏 ; 通过用户参数获取自动发现的宏

• 用zabbix-agent的实现逻辑

  1. 用户自定参数UserParameter配置两个脚本

     xxx.discovery,script1
     yyy[*],script2 $1
     
     脚本一: 返回JSON数据,里面的key为zabbix使用宏
     脚本二: 利用脚本一中宏的value作为参数,获取的详细信息

  2. 界面配置, 如ngx-vts-zbx.pdf所示

zabbix-client

• config

  # zabbix agent config
  #  -c可选 默认为 http://127.0.0.1/ngx_status/format/json
  
  # cat /usr/local/zabbix/conf/zabbix_agentd/userparameter_ngx_vts.conf
  UserParameter=serverzones.discovery,/bin/nginx-vts-zbx -c "http://xxx/ngx_status/format/json" -s
  UserParameter=serverzone.reqs[*],/bin/nginx-vts-zbx -s -o $1
  
  UserParameter=upstreamzones.discovery,/bin/nginx-vts-zbx -u
  UserParameter=upstreamzone.reqs[*],/bin/nginx-vts-zbx -u -o $1
  
  # restart 
  ps aux |grep zabbix |grep -v grep | awk '{print $2}' | xargs kill
  /usr/local/zabbix/sbin/zabbix_agentd -c /usr/local/zabbix/conf/zabbix_agentd.conf

• confirm

  # /usr/local/zabbix/sbin/zabbix_agentd -c /usr/local/zabbix/conf/zabbix_agentd.conf -t serverzones.discovery
  {
      "data": [
              {
                  "{#SERVERZONE}": "test.abc.com"
              },
              {
                  "{#SERVERZONE}": "all"
              }
         ]
  }

  # /usr/local/zabbix/sbin/zabbix_agentd -c /usr/local/zabbix/conf/zabbix_agentd.conf -t serverzone.reqs["test.abc.com"]
  10

  # /usr/local/zabbix/sbin/zabbix_agentd -c /usr/local/zabbix/conf/zabbix_agentd.conf -t upstreamzones.discovery
  {
       "data": [
           {
               "{#UPSTREAMNAME}": "test_upstream-10.11.100.17:9000"
           },
           {
               "{#UPSTREAMNAME}": "test_upstream-10.11.100.79:9000"
           }
       ]
   }

  # /usr/local/zabbix/sbin/zabbix_agentd -c /usr/local/zabbix/conf/zabbix_agentd.conf -t upstreamzone.reqs["127.0.0.1:9000"]
  9

trobule

• Value should be a JSON object.

  # zabbix server执行, 返回为空数据 { "data": [] }, 重启agent正常 :)
  zabbix_get -s 10.11.9.23 -k "serverzones.discovery"

• Cannot create item: item with the same key “upstreamzone.reqs[{#UPSTRAMNAME}]” already exists

  返回的key{#xxx}必须为大写{#XXX}
  {#UPSTREAMNAME} 拼写错误 {#UPSTRAMNAME} 

• ZBX_NOTSUPPORTED: Special characters "\, ', ", `, *, ?, [, ], {, }, ~, $, !, &, ;, (, ), <, >, |, #, @, 0x0a" are not allowed

  # 替换*为all或者其他
  strings.Replace(o, "*", "all", -1)