Docker(七) Docker监控平台

linuxx

监控维度

• 主机维度
  
  
  
  • 主机cpu
    
  • 主机内存
    
  • 主机本地镜像情况
    
  • 主机上容器运行情况
    
  
  
• 镜像维度
  
  
  
  • 镜像的基本信息
    
  • 镜像与容器的基本信息
    
  • 镜像构建的历史信息(层级依赖信息)
    
  
  
• 容器维度
  
  
  
  • 容器基本信息
    
  • 容器的运行状态信息
    
  • 容器用量信息
    
  
  

docker监控命令

docker ps
docker images
docker stats
# 注意：docker stats只有选择libcontainer作为执行驱动时才可以使用
# docker stats时常会有一些限制,使用stats api会显示更多的信息
echo -e "GET /containers/tools/stats HTTP/1.0\r\n" | nc -U /var/run/docker.sock
docker inspac
docker top
docker port
cAdvisor
　　google cAdvisor的一些特性：

• 拥有远程管理的丰富api支持api docs
  
• 有web ui管理界面
  
• 是google除支持k8s外的另一个项目
  
• 监控信息输出到influxdb数据库进行存储和读取,支持很多其他的插件
  
• 支持将容器的统计信息一prometheus标准指标形式输出并存储在/metrics HTTP服务端点
  

sudo docker run \
--volume=/:/rootfs:ro \
--volume=/var/run:/var/run:rw \
--volume=/sys:/sys:ro \
--volume=/var/lib/docker/:/var/lib/docker:ro \
--volume=/dev/disk/:/dev/disk:ro \
--publish=8080:8080 \
--detach=true \
--name=cadvisor \
google/cadvisor:latest
DataDog

SoundCloud的Promtheus

　　Promtheus是一个开源服务监控系统与时间序列数据库,结合exporter使用,exporter是基于prometheus开放的Http接口二次开发用来抓取应用程序的指标数据

Global setting

network configure

docker network create --driver bridge --subnet 10.0.0.0/24 --gateway 10.0.0.1 monitor
Grafana configure
　　Grafana Template

# pull grafana image
docker pull grafana/grafana
grafana目录

• 配置文件/etc/grafana/
  
• sqlite3 database file /var/lib/grafana
  

grafana在docker中的环境变量

• GF_SERVER_ROOT_URL=http://grafana.server.name 指定grafana的访问路径
  
• GF_SECURITY_ADMIN_PASSWORD=secret 指定grafana的登录密码
  
• GF_INSTALL_PLUGINS=grafana-clock-panel,grafana-simple-json-datasource 指定安装插件的变量
  

构建持久性存储

docker run \
-d \
-v /var/lib/grafana \
--name grafana-storage \
busybox:latest
启动容器

# 启动Grafana容器
docker run \
-d \
-p 3000:3000 \
--name grafana \
--volumes-from grafana-storage \
-e "GF_INSTALL_PLUGINS=grafana-clock-panel,grafana-simple-json-datasource" \
-e "GF_SERVER_ROOT_URL=http://10.0.0.10:3000" \
-e "GF_SECURITY_ADMIN_PASSWORD=marion" \
--network monitor \
--ip 10.0.0.10 \
--restart always \
grafana/grafana
# 查看配置文件以及数据目录的挂载位置
docker inspect grafana

cAdvisor

sudo docker run \
--volume=/:/rootfs:ro \
--volume=/var/run:/var/run:rw \
--volume=/sys:/sys:ro \
--volume=/var/lib/docker/:/var/lib/docker:ro \
--volume=/dev/disk/:/dev/disk:ro \
--detach=true \
--name=cadvisor \
--network monitor \
--ip 10.0.0.11 \
google/cadvisor:latest
Deploy by docker-compose yaml file

docker-compose.yml

prometheus:
image: prom/prometheus:latest
container_name: monitoring_prometheus
restart: unless-stopped
volumes:
- ./data/prometheus/config:/etc/prometheus/
- ./data/prometheus/data:/prometheus
command:
- '-config.file=/etc/prometheus/prometheus.yml'
- '-storage.local.path=/prometheus'
- '-alertmanager.url=http://alertmanager:9093'
expose:
- 9090
ports:
- 9090:9090
links:
- cadvisor:cadvisor
- node-exporter:node-exporter
node-exporter:
image: prom/node-exporter:latest
container_name: monitoring_node_exporter
restart: unless-stopped
expose:
- 9100
cadvisor:,
image: google/cadvisor:latest
container_name: monitoring_cadvisor
restart: unless-stopped
volumes:
- /:/rootfs:ro
- /var/run:/var/run:rw
- /sys:/sys:ro
- /var/lib/docker/:/var/lib/docker:ro
expose:
- 8080
grafana:
image: grafana/grafana:latest
container_name: monitoring_grafana
restart: unless-stopped
links:
- prometheus:prometheus
volumes:
- ./data/grafana:/var/lib/grafana
environment:
- GF_SECURITY_ADMIN_PASSWORD=MYPASSWORT
- GF_USERS_ALLOW_SIGN_UP=false
- GF_SERVER_DOMAIN=myrul.com
- GF_SMTP_ENABLED=true
- GF_SMTP_HOST=smtp.gmail.com:587
- GF_SMTP_USER=myadrress@gmail.com
- GF_SMTP_PASSWORD=mypassword
- GF_SMTP_FROM_ADDRESS=myaddress@gmail.com

prometheus.yml

# my global config
global:
scrape_interval:     120s # By default, scrape targets every 15 seconds.
evaluation_interval: 120s # By default, scrape targets every 15 seconds.
# scrape_timeout is set to the global default (10s).
# Attach these labels to any time series or alerts when communicating with
# external systems (federation, remote storage, Alertmanager).
external_labels:
monitor: 'my-project'
# Load and evaluate rules in this file every 'evaluation_interval' seconds.
rule_files:
# - "alert.rules"
# - "first.rules"
# - "second.rules"
# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
# The job name is added as a label `job=` to any timeseries scraped from this config.
- job_name: 'prometheus'
# Override the global default and scrape targets from this job every 5 seconds.
scrape_interval: 120s
# metrics_path defaults to '/metrics'
# scheme defaults to 'http'.
static_configs:
- targets: ['localhost:9090','cadvisor:8080','node-exporter:9100', 'nginx-exporter:9113']
command

docker-compose up -d
sysdig
　　sysdig offical

docker pull sysdig/sysdig
docker run -i -t --name sysdig --privileged -v /var/run/docker.sock:/host/var/run/docker.sock -v /dev:/host/dev -v /proc:/host/proc:ro -v /boot:/host/boot:ro -v /lib/modules:/host/lib/modules:ro -v /usr:/host/usr:ro sysdig/sysdig
docker container exec -it sysdig bash
csysdig
Weave Scope
　　weave scope

常用的容器监控工具
　　Docker Runtime Metrics Guide