- Monitor container metrics
- Send Alerts when a container is stopped / down
We are going to use the following tools.
cadvisor — https://github.com/google/cadvisor
Analyzes resource usage and performance characteristics of running containers.
Prometheus — https://prometheus.io/
Event monitoring and alerting. It records real-time metrics in a time series database built using an HTTP pull model, with flexible queries and real-time alerting.
Grafana — https://grafana.com/
Multi-platform open-source analytics and interactive visualization web application. It provides charts, graphs, and alerts.
Install cadvisor on the host you want to monitor
docker run -d — restart always — name cadvisor -p 8080:8080 -v “/:/rootfs:ro” -v “/var/run:/var/run:rw” -v “/sys:/sys:ro” -v “/var/lib/docker/:/var/lib/docker:ro” google/cadvisor:latest
Verify installation cadvisor
docker ps -a — check for a container named — cadvisor
Navigate to http://hostIP:8080/containers/ to view a list of containers picked up by cadvisor
Adding a new target on Prometheus
Edit prometheus.yml to add the ‘cadvisor’ job
- job_name: ‘cadvisor’
— targets: [‘cadvisorIP:8080’]
Verify target has been added successfully
You should see it listed
Navigate to Grafana Dashboard
import the dashboard template for container monitoring. Make sure to select the correct data source.
We can use a cool dashboard already created by Brian Christner -> https://github.com/vegasbrianc
Here is the dashboard
Grafana menu -> Dashboards -> Import and provide the Dashboard ID #179
To get alerts on container stopped events
Create a new panel on the dashboard with the following Metric
To create alerts for a particular container
Navigate to Alert on the panel options,
Create a condition Ie
When count() of query(A,5m,now) IS BELOW 1
Then select the Notification channel/group and enter the message to be sent if the count is below 0.
Telegram alerts on Grafana
To create a telegram bot
Go to -> https://core.telegram.org/bots#creating-a-new-bot
To get a telegram chat id use