[Systems] Docker monitoring

[Samuel Cantero]

Hi all,

Finally I've gotten some time and written the plugin to monitor docker
container CPU usage [1]. You can check the munin graph to monitor the
docker's container CPU usage in the following URL:

http://munin.sugarlabs.org/sugarlabs.org/freedom.sugarlabs.org/docker_cpu.html

The next step is to find out if there a possibility to monitor docker
container network usage and define the critical and warning thresholds for
the docker_cpu and docker_memory plugins.

Since now we can continue improving our docker infra given that we can
monitor the memory and CPU usage for every container.

Greetings,

[1] https://github.com/scanterog/munin-plugin-docker
<https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fscanterog%2Fmunin-plugin-docker&sa=D&sntz=1&usg=AFQjCNE1BrM9kzMYcCu0yDluDMzdoNlAdA>

On Tue, Sep 8, 2015 at 10:34 PM, Samuel Cantero <scg at sugarlabs.org> wrote:

> Hi all,
>
> I've been investigating how to setup a docker monitoring system for our
> docker infra. Basically the solutions consists of three components:
>
>    - cAdvisor to monitor the resource (CPU and memory) usage in
>    containers. By default, cAdvisor stores the historical data for 1 minute in
>    memory.
>    - In order to keep data over time we need a time series database. We
>    have two main options: influxDB and prometheus.
>    - In order to have some nice charts we can use grafana (a graph editor
>    for influxdb) or promdash (dashboard builder for Prometheus).
>
> The preferred stack is cadvisor+prometheus+promdash (or grafana). Why?
> Because cAdvisor exposes container statistics as Prometheus metrics out of
> the box.
>
> As you can see, we need at least three components in order to achieve
> docker monitoring. Furthermore, cAvisor consumes a lot of CPU (~20%) and
> prometheus a lot of RAM (~3GB). You can check this on cAdvisor.
>
> For this reason, I've written a plugin for munin to monitor docker
> container memory usage. This way we can leverage our munin installation
> (and its RRDtool) and avoid the cadvisor+prometheus+promdash stack :)
>
> You can check the munin graph in the following URL:
>
>
> http://munin.sugarlabs.org/sugarlabs.org/freedom.sugarlabs.org/docker_memory.html
>
> The next step is to write another plugin to monitor docker container CPU
> usage. This is a little more complicated. The jiffy value is a whole new
> world for me. I have to investigate how to get the CPU usage in % using the
> cumulative jiffie value exposed by the control groups on a
> pseudo-filesystem (similar to /proc/stat).  Any help is welcome.
>
> *Off-topic: *In order to improve our wiki response time I've tried to
> deploy mediawiki with HHVM but it was not successful. I am not sure if they
> are working on it. The mediawiki last version has experimental support for
> HHVM 3.3.0 [1]. The last available HHVM version is 3.9.1.
>
> Greetings,
>
> [1] https://www.mediawiki.org/wiki/Release_notes/1.25
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sugarlabs.org/private/systems/attachments/20150928/a5b20f06/attachment.html>