[Systems] Docker resource limits

[Samuel Cantero]

Hi Sam,

I was checking the resource limits on each container. This is what I've got:

   - The following containers have a limit of 128 MB of memory
   RAM: socialhelp (30% usage of 128 MB), helpserver (18% usage of 128
   MB), web (58% usage of 128 MB), hook (22% usage of 128 MB), bot-master (28%
   usage of 128 MB), and bot (64% usage of 128 MB). None of them have a CPU
   usage of more than 2%.
   - The following containers have a limit of 200 MB of memory RAM:
   bundlebin (17% usage of 200MB). This container doesn't have a CPU usage of
   more than 2%.
   - The following containers have a limit of 512 MB of memory RAM: parsoid
   (99% usage of 512 MB). Maybe we should increase the assigned memory.
   - The following containers have no limit yet: kafkadocker_zookeeper_1
   (372 MB of memory usage), kafkadocker_kafka_1 (577 MB of memory usage),
   cadvisor (129 MB of memory usage), app (1.07 GB of memory
   usage), agitated_torvalds (29 MB of memory usage).
   - I've found that cadvisor has a CPU usage of around 12% and the app
   container sometimes have a CPU usage of around 5 %.

One important thing that we must apply is the swap memory limit. None of
them have a swap memory limit. We can do that with the --memory-swap.

Of course all of those values can change over time. According to cadvisor's
page, we should have historical resource usage per container, but I didn't
find it. Has you found it?

Finally, how have you implemented the CPU limit? I can see that every
container have 1024 shares and all of the cores allowed (0 to 7). I guess
that you have used --cpu-quota=10000 considering the default *cpu-period
(100 ms)*. Is that correct? Docker does a great job balancing the load
between the available cores.

Thanks Sam,

Samuel C.

PD: Sam, if you want to list the container per name instead of per id you
need to write http://freedom.sugarlabs.org:8080/docker/ instead of
http://freedom.sugarlabs.org:8080/containers/docker.

On Sat, Jul 18, 2015 at 7:21 AM, Sam P. <sam at sam.today> wrote:

> Hi Samuel,
>
> I had a little hack on the docker resource limits.  I attempted to start
> using the --memory and --cpu-quota flag with docker-compose, however that
> is not supported via compose because it is not supported through the python
> api wrapper.  So I made a little tool that just runs the docker command
> from arguments in a yml file [1].
>
> I've added resource limits to the containers with around 128m of ram and
> 10% cpu.  (parsoid got 512m because that is what it was using!)
>
> I am yet to add resource limits to discourse (got to do that via the
> discourse launcher) or kafka (I'm using a docker-compose script to run
> kafka with zookeeper) or cadvisor.
>
> Also, cadvisor is consistently using ~10% cpu (even when I'm not looking
> at it).   It is stays in about the top 3 processes when viewing it in
> htop.  Maybe this is something to look into.  (It also stresses out
> firefox, so I'm starting to enjoy "docker stats")
>
> Anyway, the infra is now in place to look into further using docker!
>
> Thanks,
> Sam
>
> [1]  https://github.com/sugarlabs-infra/container.yml
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sugarlabs.org/private/systems/attachments/20150718/92d72f4c/attachment.html>