[Systems] Docker containers in SL

[Sam Parkinson]

Hi Samuel,

Thanks for compiling this information.

On Thu, Mar 10, 2016 at 3:23 AM, Samuel Cantero <scanterog at gmail.com> 
wrote:
> Hi all,
> 
> This email is to make some notes regarding to our current services 
> running in Docker containers. The main idea is to enable anyone to 
> fix a container problem or deploy a new one. This will be a long 
> email, sorry for that. After this email thread, we should write some 
> documentation in our wiki.
> 
> How are we running Docker currently?
> 
> We are using a bridged network with Docker and it lets us publish a 
> service using a specific port in freedom. Then we proxy-pass the 
> service through nginx (port 80).
> There is a bridge called docker0 and every container has an interface 
> inside this bridge. This is managed automatically by Docker.
> 
> What are the services currently running in a Docker container?
> 
> The current running containers are:
> 
> 1) org.sugarlabs.bugs.
> 
> Trac instance. Listening on port 5007.
> 
> You can check this on 18.85.44.59:5007. Then we proxy this in nginx 
> (/etc/nginx/sites-available/bugs.sugarlabs.org) using:
> 
> location / {
>                 proxy_pass http://localhost:5007;
>         }
> 
> 2) org.sugarlabs.hook.
> 
> Web hook for SL activities? Listening on port 5004.
> 
> 3) kafka: linked with zookeeper. Listening on port 9092.
> 4) zookeeper. Listening on port 2181. Locally open ports 2888 and 
> 3888.
> 
> What are we using kafka and zookeeper?

No longer.  It was also2 stuff.  I have killed those services now, as I 
should have killed them before.

> 
> 5) org.sugarlabs.socialhelp_sso.
> 
> Apparently for socialhelp.sugarlabs.org/sso. is it working? Listening 
> on port 5005.

Ah, I got the name wrong.  It is actually 
use-socialhelp.sugarlabs.org/sso.  To test that it is working, you can 
go to "socialhelp.sugarlabs.org" and then press log in.  You should 
then be redirected to the socialhelp sso page.

> 
> 6) org.sugarlabs.bundlebin.
> 
> Like a pastebin for SL. The code is hosted on 
> http://github.com/samdroid-apps/bundlebin. Listening on port 5000.
> 
> 7) org.sugarlabs.nagios.
> 
> Nagios instance. Currently not working the postfix service inside the 
> container. I should fix this or we should create a common postfix 
> container in order to link this and other containers to it.
> 
> Listening on port 8081.
> 
> 8) today.sam.sugarstick-creator
> 
> Listening on port 5008.
> Can you explain a little more about this?

Tracker/emailer for sugarstick-creator project.  It was basically an 
experiment that didn't end up shipping with the new site.

I have removed and shut down this container.

> 9) rethinkdb
> 
> Only listening locally on ports 8080, 28015 and 29015. What is the 
> purpose of this container?

RethinkDB container that was linked to sugar-stick-creator.  Not a very 
descriptive name.  I have killed it as the parent is killed.

> 
> 10) org.sugarlabs.activities-2
> 
> Listening on port 5006.
> Self descriptive. are we going to kill it?

Yes.

> 
> 11) org.sugarlabs.use-socialhelp
> 
> Listening on port 5003.
> 
> 12) org.sugarlabs.help_rebuilder
> 
> Not listening. According to main.py, this is used with kafka.

Yep, this one is the painful one that was integrated with aslo2 through 
the hook service.  I'm probably just going to move all the webhooks 
into 1 container instead of having kafka between them.

Thanks,
Sam

> 
> 13) local_discourse/app
> 
> Listening on port 8002 (web) and 2222 (ssh). Here we host 
> socialhelp.sl.o.
> 
> Sam, if you can give more information about each container would be 
> pretty useful for everyone.
> 
> How we build the images and containers?
> 
> Docker can build images automatically by reading instructions from a 
> Dockerfile, a text file that contains all the commands, in order, 
> needed to build a given image. We host all the files needed by a 
> every container in /containers. Actually, we can find there a symlink 
> for every container. For example:
> 
> org.sugarlabs.bugs -> /srv/containers/org.sugarlabs.bugs/
> 
> In case we want to deploy a new service, we can write our own 
> Dockerfile or we can get one from DockerHub (docker image repository).
> 
> We can check the built images in our host with:
> 
> docker images
> 
> We can create a image from a Dockerfile executing the following 
> command:
> 
> docker build -t OUR-IMAGE-NAME .
> 
> In the last example, the cwd is the container folder where the 
> dockerfile resides. Once the image is created, we can create a 
> container (that run as a daemon) from that image executing:
> 
> docker run -d --name CONTAINER-NAME IMAGE-NAME
> 
> In the last command, we are not publishing our service. In order to 
> bind a container to a specific port we should the -p flag. For 
> example;
> 
> docker run -d -p 8081:80 --name CONTAINER-NAME IMAGE-NAME

I'm gonna chip in here an say please use container.yml or docker 
compose or something to that effect.  Having 30 different docker run 
commands that need to be run is very messy, and it much better if the 
runtime information (volumes, ports, cpu quote) is stored in some form 
of a file.

> 
> This would map port 80 inside the container to port 8081 on Freedom.
> 
> We can customize our container at the moment of its deploying giving 
> to the docker run command some options. For example:
> 
> --cpu-quota: in order to limit CPU usage. We have another email 
> thread explaining its usage.
> --memory: memory limit. We have another email thread explaining its 
> usage.
> --restart: restart policy to apply when a container exists.
> --volume: bind mount a volume. Very useful to keep container files 
> inside our own chosen location instead of a docker default location. 
> For example: keeping configuration files of nagios inside 
> /srv/nagios3. This let us re-create the entire container every time 
> we need/want without losing any data.
> 
> Example:
> 
> docker run -d -p 8081:80 --cpu-quota=20000 --memory=512M 
> --volume=/srv/nagios3:/etc/nagios3 --name org.sugarlabs.nagios nagios
> 
> In Sugar Labs, we use a Sam's script called container.yml [1] in 
> order to make docker container configurations live in files. So, we 
> can store all this container configuration inside a container.yml 
> file :-)
> 
> For example: container.yml
> ports:
>  - 8081:80
> 
> volumes:
>  - /srv/nagios3:/etc/nagios3
>  - /srv/nagios-plugins:/usr/lib/nagios/plugins
> 
> memory: 512m
> cpu: 20
> 
> In order to run the container with the settings defined in the 
> container.yml file, we simply do:
> 
> container.yml start -d
> 
> We can also build images using container.yml script. It is a good 
> idea to read container.yml code in order to understand how it works.
> 
> How can we check a containers' status?
> 
> docker ps
> 
> How can we stop/start a container?
> 
> Stop: docker stop CONTAINER-NAME
> 
> Start: docker start CONTAINER-NAME
> 
> We can also use start/stop from sam's script.
> 
> How can we get a console from a Container?
> 
> docker exec -it [CONTAINER-ID or CONTAINER-NAME] bash
> 
> Best regards,
> 
> Samuel C.
> 
> 
> [1] https://github.com/sugarlabs-infra/container.yml
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sugarlabs.org/archive/systems/attachments/20160310/3c87b3dc/attachment.html>