phusion baseimage

[moltar]

Have you seen this? Why not use that as a base image for this setup?

[clemtibs]

I have actually seen that image before, and there is a lot to like about the things they've put in there. The short answer I guess as to why I don't use it is because I don't want to include so many things in a default image until I know I need them. I'd rather have many small images that are layered that add only the features they need instead of one big fat one that makes many assumptions about things that are needed. And it's not even a size thing, it only weighs about 80mb more than mine, I guess it's just principal.

This image strikes me as really trying to get a container to be a full blown virtual environment for long running and multi-purpose containers. I could definitely see this image as a host for a "docker-in-docker" situation if I were to use it as my virtual "host" to make sure things are as close to normal as possible. I can even see this as a base for situations where docker is used to run remote X servers and other user/gui tools that expect such things as what they added in there. But for now, I'll leave it out until I know i need it or run up against problems in my images. And I may even just add pieces of their image as I need them into my images. To their credit too, I now know what to look out for as possible blind spots as I create my images.

Did you have a specific need that their images offered?

[moltar]

I just wanted to know if you were aware of that, before building your own base images. I like that it makes syslog and cron work. It has runit built in, supervisor is nice too, but I think runit is more stable and less memory.

Also their start up script has command flags that can disable certain behaviours.

IMO it'd be nice combine the work they did, with your ideas. Unless your container addresses all those issues that they describe already here and any other that came in via tickets and Google Group discussion. I don't remember exactly the problems they were running into, but there were some.

Anyway, I am not married to it, and I haven't really explored what you are offering. I was just throwing it out there to see if you had any thoughts.

[clemtibs]

I appreciate the suggestion. I'll keep this open for further discussion and I'll think on it too. Right now I have more ideas and strategy than actual examples and testing regarding my images, so I'll let myself run into these issues as Radial matures and keep their solutions in mind. My hunch is that it's easier from a "drop-in" configuration perspective to use supervisor for most of these tasks, but I'll definitely be revisiting this at a later date.

[damm]

My view on the phusion base image is:

1. It's heavy (and every image you FROM it will be it's size and larger)
   It's no fun shipping large images :(
2. It has a /lot/ of components out of the box that not everyone will use
3. It makes every container a server
   Which goes against the notion that Docker containers should be a single process. People do this every day but it doesn't mean it's right.

As far as best practices go; it could be lighter and provide some of the components only if they are requested. There is Ubuntu Upstart which is a lot leaner and you can add the components as you need them.

Personally my containers ship all my logs to a central store; I also ship all my docker logs to the same endpoint. If your going to use a Container as a full server you really need to log what's going on because it's basically Russian roulette.

[moltar]

@damn:

1. It's only heavy the first time you have to download it, as @brianclements himself said, the difference is only 80mb. Since the baseimage is just a layer, then anything you build on top will be counted as additional size.
2. I think the only extra is SSH daemon. Which is disabled by default.
3. I don't think there is a solid defined use case for single process. I don't think the creators ever said that, unless I just missed it. I've heard this notion being thrown around, but I haven't really seen the evidence. I generally agree that that the design ultimately should be very modular, but not having syslog or cron (or even SSH sometimes), might not very useful.

[damm]

@moltar also we should keep in mind that anything from Phusion is a product; it's goal it to make users who use those products happier. That's not a bad thing; and I do think both answers are completely valid.

I wish I used single process more.

Refactoring is a pain to adopt better standards.

[clemtibs]

@moltar @damm
While I've never seen any of the Docker devs specifically oppose using Docker containers as full servers, whenever I've seen explanations/powerpoints/discussions they've always been coming from the "process container" standpoint with separation of concerns as a general rule. So I'm pretty certain that was always their mindset. Now I know that eventually they plan on allowing multiple processes per container, probably by way of multiple CMD|ENTRYPOINT entries, and it remains to be seen how they do that init-wise. I think that is a problem for their Upstart image to solve.

As for Phusion's selling points, I did some homework on it and am riffing off that list:

A correct init process

Supervisor can manage the processes just fine. There shouldn't be rogue processes going on anyway that can go "zombie", so supervisor scripts for start/stop and other admin processes are the way to go. Just SSH into the container, and everything you would ever want to do to your process should be pre-scripted and then run with supervisorctl {some-task}. I've also seen mentions of future docker features that can send custom signals into your container to help with graceful shutdowns and the like. But until then, it shouldn't be too cumbersome to use SSH for delicate shutdowns.

Fixes APT incompatibilities with Docker

This fix was baked into the official Ubuntu image eventually anyway. So I'm not sure why they are taking credit for it on their page.

sudo docker run ubuntu:14.04 ls -hal /sbin/initctl should show a symlink to /bin/true

syslog-ng

syslog-ng looks great. I don't have any experience with it however, but it seems they're only using it for basic features, which of course, Supervisor is already doing by being the init system. They also seem to forget that Docker containers share the kernel with the host, so outside of your specific process's log (Supervisor), the output of syslog and dmesg aren't relevant or unique to your container (dmesg in your container is the same output as running it on the host).

logrotate

Supervisor

ssh server

got one of those too by default in my spoke container. Same hardened settings. Key-only access. disable via Supervisor .ini if you like.

cron

I can't stand using cron on real systems, and I hope I never have to use it in my containers. I think what they need it for is log-rotate tasks. To that I say "Supervisor".

runit

I thought we already handled init? Not sure if I'm comfortable using runit, I've never really heard of it before or know anything about. It In any case....Supervisor!

setuser

This is kinda cool. But again...Supervisor can run processes as any user desired.

So it really comes down to whether we like doing everything in one program, and specifying all of init/logs/start|stop/processes/user/etc.. in one configuration file (Supervisor) or having to make our own init scripts and try to catch everything in our own nonstandard scripts for our own non-standard init system... no, thank you. 1 program doing everything is better then >1 programs in this case, even at the cost of a couple megabytes of ram (Supervisor is ~13mb I think, but my image is smaller 😋 ) Besides, one of my main goals with Radial was to keep configuration simple and very manageable so that I can splice different pieces together with git and not add very many files when building apps.

Thanks anyway @moltar for the suggestion, I saw that image before and how popular it was, but didn't really dig deeper into their methods. Glad you gave me a reason to do that!

[damm]

@moltar @damm
While I've never seen any of the Docker devs specifically oppose using Docker containers as full servers, whenever I've seen explanations/powerpoints/discussions they've always been coming from the "process container" standpoint with separation of concerns as a general rule. So I'm pretty certain that was always their mindset. Now I know that eventually they plan on allowing multiple processes per container, probably by way of multiple CMD|ENTRYPOINT entries, and it remains to be seen how they do that init-wise. I think that is a problem for their Upstart image to solve.

Basically they tried nicely but there were enough (sometimes rude) users who did not take the suggestion to not use a container as a full server well. You still see people mentioning how you should use single process containers but you can't force people to get it. So they don't beat a dead horse.

Upstart

We should talk about some of the problems about Upstart before we start selling it; There was someone in #docker the other day having problems getting mysql-server to start using the upstart by default; tianon was going to look into this I believe.

I believe when init starts it assumes certain things are going; and does not kick off everything.

• I have to use a upstart file called 'start.conf' to finish starting the services that do not start properly with init by default.

start on (local-filesystems)

task

pre-script script
  /sbin/start runsvdir
  /sbin/start cron

Logging

Really we don't want any logging collection done in the container; /dev/log can drop if there is enough messages coming through so forwarding your logs to a central point (via logstash or whatever) is the way to go.

• It maybe acceptable to configure remote logging; but syslog in a container is a knee jerk reaction because you enabled other services and you want to know what they are doing.

SSH

There's people for or against this. I'm against this because unless you do this with SSH Host Certificates you cannot validate the public RSA key. Basically if your throwing up SSH you should give it a certificate; if you can't you should not /enable it/ because it's a door. If you need access to your container use nsinit or nsenter.

Cron

There are many alternatives to cron out there; truthfully vixie cron is horrible and is known to loosing the crontabs from time to time. You should just replace cron instead of using it.

Runit

Huh?

Security

You mention setuser; but really we shouldn't be giving the user root at all. These are containers; you could treat them as a hands off approach. But really allowing SSH connections to sudo is just asking for you to get owned.

[moltar]

If we were to head single process per container, then why do we even need supervisor or upstart or any of that? Just run it via CMD / ENTRYPOINT and log to stdout. That would also prevent SSH, and log collection from inside the container.

[moltar]

@brianclements: setuser especially shines when you actually build the container and need a bunch of stuff done from other user's permissions, this saves from chown'ing everything at the end.

[moltar]

@damm: what the good alternatives to cron?

P.S. Sorry for multiple messages - early - mind is all scattered.

[damm]

@moltar here are 3 alternatives; there are more out there

1. ResqueScheduler
   Often abused as cron
2. Chronos
   I doubt this will be picked often due to it's dependencies; but I could be wrong.
3. Celery
   If you use Python they support periodic tasks.

Additionally if you use a single container; there maybe a time when you need to say deploy or run tasks? Either have a image that you would --volume-from to get your content and run your deploy script and exit.

Re-using your images and containers to get to your content also provides a more secure mechanism than SSH'ing into a public port.

I avoid using SSH because I don't like people making changes to a container when it's live; often those changes don't go back to the Dockerfile in error (if your in a hurry!)

and really my point about setuser is don't. Just set your USER in your Dockerfile as a non privileged user; don't allow them to su/sudo and win.

[moltar]

@damm: these are not cron replacements!

1. Assumes one is using ruby, how about regular shell scripts? Also would have a huge dep chain to even set that up.
2. Also too big for simply cron tasks
3. Assumes you are using Python.

Sometimes you just need to run once-in-a-while maintenance tasks that cron handles best. It is not meant to be reliable like MQs. Stuff like, deleting temporary files, clearing expired sessions from the database, fetching some data from a remote location, that is not mission critical (e.g. maxmind updates), and so on.

Alternative, I guess, it would be possible to run new docker container from the host via host's cron, attaching needed data via --volume-from and then doing said maintenance.