YHK - yet another hacker news clone

Docker is weird

116 points psxuaw 2 hours ago 42 comments

shykes 1 hour ago | parent

Hi, I'm the creator of Docker and came across this post. It's always fun to see other engineers think about how you designed a system. I haven't been involved in developing Docker in many years, but it's a fun trip down memory lane.

I'm happy to answer any questions you may have.

irishcoffee 1 hour ago | parent

Oh wow, hi! I think my biggest “gripe” with docker is that everyone grabs it as a hammer and views many problems as a nail. Am I way off base with that? What was your original vision for how docker might be used in the wild?

skinfaxi 1 hour ago | parent

Given how things have changed since your creation and the way things are going, are there any things you would have done differently?

gradus_ad 1 hour ago | parent

General question. When you were designing Docker, did you anticipate the success it would achieve?

shykes 1 hour ago | parent

To give a sense of time: Docker launched in 2013, but it was one of many iterations of the same project, which I had been working on continuously since 2008. Over those 5 years, virtually nobody cared or understood what the hell I was working on. I lived in France and had no connection to the tech industry.

So, not only did I not anticipate this kind of success, I did not even have a reference frame for what success might even look like. All I knew was, I had an itch that I had to scratch, and I had nothing better to do :)

Waterluvian 1 hour ago | parent

Was it always called Docker? Do you think the branding played a larger role in success than you would have expected?

ak217 14 minutes ago | parent

I believe Docker was always called Docker, but it grew directly out of the previous generation of the technology called dotCloud (the company Solomon founded), which was using LXC with AUFS.

I think the breakthrough with Docker was bundling the AUFS-based image layer format with a centralized container image sharing registry. This was revolutionary at the time. But another huge element of success was Solomon and others evangelizing the technology relentlessly at meetups and conferences. I distinctly remember attending a demo in 2012 where the crowd was very excited to see the Docker workflow that we now take for granted, in large part because Solomon was so good at presenting it. At the time, I was working at a cloud PaaS company where we developed a similar technology on top of LXC and BTRFS (but no image registry). Docker really made me think hard about what we were doing, although I didn't end up making any moves.

https://jpetazzo.github.io/2017/02/24/from-dotcloud-to-docke...

hack1312 10 minutes ago | parent

It used to be dotCloud iirc

ecliptik 49 minutes ago | parent

In those years in the systems group I was working in at a fortune 500, we were very interested in this type of technology. Mostly using Solaris Zones and cgroups and trying to do this ourselves with limited results.

When Docker came out it solved so many issues we were having. Thank you for scratching that itch.

xtracto 41 minutes ago | parent

I remember in 2013 the first time someone showed me Docker [im in Mexico]. My first thought is that we already had Vagrant for setting up and running virtual machines through console.

I didnt get it at that time haha.

layer8 1 hour ago | parent

If you were implementing an operating system (free from compatibility concerns), what would you do differently to better support lightweight containers?

figmert 1 hour ago | parent

I'm curious what made think of it in the first place? What was your thought process that made you think that these Linux primitives give the opportunity to make it a platform for distributing and deploying packaged applications and eliminating a lot of the issue around "it works my machine" argument?

You may have already answered this somewhere, so feel free to tell me what to search for or if you have a link to hand, that works too :)

bijmtir 23 minutes ago | parent

Docker was basically a commercialisation of lxc and now we got nspawn going full circle.

iJohnDoe 13 minutes ago | parent

I’m a huge fan of nspawn. Always wanted to use VMs instead of Docker and nspawn feels like a natural choice.

miladyincontrol 9 minutes ago | parent

Its nice having the flexibility for both system containers and application containers. It does feel though that most people forget that nspawn does both, and that it only can do system containers.

lostlogin 7 minutes ago | parent

The LXC way of getting an up per container is pretty great - I’m basic bitch with this sort of thing so I assume it’s less of an issue for the experts but really enjoyed the IP per container when I hit it.

shykes 4 minutes ago | parent

This is incorrect.

Docker in its earlier incarnations was based on openvz, which predates lxc. The lxc project was initially a kernel development project: a collection of kernel patches to add container support. The userland tool was just a companion. When it became apparent that these patches would "win" the race to be upstreamed even though openvz was much more mature, we switched our tooling to lxc. The patches matured over time, but the userland tool (what we now know as "lxc") was finicky.

The original open-source release of Docker wrapped lxc for low-level container operations. But it was just too unreliable and we swapped it out for an in-house Go implementation called "libcontainer".

okanat 1 hour ago | parent

Would you consider writing Docker/moby in a different language, if you were to create it scratch today?

shykes 1 hour ago | parent

Oh gosh, the choice of language is such a context-specific design decision. The first versions of what became Docker were written in Python. The choice of Go was a mix of nerdy excitement ("it's like the best of Python and C!"), and adoption strategy: back then devops tools were segmented by language runtime: ruby, python or Java mostly. Shipping a compiled binary allowed us to be adopted by all tribes.

None of these things can be transposed today, so I guess I can't answer...

For what it's worth, my current project (dagger) is written in Go.

Tenemo 59 minutes ago | parent

Do you think e.g. students learning docker syntax at a uni is beneficial in 2026 or you think LLM-assisted programming means that Docker will become an abstraction under an abstraction and its setup will be mostly automated by LLMs? I find myself very rarely debugging or handwriting dockerfiles nowadays!

craftkiller 13 minutes ago | parent

This is a strange question because you could learn Dockerfile syntax in a single night. There are only 18 directives, and the average Dockerfile uses like 6 of them. 90%+ of Dockerfiles is just "run this shell command" and "copy this file into the docker image". The only time it flirts with complexity is multi-stage Dockerfiles, but 90%+ of Dockerfiles are a single stage.

https://docs.docker.com/reference/dockerfile/

xyzzy_plugh 11 minutes ago | parent

I'm gonna be real with you. No one needs to know docker syntax. Most engineers just copy and paste chunks of Dockerfiles around anyways.

Docker syntax is one of those things that just immediately falls out of my head when I close the editor. I don't care about it at all.

Even better: there are many, many tools that build OCI images without using the Docker syntax. I haven't hand-written a Dockerfile in at least six years.

LLMs have not loved the needle here in my sphere, it was already bouncing off of the zero on the gauge.

edit: this isn't to say the syntax is good or bad (though there are several questionable parts of it) but extremely simple, and concise enough that you can just look up literally any command with great ease. It's surely a large part of its success, for better or for worse.

davnicwil 55 minutes ago | parent

I'm a big fan of docker and mainly just want to say thanks for bringing it about.

I remember my first time trying to use a container was about 2015 on a beach cafe's wifi in Bocas del Toro, Panama. You can imagine how it went...

After a brief hiatus though I was back to it and since then it's been a core part of many a local dev stack and even production stack (compose) I've set up. I recall the magic moment for me being when I first realised you could mount src to use all your local native dev tools while completely virtualising everything else, including and especially whatever combo of runtimes and dependencies you need.

It was just a lightbulb moment, like it entirely solves 'works on my machine', setting new people up on a project, it just seems like how software engineering ought to work and once you have experienced it it's difficult to go back.

I'm curious, as its creator are you still using it, or as someone who can clearly look ahead to experimental ways of doing things are you on to the next thing whatever that may be? If so, what is it? :-)

Fabricio20 44 minutes ago | parent

Thanks for developing such a valuable tool!

999900000999 5 minutes ago | parent

Thank you.

For one, did you get rich beyond all comprehension and did you start a band with your wealth?

Second, do you still program ?

I was lucky enough to get in this industry when Docker was new. Sure beats trying to figure out why X happens on machine A and doesn’t on machine B.

suprjami 1 hour ago | parent

Always nice to see someone demystifying a complex thing by learning the building blocks it's made from.

Knowing this, you might find rootless Podman's fork-exec method of starting containers to be more sensible. I certainly do. No root, no daemon, no socket.

If you want to learn more about the Linux Kernel namespaces which work together to make a container, this LWN series is really good: https://lwn.net/Articles/531114/

QGQBGdeZREunxLe 1 hour ago | parent

A good talk on the tech behind containers.

Containers From Scratch • Liz Rice • GOTO 2018

https://www.youtube.com/watch?v=8fi7uSYlOdc

tyingq 1 hour ago | parent

Poke around in bocker[1]. For technical people, it makes the same points the article makes in a way that's very clear.

[1] "Docker implemented in around 100 lines of bash" https://github.com/p8952/bocker

gzapp 1 hour ago | parent

That part about the threading was a huge source of issues back in the early days. Goroutines could hop threads and it's the threads namespace/cgroup configuration that gets inherited when forking..

One of the gnarliest issues I recall was where network namespaces could occasionally not be cleaned up correctly. So "randomly" (but more likely under certain loads) the networking of a reused thread would get jacked and Docker could no longer pull images.

j16sdiz 40 minutes ago | parent

Every statement in the article make me ask: "what's weird about it?"

Everything in that list feel natural when you know how things evolves over many years. Maybe it is just me getting older and less curious.

paxy 37 minutes ago | parent

A lot of people start using Docker as a black box and have the mental model of a VM, and when they start digging into the internals all the "weirdness" starts to exhibit itself.

stingraycharles 36 minutes ago | parent

And not a single mention of the actual weird decision, a client / server model which complicates permission management by at least two orders of magnitude.

slowin 31 minutes ago | parent

This is why I prefer Podman.

modeless 9 minutes ago | parent

Yeah, the daemon is the real problem with Docker.

assimpleaspossi 33 minutes ago | parent

>>Images are just folders that Docker puts somewhere and then calls chroot (or pivot_root) to make that folder your container process’s new root directory.

And folders are graphical representations of directories--sometimes, not always--and not directories themselves. When will the newbies get this?

sophacles 15 minutes ago | parent

Directory is a bad metaphor too.

slowin 33 minutes ago | parent

I prefer Podman most of the time, but Docker is/was amazing and a great DX put on top of the so called "5 different linux kernel features in trenchcoat". That's the selling point, it's super lightweight and takes full advantage of the OS features that were already there (or being developed) but were awkward to use and required tons of specialized knowledge.