Hello 👋

Ivan's here, with a monthly roundup of all things Server Side for September. The issue is well overdue, but I have a great excuse - I've been heads down finishing the first module of my hands-on Docker Roadmap, and I'm happy to announce it's finally ready! 🎉

What I was working on

The primary focus has been on the Docker Roadmap, but this work has also resulted in a nice spin-off tutorial, which ended up being one of the most-read articles I've written 🚀 Plus, I also added some gamification mechanics to incentivise people to solve more challenges.

Docker Roadmap

Completing the first module is a significant accomplishment - this month alone, I prepared a dozen challenges and drew numerous diagrams.

If you go through the 14 challenges of the "must have" path, you'll become fluent in all things running and managing containers, and if you augment it with the "digging deeper" materials, you'll also expand your understanding to other container runtimes such as Podman, containerd, and nerdctl, and the underlying Linux concepts such as namespaces and cgroups.

Three things about this roadmap that set it apart:

• No prior Docker or containerization knowledge is assumed - the challenges are intentionally designed and ordered in such a way that you start with the docker run basics and gradually build up your skills.
• No tutorial trap whatsoever. I call this approach "practice first, read up second". You jump straight into running simple containers and cover the knowledge gaps "on demand."
• Lots of visual explanations on the way.

For the most comprehensive experience, including the prerequisites and the "digging deeper" path, you can refer directly to the Roadmap, but if you're only interested in the "must have" part, it's also available as a focused skill path: Docker 101 - Run and Manage Containers.

My personal belief is that anyone who solves the first module end-to-end will find themselves far ahead of the pack in understanding Docker.

Tutorial: How Container Filesystem Works

While double-checking some implementation details for one of the Docker 101 challenges, I accidentally found myself falling into a rabbit hole of how exactly container filesystems are prepared. It took me several weeks to get to the bottom of it and then write my findings down in the form of the traditional "iximiuz deep dive". Luckily, such diversions often lead to the best-performing posts.

The How Container Filesystem Works: Building a Docker-like Container From Scratch tutorial has already made it twice to the first page of Hacker News (1, 2) and has also been mentioned in several large newsletters, making September by far the most fruitful month for iximiuz Labs (including the good old blog days):

The influx of extra visitors, as expected, resulted in more sign-ups, so another record has been broken. The number of new monthly Labs users has been steadily growing, but the spike triggered by the release of Challenges as a content format back in January 2024 set such a high mark that it took 18 months to surpass it 🚀

Reminder: Get Better Playgrounds by Solving Challenges

In September, I added a tiny bit of gamification to iximiuz Labs, hoping to incentivise people to do more hands-on learning. Two weeks in, I can clearly see that it was a good move!

Here is a quick reminder if you missed the original announcement. From September 22nd, you can improve your playgrounds by solving challenges:

• Each successful submission permanently increases the maximum daily usage time by 5 minutes.
• Solving 3 challenges makes your playgrounds' Internet access unlimited.
• Solving 10 challenges grants you a month of free Premium access (one time) 🎉

Based on the feedback I've got so far, people are enjoying both solving the challenges and receiving the free perks, and the above graph provides perfect confirmation.

Speaking of challenges, you can now subscribe to notifications when a new challenge is released. The knob appears upon a successful submission and can also be found on the labs profile page. Plus, for your convenience, you can click here.

This way, you wouldn't miss the new challenges I released in September 😉

Linux

• ​Mount an Existing Directory at a New Location (bind mount)​
• ​Pause and Resume Linux Processes with cgroup Freezer​

Docker

• ​Stop and Restart a Containerized Application Preserving Its Data​
• ​Docker Volumes: Upgrade a Containerized App Without Losing Data​
• ​Avoid Data Loss When Running Database Containers: Named vs Anonymous Volumes​
• ​Free Up Disk Space by Removing Unused Docker Volumes​
• ​Restart an Application Container on Failure Automatically​
• ​Signal a Running Containerized Application​
• ​Pause and Unpause a Containerized Application​
• ​Create, Start, Pause, Unpause, Stop, Restart, Kill, and Remove a Container​
• ​Run a Sidecar Container in the Namespace of Another Container​
• ​Execute Host Commands Inside a Running Container​

What I was reading

​Kubernetes v1.34: Autoconfiguration for Node Cgroup Driver Goes GA - Many people get genuinely surprised when I mention that Docker and Kubernetes (often) use systemd as a "cgroup driver". The thing is that cgroups are managed by editing a virtual cgroupfs (/sys/fs/cgroup), and as with any set of shared files, you only want one process writing to them. On systemd-backed Linux distros (most of them), you actually don't have much choice because systemd itself is a heavy user of /sys/fs/cgroup (with or without Docker or Kubernetes installed). So most of the time, you'd want to delegate cgroup operations to systemd. And so do Docker and Kubernetes. However, in the case of Kubernetes, there are two components that require cgroup manipulation - kubelet and the CRI runtime (containerd, cri-o, etc.), and it's very important that they both use the same driver (either direct cgroupfs manipulation or systemd).

​Kubernetes v1.34: Use An Init Container To Define App Environment Variables - A new alpha feature that allows to populate a (regular) file from an initContainer and then make Kubernetes parse the file's content into environment variables for other containers of the pod. Sounds neat and I particularly admire the clever use of initContainers, but since it's a regular file (the implementation relies on an emptyDir volume), there is a risk of leaking secrets (from env variables) to any other workloads/operators with full access to the host's filesystem.

​Kubernetes Pod Scheduling: Tutorial and Best Practices - A 6-chapter series on all things pod scheduling that walks you through the basic concepts, and then dives into node selectors and taints, pod disruption budgets, affinity and anti-affinity, and more. All with a bunch of demos and practical examples.

​Microservices Are a Tax Your Startup Probably Can't Afford - I wouldn't communicate it better! Two things young(er) teams and companies should avoid like fire are Kubernetes and microservices. If you're starting from scratch, jumping straight to microservices will most definitely make you pick the wrong service boundaries. On top of that, you'll have to deal with multiple repositories, hence multiple toolsets, coding standards, and CIs (and these things sprawl), and setting up local dev environments will become a nightmare. Add to it communication issues, overcomplicated monitoring and observability setups, and you'll clearly see how a monolith would have been such a saner choice. The only few reasons to introduce a service (I wouldn't even call it a microservice) are when you have to scale the new component separately from the existing one(s) and the new component has to use a different stack (e.g., a BFF and user-facing API written in Node.js, but the compute-heavy background jobs written in Go and scaled separately).

​5 Core Distributed Concepts Every Developer Should Know - I still remember that epiphany moment when I realized for the first time that my design might be fighting the CAP theorem, and I'm trying to deliver a theoretically impossible solution. This (relatively short) post summarizes the most important distributed system design concepts rather well: the CAP theorem, vertical and horizontal scaling, fault tolerance patterns, and more. Do recommend.

​90% - AI (agents) write 90% of the code in Armin Ronacher's new project. And this is perfectly fine and doesn't diminish the importance of the skilled engineer behind them. A very pragmatic take, which is very close to my own late experience with iximiuz Labs and a few other projects.

​Rebuilding Devin for Claude Sonnet 4.5: Lessons and Challenges - One interesting thing about agent development is that "tricks" that made your agent perform well with a model A will not always make your agent perform as well with a model B, even if B is more powerful than A. And it's not limited to just the system prompt - context management and tool calling "best practices" can differ quite a bit, too. The Cognition team described this experience rather well.

Wrapping up

Now that the first roadmap module is done, I'm finally returning to another long-awaited piece - Persistent Playgrounds. Let's see if I can actually ship this feature within a two-week timeframe this time. But with the new ZFS-powered backend, it should be much more realistic 🤞

If you found this update helpful, please consider supporting my work by getting iximiuz Labs Premium (it comes with a bunch of perks).

Ah, and almost forgot! The iximiuz Labs site is now available at ServerLabs.io, making it easier than ever to share the link with a friend.

Happy learning!

Ivan