profile

Ivan on the Server Side

New ways to experiment on iximiuz Labs: instant clones and click-ops playground creation


Hello πŸ‘‹

A week ago, I announced the release of Persistent Playgrounds. The feature was long-awaited and garnered significant initial traction and constructive feedback. In this follow-up email, I want to introduce the very first improvement release and outline a couple of powerful new use cases enabled by Persistent Playgrounds.

But before we dive into it, I'm obligated to remind you about the ongoing Black Friday sale, and that it is the last (hence best) chance to opt in for the all-inclusive lifetime premium plan at a ridiculously low price. The sale ends in a few days, and the all-inclusive plan itself will be decommissioned by the end of the year (of course, all existing members will retain their benefits indefinitely).


First, a very quick intro of two new QoL improvements I just shipped:

Playground runs can now be titled (regardless of whether it's a persistent run or not) - simply click on the title of a running playground and type in whatever you like, replacing the default Run 69270... string. Note that a playground run title is not the same as the playground name (e.g., a playground named "Docker" can have many running or stopped instances of it titled "My Docker sandbox", "Conference talk demo", "Issue #423 repro", or any other creative wording you can think of).

Stopped playground can now be found in the dashboard (actually, they've become front and center of it). You can stop running or restart stopped playground runs, clone or save runs as custom playgrounds (more on this below), and see the current storage usage. This is also where the above titles will come in handy - it's much easier to keep track of things when you can "label" them meaningfully (instead of trying to remember what hex ID corresponds to what saved playground run).

Now, moving to more interesting additions. I've just shipped two major features that take the recently released persistent playgrounds to an absolutely different level. You can now:

  • Clone a stopped persistent playground run - the operation is almost instant and takes no extra storage space.
  • Save a stopped persistent playground run as a custom playground - the saved "snapshot" can then be used either as a standalone playground or embedded into a tutorial, challenge, or a course lesson.

Both Clone and Save operations are available through the dashboard's new Persistent Runs widget and the corresponding buttons on the stopped run page (see the screenshot below):

Let's look at a few use cases for iximiuz Labs Playgrounds that, in particular, highlight the new capabilities.


A Cheap yet powerful homelab alternative

The very first release of iximiuz Labs Playgrounds back in 2023 (oh my...) already had VMs that felt very close to real-world servers. I intentionally chose Firecracker over (overly) hardened containers to provide execution environments capable of running all typical workloads, such as Docker, Kubernetes, and even kernel-picky software like Cilium or Tetragon. Yes, it took me a lot of extra effort to tweak the Firecracker's stock kernels into much more fully-fledged variants, but it’s definitely worth it.

In three years of the platform's existence, I've added the ability to customize playgrounds with user-defined init scripts, assemble playgrounds from an arbitrary set of virtual machines (up to 5 per playground), connect playground VMs into multiple LANs simultaneously, use multiple drives per machine, and even bring your own rootfs images.

​Finally, in the latest release, I added the ability to "switch off" and "turn on" the playgrounds, picking up from where you left off (before that, switching off a playground would fully destroy it, including any data you may have left on its drives).

All these capabilities make a multi-node iximiuz Labs playground as capable as a typical homelab. You can set up your own Kubernetes cluster, a 3-tier web app, a CI/CD pipeline, a fancy network topology, or even a sandbox execution environment for a herd of agents, and when you’re done for the day, the results of your work won't go away. Just hit the Stop button and continue tomorrow, next weekend, or when you're back from vacation.

Now speaking of vacations, digital nomad lifestyle, or working from unusual places, this is one of the key reasons I've personally never had a homelab. I like the idea, and I believe it's a must for any true server-side guru to have their own fleet of servers to manage, but the physical format of the homelab has always sounded too limiting for me. At first, I was substituting for it with more powerful laptops where I'd run a bunch of local VMs, but when Apple Silicon "ruined" this use case (VirtualBox stopped working on early-day Apple's Arm chips), I switched to renting first remote virtual machines and then bare-metal servers to play with various tech.

But the thing is, if you want a 5-node HA Kubernetes cluster home lab, hosting it on 5 DigitalOcean droplets (or the like VPS) would cost you at least $20/mo. And what if you want several labs? The price of learning and experimentation becomes unbearable very quickly. Renting a single large Hetzner bare-metal server and slicing into a dozen VMs can be a "cheaper" alternative at a certain scale, but the starting price jumps to $40/mo. Plus, slicing itself isn't trivial.

The upside of a real homelab, DigitalOcean, and Hetzner is that you can keep your workloads running overnight. But do you really need your test Kubernetes cluster available 24/7?

At the same time, on iximiuz Labs, you can host a close to unlimited number of labs (no pun intended), with up to 100 GB of "paused" VMs, which you can restart in under a minute. Plus, you can enhance your labs with a nicely looking "front page", configure access control, and share your labs publicly or with a limited set of collaborators (e.g., your study group). And of course, you will also benefit from the ever-growing collection of the official and community-maintained playgrounds. All this for just $12/mo (undiscounted, the actual price is often much lower thanks to regional PPP discounts). Honestly, if the physical aesthetic of a homelab isn't your top priority (this part has its charm, I admit), choosing iximiuz Labs should be a no-brainer.

Instantly replicate your environments

This is a brand new feature with a quite advanced implementation but an extremely simple UX - you'll get the idea in no time.

Let's say you started a new Linux VM, installed a bunch of packages on it, configured some services, maybe cloned some repos, and adjusted some code to set up the initial stage for your experiment. And now you want to try several ideas, but some of them might be mutually exclusive - they can spoil or damage the setup beyond recovery. A typical example is upgrading some system packages or partitioning a drive (I bricked several servers last week while upgrading them from MBR to GPT).

In an ideal world, you'd make your provisioning steps fully scripted and re-run them against a fresh VM before each attempt. But who thinks of such things in advance… I definitely don't. So, in real life, this situation would often mean you just got one shot or signed yourself up for a "fun" manual cleanup exercise after each unsuccessful attempt.

...unless you were running your experiment on iximiuz Labs! πŸš€

Since just a few days ago, it has become possible to stop the original playground session and create as many clones as you like with just 1+N clicks, where N is the number of attempts you need until the problem is solved πŸ˜‰

The video demo is somewhat lengthy (and boring), but I encourage you to just give this feature a try - launch a new playground, make some changes, stop it, and then click the Clone button, the more times the merrier:

video preview​

The applicability of this feature is not limited to vanilla Linux VMs. It works perfectly for complex multi-VM playgrounds, including Kubernetes clusters with a bunch of Helm charts installed. And the best part of it - clones are created almost instantly and take no extra space upon creation due to ZFS's copy-on-write magic and content-addressable storage of remote snapshots (carefully crafted by yours truly).

Save a playground run as a new custom playground

The relationship between a Playground and a [playground] Run is the same as between a Class and an Object in programming. A Class is a reusable read-only template that can be used to instantiate any number of Objects. Such objects can be literally identical (modulo their memory address) or differ slightly because different parameters were used at the construction time. And of course, any modifications of the objects have no effect on their (immutable) class.

The same is true for iximiuz Labs Playgrounds - you can click the Start button on the K3s playground page as many times as you like, and each time you'll get a new copy running. By default, the playground runs will be identical, but if you tweak the startup configuration, the new instance will have fewer nodes or different IP addresses assigned, etc. Conversely, any changes you make in the running instances of the playground won't impact future runs because the playground itself remains an immutable template. That's how it worked from day 1, and how it'll remain.

However, the feature I'm about to introduce has just taken the iximiuz Labs playgrounds model from the traditional Class-Object inheritance model to the rather superior prototype chain-based inheritance implementation. If you're familiar with JavaScript's way of cooking inheritance, you likely already got the idea, but if not, it'll be easy to understand with a practical example.

As always, you start with a template - any existing playground. You launch a new playground run, modify it any way you like by executing arbitrary shell commands, launching services, installing Helm charts, publishing ports, adding extra tabs, etc. And then you stop it. What you've got now is both a stopped persistent playground run and a potential new playground template!

At this point, you have 3 options:

  • Mutate - restart the stopped run and continue working in it
  • Clone - create a "shallow" copy of the stopped run and mutate it instead
  • Templatize - save the stopped playground run as a read-only template

Practically, it has two major applications.

Share your playground runs

Sharing a playground has always been easy. You can set its visibility to "public" or "limited" (to certain user IDs or GitHub nicknames) and send the playground page's link to whomever you like. At the same time, sharing a playground run is deliberately made impossible - it's an ephemeral entity, so by the time the recipient opens the link, the run can already be gone, and it's also a much more sensitive entity (e.g., it can have your SSH agent forwarded if you opted in for this feature).

But at the same time, situations when you somehow produced a certain state (of a single VM or a cluster) and now would absolutely love to "snapshot" it and share the result with someone are very common. The most typical example is working on a bug repro. You managed to reproduce the customer's issue on your machine, you can even stop this run (by making it persistent) to continue working on it tomorrow, but wouldn't it be great if you were also able to share it with a teammate who happens to be in a very distant time zone?

Well, now you can! Simply stop the run and click the Save as Playground button 😎

What you'll get is a good old (shareable) playground, whose initial state will be the "snapshotted" state of your very own playground session. And you'll still be able to continue working on your own mutable copy tomorrow as usual.

"Click-ops" playground creation

Historically, there have been two ways to create custom playgrounds:

  • Add cloud-init-like provisioning scripts to one of the existing base playgrounds (easier to configure, but the startup is slower because scripts have to rerun each time).
  • Create a flexbox-based playground using your own rootfs image(s) stored in a 3rd-party container registry (harder to configure, but the startup is usually significantly faster).

But wouldn't it be fun if you could start an existing playground, make any changes you like by running ad-hoc shell commands, and then save the result as a new custom playground?

Well, now you can! With the exact same sequence of actions as in the previous example: start a new playground as usual, adjust it, stop the run (making it persistent), and click the Save as Playground button πŸ§™


Wrapping up

Persistent playgrounds have been both the most frequently requested and the hardest to pull off feature. It was definitely the most challenging engineering problem I've solved so far in my 15 years of developing software. But the impact of introducing it is huge. Just look at the labs usage charts - the number of hours people spend in playgrounds per day has immediately doubled the moment I shipped it, and we're only starting to realize all the new use cases it enables.

Happy hacking!

Ivan

Ivan on the Server Side

A satellite project of labs.iximiuz.com - an indie learning platform to master Linux, Containers, and Kubernetes the hands-on way πŸš€

Share this page