Why I Containerized My Entire Homelab (And the One Mistake That Broke Everything First)

The old homelab only exists in memory now, and not by choice. I never documented anything. Config files weren’t saved, decisions weren’t written down, and the only record of what I’d done was whatever I could still recall two years later when something inevitably broke (usually at the worst possible time, because of course).

I ran bare metal across a few boxes over the years. Started with an Intel Atom x5-z8350 with 2GB of RAM, a cheap little machine that punched well above its weight until the eMMC drive died on it. Non-replaceable, the eMMC was soldered to the board, so it became a machine that booted off a 128GB flash drive instead. Because of course it did, what else was I going to do, throw it away? That’s how it ran for a while, until a Dell Optiplex 7010 showed up at a price I couldn’t pass on: i5-3470, 8GB of RAM, an actual machine with actual storage. The Optiplex took over. Services installed directly on the OS, dependencies fighting each other for the same Python version, an NFS share feeding a Kodi install on a Raspberry Pi, and no record anywhere of how any of it was configured. It worked, until an update broke something, then I’d spend a weekend untangling it, then I’d forget how I fixed it, and six months later I’d fight the same problem again. Then life intervened and I walked away for seven years.

The Optiplex was still running when I left. Some hardware just won’t die, and the 7010 is a good example. It probably could have kept going for another decade.

Christmas came and my gifts were wrapped under the tree: Ryzen, RAM, NVMe, Arc A380, all of it, exactly what I’d been hinting at for months. Seven days before I was supposed to open any of it, I had a heart attack. The new build sat under the tree, fully wrapped, sealed in shipping boxes, while I was in the hospital being told to relax. I would have loved to see the staff’s faces if I’d tried assembling a home server from the hospital bed. As it turned out, having something to do during recovery was probably one of the better things that could have been waiting for me. The 85TB of spinning drives came with me from the old system, because data doesn’t care what box it lives on. Everything else started over. And I made a decision somewhere between the hospital and the workbench: I was not recreating the old mess. Docker, all of it, day one.

Why Docker

Two things broke the old setup repeatedly. Services that stepped on each other (because they shared the same OS and the same Python and the same everything else), and configs that existed nowhere except whatever I could still remember about them. A kernel update would shift something subtle and I’d be debugging with no baseline to return to. A Python version conflict would surface six months after I’d already forgotten the dependency chain that caused it. Fix it, forget it, fight it again. Repeat for years.

Docker fixes both of those problems, more or less by accident. Every service runs in its own container with its own filesystem and its own runtime, so nothing on the host can conflict with anything else, no matter how many Python versions are involved. Break one container and the rest keep running, blissfully unaware. More practically, and this is the part that actually matters: the entire setup lives in a docker-compose.yml file. Version it, back it up, read it back a year later and you know exactly what you did. That alone was worth the learning curve.

And when I eventually move balor to new hardware, it won’t be a reinstall. It’ll be copying files and running docker compose up. Not there yet (the Ryzen has plenty of life in it), but knowing it’s possible is a completely different feeling than the old way of doing things.

Going All In

I didn’t migrate gradually, because there was nothing to migrate from. The new box was empty and the Optiplex was still sitting there if I needed to reference anything, so I just started building from scratch. One service at a time: stand it up, figure out the compose file, get it talking to the data, move on to the next. No hybrid period, no half-the-stack-containerized awkwardness. By the end of it I had 32 containers running and compose files I could actually read and understand a month later.

The layout I landed on: compose files under /mnt/dockers/dockercompose/, one directory per service. Persistent data under /mnt/dockers/Docker/, same structure. Nothing monolithic, nothing shared between stacks that doesn’t need to be. You can restart Jellyfin without Nextcloud knowing anything about it, which is exactly the point.

/mnt/dockers/dockercompose/
  jellyfin/
    docker-compose.yml
  sonarr/
    docker-compose.yml
  nextcloud/
    docker-compose.yml
  ...

/mnt/dockers/Docker/
  Jellyfin/
  sonarr/
  nextcloud/
  ...

The Thing That Broke Everything First

Nobody explains this well enough, and it’s the one thing that’ll have you ready to throw the whole stack in the trash, so here it is.

A media stack has Jellyfin pulling files that Sonarr, Radarr, Lidarr, and Bazarr are all writing to, plus whatever downloader is feeding the whole thing. Every one of those services is in its own container, and every container has its own filesystem. Your media pool lives on the host at something like /mnt/merged, but a container has no idea that path exists unless you explicitly mount it in the compose file. Most people figure that part out pretty quickly.

The part that causes real chaos is mounting it at different paths in different containers, and it’s the kind of mistake that’ll have you doubting your sanity.

Picture this: Sonarr finishes a download and dutifully moves it to /data/tv/Show Name/. Jellyfin is over here looking for things under /media/TV/. The file is sitting right there on the drives, both containers are running, nothing is broken at the OS level, and Jellyfin’s library scan returns absolutely nothing. Sonarr thinks it did its job. Jellyfin thinks there’s nothing there. You’re pulling your hair out staring at a folder full of media that nothing in the stack can see. I hit this. Both directions at once, naturally. Took longer than I’d like to admit to figure out what was happening.

The fix is embarrassingly simple once you see it: mount your storage at the same path inside every container that touches media. Here the pool is /mnt/dockers/merged on the host and /media inside every container, no exceptions. One line in every compose file, identical across the board:

volumes:
  - /mnt/dockers/merged:/media

When Sonarr moves something to /media/TV/Show Name/, Jellyfin already knows where /media/TV/ is, because they’re looking at the same path. They’re speaking the same language because they’re reading from the same map.

Day to Day

Thirty-two containers, and the host mostly just sits there humming. Updating a service is docker compose pull and docker compose up -d, and that’s it (I'm lazy, I aliased this on my box to dockerupdate in .bashrc). The old service stops, the new image comes up, the compose file hasn’t changed. If something breaks (which happens occasionally, because the people maintaining these images are human too), I roll back to the previous image tag in the compose file and bring it back up. The host OS has no idea what version of Jellyfin is running and doesn’t need to.

Portainer handles the UI when I want one, syslog-ng aggregates logs across containers, but the compose files are what actually matters. If the machine died tomorrow I’d restore those files and the data volumes and be back. That was never true before. Before, if the machine died, I was starting from scratch and guessing, possibly while crying.

The old setup wasn’t badly built. It was just never written down anywhere except in my head. This one is. That’s the whole difference, and it’s a much bigger one than I thought it would be.