Managing small nix fleet

How I've commited to nixos and what I feel now.

Table of Contents

Prelude

Over the last six months, I’ve deeply immersed myself in Nix, NixOS, and its ecosystem and tooling.

Since then, I’ve been managing 3 aarch64 SBCs, 3 aarch64 servers, 1 laptop, and 1 PC — all using NixOS.

My current setup is a single repository acting as a snowflake, describing all these systems. Deploying configurations is seamless with the excellent deploy-rs. Alternatively, configurations can be deployed locally by cloning the repository and running the standard sudo nixos-rebuild test --flake .#. For new systems, I can render an SD/SSD image and flash it directly to the target storage using dd.

I primarily use the laptop for coding and the PC for gaming and compiling large projects. The servers, of course, do server things: two of them host my self-hosted apps and their databases—no Kubernetes, no Docker, just pure applications with Nginx as a reverse proxy. One server provides a VPN for my relatives and friends living in dystopian countries with harsh online censorship.

Among the SBCs, one is a Grafana kiosk, another runs *arr services and Jellyfin, and the third acts as a low-fidelity staging environment where I can safely break things.

So with all this diverse range of applications, I think I may have oppinion or two.

Things I'm glad to have

Rich declarative configuration out of the box

NixOS comes with a huge set of options to configure the system declaratively. It covers all base needs — bootloader, fstab, display managers, Mesa drivers, and everything else required for a fully functional system that meets daily needs. No magic happens here: if you can't make your workflow fit a regular Ubuntu setup, Nix won’t save you.

Additionally, NixOS offers versatile configurations for different services. Be it Nginx, PostgreSQL, Redis, *arr, Wakapi, Docker, Gitea, Grafana, Prometheus, or Nextcloud — you name it. Support quality varies across services. In the best case, you get rich, typed configurations; on average, you get a predefined systemd service and the ability to inject your configuration as a Nix-lang key-value map. Worst case? You still have Docker, which Nix supports quite well — allowing you to define containers similarly to Docker Compose but with Nix syntax.

Community driven configuration modules

Community involvement is crucial for any large project, and the Nix community delivers great tools.

I use not so much things in my daily workflow, but here is a list:

  • stylix awesome package for applying system wide styling. Like you don't need to go through all supported apps and change color scheme. Pick one of base16 and you will get theme almost everywhere.
  • Home Manage Simple explanation but it do not explain depth of module: "It is like nixos but for your user home folder". Manage your user personal belongings in a nix way. Install apps only for this user, configure Sway, Firefox, git, gpg, ssh, shell.
  • sops-nix Provide secrets on machine or user level. It may work for you too.
  • nur-expressions I use it to manage addons for my firefox.
  • nixos-hardware Self explanatory. Bits for fast prototyping hardware configuration.
  • deploy-rs Spread your flakes on your fleet. Designed for remote provision, but works for local too.
  • nixos-rk3588 Usefull set of modules for RK3588 SBCs. Sadly it is archived now.
  • nixos-infect Infect machine from your cloud provider with Nixos.

Devshell

For me it is a huge deal. I use it both at work and for personal projects. Combining flakes, direnv, and nix-direnv gives me ready-to-use development environments instantly. If you’re curious, read Amos's excellent series: https://fasterthanli.me/series/building-a-rust-service-with-nix/part-10

Overlays and you own apps

Nix makes it simple to package apps if they use common build systems like Meson. I build a couple of 3rd party forks of mpp, ffmpeg and libmali myself. Works fine. Also you can modify any package from nixpkgs with reach overlay system.

Things I'm not impressed with

Stability

I’m used to rolling-release distros, so I understand "stability" in my own way: I don’t update systems during the workweek, only on Friday evenings. Tbh, this stability is good enough. Paired with generations and away to rollback on boot you get a robust system.

However, configuration stability is a mixed bag. Even on release branches of nixpkgs, I occasionally encounter breaking changes in configuration options over short timeframes. Community-driven modules are less stable.

I don’t blame anyone—this is just how decentralized projects work. Pinning module versions and updating them sparingly helps manage this pain point.

Systemd

Welp, it’s too late to blame systemd for everything, but in NixOS, services, hooks, and one-time jobs all rely exclusively on systemd units. That’s it. I’m not a fan.

Things I hate

Language

Nix is a pure, functional, lazy, declarative, and reproducible programming language.

Just keep in mind that it is not general purpose programming language. If you used to something mainstream you will suffer. And LSP would not make your life easier.

Runtime/execution

The first time you hit infinite recursion encountered, you’ll start appreciating Rust’s compiler errors. Nix being pure and lazy means you can’t simply scatter logs for debugging without jumping through hoops.

What is next

With this pain points I don't see myself using nixos in distant future. I've heard that community works on a new language — let’s see how that turns out.

For now I will keep my flakes for fleet since it mostly works and do the job. Ofcourse I will use dev shell direnv integration everywhere I can. But I don't feel a joy using nixos and what is the point of investing your time in things do not make your happy. Now I look into gnu/guix. I like s-expr language and think that guix way of managing running services is a breethe of fresh air.

So long.