“Can we have something better than Google Photos to manage our travel pics?” - my partner innocently asked, circa 2019.
At the time I took a look around for options but I couldn’t find any apps that would feel like a suitable replacement for Google Photos. But I did spend a lot of time researching the world of self-hosting, and that’s how I bumped into Commafeed - an RSS reader that feels very familiar to old school Google Reader users like myself. Commafeed was the first app I started self hosting, and I used it to replace Inoreader, which was my feed reader at the time. This was the beginning of it all.
I started by just using my desktop computer - as a lifelong gamer, I always have a somewhat updated PC at all times. But once I added Home Assistant it was obvious that I needed always-on infra. That’s when I bought my first Raspberry Pi, the perfect companion to Home Assistant.
Over the years I kept steadily adding more self-hosted apps. Added Mealie for my recipes (I’m also an avid cook), a local instance of Searxng, Navidrome, Jellyfin, Cloudbeaver, and so on. My tiny RPi quickly evolved into a couple of Mini PCs, and then later a NAS where I started keeping a local copy of all our pictures, while still using Google Photos as the primary.
When I moved my photos to Immich somewhere in 2024, it was clear I needed a better home server. Immich uses CLIP models for smart search, and processing every photo in our library took many hours on the mini PC I was running at the time. Given the “one more self-hosted app” trajectory, I decided to go big. My current box is:
- CPU: AMD EPYC 7513, 32c / 64t
- RAM: 512 GB ECC DDR4
- GPU: NVIDIA RTX 3090 (24 GB)
- Storage: a lot of flash in ZFS pools
It’s clearly more compute than I need to run Immich, but that’s the point - now I have a lot of headroom to play with! All that DDR4 RAM in particular gives me a lot of flexibility to explore hybrid GPU+CPU setups to serve large models.
Once I got all this power (and data!) at my disposal, the data engineer in me went straight for the obvious target: squeezing every last drop of performance out of local model serving. The local inference space has been evolving at a crazy pace over the last couple of years (llama.cpp, ollama, etc.) and I’ve spent a ridiculous amount of time benchmarking and tuning. My next post is about my current model serving setup, and the several iterations I’ve gone through to get there.
This blog is the intentional version of my accidental homelab. The next posts will go deeper on local model serving and the trade-offs I’m making along the way. I’m a firm believer in private, local apps and AI, running on your own devices - and I want to share what I explore in my own time, outside my day job at Meta. More about me here.