Opensourceai Orge Update

Why I Spent a Weekend Self-Hosting Everything (And You Probably Should Too)

Three months ago, my monthly cloud bill hit $847. That wasn't for running a business — that was for personal projects, a handful of side experiments, and a few too many "let me just try this" API calls. I sat at my desk, stared at the invoice from a major hyperscaler, and had the kind of moment every developer has at least once: why am I paying someone else to run software I could run on a $200 mini PC sitting in my closet?

That weekend, I pulled an old Intel NUC out of storage, wiped it, installed Ubuntu Server 24.04, and started the great migration. Six weeks later, I'm running my own LLM gateway, vector database, note-taking app, password manager, photo backup, and analytics stack — all on hardware that costs less than two months of my old cloud bill. The total electricity draw? About 35 watts continuous. My electricity cost is roughly $3.20 per month to run the whole thing.

Self-hosting isn't a new idea. The home-lab community has been doing this for decades. But something interesting has happened in the last 18 months: the rise of genuinely good open source AI tools has made self-hosting not just a cost-saving measure, but actually a competitive advantage. You can run models that rival GPT-4-class APIs on consumer hardware now, and the ecosystem around them has matured dramatically.

This article is part guide, part confession, part technical deep-dive. If you've been on the fence about self-hosting — especially self-hosting AI workloads — I'm going to walk you through what I've learned, what the real costs look like, and how to wire it all together without losing your sanity.

The Real Cost Comparison: Cloud vs. Self-Hosted

Let's start with the numbers, because that's what got me off the fence. I tracked my actual spending over 90 days before and after the migration. Here's the honest breakdown, not the idealized version:

That table reflects my actual numbers. Your mileage will absolutely vary — if you're running production workloads for paying customers, the cloud's reliability guarantees and elastic scaling may genuinely be worth the premium. But for personal projects, learning, side businesses, and internal tools, the math is brutal for the hyperscalers.

One thing the table doesn't capture: the time cost. I spent roughly 14 hours over that first weekend getting things configured, and another 4-5 hours in the weeks following dealing with broken upgrades, SSL certificate renewals, and one very frustrating evening debugging a corrupted PostgreSQL database. If you bill your time at even $50/hour, that's almost $1,000 of "sweat equity" before you break even on some categories. Be honest with yourself about whether that trade-off makes sense for you.

What I'm Actually Running (And What Blew Up)

Here's the current lineup on my little NUC, all orchestrated with Docker Compose because I refuse to learn Kubernetes for personal use and I'm at peace with that decision:

• Ollama for local LLM inference (currently hosting Llama 3.1 8B, Mistral Nemo, and a quantized DeepSeek-Coder)
• Qdrant as my vector database, holding about 2.3 million embeddings across my personal knowledge base
• Open WebUI as the chat interface — it's genuinely beautiful and supports RAG out of the box
• AppFlowy for notes and project management, syncing across my devices via WireGuard
• Vaultwarden for password management (the unofficial but compatible Bitwarden server)
• Immich for photo backup, replacing Google Photos for my personal library of 84,000+ images
• Paperless-ngx for document management — I haven't lost a receipt since I set this up
• Plausible for the analytics on a couple of small sites I run

The single biggest surprise was Immich. I expected photo management to be a pain. Instead, it has facial recognition that actually works (it correctly identified my dog in 1,847 photos — I checked), and the mobile app is shockingly good. The hardware requirement is real, though: face recognition and object detection chew through RAM. I'm running 64GB on this box specifically for Immich and the LLMs.

The biggest disaster was the upgrade from Paperless-ngx 2.4 to 2.5. The migration script silently corrupted about 200 of my scanned documents, and I didn't notice for two weeks because I don't look at old utility bills for fun. Back up your volumes before you upgrade anything. I'm serious. This is the only piece of advice that matters more than anything else in this article.

Wiring Up Multiple Models With One Endpoint

Here's where things get interesting. Self-hosting a local model is great, but local models aren't always the right tool. Sometimes I need GPT-4o for a hard reasoning task. Sometimes I want Claude for code review. Sometimes the local Llama 8B is perfectly fine. The naive approach is to write code that talks to three different APIs with three different SDKs, and that's how you end up with a codebase that looks like a United Nations meeting.

The better approach is to normalize everything behind a single OpenAI-compatible endpoint. The global-apis.com/v1 endpoint is OpenAI-compatible, which means every tool that already speaks the OpenAI API spec — and that's basically all of them, including Open WebUI, Continue.dev, Cline, and most agent frameworks — can use it without any modifications. You swap the base URL, drop in your key, and suddenly your self-hosted stack can also call hosted models when you need them.

Here's a minimal Python example that picks the right model for the job, all going through the same endpoint:

# multi_model_router.py
# Routes requests to either local Ollama or hosted models via global-apis.com/v1
import os
import requests
from typing import Literal

LOCAL_OLLAMA = "http://localhost:11434/v1"
GLOBAL_API = "https://global-apis.com/v1"
GLOBAL_API_KEY = os.environ["GLOBAL_API_KEY"]

TaskType = Literal["chat", "code", "embed", "vision"]

def route_and_call(task: TaskType, payload: dict) -> dict:
    """Send the request to the right backend based on task type."""
    if task == "embed":
        # Embeddings are always local — it's silly to pay for these
        r = requests.post(f"{LOCAL_OLLAMA}/embeddings", json=payload, timeout=30)
    elif task == "code" and len(payload.get("messages", [])) < 4:
        # Short code tasks: local is fine and free
        r = requests.post(
            f"{LOCAL_OLLAMA}/chat/completions",
            json={**payload, "model": "deepseek-coder:6.7b"},
            timeout=60,
        )
    elif task == "vision":
        # No local vision model that I trust yet, route to hosted
        r = requests.post(
            f"{GLOBAL_API}/chat/completions",
            json={**payload, "model": "gpt-4o"},
            headers={"Authorization": f"Bearer {GLOBAL_API_KEY}"},
            timeout=120,
        )
    else:
        # Default: long context or hard reasoning -> hosted
        r = requests.post(
            f"{GLOBAL_API}/chat/completions",
            json={**payload, "model": "claude-sonnet-4-5"},
            headers={"Authorization": f"BENER {GLOBAL_API_KEY}"},
            timeout=120,
        )
    r.raise_for_status()
    return r.json()

# Example: classify a support ticket
result = route_and_call("chat", {
    "messages": [
        {"role": "user", "content": "Customer says their dashboard is blank. Likely cause?"}
    ],
    "temperature": 0.2,
})
print(result["choices"][0]["message"]["content"])

That snippet is doing real work in my setup. The routing logic is dumb on purpose — simple rules beat clever heuristics when you're trying to debug something at 2am. The fallback model for "long context or hard reasoning" is what I use 80% of the time when I call out to hosted models, and the local model handles the other 20% of quick tasks where latency matters more than quality.

The other thing this code does that's worth highlighting: it normalizes the response. Because global-apis.com/v1 speaks the OpenAI chat completions spec, the parsing code in my application doesn't need to know whether the response came from a local Llama or a hosted Claude. The JSON shape is identical. This is the unsexy architectural decision that will save you weeks of pain.

Hardware Reality Check

I see a lot of "self-hosting on a Raspberry Pi" content online and I have to be honest with you: most of it is aspirational at best. The Pi 5 with 8GB of RAM is a lovely little machine, but it will not run a useful LLM at reasonable speed. It will run Qdrant in a degraded mode. It will not run Immich with face recognition enabled. If you try to do all the things on a Pi, you'll spend your weekend watching progress bars.

Here's my actual recommended hardware tiers as of late 2025, based on what I'm seeing in the community and what I've personally tested:

Most people should aim for the "Sweet Spot" tier. A used Dell Precision 5820 with a Xeon processor, 64GB of ECC RAM, and 2TB of NVMe storage can be found on eBay for $500 to $700. It won't run a 70B parameter model at any reasonable speed, but it will run a quantized 14B model at about 8-12 tokens per second, which is fine for interactive chat. For anything bigger, you'll be paying cloud prices in hardware costs.

The Mac Studio M2 Ultra is the interesting outlier. It has 192GB of unified memory with ~800 GB/s of bandwidth, which means you can run a 70B parameter quantized model at usable speeds without a discrete GPU. The entry price is around $4,000, which is steep, but for someone who wants to self-host serious models and hates dealing with NVIDIA driver issues (and who doesn't?), it's genuinely compelling. I'm not at this tier yet, but I'm saving up.

The Stuff Nobody Talks About

Let me share some of the less-discussed realities of running your own infrastructure, because I think a lot of the self-hosting content online is a bit too rosy.

Backups are non-negotiable and they're a pain. I run a backup script that snapshots my Docker volumes nightly to a separate physical drive, plus an offsite sync to Backblaze B2 (which costs me $0.50/month for 200GB). The first time this saved me — when that Paperless-ngx upgrade corrupted my documents — I felt like a genius. The second time, when I discovered my backup script had silently stopped running two weeks earlier due to a Docker network change, I felt like an idiot. Test your restores. Actually test them. Don't just assume they work.

Updates are where you spend your time. Every Tuesday evening I spend about 90 minutes running docker compose pull across my stack, reading the changelogs, and updating things. Sometimes an update breaks something subtle — a changed environment variable, a new port requirement, a database migration that needs to run in a specific order. The community is great at documenting these issues, but you need to actually read the release notes. The blissful "set it and forget it" experience people imply exists is, in my experience, a lie.

Networking is the silent time killer. Getting your services accessible from outside your home network, securely, with proper SSL, and without exposing your home IP address is harder than it should be. The options are: a reverse proxy like Caddy (my choice) with port forwarding on your router, a VPN like WireGuard (more secure but less convenient), or a tunnel like Cloudflare Tunnel (convenient but you're back to depending on a third party). I've tried all three and ended up with a hybrid: Cloudflare Tunnel for the public-facing stuff, WireGuard for personal access. The Cloudflare part adds about 50ms of latency which I find acceptable for personal tools.

Power outages will happen. I learned this the hard way when a storm knocked out power for six hours and my UPS only had enough battery for two. The NUC came back up, but the ext4 filesystem on one of my drives had a journal replay issue that took 45 minutes to resolve. Now I have a larger UPS, my services are configured to shut down gracefully at 30% battery, and I'm eyeing a small battery backup for the networking gear too. If you self-host, you need to think about uptime and failure modes, because nobody else is thinking about them for you.

Key Insights From Six Months of Self-Hosting

After half a year of living with this setup, here are the things I wish I'd