Should I self-host an LLM?

Self-host when: you process 100,000+ queries per month (cost break-even), you handle sensitive data that cannot leave your infrastructure (healthcare, finance, legal), you need guaranteed uptime independent of API provider outages, or you want to fine-tune and deploy custom models without per-query costs. Don't self-host when: your volume is under 50,000 queries/month (API is cheaper), you lack ML engineering talent, you need the latest models immediately (APIs update first), or you need multi-model flexibility (switching between GPT-4o and Claude requires separate deployments). Infrastructure cost: a single NVIDIA A100 GPU server costs $2-3/hour on AWS/GCP, or $1,500-$2,000/month. This runs Llama 3 70B at ~30 tokens/second, handling ~50,000 queries per day. Two GPUs for redundancy: $3,000-$4,000/month. Frameworks: vLLM (fastest inference server), Ollama (easiest local setup), TGI (HuggingFace, good for production), and Triton (NVIDIA, best GPU utilization). Operational requirements: GPU monitoring, model updates, load balancing, fallback to API when self-hosted is down, and evaluation pipeline to compare self-hosted quality against API baselines.