Air-Gapped LLM: How to Run Language Models With Zero Internet (2026)

An air-gapped LLM is a language model deployment with no path to the public internet at all — inference, retrieval, embeddings and audit logs all run inside an isolated network, and model weights arrive as verified offline bundles rather than downloads. It is the strictest form of private AI, and in 2026 it is a practical engineering project rather than a defense-sector curiosity: the hardware math has collapsed (a 70-billion-parameter model that needed two data-center GPUs in 2024 now runs on a single 48 GB card with 4-bit quantization), and the deployment pattern is documented well enough to plan against. This guide covers the definitions that matter, the hardware math in plain English, what it costs, and the honest cases where an air gap is the wrong answer.

Air-Gapped vs On-Premise vs Egress-Blocked: The Distinction That Matters

Three postures get conflated, and auditors do not conflate them:

• Connected on-premise: the model runs on your hardware, but the deployment may still reach external services for updates, identity or telemetry. Data stays in-house; the network is not isolated.
• Egress-blocked: a firewall rule denies outbound traffic. Better than nothing, but if any component assumes internet access, the deployment breaks the first time it needs an update — and teams quietly reopen a rule.
• Air-gapped: engineered so nothing ever needs to call out. The practical test, used by practitioners: can this deployment operate indefinitely — including bringing on new models and applying security patches — with zero outbound flows? If updates require the enclave to reach the internet, even briefly through a proxy, it is not air-gapped.

That test is what separates a security posture from a wish. Regulated buyers get burned on it every year: the deployment that keeps an "outbound-only" path for license checks or model-registry pulls fails exactly the policy the enclave was built to satisfy.

What Actually Breaks Without Internet

Serving the model is the easy part — weights are static files and the inference engine is a local process. Everything difficult is upstream, and four failure patterns account for most real-world incidents:

• Runtime pulls. A deployment that downloads model weights on first run or resolves Python packages at build time works perfectly in staging and fails inside the perimeter. Everything must be staged before isolation.
• Phone-home components. Telemetry, license checks and "is the registry reachable" health probes fail — or worse, silently retry — inside a gap. Every component must be verified to start with no outbound request.
• The RAG embedding leak. The most common second-week egress-audit finding: a retrieval pipeline quietly calling a hosted embedding endpoint or hosted tokenizer. The corpus stays in-perimeter but every indexed document crosses the gap in the wrong direction. Embedding and reranker models must run locally on the same inference plane.
• Update drift. Without a controlled offline update pipeline, enclaves end up running year-old models. Treat model logistics as seriously as the initial deployment, or capability freezes on day one.

The Hardware Math in Plain English

Two numbers drive everything: parameter count and quantization. At full 16-bit precision a model needs roughly 2 GB of VRAM per billion parameters, plus 20–40% headroom for the KV cache under concurrent use:

• 7–8B models (Llama-class, Qwen-class): about 16 GB at FP16 — fits a single 24 GB consumer card; quantized to 4-bit, around 4–5 GB, comfortable on a 16 GB card.
• 70B-class models (Llama 3.3 70B, Qwen 72B): about 140 GB at FP16 — historically two 80 GB data-center GPUs. With 4-bit AWQ/GPTQ quantization the footprint drops to roughly 38–40 GB, which fits a single 48 GB card with minimal quality loss on most production tasks.
• Small is genuinely viable now: a 1B model at 4-bit is around 500 MB and runs on commodity CPUs with 4–8 GB of RAM — hardware already sitting in controllers and industrial machines. Distillation-focused vendors build exactly this pattern for industrial control networks where a GPU will never be approved.
• Concurrency changes the answer. Size for concurrent users at your target context length, not for one user's latency: the same 70B model at high concurrency with long documents can saturate several GPUs. Add the KV-cache headroom before you buy.

For calibration, cloud rental prices make the buy-vs-rent math concrete: a Llama-3.3-70B-class model quantized to INT4 needs about 38 GB and rents on a 48 GB L40S at roughly $0.96/hour; the same model at FP16 needs about 154 GB across two A100 80GB cards at roughly $2.86/hour. Run steady eight-hour weekdays and the rent column is $2,000–$6,000/month — permanently — which is why steady, high-volume workloads justify owning hardware and bursty ones justify not doing an air gap at all.

The Two Architectures That Actually Work

There are exactly two production-grade paths to zero-egress inference, and the choice is control versus operational overhead:

• Self-hosted open-weight models — Llama, Qwen, Mistral and similar families served with vLLM (GPU throughput) or llama.cpp/Ollama (CPU and edge), behind an OpenAI-compatible API on an internal address. Full model ownership, no per-token cost, and you size and patch everything. Weights in safetensors or GGUF land on local media; engines run fully offline once images and weights are local.
• Managed model API via private endpoint — AWS Bedrock through a VPC interface endpoint is the reference pattern: foundation-model calls stay inside the cloud's network with no internet egress and no servers to manage. You accept the vendor's model catalog and per-token pricing.

Regulated buyers increasingly run both: a private-endpoint managed API for primary workloads where a foundation model is acceptable, and self-hosted open-weight models where data ownership or a fine-tuned domain model is required. The honest default is the hybrid — a strict self-built air gap is a project, not a purchase.

Getting Models and Updates In (And Keeping Them Fresh)

The offline supply chain is the largest and most underestimated part of an air-gapped platform. Every artifact that crosses into the isolated zone — model weights, tokenizers, container images, GPU drivers, CUDA libraries, Python packages, OS packages, security signatures — needs the same controlled sequence:

1. Acquire from an approved source in a connected staging zone; record version and license.
2. Verify integrity: checksums and signatures against a manifest — inside the gap, a recorded checksum is the only provenance evidence you will have.
3. Scan and stage: malware and vulnerability scanning, then transfer across the gap on read-only removable media or an approved cross-domain solution.
4. Verify again inside the enclave before extraction — signature, per-file hashes, and a dry-run install against a staging host.
5. Install alongside, never over: each bundle installs to its own versioned directory; the previous version stays in place for at least one change cycle so rollback is a one-step symlink flip, not a restore-from-backup during a production window.

Agree the update cadence in advance — how often a new model generation is evaluated, bundled, and transferred. Defense-sector practice is now routine signed-bundle logistics; the failure mode everywhere else is the enclave that silently freezes on its first model for years because nobody owns the refresh.

What an Air-Gapped LLM Costs in 2026

The LLM deployment itself prices like fixed infrastructure — no meters, no per-token exposure:

• Entry (single team, 8B-class): a $2,000–$4,000 workstation with a 24 GB GPU runs 8B models comfortably and small quantized 30B-class models acceptably — consistent with the general on-premise pricing we published in our on-premise AI server cost guide.
• Mid (department, 70B-class quantized): roughly $10,000–$20,000 for a dual-GPU server around a 48 GB card — the 4-bit quantization lever is the difference between a two-GPU node and one.
• High-concurrency / frontier-grade: $40,000 and up for multi-GPU inference with headroom, mirroring the tiers in that same guide.

The real cost driver is operations, not hardware. Air-gapping shifts spend from infrastructure into ongoing work: manual patching, artifact validation, internal registries and mirrors, local observability (Prometheus/Grafana-class monitoring must run inside the zone), and specialist staffing. Security guidance from enterprise deployments is blunt about it: evaluate air-gapping as a total cost of ownership increase over a connected on-premise deployment, not as a server price.

When You Actually Need One (And When You Do Not)

An air gap is justified by a specific requirement, not chosen as a default security posture:

• Genuinely need it: classified or regulated networks with policy prohibiting any external AI path; OT/industrial control networks; data jurisdictions where US-controlled cloud legal exposure is unacceptable; disconnected field, vessel or remote-site operations; contractual zero-egress commitments.
• Probably do not: if the actual regulatory need is met by connected on-premise AI with strong access controls, a well-operated connected deployment is often more secure in practice than an under-resourced strict air gap that cannot be patched quickly. If your team cannot sustain an offline supply chain and local security operations, the gap becomes a liability — unpatched builds and frozen models.

The decision test from enterprise deployment guidance: start from the regulatory requirement, not from the architecture. If privacy questionnaires from your customers are the trigger, ArcBeta Solutions can assess which posture actually satisfies them — and which one you are over-buying. For the threat model behind the requirement, see our guide on whether ChatGPT is safe for company data.

Frequently Asked Questions

What is an air-gapped LLM?

A language model deployment with no routable path to the public internet in normal operation: inference, retrieval, embeddings and audit logs all run inside an isolated network, and all software and model updates arrive as verified offline bundles. The defining test is whether it can operate and be updated indefinitely with zero outbound flows.

Can ChatGPT or Claude run air-gapped?

No — commercial AI APIs assume network connectivity and are unreachable inside a gap. Air-gapped deployments require self-hosted open-weight models (or a managed API reached over a private cloud endpoint, which is isolated networking rather than a true air gap). The model, its weights, runtime and every dependency must be inside the perimeter.

How much VRAM does an air-gapped LLM need?

Roughly 2 GB per billion parameters at FP16, plus 20–40% KV-cache headroom: 7–8B models fit a single 24 GB GPU, a 70B model needs about 140 GB at full precision or roughly 38–40 GB with 4-bit quantization on one 48 GB card. Size for concurrent users at your target context length, not single-user latency.

How do model updates work without internet?

Every update is a signed offline bundle — weights, runtime and dependencies with a manifest and checksums — transferred on approved media, verified inside the enclave, and installed alongside the previous version for one-step rollback. Without an owned update cadence, deployments silently freeze on their first model for years.

What does an air-gapped LLM deployment cost?

Hardware runs from about $2,000–$4,000 for an 8B-class workstation through $10,000–$20,000 for a 70B quantized departmental server to $40,000+ for frontier-grade concurrency. The larger cost is operational: offline supply-chain management, manual patching, local monitoring and specialist staffing — evaluate it as a total-cost increase over connected on-premise AI, not a server price.