Model API
A provider owns the model server and GPUs. Your product sends requests.
OperationsLow
- Choose it when
- You need to ship, test demand, or support changing usage.
- Watch for
- Usage spend, data policy, and provider limits.
Model hosting, explained
Where does an AI model actually run?
Your app does not need a GPU just because it uses AI. First decide whether another company serves the model, a GPU wakes up on demand, or you keep the hardware running yourself.
Your serverApp or agentCPU is usually enough
API call→
Inference serverOpen modelGPU is used here▦
Rule of thumbSeparate orchestration from inference before choosing hardware.
Common assumption“I am building an agent, so I need a GPU.”
→
Better mental modelAgent = CPU workflow
Model = API or GPU
Model = API or GPU
Choose the operating model
Same AI. Very different job.
Move right only when the extra control solves a real requirement. Complexity has a monthly cost even when the dashboard calls the GPU cheap.
Serverless GPU
Your model image runs on rented GPU workers that can scale down.
OperationsMedium
- Choose it when
- You need custom weights, burst jobs, or an isolated endpoint.
- Watch for
- Cold starts, storage, worker minimums, and quotas.
Always-on GPU
You rent a GPU instance and operate the full serving stack.
OperationsHigh
- Choose it when
- Load is steady and measured, or dedicated capacity is required.
- Watch for
- Idle bill, availability, drivers, security, and scaling.
Fast capacity check
Model size → first GPU to test
For 4-bit inference, short context, and one active sequence. This answers “might it fit?”, not “will it be fast enough?”
Estimated8Bmodel class
16 GBVRAM target
Estimated24Bmodel class 24 GBVRAM target
Estimated32Bmodel class 32 GBVRAM target
Estimated70Bmodel class 48 GBVRAM target
Sourced + estimated
Parameter counts are sourced from official model cards. Weight memory and first-test tiers are transparent estimates. Quantization metadata, runtime, KV cache, long context, batching, and concurrent users need headroom.
Open the complete VRAM guide →Before renting hardware
Five questions. In this order.
If you cannot answer traffic and control requirements yet, an API is a reversible default.
- 1Do I need custom weights?No → begin with an API.
- 2Must inference stay in my environment?No → keep outsourcing GPU operations.
- 3Is traffic bursty or steady?Bursty → serverless. Steady → price a warm instance.
- 4What actually fits in VRAM?Include context, cache, and concurrency — not only weights.
- 5What did the exact workload measure?Use peak VRAM, latency, utilization, and total monthly bill.
Deep research
Model hosting research library
All the detailed provider comparisons, pricing source trails, implementation caveats, and experiment notes remain available here without dominating the page.
ai hosting RunPod pricing 2026: 42 Pod prices, 13 Serverless tiers, and break-even math RunPod pricing checked September 2026: all 42 Pod prices, Secure-vs-Community premiums, 13 Flex tiers, break-even math, storage, and billing caveats. 2026-09-17 · Read the evidence → ai hosting GPU cloud cost calculator 2026: 8 A100 rates and snapshot math Eight current A100 cloud rates with idle-cost math, exact billing-stop actions, and Thunder Compute snapshot, deletion, storage, and restore boundaries. 2026-09-16 · Read the evidence → ai hosting What GPU do you need to run Llama 70B? VRAM, context, and KV-cache guide What GPU to run Llama 70B: VRAM and KV-cache math, current cloud rates, and the Massed L40S billing and termination boundary. 2026-09-16 · Read the evidence → ai hosting A100 rental price per hour in 2026: 22 public rates checked Twenty-two public A100 rates, 720-hour estimates, and the pause, storage, resume, filesystem, and Reserved IP costs that change Jarvis Labs' low headline. 2026-09-14 · Read the evidence → ai hosting H100 rental price per hour in 2026: 19 public rates checked Nineteen public H100 rates checked September 2026, with 720-hour estimates and the Lambda guest-poweroff, termination, and filesystem traps the rate card omits. 2026-09-14 · Read the evidence → ai hosting Serverless GPU pricing in 2026: H100 rates, scale-to-zero, and idle tails Eight current H100 serverless and managed rates compared with scale-to-zero and idle tails, plus why Salad Lowest consumer-GPU pricing is not equivalent. 2026-09-13 · Read the evidence → ai hosting Replicate for AI inference APIs and jobs (July 2026): fast to ship, expensive once you buy warm control A July 2026 HostFleet review of Replicate for AI inference APIs and jobs, focused on the real cost shape of official models versus public models versus deployments, the warm-capacity tradeoff, and the operational edges buyers miss. 2026-09-03 · Read the evidence → ai hosting Replicate pricing 2026: GPU rates, setup billing, and deployment floors Replicate GPU pricing checked September 2026, with 17 hardware rates, setup-failure costs, warm deployment floors, and contract boundaries. 2026-09-03 · Read the evidence → ai hosting Baseten pricing 2026: rates, replica tails, and scale-to-zero costs Baseten GPU pricing checked September 2026, with replica lifecycle billing, staged scale-down costs, request policies, and warm-floor estimates. 2026-09-02 · Read the evidence →
Show 25 more research notes
GPU cloud free credits 2026: which trials stop before charging? 2026-08-30 Modal pricing 2026: GPU rates, region fees, and Sandbox costs 2026-08-29 Fal vs Baseten vs Modal for one warm H100 deployment (July 2026): cheapest managed floor or best serving surface? 2026-07-21 Fal for AI inference APIs and jobs (July 2026): serious serverless deployment controls, premium warm-GPU economics 2026-07-20 Modal vs RunPod for one warm inference endpoint (July 2026): cleaner Python ergonomics or cheaper GPU control? 2026-07-19 Baseten vs RunPod for one warm inference endpoint (July 2026): managed inference polish or cheaper GPU control? 2026-07-18 Baseten vs Modal vs Replicate for one warm inference endpoint (July 2026): who makes you pay for readiness? 2026-07-17 Baseten for AI inference APIs and jobs (July 2026): polished dedicated inference, pricey once replicas stay warm 2026-07-16 DigitalOcean vs Hetzner Cloud for AI side projects (July 2026): clean cloud vs cheap cloud 2026-07-11 RunPod for AI inference APIs and jobs (July 2026): flexible GPU hosting with sharp billing edges 2026-07-11 Best hosting for AI agents on a budget (June 2026): choose by workload, not by AI branding 2026-07-10 DigitalOcean Droplets for AI side projects (June 2026): what fits, what breaks, and when to pay for dedicated CPU 2026-07-10 DigitalOcean vs Hetzner vs Hostinger for AI side projects (June 2026): the honest always-on floor 2026-07-10 Hetzner Cloud for AI side projects (July 2026): cheapest serious self-hosted CPU, with the catches that still matter 2026-07-10 Modal for AI inference APIs and jobs (July 2026): brilliant for bursty GPU work, awkward as a cheap warm endpoint 2026-07-08 Hetzner vs Contabo vs Hostinger VPS for AI workloads (June 2026): which budget box actually fits an agent stack 2026-06-29 S3 alternatives for AI assets: R2 vs B2 vs Wasabi vs Tigris (June 2026) 2026-06-29 What it costs to run an AI side project on a VPS for 30 days (June 24, 2026): honest budget ranges 2026-06-24 RunPod vs Modal vs Replicate for shipping a small inference API (June 2026): who should own the endpoint, queue, and warm pool? 2026-06-22 Cloudflare Workers AI vs self-hosted GPU: when each wins (June 2026) 2026-06-18 Best VPS setup for LangGraph or CrewAI (June 2026): what fits on 4 GB, 8 GB, and beyond 2026-06-13 Hostinger VPS for AI side projects: what fits, what breaks, and when to upgrade (June 2026) 2026-06-03 Best hosts for long-running agent workers (June 2026): where always-on costs and queue limits bite 2026-06-02 Vector database hosting for small AI apps (May 2026): when pgvector wins, when Qdrant wins, and when managed vector DB is worth it 2026-05-25 OpenRouter vs Together vs Groq vs Fireworks vs Cerebras: the per-token model gateways compared (April 2026) 2026-04-27