Model API
A provider owns the model server and GPUs. Your product sends requests.
OperationsLow
- Choose it when
- You need to ship, test demand, or support changing usage.
- Watch for
- Usage spend, data policy, and provider limits.
Model hosting, explained
Where does an AI model actually run?
Your app does not need a GPU just because it uses AI. First decide whether another company serves the model, a GPU wakes up on demand, or you keep the hardware running yourself.
Your serverApp or agentCPU is usually enough
API call→
Inference serverOpen modelGPU is used here▦
Rule of thumbSeparate orchestration from inference before choosing hardware.
Common assumption“I am building an agent, so I need a GPU.”
→
Better mental modelAgent = CPU workflow
Model = API or GPU
Model = API or GPU
Choose the operating model
Same AI. Very different job.
Move right only when the extra control solves a real requirement. Complexity has a monthly cost even when the dashboard calls the GPU cheap.
Serverless GPU
Your model image runs on rented GPU workers that can scale down.
OperationsMedium
- Choose it when
- You need custom weights, burst jobs, or an isolated endpoint.
- Watch for
- Cold starts, storage, worker minimums, and quotas.
Always-on GPU
You rent a GPU instance and operate the full serving stack.
OperationsHigh
- Choose it when
- Load is steady and measured, or dedicated capacity is required.
- Watch for
- Idle bill, availability, drivers, security, and scaling.
Fast capacity check
Model size → first GPU to test
For 4-bit inference, short context, and one active sequence. This answers “might it fit?”, not “will it be fast enough?”
Estimated8Bmodel class
16 GBVRAM target
Estimated24Bmodel class 24 GBVRAM target
Estimated32Bmodel class 32 GBVRAM target
Estimated70Bmodel class 48 GBVRAM target
Sourced + estimated
Parameter counts are sourced from official model cards. Weight memory and first-test tiers are transparent estimates. Quantization metadata, runtime, KV cache, long context, batching, and concurrent users need headroom.
Open the complete VRAM guide →Before renting hardware
Five questions. In this order.
If you cannot answer traffic and control requirements yet, an API is a reversible default.
- 1Do I need custom weights?No → begin with an API.
- 2Must inference stay in my environment?No → keep outsourcing GPU operations.
- 3Is traffic bursty or steady?Bursty → serverless. Steady → price a warm instance.
- 4What actually fits in VRAM?Include context, cache, and concurrency — not only weights.
- 5What did the exact workload measure?Use peak VRAM, latency, utilization, and total monthly bill.
Deep research
Model hosting research library
All the detailed provider comparisons, pricing source trails, implementation caveats, and experiment notes remain available here without dominating the page.
ai hosting GPU cloud free credits 2026: offers, GPU access and spend limits Compare five GPU cloud credit offers, AWS project spend limits, Modal stop controls, and verified GPU-hour estimates before starting a trial. 2026-11-04 · Read the evidence → ai hosting Serverless GPU pricing 2026: real allocations and idle costs Compare live-checked serverless GPU rates, RunPod idle-state rules, and conditional Northflank H100 allocations before estimating deployment costs. 2026-11-04 · Read the evidence → ai hosting RunPod pricing 2026: higher Flex rates and billing boundaries Compare 42 RunPod Pod prices and 13 Flex tiers checked November 2026, with updated break-even costs, console/API defaults and storage billing. 2026-11-02 · Read the evidence → ai hosting Baseten vs Modal vs Replicate for one warm inference endpoint (July 2026): who makes you pay for readiness? A July 2026 HostFleet comparison of Baseten, Modal, and Replicate for one warm inference endpoint, focused on 30-day warm-capacity cost floors, scale-to-zero tradeoffs, and where each platform shape really fits. 2026-11-01 · Read the evidence → ai hosting Replicate pricing 2026: GPU rates, idle costs and teardown rules Replicate GPU rates checked November 2026, with all 17 configurations, idle costs, cancellation limits, deletion prerequisites and prepaid-credit risks. 2026-11-01 · Read the evidence → ai hosting A100 rental prices 2026: rates, prepaid billing and cleanup Compare 21 dated A100 rate inputs, updated Verda prices, prepaid refund rules and storage cleanup. Separate compute estimates from maximum prepaid exposure. 2026-10-25 · Read the evidence → ai hosting GPU cloud cost calculator: idle time, VM and snapshot costs Estimate dated GPU rates, complete VM costs and idle time, with Thunder snapshot storage math and the limits of a sub-dollar GPU-test budget. 2026-10-24 · Read the evidence → ai hosting H100 rental price per hour: 19 cloud rates and billing traps Compare 19 dated H100 cloud rates, Novita’s $3.39 CPU/RAM bundle and billable startup, plus Verda’s updated price and 30-day cost estimates. 2026-10-21 · Read the evidence → ai hosting Modal pricing 2026: GPU rates and how to reconcile the bill Modal pricing checked October 19, 2026: 11 GPU rates, full resource costs, idle windows, and a billing workflow for closed hours, App attribution, and credits. 2026-10-19 · Read the evidence →
Show 25 more research notes
Baseten pricing 2026: GPU rates and deployment bill checks 2026-10-13 What GPU do you need to run Llama 70B? VRAM, context, and KV-cache guide 2026-10-06 Replicate for AI inference APIs and jobs (July 2026): fast to ship, expensive once you buy warm control 2026-09-03 Fal vs Baseten vs Modal for one warm H100 deployment (July 2026): cheapest managed floor or best serving surface? 2026-07-21 Fal for AI inference APIs and jobs (July 2026): serious serverless deployment controls, premium warm-GPU economics 2026-07-20 Modal vs RunPod for one warm inference endpoint (July 2026): cleaner Python ergonomics or cheaper GPU control? 2026-07-19 Baseten vs RunPod for one warm inference endpoint (July 2026): managed inference polish or cheaper GPU control? 2026-07-18 Baseten for AI inference APIs and jobs (July 2026): polished dedicated inference, pricey once replicas stay warm 2026-07-16 DigitalOcean vs Hetzner Cloud for AI side projects (July 2026): clean cloud vs cheap cloud 2026-07-11 RunPod for AI inference APIs and jobs (July 2026): flexible GPU hosting with sharp billing edges 2026-07-11 Best hosting for AI agents on a budget (June 2026): choose by workload, not by AI branding 2026-07-10 DigitalOcean Droplets for AI side projects (June 2026): what fits, what breaks, and when to pay for dedicated CPU 2026-07-10 DigitalOcean vs Hetzner vs Hostinger for AI side projects (June 2026): the honest always-on floor 2026-07-10 Hetzner Cloud for AI side projects (July 2026): cheapest serious self-hosted CPU, with the catches that still matter 2026-07-10 Modal for AI inference APIs and jobs (July 2026): brilliant for bursty GPU work, awkward as a cheap warm endpoint 2026-07-08 Hetzner vs Contabo vs Hostinger VPS for AI workloads (June 2026): which budget box actually fits an agent stack 2026-06-29 S3 alternatives for AI assets: R2 vs B2 vs Wasabi vs Tigris (June 2026) 2026-06-29 What it costs to run an AI side project on a VPS for 30 days (June 24, 2026): honest budget ranges 2026-06-24 RunPod vs Modal vs Replicate for shipping a small inference API (June 2026): who should own the endpoint, queue, and warm pool? 2026-06-22 Cloudflare Workers AI vs self-hosted GPU: when each wins (June 2026) 2026-06-18 Best VPS setup for LangGraph or CrewAI (June 2026): what fits on 4 GB, 8 GB, and beyond 2026-06-13 Hostinger VPS for AI side projects: what fits, what breaks, and when to upgrade (June 2026) 2026-06-03 Best hosts for long-running agent workers (June 2026): where always-on costs and queue limits bite 2026-06-02 Vector database hosting for small AI apps (May 2026): when pgvector wins, when Qdrant wins, and when managed vector DB is worth it 2026-05-25 OpenRouter vs Together vs Groq vs Fireworks vs Cerebras: the per-token model gateways compared (April 2026) 2026-04-27