calculators

What does it cost to host an LLM?

The real question behind "AI hosting" is almost always: serverless GPU, always-on GPU, or just pay per token? This calculator compares all three for your traffic. It is an estimate — every assumption is shown below and editable. GPU rates come from our live pricing table (verified 2026-07-10).

// your workload

Model size Requests per day Avg tokens generated per request (output)

// assumptions (edit me)

Throughput on (tokens/sec) Default is a conservative single-request figure, not a measured benchmark. Batch serving can be several times higher — edit to match your setup. Serverless GPU rate ($/hr) Always-on GPU rate ($/hr) Per-token gateway price ($ per 1M output tokens, blended) Set this from the gateway you'd actually use (OpenRouter, Together, Groq, Fireworks…). Default $0.80/1M is a placeholder, not a quote.

// what this deliberately ignores

Cold starts and minimum billed idle on serverless (adds cost at low, spiky traffic).
Input-token pricing on gateways (we use one blended output rate — tune it).
Batching: a busy endpoint serves many requests concurrently, cutting effective GPU-hours. Our model bills sequential GPU-seconds, which overestimates serverless cost at high volume.
Storage, egress, and the engineering time of running your own endpoint.

GPU serverless/always-on defaults use RunPod's published rates from the live table; swap in any provider's rate. For provider selection, read RunPod vs Modal vs Replicate.

Ready to try it? These are affiliate links (labeled, same price, some include credit): RunPod ($5 credit on first $10) · Vultr ($300 trial credit) · DigitalOcean