How to choose AI infrastructure: a practical guide to model API hosts

Step 0: Vendor is not the Token Provider

“Model provider” in everyday speech hides two jobs. On AI Hippo, Vendor (model developer) is the brand that trains and ships the weights—Anthropic, OpenAI, Meta, DeepSeek. Token Provider (API host) is who serves the inference HTTP endpoint—OpenRouter, Azure OpenAI, AWS Bedrock, Groq, or the vendor’s own direct API.

You can pick Claude (vendor=anthropic) and still choose among anthropic-direct, aws-bedrock, and openrouter as hosts. Mixing the two questions—“which model?” vs “which host?”—is the fastest way to a bad buy.

Glossary anchors: Vendor vs Service provider / Token Provider. Model catalog lives under /model/vendor/; host index under /token/provider/.

Map the host landscape (what AI Hippo tracks)

This snapshot indexes 17 Token Providers in four kinds:

Direct (7): anthropic-direct, openai-direct, deepseek-direct, moonshot-direct, minimax-direct, z-ai-direct, ai21-direct—vendor-page crawls into verified list prices.

Cloud (5): aws-bedrock, azure-openai, google-vertex, together, fireworks—enterprise or serverless clouds with their own SKUs and contracts.

Aggregator (4): openrouter, groq, siliconflow, deepinfra—one API key, many upstream models.

Other (1): self-hosted—for teams that run weights themselves (no shared list-price row).

Coverage is uneven on purpose: OpenRouter holds 334 primary listings (is_primary_listing=1) used for cross-model ranking; non-OpenRouter verified rows cover 151 distinct models. Rankings default to OpenRouter for comparability; the Token matrix is where you audit host-vs-host USD/1M for the same core_model_id.

Six decision dimensions (not just unit price)

Score each candidate host on six axes before you lock a contract:

1) List-price USD/1M prompt & completion (and batch/cached tiers when present). Prefer model_bound_verified rows; treat model_bound_snapshot as provisional until a vendor crawl stabilizes.

2) Price confidence & divergence: AI Hippo suppresses non-official rows that diverge ≥20% from a verified OpenRouter quote so the matrix does not silently adopt outliers.

3) Model coverage: does this host actually list your SKU? Official models often show price parity across direct/cloud/OR (e.g. Claude Sonnet 4 at $3/$15 on anthropic-direct, aws-bedrock, and openrouter). Open-weight and mid-tier SKUs can swing hard by host.

4) Latency & throughput class: Groq-style hosts often trade higher list price for speed; cheapest DeepInfra rows may win $/token but lose p95 latency. Use latency columns when present; otherwise load-test yourself.

5) Region, data residency, and compliance: Azure OpenAI, Bedrock, and Vertex exist for enterprise IAM, private networking, and residency clauses that a pure aggregator invoice cannot replace.

6) Ops surface area: one OpenRouter key vs N vendor keys, failover routing, invoice consolidation, and whether you need domestic_pay or badge_cheapest signals in the matrix.

If two hosts tie on verified price, decide on axes 4–6—not on marketing slides.

Path A — Official direct: when the vendor API wins

Choose anthropic-direct, openai-direct, deepseek-direct (and peers) when you need: newest SKUs on day one, feature flags (batch, cached input, org spend caps) that aggregators lag, or a single commercial relationship for support and DPA.

Hard numbers from this snapshot: DeepSeek V4 Pro is $0.435/$0.870 per 1M on deepseek-direct and the OpenRouter primary listing—while DeepInfra lists $1.30/$2.60, SiliconFlow ~$1.50/$3.14, and Fireworks/Together $1.74/$3.48. That is roughly 3–4× more on some aggregators for the same model id. If V4 Pro is your production workhorse, direct (or OR when it mirrors direct) is the cost-rational default.

Conversely, Claude Sonnet 4 shows $3/$15 across anthropic-direct, aws-bedrock, and openrouter—price is not the differentiator; contract, region, and IAM are.

Trade-offs: N vendor keys, fragmented invoices, and you still need a second host for models outside that vendor.

Path B — Aggregators: one key, many models, watch the spread

OpenRouter, Groq, SiliconFlow, and DeepInfra win when you prototype across vendors, need failover routing, or want one invoice while the model shortlist is still fluid. OpenRouter’s 334 primary listings make it the default comparability lane on AI Hippo rankings.

But “aggregator” is not a synonym for “cheapest.” Llama 3.3 70B Instruct in this snapshot: DeepInfra $0.10/$0.32 (verified), OpenRouter primary $0.13/$0.40, Groq $0.59/$0.79 (snapshot), Fireworks $0.90/$0.90, Together $1.04/$1.04. Input list price spans roughly 10× from cheapest to dearest host—for the same open-weight id.

DeepSeek V4 Flash is tighter: DeepInfra $0.09/$0.18, OpenRouter ~$0.094/$0.188, SiliconFlow $0.13/$0.28, deepseek-direct and Fireworks $0.14/$0.28. Here an aggregator can undercut direct.

Rule: for each production SKU, open the Token matrix and sort by verified USD/1M—do not assume the host that won last quarter still wins.

Path C — Cloud platforms: compliance and accounts beat list price

Pick aws-bedrock, azure-openai, or google-vertex when procurement already standardized on one cloud, you need VPC/private-link networking, regional residency, committed-use discounts, or unified IAM with the rest of the stack. Together and Fireworks sit in the “cloud/serverless inference” band: strong for open-weight fleets with cloud-style ops, without being a full hyperscaler account.

Price reality check: GPT-4o sits at $2.50/$10.00 on openai-direct, azure-openai, and openrouter in this snapshot—again, host choice is about where the tokens run and who signs the DPA, not a secret discount in the public list.

gpt-oss-120b shows a different pattern: OpenRouter primary $0.037/$0.170, SiliconFlow $0.05/$0.45, while aws-bedrock / Fireworks / Groq / Together cluster around $0.15/$0.60. If you must stay inside an AWS estate, paying the Bedrock row can still beat standing up a second vendor relationship—even when OR is cheaper on paper.

Never treat cloud as “always more expensive” or “always safer” without reading the verified row and your contract addenda.

Cost in practice: same model, five hosts, two workloads

Worked example on Llama 3.3 70B Instruct (list prices above). Cost = input_tokens×in_price + output_tokens×out_price.

Scenario 1 — agentic coding turn (100K in + 20K out): DeepInfra ≈ $0.0164; OpenRouter ≈ $0.021; Groq ≈ $0.075; Fireworks ≈ $0.108; Together ≈ $0.125. Cheapest verified host is ~7.6× cheaper than Together on this turn.

Scenario 2 — RAG batch pass (500K in + 5K out): DeepInfra ≈ $0.0516; OpenRouter ≈ $0.067; Groq ≈ $0.299; Fireworks ≈ $0.455; Together ≈ $0.525.

Second example — DeepSeek V4 Pro production chat (100K in + 20K out): deepseek-direct / OpenRouter ≈ $0.061; DeepInfra ≈ $0.182; Fireworks/Together ≈ $0.244. Paying an aggregator “for convenience” without checking the matrix can 3–4× a single turn.

These are spec and list-price comparisons from the site snapshot—not capability benchmarks and not negotiated enterprise rates.

Decision checklist: pick a path, then verify on AI Hippo

1) Freeze the model shortlist (capability / context / tools) using rankings and /compare/—that is the Vendor question.

2) For each shortlisted core_model_id, open /token/ and read every host row: verified vs snapshot, Δ% vs OpenRouter, badges for cheapest/fastest/domestic.

3) If you need one key and fast experimentation → start Path B (often OpenRouter), then promote winners to direct or cloud.

4) If one SKU dominates spend and direct ≤ aggregator verified price → Path A.

5) If legal/security requires a named cloud account → Path C, then check whether the cloud row is price-parity or a premium you accept.

6) Re-check after each pricing sync (pipeline ~6h). Read the companion insight on how to read the pricing matrix before filing price-correction tickets.

7) Self-host only when utilization, ops talent, and license terms beat the cheapest verified API row for your volume—link /compare/llama-vs-api-hosted/ for that fork.

Do not skip step 2. The matrix exists so “AI infrastructure” choices are auditable, not anecdotal.

Sources

• Token provider hub
• Token hub matrix
• Multi-source cost guide
• Glossary