On 2026-08-06, Hugging Face announced Baseten joining Inference Providers. Once it's live, append :baseten to a model ID and use the OpenAI-compatible endpoint to pin chat completion requests to Baseten. On the partner matrix, Baseten currently supports two kinds of chat completion: LLM conversation and VLM visual conversation (source: Inference Providers docs). The blog post lists Kimi K3, DeepSeek V4 Flash, and GLM-5.2 among the initial callable models.
Decide where the bill lands before you start. If you want one Hugging Face Token for both calls and billing, use HF routed. If you already have a Baseten contract, credits, or a key-management process, save a Baseten API Key in your Hugging Face account settings and route requests directly to Baseten. Whichever you pick, the acceptance criteria are the same four: the code explicitly pins :baseten, the test environment returns the expected structure, the bill lands in the account you chose, and failures can switch to a pre-verified fallback provider or model.
This guide covers using Baseten through Hugging Face Inference Providers, not deploying on the Baseten standalone platform (Model APIs / Truss). The two entry points differ in auth and billing; don't mix them.
Decide Between HF Routed and a Baseten Key
Both modes share the same Hugging Face Inference Providers entry point; they differ in auth, request path, and billing. Hugging Face's announcement and billing docs set the rules as of 2026-08-06.
| Option | Auth & Request | Billing | When to Use |
|---|---|---|---|
| HF routed | Uses a Hugging Face Token; HF Router forwards to Baseten | Billed to your Hugging Face account; monthly credits apply | One Token across providers, or a low-cost check using monthly credits |
| Baseten Key | Saves a Baseten API Key in Hugging Face settings; requests call Baseten directly | Billed to your Baseten account; HF monthly credits don't apply | Existing Baseten credits, contracts, permissions, or cost governance |
Per the billing docs, free users get $0.10 of monthly inference credits, PRO users $2.00, and Team or Enterprise $2.00 per seat; routed requests are billed at standard provider rates with no markup from Hugging Face. These are dynamic commercial terms—recheck your account's billing page before launch and don't hard-code the amounts into a budget model.
To use org credits and centralize billing on Team or Enterprise, routed requests must explicitly carry an X-HF-Bill-To: <org-name> HTTP header; otherwise the cost lands on the personal token and never reaches the org (source: billing docs).
If your team doesn't have a Baseten account yet, start with HF routed to spend monthly credits on a technical validation—this skips opening a separate account and configuring a key. If your org requires the key ownership, billing entity, or purchase contract to sit with Baseten, choose the key from the start to avoid migrating the auth path after validation.
Run a Minimal Call with an HF Token
HF routed mode only needs a Hugging Face Token. Create a fine-grained token with the Make calls to Inference Providers permission (token settings), put it in an environment variable, and never write it into scripts, notebooks, or Git repos:
pip install openai
export HF_TOKEN="<your-hugging-face-token>"
The official Python example points base_url at the HF Router's OpenAI-compatible entry and calls chat.completions:
import os
from openai import OpenAI
client = OpenAI(
base_url="https://router.huggingface.co/v1",
api_key=os.environ["HF_TOKEN"],
)
completion = client.chat.completions.create(
model="deepseek-ai/DeepSeek-V4-Flash-0731:baseten",
messages=[{"role": "user", "content": "Write a Python function that returns the nth Fibonacci number using memoization."}],
)
print(completion.choices[0].message.content)
JavaScript uses the same Router, token, and model string. Install the OpenAI SDK first:
npm install openai
import { OpenAI } from "openai";
const client = new OpenAI({
baseURL: "https://router.huggingface.co/v1",
apiKey: process.env.HF_TOKEN,
});
const completion = await client.chat.completions.create({
model: "deepseek-ai/DeepSeek-V4-Flash-0731:baseten",
messages: [{ role: "user", content: "Write a Python function that returns the nth Fibonacci number using memoization." }],
});
console.log(completion.choices[0].message.content);
Both snippets are based on Hugging Face's official examples from 2026-08-06. Note that the OpenAI-compatible endpoint currently only supports chat completion; image, speech, and embedding tasks need the Hugging Face native inference clients (Inference Providers docs). The model version, available regions, and provider capacity in the examples can change; inject the full model ID through configuration in production instead of scattering it across business modules.
Pin Routing with :baseten
deepseek-ai/DeepSeek-V4-Flash-0731:baseten splits into two parts: the part before the colon is the Hugging Face model ID, and the part after is the target provider. Keeping :baseten pins routing to Baseten.
Without a suffix, the OpenAI-compatible endpoint defaults to the fastest available provider for the model (equivalent to :fastest); :cheapest picks the most cost-efficient one, and :preferred follows your preference order in Inference Provider settings (source: Provider Selection docs). So you can't lock routing by putting Baseten first in settings—you must explicitly include :baseten.
Keep the model and provider in separate config, then assemble them at the call layer:
export HF_MODEL_ID="deepseek-ai/DeepSeek-V4-Flash-0731"
export HF_INFERENCE_PROVIDER="baseten"
model = f"{os.environ['HF_MODEL_ID']}:{os.environ['HF_INFERENCE_PROVIDER']}"
Reject an empty provider value at startup so a deployment mistake can't silently change routing. When you switch to a fallback provider, only the model string needs to change in config—but once the target model changes, re-verify behavior against the same set of business samples.
Don't assume every model is callable through Baseten. Hugging Face's 2026-08-06 announcement only confirms initial support for LLM and VLM chat; other tasks roll out later. Whether a model is available depends on the Hub model page and provider list at call time.
Adjust Account Settings for a Baseten Key
Hugging Face's official flow lets you add an API key for a provider you've registered and reorder provider preferences. With a Baseten key, calls are billed to your Baseten account and HF monthly credits don't apply.
Configure in this order:
- Create a purpose-specific, individually rotatable API key in your Baseten account settings. Whether Baseten distinguishes personal and team keys, and each key's permission scope, depends on the current Baseten settings page; team accounts should confirm key ownership and least privilege.
- Save the key in Hugging Face's Inference Providers settings.
- Confirm the code still pins
:baseten, unaffected by preference order. - Send a small test request and check usage records in both Hugging Face and Baseten.
- Rotate the test key and confirm the app fails loudly when the old key expires. Whether an expired key silently falls back to HF routed depends on the current routing implementation—verify with a real test before launch, don't assume from docs.
Migrate Existing OpenAI SDK Calls
For apps already using chat.completions.create, the minimal change is usually three edits: point base_url at the HF Router, switch auth to HF_TOKEN, and change the model to a Hugging Face model ID with :baseten. The messages array and the code that reads choices[0].message.content can stay as-is.
Don't stop at "text came back." At minimum, check against existing business samples:
- Whether system prompts, user prompts, and multi-turn context behave as expected
- Whether max output length, stop conditions, and sampling parameters are supported by the target model
- Whether streaming and non-streaming return structures match your parser contract
- Whether timeouts, rate limits, auth failures, and unavailable providers enter a clear retry or fallback path
- Whether input/output tokens and request counts can be verified in the chosen billing account
Parameter support differs by the target model's and provider's current interface. Extension parameters accepted by an old provider may be ignored or rejected; before migrating, remove unused parameters, then re-add them one at a time while logging the failure behavior.
Configure a Controlled Fallback Path
A unified entry point is not automatic fault tolerance. Don't retry every error against another provider—auth errors, parameter errors, and content-policy rejections can be amplified by blind retries.
Build a minimal policy by error type:
| Situation | Action |
|---|---|
| Network timeout or transient 5xx | Retry the same provider with backoff a limited number of times, then switch to a fallback provider |
| Rate limit | Respect the retry hint; switch to a fallback provider or queue past the latency budget |
| 401/403 | Stop automatic retries; check the token, key, and account permissions |
| 400 or unsupported parameter | Stop provider switching; fix the model ID, messages, or parameters |
| Model temporarily unavailable | Switch to a pre-verified fallback model and log the actual model |
Fallback models must be pre-verified with the same set of task samples. A model switch can change output format, instruction following, and context handling—HTTP success alone doesn't make them equivalent.
Pre-Launch Acceptance Checklist
- Chose HF routed or a Baseten key, and recorded the billing destination
- Team or enterprise confirmed the
X-HF-Bill-Tobilling destination for routed requests - Token and API key live only in secret management or deployment env vars
- Full model ID is injected via config and keeps
:baseten - Confirmed the target model currently supports text or visual conversation
- Minimal Python or JavaScript call returns and parses
message.content - Existing business samples cover context, output structure, streaming, and parameter compatibility
- 401/403, 400, rate limit, timeout, and 5xx each have a distinct handling path
- Fallback provider or model passed the same sample set
- Test request usage and cost appear in the expected account
- Rechecked current model list, rates, credits, and regional availability before launch
After Baseten joins Hugging Face Inference Providers, the integration-layer work is smaller—model strings, SDK init, and auth all unify behind the HF interface. But the model-migration verification itself (parameters, output structure, fallback behavior) doesn't shrink just because the entry point changed. :baseten only pins routing; confirm billing separately, and re-test behavior differences against business samples.