On 2026-06-26, Hugging Face announced that you can spin up a vLLM inference server on HF Jobs with a single command (source: Hugging Face Blog). Concretely, hf jobs run takes a container image and a launch command, exposes port 8000 to give you an OpenAI-compatible endpoint, and bills per second. If you just need temporary tests, one-off evals, batch generation, or a backend for a coding agent like Pi, Jobs is faster to spin up and cheaper than Inference Endpoints. If you need a long-lived production endpoint, scale-to-zero, or fine-grained access control, go with Inference Endpoints.
Prerequisites
- Hugging Face account, logged in via CLI:
hf auth login huggingface_hub >= 1.20.0:pip install -U "huggingface_hub>=1.20.0"- A payment method or positive prepaid balance (Jobs bills per minute of hardware usage)
Source: HF official article, 2026-06-26.
Launch
Using Qwen/Qwen3-4B as an example, spinning up a server on A10G with a 2-hour auto-timeout:
hf jobs run --flavor a10g-large --expose 8000 --timeout 2h \
vllm/vllm-openai:latest \
vllm serve Qwen/Qwen3-4B --host 0.0.0.0 --port 8000
Once started, it returns the job URL and the exposed port URL in this format:
https://<job_id>--8000.hf.jobs/v1
After a minute or two, the logs show Application startup complete—your server is ready.
Usage
curl
curl https://<job_id>--8000.hf.jobs/v1/chat/completions \
-H "Authorization: Bearer $(hf auth token)" \
-H "Content-Type: application/json" \
-d '{
"model": "Qwen/Qwen3-4B",
"messages": [{"role": "user", "content": "Hello!"}],
"chat_template_kwargs": {"enable_thinking": false}
}'
Returns an OpenAI-style JSON response.
Python SDK
from huggingface_hub import get_token
from openai import OpenAI
client = OpenAI(
base_url="https://<job_id>--8000.hf.jobs/v1",
api_key=get_token(),
)
resp = client.chat.completions.create(
model="Qwen/Qwen3-4B",
messages=[{"role": "user", "content": "Hello!"}],
extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
print(resp.choices[0].message.content)
Check available models
curl https://<job_id>--8000.hf.jobs/v1/models \
-H "Authorization: Bearer $(hf auth token)"
Endpoint security
The exposed port URL is not a public endpoint. Every request must carry an HF token with read access to the job's namespace (source: HF official article, 2026-06-26). If you need finer-grained access control or public access, put a gateway in front, or use Inference Endpoints instead.
Stop
Billed per second, so stop when done to avoid waste:
hf jobs cancel <job_id>
The --timeout is a safety net, but cancelling manually saves more money. An a10g-large runs at roughly $1.50/hour.
Advanced Usage
Chat UI with Gradio
The HF article includes a full Gradio example: after launching the job, run a local Python script (using the OpenAI client with chat template kwargs for reasoning), open http://127.0.0.1:7860, and you get a chat interface with a reasoning panel (source: HF official article, 2026-06-26).
SSH into the container
Add --ssh when launching, then use hf jobs ssh <job_id> to get a shell inside the container to run nvidia-smi, monitor processes, or debug startup failures. Make sure your SSH public key is registered at huggingface.co/settings/keys (source: HF official article, 2026-06-26).
hf jobs run --flavor a10g-large --expose 8000 --timeout 2h --ssh \
vllm/vllm-openai:latest \
vllm serve Qwen/Qwen3-4B --host 0.0.0.0 --port 8000
hf jobs ssh <job_id>
Backend for Pi coding agent
Add --enable-auto-tool-choice --tool-call-parser hermes to the launch command, and use a stronger model (the HF article uses Qwen3.5-122B-A10B):
hf jobs run --flavor h200x2 --expose 8000 --timeout 2h \
vllm/vllm-openai:latest \
vllm serve Qwen/Qwen3.5-122B-A10B \
--host 0.0.0.0 --port 8000 --tensor-parallel-size 2 \
--max-model-len 32768 --max-num-seqs 256 \
--reasoning-parser deepseek_r1 \
--enable-auto-tool-choice --tool-call-parser hermes
Then configure a custom provider in ~/.pi/agent/models.json:
{
"providers": {
"hf-jobs": {
"baseUrl": "https://<job_id>--8000.hf.jobs/v1",
"api": "openai-completions",
"apiKey": "!hf auth token",
"models": [
{ "id": "Qwen/Qwen3.5-122B-A10B" }
]
}
}
}
Then launch pi to get a coding agent powered by your self-hosted model (source: HF official article, 2026-06-26).
When to Use Jobs vs. Inference Endpoints
| Scenario | Recommendation |
|---|---|
| Temporary tests, one-off evals, batch generation | HF Jobs |
| Long-lived stable endpoints, needs scale-to-zero | Inference Endpoints |
| Debugging container startup or inference issues | HF Jobs (with --ssh) |
| Needs public/protected/private fine-grained access control | Inference Endpoints |
| Self-hosted coding agent (Pi, etc.) during development | HF Jobs |
| Reusing job images in CI/CD | HF Jobs |
| Production-facing API | Inference Endpoints + gateway |
Source: HF official article, 2026-06-26.
FAQ
What models does the vLLM server support?
Any model vLLM supports—Qwen, Llama, Mistral, DeepSeek, etc. Note that some models may need additional tokenizer or chat template adjustments.
How much does per-second billing cost?
An A10G large runs at roughly $1.50/hour, H200 at several dollars per hour. Check the HF Jobs pricing page for exact rates.
Is the exposed port public?
No. The HF article explains that endpoint access is gated: every request needs an HF token with read access to the job's namespace. It's not a public endpoint and shouldn't be used for direct external access.
Can I use other inference engines?
The HF article says the same --expose port pattern works with llama.cpp (GGUF), SGLang, and other OpenAI-compatible engines, as long as the image exposes the corresponding port. This guide covers vLLM only; see the Serve Models on Jobs guide for other engines.
How does this differ from Hugging Face Spaces?
Spaces is for interactive demos and prototyping, with limits on GPU specs and run duration. Jobs is closer to a bare container, giving you precise control over vLLM flags, hardware specs, and runtime.
What is Pi?
Pi is a provider-agnostic agent harness that supports custom provider configurations for self-hosted model backends. See the Pi project page for the README and docs.