On 2026-07-23, NVIDIA published an official tutorial showing how to run 100 reinforcement learning steps on Nemotron 3 Nano with Prime Intellect Lab and then retest it on the same 32 held-out problems. NVIDIA recorded an increase from 7 correct answers to 29. To attribute that change to training, the baseline and retest must use the same problems, environment, and sampling settings. This article was verified on 2026-07-24.
Reproducing the example on the same basis requires prime==0.6.17, math-python@0.1.10, and the Nano BF16 model. Run the 32-problem baseline first, train for 100 steps with reinforcement learning with verifiable rewards (RLVR), and then retest by changing only the model identifier. Training produces a LoRA adapter; it does not rewrite the full base-model weights.
Before starting, your Prime Intellect account must have Hosted Training access and billing configured. The task also needs a verifier or reward function that can score outcomes automatically. If the objective can only be described as "give a better answer," design the evaluation rules before starting a run. NVIDIA's result applies only to math-python; customer-support, retrieval, language-specific, and high-risk tasks need their own training set, test set, and verifier.
Pin the Four Reproduction Constraints in NVIDIA's Example
Keep the following four values unchanged when reproducing the official result. Model availability and pricing still come from the live command-line interface (CLI) catalog.
| Item | Value Pinned in NVIDIA's Example |
|---|---|
| Python | 3.13 |
| Prime CLI | 0.6.17 |
| Base model | nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 |
| Training environment | primeintellect/math-python@0.1.10 |
Run prime train models and prime inference models. Continue only if the Nano model appears in the output and you have checked the current training, input, and output prices. NVIDIA's "less than $5" figure is the total spend for that specific run, not a standing price.
Use public or synthetic data for the first experiment. Do not put production personal data, medical, legal, or financial decision-making tasks, or any task without an independent test set directly into this workflow. If you only need a temporary inference service, see vLLM on Hugging Face Jobs.
Establish a 32-Problem Baseline with the Original Nano Model
Install the CLI version pinned by NVIDIA, initialize the Lab workspace, and inspect the environment package. If your organization prohibits curl | sh, use its approved method for installing uv.
curl -LsSf https://astral.sh/uv/install.sh | sh
uv python install 3.13
uv tool install --python 3.13 --force "prime==0.6.17"
prime --version
prime login
mkdir -p nemotron-customization
cd nemotron-customization
prime lab setup
prime env info primeintellect/math-python --version 0.1.10
prime env inspect primeintellect/math-python@0.1.10 README.md
uv add "math_python==0.1.10" \
--index https://hub.primeintellect.ai/primeintellect/simple/
uv lock --check
prime train models
prime inference models
Save the evaluation command as a script. The baseline and retest will pass different model identifiers to the same file, so the evaluation parameters cannot drift through manual copying.
mkdir -p scripts
cat > scripts/run-math-eval.sh <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
eval_model="${1:?Usage: scripts/run-math-eval.sh <model-id>}"
math_eval_args='{"dataset_name":"math","dataset_split":"test","num_train_examples":-1,"max_turns":5,"sandbox_client_max_workers":8}'
math_sampling='{"max_tokens":2048,"temperature":1.0,"extra_body":{"chat_template_kwargs":{"enable_thinking":false}}}'
prime eval run math-python \
--provider prime \
--model "$eval_model" \
--num-examples 32 \
--rollouts-per-example 1 \
--max-concurrent 8 \
--env-args "$math_eval_args" \
--sampling-args "$math_sampling" \
--timeout 300 \
--max-retries 2 \
--save-results \
--disable-tui \
--abbreviated-summary
EOF
chmod +x scripts/run-math-eval.sh
export BASE_MODEL="nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16"
scripts/run-math-eval.sh "$BASE_MODEL"
--save-results creates a result file. Record the actual path for the baseline and preserve a separate result from the retest. Compare individual examples later instead of relying only on the summary score.
The math-python environment asks the model to calculate with Python, NumPy, SymPy, or SciPy and put its final answer in \boxed{}. This example limits each rollout to five assistant turns. Repeated tool calls without a final answer receive a score of 0 after the turn limit is reached or the required \boxed{} answer is missing.
NVIDIA reported a baseline mean reward of 0.219 with a standard deviation of 0.413. The environment uses binary rewards: 1 for a correct answer and 0 for an incorrect or missing answer. A mean of 0.219 therefore represents 7/32, or 21.9%. Save at least the per-problem outcome and full trajectory. Track output length and truncation separately rather than keeping only the mean. Source: NVIDIA's official tutorial
Train a Nemotron 3 Nano LoRA Adapter with 100 RLVR Steps
Create configs/rl/nemotron-3-nano-math-python-100step.toml. The training configuration uses dataset_split = "train", separate from the test split used above. It disables thinking, limits each trajectory to five assistant turns, and allows up to 2,048 tokens per turn, matching the baseline settings.
name = "nemotron-3-nano-math-python-100step"
model = "nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16"
loss = "rl"
max_steps = 100
batch_size = 32
rollouts_per_example = 8
max_inflight_rollouts = 32
learning_rate = 2e-5
lora_alpha = 16
[sampling]
max_tokens = 2048
temperature = 1.0
enable_thinking = false
[adapters]
interval = 0
keep_last = 1
[[env]]
id = "primeintellect/math-python@0.1.10"
args = { dataset_name = "math", dataset_split = "train", num_train_examples = -1, max_turns = 5 }
To match NVIDIA's pinned prime==0.6.17 example, omit run:
prime train configs/rl/nemotron-3-nano-math-python-100step.toml
That release also accepts the prime train run <config> form shown in the current documentation; both forms start the same training flow. The command above stays aligned with NVIDIA's example. Before upgrading the CLI, run prime train --help and then follow the Prime Intellect Hosted Training documentation.
Before confirming the run, check the model ID and math-python@0.1.10 environment again. The CLI output must show an acceptable live price and a successful environment action check. Save the run ID and dashboard URL.
export RUN_ID="<run-id>"
# Terminal A: follow logs until training ends or you press Ctrl-C
prime train logs "$RUN_ID" --follow
# Terminal B: watch spend until you press Ctrl-C
prime train usage "$RUN_ID" --watch
# Any terminal: inspect progress and the reward distribution
prime train progress "$RUN_ID"
prime train distributions "$RUN_ID" --type rewards
# Any terminal: inspect rollouts from a selected training step
export STEP="<step>"
prime train rollouts "$RUN_ID" --step "$STEP"
These monitoring commands are not a sequential script. Run the log and spend watchers in separate terminals. Use progress to find an available training step, check whether the reward distribution has meaningful variation with distributions, set STEP, and then read the corresponding trajectories. High-reward trajectories must actually solve the problem; low-reward trajectories should be classified as calculation, formatting, or tool-use failures. If every attempt receives the same reward, adjust the environment or tasks instead of adding more training steps. If rewards rise while the solution process degrades, check for train/test leakage, truncation, and reward hacking.
Deploy the Final Adapter and Retest by Changing Only the Model Identifier
When the trainer exits, confirm that the final checkpoint produced a LoRA adapter with READY status. A stopped training process without a READY adapter has not produced the required artifact.
export RUN_ID="<run-id>"
prime train checkpoints "$RUN_ID" --status READY
prime deployments list
export ADAPTER_ID="<adapter-id>"
export BASE_MODEL="nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16"
export ADAPTED_MODEL="$BASE_MODEL:$ADAPTER_ID"
prime deployments create "$ADAPTER_ID" --plain
COLUMNS=300 NO_COLOR=1 prime deployments list --plain
After the state changes to DEPLOYED, retest with the same script and pass only the new model identifier. The test set, environment, and sampling parameters are already fixed in that file; do not create a second copy of the evaluation command.
scripts/run-math-eval.sh "$ADAPTED_MODEL"
For NVIDIA's binary evaluation, mean reward equals accuracy. Moving from 21.9% to 90.6% is an increase of about 68.7 percentage points, not a 68.7% relative increase.
| Metric | Original Nano | After LoRA Adaptation |
|---|---|---|
| Correct answers | 7/32 | 29/32 |
| Mean reward / accuracy | 21.9% | 90.6% |
| Standard deviation | 0.413 | 0.291 |
| Problems with a changed outcome | — | 23 improved, 1 regressed |
NVIDIA's 100-step run cost less than $5 in total. That result shows only that the pinned math experiment worked; it does not predict gains on another dataset or replace task-specific evaluation. Source: NVIDIA's official tutorial
The inference deployment continues to incur charges. Unload the adapter when the experiment is over:
export ADAPTER_ID="<adapter-id>"
prime deployments delete "$ADAPTER_ID" --plain
prime deployments list
Here, delete unloads the inference service but keeps the adapter file so that it can be deployed again later. The state should move from DEPLOYED through UNLOADING and finally return to NOT_DEPLOYED. Source: Prime Intellect's LoRA deployment documentation
Decide Whether to Move to Super or Ultra Only After Nano Works
Do not copy Nano's configuration and cost assumptions directly to a larger model. NVIDIA's tutorial provides the two identifiers below. Before using either one, run prime train models again to check account access, live pricing, and hardware availability.
| Model | Identifier in NVIDIA's Example |
|---|---|
| Nemotron 3 Super | nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16 |
| Nemotron 3 Ultra | nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4 |
Use Nano to validate the environment, reward, and retest loop first. If Nano still fails repeatedly on the same difficult task group after you rule out verifier errors, leakage, and truncation, then consider model capacity as a possible constraint. Before running Super or Ultra, write down a cost ceiling and minimum required improvement, and run a small baseline under the same settings. Stop if the result does not reach that threshold. If a model is visible but your account lacks training access, do not guess a substitute model ID.
Redesign the Verifier Before Adapting the Math Example to a Business Task
RLVR fits tasks whose outcomes can be checked by a program or rule, including math, code tests, structured-data conversion, and SQL with schema validation. A citation checker can confirm that a citation comes from the supplied material, but cannot automatically establish the quality of the conclusion. Customer-service tone, open-ended writing, and high-risk advice are poor candidates for a first RLVR migration.
Rebuild the experiment in this order:
- Express the business success condition as an executable verifier and list acceptable failure boundaries.
- Separate the training set, development set, and final test set that will never be used for tuning.
- Run the baseline repeatedly on the original model and confirm that its score is neither all zero nor close to perfect.
- After training, compare regressions, cost, and output length by task group before deciding whether to continue.
If you have high-quality input-answer pairs but no reliable verifier, start with supervised fine-tuning (SFT) rather than RLVR. Both methods still need an independent test set; training reward or training loss cannot replace release evaluation.
Define Measurable Release Criteria Before Sending Production Traffic
NVIDIA's 32-problem result only shows that the example ran successfully. Before a business deployment, freeze the test set and define the minimum overall score, critical-task floor, allowed regression count, and daily inference budget before looking at the retest. The criteria below are recommendations from AIGCDev, not production approval criteria issued by NVIDIA or Prime Intellect.
| Check | Release Condition |
|---|---|
| Reproducibility and regression | Retest the same adapter independently at least three times and meet the predefined overall score each time; critical task groups must not regress, and other regressions must stay within the predefined allowance |
| Data | Training data, evaluation data, and their logs pass privacy, licensing, and internal retention checks |
| Operations | A rollback to the original model has been exercised, spend alerts are active, and unused inference deployments can be unloaded |
Keep the adapter in the experiment environment until every condition has been verified. A higher mean score cannot offset regressions on critical examples or prove that performance also improved for other languages, long contexts, or multi-tool tasks.
Troubleshoot by Symptom
| Symptom | Check First |
|---|---|
| Baseline reward is always 0 | Confirm that the model and environment are available, then inspect failed trajectories for answer-format errors, exhausted turns, or excessive problem difficulty |
| Training reward rises but retest performance does not | Check whether you evaluated only the training set, changed evaluation parameters, or introduced reward hacking |
| Mean score rises but critical problems regress | Group test problems by business risk instead of allowing an overall mean to hide critical failures |
| Charges continue after the experiment | Confirm that prime deployments list eventually shows NOT_DEPLOYED; any other state, especially UNLOADING or UNLOAD_FAILED, means cleanup is not complete |