Adding tools, memory, and multi-step execution to an AI agent does not prevent dropped fields, duplicate calls, or drift on long tasks. When those failures appear, inspect how the harness passes context, executes actions, stores state, and decides that a task is complete.
On 2026-07-27, NVIDIA Developer Blog published Six Agent Harness Capabilities for Higher Model Performance, introducing NVIDIA Labs Object-Oriented Agents (NOOA). As of 2026-07-28, the NOOA repository is public, but its CHANGELOG still lists the initial public release under Unreleased. The code and evaluation methods are public, so teams can inspect and reproduce the work; NOOA remains a research preview rather than a stable replacement for an existing harness.
Use the six capabilities to audit the current system: fix input/output contracts and context references, isolate model-generated code and tighten the execution loop, then make state explicit while limiting model-callable harness APIs. Keep every change in an isolated environment until you have a fixed evaluation set, complete trajectories, and a rollback path.
Who should audit the harness first
The checklist is for developers and platform teams that already have an agent prototype, can retain task trajectories, and are willing to compare the old and new systems with the same model and task set. If single-turn answers are poor, inspect the model, prompts, and retrieval first. If the baseline cannot be reproduced, fix evaluation and logging before changing the harness.
| Current symptom | First place to inspect |
|---|---|
| Tool results are long and the prompt keeps growing | Object references and bounded previews |
| Output fields are missing or frequently invalid | Type contracts and return-value validation |
| Long tasks lose state or repeat work | Explicit state and event history |
| No fixed evaluation set or complete trajectory | Pause the harness change and establish a baseline |
Write down four completion checks before implementation: whether task success improves, tokens per task fall, failure categories shrink, and which previously passing tasks regress. If one of those questions cannot be answered, do not send production traffic to the new harness.
How NOOA represents an agent
NOOA represents an agent as one Python class. Method signatures and docstrings define the task and type boundary, while fields hold model-visible state. A method body containing ... is executed by an LLM-driven loop; a normal method body runs as deterministic code. The NVIDIA technical report describes the programming model and evaluation methodology in detail.
| Python construct | Responsibility in NOOA |
|---|---|
| One class | Agent capabilities, state, and prompt boundary |
| Method signature and docstring | Typed inputs/outputs and task instruction |
... method body |
Model-driven generation or a CodeAct loop |
| Normal methods and fields | Deterministic rules, external capabilities, and explicit state |
A CodeAct loop lets the model write Python, call methods on the current object, and operate on objects in the execution environment. NOOA checks the abstract syntax tree and restricts modules, but those measures do not isolate the host. NVIDIA requires agents that execute model-generated code to run in an OS-level sandbox such as a container, virtual machine, or NVIDIA OpenShell. The repository safety note specifically warns about file deletion, environment modification, and data exfiltration.
The second column below describes the official NOOA mechanism. The third contains conservative recommendations for an existing production system; do not treat the two columns as the same claim.
| Official capability | NOOA mechanism | What to inspect in an existing system |
|---|---|---|
| Typed input/output | Method arguments and return values are type-validated | Whether tool arguments, tool results, and final answers have executable validation |
| Pass by reference | Objects stay in the execution environment while the model sees bounded previews | Whether large objects are repeatedly serialized into the prompt |
| Code as action | The model writes and runs Python inside the execution loop | Whether code runs in isolation and side effects pass deterministic gates |
| Programmable loop engineering | Developers and the model can express loops in ordinary Python | Whether stop, retry, budget, and permission rules are system-enforced |
| Explicit object state | Typed state lives on the agent object | Whether task, evidence, and risk state exist outside conversation history |
| Model-callable harness APIs | The model can inspect context blocks and event history | Whether read-only inspection is available before any write permission |
Phase 1: Fix contracts and context
Start with two problems that can be measured directly: input/output validation failures and large objects repeatedly entering the prompt. After the change, you should be able to track both the validation-failure rate and tokens per task.
Step 1: Replace free-text boundaries with type contracts
Telling a model to "return JSON" does not create a contract. Inputs, tool results, and final answers must pass a schema or equivalent code validation. On failure, return field-level errors to the execution loop so it can retry or stop.
| Boundary | Required validation | Failure handling |
|---|---|---|
| Tool input | Types, required fields, enum values, and length limits | Reject execution and return field-level errors |
| Tool output | Success state, error code, and critical result fields | Retry or stop according to the error category |
| Final answer | Task status, evidence, limitations, and next action | Do not deliver an answer with required fields missing |
The following example shows a field contract, not an executable JSON Schema. Implement the field names and validators for the actual business domain.
{
"action": "create_ticket",
"input": {
"customer_id": "string",
"priority": "low | medium | high",
"evidence_ids": ["string"]
},
"output": {
"status": "created | rejected",
"ticket_id": "string | null",
"errors": ["string"]
}
}
Count validation failures, successful automatic corrections, and final stops separately. Format errors that previously required manual log reading must become queryable metrics.
Step 2: Keep large objects in the execution environment
NOOA pass-by-reference keeps live Python objects in the execution environment. The model first sees a bounded preview and queries through methods when it needs the full value. The technical report identifies this as one mechanism that reduced context tokens on SWE-bench Verified.
An existing system may not support live Python references. Start with auditable object handles:
- Store a long document as a
doc_id; put only its summary, section index, and retrievable excerpts in context. - Return a retrieval result's title, source, summary, and evidence ID instead of inserting every result.
- Query tables through functions that return small windows and record the filter and row range.
- Keep logs, images, and binary data behind resource references; expose an error summary or extracted result only when needed.
Every handle should record its source and update time, and make object size, queryable fields, and access permissions inspectable. Monitor average prompt tokens and task success together. If success falls, expand the preview or add a query interface before release.
Phase 2: Constrain code and the execution loop
Letting the model write code also expands what it can do to files, networks, and processes. Run that code in an OS-level sandbox first, then use an outer loop to enforce budgets, permissions, approvals, and stop conditions.
Step 3: Put model-generated code in a sandbox
NOOA's CodeAct strategy executes model-generated Python inside the agent process. The limitations section of the technical report states that in-process validation does not protect the host. In production, let the model make semantic judgments and propose an action; deterministic code must own permission checks, idempotency, auditing, and final commit.
| Action type | What the model may decide | What the system must enforce |
|---|---|---|
| Read information | Query target and conditions | Permission, scope, and returned fields |
| Draft content | Candidate content | Format, sensitive-data, and citation checks |
| Change system state | Change intent and draft parameters | Idempotency, approval, audit, and commit |
| High-risk operation | Plan awaiting confirmation | Stop by default and wait for human approval |
At minimum, the container or virtual machine must restrict mounted directories, network egress, credentials, and process resources. When an action fails, the trajectory should distinguish model or contract errors, permission or policy denials, and external-system failures. If the cause cannot be identified, stop adding tools.
Step 4: Put stop and retry rules in code
NOOA's programmable loop lets developers and the model express control flow in ordinary Python. The model may orchestrate constrained inner steps, but completion, maximum steps, cost, and high-risk approval must be controlled by an outer program the model cannot bypass.
- Completion and stop: define the required fields and evidence, maximum steps, and human-escalation conditions.
- Retry and budget: retry only transient failures and cap attempts, tokens, spend, and total duration.
- Tools and permissions: expose tools by task phase; pause and request approval before expanding permissions.
- Context and failure output: define what enters short-term context and return an actionable failure reason to the user.
The system prompt explains the rules; the program boundary enforces them. Tests must cover normal completion, retryable errors, non-retryable errors, and budget exhaustion.
Phase 3: Separate state and limit self-management APIs
Long tasks need completion criteria, evidence, tool results, and risk flags stored as inspectable state. Give the model read-only access to events and context first. Open write-capable self-management APIs only after permission controls, auditing, and replay are stable.
Step 5: Move state out of conversation history
NOOA keeps explicit object state on the agent instance and re-renders public fields from the live object on every turn rather than rebuilding them from conversation history. An existing system can begin with four typed state groups.
| State | Example fields | Purpose |
|---|---|---|
| Task state | goal, current_step, blocked_reason |
Select the next action and decide completion |
| Evidence state | source_ids, verified_claims, unknowns |
Avoid duplicate verification and unsupported conclusions |
| Tool state | failed_calls, resource_refs |
Control retries and resource references |
| Risk state | needs_approval, policy_flags, rollback_plan |
Block unapproved side effects |
State must be serializable, auditable, and replayable. A plan stored only in conversation history is not reliable state; history compression or cross-session recovery may lose it.
Step 6: Start with read-only harness APIs
NOOA exposes context blocks, per-turn dynamic context, and event history as model-callable APIs; developers decide which interfaces each agent can see. Under the technical report's definition, these are not ordinary tool calls: they let the model inspect and manage its own working context.
- Read-only inspection: view event history, resource references, and current state.
- Constrained writes: add only working notes or flags for facts that still need verification.
- Memory candidates: submit long-term memory candidates for policy or human review.
- Production APIs: expose only operations that have passed permission, audit, and replay validation.
The optional long-term memory described in the NVIDIA technical blog records types, importance, tags, and relationships, then merges duplicate or conflicting records after a task. Those are research mechanisms; they do not make business memory automatically trustworthy. A production system still needs source, write reason, correction, and deletion paths.
Use the same task set to decide release
NVIDIA reports that with GPT-5.5 at xhigh reasoning effort on SWE-bench Verified, NOOA reached 82.2%, averaging about 28 model calls and 1.1 million tokens per task. The comparison PI harness reached 78.2% with 66 calls and 2.2 million tokens. Those results apply only to the report's model, agent, tools, and evaluation setup; they are not a promised gain for customer-support, knowledge-base, or operations agents. See the NVIDIA technical report.
| Metric | How to record it |
|---|---|
| Task success and regressions | Use a fixed task set and list tasks that passed before but fail now |
| Tokens and model calls | Record prompt, output, tool-result tokens, and call count separately |
| Contract and tool errors | Classify validation, permission, external-system, and stop causes |
| Trajectory reviewability | Check whether a reviewer can replay critical actions and locate the failure source |
Change one capability at a time while holding the model, task set, and scoring rules constant. Test object references only after type contracts are stable. Open state writes or long-term memory only after the code sandbox and outer loop pass fault tests.
Production release conditions
- Model-generated code always runs in a constrained container, virtual machine, or OpenShell.
- Deterministic code owns access control and idempotency; the program also enforces approval, budget, and stop rules.
- The fixed task set covers success, regression, failure, and high-risk denial, and the rollback has been rehearsed.
- The trajectory is sufficient to reconstruct model actions, system gates, external side effects, and human approvals.
The scope here is the interface between the model and its harness, not the complete safety controls for long tasks or the process for building an evaluation set. Long-running autonomous work also needs the long-horizon safety gates. If no fixed task set exists, establish one with the AI agent evaluation workflow. Before model-generated code reaches a real environment, check the isolation boundary with the NVIDIA OpenShell and local-agent safety checklist.
FAQ
Should NOOA directly replace LangGraph, AutoGen, or an in-house framework?
No. As of 2026-07-28, the code is public, but the repository has no stable version tag, so NOOA should still be treated as a research preview. Audit the current system against the six capabilities first, then use an apples-to-apples evaluation to decide whether to use the NOOA implementation.
How can I evaluate without SWE-bench or CyberGym?
Use a fixed task set from your own domain. A customer-support agent can use redacted historical tickets with human scoring, while a coding agent can use unit tests and a review checklist. Score a research agent on citation accuracy and evidence coverage. The baseline and new system must use the same tasks, model, and scoring rules.
Can the memory system write every model conclusion automatically?
No. Automatic writes carry wrong conclusions into later tasks. Long-term memory should retain the source, timestamp, write reason, and correction state; high-risk facts also need policy or human confirmation.
When should a team pause the harness change?
Pause if complete trajectories cannot be stored, the baseline cannot be reproduced, or the team cannot tell whether a failure came from the model or the system boundary. Adding more tools or memory would destroy comparability between the old and new systems. Resume only after logging, the evaluation set, and a minimum type contract are in place.