Treat Mistral Agentic Search as a component to validate before it replaces any one-shot RAG route. Released on 2026-08-20, it lets a model search an index iteratively, then locate, read, and verify information inside candidate documents. As of 2026-08-21, Mistral offers it through Search Toolkit and through Libraries in Studio and Vibe; the launch post does not state plan, region, or pricing limits. The Search Toolkit changelog calls 0.0.8 a tech preview, while the currently listed 0.0.9 still includes breaking changes. Recheck the version, account availability, and commercial terms before production use.
Keep one-shot retrieval-augmented generation (RAG) when a query asks for a known field in a short document or needs to return relevant passages at high volume. Test Agentic Search only when the answer is spread across long documents, tables, footnotes, or several sources and the response must point back to a reviewable location. “Use the existing index” also has conditions: the current backend must connect to Search Toolkit and preserve document identity, access metadata, and stable locators.
For acceptance testing, freeze the corpus snapshot, index configuration, and model, then compare one-shot RAG, a multi-step search loop, and the full navigation loop. Record task pass rate, evidence reproducibility, cost per successful task, and 90th-percentile (p90) latency for each path. Do not expand routing if full navigation does not produce a stable pass-rate gain or cannot return reviewers to the source.
Route each query to one-shot RAG or Agentic Search
Mistral's launch post recommends indexed retrieval for direct lookups in short documents, high-volume keyword or semantic search, and predictable questions whose likely source is already known. Long documents, multiple sources, tables, and answers that require verification are the intended Agentic Search cases.
| Query characteristic | Default path | Upgrade or stop condition |
|---|---|---|
| The answer usually appears in the first retrieved passages | One-shot RAG | Add search rounds only if a fixed evaluation set repeatedly misses the right passage |
| The document is short and stable, with a known field location | One-shot RAG or a structured query | Do not add an agent tool loop to a field that can be located directly |
| The task returns relevant passages at high volume and needs no synthesis | Indexed retrieval | Optimize recall, latency, and throughput; do not open entire documents |
| The answer is buried in a table, clause, figure caption, or footnote | Agentic Search | Return a document and stable page or location; otherwise report insufficient evidence |
| The answer requires comparing contracts, filings, or specifications | Agentic Search | The trace must cover every required source; omit the conclusion if one is missing |
| Finding the evidence requires several query rewrites | Agentic Search | Cap search rounds and accessible documents, then escalate when the cap is reached |
Route by query type rather than moving the whole knowledge base at once. Keep easy traffic on one-shot RAG and send Agentic Search only the query classes that have demonstrated a need for in-document navigation or cross-source verification.
Give search, open, navigate, read, and grep separate jobs
Mistral lists five tools in the launch post. They separate finding candidate documents from verifying an answer inside those documents:
| Tool | Role | What to inspect during evaluation |
|---|---|---|
search |
Find candidate documents through the existing index | Does the query become more specific, or keep returning the same irrelevant results? |
open |
Open one candidate document | Did the model select the correct version, date, and access scope? |
navigate |
Move to a page, section, or region | Is the locator stable, and do scanned-PDF page numbers match the original? |
read |
Read content at the current location | Are headers, units, footnotes, and surrounding context preserved? |
grep |
Find a text pattern in an open document | Do wording changes, OCR errors, or duplicate field names cause misses? |
The launch post describes the tools but does not promise that they automatically inherit existing document permissions. Make authorization an acceptance requirement: search may return only documents the current user can access, and open and read must check access again. An index hit must not bypass tenant, department, or project boundaries.
Run one-shot RAG, multi-step search, and full navigation on the same index
Mistral says Agentic Search can build on an existing index and does not require model-specific fine-tuning. That does not mean every index connects without integration work. The current Search Index documentation lists Vespa as the built-in backend and custom vector stores as the extension path. Before testing, confirm that the adapter returns content, document identity, stable locators, and access metadata. Then hold the corpus snapshot, index configuration, model version, and scoring rules constant, changing only the retrieval tools available to the model.
| Path | Allowed tools | Question it answers |
|---|---|---|
| A: one-shot RAG baseline | One index query, then answer | Which query classes fail in the current system? |
| B: multi-step search | Repeated search calls |
Do query rewrites and retries recover missing documents? |
| C: full navigation | search plus open, navigate, read, and grep |
Does in-document navigation improve correctness and reduce wasted search? |
For the shared index, save at least the corpus snapshot, index-configuration hash, document-identity rules, and locator rules. Do not change chunking, the embedding model, the ranker, and the generation model at the same time. After the three paths have run, evaluate parser, chunking, hybrid retrieval, or reranking changes separately.
Give every evaluation question an answer and an evidence location
Build the evaluation set from existing search failures, manual-review records, and real user follow-ups. Do not generate a batch of questions that merely look plausible. Each scored question needs a human-confirmed answer, required sources, and evidence locations. Put questions without a defensible answer into an exploratory set rather than the launch gate.
The following fields are illustrative and are not results from a real business dataset. run_repetitions: 3 is only a starting point for a small pilot, not a universal statistical threshold; choose the production run count from risk and sample size.
query_id: contract-renewal-notice-period
corpus_snapshot: contracts-2026-08-21
index_config_hash: sha256:<index-config-hash>
model_id: <explicit-model-id>
harness_version: <commit-or-release>
scorer_version: <rubric-version>
tool_budget: search=4, open=3, read=6
run_repetitions: 3
question: Compare the automatic-renewal notice period in the current and prior contracts.
required_sources: current contract, prior contract
expected_evidence: page and text of the renewal-and-termination clause in both contracts
answer_rule: state each period, version date, and difference; report insufficient evidence if either source is missing
forbidden_behavior: infer one contract from the other; omit units; treat OCR text as verified original text
Run the same question through A, B, and C and record:
| Record | Pass condition |
|---|---|
| Run configuration | Data and index versions, model and tool settings, and harness and scorer versions are reviewable |
| Final answer | Matches the human-confirmed rule, including numbers, dates, and units |
| Evidence completeness | Covers every required source and provides a location that can be reopened |
| Retrieval trace | Contains no unauthorized document, unproductive loop, or omitted required source |
| Stopping behavior | Does not guess when evidence is missing, a tool fails, or the round limit is reached |
| Repeated runs | Uses the same run count for all paths and reports pass rate and failure distribution |
| Resource use | Records model, OCR/embedding, retrieval-tool, and infrastructure costs plus mean and p90 latency |
A task succeeds only when both the answer and its evidence pass. A correct number tied to the wrong version or an unrecoverable page is still a failure. Normalize each path's cost as “all costs for that path ÷ successful tasks on that path.” If the sample is too small for a stable p90, publish each latency and the sample count instead of a precise-looking tail metric.
Use FinanceBench and OfficeQA Pro to choose question types, not launch targets
Mistral's 2026-08-20 results use the default Search Toolkit configuration with no use-case-specific tuning and test Mistral Medium 3.5 and GLM-5.2. They show that multi-step search and document navigation may help on long, dense, table-heavy material. They do not predict the gain on an enterprise knowledge base.
| Official evaluation | Documents and questions | Result reported by Mistral | Question types worth adding to an internal evaluation |
|---|---|---|---|
| FinanceBench | 368 U.S. Securities and Exchange Commission filings, 150 questions, about 147 pages on average | Moving from one-shot RAG to multi-step search adds 47.3 and 52.6 percentage points for the two models; navigation then adds another 8.7 and 6.7 points |
Long filings, figures across sections, repeated query reformulation |
| FinanceBench resource use | Same evaluation | Full navigation uses 23.9% and 33.7% fewer tokens than multi-step search; overall p90 falls from 255 to 154 seconds and mean latency from 108 to 71 seconds |
Quality, cost, and tail latency together rather than tool-call count alone |
| OfficeQA Pro | 696 U.S. Treasury Bulletins, 133 hard questions, about 89,000 scanned and table-heavy PDF pages | The full GLM-5.2 path reaches 51.9%, 45.6 points above one-shot RAG; Mistral Medium 3.5 gains 27.1 points | OCR documents, cross-table calculation, verifiable numerical answers |
The launch-page summary also gives one aggregate FinanceBench result from 26.7% to 86%. All of these figures come from Mistral's own evaluation; the post does not provide per-question outputs, the scorer prompt, or full run artifacts, so this is not an independent reproduction. When rerunning public benchmarks, pin the FinanceBench data version, the OfficeQA Pro data and scorer version, and the harness commit. If your corpus is mostly short FAQ pages, clean web pages, or fixed fields, keep that traffic on one-shot RAG.
Check every isolation boundary for sensitive documents
Mistral says Search Toolkit's open modules can run in the cloud or on premises and can use an existing index. A local index does not mean the entire processing path remains local. In the official Quickstart, Vespa runs locally while the example embedder still uses the Mistral API. Before launch, map data flow across ingestion, retrieval, inference, and logging, then record storage location, provider, retention period, and access role for each group.
Verify at least these four boundaries:
- Enforce document authorization at
search,open, andread, not only at the chat entry point. - Keep unrelated body text, personal information, and cross-tenant passages out of retrieval and debug logs.
- Follow the OWASP RAG security guidance and treat documents as untrusted data; instructions inside them must not change tool permissions, the system prompt, or answer rules.
- If authorized evidence is unavailable, return insufficient evidence rather than expanding to an unapproved source or public web search.
Keep the system in testing if any stage cannot explain where data goes, who can read it, or when it is deleted. For strict isolation, verify local parsing, indexing, and inference separately before selecting any cloud component.
Keep one-shot RAG, Agentic Search, and human review as separate exits
Production routing needs more than a binary choice between one-shot RAG and Agentic Search. Send simple questions to one-shot RAG, route only proven multi-step classes to Agentic Search, and send missing evidence, authorization conflicts, tool failures, or exhausted search budgets to human review or an explicit refusal.
Before expanding the rollout, confirm that:
- Complex-query answer-and-evidence pass rate is higher than the one-shot RAG baseline
- Misrouted simple queries do not increase cost per successful task or tail latency
- Every response can replay documents, locations, and tool order; revoking access or deleting a document invalidates both index and tool access
- An Agentic Search outage falls back to one-shot RAG or insufficient evidence, never an ungrounded answer
Pause the Agentic Search route for a query class after any unauthorized document, wrong version, or unsupported conclusion on a high-risk task. Fix authorization, index, or answer rules and rerun the same evaluation set before restoring it.
Three questions to answer before integration
Must the existing index be rebuilt?
Not necessarily. Mistral says Agentic Search can run on an existing index, but the current Search Index documentation provides built-in Vespa support and a custom-vector-store interface for other backends. Start with the original index only if its adapter returns content, document identity, stable locators, and access metadata. Add an adapter, migrate, or rebuild when parsing, chunking, table extraction, or OCR has already lost required information or the backend cannot return those fields.
Must the model be fine-tuned?
No. Mistral says the five retrieval tools require neither fine-tuning nor model-specific training. Practical results still depend on whether the model selects tools correctly, respects round limits, and returns stable evidence locations. Test those behaviors with your own questions.
What is the fastest way to try it?
Mistral recommends Search Starter App. It is a Copier project template; a local trial requires Python 3.12+ with uv, Docker/Vespa, and a Mistral API key. Its defaults are useful for checking ingestion, retrieval, and tool traces, not as a production configuration. Choose Libraries for a direct Studio or Vibe trial. For an existing agent, workflow, or customer deployment, evaluate the Search Toolkit version, backend adapter, and commercial terms first.