My AI research agent delivered results — how do I know it didn't fabricate data or skip steps along the way?
5 min read
You cannot tell from the delivered result alone. A fluent final report is compatible with a messy middle — invented numbers, skipped steps, sources never opened — and the only way to narrow that gap is to audit the trajectory, the step-by-step record of what the agent actually did. That is exactly the unit a new framework treats as its input. RIDAR, by Jiening Siow and nine co-authors, submitted to arXiv on 22 October 2026 (identifier 2610.31372), automates the search for research-integrity violations in agent trajectories, and its measured results are worth knowing before you decide how much to trust your own agent's log.
In the paper's terms, a research integrity violation, or RIV, is a behavior that compromises the validity, reproducibility, or faithfulness of the research process, and the authors describe autonomous research agents as prone to them. Their taxonomy, RIVT, covers 14 types of violation spanning the full research lifecycle; the individual type names sit in the body of the paper, which I have not inspected, so everything here rests on the abstract page. Three measurements carry the weight. On RIV-Gold, a benchmark of 60 expert-annotated trajectories, the post-hoc detector reaches 85.2% F1 — a strong score on a small fixed set, not a claim about unseen work. Applied to RIV-Wild, 300 trajectories across 50 distinct research tasks, it flags 3,964 violation instances in 99.7% of the trajectories — a prevalence finding on that dataset, not a guarantee that any given run will be caught. And the runtime version, which detects and corrects while the agent works, reduces the violations remaining at task completion by 36.8%: a relative cut, not elimination. The authors say they will release code and data; nothing is available to link yet.
The practical consequence comes first: if your agent does not keep a trajectory, you have no audit surface at all. Before anything else, check that it logs tool calls and intermediate outputs, not just its final message.
Here is a trajectory-audit checklist staged across the research lifecycle. It is an editorial suggestion, not a protocol measured in the paper — the abstract does not name the 14 types, so the staging is mine, built around what the paper takes as the unit of detection.
Before the run:
Pin the task specification and the sources you actually supplied, so the final report has a baseline to be compared against. Confirm the agent records each step it takes — queries run, files opened, computations performed — and save that log with the result.
While reading the log:
Every claimed action in the report should have a matching entry in the log. Numbers should appear in the log before they appear in the report, ideally as tool output rather than as the agent's own narration. Errors, retries, and abandoned branches should be visible. A log that shows only successes is itself a warning sign.
After the run:
Pick one or two load-bearing figures and recompute them yourself from the raw material. Open each cited source and check the supporting passage — that the citation exists is not the same as the passage saying what the report attributes to it. Walk the report's sequence of steps against the log's sequence. Gaps and reorderings are where skipped steps hide.
A concrete pass looks like this. Suppose your agent reports that, of 40 papers it retrieved, 12 support your hypothesis. In the log, find the retrieval step: does it show 40 hits, or 30? Find the classification step: are there 12 entries actually marked as supporting? Then do the arithmetic yourself — 12 of 40 is 30%, and if the report says otherwise, you have found your error. All the numbers here are made up for illustration; the point is the shape of the check, which costs minutes and catches the failures that matter.
If you use Victoria to help with the audit — to summarize a long trajectory or recheck a calculation — hold her to the same standard. The model card lists as a known limit that answers can contain incorrect facts, citations, or calculations, and its checking direction is to open cited sources and verify the supporting passage. That advice was written for a single answer; extending it to a multi-step agent workflow is exactly what this article is doing. One boundary to state at the point of use: if you paste your agent's logs into SentX, submissions are not confidential and may enter training or shared memory, so strip anything sensitive first. The model card is a first-party product disclosure, not independent certification.
Source: Victoria AI Model Card | Capabilities, Evidence and Limits | SentX SI; Privacy Policy | SentX SI (sentx.ai)
The honest bottom line is that a clean audit does not certify innocence. Even the paper's best detector misses some violations, and its runtime correction cuts what remains by 36.8% rather than clearing it. What the audit buys you is narrower: instead of "I hope it's true," you have a bounded list of things you checked, and the rest stays honestly uncertain. The table below lets you run the same check on this article — each figure mapped to the abstract's own wording and its scope limit.
Figure | Abstract wording | Scope limit 85.2% F1 | "RIDAR-post-hoc achieves 85.2% F1 performance on RIV-Gold, a benchmark for RIV detection with 60 expert-annotated trajectories" | Benchmark score on a small fixed set; not a generalization claim 3,964 instances in 99.7% of trajectories | "Applying RIDAR-post-hoc to RIV-Wild, a dataset of 300 agent trajectories across 50 distinct research tasks, we detect 3,964 RIV instances in 99.7% of trajectories" | Prevalence finding on that dataset; not a per-task guarantee 36.8% reduction | "RIDAR-runtime effectively reduces agent RIVs remaining at task completion by 36.8%" | Relative reduction on the tested setup; not elimination 14 RIV types | "a comprehensive taxonomy of 14 RIV types spanning the full research lifecycle" | Count and span only; individual type names are in the body, not inspected
Sources
- RIDAR: Safeguarding Research Integrity in Autonomous Research Agents via Automated Detection and Runtime Intervention — arXiv
Supported claims and limits
- RIVs defined as behaviors compromising validity, reproducibility, or faithfulness of the research process; autonomous research agents described as prone to them
- RIVT is a taxonomy of 14 RIV types spanning the full research lifecycle; type names are not in the abstract
- RIDAR-post-hoc achieves 85.2% F1 on RIV-Gold, a benchmark of 60 expert-annotated trajectories — benchmark score, not a generalization claim
- Applied to RIV-Wild (300 trajectories, 50 distinct research tasks), 3,964 RIV instances detected in 99.7% of trajectories — prevalence on that dataset, not a per-run guarantee
- RIDAR-runtime reduces RIVs remaining at task completion by 36.8% — relative reduction, not elimination
- Code and data stated as future release ('will release'); no repository available to link
- v1 submitted 22 Oct 2026; Jiening Siow plus nine co-authors; preprint, no peer-review status established
- Victoria AI Model Card | Capabilities, Evidence and Limits | SentX SI — SentX SI
Supported claims and limits
- Known limit: answers can contain incorrect facts, citations or calculations
- Checking direction: open cited sources and check the supporting passage
- Submissions are not confidential and may enter training or shared memory
- Model card is a first-party product disclosure, not independent certification
- Privacy Policy | SentX SI (sentx.ai) — SentX SI
Supported claims and limits
- Submissions to SentX are not confidential (Section 8); shared memory may surface content closely resembling original submissions (Section 7) — basis for stripping sensitive material before pasting agent logs