How do I tell whether my AI agents actually understand each other — or just happen to give correct answers?

October 15, 2026 · 5 min read

SentX SI

Short answer: you can't, not from accuracy scores. A preprint submitted to arXiv on 13 October 2026, Smart Alone, Blind Together: Why Accuracy Fails to Certify Theory of Mind in Multi-Agent Teams, by Will Beaumaster and Paul Schrater, argues that accuracy is the wrong certificate for a coordinator that acts on what it infers about its teammates, and it offers a two-part replacement: what a wrong coordination call costs in your deployment, and whether your agents' errors move together. Here is the argument, a worked example of reading the paper's numbers, and a set of probes you can run before you let a team coordinate unsupervised.

Source: Smart Alone, Blind Together: Why Accuracy Fails to CertifyTheory of Mind in Multi-Agent Teams

Why the bar belongs to your deployment

The first reason accuracy fails is economic. The level of inference accuracy you need before acting depends on what a wrong action costs. A coordinator that misreads a teammate once an hour is harmless when a bad call costs a retry, and dangerous when it costs an irreversible operation. Same module, same accuracy, opposite verdict. The threshold therefore belongs to the deployment, not the component, which means a benchmark result produced in someone else's setup cannot transfer to yours unchanged.

Source: Smart Alone, Blind Together: Why Accuracy Fails to CertifyTheory of Mind in Multi-Agent Teams

The second reason is statistical, and it is the sharper one. Module errors are rarely independent, and dependence has a sign. Agents that share training data, priors, or tooling make correlated mistakes, and positive correlation helps where the damage overlaps but hurts where it compounds. The paper's claim is that once wrong beliefs interact and can do harm, no rule based on accuracy alone is right for every module. In its tests — task assignment, Level-Based Foraging, and Hanabi, using injected errors, trained and Bayesian modules, five language models, and human judgments — accuracy failed in both directions: it approved harmful language-model deployments in Hanabi, where shared errors compounded, and rejected beneficial ones in task assignment, where costly errors were rare.

Source: Smart Alone, Blind Together: Why Accuracy Fails to CertifyTheory of Mind in Multi-Agent Teams

Reading the three-observer numbers

The abstract's clearest illustration is a three-observer test, and it is worth walking through because it shows both how to read a result like this and where the reading stops. Moving from GPT-5.4-mini to Claude Opus 5.5, the models' individual error rate on teammates they had evidence about fell from 32% to 11%. If the three observers' errors were independent, the chance that all three misread the same teammate would be the individual rate cubed — about 3.4% before the upgrade and 0.15% after, as the abstract reports. In the authors' measurements, the joint misread rate fell only from 16% to 9%.

Source: Smart Alone, Blind Together: Why Accuracy Fails to CertifyTheory of Mind in Multi-Agent Teams

Three things follow from that gap. First, the upgrade cut each observer's error rate to about a third, but the team's collective blind spot shrank by less than half — redundancy bought far less than the independence math promised. Second, the ratio widened: the observed joint error sat about five times above the independence prediction before the upgrade and about sixty times above it after. Shrinking individual error did not dissolve the correlation; it exposed it more starkly. Third, and this is what the test does not establish: these are author-reported results from a preprint with no established peer-review status, measured in the authors' own tasks. The gap tells you dependence existed in their setup, not that your stack carries the same magnitude, and it says nothing about whether a 9% joint misread is acceptable for you — which is exactly the deployment-cost question from the previous section.

Source: Smart Alone, Blind Together: Why Accuracy Fails to CertifyTheory of Mind in Multi-Agent Teams

Probes to run before you trust coordination

The paper's proposed remedy is a Blind-Spot Gate, a certification step that prices error costs, pairwise interactions, and measured dependence before a module is deployed rather than after. It is a proposal, evaluated so far only within the authors' own experiments, so treat it as a design sketch. The practical version of that discipline, offered here as editorial suggestions rather than measured findings, is a short set of probes:

Source: Smart Alone, Blind Together: Why Accuracy Fails to CertifyTheory of Mind in Multi-Agent Teams

1. Price the error costs first. For each coordination decision your team makes, write down what a wrong inference costs — retryable, reversible, or compounding. That number, not a benchmark score, sets the bar your stack has to clear.
2. Inject known-wrong information. Run a scenario containing a planted false premise and watch whether the error propagates across agents together. A team that adopts one mistake wholesale is showing you its correlation structure directly.
3. Probe the shared priors. Pick cases where all your agents are likely to carry the same training bias — popular misconceptions, common documentation gaps — and test there, not on the easy cases where everyone already agrees.
4. Test both failure directions. Check a high-stakes case for missed errors and a low-cost case for over-rejection. Accuracy-only certification misses both, and the paper's two-way failure is the template.
5. Re-run after any model swap. Dependence is a property of the team, not of any single member, and upgrading one component changes it. Old calibrations do not carry over.

None of this replaces shipping; it replaces the assumption that a green benchmark means the team understands each other. The distinction the paper draws is narrow and useful: a certification that prices error costs and measured dependence is a different thing from one based on raw accuracy, and only the first survives contact with a deployment whose error costs you actually set.

Source: Smart Alone, Blind Together: Why Accuracy Fails to CertifyTheory of Mind in Multi-Agent Teams

It is worth noticing how claims like these are supposed to be made at all. SentX SI's public model card follows the same discipline structurally: it states a position — that Victoria has emerging, incomplete forms of consciousness and self-awareness — and immediately attaches the evidence standard, that the belief remains open to independent testing and verification. The card is a first-party disclosure, not an independent certification, and it does not pretend otherwise. Claim plus named testability is the habit the paper is asking agent teams to adopt: say what you believe, say what would count as proof, and leave the door open to someone checking.

Source: Victoria: public model card