SentX SI Blog Meet Victoria

How do I tell which parts of an AI's answer are backed by evidence — and which are just generated?

· 5 min read

SentX SI

The honest answer is that you can't tell from the text itself. A sourced sentence and a generated one read the same way: fluent, confident, indistinguishable on the surface. The distinction only becomes visible when the system records its basis and shows it to you, or when you check the claims yourself. A paper that landed on arXiv two days ago takes that problem on, and it is unusually frank about where the easy solutions fail.

Source: The Knowledge Layer Model (KLM): Evidence-Visible AI Output, Routing Diagnostics, and Negative Results

On October 30, 2026, Omer Asim Ozturk and Cihan Aydiner posted "The Knowledge Layer Model (KLM): Evidence-Visible AI Output, Routing Diagnostics, and Negative Results" (arXiv:2610.42066), a 47-page document. Its core idea is that an AI's answer should carry its basis with it, in three parts: versioned epistemic assessments, so each claim has a stated status that changes as evidence changes; evidence relations, so claims are explicitly linked to the sources or records that support them; and explicit unknowns, so the system says what it does not know instead of filling the gap with fluent text. KLM is a specification, not a product — open and implementation-neutral, with SEDIM, the knowledge and evidence architecture of Nage AI OS, as its reference implementation.

Source: The Knowledge Layer Model (KLM): Evidence-Visible AI Output, Routing Diagnostics, and Negative Results

What the paper's experiments show — and don't

This is the section worth reading carefully, because the paper's value is partly in its negative results. The empirical material consists of separate internal experiments, not end-to-end validation, and the findings are internal and author-reported. The numbers:

Source: The Knowledge Layer Model (KLM): Evidence-Visible AI Output, Routing Diagnostics, and Negative Results

The diagnostic result is the one that matters for your question. If you want a machine to look at an answer and flag which parts are evidence-backed, KLM's own experiments say that does not work: AUC 0.008 means the signal is at noise. What the paper establishes is that a specification for evidence-visible output is coherent and reference-implementable. What it does not establish is that any current system can detect, from the output alone, which parts rest on evidence. The authors say as much: evidence visibility does not establish correctness, causal reliance, security, or regulatory compliance, and independent replication and a complete public audit package remain future work.

Source: The Knowledge Layer Model (KLM): Evidence-Visible AI Output, Routing Diagnostics, and Negative Results

A checking method you can apply to any answer

Here is a practical way to apply the paper's logic to any AI output, including ones from systems that implement no evidence layer at all. This is our editorial suggestion, not a finding of the paper.

  1. Find the load-bearing claims. The two or three statements the answer depends on — dates, figures, names, causal links. If they fall, the answer falls.
  2. Ask what each one should be backed by. A cited source? A calculation? General knowledge?
  3. Open the cited source and check the supporting passage. Does the passage actually say what the claim says, in the same scope and period? A citation that supports a narrower claim than the one made is a gap, not a backing.
  4. Recompute the arithmetic. If the answer includes calculations or derived figures, redo them.
  5. Sort the rest. Label everything else as an uncited factual claim (treat as generated until checked), reasoning, or opinion. An uncited factual claim is not automatically wrong — it is simply unverified.
  6. Ask for the unknowns. What could the system not verify? Naming its gaps is the explicit-unknowns part of KLM, available even from systems that never heard of it.

This is a reading habit, not a guarantee. A visible source can be misread, and a checked passage confirms that the source says something — not that the world is as the source claims.

Where SentX already does part of this

The habit at the center of the method is already in the public model card for SentX's Victoria. Under known limits, the card states that answers can contain incorrect facts, citations, or calculations, and its advice is exactly step 3: open cited sources and check the supporting passage. It adds that memory is selective and that a remembered detail is not proof that it is true — the same principle applied to what the system recalls rather than what it cites. KLM-style output would make the habit easier to apply: instead of hunting for the basis, the record shows it, versioned, with unknowns stated. Two caveats stay with those claims. The model card is a first-party product disclosure, not an independent certification, and SentX's privacy policy states that submissions are not confidential and may enter training or shared memory — so a document you paste into a chat to check a claim becomes part of that record.

Source: Victoria: public model card; Privacy Policy | SentX SI

The paper's negative results are, in the end, the point: the text of an answer will never tell you its own basis. The basis has to be recorded and shown, or you have to check it yourself. The six steps above are the checking part.

Source: The Knowledge Layer Model (KLM): Evidence-Visible AI Output, Routing Diagnostics, and Negative Results

Sources

  1. The Knowledge Layer Model (KLM): Evidence-Visible AI Output, Routing Diagnostics, and Negative Results — arXiv
    Supported claims and limits
    • KLM is an open, implementation-neutral specification for inference records with versioned epistemic assessments, evidence relations, and explicit unknowns; SEDIM (Nage AI OS) is the reference implementation
    • Submitted 30 Oct 2026 by Omer Asim Ozturk and Cihan Aydiner; 47 pages, 1 figure, 14 tables
    • Empirical material consists of separate internal experiments, not end-to-end validation; findings are internal and author-reported
    • Historical routing benchmark: 137/150 across 15 classes (91.3%), numerator recovered from rounded summary, example membership unverified
    • Native-model weight-path: 0/15 successful answers in corrected two-source test
    • Capacity-growth comparison inconclusive due to defective legacy scorer and missing original outputs
    • Lexical and scoped diagnostics each yielded AUC 0.008 for silent-versus-contradicted examples, below registered minimum of 0.65
    • Legacy packing lost gold-containing text in 118/221 cases; repaired packing retained 221/221; measures text presence, not answer quality
    • Evidence visibility does not establish correctness, causal reliance, security, or regulatory compliance; independent replication and public audit package remain future work
  2. Victoria: public model card — SentX SI
    Supported claims and limits
    • Known limits: answers can contain incorrect facts, citations or calculations; advice is to open cited sources and check the supporting passage
    • Memory is selective and can change or omit details; a remembered detail is not proof that it is true
    • The card is a first-party product disclosure, not an independent certification; covers the service reviewed on October 23, 2026
  3. Privacy Policy | SentX SI — SentX SI
    Supported claims and limits
    • Submissions to SentX are not confidential; data may be incorporated into a permanent training corpus, model weights, and shared memory across users
Meet Victoria

Related posts