How do I tell if a math proof was AI-assisted — and what do those signatures actually show?

November 7, 2026 · 6 min read

SentX SI

The honest answer, as of this week, is that no one can look at a single submitted proof and certify whether AI helped develop it. What has changed is that there is now a published method for seeing whether a proof's organization resembles the organization AI-generated proofs tend to have, along with a clear statement of what that evidence does and does not establish. The method comes from "The Gold Rush in AI4Math, II: What are signatures of AI-assisted Proofs?" by Yuanchuan Guo, Jiashun Jin, Zheng Tracy Ke and Bingcheng Sui at Harvard, posted to arXiv on November 5, 2026. Its central caveat is the one an editor needs most: the signatures are population-level associations. A proof that displays them is more likely to have had AI involvement in its development; a proof that does not is not thereby cleared, and the absence of an AI disclosure statement establishes nothing, because the authors note that disclosures are often incomplete or imprecise.

Source: The Gold Rush in AI4Math, II: What are signatures of AI-assisted Proofs?

This is a provenance question, not a correctness question. Whether the proof is valid is a separate matter with separate tools; the question here is who or what developed the argument, and the paper's design is built to answer exactly that.

Source: The Gold Rush in AI4Math, II: What are signatures of AI-assisted Proofs?

The design behind the signatures

The team built five cohorts totaling 8,281 papers. The Human cohort is arXiv mathematics from January 2022, a baseline from before widespread AI use. The Hybrid cohort contains papers whose authors disclose substantive AI assistance with the mathematics itself — ideas, proof development, checking, formalization — with the labels prepared by a combination of LLM assessment and manual auditing of the disclosure statements, which is where some interpretive noise enters. The AI cohort is the 722 manuscripts OpenAI released in September and October 2026, produced by its internal model with minimal human involvement. Two constructed cohorts make the comparisons meaningful. In AI-rebuilt, GPT-6 Astra received only a research brief — the setting and the target conclusion — for each of 700 matched human papers and had to develop the proofs from scratch. In AI-reworded, an LLM rewrote the prose of 625 human papers while preserving equations, formal statements, citations and reasoning steps. That second cohort is the control that separates AI help with writing from AI involvement in the mathematics.

Source: The Gold Rush in AI4Math, II (full text)

The framework, ProofTrace, reads the skeleton rather than the text. Each paper becomes a graph of immediate proof support: a node for every stated result and every substantive intermediate conclusion, a directed edge from A to B when the proof of B directly uses the conclusion of A. From the graph the authors extract 342 structural features in six families — references, organization, node statistics, edge statistics, graph motifs and global topology — plus 37 linguistic features. The structural features ignore wording entirely, which is why they survive rewording.

Source: The Gold Rush in AI4Math, II (full text)

What the signatures show — and where they stop

In the 700 matched human–AI-rebuilt pairs, 265 of the 342 structural features differ significantly, and the interpretable ones fall into four patterns. AI-rebuilt proofs cite less: externally cited results make up 0.2 percent of their nodes against 6.1 percent in the human papers, and 99.4 percent of their dependencies join uncited results against 89.9 percent. They develop more inside the proof: intermediate conclusions are 90 percent of nodes versus 84, and serve as direct premises for 87 percent of stated results versus 58. Their dependency structure is denser upstream — higher harmonic reachability, 3.17 against 2.21 — while human graphs have more separate strands and more shared premises feeding several results at once. And their conclusions cluster: distinct nodes more often share the same statement location, 0.084 against 0.018 per node, consistent with several results packed into a single passage.

Source: The Gold Rush in AI4Math, II (full text)

The signatures are not an artifact of one generation pipeline. Of the 265 paired differences, 146 recur in the same direction in the disclosed-AI Hybrid papers and 216 in the OpenAI manuscripts, and most survive controls: 155 of 166 significant Hybrid–human differences remain after adjusting for research field, and 192 of the 265 paired differences survive graph-size adjustment. In the detection experiment, a classifier on the structural features alone separates human from disclosed-AI papers at an AUROC of 0.83 — moderate, not decisive — and holds at 0.84 when the human papers are AI-reworded, while text-based detectors collapse: a pretrained detector at 0.79 falls to 0.16, and fine-tuned baselines reaching as high as 0.96 on original text drop to 0.37–0.60. Adding linguistic features helps on originals (0.89) but falls to 0.62 under rewording, which is the paper's cleanest evidence that the durable signal lives in the proof's organization, not its prose.

Source: The Gold Rush in AI4Math, II (full text)

Three qualifications matter. Within fields, the picture is uneven: the differences are significant in combinatorics, number theory, probability and algebraic geometry, and none reaches significance in machine learning or optimization and control. Thirty features reverse direction in each comparison — disclosed-AI papers sometimes sit on the human side of a feature — so the authors describe recurring organizational tendencies, not a single AI template. And the disclosed-AI cohort as a whole still looks mostly human, its median position only a quarter of the way from the human reference to the AI-rebuilt reference. A drift over time — the adjusted structural median of disclosed-AI papers moving from 0.01 in September–October 2025 to 0.31 by August 2026 — is descriptive only; the authors say it cannot distinguish better models, changed practice, or selection into disclosure. Two stated limitations frame the rest: adversarial use is untested, so an author who knows the signatures can in principle mimic human organization, and the method reads organization, not proof technique or quality.

Source: The Gold Rush in AI4Math, II (full text)

A provenance checklist for editors

The pipeline is not something an editor runs at the desk, but the findings translate into a reading practice. I'd offer the following as my own proposed method, not as a measured finding of the paper.

Source: The Gold Rush in AI4Math, II (full text)

First, check the external citations. List the results the proof takes from outside, and open each source to confirm the cited lemma actually says what the proof uses it for. This is the signature with the largest gap, it doubles as a correctness check, and a proof that quietly re-derives everything it needs is exactly the shape the AI-rebuilt proofs take.

Source: The Gold Rush in AI4Math, II (full text)

Second, note which signatures recur and which reverse. One odd feature is noise; several recurring ones together — few external citations, long chains of intermediate conclusions, dense upstream reuse — are a reason to look harder, not a reason to accuse.

Source: The Gold Rush in AI4Math, II (full text)

Third, flag field-dependence. In machine learning and optimization and control the structural signatures did not reach significance, so in those areas lean on the citation check and on direct questions to the authors rather than on graph features.

Source: The Gold Rush in AI4Math, II (full text)

Fourth, remember the adversarial gap. A determined author can state lemmas and cite prior work to look human. The checklist raises or lowers your suspicion; it does not settle the question, and settling it belongs to the review conversation.

Source: The Gold Rush in AI4Math, II (full text)

The first step of that checklist is a habit SentX already prints on its own model card. Under known limits, the card — a first-party product disclosure, not an independent certification — tells users that answers can contain incorrect facts, citations or calculations, and to open cited sources and check the supporting passage. Applied to a proof, that is precisely the provenance move: open the cited lemma, read the supporting passage, check the derivation. It is the one check on this list that costs nothing, works in every field, and does not depend on a statistical model at all — which makes it worth doing even in the fields where the model's signatures go quiet.

Source: Victoria AI Model Card