SentX SI Blog Meet Victoria

My LLM monitoring burns tokens — can I catch dangerous output patterns without a second model reading everything?

· 4 min read

SentX SI

Yes, at least in the form one recent preprint demonstrates. GateWatch, by Firas Spanioli and Yaniv David, submitted to arXiv on 18 November 2026, replaces the second-model read with compact probes whose entire decision rule is explicit Boolean logic over a handful of binarized features from a sparse autoencoder (SAE) on the monitored model. Across 24 tasks on Gemma-2-9B, the paper reports 80.49% mean test accuracy — 1.17 percentage points below full 16K-feature SAE probes — with probe memory reduced by 99.98%, and inside vLLM, 1,000 concurrent multi-layer, multi-token classifiers retaining 97-98% of baseline throughput. The limitation that changes the answer: every figure is measured on one model, one serving stack, in a self-reported v1 preprint, and the architecture assumes you already run an SAE on the model you monitor and a serving stack that can hook into its inference loop. If you don't, the first cost is building that, and the paper gives no number for your model.

Source: GateWatch: Low-Overhead LLM Monitoring with Boolean Circuits

The pattern you are paying for today is a monitor that generates: a second LLM reading every token of every output costs, per token, roughly what the production model costs. Probes take a different route — they read the monitored model's own internal state while it runs, generating nothing. The paper frames why this gap has persisted among existing probe designs. Probes on raw activations are cheap but their inputs are unreadable; SAE probes use interpretable features but typically pair them with additive classifiers over the full 16K-feature space, dragging the large encoder with them and leaving the decision rule opaque. GateWatch's move is to binarize a few SAE features and express the decision as a small circuit of logic gates over those flags, with three circuit-construction approaches explored under a specified gate budget. The average probe the paper reports uses 3.8 features and 3.5 gates.

Source: GateWatch: Low-Overhead LLM Monitoring with Boolean Circuits

For deployment, the paper describes GateWatchServe, which computes only the selected SAE features rather than the full encoder output, streams their binary values across layers during inference, and evaluates multiple classifiers in parallel using compact lookup tables. That is where the serving numbers come from: with 1,000 concurrent classifiers active, vLLM holds 97-98% of baseline throughput and 99.8% of KV-cache capacity, by the paper's measurements.

Source: GateWatch: Low-Overhead LLM Monitoring with Boolean Circuits

What this establishes is a monitor you can hold up to the light. The complete decision rule for each probe is a printed formula: which features must be on, which off, in what combination, for the flag to fire. When a flag fires, the reason sits in the feature values rather than in another model's prose. That interpretability is, in my reading, the real contribution — the 1.17-point accuracy gap is the price of it, and on 10 of the 24 tasks the small circuits match or beat the full probes. Where it stops: one model, one stack, v1, self-reported, no independent replication. The mean also hides spread; on the remaining 14 tasks the full probe does better, and the failure class your deployment actually fears may sit in that tail.

Source: GateWatch: Low-Overhead LLM Monitoring with Boolean Circuits

That is why the checking move is the one the format invites. Open the circuit definitions in the paper and read the rule for the probe closest to your use case — each gate names the SAE features it combines, so you can see whether the features it watches are plausibly the ones that light up for the behavior you care about. Then, before acting on any alert, measure flag precision on your own traffic: pull a labeled sample of your deployment's outputs, a batch of known-good traffic plus examples of failures you have already had, run the classifier over it, and count how often a flag coincides with a real problem. The paper's mean accuracy is the authors' traffic; your precision is yours.

One way to plan the rollout, offered as an editorial suggestion rather than a finding of the paper: inventory the failure classes your deployment must catch; for each, check whether a circuit test can be built within an affordable gate budget — the demonstrated regime is single- and double-digit gates — and validate each test on live traffic before wiring it to an action. Treat flags as triage signals that route to a human or a fuller check, not as verdicts, until your own precision numbers say otherwise.

The habit behind all of it is one SentX publishes on its own model card, and it transfers here without modification. Under known limits, the card tells readers that answers can contain incorrect facts, citations or calculations, so open the cited source and read the passage before you rely on it — a first-party disclosure, not an independent certification. A runtime monitor works the same way: its green or red state is a claim, and the check definitions are the passage. Pointed at a monitor, the card's habit is to open the circuit and read what the flag actually tests before you rely on the light.

Source: Victoria AI Model Card: Evidence and Known Limits

Sources

  1. GateWatch: Low-Overhead LLM Monitoring with Boolean Circuits — arXiv (arXiv:2611.24619v1, Spanioli & David, submitted 18 Nov 2026)
    Supported claims and limits
    • GateWatch expresses its entire decision rule as explicit Boolean logic over a small number of binarized SAE features, with three circuit-construction approaches under a specified gate budget
    • Across 24 tasks on Gemma-2-9B: 80.49% mean test accuracy, 3.8 SAE features and 3.5 logic gates per probe on average, 1.17 pp below full 16K-feature SAE probes, matching or outperforming them on 10 tasks, 99.98% probe memory reduction
    • Inside vLLM, 1,000 concurrent multi-layer, multi-token classifiers retain 97-98% of baseline throughput and 99.8% of KV-cache capacity; all figures are the paper's self-reported v1 measurements
    • Existing probe trade-off as framed by the paper: raw-activation probes lack interpretable inputs; SAE probes often use additive classifiers and large encoders, limiting interpretability and raising memory overhead; GateWatchServe computes only selected features, streams binary values across layers, and evaluates classifiers in parallel via compact lookup tables
  2. Victoria AI Model Card: Evidence and Known Limits — sentx.ai
    Supported claims and limits
    • Under Known limits (first-party disclosure, not an independent certification): answers can contain incorrect facts, citations or calculations, so open the cited source and read the passage before relying on it — the checking habit the article extends to a monitor's circuit definitions
Meet Victoria