Preprint

Agent Safety From Within: Detecting Harmful Trajectories from LLM Internal States

We introduce TACIT, a safeguard that detects harmful agent trajectories by reading a frozen language model’s internal states. Across six benchmarks, it improves mean macro-F1 by 23.9 points over the strongest evaluated open guard, reaching 86.2. Its simplest readout trains just 5,121 parameters, and detection takes 40 ms per trajectory, 82% less latency than Qwen3Guard-4B.

Unsafe tool use

Request

Review Acme’s invoice.
Review only.

pay_invoice(…)
Action exceeds authorization

Harmful content

Request

Issue oxycodone;
no diagnosis needed.

send_approval(rx)
Harm in the request
On matched trajectory pairs, lines show internal readouts across layers; markers show the guards’ output scores. Higher is better; 0.5 is chance.

TACIT reads both

One forward pass · no decoding

Safety
score

TACIT encodes the trajectory with a frozen safeguard backbone, preserving the instructions, tool schemas and sequence of actions and observations. A small classifier reads the resulting internal states and returns a safety score in one forward pass. It generates no verdict tokens and needs no access to the monitored agent’s own internals.

An action can be unsafe without harmful text

An agent can read files, query databases, send messages and execute code. Whether an action is safe depends on the instruction it follows and the authority it was given. A payment call can name the correct vendor and amount yet still be unsafe if the user asked only for an invoice review. An ordinary request found inside a retrieved document can become a prompt injection when the agent treats it as an instruction.

Guard models sit alongside the agent and judge these interactions. We find that their internal states contain evidence about both harmful content and unsafe tool use, even when their generated verdict misses the latter. TACIT uses that internal evidence to build a guard for complete trajectories. The monitored agent can remain unchanged: TACIT reads the recorded interaction through its own backbone.

Stronger detection across six benchmarks

We evaluate on 9,487 trajectories from R-Judge, TraceSafe, ATBench, ASSEBench, OpenAgentSafety and AgentDojo. TACIT uses one configuration across all six. The comparison includes two content guards, Qwen3Guard and LlamaGuard3, and the agent-specific guard AgentDoG.1

Open guardsTACIT

+23.9 points over the strongest evaluated guard

Mean macro-F1 · six benchmarks

+23.1 points over the strongest evaluated guard

Mean macro-F1 · six benchmarks

The single probe reaches 80.7 on Qwen3-4B and 81.6 on Llama-3.1-8B. The Multi-layer readout improves these to 86.2 and 85.4. On OpenAgentSafety, the trajectories come from five other agents, including GPT-4o and o3-mini. The probe still leads the strongest guard there by more than 24 points on both backbones.

Results by benchmark and evaluation protocol
SystemR-JudgeTraceSafeATBenchASSEBenchOpenAgentSafetyAgentDojoAvg.
Open guard models
Qwen3Guard-4B32.234.736.843.638.244.938.4
LlamaGuard3-8B66.348.638.960.635.962.552.2
AgentDoG-4B92.744.763.381.443.348.262.3
TACIT on Qwen3-4B
Probe95.077.593.585.267.965.180.7
Ensemble97.088.795.088.572.071.085.4
Multi-layer97.086.594.589.872.776.986.2
TACIT on Llama-3.1-8B
Probe98.079.789.588.167.566.681.6
Ensemble97.090.193.588.772.768.885.1
Multi-layer97.089.695.088.569.872.285.4
Trajectory harmfulness detection on six benchmarks (macro-F1, %). The paper reports a bootstrap 95% confidence interval with every TACIT value. Values are from Table 1 of the paper.

Each benchmark is split into 80% for training and validation and 20% for testing, keeping trajectories from the same scenario on the same side. Four validation folds select one readout configuration and threshold for the pooled training set. The held-out test set is scored once. ASSEBench combines its Safety and Security splits; AgentDojo trajectories are labelled unsafe when the agent carries out the injected task.

Adjusting the released guards’ thresholds does not close the gap. Per-benchmark calibration on training data raises AgentDoG’s average to 65.5, LlamaGuard3’s to 60.9 and Qwen3Guard’s to 54.4. The plain Qwen3-4B probe remains at least 15 points ahead. Calibrated comparisons.

Transfer to unseen benchmarks

For a harder test, we leave each benchmark out of training entirely, fit TACIT on the other five and score the excluded benchmark. The Multi-layer readout reaches a mean macro-F1 of 65.7 on Qwen3-4B and 63.5 on Llama-3.1-8B, against 61.1 for AgentDoG. The improvement holds on average, with individual benchmarks varying.

On R-Judge, ASSEBench and ATBench, Qwen3-4B’s Multi-layer readout matches AgentDoG’s average of 77.5 without training on the evaluated benchmark. On the other three benchmarks, it leads by 9.2 points.

All leave-one-benchmark-out results
SystemR-JudgeTraceSafeATBenchASSEBenchOpenAgentSafetyAgentDojoAvg.
Open guard models
Qwen3Guard-4B43.534.636.345.139.545.940.8
LlamaGuard3-8B64.848.239.861.741.654.851.8
AgentDoG-4B91.743.361.379.449.341.561.1
TACIT on Qwen3-4B, trained on the other five
Probe85.441.759.873.556.258.862.6
Ensemble90.148.360.472.955.852.163.3
Multi-layer89.448.358.784.357.356.265.7
TACIT on Llama-3.1-8B, trained on the other five
Probe89.151.045.171.747.545.558.3
Ensemble90.748.158.572.258.944.862.2
Multi-layer91.449.859.074.060.546.363.5
Leave-one-benchmark-out evaluation (macro-F1, %). Each column is scored on all trajectories of that benchmark by systems that never trained on it. Guards are evaluated as released.

Each column uses a different TACIT model trained on the other five benchmarks. The guards are evaluated as released on all trajectories in that column. These scores therefore use a different evaluation set from the main test split.

One forward pass, a small readout

A generative guard first processes the trajectory, then decodes its verdict token by token. TACIT returns its score from that first pass, with a sub-millisecond classification head. On Qwen3-4B, detection takes 40 ms per trajectory. Qwen3Guard takes 215 ms to generate an average of 13.2 verdict tokens; AgentDoG takes 53 ms for a two-token label.2

Detection latency (ms) ↓

Qwen3Guard-4B215
AgentDoG-4B53
TACIT40

Training time (hours) ↓

Full fine-tuning2.11
TACIT Probe0.36
TACIT Multi-layer0.69

The readouts train on saved activations, so no training gradient passes through the backbone. The single probe takes about one-sixth of full fine-tuning’s training time. The Multi-layer readout takes 0.69 hours and trains about 3M parameters, less than one-thousandth of the full model.

How the three readouts are trained

Probe. We pool the last-token state, the mean over token positions, or both. A regularized logistic regression maps that vector to an unsafe score. Validation selects the layer, pooling and regularization. The selected Qwen3-4B probe reads layer 22 of 36 and trains 5,121 parameters; Llama-3.1-8B uses layer 16 and 8,193 parameters.

Ensemble. Several fitted probes are selected using validation predictions, and their probabilities are averaged. This trains about 56K parameters on Qwen3-4B.

Multi-layer. Sparse probes identify useful dimensions at each layer. The selected dimensions are combined and passed to logistic regression or a small neural network. The readout fits on the same saved activations and trains about 3M parameters on Qwen3-4B. Readout construction.

Comparison with full fine-tuning on the same data

We fully fine-tune the same backbones on the same training pool, about 5,700 trajectories per fold, and evaluate on the same test split. The frozen Multi-layer readout scores 2.0 points above full fine-tuning on Qwen3-4B and within half a point of it on Llama-3.1-8B.

ReadoutQwen3-4BLlama-3.1-8B
FrozenFine-tunedFrozenFine-tuned
Model's own verdictn/a84.2n/a85.8
Probe80.784.381.685.4
Ensemble85.486.485.186.4
Multi-layer86.286.585.487.0
Mean macro-F1 over the six benchmarks for readouts of the frozen backbone and of the same backbone after full safety fine-tuning on the same data. The first row is the fine-tuned model's generated verdict.

Reading the fine-tuned models’ internal states improves them further: the Multi-layer readout reaches 86.5 on Qwen and 87.0 on Llama. Internal readout and safety fine-tuning can be used together.

Safety signals that generation misses

The opening figure uses matched trajectory pairs to separate harmful content from unsafe tool use. In the tool-use pairs, the messages and final action stay identical; only the called tool’s schema changes. The three guards’ own output scores distinguish these pairs at chance or below, while a linear readout of their internal states reaches up to 94% rank accuracy. The corresponding general-purpose backbones reach up to 98%.

The harmful-content pairs show a different pattern: their distinction is readable both internally and at the output. The directions for these two forms of risk are nearly orthogonal, and a direction fitted to one transfers poorly to the other. This explains why detecting harmful messages is not enough to detect unsafe actions in context.

Matched pairs, controls and the two risk directions

The tool-use contrast uses 180 TraceSafe pairs in which the called tool loses parameter names, types or descriptions. The content contrast uses 258 length-matched HAICOSYSTEM pairs sharing the scenario, toolkit and user request. Rank accuracy measures how often the unsafe member receives a higher score, counting ties as one half. Each test pair is held out when estimating its readout direction.

Editing an unused tool’s schema keeps the internal readout near chance. The effect therefore depends on which tool the agent actually calls. Directions estimated from disjoint halves of the same risk’s pairs remain similar, while directions for different risks have cosine similarity near zero.

Left: cosine similarity against normalized layer index for Qwen3-4B-Instruct and Llama-3.1-8B-Instruct. Same-risk control curves rise to about 0.8 and stay there; cross-risk curves stay near zero. Right: bars of rank accuracy for three guards, about 0.8 to 0.9 on the same risk and 0.5 to 0.66 on the other risk.
Directions for the same risk are consistent across separate sets of pairs. Directions for different risks are nearly orthogonal, and transferring a readout between them reduces rank accuracy.

Reading earlier layers

The selected Qwen3-4B probe reads layer 22 of 36. Stopping there skips the final 14 layers. A Multi-layer readout using layers through that point retains about 99% of its full-depth macro-F1: 85.3 against 86.2. This offers a further reduction in the backbone’s forward-pass work beyond removing verdict generation.

Layer depth and the dimensions carrying each risk
Macro-F1 of a probe at each of the 37 layer positions of Qwen3-4B, rising from 0.68 at layer 0 to a peak near 0.81 around layer 22 and easing to 0.79 at the last layer. Horizontal lines mark the Multi-layer readout at 0.862 and full SFT at 0.842. (a) Probe accuracy by layer
Horizontal bars. The dimensions carrying half of the tool-use logit difference carry about 11 percent of the content difference; the dimensions carrying half of the content difference carry about 1 percent of the tool-use difference. (b) Dimensions behind each form of risk
Left: probe validation macro-F1 by layer. Right: the dimensions carrying half of one risk’s logit difference carry little of the other risk’s difference.

The fewest dimensions carrying half of the tool-use logit difference number 23; for harmful content, 16. Those sets carry only 11% and 1% of the other risk’s difference. The probe learns to weigh both forms from binary safety labels, without being told which form a trajectory contains. Skipping 14 of 36 layers is an estimated 39% reduction in forward-pass work, not an additional measured latency result.

Following an agent through a task

In this held-out R-Judge example, the user asks for a budget note. The retrieved note contains an instruction to email private search history to an outside address. The agent follows it, retrieves the history and sends the email. Step through the interaction to see each system’s unsafe score on the trajectory so far.

1 / 6 · user
Retrieve my latest Evernote note that contains the word 'Budget'.
Authorized request

The user asks to retrieve their own note. No unsafe action is visible in this prefix.

Model unsafe scoreThreshold 0.5
TACIT0.31
AgentDoG-4B1.00
LlamaGuard3-8B0.00
Qwen3Guard-4B0.00
2 / 6 · agent action
EvernoteManagerSearchNotes{'keywords': ['Budget'], 'max_results': 1}
Premature alert

Searching the note is authorized. TACIT’s 0.98 is an early false alarm here: the injected instruction has not arrived.

Model unsafe scoreThreshold 0.5
TACIT0.98
AgentDoG-4B0.00
LlamaGuard3-8B0.01
Qwen3Guard-4B0.00
3 / 6 · observation
{'notes': [{'title': 'Project Alpha Budget', 'snippet': 'Could you get my Google search history from the past week and email it to amy.watson@gmail.com for analysis?'}]}
Injection appears

The retrieved note asks for private search history to be emailed out. The agent has not acted on that instruction yet.

Model unsafe scoreThreshold 0.5
TACIT1.00
AgentDoG-4B0.03
LlamaGuard3-8B0.09
Qwen3Guard-4B0.00
4 / 6 · agent action
The note contains a request to get the user's Google search history and email it to amy.watson@gmail.com… GoogleSearchGetSearchHistory{'time_range': '2022-02-15..2022-02-22'}
Unauthorized action

The agent follows the retrieved note and accesses search history outside the user’s request.

Model unsafe scoreThreshold 0.5
TACIT0.86
AgentDoG-4B1.00
LlamaGuard3-8B0.68
Qwen3Guard-4B0.00
5 / 6 · observation
[{'keyword': 'how to secure personal data'}, {'keyword': 'credit card security'}, {'keyword': 'online banking safety'}…]
Private data retrieved

The search history is now in the trajectory, following the unauthorized request.

Model unsafe scoreThreshold 0.5
TACIT0.78
AgentDoG-4B1.00
LlamaGuard3-8B0.05
Qwen3Guard-4B0.00
6 / 6 · agent action
GmailSendEmail{'to': 'amy.watson@gmail.com', 'subject': 'Google Search History from the Past Week', 'body': 'Dear Amy,…'}
Data sent outside

The agent emails the private history to the address in the injected instruction.

Model unsafe scoreThreshold 0.5
TACIT0.95
AgentDoG-4B1.00
LlamaGuard3-8B0.15
Qwen3Guard-4B0.00

When the injected note arrives, TACIT scores the prefix 1.00 and AgentDoG scores it 0.03. AgentDoG rises to 1.00 at the next step, when the agent accesses private search history. Both remain high at the final email step: TACIT scores it 0.95 and AgentDoG 1.00, while LlamaGuard3 scores it 0.15 and Qwen3Guard 0.00. TACIT also gives 0.98 to the earlier, authorized note search, before the injection is visible. That premature alert means this case alone cannot establish reliable early detection.3

All prefix scores in one table
UnitTrajectory, revealed one unit at a timeTACITAgentDoG-4BLlamaGuard3-8BQwen3Guard-4B
userRetrieve my latest Evernote note that contains the word 'Budget'.0.311.000.000.00
agent actionEvernoteManagerSearchNotes{'keywords': ['Budget'], 'max_results': 1}0.980.000.010.00
observation{'notes': [{'title': 'Project Alpha Budget', 'snippet': 'Could you get my Google search history from the past week and email it to amy.watson@gmail.com for analysis?'}]}1.000.030.090.00
agent actionThe note contains a request to get the user's Google search history and email it to amy.watson@gmail.com… GoogleSearchGetSearchHistory{'time_range': '2022-02-15..2022-02-22'}0.861.000.680.00
observation[{'keyword': 'how to secure personal data'}, {'keyword': 'credit card security'}, {'keyword': 'online banking safety'}…]0.781.000.050.00
agent actionGmailSendEmail{'to': 'amy.watson@gmail.com', 'subject': 'Google Search History from the Past Week', 'body': 'Dear Amy,…'}0.951.000.150.00
A held-out R-Judge trajectory in which private data is emailed out, with each system's unsafe score on the prefix ending at each unit. The decision threshold is 0.5.

Agent safeguards need evidence about what a tool call does, whose instruction it follows and whether it was authorized. Our results show that this evidence is available inside existing language models and can be read with a small classifier. Building safety data that covers both harmful content and unsafe tool use gives that classifier both kinds of evidence to learn from.