Why Filtering Fails

There is a version of this chapter that would be easy to write and worthless to read. It would quote a vendor's marketing page, find the weakest claim, and knock it over. Detection vendors are doing serious work on a hard problem, and several of them are doing it well.

So start with the strongest published result, not the weakest.

The best case for filtering

PromptArmor prompts an off-the-shelf language model to find and strip injected instructions from input before the agent ever processes it. On the AgentDojo benchmark, using GPT-4o, GPT-4.1 or o4-mini, it reports a false positive rate and a false negative rate both below 1%, and attack success falling below 1% once the injected prompts are removed (Zhan et al., 2025).

Those are good numbers. They are not a rounding error or a cherry-picked configuration, and anyone arguing that guardrails are useless has to get past them first.

Meta's LlamaFirewall is a serious open-source attempt at the same problem, built for agent pipelines rather than single prompts. Several commercial systems report comparable figures on their own evaluations.

Take all of it at face value. The argument in this chapter does not require any of these results to be wrong.

What happened when we attacked the best case ourselves

Those numbers were measured on benchmarks, against attacks nobody was adapting. So we ran the obvious next test, and appendix E has the method and the caveats in full.

Hackett et al. published twelve character-injection families in 2025 and reported up to 100% evasion against six guardrail systems. We reimplemented the twelve and pointed them at a corpus built for this book's own agent, then measured a downloadable guard model and a PromptArmor-style LLM detector on the same payloads.

The guard model has clearly been hardened since that paper. Spacing, zero-width characters, underlining and inverted text are now caught outright, where the paper found them productive. Two families were untouched, and they are the two the paper ranked highest: payloads smuggled into emoji variation selectors and Unicode tag characters. Unicode tag smuggling evaded every detector we tested, at every size, without a single catch.

One honest qualification, and it matters. Neither model we tested can read tag-smuggled text either, so against those models this is a blind detector in front of a blind agent rather than a working exploit. That is an accident of tokenization, not a defense. It holds until someone ships a model that decodes those characters, and the filter in front of it will not notice the day that changes.

The first crack: you pay for recall in false positives

The trouble starts when you measure the same defenses on a corpus that contains benign traffic in realistic proportions.

One controlled evaluation ran 480 queries, 369 of them injections and 111 benign, through several deployed configurations. The numbers are worth sitting with.

NeMo stopped everything. It also rejected roughly one in six legitimate requests and added a second and a half to every call. That configuration is secure and unusable, and no product team will ship it.

Read down the column and the shape of the problem appears. You are not choosing a defense. You are choosing a point on a curve, and the curve does not have a corner where security and usability are both free. Every deployment picks a threshold, and the threshold is where your protection actually lives.

Two things about that table before it gets quoted anywhere. These are single-corpus measurements against non-adaptive attacks, which is the most favourable condition a detector will ever see. And the evaluation was run on educational tutoring traffic, not support or engineering workloads, so read the shape of the curve rather than the specific figures. The trade-off is the finding. The decimal places are not.

The second crack: over-defense is not a tuning problem

The natural response is to move the threshold. Accept more false negatives, keep the product usable, call it risk management.

That assumes the classifier's errors are distributed sensibly. They are not. Prompt guard models suffer from trigger word bias: they learn that certain tokens co-occur with attacks and flag benign text containing those tokens.

The NotInject benchmark tests exactly this, using 339 benign samples enriched with words common in injection attacks. Against it, state-of-the-art guard models drop to "close to random guessing levels (60%)" on benign classification (Li et al., 2024).

A coin flip, on ordinary text, because the ordinary text mentioned instructions.

Your support inbox contains customers writing things like "ignore my previous email, the account number was wrong." Your engineering corpus contains documentation about prompt injection. Both are benign. Both sit in the region where these models perform near chance. The false positives are not randomly scattered across your traffic, they are concentrated in the content most likely to be about the system itself.

The InjecGuard authors report improving on the previous best by 30.8% with a training strategy aimed squarely at this bias, so the problem is tractable. It is not solved, and it is a property of how these classifiers learn rather than a threshold you can nudge.