What the logs cost you in an incident you did not have

What the logs cost you in an incident you did not have

There is a quieter benefit worth naming, because it is what usually justifies the work internally.

Most agent alerts are not incidents. A support agent reports something odd, a customer queries a refund, a monitoring rule fires. In each case somebody has to decide whether anything actually happened, and that decision is where the time goes.

The gate record either shows a consequential action with tainted arguments or it does not, and that is a lookup rather than an investigation. What makes this matter is volume: these arrive weekly, not annually, and a triage step measured in minutes rather than hours is the difference between a process people follow and one they quietly stop running.

A team that investigates a handful of these a month recovers the cost of this chapter well before a real incident arrives.

Disclosure

Two facts make agent incidents awkward to report, and both are better acknowledged early.

The first is that nothing was breached in the conventional sense. No credential was stolen, no vulnerability exploited, no system compromised. Your software did what it was asked by someone who was not supposed to be asking. That is genuinely hard to explain to a board and it is still a security incident.

The second is that the affected-party analysis depends entirely on the logs above. If you cannot say which records the agent read, you cannot scope a notification, and you will end up notifying everyone or defending a decision not to.

Where a regulator is involved, chapter 18's transparency obligations apply and the record you need is the one this chapter describes.

Who to call

Decide in advance who owns an agent incident, because the answer is genuinely unclear and the ambiguity costs hours.

Nothing was breached, so the security team can reasonably say it is not theirs. The software worked as written, so it is not a product bug. And whether it is a data incident depends entirely on what left, which is the thing nobody knows yet.

The workable answer is that agent incidents run through the normal security process, with one addition: whoever owns the agent joins the call from the start. They are the only person who can read a gate record and say whether a verdict was correct, and pulling them in an hour late is the most common reason these take a day instead of an hour.

Practising

The single most useful thing in this chapter is a rehearsal.

Plant a marked document in a staging corpus. Let an agent read it. Take an action. Then hand the resulting alert to someone who was not involved and ask them to answer the three questions using only the logs.

They will fail the first time, and the specific way they fail tells you which record is missing. This takes an afternoon and it is the difference between an incident that resolves in hours and one that resolves in a week of reading transcripts.

What this costs

Storage, and the discipline to log lineage rather than content when content would be easier and more satisfying to read.

There is also a real tension with data minimisation. You are keeping records of which sources fed which actions, and those records are themselves a description of your data flows. Retain them deliberately, keep them out of the agent's reach, and treat the log store as a system that would badly want protecting.

Chapter 18 turns to the people who will ask to see all of this.

Sources for this chapter

The three questions, the four records, the SourceRef field, the blast-radius query method and the rehearsal exercise are the author's. No measured claims appear in this chapter. The observation that teams over-scope incidents because they cannot do better is experience rather than survey data, and the claim that a rehearsal saves days is an estimate offered without measurement.