Shield Agent AI

All 13 questions

Question 9 of 13 · Evidence

Do you keep your own record of agent actions, independent of the AI provider?

Clients also ask it like this:

  • Do you have audit logs of what AI agents did, not only the vendor's logs?
  • Where is the record of agent activity stored and who can change it?
  • Can you show what an agent did on our project last month?

What the client is checking

The client wants a record that you control and that does not depend on a vendor. They check where it lives, how long it lasts and who can edit it.

Three draft answers

Pick the level that is true for you today. Every statement marked [VERIFY] must be true for your organisation before you send it. Delete a statement that is not true. Fill each {{placeholder}}.

Not yet in place

We rely on what the AI provider and the developers' own tools show. [VERIFY] We do not keep our own record of agent actions. [VERIFY] We will start by {{date}}. [VERIFY]

Policy only

Our policy requires a record of agent actions on client work. [VERIFY] Developers keep the session history that their tools store on their own machines. [VERIFY] These records are not gathered in one place and are not protected against change. [VERIFY]

Enforced, with evidence

We keep our own record of agent actions: {{what is recorded}}. [VERIFY] It is stored in {{place}} and kept for {{period}}. [VERIFY] Only {{roles}} can change or delete it, and changes are logged. [VERIFY] The record does not depend on the AI provider. [VERIFY] We check that it is complete with {{test}}. [VERIFY]

Evidence to attach for the strongest answer

  • A redacted sample of the record.
  • The storage and retention settings.
  • The access list of who can read, change and delete the record.
  • The result of the completeness test.
  • The retention rule.

How to move up one level

From not yet in place to policy only

Decide what to record (tool, action class, project, time, result). Collect the records of all developers into one place. Set a retention period.

From policy only to enforced, with evidence

Store the record where developers cannot change it alone. Log every change. Test regularly that every agent action shows up in the record.

Where KYA fits

KYA writes a local record, ~/.kya/trail.jsonl, for all projects on one machine. Each event carries the project folder name, a verdict (ALLOW, DENY or REQUIRE_APPROVE) and an outcome (ran, blocked, held or error). Write and edit calls may store a clipped, redacted change preview. The record is separate from the AI provider.

The local record is a plain file on the developer's machine. It rotates at 8 MB and keeps one archive, so it uses about 16 MB at most and older events are deleted. A person with access to the machine can edit it. It is not a long-term archive and not tamper-proof.

A hosted workspace receives sessions and events from the machines that report to it. Its trail history window follows the plan. The docs list 30 to 365 days. The docs do not say what happens to events after the window. Get the retention term in writing before you promise it. kya import can bring in logs from other tools. kya certify --sign writes a signed evidence bundle. A key made on the machine proves that the bundle did not change and that the key stayed the same. It does not prove who you are.

Get all 13 answers in one document

Choose the level that is true for each group. You get a preview at once and the full pack by email.

Make my Answer Pack

Previous question Next question

These are drafts. Send only what is true for your organisation. This is not legal advice.