What an AI Decision Record Must Contain to Survive Audit

There are three rooms in which an organization is asked to account for a decision its AI helped make. The furniture differs; the question does not.

In the first room sits an auditor or a regulator. Under the EU AI Act, high-risk systems must "technically allow for the automatic recording of events" over their lifetime, and both providers and deployers must retain those logs. An ISO/IEC 42001 certification auditor asks for documented evidence that the controls you describe actually operated. A bank's model risk examiner, working from guidance that predates the current AI wave by more than a decade, asks for the model inventory, the validation evidence, and the change history.

In the second room sits opposing counsel. The vocabulary changes (authentication, foundation, chain of custody), but the demand is the same, sharpened by the fact that the person asking is paid to find the gap. Under Federal Rule of Evidence 901, your record is admitted only if you can produce "evidence sufficient to support a finding that the item is what the proponent claims it is." In a corporate deposition, a designated witness will be asked, under oath: who created this record, when, has it been altered since, how would you know, and who could alter it.

In the third room sits your own team, eighteen months after the fact, because a threshold has been found defective and someone has to determine which past decisions used it, and for whom.

All three rooms ask for the same object: a record of this decision: what the system saw, which version ran, which rules applied, who acted, when, provably unaltered, producible now. Most organizations, asked for that object, produce logs instead. We have written before about why that substitution fails in general, and about what it costs when the failure surfaces in federal discovery. This piece is the constructive half: the anatomy of the record itself.

1. The tests are older than the technology

It is tempting to treat AI recordkeeping as a new problem awaiting new rules. The opposite is closer to true. The law of evidence has spent decades (in one instance, eight of them) defining what a trustworthy record looks like, and the definition transfers to AI decisions without modification. What AI changes is not the test. It is how completely ordinary logging fails it.

Four authorities do most of the work.

Rule 901: authentication. The proponent of any exhibit must show the item is what they claim it is. Rule 901(b)(9) offers one route tailor-made for machine output: evidence "describing a process or system and showing that it produces an accurate result." Read that as a design requirement. If your record cannot be accompanied by a credible account of the process that produced it, it does not get in.

Rules 902(13) and 902(14): self-authentication. Added in December 2017, these allow two kinds of electronic records to authenticate themselves, without a live witness: records "generated by an electronic process or system that produces an accurate result," and data copied from a device or file where the copy is "authenticated by a process of digital identification." The advisory committee's paradigm for that phrase is hash verification: a record whose fingerprint can be recomputed and matched. These rules are, in effect, a standing invitation: build records that carry their own proof of integrity, and litigation gets cheaper and safer. Records that cannot prove their own integrity face the witness stand instead.

Rule 803(6): the business records exception. Hearsay rules would exclude most corporate records, except that records of a regularly conducted activity are admitted when they were made "at or near the time" of the event, by someone with knowledge, as a regular practice, and when the opponent cannot show that "the source of information or the method or circumstances of preparation indicate a lack of trustworthiness." Every clause of that sentence is a test your record either passes at creation time or fails forever.

Palmer v. Hoffman: the contemporaneity trap. In 1943 the Supreme Court held that an accident report prepared by a railroad with an eye toward litigation was not a record made "in the regular course of business," and excluded it. The modern translation is direct: a decision narrative assembled after the dispute arises is not a business record. It is a litigation exhibit, prepared by an interested party, and it will be treated as one. The record that protects you is the one written before anyone knew it would be needed, which means it must be written for every decision, automatically, or it does not exist for the decision that matters.

Anyone who wants the full map of how these rules apply to electronic evidence should read Lorraine v. Markel, the 2007 opinion that remains the standard tour of authentication, hearsay, and the ways electronically stored information fails both. The relevant point here is simpler: none of this is new, none of it is AI-specific, and all of it was knowable before your system issued its first decision.

Regulators are converging on the same demands from the other direction. The EU AI Act's Article 12 requires lifetime event recording for high-risk systems; Articles 19 and 26(6) set retention floors for providers and deployers (application dates for Annex III high-risk systems are in flux; see our Mobley analysis on the Digital Omnibus deferral); Article 86 gives affected individuals a right to "clear and meaningful explanations of the role of the AI system in the decision-making procedure." GDPR Article 22 constrains solely automated decisions and its transparency articles require "meaningful information about the logic involved." In US consumer credit, Regulation B requires adverse-action notices with specific reasons, and the CFPB's 2022 circular closed the obvious escape, stating that creditors cannot excuse vague reasons on the ground that their algorithm is too complex to explain. NIST's AI Risk Management Framework and ISO/IEC 42001 both make traceability and documented evidence of operation central to what "managed" means.

Different rooms. Same object. Here is what it contains.

2. The seven elements

Each element below is one field of the record, and each exists to pass a specific test from the rules above. For each: what it must contain, and why the operational log (the thing most organizations would actually produce) fails the test the element exists to pass.

Figure 1Seven questions, two witnesses

01
Inputs, as seen“What data did the system actually see?”
An applicant ID: a pointer into a live database that has been updated many times since. Dereferencing it today retrieves today's truth, not the truth the model saw.
Fails
02
Model and system version“Which build produced this output?”
Whatever the deploy timeline implies. Reconstructing it means joining deploy logs, CI history, and a git tag: three systems, three retention policies, zero designed to be joined.
Fails
03
Policy and thresholds in force“Which rules authorized this outcome?”
Current configuration only. The value in force eighteen months ago lives, at best, in a change-management ticket describing an intention.
Fails
04
Human actor and authority“Who reviewed it, what did they see, could they have overridden it?”
User 4471 clicked approve at 14:32. What they were shown and what they could have done was rendered from application state that no longer exists.
Fails
05
Timestamp integrity“How do you know when this record was made?”
Our server said so. A timestamp written by the application, on the application's clock, in a file the application can rewrite.
Fails
06
Tamper evidence“How do you know it hasn't been altered since?”
Our access controls are good. Testimony, from the party with the most at stake, about systems it administers itself.
Fails
07
Retention and export“Can you produce it now, and will it verify in our hands?”
Rotation deleted it on schedule, and what survives exports as text with none of the integrity apparatus attached.
Fails

The questions are the ones asked in the three rooms this piece opened with: by auditors, by opposing counsel at deposition, and by your own reviewers. Switch the witness and the same seven questions get materially different answers: 0 of 7 hold under the current one.

2.1 The inputs, as the system saw them

The record must contain the data actually presented to the system at decision time, copied into the record, not referenced by it.

This distinction does more work than any other in the anatomy. Operational systems log identifiers: an applicant ID, an account number, a document key. The identifier is a pointer into a live database, and live databases exist to be updated. The profile is enriched, the address corrected, the income figure refreshed. Eighteen months later, dereferencing the pointer retrieves today's truth, not the truth the model saw, and the difference between those two is frequently the entire dispute. A record that references its inputs is a record that quietly rewrites itself every time the source data changes, which is precisely the "circumstances of preparation" problem Rule 803(6) invites your opponent to raise.

Why logs fail: logs are written to be small and fast. Copying full input state into every event is exactly what logging best practice tells engineers not to do.

2.2 The model and system version

The record must name the exact system that produced the output: model identifier and version, prompt or feature-pipeline revision, and for retrieval-augmented systems, some identification of the knowledge state consulted, bound to the individual decision, not to a deployment window.

Model risk guidance has required version discipline at the system level since the Federal Reserve's SR 11-7 in 2011: an inventory of models, documentation, validation, change control. The decision record needs the same fact at the decision level, because "which build was live on March 14?" is answered today by an archaeology project across deploy logs, CI history, and a git tag: three systems with three retention policies, none of them designed to be joined, none of them preserved with evidence in mind. When the model has since been retired, the join may no longer be possible at all.

Why logs fail: version identity lives in the deployment system's records, not the decision event. The binding between them is an inference, and inference is what cross-examination is for.

2.3 The policy and thresholds in force

The record must state which rules authorized the outcome (the threshold, the eligibility criteria, the routing logic), with a revision identifier for the policy in effect at that moment.

This is the element internal reviewers need most. When a threshold is later found defective, the first question is which decisions did it touch? With policy revisions sealed into each record, that is a query. Without them, it is the multi-week reconstruction exercise that consumes organizations in every remediation.

Why logs fail: configuration is stored as current state, by design. The history, where it exists at all, lives in change-management tickets that describe an intention to change something, not proof of what value a given decision actually used.

2.4 The human actor and their authority

Where a person reviewed the output, the record must capture who they were, what they were shown, what authority they had (could they have overridden it?), and what they did.

Every serious oversight regime assumes this element exists. The EU AI Act's Article 14 requires that human overseers be able to correctly interpret the system's output, decide not to use it, or intervene and override it. Regulation B requires specific reasons for adverse action, and a reviewer who was shown only a score cannot supply them. In litigation, "a human reviewed it" is a defense exactly as strong as the record of what that human saw and could have done: a bare approval timestamp establishes presence, not oversight, and the gap between the two is where the term rubber stamp enters the deposition.

Why logs fail: audit trails in most enterprise software record that user 4471 clicked approve at 14:32. Everything that makes the review meaningful (the information presented, the authority held, the alternative available) is rendered from application state that has since changed, and was never written anywhere.

2.5 Timestamp integrity

The record must be provably of its time: timestamped at creation by a mechanism the organization cannot quietly adjust.

Rule 803(6) admits records made "at or near the time." Palmer excludes reconstructions. Both tests assume the timestamp itself is trustworthy, and a timestamp written by an application server is an assertion, not a proof. Clocks skew, pipelines delay, backfills exist, and every engineer knows it. The established answer is trusted timestamping of the kind standardized in RFC 3161 a quarter-century ago: a cryptographic binding between the record's fingerprint and a time asserted by an authority outside the writer's control. A record sealed this way does not ask the examiner to trust your clock. It lets them check.

Why logs fail: log timestamps are claims made by the writer, on the writer's clock, in a file the writer can rewrite.

2.6 Tamper evidence

The record must be able to demonstrate, to a party who does not trust you, that it has not changed since creation.

This is the property Rules 902(13) and (14) reward: a record whose integrity is established "by a process of digital identification" (a hash, a signature, a verifiable chain) can authenticate itself on certification, without a parade of witnesses. It is also the oldest idea in this anatomy. SEC Rule 17a-4 required broker-dealer records in non-rewriteable, non-erasable form long before AI complicated the picture, and NIST's digital-evidence work has always started from the same premise: digital records are uniquely susceptible to modification without trace, so integrity must be engineered in rather than asserted afterward.

An operational log's answer to "how do you know it wasn't altered?" is our access controls are good. That is testimony: from the party with the most at stake, about systems it administers itself. A sealed record's answer is recompute the hash. One of these survives a hostile examiner.

Why logs fail: logs are writable by the systems that produce them and by the administrators who operate those systems. The capability to alter is enough to found the question, even where no alteration occurred.

2.7 Retention and export

The record must outlive the system that wrote it, and leave it in a form that remains verifiable outside.

The statutory retention floors are deceptively short. The EU AI Act requires providers and deployers to keep logs at least six months. Regulation B requires 25 months for consumer credit applications. But the floor is not the exposure: limitation periods and discovery reach run far longer, and they are set by the dispute, not the statute. The discovery window in Mobley v. Workday opens in September 2020: decisions made on systems that have since been retrained, reconfigured, and retired several times over. A record retained inside a system that no longer exists was not retained in any sense that matters.

Export is the half of this element organizations forget. When the request comes (from a regulator, a litigant, an auditor), the deliverable is a coherent, verifiable bundle produced under controlled access, not read access to a log cluster. If exporting a decision's record strips the integrity evidence that made it trustworthy, the export is a photocopy of a sealed envelope: the contents arrived, and the proof stayed behind.

Why logs fail: log retention is sized to storage budgets and debugging horizons, so rotation destroys evidence on schedule, indifferent to what it deletes. And logs export as text, carrying none of the integrity apparatus an examiner will ask for, because they never had it.

3. One design decision, seven failures

The pattern across all seven elements is worth stating once, because it explains why the fix is architectural rather than incremental.

Operational logs fail these tests not through seven separate oversights but through one coherent design decision: a log is written by the system, for the system's operators, in the system's vocabulary, at the system's convenience. NIST's log management guidance has described logs this way for two decades: instruments of troubleshooting, security monitoring, and operational health. For that job, referencing mutable state is efficient, writer-controlled timestamps are adequate, mutability is a feature, and short retention is fiscal prudence. Logging is not broken. It is a well-designed tool for a different job.

The examiner's record inverts every one of those choices. It is written about a decision, for a stranger, in the vocabulary of the decision (inputs, version, policy, actor, time), at the moment of issue, sealed against everyone including its author, retained on the exposure horizon rather than the debugging one. You do not get from one to the other by logging more. You get there by writing a second kind of object.

4. The maturity path

Most organizations discover this gap all at once, usually in one of the three rooms, and the natural response is to treat it as one enormous project. It is better understood as four distinct capabilities, each of which is useful before the next exists, and each of which answers to a different examiner.

Figure 2Four stages, and which examiner each survives

Stage 0 · TelemetryCan prove: “The system was running.”

Application logs, traces, debug output: the operational default, and where most AI deployments are today.

Still fails · Cannot name the decision, let alone reconstruct it. Every question in Figure 1 gets the log's answer.

Ops reviewInternal auditAuditor / regulatorOpposing counsel

Stages are ordered by dependency, not importance: sealing a log that lacks bound context produces a tamper-evident record of not enough. And progress is prospective only: every decision issued at stage 0 is at stage 0 forever, which is why the start date drives exposure more than the pace.

Stage 0: telemetry. Application logs, APM traces, debug output. Proves the system was running. Survives an operations review and nothing else. This is where most AI deployments are today, including sophisticated ones.

Stage 1: decision events. The decision becomes a first-class recorded object: one event per decision, with a stable identifier, the output, and correlation keys to the surrounding workflow. This is the cheapest step with the largest return, because it creates the object everything later attaches to. It proves a decision happened and what it concluded. It survives internal audit.

Stage 2: bound context. The event now carries the anatomy: inputs as seen, model and policy versions, thresholds in force, human actor and authority, all captured at decision time, not joined afterward. This is the stage at which reconstruction becomes a query rather than a project, and the stage most regulatory frameworks are actually asking for. It survives the auditor and, in most postures, the regulator.

Stage 3: sealed evidence. Integrity and time become provable: records are hashed and sealed at issue, timestamps are anchored outside the writer's control, verification is available to a third party, and export produces a bundle that remains verifiable in the examiner's hands. This is the stage the Federal Rules quietly describe: the record that authenticates itself under 902(13) and (14), holds its foundation under 803(6), and turns the deposition's sharpest question into a demonstration. It is the only stage built for the second room.

Two properties of this path deserve emphasis. First, the stages are ordered by dependency, not by importance: sealing a log that lacks bound context produces a tamper-evident record of not enough. Second, and this is the uncomfortable one, progress along the path is prospective only. Reaching Stage 3 next year does nothing for this year's decisions. Every decision issued at Stage 0 is at Stage 0 forever, which is why the date you start is a bigger driver of exposure than the pace you move.

The diagnostic we recommend has not changed since our Mobley analysis: pick one real decision from eighteen months ago and attempt to produce its full record: inputs, version, policy, reviewer, timestamp, proof of integrity. Time the exercise. The stage you are at will be obvious within the hour, and the result will make the case to your board more effectively than any framework document.

This anatomy is also, not coincidentally, a description of what Sigilith's platform writes: a sealed, self-verifying record of each governed decision, created at the moment of issue, built for the rooms where it will eventually be read. The two-minute version of the reconstruction exercise is the fastest way to feel the difference between a system that logs and a system that can answer.

Sources

Rules of evidence and case law

Federal Rules of Evidence, Rule 901: Authenticating or Identifying Evidence (Cornell LII)
Federal Rules of Evidence, Rule 902: Evidence That Is Self-Authenticating, subsections (13) and (14), effective 1 December 2017 (Cornell LII)
Federal Rules of Evidence, Rule 803(6): Records of a Regularly Conducted Activity (Cornell LII)
Palmer v. Hoffman, 318 U.S. 109 (1943)
Lorraine v. Markel American Insurance Co., 241 F.R.D. 534 (D. Md. 2007)
Federal Rules of Civil Procedure, Rule 30(b)(6) and Rule 34 (Cornell LII)

Regulation

EU Artificial Intelligence Act, Regulation (EU) 2024/1689: Article 12: Record-Keeping, Article 14: Human Oversight, Article 19: Automatically Generated Logs, Article 26: Obligations of Deployers, Article 86: Right to Explanation; full text
GDPR, Regulation (EU) 2016/679: Article 22: Automated individual decision-making
CFPB, Circular 2022-03: Adverse action notification requirements in connection with credit decisions based on complex algorithms, 26 May 2022
Regulation B, 12 CFR Part 1002: §1002.9 (adverse action notification) and §1002.12(b) (record retention)
Federal Reserve, SR 11-7: Guidance on Model Risk Management, 4 April 2011
U.S. Securities and Exchange Commission, Rule 17a-4: Records to be preserved

Standards and technical guidance

ISO/IEC, 42001:2023 (AI Management Systems)
NIST, AI Risk Management Framework (AI RMF 1.0)
NIST, SP 800-92: Guide to Computer Security Log Management
NIST, IR 8387: Digital Evidence Preservation, 2022
IETF, RFC 3161: Internet X.509 Public Key Infrastructure Time-Stamp Protocol, August 2001