Decision Provenance Stores: A Reference Architecture

Regulators asking "why did the system deny this claim" and an engineering team asking "why did accuracy drop 4 points on Tuesday" are asking the same underlying question in different vocabulary: what evidence, model version, and policy actually produced this specific decision. Most organizations we've audited can answer that question for maybe the last few days, if the right person is still around to reconstruct it by hand from logs scattered across three systems. That's not an audit gap you patch with better logging. It's a missing piece of core infrastructure, and building it deliberately — as a decision provenance store — is one of the highest-leverage pieces of a decision intelligence platform, because everything downstream of it, from compliance to model improvement, depends on records that were designed to answer this question from the start.

What the store actually records

Every automated decision our reference architecture governs writes one immutable provenance record at decision time, not reconstructed after the fact from logs. That record captures four things: the input evidence (a content-addressed reference to the exact data — feature values, retrieved documents, sensor readings — the decision consumed, not a pointer to a mutable table that might look different tomorrow), the model or policy version (a hash, not a version number alone, because version numbers get reused across hotfixes more often than teams admit), the decision path (which rules fired, which agent or model made which sub-call, in what order — the full execution trace, not just the final output), and the outcome (the decision itself, plus, whenever it becomes available later, the real-world result the decision was trying to predict or optimize for, linked back to this same record).

Why content-addressing the evidence matters more than it sounds like it should

The single most common failure we see in provenance systems built without this discipline: the record points to "customer_risk_table, row 4471" instead of the actual values that row held at decision time, and six weeks later, when someone asks why a decision was made, that row has been updated three times and the provenance record now silently points to different evidence than what actually informed the decision. Content-addressing — hashing the actual input payload and storing that hash alongside a snapshot of the payload itself — makes the record actually immutable in the way an audit requires, not immutable in name while mutable in substance underneath.

The architecture, layer by layer

At the write path, every decisioning service — whether it's a rules engine, an ML model, or an agent orchestration layer — emits a provenance event synchronously, before the decision is allowed to take effect downstream, to an append-only event log (we build this on the same Kafka backbone that carries the rest of the agent mesh's traffic, on a dedicated topic partitioned by decision domain). A stream processor consumes that topic and writes into two places: a queryable store, indexed by entity (customer, claim, shipment) and by time, optimized for "show me every decision that touched this account," and a cold, cryptographically chained archive optimized for tamper-evidence over long retention windows, which is the property an actual regulatory audit cares about most. On the read path, we build two access patterns from day one, because they get requested immediately once the store exists: a case-lookup API for compliance and support teams asking about one specific decision, and an aggregate-analysis interface for the model team asking a population-level question — which model version's decisions correlate with the outcome drift we're seeing this month.

The part teams under-invest in: linking outcomes back

A provenance store that only records decisions is half the system. The record isn't done providing value until the real-world outcome — did the customer default, was the maintenance prediction correct, did the fraud flag hold up on manual review — gets written back and linked to the original decision record, on whatever delay is natural to that outcome (minutes for some decisions, months for others). That link is what turns a compliance archive into an improvement feedback loop: without it, you can answer "why did we decide this" but not "was this model version's decision policy actually good," which is the question that should be driving the next model iteration.

Teams that build this expecting only a compliance artifact are usually surprised by what else it enables: the ability to precisely bound the blast radius of a bad model version the day it's discovered — query every decision that used that version's hash, not an estimate — and the ability to run counterfactual analysis against real historical evidence instead of a synthetic backtest, because the actual inputs at decision time are still sitting there, exactly as they were, content-addressed and unchanged.