What Building Data Pipelines Taught Me About "AI Accountability"
Most accountability frameworks are written by people who've never had to debug a production model at 2am. Here's what the engineering side actually needs policy to understand.
I've spent years building data and AI pipelines that decides what data a model sees, how fresh it is, and what happens when something upstream quietly breaks. More recently, through a Decision Intelligence and AI Fellowship, I've started paying closer attention to the policy conversations happening around "AI accountability." And honestly? A lot of it reads like it was written by people who've only ever seen a model from the outside.
Here are four things the pipeline actually teaches you and the policy gap each one exposes.
1. Accountability isn't a moment, it's a chain
When something goes wrong with a model's output, the instinct is to ask "who approved this model?" But by the time a prediction reaches a user, it's passed through data collection, cleaning, feature engineering, training, validation, deployment, and monitoring, each stage often owned by different teams, sometimes different companies.
The policy gap: most frameworks assign accountability to a single point (the deployer, the "AI system") instead of the chain. That's like blaming the driver for a car crash caused by a faulty part installed three suppliers back. Real accountability frameworks need to trace responsibility across the pipeline, not just at the point of output.
2. "Explainability" and "debuggability" are not the same thing
Policy often talks about explainability as if it's one thing, a clean, human-readable reason for a decision. But when I'm debugging a production model at 2am, I don't need a philosophical explanation. I need to know: did the input distribution shift? Did a feature pipeline silently fail? Did we deploy a stale model?
The policy gap: regulations tend to demand explanations aimed at end users, but say almost nothing about the operational debuggability that would actually let engineers catch problems before they become incidents. You can have a perfectly "explainable" model that nobody can actually troubleshoot.
3. Most failures aren't the model's fault instead they're data's fault
In my experience, the majority of production incidents trace back to data: a schema change nobody flagged, a source that changed its update frequency, a label definition that drifted over time. The model just faithfully learned from bad inputs.
The policy gap: accountability frameworks are overwhelmingly model-centric. Audits ask about training data provenance once, at the start, then move on to model behavior. But data is not static, it keeps moving after deployment, and that's where things quietly break. Policy needs to treat data pipelines as an ongoing accountability surface, not a box to check at model release.
4. Monitoring is where accountability actually lives and it's the least regulated part
The parts of my job that most directly protect people from bad AI outcomes aren't the model architecture decisions, they're the monitoring, alerting, and rollback systems. Whether we catch drift in hours or in months is the real difference between a minor bug and a headline.
The policy gap: almost nothing in current accountability frameworks specifies requirements for post-deployment monitoring including how often, what thresholds, who's watching. It's treated as an engineering best practice, not a governance requirement. That's backwards. Monitoring infrastructure is accountability infrastructure.
Where this leaves me
I don't think engineers should be writing policy alone, any more than policy should be written without engineers in the room. But there's a translation gap, and it goes both ways. Engineers assume the operational realities are obvious; policymakers often don't have visibility into them at all.
That's the gap I'm trying to sit in, close enough to the pipelines to know where they actually break, and increasingly curious about how to make that knowledge legible to the people writing the rules.