AINN All articles
AI Policy & Law

Merge Without a Signature: The Accountability Gap in AI-Assisted Software

AINN
Merge Without a Signature: The Accountability Gap in AI-Assisted Software

Photo: DFID - UK Department for International Development, CC BY 2.0, via Wikimedia Commons

A process that has quietly stopped meaning anything

Most American software organizations still describe themselves as having human review. A pull request is opened, a colleague reads the diff, a comment or two is exchanged, and an approval is recorded. The process looks intact. The question worth asking is whether the person approving could explain the change to an auditor, or a judge, a year later.

Increasingly, the answer is no. As AI assistants generate more of the code that reaches production, review has compressed into a shorter, faster, and less legible act. Reviewers are evaluating more volume with less context, and the approval they record is a click, not a statement of understanding. The form survives; the substance has thinned.

Three questions no one can currently answer

The gap is easiest to see in the questions that arise after an incident. If an AI-assisted change causes an outage, an injury, or a financial loss, the organization needs to establish who decided to ship it, on what information, and with what authority. In many teams today, none of the three is answerable in a documented way.

Who decided? The commit carries the name of the developer, but the change may have been drafted largely by a model, revised by another model, and approved by a colleague reviewing a summary rather than the diff. None of these roles map cleanly onto the concept of an author.

On what information? Reviewers often see what the tool shows them. If the assistant summarizes its own change, the reviewer may be auditing the summary rather than the code. That is a meaningful difference in evidentiary weight.

With what authority? In most organizations, the right to merge is a repository permission, not a defined delegation of responsibility for design decisions. Nothing establishes that the approver was competent to assess the specific risk being introduced — a cryptographic change, a billing path, a safety interlock.

Why the legal question is closer than it feels

United States law does not presently impose a general duty on software authors to prove how a particular line was written. But liability rarely arrives as a pure software question. It arrives through contracts, consumer protection statutes, sector regulators, and negligence claims, each asking the same underlying thing: did the organization exercise reasonable care, and can it demonstrate that it did?

Documentation is where that demonstration lives. A firm that cannot produce a coherent record of human judgment will find itself arguing reasonableness in the abstract — an uncomfortable posture when the opposing side can point to the tool that generated the change and the absence of any substantive check on it.

Contractual allocation matters here too. Enterprise agreements with AI vendors vary widely on indemnification for generated code, and the scope of those provisions is often narrower than engineering leaders assume. A vendor indemnity that covers output does not necessarily cover the downstream decision to deploy it, particularly where the customer modified, combined, or misapplied the output. The indemnity is not a substitute for internal control; it is a second layer that may or may not engage.

What meaningful sign-off would require

A workable model does not need to reimpose line-by-line review on every change, which would destroy the productivity gains that justified the tools in the first place. It needs to make the human decision legible and proportional to risk.

Risk-tiered approval authority. Low-risk changes can travel with lightweight attestation. Changes touching authentication, payment flows, regulated data, or safety-relevant logic should require an approver with named competence and a recorded rationale, not merely a permission.

Provenance attached to the change. Records should indicate which portions were model-generated, which tools were used, what prompts or specifications drove the work, and what validation (tests, static analysis, manual inspection) was performed. This is metadata, not surveillance; it is the equivalent of a lab notebook.

Explicit attestation language. An approval should assert something specific — that the approver read the change, understood its intent, and verified the stated validation. Vague approval buttons invite vague attention.

Non-delegable decisions. Some judgments should not be delegated to a model-assisted process at all. Defining that list is an engineering-management task, and the absence of a list is itself a finding.

The organizational obstacle

None of this is technically hard. Version control already records who merged what and when. Adding attestation fields, provenance tags, and approval-level requirements is largely a configuration exercise. The obstacle is cultural: these practices impose friction on the exact workflow that AI tools were adopted to accelerate.

That friction is the point. The productivity gain from AI-assisted development is real, and it is not going away. But an organization that captures the speed while discarding the accountability has not modernized its engineering process — it has removed one.

A near-term agenda

Three steps would close most of the gap for a typical US engineering team within a year. First, define which classes of change require named, competent approval and encode that requirement in the tooling rather than in a wiki. Second, require provenance metadata on AI-assisted changes so that post-incident reconstruction is possible. Third, have counsel review AI vendor agreements specifically for what happens when generated code causes harm downstream — and adjust internal policy to cover what the contract does not.

Software has always carried implicit trust: that the person who approved a change understood it. AI assistants have not broken that trust. They have revealed that in many organizations it was thinner than anyone wanted to admit.

All Articles

Related Articles

The Checkout Problem: Agentic Commerce Needs a Chain of Authorization

The Checkout Problem: Agentic Commerce Needs a Chain of Authorization

Megawatts Before Megaflops: The Power Queue Rewriting AI's Expansion Map

Megawatts Before Megaflops: The Power Queue Rewriting AI's Expansion Map

The Rubber Stamp: Five Ways Human Review Fails AI Systems

The Rubber Stamp: Five Ways Human Review Fails AI Systems