What counts as evidence that an agent finished?

First, the intuition.

A log entry saying “done” is a claim, not evidence. A trustworthy trail connects the claim to a concrete diff, a test invocation, and the outcome after the last edit.

What the researchers actually did

Soei proposes an acknowledgement-aware event ledger, provenance graph, and verification layer. Its observational analysis examines 45 real coding-agent sessions.

What the results suggest

• The authors estimate that 23.4% of sampled code-editing episodes lack an independent check after the last edit.
• They estimate that 57.8% of agent-marked completed to-dos lack supporting proof within the relevant interval.
• Their prototype explores replayable, integrity-checked evidence records.

How convincing is the evidence?

Real-session traces and audit rules illuminate a concrete failure mode: claims and actions can become disconnected from verifiable outcomes.

What remains uncertain?

Only 45 sessions were audited. Operational definitions of proof and independent checking matter. This is not proof that the proposed ledger reduces real-world defects.

Connections worth exploring

• Provenance graphs resemble data lineage in scientific computing.
• An evidence ledger must protect privacy as well as integrity.
• The verifier should examine the final artifact rather than earlier test output.