Why can coding agents complete individual features but fail when a codebase must keep evolving?
First, the intuition.
A wrong assumption in an early patch becomes part of the environment for every later patch. If each step is graded in isolation, that accumulating failure is mostly invisible.
What the researchers actually did
FeatureArc evaluates agents over 73 evolving-feature tasks, each with two to ten sequential stages in a persistent workspace. Tests cover newly required behavior and regressions in old behavior.
What the results suggest
- The reported pooled first-stage success is 62.7%, while complete evolving-feature success falls to 28.5%.
- Inherited defects appear in 72.9% of failed later stages.
- After an earlier stage fails, the problem often continues without an explicit diagnosis.
How convincing is the evidence?
Persistent workspaces, sequential requirements, hidden acceptance tests, and regression checks make this much more realistic than independent one-shot questions.
What remains uncertain?
The experiment deliberately limits feedback and rollback, so it should not be interpreted as the best reliability an agent with stronger verification could achieve. The available tasks are still a limited sample.
Connections worth exploring
- Compounding errors also appear in planning, control systems, and data pipelines.
- Long-horizon success needs chain-level metrics, not averages over separate steps.
- Independent regression tests serve as an external memory of what must remain true.
One-step competence is not long-term reliability.
Yunlong Wang et al. · Oct 27, 2026
Our analysis is not an independent replication.