How Cognition Tripled Merged PRs Per Week Using Claude to Power Devin, Its Autonomous AI Engineer

Cognition launched Devin in early 2024 as one of the first AI software engineers — a product designed not to complete individual code suggestions but to take a well-scoped ticket and own the entire trajectory from understanding to shipped PR. The bar is categorically different from a code-completion tool. Users under-specify tasks by default; the agent has to clarify intent, infer context it wasn't given, and sustain focus across a long, multi-step sequence without drifting. A wrong starting inference doesn't just produce a bad line of code — it sends the entire trajectory off course.

The primary failure mode for autonomous agents before current frontier models was consistency: trajectories would sometimes work, but output quality degraded over long contexts and variance was too high to trust autonomously. Cognition's engineering work focuses heavily on trajectory monitoring — detecting when an agent is going off track and steering it back — but that only works if the underlying model can sustain coherent, long-running execution in the first place.

Cognition routes across multiple models, directing work to the model best suited for each task. For long-horizon, agentic execution it routes to Claude. Three properties drive the routing decision. First, sustained performance over long contexts: Claude models have been ahead of the curve at following through and consistently working on longer-running tasks without losing track of the original objective. Second, intelligent tool use: Devin gives the model access to the PR itself, commit history, codebase files, clarification loops, and more — using each appropriately requires a specific kind of judgment. Third, intent expansion: the ability to take a two-line description and correctly expand it to all the implied details, without requiring fully-specified instructions.

Cognition runs a comprehensive internal evaluation suite covering mechanical code editing, file localisation, and end-to-end software engineering tasks. When Claude Sonnet 3.6 came out in 2024, the eval results drove the adoption decision. Since routing long-horizon work to Claude Sonnet 3.6, Cognition recorded a 3.5× increase in merged PRs per week — the metric their enterprise customers (Goldman Sachs, Mercedes-Benz, the US Army) hold them to: a well-scoped ticket comes in, the agent ships it, the PR gets merged.

The longer-term direction for Cognition is moving Devin beyond individual contributor work toward higher-level engineering goals — design decisions, architecture trade-offs, and cross-system coordination — which requires even deeper capability in the long-horizon agentic execution that Claude underpins today.