Framework Conventions Cut AI Coding Agent Accuracy by 30 Points

May 24, 2026 | Sources: arxiv 2605.06445, Forge (antoinezambelli/forge)

New research: LLM coding agents suffer measurable performance collapse as framework conventions accumulate, dropping 30 points between minimal and convention-heavy stacks.

Framework selection has always involved tradeoffs: ecosystem size, hiring pool, documentation quality, community momentum. A paper published this week adds a new axis to that decision. Researchers measured what they call "constraint decay" in LLM coding agents, the pattern where agent accuracy degrades as structural requirements (schemas, ORMs, architectural patterns, dependency injection) accumulate in a codebase. Agents that succeed reliably with minimal frameworks fall apart when asked to work within convention-heavy ones, losing 30 points of accuracy between the two extremes.

The root cause is specific: defects that stem from implicit behavior. Convention-heavy frameworks encode rules that the developer is expected to already know. Django's ORM (object-relational mapper) expects you to know that select_related is for foreign keys and prefetch_related is for many-to-many. FastAPI's dependency injection expects you to understand the resolution order without it being stated anywhere in the code. Next.js App Router expects you to know which components render on the server by default, where use client must appear, how layout nesting affects data fetching, and which files are treated as route segments versus utilities. None of that is in the code an agent is currently reading. It exists in documentation and training data, and agents get it wrong at a measurable rate.

Same model, same prompts, same task complexity. The conventions are the variable.

A separate project this week reinforces the other side of the equation. Forge, an open-source agent harness, demonstrated that structural guardrails (response validation, rescue parsing, retry loops) can take an 8B parameter model from single-digit accuracy to 84% on agentic tasks. Applied to Claude Sonnet 4.6, the same guardrails pushed accuracy from 85% to 98%. The variable is the infrastructure around the model.

Why This Matters for Next.js Developers

Next.js App Router is a particularly dense convention surface. Server Components are the default, but the boundary between server and client is implicit: an agent adding an event handler or a useState call to a Server Component will generate broken code that passes a type check and fails at runtime. The use client directive must appear at the top of a file, not inline. Route segment behavior (layouts, loading states, error boundaries) is determined by filename, not by any declaration in the file itself. Data fetching patterns changed significantly between the Pages Router and App Router, and agents trained on a large corpus of pre-App-Router Next.js code carry that prior into every generation.

This shows up most in two places: data-layer work (fetch caching, revalidation, route handlers) and component boundary decisions. These are exactly the defect categories the research identifies as highest-risk in convention-heavy frameworks.

Three things that help agents work more reliably in a Next.js codebase.

Make the server/client boundary explicit in your conventions. A CLAUDE.md or agent instruction file that states which directories are server-only, which are client-only, and what the team's pattern is for shared utilities removes the biggest source of implicit guesswork. Agents that have this context make fewer boundary errors.

Wrap implicit Next.js patterns in explicit utilities. Data fetching functions with typed return signatures, a single fetchWithCache wrapper that encapsulates your revalidation strategy, typed route parameter schemas. Each wrapper converts an implicit convention into something the agent can read and reason about rather than guess.

Treat App Router migration code with elevated scrutiny. If your codebase has a mix of Pages Router and App Router patterns, the agent's training data is working against you. Flag mixed-router files explicitly and review agent output on them before committing.

Forge's results show that a 98% ceiling is achievable with current models when the surrounding system validates outputs, retries on structural failures, and enforces response schemas. If your agent pipeline has no retry logic, no output validation, and no schema enforcement, that is a larger reliability gap than any model swap will close.

As agents take on more implementation work, the gap between implicit and explicit frameworks will compound. Teams making stack decisions today are locking in their agent productivity ceiling for the next two to three years. For teams already committed to Next.js, the answer is not to switch; it is to make the implicit explicit before the agent touches it.