AI-assisted delivery is blowing up estimation variance: one ticket lands in an hour, the next takes three days of review, QA, and undoing plausible-looking mistakes. If you’re still treating story points as a stable proxy for effort, you’re optimising the average while the business bleeds on the outliers.
The punchline: stop arguing about whether AI makes developers “faster”. Start modelling where uncertainty comes from, track it explicitly, and price that risk into fixed-bid scopes.
The problem: AI doesn’t just change velocity, it changes variance
AI coding tools change the shape of work. They collapse some tasks (boilerplate, scaffolding, straightforward refactors) and expand others (anything with hidden constraints, messy legacy behaviour, non-obvious security properties, or unclear acceptance criteria).
That means two things happen at once:
- The median time-to-merge might improve.
- The spread (variance) gets worse, because the “looks done” state arrives earlier than the “is actually correct, secure, and maintainable” state.
If you’re running fixed-bid projects (or even fixed-scope internal commitments), variance is the thing that kills you. A handful of rework-heavy tickets will consume your contingency faster than any steady, predictable drift.
This isn’t just a vibe. DORA’s generative AI report found that increased AI adoption was associated with decreased delivery throughput and stability in their analysis, which is exactly the pattern you’d expect when rework and instability dominate the system. (dora.dev)
And practitioners are already circling the right issue: estimating the work is now partly about whether you gave the ticket enough context for AI-assisted implementation to go well, and how you account for verification effort. (reddit.com)
Why story points fail harder under AI
Story points were never a physics unit. They worked (when they worked) because teams had a relatively stable relationship between:
- The shape of work
- The team’s shared understanding of “done”
- The cost of integration and verification
AI breaks that relationship in a specific way: it decouples typing from correctness.
Failure mode 1: “Implementation is done” arrives early
With AI, a developer can produce a lot of code quickly. The ticket feels like it’s progressing. But the true cost is often in:
- Code review: does it match system conventions, edge cases, error semantics?
- QA: did we accidentally change behaviour outside the happy path?
- Security: did we introduce a known weakness in a place static analysis won’t scream?
There is a body of research showing security weaknesses can appear in AI-generated code and that automated checks (for example, CodeQL-style analysis) are commonly used to evaluate these risks. You don’t need to be alarmist to recognise the cost: security review and remediation effort is real work, and it’s spiky work. (doi.org)
Failure mode 2: “The ticket was small” hides the actual driver
In AI-heavy delivery, the main driver often isn’t “how much code”; it’s:
- How precisely the expected behaviour is specified
- How constrained the environment is (framework, legacy boundaries, platform restrictions)
- How expensive it is to verify (test harness quality, staging parity, observability)
Those aren’t cleanly captured by a single scalar (points). You can debate the number, but you’re still compressing the interesting part out of the discussion.
Failure mode 3: points become a proxy for trust
When output becomes unpredictable, organisations tend to respond by tightening controls:
- More estimation ceremony
- More checkpoints
- More “prove it” gating
That makes the system even slower. It also incentivises developers to avoid high-uncertainty work, because it’s the work that “blows the sprint”.
The alternative: a 3-bucket uncertainty ledger
An uncertainty ledger is not “no estimates”. It’s an explicit agreement that we will track variance drivers rather than pretending we can forecast effort precisely.
The model is deliberately simple: for each ticket (or epic, if you’re scoping a fixed bid), you allocate work into three buckets.
- A: Deterministic build
- B: Human verification and integration
- C: Unknowns and rework risk
You are not trying to predict the exact hours. You are trying to predict where time can explode.
Bucket A: Deterministic build
This is the part AI often crushes.
Examples:
- CRUD endpoints that follow an existing pattern
- UI components that reuse established design system primitives
- Straight-line data transformations with clear inputs/outputs
What you look for when scoping:
- Existing reference implementations
- Strong conventions
- Low coupling to legacy behaviour
If a ticket is mostly Bucket A, AI will likely reduce cycle time without much drama.
Bucket B: Human verification and integration
This is the part many teams forget to price.
Examples:
- Writing tests that actually encode tricky business rules
- Updating observability (logs/metrics/traces) so you can detect regressions
- Reviewing for maintainability, performance implications, and security posture
- Coordinating with other teams, data owners, platform constraints
This bucket is where AI can be neutral or even negative: the code is produced quickly, but humans still have to validate behaviour across a system.
A practical rule: if you can’t clearly describe the verification plan, you don’t have a small ticket; you have an under-specified ticket.
Bucket C: Unknowns and rework risk
Bucket C is the explicit admission that some work will not go linearly.
Common variance drivers under AI:
- Ambiguous acceptance criteria (“works like X” without edge cases)
- Legacy behaviour that isn’t codified in tests
- Cross-cutting concerns (auth, permissions, caching, data migrations)
- Non-local constraints (performance SLOs, compliance, auditability)
- Flaky environments (staging differs from prod; incomplete test data)
This is also where you put “AI risk” honestly:
- The model generates plausible code that compiles but violates domain rules
- The model follows a library pattern that is subtly wrong for your version/config
- The model introduces security weaknesses that require expert review
Research in this area is evolving, but the conservative takeaway is stable: you cannot outsource accountability, and you should expect non-zero remediation work in security-sensitive codepaths. (doi.org)
How to use the ledger in real scoping (including fixed bids)
You can run the ledger at ticket level, but it’s most valuable at the boundary where money and commitments happen: proposals, milestones, and release scopes.
- Break the scope into thin vertical slices (the smallest user-visible units you can ship).
- For each slice, assign a rough split across A/B/C.
- Price the project off:
- The sum of A and B (your baseline)
- Plus a risk premium based on C
- Plus a management reserve for coordination and unknown unknowns
This is the key move: Bucket C is priced explicitly, not hidden in optimistic point estimates.
A concrete heuristic for founders and heads of engineering
You don’t need a Monte Carlo spreadsheet to get value. You need repeatable questions that surface the variance.
For each item, ask:
- What would make this 10× faster?
- What would make this 10× slower?
- If we had to prove correctness to a sceptical reviewer, what artefacts would we need (tests, logs, docs, threat model notes)?
Then map the answers:
- “10× faster” factors usually reduce A.
- “10× slower” factors usually inflate B or C.
Operating rules that keep the ledger honest
A model is only useful if it changes behaviour.
Rule 1: Do not allow Bucket C to hide indefinitely
If a ticket repeatedly lands in C, stop calling it a ticket and convert it to a timeboxed discovery.
- Define the question you’re answering.
- Define the output (prototype, spike, decision record, test harness improvement).
- Define what gets cut if the answer is “this is riskier than we thought”.
Rule 2: Separate “AI did it” from “we can ship it”
Track at least two timestamps:
- Code drafted (often AI-assisted)
- Code accepted (reviewed, tested, deployed)
The gap between these is the story. It’s where rework lives.
This aligns with what practitioners are debating right now: separating AI-assisted creation from human-verified completion, and focusing on what correlates with rework risk rather than raw output. (reddit.com)
Rule 3: Tighten acceptance criteria, not control
When variance rises, the worst response is more ceremonial estimation. The better response is:
- Sharper definitions of done
- Better test scaffolding
- Clearer ownership of cross-cutting constraints
If you can’t afford those, you can’t afford a fixed bid on that scope.
When you should not use this approach
There are cases where the ledger is the wrong tool.
- If you already have high predictability and low rework, and story points are not harming you, keep it simple.
- If you’re doing pure R&D where outcomes are unknown, you may be better with explicit time-and-materials plus a kill switch.
- If leadership is going to misuse the ledger as a performance tool (“why is your C so high?”), don’t deploy it. You’ll get fake numbers.
Where codeversols fits (briefly, honestly)
If you’re scoping AI-heavy delivery and your risk is in verification, not typing, the practical work is usually: test harness upgrades, CI/CD tightening, security review patterns, and carving scope into shippable slices. That’s the sort of delivery engineering we do at codeversols across web, mobile, AI, and cloud: making it possible to ship safely and predictably when the implementation layer has become cheap but correctness has not.
Close
AI-assisted engineering didn’t kill estimation; it killed the idea that a single number can stand in for uncertainty. Replace story points with a three-bucket uncertainty ledger, track the gap between “drafted” and “accepted”, and price rework risk explicitly into fixed-bid scopes. When you do that, you stop arguing about whether AI is making you faster and start managing the thing that actually moves your delivery date: variance.
