AI Implementation Plan Review: Red-Team It Before Coding
Summary: Once the spec is locked, VAON engineers use Claude Code in plan mode to draft the implementation plan. The plan then goes through two challenge rounds: a red-team round that tries to make it fail, and a validation round that checks it against the spec. Open questions go to the client, never to guesswork. A Tech Lead decides what changes.
Contents
- What goes into an AI-drafted implementation plan?
- Why does a tidy AI plan still need to be attacked?
- What does a red-team round ask?
- What does validating the plan against the spec check?
- Who decides what to fix after the two rounds?
- When is a plan red-team not needed?
- FAQ
1. What goes into an AI-drafted implementation plan?
An AI-drafted implementation plan at VAON has four parts: phases, the files that will change, done criteria for each phase, and known risks. The plan is written only after the spec has been agreed with you in writing. See the gap questions AI lists before a spec is locked.
Engineers write the plan with Claude Code in plan mode. Anthropic's documentation says that in this mode Claude reads files and proposes a plan, but makes no edits until you approve.
| Part of the plan | What it records | How it protects you |
|---|---|---|
| Phases | Order of work, safe stopping points | Each short block of 1 to 2 weeks ends with something that runs |
| Files to change | Scope of the change in the code | You see in advance which parts of the system are touched |
| Done criteria | Conditions a test or a screen action can confirm | Both sides agree on what "finished" means |
| Risks | What can break, how to roll back | Risks are discussed before effort goes into code |
The plan and its two challenge rounds are steps 4 and 5 of VAON's Claude Code development workflow. It is the standard VAON applies to most projects, not yet all.
2. Why does a tidy AI plan still need to be attacked?
A tidy AI plan still needs to be attacked because the neater a plan looks, the harder it is to spot a missing assumption. Every heading can be filled in and the business logic can still be wrong.
Speed is the second reason. AI writes code fast, so a wrong plan becomes wrong code fast. A finished module costs far more to fix than a plan.
For a buyer, this is the cheapest point to stop a risk: no code yet, no data moved.
3. What does a red-team round ask?
A plan red-team is a round where the AI plays the person trying to make the plan fail. It returns questions and failure scenarios only, never a rewritten plan, so the reader sees where the problem is.
Engineers run it apart from the planning session, in a new session or a Claude Code subagent, which starts with a fresh, isolated context window. It gets only the spec and the plan.
Red-team question categories below are a generic illustration, not from any client project:
| Category | Example question | If it hits, fix |
|---|---|---|
| Rollback | A bug appears after release. How do we return to the old version? Is new data lost? | Risks, release step |
| Data migration stops halfway | Migration stops at the midpoint. Where does it restart? Does a rerun write duplicates? | Phase split, rerun procedure |
| Permissions widen | After this change, can any role see data it could not see before? | Files to change, permission tests |
| Concurrency | Two users edit the same record, or a job runs twice. What happens? | Done criteria, locking approach |
| External API failure | The partner API does not respond. Retry or fail? Can a retry create two orders? | Risks, error handling |
| New dependency | Does the package we plan to add exist on the official registry? Who maintains it? | Make adding it a separate task |
| Tests | If this risk happens, which test turns red? If none does, what do we add? | Done criteria, test plan |
AI can suggest package names that do not exist, so on every project VAON checks package names before installing.
4. What does validating the plan against the spec check?
Validation is a round where the AI maps every spec requirement to the plan. The red-team asks "how does this break?". Validation asks "does this match what we agreed?". It looks for:
- Requirements in the spec that the plan never mentions.
- Work in the plan that the spec does not justify.
- Questions the spec cannot answer.
The third type has a firm rule: engineers do not guess, they ask you. An illustration: the spec says "large orders need manager approval" but never says what counts as large. The AI could pick a number itself. A plausible wrong number is still wrong, and code gets built on it fast.
Validation also checks done criteria. Nobody can say whether "search is faster" is finished, so each criterion must be confirmable by a test or a screen action. How done criteria become test cases is covered in AI-generated tests inside the pull request.
5. Who decides what to fix after the two rounds?
A Tech Lead or senior engineer reads the results of both rounds and decides what to fix. Not every finding gets fixed: some are handled in another phase or sit outside scope.
Unfixed findings are recorded with a one-line reason, so nobody raises them again. The plan moves to coding only after someone other than its author has challenged and confirmed it.
At VAON, client questions from validation go into the weekly report, under items that need your decision. If an answer widens the scope, it follows the written change-control process: VAON sends the price and schedule impact, and work starts only after you approve. No approval means no work and no charge. The CTO and a Tech Lead are involved in every project, so the reader of red-team results knows the architecture.
6. When is a plan red-team not needed?
A plan red-team is not needed for every change, and it has limits of its own:
- Very small changes. A label fix does not need two challenge rounds. The process scales with the change.
- No permission to share code or documents with AI. Steps where AI reads the code do not run. Engineers challenge the plan themselves, which is slower.
- AI does not know your unwritten rules. It asks well about familiar failures, not about rules nobody wrote down.
Without someone on your side to answer business questions, validation produces questions, not answers. VAON has not published any measurement of this step's effect.
7. FAQ
Do I have to read the whole implementation plan? No. You receive the parts that concern you: the phases, the done criteria and the list of questions that need your answer. The full plan sits in the project records alongside the review and test records, and you can open it whenever you need.
Do two challenge rounds slow the project down? They add time at the planning step, which is the cheapest place to fix things. A gap found while there is only a plan costs a few lines of text. The same gap found after data has moved means fixing code, fixing data and testing again. For small changes, VAON shortens this step.
Why not let the AI answer the open spec questions itself? Because the AI will choose a plausible answer, not the answer that is right for your company. Approval thresholds, rounding rules and who may see which data are your decisions. A wrong guess there becomes wrong code very quickly. So VAON engineers turn those points into questions and send them to you.
What if my answer widens the scope? The extra work follows the change-control process. VAON sends you the price and the schedule impact in writing, and the work starts only after you approve. If you do not approve, that work is not done and you are not charged for it.
Does the red-team put my source code into an AI tool? Only with your permission. Before work starts, VAON agrees with you which data may be used, who may see it and whether code may go into AI tools. Without that permission, VAON does not put code or documents into AI, and engineers challenge the plan themselves. Slower, but true to the agreement.
One thing you can do today: for the next large change on your project, ask your development team, "If the data migration stops halfway, how do we roll back? Who has read that part of the plan?" If the plan does not answer that, it has not been challenged.
To see how these rounds apply to your project, book a free 60-minute consultation. VAON's CEO joins the call, no documents are needed, and there is no commitment.
