Passing tests doesn’t mean AI-generated code is ready for production. As AI codi
By Yananchenko Ss · November 18, 2026 · Curated by George's Blog
Passing tests doesn’t mean AI-generated code is ready for production.
As AI coding agents become part of everyday software development, the challenge is no longer just generating code. It’s making sure the result works reliably.
Anthropic recently shared how the Claude Code team approaches verification. One point stood out to me: passing individual tests doesn’t necessarily mean a feature works as expected. You need to verify how the entire system behaves.
Here’s how I’d apply their approach to eCommerce:
1. Test the whole journey.
A cart that works and a payment module that works can still add up to a checkout that doesn’t. End-to-end tests go through the store the way a customer does: product page, cart, checkout, payment, confirmation. If the customer can’t finish the purchase, it doesn’t matter how many unit tests passed.
2. Try to break your own rules.
Every store has rules that must never break: an order total can’t go negative, a discount can’t exceed the price, a refund can’t exceed what the customer paid. Property-based testing takes a rule like that and throws thousands of generated inputs at it to see if anything breaks it. Bundles, tiered pricing and stacked promos are exactly where those surprises hide.
3. Compare before and after.
Stores that have been live for years carry a lot of history: custom logic, app integrations, workarounds nobody remembers adding. Change one thing, and something three layers away can shift. Running the old and new versions on the same inputs shows exactly what changed. Some differences are intended and some aren’t, and the point is to sort them out before customers notice.
4. Don’t fix the same bug twice.
If the same type of issue slips through more than once, a fix isn’t enough. It should become a rule or an automated check. Just review that list regularly, or it turns into noise.
5. Know when to hand it to a human.
Claude gets three rounds to fix a failing check. If it still fails, a person steps in. Just as important: the code being reviewed can’t touch the rules it’s reviewed against. If an agent can rewrite its own tests, passing them proves nothing.
The effectiveness of these practices depends on one thing that’s often overlooked: understanding the business logic behind each store. An AI agent doesn’t automatically know that wholesale customers have different pricing, that certain products can’t be shipped to specific regions, or that some promotions can be combined while others can’t. These rules come from the merchant and the team. Without them, even well-designed tests can miss issues that directly affect the customer experience and revenue.
For teams building with AI, speed of development and delivery is only one side of the equation. Just as important is having a reliable way to verify that the result actually works as intended.
#AI #AICoding #eCommerce #SoftwareDevelopment #QualityAssurance