WINDAGs Blog

It's 9pm on a Tuesday. You've got two hours before bed and a Stripe webhook that double-charges customers under retries. You ask Sonnet. It answers — confidently, fluently, with a code block and everything. Sometimes it's confidently wrong, in the specific way that ships Friday and pages you Tuesday. That's the tax on confident-but-generic AI: not the hallucinations you catch, the ones you don't.

It's Saturday. You're four hours into a side project. You ask your AI to "set up a Stripe webhook handler with idempotency and a retry policy that doesn't blow up under bursts." It writes you something that looks fine. It misses the signature verification, picks a backoff strategy that'll silently double-charge, and uses an API shape that was renamed in 2024.

You ask your AI to "design a paginated GraphQL endpoint that won't fall over at scale." It reaches for the textbook answer — LIMIT and OFFSET. Which works fine, right up until your dataset hits a million rows and every page-2 request starts scanning the whole table.

Every team has the senior engineer who read the book. The one who can tell you, in twenty seconds, why you should not use that pattern, where the gotchas are, and which page the load-bearing argument starts on. They're an institutional asset. They're also a single point of failure — when they leave, the team relearns everything the hard way.

You roll out an AI tooling guide to the team. It's well-organized, well-written, has examples. Three weeks later, you watch a junior dev follow it step-by-step and produce something that compiles, passes the tests you wrote, and is quietly wrong in a way only a senior would catch on review. The guide explained what things are. It did not say how to decide what to do when the obvious answer is the wrong one.

It's Tuesday at 9:42 AM. You open the laptop. The repo is right there. You stare at it.

It's late. You've been on a long agent session. Your agent took twelve seconds to think before answering an easy question and you watched the spinner. The thing is loaded with skill prompts — 200-line expert documents, dozens of them, every one of them eating context before the model can even read the question. And you start to wonder how much of those tokens is the skill teaching the model something it doesn't know, vs. the skill telling the model things it had already seen a thousand times during pretraining.

TL;DR: We ran two competing AI skill-evaluation tools against each other — Anthropic's open-source skill-creator and our skill-architect. Both violated their own rules. Neither caught what the other caught. Self-evaluation has a structural blind spot that cross-evaluation fills. The experiment converged to A- from both directions.

We have 191 Claude Code skills. They cover everything from Drizzle migrations to Jungian psychology, from drone inspection to wedding photography, from pixel art to HIPAA compliance.