Across Wayfair, we're investing heavily in applied AI, piloting different tools and approaches across design, engineering and product to make everyday work faster and better. This is the story of one of those experiments, and it starts with a chore every product team knows: design QA, the check between "built" and "shipped" where a designer compares the coded screen against the Figma design and flags what drifted. A heading a few pixels too small. A link that picked up the wrong font. Spacing that collapses the moment one more component lands on the page.
We estimate that this check costs each of our designers roughly a week per quarter, done manually. So, we set out to hand the comparison to AI to give designers that week back for problems only they can solve, and give developers findings they can fix without a meeting.
What We Measured
Before the story, the numbers. These are from our pilots of the final version, with a designer reviewing every result:
- ~70 audits across eight products, spanning web and mobile, for both internal and external users
- ~1,251 discrepancies identified, each cross-checked by a human
- ~90% precision and ~86% recall on those early, deeply reviewed pilots
- About five minutes and roughly $1 per audit, around a tenth of what our first automated version cost to run
One important caveat: The "week per quarter" figure is our estimate of manual effort, not measured time returned. What we've measured is how fast, cheap and accurate the audit itself is.
How It Works Today
A developer or designer types "/design-QA" in our AI-native code editor. The skill reads the Figma design and the live build, then returns a table of mismatches with the exact file and property to change. Typing "/Fix" applies those changes in the code. The back-and-forth that used to run between a designer, developer and PM now runs between the skill and the codebase.
Three Attempts, One Skill
We didn't land here on the first try. It took three versions, and each one existed to answer a question the previous one couldn't.
The manual version proved it was possible. We started by mapping the real process: what happens between a designer handing off a feature and a developer resolving QA feedback, and what lives in our repos along the way. Then we replicated that process with AI, one prompt at a time. It worked. The AI could compare a design to a build and find real discrepancies. But it also revealed a second cost we hadn't planned for. Every run burned a lot of compute (tokens, the unit AI usage is billed in), needed both a designer and a developer standing by and produced findings we then had to rework by hand. We now had two costs to drive down at once: designer hours and the cost of the check itself.
The measured kit told us what to trust. Instead of jumping straight to something lighter, we built something heavier on purpose. Think of it as the instrumented version: a pipeline that inspected every element on the screen and scored each finding against a rubric, so we could see exactly where the AI was right, where it was wrong and why. Was it misjudging the layout? Inventing color values? Missing accessibility issues? We also ran repeated end-to-end audits, with no human stepping in, to measure how long a fully hands-off run really took and what it cost.
Two things came out of this. First, a map of the failure modes, which told us which checks were reliable and which needed guardrails. Second, an uncomfortable discovery: Our Figma files weren't AI-ready. Older files were full of hard-coded values and one-off shapes, where a tool can't tell an intentional choice from a stray mistake. A file built from named, reusable components with colors and spacing bound to design tokens is a different story; the tool can read design intent directly instead of guessing at it from pixels. So alongside the audit we built a small design-readiness check to get our files up to that standard.
The skill made it cheap enough to run on everything. With the failure modes mapped and the cost leaks found, we distilled the kit down to a single skill that runs inside the editor. Same quality bar, roughly a tenth of the cost: about five minutes and $1 per audit. We trust the simple version precisely because we built and measured the complicated one first. As Paul Rand put it, "Design is so simple. That's why it's so complicated."
What the Pilots Say
This is the part that matters most, so here is exactly how we measured it.
We ran about 70 audits with the final skill across eight products, covering web and mobile surfaces used by both internal teams and external customers. Every finding the skill produced was reviewed by a designer and marked as one of three things: confirmed, false alarm or won't-fix.
From that, we tracked two numbers.
Precision asks: Can I trust a finding? It's the share of everything the skill flagged that turned out to be a real, actionable defect: confirmed ÷ (confirmed + false alarms + won't-fix). We deliberately counted won't-fix items against precision rather than ignoring them, so the number reflects what a developer would actually experience.
Recall asks: Is it safe to rely on? It's the share of real defects, as found by a full human audit of the same screen, that the skill caught.
On these early pilots, precision landed around 90% and recall around 86%. Those are pilot numbers with a human still in the loop, not a validated long-run average, and we track them per pilot on a live dashboard so we can pause and adjust if the signal slips for a particular team. So far, across eight products, we've needed to customize the skill once, for the team working on our mobile interface, and that took a single prompt.
The Real Goal: Engineering Design QA Out of the Process
We're not just trying to make design QA faster. We're trying to make it unnecessary.
The mechanism is simple. Every fix we push upstream, into a shared token, component, or template, removes a whole category of bug before it can happen again. As our Figma files become more structured, and as design tools and code editors get better at reading each other directly, the skill has less to untangle: fewer issues to find, fewer to fix and a cost per audit heading toward pennies. We're also pointing it at legacy products to clean up screens that shipped years ago. None of this is finished, and we're not claiming it is. But the direction is clear, and every release moves the floor down a little. Our core metric going forward isn't how good the tool is. It's how little we need it.
What we used: An AI-native code editor with a design-tool integration for reading Figma, plus a browser inspection step for reading the live build. Nothing in the approach depends on a specific vendor; it should port to any comparable setup.
Note: Design QA at Wayfair is an ongoing experiment in applied AI. The figures here reflect pilot results specific to our own products and workflows, and the team is continuing to validate performance over time.
