Project case study
AI Quality and Release Verification
At The Home Depot, I built a workflow to check the numbers in AI-generated sales presentations, turn discrepancies into engineering handoffs, and verify that fixes were correct for the business.
My role: Validation design, business verification and workflow development with AI coding agents. Engineering implemented product fixes.
Watch the interactive walkthroughHow it works
Extract the claims
Printed figures become a list of checkable claims.
Slide tables, tiles and sentences are staged as structured claims. Model-assisted extraction identifies the claim without deciding whether it is true.
Check two paths
Known metrics and assisted checks stay separate.
Tested queries validate defined metrics. Assisted queries handle other claims; suspected failures can be checked with a second query. Undefined metrics are refused.
Recheck the fix
Before, expected and now expose a failed correction.
The verification workflow compares revised output against acceptance criteria and drafts the handoff. A person reviews and posts the result.
Define the metric
One agreed formula gives future checks a target.
A metric card records the formula, scope and owner. The example then recalculates the result; signing a card alone does not change a dashboard.
A fix is not finished because a ticket says it is.
In my Home Depot work, I connected the checker to the process around it. Findings became a shared issue board and a handoff page. A verification run compared the previous output, the expected value and the revised output. My job was to establish what was wrong and whether the fix held up; engineering owned the fixes.
- Focus
- Claim extraction, query-based validation, issue tracking and release checks
- Status
- Checker and workflow shipped internally; metric governance in progress
How it works
Structured claims enter a staging table. Tested queries check defined metrics; an assisted path handles other claims without blending the two counts. Defect cards carry expected behavior into a review workflow, and subsequent output is checked against those same criteria.
A decision that mattered
I kept tested metric queries separate from AI-assisted queries and made unknown definitions visible. I also counted extractable numbers inside sentences, even when that made coverage look worse. Later, tracing defects back to inconsistent definitions led to a proposal for owner-approved metric cards.
Status
The checker and verification workflow were shipped internally; metric governance is still being developed. The fictional walkthrough shows the method, not employer accuracy or time-saving results.