Business Audit Tool

The problem

An agency audit of a business’s digital presence takes hours of manual checking and produces inconsistent, vibes-based grades. Automating it with an LLM alone makes it worse: the model will confidently invent findings. The tool had to be fast, repeatable, and impossible to bluff.

What I built

A full-stack product: enter a business, and a multi-agent pipeline crawls and analyzes its digital presence, then produces a graded report. Split web/worker architecture with a Postgres job queue; 226 passing tests.

Decisions worth noting

• The LLM never decides the grade. All A–F grades come from a deterministic weighted scoring engine. Claude extracts signals and writes prose — language tasks only.
• Code-enforced grounding. Every AI claim must carry evidence from the actual crawl, or a grounding layer strips it before the report. Hallucination prevention is structural, not a prompt instruction — the system fails closed.
• Model tiering. Haiku-class models do extraction; Sonnet-class does synthesis. Prompt-cache write/read pricing is modeled explicitly, which is the difference between a demo and a product with a cost envelope.
• Redelivery-safe workers. pg-boss jobs use atomic-claim idempotency, so a redelivered job can’t double-process an audit.

My role

Solo-architected and built at Alyne in a short, heavily AI-assisted sprint (95 commits in two days) — Claude Code wrote much of the implementation; the architecture above is what I own and can defend in detail. It’s an internal Alyne tool: the source stays private and there’s no public demo — this write-up is the case study.