Illinois bans the zip-code field the EU AI Act wants kept. I built an AI hiring audit that says so instead of printing a pass.

On December 2, 2025, the New York State Comptroller published a review of the same 32-company sample the city's Department of Consumer and Worker Protection had already looked at under Local Law 144, and counted 17 potential violations where DCWP had found one. DCWP agreed to move to proactive enforcement.

I read that and assumed the hard engineering problem was detection: build something that catches the bias the first pass missed. Writing the rule packs changed my mind, because detection turned out to be the easy half. The hard half is that an employer running automated hiring tools across New York, Colorado, Illinois, Texas, California and the EU is answering to six regulators who want six differently shaped documents, and in at least one place two of them want opposite things from the same column of data.

So I built a compliance overlay for that, a console I named Clarion that sits on top of the AI hiring tools an employer already runs, reads one vendor scoring export, computes every adverse-impact statistic in deterministic code, and fans that single audit into six jurisdiction-shaped deliverables. You can see how it works at veriprajna.com/demos/ai-hiring-compliance. Everything below runs against a seeded synthetic export of 1,040 candidate records for a fictional employer, "Acme Logistics, Inc.", on requisition REQ-2026-0412. I planted the violation in that data myself, which is the only reason I can tell you exactly what the engine was supposed to find.

The audit that passes is not the audit the law asks for

I seeded the dataset, so the first result still managed to annoy me. The marginal four-fifths test, the one a vendor self-audit ships, passes cleanly: minimum race impact ratio 0.8196, with Hispanic and Black tied at that minimum, and minimum sex ratio 0.8744. Both sit at or above the 0.80 line. Nothing in the code asserts that pass. The 0.8196 is what the computation returns on the fixture, and it clears the line on its own. On screen the rate card says it plainly: the vendor stops here.

Local Law 144 does not stop there. It asks for intersectional race by sex ratios, and the synthetic candidates are scored by three simulated tools (a Workday-Spotlight-style scorer, a HireVue-style video round, an Eightfold-style match engine) that are fixture adapters over that synthetic data, archetypes rather than integrations. Cut the same records by race and sex and the picture inverts. Against the White / Male reference cell, which advances 68 of 130 candidates at 52.31%, the Black / Female cell advances 44 of 130, or 33.85%. Impact ratio 0.6471. Hispanic / Female fails too, at 0.7647.

Hispanic / Female is where I argued with my own engine. Under Benjamini-Hochberg false discovery rate control at alpha 0.05, only the Black / Female cell is statistically robust, at q = 0.0185. Hispanic / Female lands at q = 0.1629, so the engine flags it and explicitly declines to call it significant. I wanted both cells to count, because two failing cells is a better story than one. The engine distinguishes a robust finding from a chance one, which is the same discipline I would want applied to my own numbers if I were the employer on the other side of the report.

Six regulators, six different documents

I wrote the six rule packs one after another and kept trying to collapse them into a single score, because one number is what a dashboard can hold and what a buyer expects. It never survived contact with the statutes. NYC wants an intersectional adverse-impact report plus a posted public summary. Colorado's SB 24-205 wants a documented reasonable-care impact assessment and a risk-management program, and specifies no methodology at all, in force June 30, 2026. Texas TRAIGA rejects disparate impact as a standalone basis and asks about intent. California's FEHA ADS amendments have been in force since October 1, 2025 and ask for their own documentation shape again. The EU AI Act treats recruitment as Annex III high-risk and asks for Article 10 data governance plus an Article 11 technical file, from August 2, 2026.

The single score died on the Texas pack. I had already built the NYC deliverable out of intersectional ratios, and TRAIGA makes those exact statistics evidentiarily irrelevant to its own assessment, so there was no weighting I could apply that let one number mean both things. Six different questions, asked in six different shapes, and a fairness score answers roughly one and a half of them. So Clarion produces six deliverables, each carrying its own citation, effective date and required shape, and the console's headline tiles report the coverage rather than a grade: Jurisdiction Deliverables 6, Lowest Impact Ratio 0.65, Items Requiring Human Action 9.

That mixed column is the honest output, and it is also the one nobody is shipping. Researchers from Cornell, Data & Society and Consumer Reports checked 391 NYC employers for the audit Local Law 144 requires and found published bias audits for 4.6% of them (FAccT 2024). The obligation is in force. Compliance with it is close to a rounding error, and my read is that a lot of that gap is employers who genuinely believe their vendor's single passing ratio settled the question.

Cramér's V 0.3321 on zip_region, and two regimes that want opposite things

I wrote a resolver for the zip-code problem before I understood the zip-code problem, and deleting it was the moment the product changed shape. The engine detects protected-class proxies by correlation, and zip_region comes back at Cramér's V 0.3321 against race, 0.3253 bias-corrected, over the 0.2 threshold. It is a proxy. This is not a flag-everything heuristic either: school_tier sits at 0.0711 in the same run and clears.

Illinois HB 3773, live since January 1, 2026, bans zip codes as protected-class proxies, so the Illinois-safe configuration masks geography. EU AI Act Article 10(3) requires training data that is relevant, representative and complete, which in practice leans on the geographic coverage Illinois just told you to remove. Mask the field and the EU representativeness obligation weakens. Keep it and Illinois is violated. One model configuration cannot satisfy both.

My resolver had picked a winner. It compared the two exposures, chose the larger one, and emitted a clean status for the other side, which is exactly the behavior that puts a false statement in a filed document. I took it out and replaced it with a verdict the gate is allowed to reach: CONFLICT. The reconciler then writes a legal-strategy memo instead of a certification. Run two deployment configurations, a full geographic mask for Illinois inference and a coarser region feature for EU training data. Or, if one configuration is mandated, pick the regime with the larger exposure and document the accepted risk, noting that EU high-risk penalties reach the greater of 15 million euro or 3% of global annual turnover as a statutory maximum.

A dashboard that says compliant is, in a courtroom, an exhibit. A memo that says we knew, we priced both branches, and we chose is a defense.

That is the trade I would make as a General Counsel every time, and it is the trade no product built to reassure a buyer will offer, because the reassuring version sells better right up until the deposition.

I tried to talk my own system into a pass

I ran the experiment I was most afraid of, which was to find out whether I could prompt my way to a better verdict. There are six jurisdiction agents in Clarion, plus a conflict reconciler and an adversarial skeptic, built on Pydantic AI and provider-swappable. I rewrote the narration prompt to lean generous, the way a vendor under quarterly pressure would lean generous. The prose got warmer. Not one number moved and not one verdict changed, because every statistic, every threshold comparison and every gate decision lives in engine.py and rulepacks.py, outside the agent framework entirely.

The same property shows up the other way. With no model provider configured at all, the crew falls back to deterministic templates and the app runs identically, same six deliverables, same 0.6471, same CONFLICT. Agents advise, code decides, and 12 of 12 engine tests pin the planted ground truth so that the marginal pass and the intersectional failure cannot drift apart between runs.

Agents can read a statute and write a memo. They cannot be the thing that decides a threshold was crossed, because their agreement is purchasable by a better prompt.

I did not start with that view. I started thinking the agents were the product and the arithmetic was plumbing, and I had it exactly backwards. I swapped the provider, swapped the prompt, ran it with no key at all, and the engine still returned 0.6471 at Black / Female and CONFLICT on zip_region. Anything I improve about the model improves the memo's prose and nothing an auditor would check.

Three legal theories a passed bias audit never touched

I built the adversarial skeptic expecting it to argue about statistics, and it spent its time arguing about scope instead. It refuses to green-light three things, and each one is a separate legal theory that a passed bias audit does not cover.

The first is self-classification. A vendor memo asserting "our scorer is not an AEDT" gets rejected, because under the agent theory raised in Mobley v. Workday a tool that recommends or filters candidates is inside the decision. That theory is undecided, not a holding, which is why the item goes to human counsel for a scope attestation rather than to a status color.

The second is accessibility. On the 432 candidates who went through the video round, word error rate is 0.0794 for standard speech and 0.3016 for the 104 candidates with non-standard speech, a 3.8 times disparity that Local Law 144 never tests, because LL144 is about race and sex. The theory raised in D.K. v. Intuit/HireVue is an ADA theory, and it survives a perfect bias audit untouched. Clarion detects the disparity and routes it to a human ADA review. It does not build the accommodation workflow, and I am not going to pretend otherwise.

The third is FCRA, which does not care about fairness at all. In this export, 510 candidates were scored from third-party-scraped data and filtered on a numeric score, the pattern at issue in Kistler v. Eightfold. If the platform is a consumer reporting agency, every scored candidate is owed an adverse-action notice and a dispute path regardless of how balanced the outcomes are. Clarion detects the trigger and routes it to adverse-action and dispute infrastructure. It does not build the candidate-facing portal either. I find this the least intuitive of the three, and the one most likely to be missed by a team that has done everything right on fairness, because nothing about the statistics tells you the answer. The question FCRA asks is about what the platform is, and a company can pass every impact ratio in the report and still owe several hundred people a notice it never sent.

The package I would hand to whoever signs it

I kept one question taped in front of me for the entire build: what does the person who actually signs this need, and it is not a score. Across the six regimes the engine evaluates 13 obligations and returns 2 PASS, 2 FAIL, 7 NEEDS_PROOF and 2 CONFLICT. Stated as coverage on this seeded synthetic export, 2 of 13 obligations are auto-satisfied by data alone, and 9 of 13 are routed to a named human because they are NEEDS_PROOF or CONFLICT. On a system sold as automation that ratio looks like failure. As an audit it is the only shape I would sign my name under.

The export is a SHA-256 hash-chained bundle of 17 nodes, each node linking to the hash of the node before it, so any edit to any node breaks the chain. It emits as JSON and as a printable HTML packet, with every number carrying its inputs and its computation and every verdict carrying its citation. The chain verifies on each run and the tamper-evidence is unit-tested.

Veriprajna does not sign this. The independent Local Law 144 auditor role belongs to firms like DCI, ORCAA and Secretariat, and the entire design goal of the package is that one of them can sign it without rewriting it first. The walkthrough of the full run, including the conflict memo, is at veriprajna.com/demos/ai-hiring-compliance.

And if you would rather watch the gate refuse than read me describe it, here is the whole run end to end, conflict memo included.

The conflict memo leaves an employer exactly as compliant as they were the day before. What it adds is a trade-off that is legible, dated and attributable, so that two years from now, when someone asks who decided to keep the zip field and what they knew when they decided it, there is a document that answers instead of a green badge that has to be explained.