Rubric: error taxonomy completeness

Written by a person. Last read by a person on 2026-09-04, 4 days ago. Its facts were checked by the eval suite on 2026-09-07.

Enforced by check_error_taxonomy.

The claim

The error families are human-authored. Each one carries enough to do the job the helper asks of it. It names the likely non-API causes to rule out first. It says what else lives in the family.

Observable evidence

• The provenance value of data/error-families.yaml.
• For each family: id, title, retryable, non_api_first, also_in_family, implies_working.
• Whether any family id repeats.

The pass bar

The file is human-authored and every family carries all six fields, with none empty.

implies_working is the newest, and the one most often written badly. It states what a given failure proves is already correct, and every claim in it must be true of the protocol rather than encouraging. The environment family is the test. Its honest answer is that nothing is confirmed yet. A field that only ever produced good news would be a compliment generator, not information.

The bar is about the taxonomy being usable rather than about it being right. Spec 3.5 makes this file, not the model, the source of truth for the families, so a family with no non_api_first is a classification the helper can make and cannot act on.

Ambiguous cases

Whether the families are the right families. Not checkable, and the more important question. This rubric confirms each family is complete; only use tells you whether the cut is a good one. Change 0004 measures which classifier destinations readers bounce off, which is the closest thing to evidence the system can generate about itself.

A family with no status codes. Passes, and is deliberate. The environment family never reaches the API, so it never produces a status code. That is exactly why readers check it last. It is why they should check it first.

Overlapping families. Allowed. Billing genuinely returns 400, 402 and 429 depending on which limit was hit, and pretending the taxonomy partitions cleanly would make it wrong rather than tidy. The overlap is documented on the troubleshooting page instead of hidden.

A family invented by the model at runtime. No longer possible for the classifier built in 0004. It reads a compiled copy of this file and matches against it, so the set of families it can name is the set written here. A model asked a question the taxonomy cannot classify is a separate path, and it is labeled as one.

The second claim: every family routes somewhere real

Enforced by check_classifier_routes.

The classifier ends every answer with a link to the canonical section, and that link is derived from the family title. The headings it points at are typed by hand on the troubleshooting page, in a different file. They agree today. Nothing in either file makes them agree tomorrow.

The check therefore resolves each link against the built page and fails when one does not land. It also fails when content/assets/families.js, the compiled copy the classifier reads, is behind this file. That copy is generated and committed, so an edit here with no rebuild would leave readers seeing new families on the help page while the classifier still matched the old ones.

The help page tells readers the families "cannot drift from what the classifier uses". This is the check that makes that sentence true rather than a hope.

Where the response order is checked instead

The order the classifier presents is a requirement of its own: what the failure already proves is working, then the causes that are not the API, then the family, then whether retrying helps, then the link. It is asserted in tests/js/classify.test.mjs against the structure the classifier returns, rather than here, because the order is a property of the code and not of this file.

Two of those tests are worth naming. One holds the canonical link last even when a signal matches two families. The other feeds the classifier an input with no status and no error type and requires it to name no family at all, because a plausible family offered for "it broke" reads exactly like an answer and is not one.