Screening insurance applications with an LLM that never decides

Insurance intake is mostly transcription. An application arrives as a filled-in form with free-text fields and a handful of uploaded documents, and before anyone can make a judgment call someone has to read all of it, pull the relevant values into a shape they can compare, and notice the parts that don't line up. On Insur — where insurance organizations build their own application forms and run submissions through a review pipeline — that first-line pass is the bottleneck. It's also exactly what a language model is good at.

The obvious next step is the wrong one. Once a model can read an application well enough to summarize it, it's tempting to let it approve the easy ones. Don't. A denied application is a decision a person has to be able to explain, and "the model was fairly confident" is not an explanation. So we built the screening stage to be structurally incapable of deciding anything.

The model is a stage, not a branch

Submissions move through a fixed pipeline:

Submitted → AI Screening → Pending Review → In Review → Approved | Rejected

The important property is that AI screening has exactly one successful exit, and it isn't a fork. The model's output can't route a submission anywhere a human wouldn't have sent it — it only ever lands in Pending Review with more information attached than it arrived with.

That's enforced where transitions are enforced, not by convention:

// Which actor is allowed to perform which transition.
const ALLOWED_TRANSITIONS = {
  system: {
    SUBMITTED: ["AI_SCREENING"],
    AI_SCREENING: ["PENDING_REVIEW"], // the only exit. no approve, no reject.
  },
  user: {
    PENDING_REVIEW: ["IN_REVIEW"],
    IN_REVIEW: ["APPROVED", "REJECTED", "PENDING_REVIEW"],
  },
};

APPROVED and REJECTED do not appear anywhere in the system map. There is no configuration flag, no auto-approve threshold, no "confidence above 0.95" shortcut waiting to be turned on in a future sprint. If someone later decides the model should be able to close out clean applications, that's a code change that shows up in review — which is where a decision like that belongs.

Three outputs, three levels of trust

A screening run doesn't produce "a result". It produces three things, and they are not equally trustworthy:

type ScreeningResult = {
  // 1. Extraction — structured, checkable, the highest-trust output.
  fields: Record<string, ExtractedValue>;
  // 2. Anomalies — claims about the application that a human must confirm.
  anomalies: Anomaly[];
  // 3. Summary — prose for a reader. Never parsed, never acted on by code.
  summary: string;
};
 
type ExtractedValue = {
  value: string | number | null;
  sourceField: string | null; // which form field or document it came from
};

Extraction is checkable — a date is a date, a policy amount is a number, and most of it can be validated against the form's own schema. Anomalies are assertions, and assertions can be wrong. The summary is the loosest output of the three, which is why nothing downstream reads it. It is text for a person, and it is treated as text for a person: rendered on the review screen, never parsed, never used as a key, never fed into another decision.

Keeping those separate matters more than it sounds like it should. The moment one blob of model output serves both the reviewer's eyes and the application's control flow, a hallucinated sentence stops being a bad paragraph and becomes a bug.

Constrain the output, then validate it anyway

We ask for structured output — a JSON schema on the request, so the model is constrained to the shape we want. That gets you well-formed JSON. It does not get you correct JSON, and it doesn't survive a bad response, a truncated generation, or a model version that quietly starts interpreting a field differently.

So every response is parsed and validated on the way in, and a validation failure is a screening failure, not a partial success:

const result = ScreeningSchema.safeParse(raw);
if (!result.success) {
  return this.degrade(submissionId, "Screening output failed validation");
}

The other rule that does a surprising amount of work: null beats a guess. ExtractedValue.value is nullable and the prompt is explicit that an absent value should come back as null rather than an inference. A blank field on the review screen tells a reviewer "read this one yourself." A plausible wrong number tells them nothing at all, because they have no way to know it's wrong without doing the work the extraction was supposed to save. Fields carry sourceField for the same reason — a value a reviewer can trace back to where it came from is a value they can accept in a second.

Anomalies need evidence, not adjectives

The flagging half of screening is the part most likely to be useless. A model will happily tell you a submission "appears unusual," which costs a reviewer time and gives them nothing. So an anomaly is only allowed to exist if it points at something:

type Anomaly = {
  field: string;   // what it's about
  reason: string;  // why, in terms of the application's own contents
};

Both are required. A flag that can't name a field doesn't get emitted, and one that can't state a reason grounded in the submission's contents gets dropped in validation. The trade is quantity for signal: fewer flags, but each one cheap to confirm or dismiss instead of re-deriving from scratch.

Dismissal is part of the design too. Every anomaly is something a reviewer can wave off, and their doing so is recorded against the submission. A flag is a suggestion with a receipt, not a mark on the file.

Failure is a state, not an exception

The screening stage sits between an applicant hitting submit and a human opening the submission, which means it is the one part of the pipeline that absolutely cannot become a hard dependency. Models time out. Providers have bad afternoons. Rate limits exist.

So screening degrades instead of blocking. On a failure — timeout, validation error, provider error, anything — the submission still moves to Pending Review, with the screening block empty and a visible note explaining that it wasn't screened:

private async degrade(submissionId: string, note: string) {
  await this.submissions.transition(submissionId, "PENDING_REVIEW", {
    screening: null,
    screeningNote: note,
  });
}

A reviewer opening that submission sees an application with no AI assist, which is exactly the job they were doing before any of this existed. The work gets slower, never stuck. Runs are keyed by submission id so a retry can't produce a second screening block, and the whole thing is queued rather than inline — nobody's submit button waits on a model.

That's the shape of the trade: screening is worth building because it removes the transcription pass, and it's only safe to build if the product is still correct when it's completely absent.

What this is built to buy

The goal is to take the first-line pass close to zero — not because the model decides, but because the reviewer stops doing data entry before they can start doing their job. They open a submission and the values are already laid out, the things worth a second look are already pointed at, and the free-text is already summarized. The decision itself is untouched. It's still a person, still explainable, still theirs.

That distinction is the whole design. The number of judgments being made doesn't change; what's removed is the work that happens before the judgment.

Worth stating plainly: Insur is still in development, and this pipeline hasn't been put in front of a live review team yet. Everything above is how the system is built and why — not a measured result. The screening stage is designed on the assumption that its output will sometimes be wrong, and that assumption is exactly what real reviewers will test.

Where this stops working

This holds because of one economic fact about the domain: a human looking at every submission is affordable. Insurance applications are high-value and comparatively low-volume, so a person per submission is a cost the business can carry, and the model only has to be useful — never authoritative.

Flip that and the design has to change. A workflow with a hundred times the volume and a fraction of the value per item can't put a human on everything, and at that point you need something we deliberately don't have here: real auto-disposition, with the calibration, monitoring, and appeal path that a decision made without a human requires. That's a much larger commitment than a screening stage, and it isn't one you should back into by raising a threshold on a system that was never built for it.

The other direction is more likely, and cheaper: some of what the model does here doesn't need a model. Deterministic rules over the extracted fields handle the boring, unambiguous checks — a missing required document, a value outside a declared range — faster and more predictably than any prompt. The model is worth its cost on the parts that are genuinely fuzzy: unstructured text, inconsistencies across documents, the summary. Everywhere else, the boring answer is still the right one.