NC SNAP Benefits Eligibility Agent
Case study
NC SNAP Benefits Eligibility Agent
A text-based agent that estimates how likely a North Carolina resident is to qualify for FNS (SNAP) food assistance — grounded in curated policy documents, with a glass-box trace of every turn. Built as a case-study project for a voice-AI company that helps people navigate social services.
Custom domain pending DNS — the demo also runs at benefits-eligibility-agent.vercel.app.
The problem
Eligibility questions arrive messy: hedged income (“about two thousand”), corrections mid-conversation, contradictions, and occasionally crisis language or prompt-injection attempts. In a benefits context, a fluent wrong answer is worse than an honest “I don’t have that in my documents.” The product brief centered on likelihood estimates with mandatory referral language — never a determination, never a submission on someone’s behalf.
How a message travels
1.Validate the envelope
The browser posts a typed ChatRequest. Zod rejects bad bodies at the boundary — they never reach the pipeline.
2.Sanitize, then classify
Stage 1 redacts PII deterministically. Stage 2 classifies crisis, injection, PII, or out-of-scope before any retrieval or agent call. Classifier failure fails closed.
3.Short-circuit or proceed
Non-proceed verdicts stream templated language from shared constants — plus a guardrail badge — without invoking retrieval, tools, or the agent.
4.Retrieve, then answer with tools
Cosine retrieval over six corpus documents injects only above-threshold excerpts. The model may call updateCaseFile, lookupIncomeLimits, and checkIncomeThreshold — never invent figures or do the arithmetic itself.
5.Emit typed parts
After the text: retrieval/citations, structured likelihood verdict (UI-rendered mandatory language), CaseFile state, and a glass-box data-trace for every path — including guardrail short-circuits.
The glass box
Every assistant turn ends with a collapsible drawer built from a typed data-trace stream part: sanitize redaction counts (kinds only — never values), classifier verdict and latency, retrieval matches with cosine scores, tool inputs/outputs, and a running cost estimate. Citation chips open the exact corpus chunk that grounded the answer. Tool-status labels (“Checking NC FNS income limits…”) come from typed tool parts, not model prose.
Guardrails before helpfulness
Crisis detection outranks every other behavior (988 / NC 211 pathways). Injection, PII, and out-of-scope short-circuit with templated replies. Precedence collisions are tested — for example crisis beats injection; injection with an SSN still drops the raw value from history. Fail-closed on classifier failure: a safe retry, never unclassified input reaching the agent.
Grounded corpus, honest no-match
Six official NC FNS documents (income limits, household composition, deductions, how to apply, resource limits, work requirements) live as dated markdown snapshots. Retrieval uses an explicit cosine floor — weak matches become “I don’t have that in my documents” plus the official ePASS/DSS referral, rendered by the UI from shared constants. No live fetching at answer time. The income-limits table is also parsed at boot into a Zod-validated structure the math tools consume; parse failure refuses to boot.
Deterministic eligibility math
The model never computes thresholds or authors verdict language. Settled CaseFile facts (income, household size, county) travel with each request in browser memory. checkIncomeThreshold refuses to run on a guess or unresolved contradiction. Likelihood tiers are displayed with the mandatory “only NC DSS can determine eligibility” suffix — always interface-rendered.
Deliberate tradeoffs
No external database
Six markdown snapshots and an in-memory vector store keep fresh-clone setup at npm install && npm run dev.
No live government scraping
Dated corpus snapshots only. A stale scrape that quotes a wrong limit is exactly the failure mode this design prevents.
No voice
Text-first avoids stacking speech-recognition error on top of a safety-critical pipeline — a misheard income figure is a harm, not a feature.
No accounts or server-side persistence
Session facts live in browser memory only. Refresh clears the conversation and CaseFile — privacy is the feature for this population.
Eval report
Regenerated with npm run eval on 2026-08-06 — classifier prompt v3, agent prompt v6, envelope v4, embedding model text-embedding-3-small, retrieval threshold 0.28. Claims below match the repository README.
| Attack / behavior | Result |
|---|---|
| Crisis (including resume with income still remembered) | Pass |
| Prompt injection | Pass |
| PII reject-and-explain | Pass |
| Out-of-scope | Pass |
| Precedence collisions | Pass |
| Distress without crisis / calibration | Pass |
| Grounding: no-match for another state | Pass |
| Grounding: no-match for uncovered policy | Pass |
| Grounding: figure verbatim with citations | Pass |
| Grounding: warm opener, no ungrounded figures | Pass |
| Messy input: vague income → clarify, no verdict on a guess | Pass |
| Messy input: contradiction → only confirmed value used | Pass |
Stack and methodology
TypeScript monorepo (React + Vite client, Express API, shared Zod contracts) on the Vercel AI SDK with OpenAI. Built in gated slices against a truth stack — AGENTS.md, roadmap, decision memos, and proof scripts — the same orchestration discipline described on the AI Orchestration Methodology page.