Turning free-text complaints into routed, auditable, reportable cases

The situation

Grievances arrived through a call centre, a web and mobile portal, and paper forms scanned in from local offices, in mixed languages and wildly varying specificity. Staff read each one, decided which department it belonged to, and logged a summary before routing it on. Volume grew faster than review capacity, categorisation varied by which officer happened to read a given complaint, and assembling the monthly performance report for leadership took a team most of two weeks, working from spreadsheets that frequently did not agree with each other.

The constraint

• A mis-routed grievance is not an abstract error rate, it is a real delay for the person who filed it, so accuracy carried more weight here than in most classification problems.
• The system had to run inside a government-approved data environment, not call an external API, so classification runs on a self-hosted, open-weight model rather than a commercial one.
• Personal information — names, addresses, sometimes ID numbers — appears inside the free text of a grievance, not only in structured fields, so redaction had to work on unstructured text.
• Leadership needed a reporting number they could stand behind in a review meeting, so a system that quietly recategorises and produces a different total each week would have been worse than the manual process it replaced.

What we built

Classification with a citable reason

The model assigns department, category and urgency, and must point to the specific phrase in the grievance text supporting each field — the same structured-with-citation approach we use for contact centre call extraction, applied here to public complaint text. A reviewing officer checks a routing decision in seconds instead of re-reading the whole complaint.

A graded set built with the officers who already do this

Experienced grievance officers helped build a graded set of real, anonymised grievances with agreed correct routing and category. It became the acceptance bar and the regression suite for every model change, the same discipline we apply to any production classification pipeline.

Redaction inside the free text

Personal identifiers inside the grievance body are detected and masked before the text is visible to anyone beyond the assigned handling officer. The redaction model is evaluated against its own graded set rather than assumed to work because the extraction model does.

SLA and escalation that matches how the organisation holds itself accountable

Routing carries a department-specific SLA clock and an automatic escalation path once a grievance ages past threshold, visible to supervisors — replacing a manual follow-up spreadsheet with workflow the system enforces on its own.

Reporting as a query, with a fixed definition

A trend and performance layer by department, category, region and ageing, built on metric definitions fixed in code, so the monthly leadership number is reproducible and comparable month over month instead of reassembled from source every time.

What changed

First-pass routing accuracy improved markedly, officer-confirmed against the graded set, and the monthly leadership report went from most of two weeks of manual assembly to same-day. Median time to first assignment fell by more than half, which is the number that actually shortens a citizen's wait.

What we would do differently

The graded set was built from grievances that had, by definition, already been resolved correctly by hand, which skewed it toward the clearer cases. The model scored well in evaluation and then struggled on the genuinely ambiguous grievances that experienced officers route on institutional memory the text alone does not capture. We should have deliberately oversampled ambiguous, borderline cases into the graded set from the start, rather than discovering the gap after go-live.

Outcomes

Grievances routed correctly on first pass

a marked, officer-confirmed improvement

Assembling the monthly leadership report

weeks → same day

Median time to first assignment

cut by more than half

Client identity withheld under a mutual NDA. Figures are illustrative — rounded and directional, meant to show the shape of the change rather than an audited result. We will walk through the real numbers, and how they were measured, under NDA on a call.