Deployed at a top-10 Hong Kong insurer
Two AI tools I built for one client, described from the users’ side: what their day looked like before, what I built, and how we know it helped.
1The client and its constraints
The client is one of Hong Kong’s ten largest insurers. Three things about it shaped every design decision.
Bilingual work
Staff move between Cantonese and English, often within one sentence.
Confidential data
Meetings and claim files contain customer and business information.
A regulated business
Any output may be audited later, so a person has to stay responsible for it.
2Deployment one: meeting minutes
Before
After each meeting, a staff member typed up the minutes by hand, then copied them into the firm’s Word template. General-purpose transcription tools didn’t help much. They tend to turn spoken Cantonese into formal written Chinese, which reads like Mandarin, and they lose the English terms people drop into their sentences.
What I built
- Speaker separationwho is speaking, and when
- Split at pausesshorter pieces, cut at quiet moments
- Cantonese transcriptioncolloquial Cantonese, English kept in English
- Structured summarydecisions, action items
- The firm’s templatethe Word minutes staff already use
The speech model is trained on Cantonese. A short prompt tells it how to write: colloquial Cantonese, with English words left in English. Users can also add a list of company terms.
Result
I benchmarked the transcription on a public test set of 266 clips of real Hong Kong speech from TV and interviews, where speakers mix Cantonese and English. The error rate is the share of Chinese characters and English words that came out wrong, missing or extra. The comparison is WhisperX, a widely used open-source pipeline running Whisper large-v3. Every setup used the same speaker-separation step, so the comparison isolates the transcription itself.
| Setup | Error rate lower is better | English words correct |
|---|---|---|
| WhisperX, default settings | 27.9% | 46% |
| WhisperX, with my Cantonese prompt | 17.9% | 53% |
| My pipeline, with a Cantonese-trained model | 9.3% | 82% |
The middle row shows where the gain comes from. The prompt alone removes about a third of WhisperX’s errors on the same model. Switching to a model trained on Cantonese then halves what is left.
The company-term list wasn’t tested, since it doesn’t apply to public TV clips. These clips also aren’t the client’s own meetings, so the figures show how well the pipeline handles the language, not how accurate any particular meeting’s minutes are.
3Deployment two: claim emails to Excel
Before
A claim is a customer’s request for the insurer to pay out under their policy. It arrives as an email with attachments: forms, receipts, medical documents. A claims handler, the staff member who assesses it, opened each attachment, found the relevant evidence, and typed it into an Excel workbook used to assess the claim. Each case took about 23 minutes, and most of that went to reading and retyping rather than judging the claim.
What I built
An AI agent works through the same steps the handler did.
- Readthe email and every attachment
- Pull out evidencedates, amounts, diagnoses, policy details
- Sort into fieldsthe fixed set the assessment needs
- Fill the workbook
- Handler reviewschecks and decides
Result
The client timed 30 cases. With the agent, each took about 5 minutes, down from 23. The handler’s time now goes to checking and deciding instead of copying.
4What deploying taught me
Fit the format people already use
Staff already had a minutes template that their managers expected. Output in a new format would have meant more work for them, so the pipeline fills in their existing template instead.
The agent prepares, the person decides
In a regulated business, someone has to answer for every claim decision and every set of minutes. Both tools stop at a draft. The handler or the minute-taker reviews it and signs off.
Fit the language before reaching for a bigger model
A short prompt about how to write Cantonese cut errors on the same model from 27.9% to 17.9%. The next big gain came from a model trained on Cantonese, not from a larger general-purpose one.
5What I’d measure next
The benchmark and the timing show that both tools work. They don’t show whether people keep using them, and no usage data has been collected yet. Next, I would track:
- How long each tool has been in use, and whether usage holds up after the first few weeks.
- Cases per week going through the claims agent, to turn the per-case saving into hours saved per team.
- The fields people edit most in the drafts. Those edits show exactly where the tools still get things wrong, and therefore what to fix first.