Paper digest: speech AI and Caribbean voices
Whisper transcribes standard English well, but gets most Jamaican Patois words wrong. What three papers show, and how to test speech tools yourself.
Speech recognition software now transcribes standard English almost as well as people do. On Jamaican Patois music, the same software got about 89 percent of words wrong in a 2025 study.
For Caribbean businesses that run on voice notes, phone calls and radio, that gap matters. This digest covers three papers: OpenAI's Whisper, a Jamaican study on Patois music, and a proposed emergency call system built for Caribbean accents.
The short version
Training on huge amounts of internet audio made speech recognition much more robust. But Caribbean Creoles are poorly covered by that data. Fine-tuning on about 40 hours of local recordings cut errors sharply, though not to the level people expect for standard English.
The papers
| Paper | Authors | Published | Link |
|---|---|---|---|
| Robust Speech Recognition via Large-Scale Weak Supervision (Whisper) | Radford and others, OpenAI | December 2022 | arXiv:2212.04356 |
| Towards Robust Speech Recognition for Jamaican Patois Music Transcription | Madden, Stone, Johnson and Geddez | July 2025 | arXiv:2507.16834 |
| TRIDENT: A Redundant Architecture for Caribbean-Accented Emergency Speech Triage | Galbraith, Sutherland and Morgan | December 2025 | arXiv:2512.10741 |
How the research builds
Whisper: learning from the internet
Automatic speech recognition, or ASR, turns spoken audio into text. Older systems needed carefully labelled recordings, which are expensive to make.
Whisper took a different approach. The OpenAI team trained it on 680,000 hours of audio paired with transcripts collected from the internet, in many languages. The abstract reports that the resulting models work well on standard tests without extra training, and approach human accuracy and robustness. OpenAI released the models openly, which is why Whisper became the starting point for many researchers, including those in the Caribbean.
Whisper is built on the Transformer design explained in our Attention Is All You Need digest.
Jamaican Patois: local data makes the difference
Jordan Madden and colleagues collected about 42.6 hours of Jamaican Patois music and transcribed it by hand, the largest public dataset of its kind they know of. They then fine-tuned four smaller Whisper models on it. Fine-tuning means extra training on a specific kind of data.
Here are their best results. Word error rate is the share of words transcribed wrongly.
| Model | Error rate on Patois music |
|---|---|
| Whisper Large, no Patois training | 0.89 |
| Whisper Tiny, trained on 40 hours | 0.70 |
| Whisper Small, trained on 40 hours | 0.41 |
| Whisper Medium, trained on 40 hours | 0.30 |
For comparison, the paper notes that Whisper Large gets about 5 percent of standard English words wrong. Even the smallest model, about 50 times smaller than Large, beat it once trained on Patois. The authors also found that more data and bigger models both lowered errors by similar amounts.
TRIDENT: planning for mistakes
The third paper, from Elroy Galbraith and colleagues, proposes a system to help emergency call dispatchers. It uses Whisper Medium tuned on about 28,000 Caribbean broadcast clips, plus software that picks out medical details and detects stress in a caller's voice.
Its key idea is that a low-confidence transcript is useful information. Callers in crisis often shift into deeper Creole, so poor transcription combined with signs of distress can push a call up the queue for a human dispatcher.
The authors are clear that this is a design. Testing on real Caribbean emergency calls remains future work.
Limits and gaps that matter in the Caribbean
- Creole is its own language, not accented English. Whisper's English training helps with standard Caribbean English much more than with deep Creole.
- Music is not phone audio. A model trained on dancehall lyrics still has to be tested on voice notes and call centre recordings.
- Spelling is not standardised. Patois has no single agreed spelling, so error rates partly depend on how the reference transcript was written.
- Most results come from a single study. Other Caribbean languages, such as Trinidadian or Guyanese Creole, have little published data.
Who should care
If you are buying captioning, call summaries or voice tools, ask the vendor for error rates on recordings like yours, such as local voice notes or radio ads. Ask whether they trained on Caribbean speech and what happens when the system is unsure.
For researchers in the region, the Madden paper shows a practical path. Careful local transcription can improve results more cheaply than waiting for bigger global models.
Media and accessibility teams should note that 30 percent errors is a big improvement, but captions for Patois still need a human editor.
Bottom line
Whisper raised the standard for speech recognition, but it does not understand Caribbean Creoles well out of the box. Watch for new Caribbean speech datasets, especially phone and voice note audio, and for TRIDENT's first real-world tests.