Paper digest: RAG for cultural LLMs
Retrieval-augmented generation lets AI look up facts in your documents before answering. What the key papers found, and why it matters for Caribbean AI.
AI chatbots often make up facts, such as a fee, a law or an opening time, because they answer from memory. Retrieval-augmented generation, or RAG, fixes much of this by having the AI look up relevant documents first and answer from them.
For Caribbean groups planning AI built on local knowledge, such as Jamaica's Amber lab, RAG is often the quickest and cheapest way to get accurate local answers.
The papers
| Paper | Authors | Year | Link |
|---|---|---|---|
| Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks | Lewis and 11 co-authors (Facebook AI Research, University College London, New York University) | 2020, NeurIPS | arXiv:2005.11401 |
| Dense Passage Retrieval for Open-Domain Question Answering | Karpukhin and others (Facebook AI) | 2020 | arXiv:2004.04906 |
| REALM | Guu, Lee, Tung, Pasupat and Chang (Google) | 2020 | arXiv:2002.08909 |
| Leveraging Passage Retrieval with Generative Models (FiD) | Izacard and Grave | 2020 | arXiv:2007.01282 |
| Self-RAG | Asai, Wu, Wang, Sil and Hajishirzi | 2023 | arXiv:2310.11511 |
| Retrieval-Augmented Generation for Large Language Models: A Survey | Gao and others | 2023 | arXiv:2312.10997 |
Why this paper exists
Language models, built on the design in our Attention Is All You Need digest, store a surprising amount of knowledge from their training. But that memory is hard to update, can't point to a source and fails confidently when it is wrong.
Patrick Lewis and colleagues proposed pairing a language model with a searchable library of documents. They used Wikipedia in their tests. The model searches the library, pulls out the most relevant passages and writes its answer using them. Update the library and the answers change, with no retraining needed.
How it works
A RAG system has three steps:
- Take the user's question and turn it into a search.
- Find the most relevant passages in a document collection.
- Give the question and those passages to the language model to write the answer.
The search step matters a lot. Older search matched exact words. Dense retrieval, from Vladimir Karpukhin and colleagues, turns text into numerical summaries of its meaning. It can find a passage that answers the question even if it uses different words.
Results that moved
All from the paper abstracts:
| Paper | Key finding |
|---|---|
| RAG (Lewis) | Best results at the time on three open-domain question-answering tests, with more specific and factual writing than a model without retrieval |
| Dense retrieval (Karpukhin) | Found the right passage 9 to 19 percentage points more often in the top 20 results than standard keyword search |
| REALM (Guu) | Beat all previous methods on three question-answering tests by 4 to 16 percentage points |
| FiD (Izacard and Grave) | Answers got better as more passages were retrieved, showing models can combine evidence from many sources |
| Self-RAG (Asai) | Models with 7 and 13 billion parameters that decide when to search and check their own answers beat ChatGPT on several fact-based tasks |
Self-RAG points to where the field went next. Rather than always searching, a model learns when it needs to look something up and whether the passages actually support its answer.
What a Caribbean "cultural" AI needs
It helps to split a local AI project into three parts:
| Part | What it covers | Where it goes wrong |
|---|---|---|
| Language | Understanding Creole and local ways of speaking | Answers sound foreign or misread questions |
| Knowledge | Local laws, prices, procedures and rules | Makes up a fee or quotes an old rule |
| Application | The actual service, like a tutor or business helper | Can't finish the task the user needs |
RAG solves the knowledge part. Our Creole language digest covers the language part.
This split also helps with timing. A useful RAG assistant built on a country's government documents could launch in months, while training a new language model takes years.
Getting the library right
The papers tested on Wikipedia. Real projects need more care:
- Use documents you have the right to use, such as official circulars, price lists and brand guides.
- Tag each document with its date, country and topic, so the system can pick the current version.
- Combine meaning-based search with exact keyword search, which catches section numbers and plan codes.
- Show users which document the answer came from.
- Update the library whenever a document changes.
Limits
- RAG only helps if the answer is in the library and the search finds it. A bad library gives confident wrong answers.
- Search tools trained on standard English may struggle with Patois questions or mixed Creole and English messages.
- Document libraries can contain personal or confidential information, so access needs controlling.
- RAG won't make a model speak Creole naturally. That needs language training.
Longer AI memory, made cheaper by work like FlashAttention, doesn't replace RAG. Pasting a whole folder into a prompt is not the same as finding the right page.
Who should care
Caribbean governments and AI labs can start useful projects now. A searchable library of official documents, paired with an existing model, can answer questions on education, health services or business registration.
For small businesses, your price list, FAQs and brand rules are a RAG problem today. You don't need a custom AI model to stop a chatbot inventing prices.
When a vendor demos a "Caribbean AI", ask where its facts come from, how you correct a wrong one and whether it shows its sources. A local accent is not the same as accurate local knowledge.
Bottom line
RAG lets AI answer from documents you control and can update, rather than from memory. Watch for Caribbean AI projects to publish what documents their assistants draw on and how often they are refreshed.