Running a language AI for Benin on a small CPU server: what works, what breaks

Our demo translates, transcribes and speaks languages of Benin. It runs without a GPU, on a small 4-core, 8 GB server that is switched on when needed. This post explains how it is built, what we measured, and above all what is not good yet. These are low-resource languages: little text, little audio, few evaluations. In that setting a system that merely answers is dangerous; a system that shows its doubts and lets people correct it is useful.

Watch the demo video: EnariLab · Demo: translate, listen and correct the languages of Benin

In short

• We do not train a model yet. We use three open models from Meta: NLLB-200 (translation), MMS (speech to text) and MMS-TTS (speech synthesis). They are released under CC-BY-NC 4.0, so the demo is a non-commercial research project.
• Our work is around the models: checking their outputs, flagging doubt, ranking candidates, collecting consented corrections, and making all of it fit on modest hardware.
• We have no quantitative evaluation (no BLEU or chrF published) until native speakers write reference translations. What follows is timing measurements and qualitative findings.

The architecture, in one diagram

All three tools share one Python process. That is deliberate: with 8 GB you cannot afford several copies of the models.

1. Translating: generate, back-translate, rank

A single translation hides uncertainty. So we ask the model for several candidates (beam search of width 5, four candidates kept), along with the model's confidence for each: the exponential of the sequence score, i.e. a length-normalised average word probability. It is an approximation, and it is only comparable between candidates for the same sentence.

Each candidate is then translated back into the source language, and we measure the similarity between the original sentence and the back-translation with a dependency-free chrF (F-score on character n-grams).

The final score combines both signals, giving the round trip more weight than confidence, then subtracts points for recognisable defects: empty output, plain copy of the input, looping repetitions, implausible length. The exact values are starting settings, not truths, and we do not publish them: the principle matters more than the numbers.

Why weight the round trip more? For these languages the model is often very sure of itself when it is wrong. Its confidence alone does not separate good from bad. The round trip mostly catches off-topic translations.

What it does not prove. It is a weak check: the back-translation is done by the same model, so it is not an independent verification. The score sorts candidates; it does not say whether any is right. The interface shows every candidate with its score, and one click pulls one into the correction box. Only a speaker settles it.

Each request also produces a trace: the steps (prepare, generate, back-translate, rank), their durations and the scores, shown under "Technical details" and returned as JSON by the API.

2. Speech to text: a confidence per word

MMS is a CTC-decoded model. Rather than using its final output as is, we redo the greedy decoding ourselves, frame by frame, keeping each letter's probability:

# Principle (simplified): merge repeats, drop the "blank", words split on the delimiter
for t, symbol in enumerate(ids):
    if symbol == previous:              # same letter as the previous frame
        confidences[-1] = max(confidences[-1], prob[t])
        continue
    previous = symbol
    if symbol in (blank, *specials):    # nothing to write
        continue
    if symbol == delimiter:             # end of word: word confidence = mean of its letters
        close_word()
        continue
    letters.append(vocab[symbol]); confidences.append(prob[t])

A word's confidence is the mean of its letters', and a letter's is the highest across its frames. Words below a threshold are highlighted "to check": the person correcting knows where to look first.

Two safeguards: our decoding must reproduce exactly the text of the official decoder (the check is recorded in the trace), and audio is capped at 30 seconds, mono, 16 kHz, to keep the processor responsive.

MMS-1B-all ships with one adapter per language. It loads with the English adapter, then switches adapter according to the requested language. The adapter and vocabulary are shared state: the whole chain runs under a lock, and if an adapter switch fails we reset the state to "unknown" so it reloads next time.

3. Text to speech: characters the voice does not know

MMS-TTS voices (VITS, a small voice of about 140 MB per language) have a character-level vocabulary. Languages of Benin use tones and letters such as ɔ, ɛ, ɖ, ŋ, and their Unicode form matters: the same accented letter can be written as one character (composed form) or two (decomposed form). For each text we try both forms, keep the one that loses the fewest characters, and tell the person which characters the voice skipped.

Another choice: the random seed is fixed, so the same text always gives the same sound (VITS is otherwise stochastic). And the synthetic voice is never kept with the corrections.

A limit to know: for many languages, these voices were trained mostly on Bible readings. Pronunciation and tones can be wrong, and the interface says so.

4. Running all this on a CPU

Memory

In 32-bit, a 600-million-parameter model takes on the order of 2.4 GB, and a 1-billion-parameter model on the order of 4 GB. With 8 GB in total we cannot preload everything: translation and the three main voices fit, and speech to text is preloaded only if enough memory is available (we read the system's available memory at startup).

The first call

With lazy loading a model was only loaded on the first click. The result: the first person waited 15 to 30 seconds in front of a silent page, and the next ones behind them. We replaced this with a warm-up at startup, in a background thread: the page answers immediately, the translation model is loaded and runs a first dummy computation, then the voices follow. Until it is done, the page says so.

Informal measurements, one run, not a benchmark:

Call (4-core server, no GPU)	Duration
Ranked translation (4 candidates, 4 back-translations), 1st call after warm-up, end to end	about 4 s
Translation + Ewe voice, 1st call	about 8 s
A translation and a voice at the same time	6.4 s and 7.2 s

The last row shows the limit: two tools computing at the same time share the 4 cores. Warm-up removes the first-call wait, not the contention.

Queues

The web application runs one queue per tool (translation, speech to text, synthesis), with one request at a time per tool: the model and its state are shared, and each model also has its own lock. A slow transcription no longer delays a translation. More parallelism on a single tool would not help: the processor is already saturated.

button.click(translate, inputs, outputs, api_name="translate", concurrency_id="translation")
demo.queue(default_concurrency_limit=1, max_size=40)

What broke, and what we learned

• An invisible queue indicator. Our redesign hid the result area until the translation arrived. But the interface's queue indicator is drawn on the output components, so it sat on a hidden element and people saw "nothing". Fix: the result opens on click, the button locks, and a message says what is happening.
• Orphaned processes. Our restart script only stopped the main process of the "keep-awake" helper (the host pauses an idle machine after roughly 30 minutes), not its children. Each restart left some behind. Result: a load of 12 on 4 cores and an application that stopped responding. We now stop every process of the script on each restart.
• An edge relay. The host's ingress rate-limits requests: without a relay, the demo page stayed on "Loading…". The relay rewrites the host's address, caches static files, and retries requests rejected by the rate limit.

5. The API

The service exposes eight named endpoints, whose shape is frozen so tools calling them do not break: translate, translate with the full ranking detail, transcribe, transcribe with per-word confidence, speak, translate then speak, and two endpoints for sending corrections. The "detailed" variants return the full JSON: candidates, scores, warnings, steps and durations. Purely visual interface events are hidden from the API, and internal states are not exposed, so the contract stays stable as the interface evolves.

We deliberately keep the API free of session state: each call carries everything it needs, and nothing about a person is kept between two calls.

6. The correction loop

When the person corrects and submits, and only if they tick consent, we record: the text or audio, the correction, the language, the date, the model used, the AI's candidates with their scores and the rank of the one they picked up, an "unchanged" flag and a "native speaker" flag. Never a name, an email or an IP address. Corrections are written locally; a copy to a private data repository is planned but not enabled yet.

Storing the candidates and the rank has a scientific reason: it will let us check later whether our ranking really helps, by seeing whether speakers pick up the candidates we rank first more often.

One privacy detail: the interface writes imported audio to a temporary cache before consent; that cache is purged automatically after about an hour, not immediately.

7. What is poor or not evaluated

We prefer to say it plainly:

• Fon is badly translated. Our first readings show clearly poor translations. This is the problem the demo wants to expose, not a result we are proud of.
• No publishable score. We have no reference translations written by speakers. An evaluation script exists (it will compare our ranking with the model's raw first choice), but it is waiting for those references.
• The ranking is a heuristic. Same caveats as above: round trip through the same model, confidence often high on bad outputs, untuned settings.
• Seven voice-only languages. Baatonum, Dendi, Ditammari, Gen, Naténi, Waama and Yom have a voice and a transcription, but no translation model covers them, and a native speaker has not judged their quality. They are switched off by default.
• Speech to text barely tested on real voices. Audio edge cases are tested (formats, silence, too-short or corrupted files); evaluation on real speakers' voices is still to do.

Next steps, and how to help

1. Have reference translations written by speakers, to measure and tune the ranking.
2. Have speakers judge the voices, language by language, before opening the remaining seven languages.
3. Measure the reliability of per-word confidence on real voices.
4. Eventually, move from base models to small models fine-tuned on consented data.

Do you speak a language of Benin, work as a linguist, or develop software? You can request a demo, or join the contributors' list. Every correction submitted is exactly the data that is missing.