Language Support & Multilingual Architecture

This document details language support in LocalNPC, explaining why v1.0 ships English-only, the technical seams designed for multilingual expansion, and the strict engineering gate required to add any new language.

1. Shipping Status: English Only in v1.0

LocalNPC v1.0 ships with official support for English (en-US).

Both presets (Balanced, the default, and Fast) are configured with:

• ggml-tiny.en.bin (an English-only Whisper model).
• The Windows system voice for en-US: Microsoft Mark, then Microsoft Zira, then Microsoft David, then any other en-US OneCore voice, then a OneCore voice of another English region (en-GB, en-AU, ...), then a SAPI 5 voice of en-US. Nothing is downloaded for the voice.
• English text normalizer (expanding $50 to fifty dollars, dates, ordinals, and abbreviations).
• An English phonemizer. The CMU dictionary itself is not included yet: 42 words are built in. The Windows voice reads text and does not use the phonemizer.

2. The Architectural Seams

No core pipeline file (NpcPipeline.cs, NpcAgent.cs, WindowsSpeechSynthesizer.cs, WhisperRecognizer.cs) is meant to contain hardcoded English strings, terminators, or language logic. This is not fully true yet: NpcPipeline.cs still falls back to the language code "en", uses an English abbreviation list when no LanguageProfile is set, and filters a short list of English phrases out of transcripts.

All language behaviour is encapsulated behind three contracts:

graph TD
    LP[LanguageProfile Asset] --> TN[ITextNormalizer]
    LP --> PH[IPhonemizer]
    LP --> ST[Sentence Delimiters & Abbreviation Rules]
    LP --> WC[Whisper Language Code]
    LP --> SV[Windows Voices: systemVoices]

1. LanguageProfile (LanguageProfile.cs)

A ScriptableObject defining:

• Language tag (en-US, es-ES, de-DE, etc.).
• Sentence terminators (. ! ? ¿ ¡).
• Trailing quotation/bracket closers (" ' ) ] }).
• Common abbreviations (Mr., Dr., approx., vs.) to prevent premature sentence splitting.
• Whisper language code.
• systemVoices: the Windows voices to speak with, in order of preference, by display name or voice id. When none of them is installed, any OneCore voice of the language tag is used, then any OneCore voice of its main language (for en-US: en-GB, en-AU, ...), then a SAPI 5 voice of the language tag. The shared voice is chosen once for the process, from the default language profile of the NpcRuntimeConfig; an NPC can have a voice of its own (see below).

Choosing a Voice for Each NPC

By default every NPC speaks with the shared voice. To give an NPC its own, select its NpcAgent and pick a voice under Voice in the Inspector. Speaking Rate sets its speed: 1.0 is the voice's normal rate, the range is 0.6 to 1.5, and Default Speed is 0.95. From a script, set SystemVoice and SpeakingRate before the agent starts; a change after that is ignored with a warning.

• The list shows the OneCore voices installed on the PC you work on, for the language of the agent's language profile (or of the default profile of its NpcRuntimeConfig). A player's PC may have other voices.
• To list the voices installed on the player's PC from a script, for example for a voice menu of your own, call NpcAgent.GetInstalledVoices("en-US") on the main thread. Each entry has a Name, which you can give to SystemVoice, and a Language. It plays no sound; with no language tag it lists every voice, and it is empty when the voice library is missing.
• When the chosen voice is not installed on the player's PC, the profile's systemVoices are tried in order, then any voice of the language tag, then any voice of its main language, then SAPI 5. The Inspector keeps a saved voice that your PC lacks and marks it "not installed here, will fall back".
• On a PC with only one voice of the language, all NPCs sound the same; only their speaking rates differ. Nothing warns about this.
• Not verified: tested only against a fake list of voices. Not tried on a real PC with one or two voices, and not on Windows 10.

2. ITextNormalizer (INpcModules.cs)

Expands numbers, currencies, dates, and symbols into pronounceable words before the sentence goes to the voice (e.g. converting 100kg into one hundred kilograms).

3. IPhonemizer (INpcModules.cs)

Translates graphemes (letters) into phonemes (phonetic sounds). The Windows voice reads text, so v1.0 does not use a phonemizer.

3. The 7-Point Language Gate (§14.1)

Before scheduling or announcing official support for any new language, the following 7 questions must be answered affirmatively:

4. How to Implement a Community Language

If you wish to add a custom language to your game:

1. Implement ITextNormalizer:

   public class SpanishTextNormalizer : ITextNormalizer
   {
       public string Normalize(string input)
       {
           // Strip markdown, expand € / números
           return expandedText;
       }
   }

2. Optional: implement IPhonemizer (only for your own synthesizer that takes phonemes; the Windows voice does not use it):

   public class SpanishPhonemizer : IPhonemizer
   {
       public int[] Phonemize(string sentence)
       {
           // Return the phoneme ids your synthesizer expects
           return phonemeIds;
       }
   }

3. Create LanguageProfile_es-ES.asset: Set whisperLanguageCode to "es", set languageTag to "es-ES", put the assembly-qualified type names of your classes in customNormalizerType and customPhonemizerType, list the Spanish Windows voices to use in systemVoices (with an empty list, any installed es-ES voice is used), and assign the profile as the default language profile of your NpcRuntimeConfig.
4. Select Multilingual Whisper & LLM: NpcRuntimeConfig has no setting for the Whisper model or for another LLM. LocalNPC uses the speech-to-text model of the model catalogue, ggml-tiny.en.bin, which is English only, and the LLM of the chosen preset. A multilingual speech model and automatic language detection have not been tested with the built-in recognizer. To use others, write your own ISpeechRecognizer or ILanguageModel and register it before the first agent starts, with NpcServiceLocator.RegisterRecognizerFactory or NpcServiceLocator.RegisterLanguageModelFactory.

5. Roadmap & Future Languages

• Spanish (es-ES) & German (de-DE): Planned for Post-v1.0 release.
• Bengali (bn-BD): G2P dictionary research and complex TMP font shaping remain active research tasks (~180–220 hours of engineering).
• Japanese (ja-JP): The sentence splitter must first learn to end a sentence at 。 without a following space (T-151); targeted for an optional add-on package.