Quick Start Guide

Get a talking, listening NPC running in your Unity project. The QuickStart NPC is a capsule with a mouth box that opens with the loudness of its voice; the Shopkeeper and Kiosk scenes have no moving mouth. The three sample scenes draw in the Built-in Render Pipeline and in URP. HDRP is not supported.

The QuickStart folder also holds QuickStartHead.unity: the QuickStart setup with the sample head (Samples/Head/LocalNPC Head.prefab) in place of the capsule, and its own NPC to match the head: Corin of Hollowmere, a man, who speaks with the Microsoft David voice (or the default voice when David is not installed). The head opens its jaw with the loudness of the voice; other mouth shapes need the optional uLipSync package, and the head does not blink.

🚀 Setup

Step 1: Open the Setup Wizard

From the Unity menu bar, select:

Tools → LocalNPC → Setup Wizard

The wizard will verify your project settings (Unity version, 64-bit architecture). It has three tabs: 1. Environment & Hardware, 2. Model Provisioning and 3. Ready to Converse.

Step 2: Download the Balanced Preset Models

1. In the Setup Wizard, tab 2. Model Provisioning, select Balanced Preset (Recommended), the default (1.20 GB of model files). On a low-end PC, select Fast Preset (low-end PCs) instead (569 MB).
2. Click Download & Provision Balanced Suite (with Fast selected, the button says Download & Provision Fast Suite).
3. The wizard downloads these files from Hugging Face (huggingface.co), over HTTPS with the server's certificate checked, and checks each one with SHA-256:
   • LLM: qwen2.5-1.5b-instruct-q4_k_m.gguf (1.12 GB; with Fast, qwen2.5-0.5b-instruct-q4_k_m.gguf, 491 MB)
   • STT: ggml-tiny.en.bin (Whisper)

No voice activity model is downloaded: speech is detected by the built-in energy VAD. The voice needs no download. The NPC speaks with the Windows system voice (for en-US: Microsoft Mark first, then Microsoft Zira, then Microsoft David, then any other en-US voice, then a voice of another English region such as en-GB; SAPI 5 as the last fallback).

The same files can be downloaded one by one in Tools > LocalNPC > Model Manager. The speech-to-text model is the entry of the model catalogue shown there.

Step 3: Add an NPC to Your Scene

1. Create a 3D capsule character or character model:

   GameObject → 3D Object → Capsule

2. Add the NpcAgent component to the GameObject:

   Add Component → LocalNPC → NPC Agent

3. Add an AudioSource and a SpeechAudioQueue to the same GameObject, so the NPC can be heard:

   Add Component → Audio → Audio Source

   Then click Add AudioQueue in the NpcAgent Inspector. These two components are not added automatically.

4. Create or assign an NpcPersonality asset:

   • Right click in Project window: Create → LocalNPC → Personality
   • Name it CompanionPersonality.
   • Set Display Name: Companion
   • Set System Prompt: You are a helpful, witty companion traveling with the player. Keep replies to one or two sentences.
   • Set Greeting: Hello there! Where are we heading next?
   • Drag CompanionPersonality into the Personality field of NpcAgent.

Step 4: Turn On Listening

1. In the NpcAgent Inspector, under Lifecycle & Startup, tick Listen On Start (leave Prewarm On Start ticked). From code, call StartListening() after PrewarmAsync() instead.
2. That is all the microphone setup. NpcAgent starts the microphone itself when it starts listening and stops it on StopListening(). Do not add a MicrophoneCapture to another GameObject: two captures would record the same device.
3. In Windows, open Settings > Privacy & security > Microphone and allow desktop apps to use the microphone.
4. Optional: add SubtitleHud (from the samples; after you import the .unitypackage it is at Assets/LocalNPC/Samples/SubtitleHud.cs) to an empty GameObject for on-screen dialogue display.

If the microphone cannot be used, NpcAgent raises OnError with the code MicrophoneUnavailable and a message that says why. The agent stays in the Listening state. See the Troubleshooting Playbook.

Only one NpcAgent listens to the microphone at a time; a second one logs a warning and gets no voice input. While the NPC is speaking, the microphone is ignored.

Tip: In a noisy room or on loudspeakers, use a headset (the NPC's own voice can come back through the speakers).

Step 5: Press Play & Talk

1. Click Play in the Unity Editor.
2. Speak clearly into your microphone: "What can you help me with?"
3. The NPC will transcribe your voice, generate a response using the local language model (1.5B with the default Balanced preset, 0.5B with Fast), and reply out loud through the AudioSource!

If Windows has no voice for the NPC's language, the NPC answers in text only and the Console shows one warning that says how to add a voice. See Troubleshooting.

Speech-to-text runs inside the Unity process, through the whisper.cpp 1.9.5 DLLs in the package. There is no separate program to install or start.

If the speech model or one of these DLLs cannot be loaded, NpcAgent raises OnError once. The message says what is wrong and what to do, and contains the sentence "Speech input is off. Text input still works." The NPC then goes to Idle and answers typed input as usual. A missing speech model is downloaded in Tools > LocalNPC > Model Manager. See the Troubleshooting Playbook.

💡 Testing Without a Microphone

If you don't have a microphone plugged in:

1. Select your NPC GameObject in the Hierarchy during Play Mode.
2. In the NpcAgentInspector, locate the Test With Typed Input (No Microphone Required) box.
3. Type any question and click Send Prompt. The turn executes immediately with full audio and subtitles.