Raspberry Pi 2 – Speech Recognition on device

This is a lengthy post and very dry, but it provides detailed instructions for how to build and install SphinxBase and PocketSphinx and how to generate a pronunciation dictionary and a language model, all so that speech recognition can be run directly on the Raspberry Pi, without network access. Don’t expect it to be as fast as Google’s recognizer, tho …

Creating the RASPBIAN boot MicroSD

Starting with the current RASPBIAN (Debian Wheezy) image, the creation of a bootable MicroSD Card is a well understood and well documented process.

Uncompressing the zip (again, there is no better tool than The Unarchiver, if you are on a Mac) reveals the 2015-02-16-raspbian-wheezy.img

With the MicroSD (inside an SD-Card adapter – no less than 8GB) inserted into the Mac, I run the df -h command in Terminal, to find out how to address the card. Today, it showed up as /dev/disk4s1 56Mi 14Mi 42Mi 26% 512 0 100% /Volumes/boot, which means, I run something like this, to put the boot image onto the MicroSD:

… after a few minutes, once the 3.28 GB have been written onto the card, I execute:

Customizing the OS

Once booted, using the sudo raspi-config allow the customization of the OS, which means that time-zone, keyboard, and other settings are adjusted, to closely match its environment.
I usually start (PI is already connected to the internet via Ethernet Cable) with

  • updating the raspi-config
  • expanding the filesystem
  • internationalization: un-check en-GB, check en-US.UTF-8 UTF-8
  • internationalization: timezone ..
  • internationalization: keyboard: change to English US
  • setting the hostname to translator, there are too many Raspberry Pis on my home network, to leave it at the default
  • make sure SSH is enabled
  • force audio out on the 3.5mm headphone jack


Given the sparse analog-to-digital support provided by the Raspberry Pi, the probably best and easiest way to connect a decent Mic to the device, is using a USB microphone. I happen to have an older Logitech USB Mic, which works perfectly fine with the Pi.

After a reboot and now with the microphone connected, let’s get started ..
ssh pi@translator with the default password ‘raspberry’ gets me in from everywhere on my local network
cat /proc/asound/cards
0 [ALSA ]: bcm2835 - bcm2835 ALSA
bcm2835 ALSA
1 [AK5370 ]: USB-Audio - AK5370
AKM AK5370 at usb-bcm2708_usb-1.2, full speed

showing that the microphone is visible and its usb extension.
Next, I edit alsa-base.conf to load snd-usb-audio like so:
sudo nano /etc/modprobe.d/alsa-base.conf
options snd-usb-audio index=-2
options snd-usb-audio index=0
and after a sudo reboot, cat /proc/asound/cards
looks like this
0 [AK5370 ]: USB-Audio - AK5370
AKM AK5370 at usb-bcm2708_usb-1.2, full speed
1 [ALSA ]: bcm2835 - bcm2835 ALSA
bcm2835 ALSA

Recording – Playback – Test

Before worrying about Speech Recognition and Speech Synthesis, let’s make sure that the basic recording and audio playback works.
Again, I have an USB Microphone connected to the Pi, as well as a speaker, using the 3.5mm audio plug.

Installing build tools and required libraries

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install bison
sudo apt-get install libasound2-dev
sudo apt-get install swig
sudo apt-get install python-dev
sudo apt-get install mplayer
sudo reboot


sudo nano etc/asound.conf and enter something like this:


The current recording settings can be looked at with:
amixer -c 0 sget 'Mic',0
and for me that looks something like this:

alsamixer -c 0 can be used to increase the capture levels. After an increase, it looks like this:


The current playback settings can be looked at with:
amixer -c 1
alsamixer -c 0 can be used to increase the volume. After an increase,
amixer -c 1
it looks like this:

Test Recording and Playback

With the mic switched on ..
arecord -D plughw:0,0 -f cd ./test.wav .. use Control-C to stop the recording.
aplay ./test.wav

With recording and playback working, let’s get into the really cool stuff, on-device speech recognition.

Speech Recognition Toolkit

CMU Sphinx a.k.a. PocketSphinx
Currently pocket sphinx 5 pre-alpha (2015-02-15) is the most recent version. However, there are a few prerequisites that need to be installed first ..

Installing build tools and required libraries

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install bison
sudo apt-get install libasound2-dev
sudo apt-get install swig
sudo apt-get install python-dev
sudo apt-get install mplayer

Building Sphinxbase

cd ~/
wget http://sourceforge.net/projects/cmusphinx/files/sphinxbase/5prealpha/sphinxbase-5prealpha.tar.gz
tar -zxvf ./sphinxbase-5prealpha.tar.gz
cd ./sphinxbase-5prealpha
./configure --enable-fixed
make clean all
make check
sudo make install

Building PocketSphinx

cd ~/
wget http://sourceforge.net/projects/cmusphinx/files/pocketsphinx/5prealpha/pocketsphinx-5prealpha.tar.gz
tar -zxvf pocketsphinx-5prealpha.tar.gz
cd ./pocketsphinx-5prealpha
make clean all
make check
sudo make install

Creating a Language Model

Create a text file, containing a list of words/sentences we want to be recognized

For instance ..

Upload the text file here: http://www.speech.cs.cmu.edu/tools/lmtool-new.html
and then download the generated Pronunciation Dictionary and Language Model

For the the text file mentioned above, this is what the tool generates:

Pronunciation Dictionary

Language Model

Looking carefully, the Sphinx knowledge base generator provides links to the just generated files, which make sit super convenient to pull them down to the Pi. For me it generated a base set with the name 3199:
wget http://www.speech.cs.cmu.edu/tools/product/1427343814_14328/3199.dic
wget http://www.speech.cs.cmu.edu/tools/product/1427343814_14328/3199.lm

Running Speech-recognition locally on the Raspberry Pi

Finally everything is in place, SphinxBase and PocketSphinx have been building installed, a pronunciation dictionary and a language model has been created and locally stored.
During the build process, acoustic model files for the english language, were deployed here: /usr/local/share/pocketsphinx/model/en-us/en-us

.. time to try out the the recognizer:
cd ~/
export LD_LIBRARY_PATH=/usr/local/lib
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig

pocketsphinx_continuous -hmm /usr/local/share/pocketsphinx/model/en-us/en-us -lm 3199.lm -dict 3199.dic -samprate 16000/8000/48000 -inmic yes



INFO: ps_lattice.c(1380): Bestpath score: -7682
INFO: ps_lattice.c(1384): Normalizer P(O) = alpha(:285:334) = -403763
INFO: ps_lattice.c(1441): Joint P(O,S) = -426231 P(S|O) = -22468
INFO: ngram_search.c(874): bestpath 0.01 CPU 0.003 xRT
INFO: ngram_search.c(877): bestpath 0.01 wall 0.002 xRT

Live Demo

This video shows the recognizer running in keyword spotting mode, using the dictionary and model mentioned above:
pocketsphinx_continuous -lm 3199.lm -dict 3199.dic -keyphrase "OKAY PI" -kws_threshold 1e-20 -inmic yes
The purpose is to provide some indication of the recognition speed that can be expected, running PocketSphinx on the Raspberry Pi 2.

7 Comment

  1. HI, great tutorial, wondering if this would work on raspbian Jessie?

  2. getting Input overrun, read calls are too rare .. and poor recognition

  3. Hey
    any idea how to get the pocketsphinx-python package running on a raspberry pi 3?

    1. yes i started on raspberry pi 3

  4. excellent guide … but you could just connect a bash command recognizes the word

  5. it would be nice to have a tutorial how to use this with GPIOs..

  6. I have given all the commands as stated but i am getting the following error;
    INFO: continuous.c(307): pocketsphinx_continuous COMPILED ON: Apr 28 2017, AT: 09:23:15

    Error opening audio device (null) for capture: Connection refused
    FATAL: “continuous.c”, line 245: Failed to open audio device
    Please tell how to rectify this

Leave a Reply