Wolf Paulus

Chatbots 3.3 Conference

Wolf Paulus — Sat, 30 Mar 2013 04:16:05 +0000

The Gathering

The legends and current elites of AIML and Turing AI (not necessarily disjoint groups) met for their yearly gathering at Seed Philly, a Philadelphia tech startup incubator, located in the heart of Center City.

The Chatbots 3.3 conference was a fast flowing event with exciting flash-talk style presentations, always followed by Q&A segments. The amazing speaker lineup included several Loebner Prize winners, AIML Engine developers, VCs, Psychiatrists, Artificial Intelligence researcher, and best-selling authors.

Dr. Richard Wallace

The Highlights

Dr. Richard Wallace, Loebner Prize Winner 2000, 2001, 2004 and father of AIML, talked about the new AIML 2.0 Specification that focuses much on making AIML more succinct, while maintaining its simplicity. He stated that it currently took about 10,000 AIML categories to create a believable character. ALICE for instance has about 100,000 categories and the PROFESSOR, a bot with one of the largest AIML knowledge bases, has about 580,000 AIML categories. Writing that many categories is not only very time consuming (experienced AIML authors may be able to write one category per minute) but also requires a memory capacity, not available on many embedded and mobile devices. AIML 2.0 therefore tries to make AIML more efficient, allowing the creation of a believable character with as little as 1/6 of the categories needed before.

Mohan Embar

Mohan Embar, 2012 Loebner Prize Winner for his bot entry Chip Vivant, gave an informal talk entitled Moonshot or Low-Hanging Fruit, sharing his motivation for creating his bot (which is not based on AIML) and encouraging other bot authors to question their own motivation and to stay focussed.

Dave Morton

He currently works on creating a more empathetic and engaging bot that is more than simply reacting. He suggested a project that would allow to program a bot with a persons’ personality, core-values, views, achievements, etc. which would be available after the persons’ death, for relatives and friends to still communicate with the deceased.

Jeremy Spiegel, MD, Cascon Bay Medical, was making the case for the use of Chatbots in Psychiatry, Dave Morton, Lead developer of the Program-O AIML-Interpreter, talked about the improvements he is currently making on the PHP/MySQL based engine, and Wolf Paulus, Inuit Inc., talked about Voice Enabling Chatbots.

Bill DeSmedt

Bill DeSmedt, system designer, Artificial Intelligence researcher, and science fiction author (Singularity (2004), Dualism (2010)) talked about his PASCAL based deductive and inductive reasoning system that can solve reasonably sophisticated murder mystery puzzles when fed with all the facts.

Dave Morton

Chris Shaw

The event ended with an invigorating talk from the entertaining, always challenging, and thought provoking Chris Shaw. His focus was more on visual expression rather than artificial intelligence, showing many of the cutting edge visual presentations of avatars, like Zoe . Chris emphasized the importance of tell a compelling story, a bots needs to be able to play a role in an interactive drama and always needs to be able to create that moment of surprise. He concluded with stressing the significance of sound quality of a natural voice over any visual presentation.

The Conclusion

Seeing the legends of AIML and Turing AI being all invigorated and out in force, confirmed the intuition that everyone at the conference seemed to have had: all the parts and pieces are finally coming together, to create an believable virtual human.

There was a spirit of optimism that many of the more experienced could only compare with the time shorty before the word-wide-web popularized the Internet.

Installing Java 7.x and Tomcat 7.x on Debian Squeeze

Wolf Paulus — Fri, 15 Mar 2013 23:23:11 +0000

Tomcat 7 is the first Apache Tomcat release to support the Servlet 3.0, JSP 2.2, and EL 2.2 specifications. Please note that Tomcat 7 requires Java 1.6 or better so we start with installing a recent version of Oracle’s JRE.

Install Oracle JRE 7 on Debian Linux

To install Oracle’s Java Runtime with apt-get, we first need to entend the list of apt-get’s sources. Once that is done, an java-installer will actually install the Java SE Runtime Environment. Here are the steps to follow:

$ su root
# echo "deb http://ppa.launchpad.net/webupd8team/java/ubuntu precise main" > /etc/apt/sources.list.d/webupd8team-java.list
# echo "deb-src http://ppa.launchpad.net/webupd8team/java/ubuntu precise main" >> /etc/apt/sources.list.d/webupd8team-java.list
# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys EEA14886
# apt-get update
# apt-get install oracle-java7-installer
# java -version
java version "1.7.0_17"
Java(TM) SE Runtime Environment (build 1.7.0_17-b02)
Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)
# exit
$

With Java now installed we move on, installing Tomcat. However, it may be beneficial to have an dedicated user for Tomcat, in case you …
BTW, Java got installed into this location: /usr/lib/jvm/java-7-oracle

sudo adduser \
  --system \
  --shell /bin/bash \
  --gecos 'Tomcat Java Servlet and JSP engine' \
  --group \
  --disabled-password \
  --home /home/tomcat \
  tomcat

Installing Tomcat7

$ cd /tmp
$ wget http://mirror.cogentco.com/pub/apache/tomcat/tomcat-7/v7.0.37/bin/apache-tomcat-7.0.37.tar.gz
$ tar xvzf ./apache-tomcat-7.0.37.tar.gz

$ sudo mkdir /usr/share/tomcat7
$ sudo mv /tmp/apache-tomcat-7.0.37 /usr/share/tomcat7

To make it easy to replace this release with future releases, we are going to create a symbolic link that we are going to use when referring to Tomcat (after removing the old link, you might have from installing a previous version):

$ sudo rm -f /usr/share/tomcat
$ sudo ln -s /usr/share/tomcat7/apache-tomcat-7.0.37 /usr/share/tomcat

Since we created a tomcat user, he should also own all these files in

$ sudo chown -R tomcat:tomcat /usr/share/tomcat7
$ sudo chmod +x /usr/share/tomcat/bin/*.sh

If Tomcat’s default HTTP port (8080) is already in use, you need to edit the server.xml configuration file, e.g.
edit /usr/share/tomcat/conf/server.xml and replace 8080 with 8000

Starting Tomcat

$ sudo /bin/su - tomcat -c /usr/share/tomcat/bin/startup.sh

Using CATALINA_BASE:   /usr/share/tomcat
Using CATALINA_HOME:   /usr/share/tomcat
Using CATALINA_TMPDIR: /usr/share/tomcat/temp
Using JRE_HOME:        /usr
Using CLASSPATH:       /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar

Stopping Tomcat

$sudo /bin/su - tomcat -c /usr/share/tomcat/bin/shutdown.sh

Using CATALINA_BASE:   /usr/share/tomcat
Using CATALINA_HOME:   /usr/share/tomcat
Using CATALINA_TMPDIR: /usr/share/tomcat/temp
Using JRE_HOME:        /usr
Using CLASSPATH:       /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar

Tomcat with Webmin

If you were to manage your debian server with WebMin, the configuration of the start/stop behavior would look like this

which created a script just like this one:

#!/bin/sh
### BEGIN INIT INFO
# Provides: Tomcat
# Required-Start: $network
# Required-Stop: $network
# Default-Start: 2 3 5
# Description: Java Servlet and JSP Engine
### END INIT INFO

case "$1" in
'start')
	/bin/su - tomcat -c /usr/share/tomcat/bin/startup.sh
	;;
'stop')
	/bin/su - tomcat -c /usr/share/tomcat/bin/shutdown.sh
	;;
*)
	echo "Usage: $0 { start | stop }"
	;;
esac
exit 0

Speech Synthesis and the Quality of Voice

Wolf Paulus — Sat, 02 Mar 2013 00:10:39 +0000

When listing to the radio or a podcast, while driving to work, I don’t think I imagine how the person I’m listing to, looks like. Still, if later, I happen to see them for the first time, in a picture or video, I often find myself surprised.

A verbally responding mobile application has many obvious advantages. For instance, users don’t have to decipher tiny fonts on small displays, in fact, they don’t have to look at the display at all. Just like colors and typography contribute considerably to the look and feel of an application, so does the voice quality for a voice enabled mobile application.

There are at least three different approaches to synthesize text.

There might be a Text-To-Speech module built into the OS, or a separately installed Text-o-Speech engines can plug-in to the OS’s Text-To-Speech module.
Secondly, instead of requiring a separate install, a synthesizer and voices can be packaged and shipped with the application.
Lastly, a web-service can be used, to synthesize text. The advantage of this, would be a more predictable and consistent voice quality, comparatively independent from the hardware and operation system used on the mobile client.

Quality of Voice Voice quality is that component of speech which gives the primary distinction to a given speaker’s voice when pitch and loudness are excluded. It involves both phonatory and resonatory characteristics. Some of the descriptions of voice quality are harshness, breathiness and nasality.

Voice Synthesis Demo

Voiceware Co.,Ltd., located in Seoul, Korea, provides one of the highest speech synthesis quality solutions available. This however is highly subjective, see “Factors to assess voice quality” below.

Listener’s choice of what is a good voice is subjective without measurable units. However, a voice can still be described as:

Bright: Higher loudness without effort (not-shouting)
Confident: Steady Pitch and Loudness level –opposite of wavering
Melodious: Good correlation between Pitch and loudness
Resonant: Articulation tuned to Pitch – Ringing
Hoarse: Presence of noise component – absence of harmonics
Pressed: Long closed phase in vibration cycle
Breathy: Steep spectral slope, strong fundamental
Nasal: Heavily damped resonances

Factors to assess voice quality

Voice quality depends not only on the functioning of the voice box or the larynx but also on articulation (enunciation or pronunciation), rate of speech (too rapid or too slow), intonation, pauses etc.

Rate of speaking: Number of words per breath, Number of words per minute
Rhythm of Speech: Accent and Inflections (Pitch range)
Breathing Pattern: Speech as during fast walking Vs relaxed
Pitch Level: Habitual F0 or Pitch
Loudness level: Deafening (too loud) Vs Whispering (too soft)
Mood State: Sudden changes in pitch level, loudness and rhythm
Clarity of Speech: Good articulatory habit

When perceiving a persons credibility, accent is more important than looks

More TTS – Web Services

Google http://translate.google.com/translate_tts?tl=en&q=text, e.g.:
http://translate.google.com/translate_tts?tl=en_uk&q=Hello+,+how+are+you+today+?
Free text-to-speech conversion service: http://speechutil.com/
Microsoft http://www.microsofttranslator.com/dev/ an example would be http://www.bing.com/translator/
iSpeech Demo: http://www.ispeech.org/text.to.speech

Artist on Android w/ Voice Recognition

Wolf Paulus — Fri, 22 Feb 2013 04:37:28 +0000

Artist on Android

The display is the canvas and your finger becomes a paint brush.
Simply use your finger to create beautiful paintings.
Use the Menu to change color or brush styles and also to print or share your artwork via Email etc.
Publish your best work on our flickr group, straight from the app.
Using Advanced Photo Paper has produced stunning print results, worth framing.
I am totally amazed by what users have drawn with this app.

Voice Recognition

Mobile applications like Artist on Android are perfectly suited to accept voice input.
Instead of accessing the menu several times, to select a tool (like a pen or a brush), its style (like embossed or blurred), and its color, you now press the microphone button in the Actionbar and say “Lavender Blur Pen”.

The application’s vocabulary include all menu topics like “Pen, Brush, Roll, Emboss, Blur, Blend, Alpha, Erase, Reset, Save, Print, Share Publish, Explore, and Help” and also over one hundred color names. To bring up a list of all those color names, simply say “Colors”.
Speech recognition, interpretation, and processing is hard and “Artist on Android” is far from perfect. So please be gracious and generous if it not always understands. We are just getting started ..

Installing / Updating Python on OS X

Wolf Paulus — Mon, 28 Jan 2013 17:00:28 +0000

While Python comes pre-installed on OS X, Apple doesn’t do a good job on keeping the Python runtime environment up to date. Currently, on Mac OS X 10.7.4 “Lion”, entering python -V returns Python 2.7.1. Even worse, Mac OS X 10.6 “Snow Leopard” is still on a Python 2.6 release.
While the latest Python releases are always available on http://www.python.org, updating a Mac isn’t a simple, straight forward process.

Follow along and update your Mac to Python 2.7.3, or 3.3.0 or whatever the newest 2.x and 3.x release might be, when you read this. To update your Mac to something like Python 2.7.3, I assume that

your Mac-User account is setup as an “Administrator” account.
your Mac already has this folder: /System/Library/Frameworks/Python.framework/Versions/

To read about how to upgrade to Python 3.3, jump to the very bottom of this post.

1. Downloading and Installing the latest Python Release

Go to python.org and pick the most recent Python 2.x release: http://python.org/download.
I picked Python 2.7.3 Mac OS X 64-bit/32-bit x86-64/i386 Installer (for Mac OS X 10.6 and later), an about 18 MB download.
Opening the DMG and executing the installer (Python.mpkg) will install the Python release into /Library/Frameworks/Python.framework, which is not next to the other, already installed Python versions and may lead to some nasty incompatibilities. Moreover, /usr/bin/python still executes python 2.7.1., i.e. some work still needs to be done to really update Python on your Mac.

2. Moving Python into the right place

If Python 2.7.x is already available on the Mac (e.g. on OS X Lion), open a terminal and enter:

sudo rm -R /System/Library/Frameworks/Python.framework/Versions/2.7

This will delete Python 2.7.
In any case, now enter this into the terminal

sudo mv /Library/Frameworks/Python.framework/Versions/2.7 /System/Library/Frameworks/Python.framework/Versions

which moves the just installed python 2.7.3 release next to the other Python releases.

3. Fixing the Group

Setting group to wheel, just like it’s done for the already installed Python Versions:

sudo chown -R root:wheel /System/Library/Frameworks/Python.framework/Versions/2.7

4. Updating the Current Link

sudo rm /System/Library/Frameworks/Python.framework/Versions/Current
sudo ln -s /System/Library/Frameworks/Python.framework/Versions/2.7 /System/Library/Frameworks/Python.framework/Versions/Current

5. Fixing /usr/bin

Since python, pythonw, etc. are not linked but actually reside in the /usr/bin directory,
/usr/bin/python still executes python 2.7.1..
Therefore, we are going to remove the old python executable files and replacing them with links into /System/Library/Frameworks/Python.framework/Versions/Current:

5.1. Removing old copies

sudo rm /usr/bin/pydoc
sudo rm /usr/bin/python
sudo rm /usr/bin/pythonw
sudo rm /usr/bin/python-config

5.2. Creating links into /System/…/Python.framework/Versions/Current

sudo ln -s /System/Library/Frameworks/Python.framework/Versions/2.7/bin/pydoc /usr/bin/pydoc
sudo ln -s /System/Library/Frameworks/Python.framework/Versions/2.7/bin/python /usr/bin/python
sudo ln -s /System/Library/Frameworks/Python.framework/Versions/2.7/bin/pythonw /usr/bin/pythonw
sudo ln -s /System/Library/Frameworks/Python.framework/Versions/2.7/bin/python-config /usr/bin/python-config

6. Updating .bash_profile

Use TextMate, pico, or your favorite text editor to edit the hidden ~/.bash_profile file. If you want to be able to execute python tools like idle easily, without providing the path, edit the PATH for Python like this:

# Setting PATH for Python 2.7
# The orginal version is saved in .bash_profile.pysave
PATH="/System/Library/Frameworks/Python.framework/Versions/2.7/bin:${PATH}"
export PATH

However, you can also remove those four lines altogether or fall-back to the .bash_profile.pysave file.
Try it by closing the terminal app, re-opening it, and entering python -V as well as /usr/bin/python -V.

7. Updating your IDE

The last remaining step it to tell your IDE about the new Python release that you just installed.
I hope you have discovered PyCharm, JetBrain’s IDE for Python and Django.
In PyCharm, you would open its properties and under “Project Interpreters” change the Python Interpreter, by pointing it to the /System/Library/Frameworks/Python.framework/Versions/2.7 directory.

Updating to Python 3.3

Following the same procedure, you can also update to Python 3.3. E.g, I downloaded the Python 3.3.0 Mac OS X 64-bit/32-bit x86-64/i386 Installer form here: http://python.org/download/

After running the included installer, a script like this (run with sudo) should put everything into the right place and make python 3.3 the default python version.

#!/bin/bash
rm -R /System/Library/Frameworks/Python.framework/Versions/3.3
mv /Library/Frameworks/Python.framework/Versions/3.3 /System/Library/Frameworks/Python.framework/Versions
chown -R root:wheel /System/Library/Frameworks/Python.framework/Versions/3.3

rm /System/Library/Frameworks/Python.framework/Versions/Current
ln -s /System/Library/Frameworks/Python.framework/Versions/3.3 /System/Library/Frameworks/Python.framework/Versions/Current

rm /usr/bin/pydoc
rm /usr/bin/python
rm /usr/bin/pythonw
rm /usr/bin/python-config

rm /System/Library/Frameworks/Python.framework/Versions/3.3/bin/pydoc 
rm /System/Library/Frameworks/Python.framework/Versions/3.3/bin/python 
rm /System/Library/Frameworks/Python.framework/Versions/3.3/bin/pythonw 
rm /System/Library/Frameworks/Python.framework/Versions/3.3/bin/python-config 

ln -s /System/Library/Frameworks/Python.framework/Versions/3.3/bin/pydoc3 /usr/bin/pydoc
ln -s /System/Library/Frameworks/Python.framework/Versions/3.3/bin/python3 /usr/bin/python
ln -s /System/Library/Frameworks/Python.framework/Versions/3.3/bin/pythonw3 /usr/bin/pythonw
ln -s /System/Library/Frameworks/Python.framework/Versions/3.3/bin/python3-config /usr/bin/python-config

E*Trade Mobile – Voice Commands

Wolf Paulus — Sun, 27 Jan 2013 22:36:44 +0000

ETrade provides a great mobile app experience on iPhone, iPad, Android, Windows Phone, and Blackberry. I think it’s almost expected that the feature-set provided by the dedicated native mobile applications are not quite the same. The Windows version and especially the one for Blackberry fall far behind what ETrade has to offer on Android and iOS.

For instance, in April 2012, Speech Recognition was first added to their iPhone (not iPad) mobile application and later to the Android app as well; allowing the user to request stock quotes, options chains, company information, or to launch the stock order, just by using voice.

“Investors are becoming more accustomed to interacting with voice-enabled technology, and we’re proud to be one of the first to offer this innovative feature to our mobile users,” said Michael Curcio, President, ETRADE Securities. “By integrating voice technology, ETRADE provides a mobile experience unlike any other – creating a state-of-the-art and convenient approach to navigation.”

ETrade uses speech recognition and speech synthesis software provided by Nuance Communications, Inc. The application is feature-packed, comes as an 11 MB download, and is not, what you would call a thin client. Only a very few of those features however, are accessible through Voice Commands.

Available Voice Commands

Stock Quotes

“Price [symbol or name]“
“Can I get a quote on [symbol or name]?”
“Please give me the stock price on [symbol or name]“

Company News

“I would like news on [symbol or name]“
“News [symbol or name]“
“Can I get some news on [symbol or name]?”

Company Options Chains

“Can I get options chains on [symbol or name]“
“Give me options on [symbol or name]“

Screen Navigation

Take me to the dashboard”
“Show me my accounts please”
“I would like to see my alerts”
“Can I see my watchlists”
“Show me my portfolios”

Stock Order Ticket

“Buy 50 shares of [symbol or name]“
“Buy 75 shares of [symbol or name] at limit price 9 dollars and 50 cents”
“Sell my holdings on [symbol or name]“
“Sell 200 shares of [symbol or name]“
“Sell 50 shares of [symbol or name] at limit price of 10 dollars”

Chat Bot

When it comes to interpreting voice inputs, the Etrade app is much more forgiving than Google’s. For instance, while only “Take me to the dashboard” is listed as a voice command,
“Open the dashboard” or “Show me the dashboard”, or “Can I see the dashboard”, is understood and does open the application’s dashboard view. However, the monotone voice and demeanor of the bot suggests some kind of bitterness, more than even Apple’s Siri does. This of course is hight subjective.

Besides what is listed as an available voice command, the Etrade app doesn’t seem to be very communicative. Simple questions like “How did the market close today?”,“How are you?”, or “What is your name?” all trigger the same answer “One moment please – I’m sorry I could not process that.“

Dissecting the mobile application

Once the microphone button is pressed, a voice prompt is played “How can I help you?” and the application goes into listening mode. It will keep listing until a drop in amplitude is recognized. I.e., if I didn’t say anything, it kept listening, probably until the phone runs out of battery.
When an amplitude drop is recognized, the sound recording is sent for speech recognition and acknowledged with another voice prompt: “One moment please.”
The recognition of an input like “Please give me the stock price on Apple” creates a spoken output like “Price 439.88 Change Percent Minus 2.36 Percent”. The time between the end of the request and the beginning of the response is a little over 5 seconds. However, the voice prompt “One moment please” and the fact that the app shows a view with the requested quote, even before it starts playing the synthesized response, are clever tricks to obscure the relative long delay.

Since the application doesn’t seem to include the standard Nuance Voice Kit, I am assuming the sound file is not streamed during the recording (the forever listing behavior would be another indication), but rather encoded and sent via HTTP POST once the recording has stopped.
The response is synthesized by a Web-service, instead of on the mobile device itself. While this has the advantage that no high quality voice synthesizer needs to be deployed with the app, the pre-recorded and included voice prompts sound differently, in quality and also noticeably in loudness.

Final Thoughts

ETrade has done a great first step, voice-enabling their mobile application. It’s better than Google’s Voice Commands, since it’s much less rigid. And while voice and demeanor remind us of Siri, it’s good to see trailblazers like Etrade pioneering Voice User Interfaces on mobile.

Demo

Take a look and a listen how I interacted with ETrade’s mobile Android app via Voice Commands:

Sh!t we dislike – I’m Watch

Wolf Paulus — Wed, 23 Jan 2013 08:00:28 +0000

The broad introduction of Voice User Interfaces, which allow humans to interaction with computers through voice/speech, may be the next revolution, when it comes to User Interface Engineering and represent an even bigger change, when compared to the transition from text-based to graphical user interfaces. A Voice User Interfaces seems to be especially attractive on smaller devices, like small phones or watches that don’t allow for many buttons or have a touch-screen that is just not big enough, to allow for comfortable interactions.

The first generation of watches that connect to smartphone via Bluetooth is now available, most of them are not equipped with a microphone and speaker, i.e. don’t provide a good platform for a Voice User Interface.

Pebble, E-Paper Watch for iPhone and Android.
Smartwatch, Sony Android watch.
MOTOACTV, Motorola GPS Fitness Tracker with MP3 Player.
NEXD, The Next-gen Android watch.
i’m Watch, i’m SpA, Smartwatch connects via Bluetooth to a phone.

i’m Watch

However, there is i’m SpA, an Italian company, maker of the I’M WATCH, an Android 1.6 powered watch that works as a Bluetooth extension for iPhone and Android phones and comes with microphone and speaker on board.

Gestures

I really wanted to like the i’m Watch, but a couple of significant shortcomings became immediately obvious, during the setup process.
There is a delay with all touch and swipe gestures, which we are just not used to anymore. A response often comes delayed and you end up pressing buttons twice. Mainly due to the small display size, gestures seem too sensitive, i.e. minuscule gestures create unexpected effects.
The notification panel, which is heavily used, is too small and therefore hard to access, when dragging it downwards to open it.

Audio Quality

Using the watch as an Bluetooth extension for a phone, e.g. to make phone calls, is virtually impossible. The audio quality for both playback and recording is terribly bad.

Internet Access

Most applications running on the Watch need to access the Internet, which means that the smart phone that connects to the watch via Bluetooth, needs to act as hotspot for Bluetooth devices. Enabling this feature on your smartphone requires to upgrade ($$) your dataplan (at least for Verizon and AT&T customers).

Software development

Software development, at least for the purpose I wanted to use the watch for, turned into something close to a nightmare. There are two Android 1.6 images available for the Watch, one has adb enabled, i.e. allows to deploy Android apps, the other one allows to mount the watch as a USB drive – it’s either or. I went with the adb enabled image for developers, but the Watch still doesn’t easily show up with “adb devices”. Instead, one needs to setup a RNDIS network device and hardcode 10.0.0.168 as its ANDROID HOST IP. adb needs to be manally killed and restarted every time the watch gets (re-)connected. Which needs to happens very often, since a DRRS-plug is the only way to tether the watch for deployment, but while connected, the DRRS-plug also shuts-off microphone and speaker.

SpeechSynthesis

While android.speech.tts.TextToSpeech was introduced with Android 1.6, there was no implementation provided on the i’m Watch. However, after a little bit of searching around, I came across com.svox.langpack.installer.apk and PicoLangInstaller.apk, which can be installed on the watch (adb install ..). This eventually helped to enable a crummy but still understandable voice output.

//
// TextToSpeech.OnInitListener listener
//

   	@Override
   	public void onInit(int i) {
       	tv.setText("onInit");
       	tts.setPitch(1.2f);
       	tts.setSpeechRate(1.1f);
       	tv.setText("Locale:" + tts.getLanguage().toString());
       	tts.speak("Hello, how are you today?", TextToSpeech.QUEUE_FLUSH, null);
   	}

Since that didn’t take too long, I was optimistic to get SpreechRecognition working as well. However, huge portions of the Android 1.6 SDK are simply missing. E.g. RecognizerIntent is not implemented and MediaRecorder will always fail when prepare() is called. Meaning, I ended up using AudioRecord for the voice recording. Something like so:

private void startRecording() {

        mRecorder = new AudioRecord(
                MediaRecorder.AudioSource.MIC,
                RECORDER_SAMPLERATE,
                RECORDER_CHANNELS,
                RECORDER_AUDIO_ENCODING,
                mBufferSize);

        mRecorder.startRecording();
        isRecording = true;

        mRecordingThread = new Thread("Recording Thread") {
            @Override
            public void run() {
                writeAudioDataToFile();
            }
        };
        mRecordingThread.start();
    }

    /**
     * writeAudioDataToFile writes the raw recording buffer into a temp. file.
     */
    private void writeAudioDataToFile() {
        final byte data[] = new byte[mBufferSize];
        try {
            final OutputStream os = new BufferedOutputStream(new FileOutputStream(getTempFilename()));
            while (isRecording) {
                final int k = mRecorder.read(data, 0, mBufferSize);
                if (k != AudioRecord.ERROR_INVALID_OPERATION) {
                    try {
                        os.write(data, 0, k);
                    } catch (IOException e) {
                        Log.e(LOG_LAG, e.getMessage());
                    }
                }
            }
            try {
                os.flush();
                os.close();
            } catch (IOException e) {
                Log.e(LOG_LAG, e.getMessage());
            }

        } catch (FileNotFoundException e) {
            Log.e(LOG_LAG, e.getMessage());
        }
    }

    private void stopRecording() {
        if (mRecorder != null) {
            isRecording = false;
            mRecorder.stop();
            mRecorder.release();
            mRecorder = null;
            mRecordingThread = null;
        }

        final String wavFile = getFilename();
        copyWaveFile(getTempFilename(), wavFile);
        new File(getTempFilename()).delete();
        // AsyncTask calling SpeechRecognition web service ..
        new NAPI(mHandler).execute(getHttp_Api_Url(), wavFile );
    }

Unfortunately, the sound quality was once again very poor and Nuance’s Web service for Speech Recognition could not render any results. After significantly improving the wav-file using Wiretap Studio on my Mac, Nuance would finally return a result, which wasn’t even too far off. However, even if the improvement could be done on the i’m Watch itself, the total processing time was just way too long that even when considering all optimizations, we would never get into usable range.

List of things that won’t work

Sensors APIs require special library, which have not been released yet.
The i’m Watch has a custom audio system. Android Audio recording/routing APIs are not working (Manufacturer has acknowledged the problem, but no ETA)
Speech recognition APIs don’t work due to the previous point.
NetworkManager.isConnected() always returns true (Manufacturer has acknowledged the problem, but no ETA)
Option menus are not working (UI/UX decision: screen is too small for this) (Manufacturer promised to publish a fix on their GitHub account soon.)
Vibration APIs are not working (no vibration motor in the i’m Watch)
Input Method Editors are not supported (UI/UX decision: screen is too small for this) (Manufacturer promised to publish a fix for the NumPad at least on their GitHub account soon.)
Links that open the web browser are not supported. The web browser is present, but will be removed in a future release (UI/UX decision: screen is too small for this)

Final Verdict

Requested RMA. Returned i’m Watch ..

A Daunting Task – Tiffany goes to Cupertino

Wolf Paulus — Mon, 21 Jan 2013 04:12:19 +0000

TiffanyScreens in the Mac App Store

I never really liked going to “PowerPoint Meetings”, sharing the screen content with others during a meeting, usually requires to connect a projector to the presenter’s laptop. In a lengthy process, the laptop’s screen resolution and refresh-rate needed to be manually adjusted to synchronize with the projector. What follows is a lecture style presentation, featuring slides being projected, hugely magnified onto one of the meeting room’s walls; often, lights need to be dimmed, and listeners either doze off or start checking email.

A couple years back, I wrote TiffanyScreens, which allows you to share presentations (or any screen content), without requiring a projector.
Imagine a scenario, where every participant brought a laptop computer to a meeting and watched the presentation on that laptop’s display – participants sit on a table facing each other, instead of the wall. No adjustments are necessary; images are scaled automatically on arrival, to best match the receiver’s display-capability. To support lively meetings, everyone participating, can with a single button click, turn his computer into the presenting device.

During the development, I had small engineering groups in mind, but it became more and more obvious that instructors, educators, and educational institutions would appreciate TiffanyScreens the most. Listening to users’ feedback, TiffanyScreens has been iteratively improved, refined, and optimized.

For instance, to accommodate instructors and teachers, only one peer needs to have a licensed copy of the application installed and since you never know what kind of laptop users bring, TiffanyScreens is of course a multi-platform application, running on Windows, Linux, and the Mac. In fact, most of TiffanyScreens is implemented in Java.

But almost like the boiling frog story, Apple slowly turned up the heat on Java and Java developers. OS X 10.7 Lion introduced the Mac Application Store, in which Java applications were not welcome. OS X 10.8 Mountain Lion makes it extra hard for users to install applications that are not distributed through the Mac App Store.

With Apple withdrawing most of its Java support, it’s now Oracle, providing up to date Java runtime and Java SDK of Mac OS X. However, recent Zero-Day Java security vulnerabilities make it less like that users’ machines have a Java runtime installed.

Considering all these obstacles, I could have let TiffanyScreens run its course, letting it die a slow death, but during a couple of days off work, over the Christmas break, I came up with a decisive rescue plan … putting TiffanyScreens into the Mac Application Store.

Putting TiffanyScreens into the Mac Application Store

1. Administrative Tasks

Trying to put a Java desktop app into Apple’s Mac App Store is a daunting task, no mean feat, some may even call it a total waste of time. However, TiffanyScreens, the innovative presentation and screen sharing tool, used and loved by many educational institutions worldwide, must not die …

1.1. Joining the Mac Developer Program

It all started with a $99 investment, purchasing a Mac Developer ID and getting into the Mac Developer Program.

https://developer.apple.com/programs/start/standard/

1.2. Request a OS X Signing Certificate

The next step was all about preparing my machine to sign and submit a binary to the Mac App Store approval process.
https://developer.apple.com/certificates/index.action#maccertrequest

Since TiffanyScreens does neither need the iCloud nor Apple Push Notifications, I did forgo the Mac App Development Certificate but requested and installed:

Mac App Certificate
Mac Installer Certificate
Developer ID Application Certificate
Developer ID Installer Certificate

1.3. Creating an Application Bundle ID

I created and Application Bundle ID here: https://developer.apple.com/certificates/index.action#bundle
An App ID is an identifier used by iOS and Mac OS X to recognize your app. This was pretty simple and straight forward. I just needed to enter my applications name and a bundle identifier.

TiffanyScreens Pro
com.carlsbadcubes.tiffany.TiffanyScreens.pro

1.4. Creating an Application on the iTunes Store

I opened https://itunesconnect.apple.com/ and in “Manage Your Applications”, selected “Add New App” followed by “Mac OS X App”
Here, the application got defined in more detail, like it will appear in the iTunes Store. Completing this step also included deciding on a primary and secondary application category, as well as uploading a couple of screen shots. I think it’s important to note that once an application has been approved, those screen shots cannot easily be changed.

This pretty much concluded the administrative tasks and now the much more interesting engineering tasks began.

2. Engineering Tasks

2.1. Java Runtime

Since I wanted to bundle TiffanyScreens with the latest Java Runtime, I downloaded and deployed JDK 7 on my development and build machine.
http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html
Installing jdk-7u11-macosx-x64.dmg, the most recent Java release for 64-bit Intel Macs, meant getting something like this when typing java -version into a terminal.
java version "1.7.0_11" Java(TM) SE Runtime Environment (build 1.7.0_11-b21) Java HotSpot(TM) 64-Bit Server VM (build 23.6-b04, mixed mode)

In ~/.bash_profile I have JAVA_HOME declared like this:
export set JAVA_HOME=`/usr/libexec/java_home`
which in fact points here:
/System/Library/Frameworks/JavaVM.framework/Versions/Current/Commands/java_home
which is an executable and returns:
/Library/Java/JavaVirtualMachines/jdk1.7.0_11.jdk/Contents/Home
and this location contains the Java Runtime (jre) folder, which will eventually be bundled with TiffanyScreens.

2.2. A Single JAR

TiffanyScreens was already packaged to be distributed as a Mac friendly DMG. What I mean with that is that TiffanyScreens was always packaged into a single Jar. 3rd party libraries like Swixml for instance get unpacked

during the build process and re-packaged together with TiffanyScreens’ classes and resources into a single jar file.

The package target for instance in my TiffanyScreeens’ Ant build.xml looks something like this:

2.3. AppBundler Ant Target

Oracle also has a document on its site that explains some of the things I wanted to do, Bundling the JRE with Tiffany. Unfortunately, the example they show is somewhat oversimplified to be really helpful. However, it has links to the Java Application Bundler project, which contains a Jar, providing an Ant target, to bundle the JRE with a Java app.

So I download the appbundler-1.0.jar and read the related documentation.

Going back and force between the documentation and my Ant build file, I ended up with an target configured like this:

Most of tag’s attributes should be obvious, I’m using this build file for both TiffanyScreens version (Pro and Standard), which is what the bid property is used for.

The applicationCategory attribute value should of course match with what was selected as a primary category, when the application entry was created on the iTunes Store. How this value needs to be decoded can be found here.

Since I had selected ‘Productivity’ for the primary category, I used this UTI: public.app-category.productivity

The runtime tag points to the Java_Home location, from where it will pick up the jre.
The classpath tags declare additional locations that need to be put into the bundle as well.
Finally, the option tags will create in the Mac applications Contents/info.plist file.
Once the bundling has happen, as a last thing, I deleted the Java runtime’s info.plist. The submission process seems to be very sensitive about that and only after there was only a single info.plist file left in the bundle was I able to submit the application.

2.5. Application Icon

The Ant target shown above contains two references to the application’s icon, which has to be carefully constructed.
Since the introduction of the “retina display”, Mac application that want to pass the rigorous approval process, need to have an icon file containing icons all the way to 1024×1024 pixels.
After I had created the icon artwork in five resolutions, I carefully named them like so:

icon_16x16.png
icon_128x128.png
icon_256x256.png
icon_512x512.png
icon512x512@2x.png

A tool provided with the latest XCode release allow to combine the icons into a single icns file:
iconutil -c icns /Users/wolf/Work/Tiffany/res/pro.iconset

2.6. Signing the application bundle

Running the appbundle build target ./build/dist/TiffanyScreens Pro.app, already created a perfectly executable Mac application.
ant appbundle .. appbundle: [bundleapp] Creating app bundle: TiffanyScreens Std [delete] Deleting: /Users/wolf/Work/Tiffany/build/dist/TiffanyScreens Std.app/Contents/PlugIns/jdk1.7.0_11.jdk/Contents/Info.plist

BUILD SUCCESSFUL Total time: 4 seconds

Before uploading the application for approval, the application and all packaged libraries needed to be signed. Since TiffanyScreens has all its classes and resources in a single jar, this meant that I only needed to sign the TiffanyScreens.jar file and the one jnilib that I included, .. plus all the libraries in the Java runtime of course.

#!/bin/bash
echo "Using ant build-target 'appbundle' to create the app file"
ant appbundle

echo "Sign everything inside the app file"
codesign --entitlements ./tiffany.plist -s "3rd Party Mac Developer Application: Wolf Paulus" -v "./build/dist/TiffanyScreens Pro.app"
find "./build/dist/TiffanyScreens Pro.app/Contents" -type f \( -name "*.jar" -or -name "*.dylib" \) -exec codesign --verbose -f -s "3rd Party Mac Developer Application: Wolf Paulus" --entitlements tiffany.plist {} \;
find "./build/dist/TiffanyScreens Pro.app/Contents" -type f \( -name "*.exe" -or -name "*.jnilib" \) -exec codesign --verbose -f -s "3rd Party Mac Developer Application: Wolf Paulus" --entitlements tiffany.plist {} \;

echo "Verify all libraries have been signed..."
find "./build/dist/TiffanyScreens Pro.app/Contents" -type f \( -name "*.jar" -or -name "*.dylib" \) -exec codesign --verbose --verify {} \;
find "./build/dist/TiffanyScreens Pro.app/Contents" -type f \( -name "*.exe" -or -name "*.jnilib" \) -exec codesign --verbose --verify {} \;

echo "Create signed package"
productbuild --component "./build/dist/TiffanyScreens Pro.app" /Applications --sign "3rd Party Mac Developer Installer: Wolf Paulus" --product tiffany.plist TiffanyScreensPro.pkg

echo "Test pkg before uploading it to Apple"
sudo installer -store -pkg ./TiffanyScreensPro.pkg -target /

echo "more tests"
spctl --assess --verbose=4 --type execute TiffanyScreensPro.pkg
spctl --assess --verbose=4 --type install TiffanyScreensPro.pkg
codesign -v ./build/dist/TiffanyScreens\ Pro.app/

This bash script runs the Ant script, before doing the code signing (using both certificates: “3rd Party Mac Developer Application: Wolf Paulus” and “3rd Party Mac Developer Installer: Wolf Paulus”) and running two sanity checks, recommended before trying to submit the app.

The outcome of this is a *.pkg file, which is made for submission to the app-store approval process but not for installing on your own Mac.

2.7. Submitting for Approval

With the latest version of XCode installed, the Application Loader can be found by, control-clicking XCode and selecting “Show Package Contents”. The Application Loader in the Contents/Applications folder. Selecting “Deliver Your App” automatically picked up all the information I had provided, when creating the application entry in the iTunes Store. Application Loader runs a lot of tests, before finally accepting and uploading the pkg file.

Waiting

Now the waiting began. While iTunes Connect (https://itunesconnect.apple.com/WebObjects/iTunesConnect.woa) showed the current status of the approval process, the information provided here http://reviewtimes.shinydevelopment.com/ gave a pretty good indication of the average App Store review time.

A day after submitting Tiffany Screens Pro, I submitted TiffanyScreens Std, which obviously only took a fraction of the time. The only work that I really had to do, was to create that insanely great 1024×1024 icon.

Approved

Each application got approved after 8 days and is now available in the Mac application store. Take a look here:
http://itunes.com/mac/wolfpaulus

Sh!it We Dislike – Sigma 30mm f/1.4

Wolf Paulus — Sat, 19 Jan 2013 19:00:58 +0000

Sigma 30mm f/1.4 EX DC HSM Lens

I really wanted to like this lens. It has a nice build quality and has a really good feel and helf. I also liked the larger 62 mm Filter-Thread, compared to comparable lenses. However, mounting the lens already felt somewhat odd. It did not want to go on a Canon 7D body, as easily and smooth as my Canon and Tamron lenses.

Taking the 1st shots and looking at them at the camera’s LCD already was disappointing – the pictures has an odd pale almost green look. After importing the photos into Lightroom we saw the stunningly poor quality. Not even close to an inexpensive 1.8 nifty fifty and worse than any of my zoom lenses in that rage; unbelievably soft and out of focus.

Next we did the common focus test (described in much detail here: http://focustestchart.com/focus21.pdf) and the results showed that the focus was off by 15mm with a 500mm focus distance. But even the areas that looked like were in focus, were totally soft, all the way to f/5.6. Don’t know how Sigma can stay in business when delivering products like this 30mm.

Focus Test

Here is a photos we took during the focus test. To be fair to Sigma, this was the VERY best we could find.

Focal Length: 30mm
Apature: f/1.4
Exposure Time: 1/640 sec
ISO: 400

[Download original image (not cropped) 5184x3456, 13.7 MB]

Specification

Filter Thread: 62mm
Dimensions (DiameterxLength): 3.2 x 2.3″ (8.13 x 5.84 cm)
Weight: 15.1 oz (430 g)
Camera Mount Type: Canon EF-S
Format Compatibility: Canon (APS-C)
Focal Length: 30 mm
Minimum Focus Distance: 15.7″ (40 cm)
Aperture: f/1.4 to f/16
Magnification 0.09x
Angle of View 46 Degrees
Groups/Elements 7/7
Diaphragm Blades 8

Final Verdict

This was my 1st Sigma experience and I’m really glad that Amazon has such a great return policy. This lens stayed for not even 12 hours at my house. Sorry Sigma, but it will probably take a really long time, until I consider one of your products again.

Sh!t We Like – TP-Link Nano Router TL-WR702N

Wolf Paulus — Sat, 12 Jan 2013 19:36:23 +0000

Pack your own parachute and bring your own wireless router

If you ever gave a talk or presentation that involved demonstrating software running on a wireless device, you probably remember some restlessness, approaching the venue.

What would the Wifi conditions be? Would you be able to connect your device(s) to the Wifi network? Would the bandwidth be sufficient, to support a smooth demo? etc. etc.

There is no such thing as perfect and you can never be over-prepared. The same it probably true when it come to display-adapters, especially if a MacBook is your presentation tool of choice. Pack all the Thunderbolt/MiniDisplayPort to DVI/HDMI/VGA adapters you can find. And just in case, I usually export my slides to PDF and put that file on a FAT-formatted USB-Thumb-drive. Moreover, I used to schlep a standard home-office router, together with its bulky power-supply of course. But not anymore …

I recently came across a nifty and tiny Wifi Router that is not only fast enough, but can be powered with a standard 5V phone charger, using a standard Micro-USB Cable – or even better, leeching power off a standard USB-Port (e.g., one of the MacBook’s USB-Ports).

Speedtest (ISP-Cox)

I certainly had my doubts when it came to data transfer rates, but this little guy is seriously fast. Here is a real world test, comparing the TP-Link TL-WR702N to a Cisco Linksys E4200 Wireless-N Dual-Band Wi-Fi Router, on my home network:

Speed Test w/ TL-WR702N

Speed Test w/ LinkSys E4200

The TP-Link TL-WR702N uses the Atheros AR9331 chipset and is equipped with 2MB Flash memory and 16MB RAM, which unfortunately is not quite enough to replace the stock OS with an open-source Linux distribution for embedded devices, like OpenWrt or DD-WRT.

However, the stock software already provides some capabilities, often missing in many full-size home-network routers.
For instance, the TL-WR702N can be operated in five different modes:

AP Mode

This is the WR702N’s default mode. The nano router supplements a wired LAN (to which it connects, using an Ethernet cable) and enables its wireless clients, to connect to that LAN wirelessly.

Wireless Router Mode

The one and only wired port works as WAN Port, e.g. gets connected to a DSL/Cable Modem. The routers works as an DHCP server and wireless clients can obtain IP addresses and DNS automatically.

Repeater Mode

The TL-WR702N is used to extend the range of an existing AP or wireless router.
In this mode, the one and only wired port gets connected to a client laptop or PC via Ethernet cable. However, clients can also connect to the nano router wirelessly. The SSID of TL-WR702N should exactly match the SSID of the device you want to repeat. In Repeater Mode, the DHCP server defaults to off.

Bridge Mode

Pretty much the same as the Repeater Mode, but the the nano router’s SSID does not match any other Wifi-Router’s SSID on your network.

Client Mode

The TL-WR702N is basically used as a wireless network card. I.e. a computer connects via Ethernet cable to the nano router and the TL-WR702N is used to access a wireless network.

At the relatively low price (US$19 to US$29 B&H, Newegg, Amazon, ..) the size, data transfer rate, and feature-set, this is an impressive device. After running it for one week without the need for a reset, it’s now part of my travel backpack ..

TL-WR702N internal Web Server

Specifications:

Wireless Standards 802.11b, 802.11g, 802.11n
Interfaces 10/100Mbps WAN/LAN
Dimensions (WxDxH) 3.2 x 2.8 x 1.1 in (81 x 71 x 28mm)
Frequency 2.4-2.4835 GHz
Signal Rate 11n Up to 150Mbps
Signal Rate 11g Up to 54Mbps
Signal Rate 11b Up to 11Mbps
Security 64/128/152-bit WEP / WPA / WPA2,WPA-PSK / WPA2-PSK