Now most of the data entry or origination happens on the Web, where most of the forms processing has been moved to as well, i.e. OCR hasn’t been in vogue for quite a while. However, the popularity of smartphones, combined with built-in high-quality cameras has created a new category of mobile applications, benefiting greatly from OCR. Take Word-Lens (http://questvisual.com) as an example: an augmented reality translation application that tries to find out what the letters are in an image and then looks in a dictionary, to eventually draws the words back on the screen in translation.

On Device or In The Cloud ?

Before deciding on an OCR library, one needs to decide, where the OCR process should take place: on the Smartphone or in the Cloud. Each approach has its advantages.
On device OCR can be performed without requiring an Internet connection and instead of sending a photo, which can potentially be huge (many phones have 8 or 12 Mega-Pixel cameras now), the text is recognized by an on-board OCR-engine.
However, OCR-libraries tend to be large, i.e. the mobile application will be of considerable size. Depending on the amount of text that needs to be recognized and the available data transfer speed, a cloud-service may provide the result faster. A cloud-service can be updated more easily but individually optimizing (training) an OCR engine may work better when done locally on the device.

Which OCR Library to choose ?

• Wikipedia has a “non-exhaustive” but still very broad comparison of optical character recognition software here: http://en.wikipedia.org/wiki/ List_of_optical_character_recognition_software
• A comparison of some of the more popular OCR-Engines can be found here: http://www.freewaregenius.com/2011/11/01/how-to-extract-text-from-images-a- comparison-of-free-ocr-tools/
• Linux OCR software comparison with a strong focus on accuracy: http://www.splitbrain.org/blog/2010-06/15-linux_ocr_software_comparison

After taking a closer look at the all comparisons, Tesseract stands out. It provides good accuracy, it’s open-source and Apache-Licensed, and has broad language support. It was created by HP and is now developed by Google.

Also, since Tesseract is open source and Apache- Licensed, we can take the source and port it to the Android platform, or put it on a Web-server to run our very own Cloud-service.

1. Tesseract

The Tesseract OCR engine was developed at Hewlett Packard Labs and is currently sponsored by Google. It was among the top three OCR engines in terms of character accuracy in 1995. http://code.google.com/p/tesseract-ocr/

1.1. Running Tesseract locally on a Mac

Like with so make other Unix and Linux tools, Homebrew (http://mxcl.github.com/homebrew/) is the easiest and most flexible way to install the UNIX tools Apple didn’t include with OS X. Once Homebrew is installed (https://github.com/mxcl/homebrew/wiki/installation), Tesseract can be installed on OS X as easy as:
$ brew install tessertact
Once installed,
$ brew info tesseract will return something like this:

tesseract 3.00

Tesseract doesn’t come with a GUI and instead runs from a command-line interface. To OCR a TIFF-encoded image located on your desktop, you would do something like this:
$ tesseract ~/Desktop/cox.tiff ~/Desktop/cox
Using the image below, Tesseract wrote with perfect accuracy the resulting text into
~/Desktop/cox.txt

There are at least two projects, providing a GUI-front-end for Tesseract on OS X

1. TesseractGUI, a native OSX client: http://download.dv8.ro/files/TesseractGUI/
2. VietOCR, a Java Client: http://vietocr.sourceforge.net/

TesseractGUI, a native OSX Client for Tesseract

VietOCR, a Java Client for Tesseract

1.2. Running Tesseract as a Could-Service on a Linux Server

One of the fastest and easiest ways to deploy Tesseract as a Web-service, uses Tornado (http://www.tornadoweb.org/), an open source (Apache Licensed) Python non-blocking web server. Since Tesseract accepts TIFF encoded images but our Cloud-Service should rather work with the more popular JPEG image format, we also need to deploy the free Python Imaging Library (http://www.pythonware.com/products/pil/), license terms are here: http://www.pythonware.com/products/pil/license.htm

The deployment on Ubuntu 11.10 64-bit server looks something like this:

sudo apt-get install python-tornado
sudo apt-get install python-imaging
sudo apt-get install tesseract-ocr

1.2.1. The HTTP Server-Script for port 8080

#!/usr/bin/env python
import tornado.httpserver
import tornado.ioloop
import tornado.web
import pprint
import Image
from tesseract import image_to_string
import StringIO
import os.path
import uuid

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.write('</pre>
<form action="/" method="post" enctype="multipart/form-data">' '
<input type="file" name="the_file" />' '
<input type="submit" value="Submit" />' '</form>
<pre class="prettyprint">')

    def post(self):
        self.set_header("Content-Type", "text/html")
	self.write("") # create a unique ID file
        tempname = str(uuid.uuid4()) + ".jpg"
        myimg = Image.open(StringIO.StringIO(self.request.files.items()[0][1][0  ['body']))
        myfilename = os.path.join(os.path.dirname(__file__),"static",tempname); 

        # save image to file as JPEG
        myimg.save(myfilename) 

        # do OCR, print result
        self.write(image_to_string(myimg))
        self.write("")

settings = {
    "static_path": os.path.join(os.path.dirname(__file__), "static"),
}

application = tornado.web.Application([
    (r"/", MainHandler),
], **settings)

if __name__ == "__main__":
    http_server = tornado.httpserver.HTTPServer(application)
    http_server.listen(8080)
    tornado.ioloop.IOLoop.instance().start()

The Server receives a JPEG image file and stores it locally in the ./static directory, before calling image_to_string, which is defined in the Python script below:

1.2.2. image_to_string function implementation

#!/usr/bin/env python

tesseract_cmd = 'tesseract'

import Image
import StringIO
import subprocess
import sys
import os

__all__ = ['image_to_string']

def run_tesseract(input_filename, output_filename_base, lang=None, boxes=False):
    '''
    runs the command:
        `tesseract_cmd` `input_filename` `output_filename_base`

    returns the exit status of tesseract, as well as tesseract's stderr output

    '''

    command = [tesseract_cmd, input_filename, output_filename_base]

    if lang is not None:
        command += ['-l', lang]

    if boxes:
        command += ['batch.nochop', 'makebox']

    proc = subprocess.Popen(command,
            stderr=subprocess.PIPE)
    return (proc.wait(), proc.stderr.read())

def cleanup(filename):
    ''' tries to remove the given filename. Ignores non-existent files '''
    try:
        os.remove(filename)
    except OSError:
        pass

def get_errors(error_string):
    '''
    returns all lines in the error_string that start with the string "error"

    '''

    lines = error_string.splitlines()
    error_lines = tuple(line for line in lines if line.find('Error') >= 0)
    if len(error_lines) > 0:
        return '\n'.join(error_lines)
    else:
        return error_string.strip()

def tempnam():
    ''' returns a temporary file-name '''

    # prevent os.tmpname from printing an error...
    stderr = sys.stderr
    try:
        sys.stderr = StringIO.StringIO()
        return os.tempnam(None, 'tess_')
    finally:
        sys.stderr = stderr

class TesseractError(Exception):
    def __init__(self, status, message):
        self.status = status
        self.message = message
        self.args = (status, message)

def image_to_string(image, lang=None, boxes=False):
    '''
    Runs tesseract on the specified image. First, the image is written to disk,
    and then the tesseract command is run on the image. Resseract's result is
    read, and the temporary files are erased.

    '''

    input_file_name = '%s.bmp' % tempnam()
    output_file_name_base = tempnam()
    if not boxes:
        output_file_name = '%s.txt' % output_file_name_base
    else:
        output_file_name = '%s.box' % output_file_name_base
    try:
        image.save(input_file_name)
        status, error_string = run_tesseract(input_file_name,
                                             output_file_name_base,
                                             lang=lang,
                                             boxes=boxes)
        if status:
            errors = get_errors(error_string)
            raise TesseractError(status, errors)
        f = file(output_file_name)
        try:
            return f.read().strip()
        finally:
            f.close()
    finally:
        cleanup(input_file_name)
        cleanup(output_file_name)

if __name__ == '__main__':
    if len(sys.argv) == 2:
        filename = sys.argv[1]
        try:
            image = Image.open(filename)
        except IOError:
            sys.stderr.write('ERROR: Could not open file "%s"\n' % filename)
            exit(1)
        print image_to_string(image)
    elif len(sys.argv) == 4 and sys.argv[1] == '-l':
        lang = sys.argv[2]
        filename = sys.argv[3]
        try:
            image = Image.open(filename)
        except IOError:
            sys.stderr.write('ERROR: Could not open file "%s"\n' % filename)
            exit(1)
        print image_to_string(image, lang=lang)
    else:
        sys.stderr.write('Usage: python tesseract.py [-l language] input_file\n')
        exit(2)

1.2.3. The Service deploy/start Script

description  "OCR WebService"

start on runlevel [2345]
stop on runlevel [!2345]

pre-start script
mkdir /tmp/ocr

mkdir /tmp/ocr/static

cp /usr/share/ocr/*.py /tmp/ocr

end script
exec /tmp/ocr/tesserver.py

After the service has been started, it can be accessed through a Web browser like shown here: http://proton.techcasita.com:8080 I’m currently running tesseract 3.01 on Ubuntu Linux 11.10 64-bit, please be gentle, it runs on an Intel Atom CPU 330 @ 1.60GHz, 4 cores (typically found in Netbooks) The HTML encoded result looks something like this:

<html><body>Contact Us
www. cox.com
Customer Serv 760-788-9000
Repair 76O—788~71O0
Cox Telephone 888-222-7743</body></html>

1.3 Accessing the Tesseract Cloud-Service from Android

The OCRTaskActivity below utilizes Android’s built-in AsyncTask as well as Apache Software Foundation’s HttpComponent library HttpClient4.1.2, available here: http://hc.apache.org/httpcomponents-client-ga/index.html OCRTaskActivity expects the image to be passed in as the Intent Extra “ByteArray” of type ByteArray. The OCR result is returned to the calling Activity as OCR_TEXT, like shown here:

setResult(Activity.RESULT_OK, getIntent().putExtra("OCR_TEXT", result));

import android.app.Activity;
import android.graphics.BitmapFactory;
import android.os.AsyncTask;
import android.os.Bundle;
import android.util.Log;
import android.view.View;
import android.widget.ImageView;
import android.widget.ProgressBar;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.mime.HttpMultipartMode;
import org.apache.http.entity.mime.MultipartEntity;
import org.apache.http.entity.mime.content.ByteArrayBody;
import org.apache.http.entity.mime.content.StringBody;
import org.apache.http.impl.client.DefaultHttpClient;

import java.io.BufferedReader;
import java.io.InputStreamReader;

public class OCRTaskActivity extends Activity {
    private static String LOG_TAG = OCRAsyncTaskActivity.class.getSimpleName();
    private static String[] URL_STRINGS = {"http://proton.techcasita.com:8080"};

    private byte[] mBA;
    private ProgressBar mProgressBar;

    @Override
    public void onCreate(final Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.ocr);
        mBA = getIntent().getExtras().getByteArray("ByteArray");
        ImageView iv = (ImageView) findViewById(R.id.ImageView);
        iv.setImageBitmap(BitmapFactory.decodeByteArray(mBA, 0, mBA.length));
        mProgressBar = (ProgressBar) findViewById(R.id.progressBar);
        OCRTask task = new OCRTask();
        task.execute(URL_STRINGS);
    }

    private class OCRTask extends AsyncTask {
        @Override
        protected String doInBackground(final String... urls) {
            String response = "";
            for (String url : urls) {
                try {
                    response = executeMultipartPost(url, mBA);
                    Log.v(LOG_TAG, "Response:" + response);
                    break;
                } catch (Throwable ex) {
                    Log.e(LOG_TAG, "error: " + ex.getMessage());
                }
            }
            return response;
        }

        @Override
        protected void onPostExecute(final String result) {
            mProgressBar.setVisibility(View.GONE);
            setResult(Activity.RESULT_OK, getIntent().putExtra("OCR_TEXT", result));
            finish();
        }
    }

    private String executeMultipartPost(final String stringUrl, final byte[] bm) throws Exception {
        HttpClient httpClient = new DefaultHttpClient();
        HttpPost postRequest = new HttpPost(stringUrl);
        ByteArrayBody bab = new ByteArrayBody(bm, "the_image.jpg");
        MultipartEntity reqEntity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);
        reqEntity.addPart("uploaded", bab);
        reqEntity.addPart("name", new StringBody("the_file"));
        postRequest.setEntity(reqEntity);
        HttpResponse response = httpClient.execute(postRequest);
        BufferedReader reader = new BufferedReader(new InputStreamReader(response.getEntity().getContent(), "UTF-8"));
        String sResponse;
        StringBuilder s = new StringBuilder();

        while ((sResponse = reader.readLine()) != null) {
            s = s.append(sResponse).append('\n');
        }
        int i = s.indexOf("body");
        int j = s.lastIndexOf("body");
        return s.substring(i + 5, j - 2);
    }
}

This sample Android app has an Activity that sends a small JPEG image to the Cloud-Service, which is running the Tesseract OCR engine.

1.4. Building a Tesseract native Android Library to be bundled with an Android App

This approach allow an Android application to perform OCR even without a network connection. I.e. the OCR engine is on-board. There are currently two source-bases to start from, the original Tesseract project here:

1. Tesseract Tools for Android is a set of Android APIs and build files for the Tesseract OCR and Leptonica image processing libraries:

   svn checkout http://tesseract-android-tools.googlecode.com/svn/trunk/ tesseract-android-tools

2. A fork of Tesseract Tools for Android (tesseract-android-tools) that adds some additional functions:

   git clone git://github.com/rmtheis/tess-two.git

… I went with option 2.

1.4.1. Building the native lib

Each project can be build with the same build steps (see below) and neither works with Android’s NDK r7. However, going back to NDK r6b solved that problem. Here are the build steps. It takes a little while, even on a fast machine.

cd <project-directory>/tess-two
export TESSERACT_PATH=${PWD}/external/tesseract-3.01
export LEPTONICA_PATH=${PWD}/external/leptonica-1.68
export LIBJPEG_PATH=${PWD}/external/libjpeg
ndk-build
android update project --path .
ant release

The build-steps create the native libraries in the libs/armabi and libs/armabi-v7a directories.

The tess-two project can now be included as a library-project into an Android project and with the JNI layer in place, calling into the native OCR library now looks something like this:

JNI and Native Libraries

1.4.2. Developing a simple Android App with built-in OCR capabilities

...
TessBaseAPI baseApi = new TessBaseAPI();
baseApi.init(DATA_PATH, LANG);
baseApi.setImage(bitmap);
String recognizedText = baseApi.getUTF8Text();
baseApi.end();
...

1.4.2.1. Libraries / TrainedData / App Size

The native libraries are about 3 MBytes in size. Additionally, a language and font depending training resource files is needed.
The eng.traineddata file (e.g. available with the desktop version of Tesseract) is placed into the main android’s assers/tessdata folder and deployed with the application, adding another 2 MBytes to the app. However, due to compression, the actual downloadable Android application is “only” about 4.1 MBytes.

During the first start of the application, the eng.traineddata resource file is copied to the phone’s SDCard.

The ocr() method for the sample app may look something like this:

protected void ocr() { 

        BitmapFactory.Options options = new BitmapFactory.Options();
        options.inSampleSize = 2;
        Bitmap bitmap = BitmapFactory.decodeFile(IMAGE_PATH, options); 

        try {
            ExifInterface exif = new ExifInterface(IMAGE_PATH);
            int exifOrientation = exif.getAttributeInt(ExifInterface.TAG_ORIENTATION, ExifInterface.ORIENTATION_NORMAL); 

            Log.v(LOG_TAG, "Orient: " + exifOrientation); 

            int rotate = 0;
            switch (exifOrientation) {
                case ExifInterface.ORIENTATION_ROTATE_90:
                    rotate = 90;
                    break;
                case ExifInterface.ORIENTATION_ROTATE_180:
                    rotate = 180;
                    break;
                case ExifInterface.ORIENTATION_ROTATE_270:
                    rotate = 270;
                    break;
            } 

            Log.v(LOG_TAG, "Rotation: " + rotate); 

            if (rotate != 0) { 

                // Getting width & height of the given image.
                int w = bitmap.getWidth();
                int h = bitmap.getHeight(); 

                // Setting pre rotate
                Matrix mtx = new Matrix();
                mtx.preRotate(rotate); 

                // Rotating Bitmap
                bitmap = Bitmap.createBitmap(bitmap, 0, 0, w, h, mtx, false);
                // tesseract req. ARGB_8888
                bitmap = bitmap.copy(Bitmap.Config.ARGB_8888, true);
            } 

        } catch (IOException e) {
            Log.e(LOG_TAG, "Rotate or coversion failed: " + e.toString());
        } 

        ImageView iv = (ImageView) findViewById(R.id.image);
        iv.setImageBitmap(bitmap);
        iv.setVisibility(View.VISIBLE); 

        Log.v(LOG_TAG, "Before baseApi"); 

        TessBaseAPI baseApi = new TessBaseAPI();
        baseApi.setDebug(true);
        baseApi.init(DATA_PATH, LANG);
        baseApi.setImage(bitmap);
        String recognizedText = baseApi.getUTF8Text();
        baseApi.end(); 

        Log.v(LOG_TAG, "OCR Result: " + recognizedText); 

        // clean up and show
        if (LANG.equalsIgnoreCase("eng")) {
            recognizedText = recognizedText.replaceAll("[^a-zA-Z0-9]+", " ");
        }
        if (recognizedText.length() != 0) {
            ((TextView) findViewById(R.id.field)).setText(recognizedText.trim());
        }
    }

OCR on Android

The popularity of smartphones, combined with built-in high-quality cameras has created a new category of mobile applications, benefiting greatly from OCR.

OCR is very mature technology with a broad range of available libraries to chose from. There are Apache and BSD licensed, fast and accurate solutions available from the open-source community, I have taken a closer look at Tesseract, which is developed by HP and Google.

Tesseract can be used to build a Desktop application, a CloudService, and even baked into a mobile Android application, performing on-board OCR. All three variation of OCR with the Tesseract library have been demonstrated above.

Focussing on mobile applications, however, it became very clear that even on phones with a 5MP camera, the accuracy of the results still vary greatly, depending on lighting conditions, font, and font-sizes, as well as surrounding artifact.

Just like with the TeleForm application, even the best OCR engines perform purely, if the input-image has not been prepared correctly. To make OCR work on a mobile device, no matter if the OCR will eventually be run onboard or in the cloud, much development time needs to be spend to train the engine – but even more importantly, to select and prepare the image areas that will be provided as input to the OCR engine – it’s going to be all about the pre-processing.

This shows my Capture OCR sample Android-OCR application (with Tesseract OCR engine built-in), after it performed the OCR on a just taken photo of a book cover.

“Your father’s light saber. This is the weapon of a Jedi Knight. Not as clumsy or random as a blaster; an elegant weapon for a more civilized age. For over a thousand generations, the Jedi Knights were the guardians of peace and justice in the Old Republic. Before the dark times… before the Empire.”

While the Android 4.0 SDK comes with a complete set of javadocs, the source code of the SDK is missing in the SDK distribution. This is very unfortunate, since you cannot easily debug into SDK methods (at least not without running into de-compiled code) nor can you see how things actually work.

Eclipse - Source Not Found

However, there is a quick fix to that problem. I downloaded the complete Android source including the Linux, drivers, libs, etc., like explained here: http://source.android.com/source/download.html and ran small Java program on the source tree. I used to this with a simple bash script but over the last couple of Android Releases, the java source locations got a little more diverse and I started missing a couple files. So instead, this Java program walks the source tree and looks for java source files. All those will then be copied into a new location, considering their package name. Finally, the jar tool gets called to put all the source into a single bundle for easier handling.

package com.techcasita.tools;
import java.io.*;
/**
 * Find all of Android's Java Sources, consider their package names,
 * and place it in a jar file.
 *
 * @author <a href="mailto:wolf@wolfpaulus.com">Wolf Paulus</a>
 */
public class DroidSource {
    private static String TargetPath;

    public static void main(String[] args) {
        if (args.length == 2 &amp;&amp; new File(args[0]).isDirectory()) {
            DroidSource.TargetPath = args[1];
            File t = new File(args[1]);
            if ((t.exists() &amp;&amp; t.isDirectory() &amp;&amp; t.listFiles().length == 0) || t.mkdir()) {
                traverse(new File(args[0]));
                try {
                    Runtime.getRuntime().exec("jar cf " + t.getParentFile().getAbsolutePath()
+ File.separatorChar + "android-10-src.jar -C " + TargetPath + File.separatorChar + " .");
                } catch (IOException e) {
                    e.printStackTrace();
                }
            } else {
                System.out.println("Usage: " + DroidSource.class.getName()
+ "  ");
                System.out.println(" should exist and be empty");
            }
        } else {
            System.out.println("Usage: " + DroidSource.class.getName()
+ "  ");
        }
    }

    static void traverse(File file) {
        if (file != null &amp;&amp; !file.isHidden()) {
            if (file.isDirectory()) {
                if (!"out".equals(file.getName())) {
                    File[] files = file.listFiles();
                    for (File f : files) {
                        traverse(f);
                    }
                }
            } else if (file.getName().endsWith("java")) {
                String path = findPackage(file);

                if (path != null) {
                    path = path.replace('.', File.separatorChar);
                    createDirs(path);
                    copyFile(file.getAbsolutePath(), TargetPath + File.separatorChar
+ path + File.separatorChar + file.getName());
                }
            }
        }
    }

    private static String findPackage(File file) {
        final String TAG0 = "package ";
        final String TAG1 = ";";
        char[] buffer = new char[2048];

        try {
            FileReader r = new FileReader(file);
            r.read(buffer);
            StringBuffer sb = new StringBuffer();
            sb.append(buffer);
            int t0 = sb.indexOf(TAG0);
            if (t0 &lt; 0) return null;
            t0 += TAG0.length();
            int t1 = sb.indexOf(TAG1, t0);
            if (t1 &lt; 0) return null;
            if (t1 &gt; sb.indexOf(" ", t0)) return null;
            return sb.substring(t0, t1);
        } catch (IOException e) {
            return null;
        }
    }

    private static void createDirs(String path) {
        String[] sa = path.split("" + File.separatorChar);
        String name = TargetPath;
        for (String s : sa) {
            name += File.separatorChar + s;
            File f = new File(name);
            if (!f.exists()) {
                f.mkdir();
            }
        }
    }

    private static void copyFile(String srFile, String dtFile) {
        try {
            File f1 = new File(srFile);
            File f2 = new File(dtFile);
            InputStream in = new FileInputStream(f1);
            OutputStream out = new FileOutputStream(f2);

            byte[] buf = new byte[1024];
            int len;
            while ((len = in.read(buf)) &gt; 0) {
                out.write(buf, 0, len);
            }
            in.close();
            out.close();
        } catch (FileNotFoundException ex) {
            System.out.println(ex.getMessage() + " in the specified directory.");
            System.exit(0);
        } catch (IOException e) {
            System.out.println(e.getMessage());
        }
    }
}

This will generate this android-14-src.jar, (50 MB) containing all the relevant java sources and after attaching it to the SDK classes in your favorite IDE (IntelliJ in my case) you can easily step into the code, for instance during a debug session… The jar was generated from the android-4.0.1_r1 tag here http://android.git.kernel.org on Nov. 14, 2011.

Tap & Go a.k.a PayPass, is a new simple way of paying. PayPass is a payment method that lets you make purchases without having to swipe your card or provide your signature. A simple tap with a card, key fob, or mobile phone is all it takes to pay at checkout.

So this Saturday morning, I took paying with a mobile phone to the test – the only method of payment available to me was the Google Wallet application on a Samsung Nexus S Android Phone running on Sprint’s 4G Network.

Google Wallet can be linked to a Citi MasterCard, or like I did, used as a prepaid card, funded with any of my existing credit cards.

I started buying a small cup of coffee at the 7-11 at 13985 Torrey Del Mar Dr, San Diego, CA 92130
Both checkouts were equipped with PayWave contact-less hand readers and the clerk seemed quite familiar with the process.

Buying a small cup of joe for $1.49 at 7-11 was quick and eventless. Two touches with the phone and I was on my way.

Next up, the CVS/pharmacy at 9225 Twin Trails Dr, San Diego, CA 92129. I bought a toothbrush for $3.22 and once again, the checkout clerk was familiar with this form of payment, recognizing the Google Wallet app. Two touches with the phone and I was on my way to a nearby Rite Aid.

What goes hand in hand with a toothbrush … toothpaste of course. At the Rite Aid at 13167 Black Mountain Rd, San Diego, CA 92129, I bought a tube of Colgate for $5.16. All cash registers were equipped with the PayWave contact-less readers and the clerk was not surprised seeing me pay with a phone.

There is Stater Brothers Market at 9909 Carmel Mountain Rd. nearby, where I wanted to get a baguette. Unfortunately, they weren’t too keen about us filming the checkout process, but paying with the phone was almost as easy as at all previous locations. This time however, an additional interaction with the PayWave station was required, I needed to tap the “Accept” button on the reader’s small touch-screen display.

Four for four; not too bad so far. Let’s move on to some bigger stores in the immediate vicinity. At the Best Buy at 11160 Rancho Carmel Dr, San Diego, CA 92128, I bought a cool RocketFish Stylus for tablets like the iPad. Of course, it works extremely well on Android Tablets like the Xoom or the Samsung Gallaxy Tab 10.1 as well. You should try it someday with apps like Artist Pro on Android.

$18.30 would be my biggest purchase for the day and I should mention, “paying without signature” is limited to payments under $50. While all cash registers were equipped with PayWave, the one I was using was broken. However, the guys at Best Buy swiftly moved me to another line and after two touches with the phone and I was on my way.

Next stop, Sports Authority at 11690 Carmel Mountain Rd, San Diego, CA 92128. I bought a baseball for $3.22 and while the clerk was telling me that this was the first time that he saw someone paying with a phone, it worked flawlessly. Again, tap – enter Pin – tap and I was on my way .. to the Home Depot.

Last stop for the day was the Home Depot at 12185 Carmel Mountain Rd, San Diego, CA 92128, were I wanted to get a strap wrench. Only one (Number 11) of the 14 registers was equipped with a PayWave reader and the clerk seemed completely clue-less. I attempted to pay just like I did in all the other stores but the payment was rejected three times, after which I left the store empty-handed and a little disillusioned.

To be fair, we did our homework before we started. We researched, which nearby stores accepted PayPasse and planned the route accordingly. Six out of seven therefore sounds a little better then it actually is. Paying with Tap & Go is convenient, no question about it. However, don’t leave your house without a card just yet.

At a Glance

Now, don’t think just because PayPass is not ubiquitous yet, it’s not cool. We think it’s awesome and to let you experience the cool ways of Tap & Go with us, we produced this short firm, showing “A Saturday Morning with Google wallet”. We really hope you are going to like it: .. click to play ..

Direct Movie Link

http://wolfpaulus.com/GoogleWallet/GoogleWallet.mov
 
 

For the last couple of weeks now, we have started experimenting with USB-tethered Android screen capturing in combination with TiffanyScreens.

Today we are announcing

TiffanyScreens with Android ScreenCasting.

TiffanyScreens can now share the screen content of an Android phone that is connected via USB-cable to your PC or Mac, with every one else in the room, running TiffanyScreens. Think how this is going to change your UI/UX design meetings, your mobile engineering meetings, your product management / marketing meeting, or your seminars and demos.

You don’t need to have an Android SDK installed on your computer nor do you need to run any special application on your Android phone. If you present from a Windows PC, you may need to install the appropriate USB driver and USB-Debugging needs to be enabled on the phone.
With these small pre-requisites out of the way, a presenter will see a small drop-down on the lower-right, allowing him to easily switch the presentation’s source, between computer and phone screens.

“The C Programming Language” originally published in 1977. My German copy was printed in 1983.

Since most participants seem to bring a laptop, to those meetings to check Email, IM, etc. I had the idea to instead of the wall, use the laptops’ screens as a presentation canvas, i.e. we could all sit on a round table again, facing each other, instead of the wall.

I wrote a small self-contained (doesn’t use a server) multi-platform application a.k.a TiffanyScreens, to capture the presenter’s screen content and send it to the other participants. Best of all, with a simple push of a button, an observing meeting participant becomes a presenter, showing her screen’s content to the other team members.

Up to now, TiffanyScreens was implemented in pure Java. While the installer was implemented and compiled on the native platform, the application itself did not take advantage of OS native capabilities.

Up to now that is. TiffanyScreens 4.0 taps into native libraries and the results are stunning. Instead of using Java’s slow screen capture support, TiffanyScreen now uses Mac OS X native foundation to capture and send up 30 frames per second, marking a three-fold increase. Using native OS functionality also allows the selection of the monitor to capture, which may become important, if you have a 2nd monitor connected to your laptop.

By default and on systems were native support is not yet available, TiffanyScreens still uses Java’s inherent screen capture functionality. A small drop-down on the bottom-right, available when broadcasting allows the presenter to select native screen-capturing, and also to select the source monitor. Switching to native, taxes the presenters machine a little less and also improves the experience for the watching audience, e.g. by showing smoother transitions.


JNIEXPORT jbyteArray JNICALL
Java_com_carlsbadcubes_tiffany_send_ScreenCapture_getScreenShot ( JNIEnv* env, jobject thiz, jint ndid );

JNIEXPORT jint JNICALL
Java_com_carlsbadcubes_tiffany_send_ScreenCapture_getNumberOfScreens( JNIEnv* env, jobject thiz);

…

.

Apple Store NYC, iDay -3

July 2007 in NYC – 3 days before the iPhone launch

On OS X 10.7 (Lion) however, Java is initially not installed anymore. You can find the installer on Apple’s support side here or follow this installation guide, provided by Adobe. Whatever you do, when opening Terminal and running java -version, you should see something like this.

java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03-383-11A511)
Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02-383, mixed mode)

Here are the easy to follow steps to get it up and running on your Mac

1. Download a binary distribution of the core module: apache-tomcat-7.0.20.tar.gz from here. I picked the tar.gz in Binary Distributions / Core section.
2. Opening/unarchiving the archive will create a folder structure something like this in you Downloads folder: (btw, this Unarchiver app is perfect for all kinds of compressed files and superior to the built-in Archive Utility.app)
   ~/Downloads/apache-tomcat-7.0.20
3. Move the unarchived distribution to /usr/local
   sudo mkdir /usr/local
sudo mv ~/Downloads/apache-tomcat-7.0.20 /usr/local
4. To make it easy to replace this release with future releases, we are going to create a symbolic link that we are going to use when referring to Tomcat:
   sudo ln -s /usr/local/apache-tomcat-7.0.20 /Library/Tomcat
5. Change ownership of the /Libaray/Tomcat folder hierarchy:
   sudo chown -R <your_username> /Library/Tomcat
6. Make all scripts executable:
   sudo chmod +x /Library/Tomcat/bin/*.sh

Instead of using the start and stop scripts, like so:
Last login: Sun Aug 14 15:20:38 on ttys000
wpbookpro:~ wolf$ /Library/Tomcat/bin/startup.sh
Using CATALINA_BASE: /Library/Tomcat
Using CATALINA_HOME: /Library/Tomcat
Using CATALINA_TMPDIR: /Library/Tomcat/temp
Using JRE_HOME: /Library/Java/Home
Using CLASSPATH: /Library/Tomcat/bin/bootstrap.jar:/Library/Tomcat/bin/tomcat-juli.jar
wpbookpro:~ wolf$ /Library/Tomcat/bin/shutdown.sh
Using CATALINA_BASE: /Library/Tomcat
Using CATALINA_HOME: /Library/Tomcat
Using CATALINA_TMPDIR: /Library/Tomcat/temp
Using JRE_HOME: /Library/Java/Home
Using CLASSPATH: /Library/Tomcat/bin/bootstrap.jar:/Library/Tomcat/bin/tomcat-juli.jar
wpbookpro:~ wolf$

you may also want to check out Activata’s Tomcat Controller a tiny freeware app, providing a UI to quickly start/stop Tomcat.

Well, about three weeks ago, I decided to make my little demo app available in the Android Market. It’s a well known simple finger painting application, the display is the canvas and your finger becomes a paint brush.

Simply use your finger to create beautiful paintings. Use the Menu to change color or brush styles and also to print or share your artwork via Email etc. Become an Artist on Android and print some of your best creations. Using HP Advanced Photo Paper has produced stunning results, worth framing.

https://market.android.com/details?id=com.techcasita.android.creative

I have recently added some code, allowing the artwork to be directly uploaded to a group on Flickr: Artist on Android – and WOW, check out the results; there really are Artists on Android. I have also integrated Google’s Analytics library to get a better idea about how the app is being used.

Google Analytics is really a great tool, every Android app should make use of. There is a lot to learn about usage patterns and while my little app is currently installed on only about 1,200 devices, I still know a lot about how it’s used.
For instance, I know that 70 times an finger-painted image has been printed and that on average it took 89 seconds to draw the picture before it was printed.

In total, 16,534 times the color of a brush was changed and 12,973 times the canvas was cleared and the artist was starting over.
The brush type (blur, emboss, etc. ) was changed 12,533 times, blur being the most popular 3,403 times.
Users can print via Google Cloud-print or HP iPrint, requiring an HP Printer as well as HP’s iPrint for Android app being installed. Anyway, if not on the device already, I guide the user through the install process of HP iPrint which can result in some stunning prints. In total, 53 users (4.6%) went that route and installed HP iPrint on their phones.

There are more numbers that need to be interpreted and analyzed. However, just this little adventure into usage tracking with Google Analytics can help you understand your user base so much better, going from “I think our users want ..” to “I know exactly what our users want ..”.

A very concrete result for this application could be to make brush color changes easier, e.g. pulling it up in the menu tree, or to make choosing the blur brush easier.

Considering how often Android applications are being updated, we might be at the verge of a new area of customer driven User Experience Design, not relying on small groups of test users anymore, but observing real usage behavior and adjusting the user experience almost interactively.

Galaxy 10.1 Tab running a finger paint app that can print

“There was nothing you could have done, Luke, had you been there. You’d have been killed too, and the droids would now be in the hands of the Empire.”

While the Android SDK comes with a complete set of javadocs, the source code of the SDK is missing in the SDK distribution. This is very unfortunate, since you cannot easily debug into SDK methods (at least not without running into de-compiled code) nor can you see how things actually work.

Eclipse - Source Not Found

However, there is a quick fix to that problem. I downloaded the complete Android source including the Linux, drivers, libs, etc., like explained here: http://source.android.com/source/download.html and ran small Java program on the source tree. I used to this with a simple bash script but over the last couple of Android Releases, the java source locations got a little more diverse and I started missing a couple files. So instead, this Java program walks the source tree and looks for java source files. All those will then be copied into a new location, considering their package name. Finally, the jar tool gets called to put all the source into a single bundle for easier handling.

package com.techcasita.tools;
import java.io.*;
/**
 * Find all of Android's Java Sources, consider their package names,
 * and place it in a jar file.
 *
 * @author <a href="mailto:wolf@wolfpaulus.com">Wolf Paulus</a>
 */
public class DroidSource {
    private static String TargetPath;

    public static void main(String[] args) {
        if (args.length == 2 &amp;&amp; new File(args[0]).isDirectory()) {
            DroidSource.TargetPath = args[1];
            File t = new File(args[1]);
            if ((t.exists() &amp;&amp; t.isDirectory() &amp;&amp; t.listFiles().length == 0) || t.mkdir()) {
                traverse(new File(args[0]));
                try {
                    Runtime.getRuntime().exec("jar cf " + t.getParentFile().getAbsolutePath()
+ File.separatorChar + "android-10-src.jar -C " + TargetPath + File.separatorChar + " .");
                } catch (IOException e) {
                    e.printStackTrace();
                }
            } else {
                System.out.println("Usage: " + DroidSource.class.getName()
+ "  ");
                System.out.println(" should exist and be empty");
            }
        } else {
            System.out.println("Usage: " + DroidSource.class.getName()
+ "  ");
        }
    }

    static void traverse(File file) {
        if (file != null &amp;&amp; !file.isHidden()) {
            if (file.isDirectory()) {
                if (!"out".equals(file.getName())) {
                    File[] files = file.listFiles();
                    for (File f : files) {
                        traverse(f);
                    }
                }
            } else if (file.getName().endsWith("java")) {
                String path = findPackage(file);

                if (path != null) {
                    path = path.replace('.', File.separatorChar);
                    createDirs(path);
                    copyFile(file.getAbsolutePath(), TargetPath + File.separatorChar
+ path + File.separatorChar + file.getName());
                }
            }
        }
    }

    private static String findPackage(File file) {
        final String TAG0 = "package ";
        final String TAG1 = ";";
        char[] buffer = new char[2048];

        try {
            FileReader r = new FileReader(file);
            r.read(buffer);
            StringBuffer sb = new StringBuffer();
            sb.append(buffer);
            int t0 = sb.indexOf(TAG0);
            if (t0 &lt; 0) return null;
            t0 += TAG0.length();
            int t1 = sb.indexOf(TAG1, t0);
            if (t1 &lt; 0) return null;
            if (t1 &gt; sb.indexOf(" ", t0)) return null;
            return sb.substring(t0, t1);
        } catch (IOException e) {
            return null;
        }
    }

    private static void createDirs(String path) {
        String[] sa = path.split("" + File.separatorChar);
        String name = TargetPath;
        for (String s : sa) {
            name += File.separatorChar + s;
            File f = new File(name);
            if (!f.exists()) {
                f.mkdir();
            }
        }
    }

    private static void copyFile(String srFile, String dtFile) {
        try {
            File f1 = new File(srFile);
            File f2 = new File(dtFile);
            InputStream in = new FileInputStream(f1);
            OutputStream out = new FileOutputStream(f2);

            byte[] buf = new byte[1024];
            int len;
            while ((len = in.read(buf)) &gt; 0) {
                out.write(buf, 0, len);
            }
            in.close();
            out.close();
        } catch (FileNotFoundException ex) {
            System.out.println(ex.getMessage() + " in the specified directory.");
            System.exit(0);
        } catch (IOException e) {
            System.out.println(e.getMessage());
        }
    }
}

This will generate this android-10-src.jar, (36 MB) containing all the relevant java sources and after attaching it to the SDK classes in your favorite IDE (IntelliJ in my case) you can easily step into the code, for instance during a debug session… The jar was generated from the 2-3 tag here http://android.git.kernel.org on July 1st, 2011.