Wolf Paulus

Journal

Navigation Menu

Building PhoneME for Mac OSX

Posted by on Jan 19, 2009 in Embedded

A few days ago, I wrote here about how the PhoneME JavaVM can be built with and integrated into OpenWrt, the Linux distribution optimized for WiFi routers or any device that connects over Wifi. However, even with a powerful Java runtime environment available on the target hardware platform, we still need to develop and simulate/test on a development workstation, which may very well be a Mac.
Developing for the PhoneME JavaVM becomes much easier and more fun, if you have the same VM available on your development platform; so here is what I did to get the PhoneME VM build for OS X 10.5 (Intel).

Again, we need to get the sources from the SVN server on java.net here: https://phoneme.dev.java.net/svn/phoneme. Don’t get everything, just the trunk (i.e. phoneme/components/cdc/trunk) and tools (i.e. phoneme/components/tools) folder. However, trunk and tools need to end up on the same level (like siblings) on the build machine.

Unfortunately, the currently available trunk only builds on older Macs that is Macs built on the Power-PC architecture. For newer Macs, i.e. Intel based machines, a patch needs to be installed into the just created trunk folder.
Markus Heberling has made this patch available here: http://markus.heberling.net/wp-uploads/2008/04/phoneme-cdc-darwin-x86.zip. The three folders in this patch just need to be added into the source tree in trunk:

  • patch cdc/src/darwin-x86 becomes trunk/src/darwin-x86
  • patch cdc/src/darwin-x86 becomes trunk/src/darwin-x86
  • patch cdc/buld/darwin-x86-mac becomes trunk/build/darwin-x86-mac

All ducks are in a row now to kick-off the PhoneME build. But there is still one final decision to be made, what kind of PhoneME runtime should be built?
The J2ME_CLASSLIB argument provided to Make, defines what class library will eventually be created:

  • cdc – (default) will create a limited class library (cdc.jar 884 KBytes) that is meant for testing purposes only.
  • foundation – creates the full Foundation Profile class library (foundation.jar 1.475 KBytes).
  • personal – creates the full Personal Profile class library (personal.jar 2.047 KBytes).

Just like you would expect, compared to the Personal ProfileFoundation is missing java.applet.* and java.awt.*. And compared to the Foundation Profile, the cdc.jar is missing some of the networking and security related classes.
Running Java applications on a Mac that depend on the Personal Profile, also require The X Window System (available athttp://xquartz.macosforge.org/trac/wiki) and QT3, the Cross-Platform GUI application framework, installed. QT3 is best installed using FINK and Fink Commander. I usually prefer Porticus but even after several attempts could not make Personal Profile depending Java Apps run. Only after installing QT3 using Fink, did they work flawlessly.

cd trunk/build/darwin-x86-mac
make J2ME_CLASSLIB=foundation

With the Java Runtime Environment built, we can now either use and ANT script like this, to build (and deploy) a Java app:


<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2009 Wolf Paulus - http://wolfpaulus.com
OpenWrt / PhoneME / Java Project build.xml
-->
<project name="fon" default="deploy" basedir=".">

<!-- =================================================================== -->
<!-- Initialization target -->
<!-- =================================================================== -->
<target name="init" description="initialize properties and folder variables">
<tstamp/>
<!-- remote target information -->
<property name="remote.addr" value="192.168.1.1"/>
<property name="remote.uid" value="root"/>
<property name="remote.pw" value="admin"/>
<property name="remote.dir" value="/usr/java"/>

<!-- Project Attributes -->
<property name="build.id" value="1001"/>
<property name="version" value="1.0 (#${build.id})"/>
<property name="year" value="2009"/>
<property name="packages" value="com.carlsbadcubes.*"/>

<!-- Folders -->
<property name="src.dir" location="src"/>
<property name="build.dir" location="build"/>
<property name="build.src" location="build/src"/>
<property name="build.dest" location="build/classes"/>
<property name="build.javadocs" location="build/apidocs"/>

<!-- Replace tokens in source code before compilation -->
<filter token="year" value="${year}"/>
<filter token="version" value="${version}"/>
<filter token="date" value="${TODAY}"/>
<filter token="log" value="true"/>
<filter token="verbose" value="true"/>

<!-- Complier Settings -->
<property name="compile.source" value="1.4"/>
<property name="compile.target" value="1.4"/>
<property name="compile.compiler" value="modern"/>
<property name="compile.debug" value="true"/>
<property name="compile.optimize" value="true"/>
<property name="compile.deprecation" value="true"/>

<property name="JSDK.home" location="/Users/wolf/Work/Embedded/phoneme-advanced/OSX/phoneme-x86"/>
<!-- Compilation class path -->
<path id="compile.classpath">
<pathelement location="${JSDK.home}/btclasses.zip"/>
<pathelement location="${JSDK.home}/lib/foundation.jar"/>
</path>
</target>

<!-- =================================================================== -->
<!-- Clean up -->
<!-- =================================================================== -->
<target name="clean" depends="init" description="Removes build the distribution files">
<delete dir="${build.dir}"/>
</target>

<!-- =================================================================== -->
<!-- Prepares the build directories and souce -->
<!-- =================================================================== -->
<target name="prepare" depends="clean,init" description="Creates build folder(s) and copies sources">
<mkdir dir="${build.dir}"/>

<!-- create directories -->
<mkdir dir="${build.src}"/>
<mkdir dir="${build.dest}"/>

<!-- copy src files -->
<copy todir="${build.src}" filtering="yes">
<fileset dir="${src.dir}" excludes="**/*.png"/>
</copy>

<!-- copy doc images (e.g. uml diagrams)-->
<copy todir="${build.src}" filtering="no">
<fileset dir="${src.dir}" includes="**/*.png"/>
</copy>
</target>

<!-- =================================================================== -->
<!-- Compiles the source directory -->
<!-- =================================================================== -->
<target name="compile" depends="prepare" description="Compiles the source code">
<javac
srcdir="${build.src}"
destdir="${build.dest}"
debug="${compile.debug}"
optimize="${compile.optimize}"
deprecation="${compile.deprecation}"
compiler="${compile.compiler}"
target="${compile.target}"
source="${compile.source}">
<classpath refid="compile.classpath"/>
</javac>
</target>

<!-- =================================================================== -->
<!-- Package Classes into JAR -->
<!-- =================================================================== -->
<target name="package" depends="compile" description="Package Classes into JAR">

<jar destfile="${ant.project.name}.jar">
<manifest>
<attribute name="Built-By" value="${user.name}"/>
<attribute name="Implementation-Vendor" value="Carlsbad Cubes."/>
<attribute name="Implementation-Title" value="${ant.project.name}.jar"/>
<attribute name="Implementation-Version" value="${version}"/>
</manifest>
<fileset dir="${build.dest}"/>
</jar>
</target>

<!-- =================================================================== -->
<!-- Deploy to target (opt ant task, http://www.jcraft.com/jsch/) -->
<!-- =================================================================== -->
<taskdef name="scp" classname="org.apache.tools.ant.taskdefs.optional.ssh.Scp"/>
<target name="deploy" depends="package" description="Deploy on to target hardware">
<scp file="${ant.project.name}.jar" todir="${remote.uid}:${remote.pw}@${remote.addr}:${remote.dir}"/>
</target>

</project>

.. or create a new JSDK in an IDE, like IntelliJ IDEA
you may have to rename the trunk/build/darwin-x86-mac/cvm into trunk/build/darwin-x86-mac/java

As of Snow Leopard 10.6.2, I had to add -m32 to these settings in cdc/build/darwin-x86-mac/GNUmakefile:
ASM_ARCH_FLAGS = -m32 -march=i686
CC_ARCH_FLAGS = -m32 -march=i686
LINK_ARCH_FLAGS = -m32

Read More

La Fonera Pt.3

Posted by on Dec 1, 2008 in Embedded

PhoneME, a JavaVM for the Fonera FON Router

OpenWRT is a Linux distribution optimized especially for embedded devices and also builds the starting point for many popular RouterOS distributions, like DD-WRTFreeWRT, or X-Wrt. Wikipedia has details on all those project, a good starting point would be here.
Since OpenWrt is optimized for embedded systems, it doesn’t come as a surprise that features like a read-only Flash memory filesystem (SquashFS) and a read/write Flash memory filesystem (JFFS2), or BusyBox, which combines many common UNIX utilities into a single executable, are readily available to be configured into a custom image. OpenWrt also comes with a lightweight package management system (IPKG or more recently OPKG), meaning that features that have not already been built into the kernel, can be added later, at runtime.
Java-Runtime Environment would be one of those features that are not built-in but would be nice to have for some of us. So let’s take a look at how a JavaVM could be build, packaged, and deployed into an embedded system that has the OpenWrt firmware.

Fink or MacPorts

The OpenWrt build process requires that several GNU tools are installed on machine that will eventually perform the build process and while some of these tools seem to be available on OS X, they aren’t parameter compatible with the GNU version, i.e. either Fink or MacPorts is needed to get the GNU tools working.

To find out what tools are needed, open a Terminal, cd into the OpenWrt trunk folder and run make help.
After installing binutils, patch, bzip2, flex, bison, gettext, … with the help of Fink Commander, we’re ready to configure the build process by running make menuconfig

The result may come as a surprise, but the kernel configuration is done with the help of a character based UI.
The most important thing to get right here is to pick the correct Target System. For the Fonera FON, this would be Atheros 231x/5312 [2.6]. Moreover, to be able to build a JavaVM later, building the OpenWrt SDK needs to be enable here.
After a subsequent default build (entering make without any arguments) has been successfully completed, the sdk can be found here:
Kamikaze/trunk/build_dir/mips/OpenWrt-SDK-atheros-for-Darwin-i386

(If the build process fails, running make V=99 is revealing what went wrong. In my case it was a missing library. After installing it – again with the help of Fink Commander – the error message went away.)

Java

A good starting point for Java in embedded world can be found here and one actually does have a choice, when is comes to Java Virtual Machines for OpenWrt Embedded Linux.

  • JamVM
    JamVM is a new Java Virtual Machine which conforms to the JVM specification version 2 (blue book). In comparison to most other VM’s, it is extremely small, but requires the building of the GNU Classpath. Clearly the full version of Classpath isn’t going to fit onto a small router, contains some native code and need to be cross-compiled.
  • SableVM and SableVM Mini
    SableVM is a Java Bytecode Interpreter implementing Java virtual machine (JVM) specification, second edition. Its goals are to be reasonably small, fast, and efficient, as well as providing a well-designed and robust platform.
  • PhoneME
    The PhoneME project was created after opensourcing Sun implementation of Java ME. It comes in two versions: PhoneME feature software (CLDC), for mobile phones and PhoneME advanced software (CDC), for higherend devices. Reference implementations are available for x86, ARM and MIPS.

JamVM and SableVM both require the building of the GNU Classpath. Clearly the full version of Classpath isn’t going to fit onto a small router, contains some native code, and need to be cross-compiled. Taking the many posts in the OpenWrt related forums seriously, porting GNU Classpath to MIPS seemed to be a very daunting project and even if I’d be successfull, the full version of this project may be to be too to be deployed on the Fonera and still leave some room for an Java application. Considering Bug Labs performance benchmarks for Java on embedded Linux, the PhoneME JavaVM needs to be the 1st choice for a resonably small and fast JavaVM for the embedded target platform, like the Fonera FON.

PhoneME

Next step is to checkout the phoneme sources form: https://phoneme.dev.java.net/svn/phoneme/components (you may have to create an user account at the java.net site before being able to access the source repository). We need cdc’s trunk and tools’ trunk on the same level and then edit thecdc/trunk/build/linux-mips-openwrt/GNUmakefile, setting the CVM_TARGET_TOOLS_PREFIX to point to the toolchain location in the OpenWrt-SDK, like so:

CVM_TARGET_TOOLS_PREFIX ?= /Users/wolf/Work/Embedded/Kamikaze/trunk/build_dir/mips/OpenWrt-SDK-atheros-for-Darwin-i386/staging_dir/toolchain-mips_gcc4.1.2/bin/mips-linux-uclibc-

Running make in the linux-mips-openwrt folder will produce theses files:

  • linux-mips-openwrt/btclasses.zip
  • linux-mips-openwrt/testclasses.zip
  • linux-mips-openwrt/democlasses.jar
  • linux-mips-openwrt/bin/cvm
  • linux-mips-openwrt/lib/cdc.jar (or linux-mips-openwrt/lib/foundation.jar)

The J2ME_CLASSLIB argument provided to Make, defines what class library will eventually be created:

  • cdc – (default) will create a limited class library (cdc.jar 884 KBytes) that is meant for testing purposes only.
  • foundation – creates the full Foundation Profile class library (foundation.jar 1.475 KBytes).

Just like you would expect,compared to the Foundation Profile, the cdc.jar is missing some of the networking and security related classes.

Creating the ipkg Package

These files now have to be distributed into a manually created folder that needs to look exactly like this: Start by creating a cdc folder in the OpenWrt-SDK package location like here:

/Users/wolf/Work/Embedded/Kamikaze/trunk/build_dir/mips/OpenWrt-SDK-atheros-for-Darwin-i386/package/cdc

Insert this makefile into the cdc folder and also create a files folder.

include $(TOPDIR)/rules.mk
PKG_NAME:=pmea-wp
PKG_VERSION:=1_1
PKG_RELEASE:=2
PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_NAME)
include $(INCLUDE_DIR)/package.mk
define Package/pmea-wp
SUBMENU:=Java
SECTION:=lang
CATEGORY:=Languages
DEPENDS:=+libpthread
TITLE:=CDC Java runtime environment
endef
define Package/pmea-wp/description
phoneME is an open source project based on Java Micro Edition (Java ME) \
technology. phoneME Advanced is a phoneME project based on Java ME CDC \
technology targeting resource-constrained devices like smartphones, \
set-top boxes and office equipment. See https://phoneme.dev.java.net \
and http://java.sun.com/products/cdc
endef
define Build/Compile
endef
define Package/pmea-wp/install
$(INSTALL_DIR) $(1)/usr/java
$(CP) files/usr/java $(1)/usr
endef
$(eval $(call BuildPackage,pmea-wp))

Eventually, the content layout needs to look like this:

Running make in the SDK folder here:
/Users/wolf/Work/Embedded/Kamikaze/trunk/build_dir/mips/OpenWrt-SDK-atheros-for-Darwin-i386
creates the deployable package, containing the JavaVM and a set of test classes, here:
/Users/wolf/Work/Embedded/Kamikaze/trunk/build_dir/mips/OpenWrt-SDK-atheros-for-Darwin-i386/bin/packages/mips/pmea-wp_1_1-2_mips.ipk

Like declared in the makefile that was responsible for creating the ipk package, the only non-standard OpenWrt package required by PhoneME Advancedis libpthread. Running OpenWrt’s make menuconfig again, now shows Java in the Languages menu and if Java gets selected, libpthread (to be found in Base System) is automatically added.


Deployment and Test

With the package built, all what’s left to do is deploying it and running the test suite. Deploying the ipkg file is as easy is putting it into the target’s tmpfolder (running sftp on the FON and Fugu on the development host, makes this easy and fast).
After ssh- into the FON, running ipkg install /tmp/pmea-wp_1_1-2_mips.ipk or opkg install /tmp/pmea-wp_1_1-2_mips.ipk will install the JavaVM as well as the test classes.
Now executing the test is as simple as /usr/java/bin/cvm -cp /usr/java/testclasses.zip Test

Read More