How To Effectively Compare Mobile App Platforms

Some readers might already be aware of my master’s project, “Creating Multiplatform Native App Support for Locating H1B Jobs”. The project investigates the differences between the iOS and Android platforms from a developer’s perspective. My perspective was informed by the app I developed and deployed on both iTunes App Store and Google Play. Developing for both platforms gave me insight into questions a programmer needs to typically ponder over before making when assessing the pros and cons of a given platform. Some of the questions also originate from selected conversations with my classmates as well as discussions from my job interviews with top companies like Amazon and Compuware.

Learning Curve and Marketing Dominance

These days it seems you can’t turn around without running into someone with either an iPhone or an Android mobile device. It is probably not surprising that Android and iOS accounted for 90% of all smartphone sales. Although Apple seems to be selling iPhones at a fast pace, Android continues to dominate the market. Android is leading the market sector with 47% share sales while iOS is breathing down the neck of Android with 43%. [1] Therefore we can confidently say Google’s Android is in high demand today, closely followed by Apple’s iPhone.

Aside from that, the learning curve for Android development is relatively less steep than iOS. For me, my prior background in Java programming made the transition to Android development much smoother. I was able to transfer my expertise in pure Java coding to working in the Droid platform. However iOS was rather challenging. Without any prior knowledge in C programming, self-instructed lessons in Objective-C require a high level of dedication. The iOS lectures videos available on Stanford iTunes can be very condensed. As a result, I occasionally consult an alternate, basic training material “Beginning iOS5 Development – Exploring the iOS SDK”[2] authored by Mark et al.

Similarities and Differences in iOS & Android UX Design

The user experience designs of both Android and iOS are alike. The touch gesture controls are very comparable. For instance certain gestures such as tap, pinch, swipe, slide, flick, hold and drag, and double tap offer identical actions on both iOS and Android devices. Although the design treatment and UI placement might vary, basic components such as buttons, switches, checkboxes, pickers, tabs and sliders also have very similar functionalities in both platforms. Moreover, for both iOS and Android platforms, particularly in task-based applications, it is common to see hierarchical tree-based menu format where user navigates the app through a series of lists to dive deeper into specific information within the app.

However, there are some differences in the user experience designs of both platforms that strongly dictate development in either environment. The iOS smartphones comes in 2 screen sizes and resolutions. Android, on the other hand, has 4 generalized sizes and resolutions (densities). Furthermore, the tab navigation placement in both platforms is complete opposites. The tab navigation of iOS is represented as a tab bar at the bottom of the app. Android suggests tab should be placed at the top of the app.

Pros and Cons of Each Development Environment

Firstly, let us review the advantages of both development environments. Xcode for iOS platforms has storyboarding, which makes development very easy. The storyboards are nice for prototyping. The use of storyboard segues helps to identify navigation/modal relationships.  Xcode also has a user-friendly Model-View-Controller (MVC) oriented organization. The emulator for both Android and iOS platforms allow you to conveniently test apps on a device from the comfort of your computer.

Unfortunately it could be debated that as storyboards expand, it gets difficult to keep track of the navigation relationships within the app. It almost seems like a “spaghetti” code and could get a little frustrating. The Eclipse emulator could be extremely slow to start up. At some point, it maxed out my computer RAM. When an emulator starts, it takes quite a while for the apps to also load. A quick fix around this is to have an Android device if you intend to develop efficiently. Sadly, with Xcode, developers need to register their device before they can test on an iPhone.

The Application Deployment Model Between Apple and Google

Google Play developer’s account costs $25, which is a one-time fee. However the iTunes App Store charges an annual fee of $99. Also, publishing an app on Google Play is very easy. Once the developer completes development, he simply uploads the apk file(s) and instantly publishes the app on Google Play. iTunes App Store on the other hand has a very tedious review process. A developer typically submits the binary files for review and approval by the App Review Team. This review process usually takes approximately 8 business days prior to an official launch of an app on the iTunes App Store. Finally, Android developers have a variety of deployment avenues such as Amazon App Store for Android and AppsZoom, formerly AndroidZoom.

Conclusion

Personally, the choice between the 2 platforms, iOS and Android, would be a challenge. To some degree they both have unique strengths and weaknesses. It is safe to say that no one platform is a universal solution. I am a huge fan of the Xcode storyboards. For me, the simple drag, drop is fantastic. However, as a student, the $99 annual fee is an expensive cost especially when my app isn’t monetized. On the other hand, Android development makes things easy for developers just like me to deploy apps without a rigorous review process. My proficiency in Java has made the Android platform very appealing to me. But the only thing that bugs me is the complexity of manually specifying relative layouts when I drag and drop UI components on an Android canvas or layout. If I had to pick one at the end of the day, I would base my choice on the best tool that can help me accomplish the job or task I have been assigned.

Vincent Sam (aka Kobby) recently graduated with his MS from Grand Valley State University. This article was based on his experiences writing the H1B Jobs app on both the iOS and Android platforms. You can download both apps from the iTunes App Store or Google Play. Vincent currently works as a mobile app developer at Compuware in Detroit, MI.

References

[1] Kim, R. (2012, January 9). IPhone Breathing Down Neck of Android in U.S. Retrieved December 14,
2012, from GigaOm website: http://gigaom.com/2012/01/09/iphone-breathing-down-the-neck-of-
android-in-u-s/
[2] Mark, D., Nutting, J., & LaMarche, J. (2011). Beginning iOS 5 Development: Exploring the iOS SDK.
New York, NY: Springer Science and Business Media New York.

We will be having a MASL Meetup on Friday afternoon, November 30, at 3pm in B1-118 Mackinac Hall on the Allendale Campus. Anybody in the GVSU community interested in mobile technology and/or software development in general are welcome to attend.

Speaker: Sam Serpoosh – Sam is a passionate software developer and graduate student in GVSU’s School of Computing.  Sam works as a graduate assistant in the Mobile Apps & Services lab. Sam gets into writing awesome software and is interested in all things agile. You can follow Sam’s blog on software craftmanship here.

Abstract: With always-changing requirements during the software development life cycle you need to be fast and responsive. However, the only way to go fast is to go WELL! If your code is a mess and you don’t have an efficient way to test your changes you’ll be afraid to touch it, and hence you won’t be going anywhere! As Steve Freeman and Nat Pryce put it: “Fear kills Progress!”. However, we can make the fear disappear with efficient programming and software development techniques! One of the most important practices of effective software development is Test Driven Development. In this talk we will discuss these issues and try to inspire attendees to think more agile by adopting test driven development practices.

J. Engelsma, F. Jumah, A. Montoya, J. Roth, V. Vasudevan, and G. Zavitz. Shop Social: The Adventures of a Barcode Scanning App in the Wild. Fourth International Conference on Mobile Computing, Applications and Services. October 11-12, 2012. Seattle, WA.

Abstract: Mobile retail is a space rich with plausible hypotheses but sparse on longitudinal datasets that give us a corpus of user behavior to validate or disprove theories around the use of digital devices in a physical store. Popular price comparison apps such as ShopSavvy have shown that a smartphone in the aisle is a reality that brick-and-mortar retailers have to contend with. A nuanced, data-driven understanding of a smartphone powered shopper might enable store-based retailers to leverage the smartphone rather than fear it as something that leads to sales erosion. To this end, we built and deployed a novel, mobile retail app that blends mobile, media and social capabilities. In this paper, we describe the user needs and design axioms behind the app, and the data that we’ve collected over the course of its use by about 5,500 users over the period of a year.

A. Restrepo, J. Engelsma, T. Parker and J. Farris. Accessorized Therapeutic Games Experiences for Tablets. Meaningful Play 2012 Conference. October 18-20, 2012. East Lansing, MI.

Abstract: In the world of physical therapy, a number of consumer gaming devices have been used with various levels of success. Most commercially available video games are designed for the general population and are, in most cases, overwhelming and difficult for traumatic brain injury (TBI) or stroke patients to use. Specialized therapeutic medical devices are not only expensive and non-portable, they also make limited use of gamification techniques to better engage and motivate the patient. This paper examines the use of inexpensive, portable handheld devices, together with a custom sensor accessory in order to drive a set of therapist designed and configured, short video games. Games have been designed that are intended to elicit specific therapeutic movements from the patient, are customizable by the therapist for a given patient’s needs, and also produce clinical output for the therapists to use. The games have been evaluated in clinics by physical therapists who treat TBI patients, and the results indicate our approach addresses the shortcomings therapists have experienced with prior attempts at gamification in physical therapy. Moreover, game controllability by the therapist has been identified as a key component in successfully gamifying treatment of TBI patients as it allows the therapist to customize the game experience to suit a patient’s individual needs.

Where: B-1-118 Mackinac Hall (Allendale Campus)
When: 3pm – 3:50pm, Friday September 21.

Note: No Microsoft confidential proprietary information will be presented!

July 31: Mobile / Wireless Technology Symposium: Leaders in government business, government, education, manufacturing, healthcare and other critical verticals will be exposed to local, national and international experts in various mobile / wireless sectors. Attendees will participate in a strategic planning session lead by industry leaders. Our objective is to develop a strategy that can be adopted and tailored for different organizations desiring to take advantage of mobile technologies. This collaboration is the first step for “Mobile Moves Michigan” to develop strategies for Michigan to take a national leadership role in the industry.

July 31: Michigan Mobile / Wireless Technology Exhibition: Open to the general public, as well as invite-only symposium attendees, the Michigan Mobile / Wireless Technology Exhibition will enable attendees to view exhibits from Michigan-based firms currently providing and/or utilizing a variety of mobile / wireless technologies.

July 30 and 31: WIPJam + Muther! of all Hack-a-thons: Produced by internationally renowned Wireless Industry Partnership (WIP), the WIPJam + Muther! of all Hack-a-thons is a mobile developer conference and hack-a-thon event which will provide education from, and direct access to, national and international companies in the mobile / wireless industry. Developers will receive training, access to resources, and one-on-one engagement with industry experts. Participants will have the opportunity to win prizes based on apps developed during the hack-a-thon portion of the program.

For more Information: http://www.mobilemovesmichigan.com

Symposium Information: info@GoMobileMichigan.org

Exhibition Information: http://www.gomobilemichigan.org/events/mobile-moves-michigan—request-to-exhibit.html

Hack-a-thon Information: http://wipconnector.com/wipjam/entry/wipjam_muther_michigan

Venue / Hotel Information:
Motor City Casino – Hotel
2901 Grand River Ave
Reservations: 866.782.9622 Request “Mobile Moves Michigan” Rate of $139

When: 1pm – 1:50pm Thursday, February 23, 2012

Where: B1-116 Mackinac Hall

Note: Compuware will also be giving a public CIS seminar on a related topic early in the day. The MASL talk will be a deeper technical dive than the earlier presentation.

Google AppEngine Channels create a persistent connection between your client and the server running the Channel API Code. Information is passed between the two in real time without the use of polling, even between multiple clients, using the server as a mediator. For instance what if you wanted to create a Java client to use these channels to receive push notifications from a server, or to push data to android devices without the use of sockets and worrying about port numbers. Playing a simple tic-tac-toe game over the internet could get complicated if there wasn’t a way for the other player to easily notify his opponent that a move had taken place. Well… you would have been out of luck. Up until now there wasn’t a way to use Java to do this using Google’s Channel API, it was just available in JavaScript. Luckily there is a solution to this, below is a simple framework that does just that. It allows you to utilize Google’s Channel API in Java.

Framework Overview

Channel Creation

The Client instantiates an instance of the ChannelAPI. The framework then requests a channel to be created with a given key that the user has chosen. The Server creates a channel with the provided key and returns a token which the client then uses to listen to the channel.

Message Sending

The Client sends a message using its channel token. The Server receives the message and propagates it to all the other client channels with the same channel key.

Framework Details

• Channel Creation
• Client Side Message Sending
• Server Side Message Sending

Channel Creation

ChatListener chatListener = new ChatListener();
ChannelAPI channel = new ChannelAPI("http://localhost:8888", "key", chatListener);
channel.open();

There are three main parts to creating a channel:

• Base URL
• ChannelService class Implementation
• Channel Key

The Base URL is the location of where your ChannelServer is stood up. In this example dealing with our ChatServer we are going to use  localhost, so our base URL ends up being “http://localhost:8888″

The ChannelService class Implementation is just simply that. We need to create a class that implements the ChannelService interface class.

package edu.gvsu.cis.masl.chat;
import edu.gvsu.cis.masl.channelAPI.ChannelService;

public class ChatListener implements ChannelService{

In this example the “ChatListener” is our implementation of the ChannelService class. Methods from this class get called by the channel you are about to create. Of all the methods being called the one that is most relevant to our example would be the ‘onMessage’:

/**
* Method gets called when the server sends a message.
 */
@Override
public void onMessage(String message) {
	System.out.println("Server push: " + message);
}

this method gets called when the server or other clients push messages to a channel with the same key that you are using.

The Channel Key is just that, A key. You can use any string you want for it, but keep in mind that channels with the same key receive the same messages from the server. We’ll go into more detail on how the server sends messages to clients, but the highlighted line below shows how messages are sent using a Channel Key on the server side.

public class ChatServlet extends HttpServlet {
  @Override
  public void doPost(HttpServletRequest req, HttpServletResponse resp) throws IOException {
    String channelKey = req.getParameter("channelKey");
    String message = req.getParameter("message");

    //Send a message based on the 'channelKey' any channel with this key will receive the message
    ChannelService channelService = ChannelServiceFactory.getChannelService();
	channelService.sendMessage(new ChannelMessage(channelKey, message));
  }
}

Every client gets a different channel, but clients can have similar keys.

Once the client has instantiated a ChannelAPI object the class does the rest of the work setting up the channel for you. The constructor calls the method “createChannel(channelKey)” and passes it the Channel Key you defined.

public ChannelAPI(String URL, String channelKey, ChannelService channelService) throws IOException, ClientProtocolException {
    	this.clientId = null;
    	this.BASE_URL = URL;
    	this.channelId = createChannel(channelKey);
    	this.applicationKey = channelKey;

    	if (channelListener != null) {
            this.channelListener = channelService;
        }
    }

The purpose createChannel() is to prompt the server to create a Channel for us and then send back a channel token. The channel token is what we use to communicate with our channel as well as the server and other clients.

private String createChannel(String key) throws IOException, ClientProtocolException{
    	String token = "";
		HttpClient staticClient = new DefaultHttpClient();
		HttpGet httpGet = new HttpGet(BASE_URL + "/?c=" + key);
		try{
			XHR xhr = new XHR(staticClient.execute(httpGet));
			System.out.println(xhr.getResponseText());
			JSONObject json = new JSONObject(xhr.getResponseText());
			token = json.getString("token");
		} catch (JSONException e) {
			System.out.println("Error: Parsing JSON");
		}
    	return token;
    }

Client Side Message Sending

There are two main parts to sending a message from the client to the server:

• The Message
• The Location

/***
* Sends your message on the open channel
* @param message
*/
public void sendMessage(String message){
try {
		channel.send(message, "/chat");
	} catch (IOException e) {
		System.out.println("Problem Sending the Message");
	}
}

If you wanted to change where the server is looking for the messages you can change the “web.xml”:

<?xml version="1.0" encoding="utf-8"?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://java.sun.com/xml/ns/javaee"
xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" version="2.5">
	<servlet>
		<servlet-name>ChatChannelServlet</servlet-name>
		<servlet-class>edu.gvsu.cis.masl.channel.ChatChannelServlet</servlet-class>
	</servlet>
	<servlet-mapping>
		<servlet-name>ChatChannelServlet</servlet-name>
		<url-pattern>/*</url-pattern>
	</servlet-mapping>
  <servlet>
    <servlet-name>ChatServlet</servlet-name>
    <servlet-class>edu.gvsu.cis.masl.channel.ChatServlet</servlet-class>
  </servlet>
  <servlet-mapping>
    <servlet-name>ChatServlet</servlet-name>
    <url-pattern>/chat</url-pattern>
  </servlet-mapping>
</web-app>

The ‘<url-pattern>’ highlighted above is the line you would change, because the ChatServlet class is the one handling all of the messages from the clients. If you had different classes handling different types of client messages then you would change the appropriate lines in your “web.xml” to match your needs.

Server Side Message Sending

There are three main parts to sending a message on the server-side:

• ChannelService
• The ChannelKey
• The Message

//Send a message based on the 'channelKey' any channel with this key will receive the message
ChannelService channelService = ChannelServiceFactory.getChannelService();
channelService.sendMessage(new ChannelMessage(channelKey, message));

An instance of the ChannelService needs to be instantiated so we are able to pass a message through it. We instantiate it by grabbing an instance of the class from the ChannelServiceFactory. As seen in the highlighted line above. We then call the channelService.sendMessage() function and pass it an instance of a ChannelMessage. Seen below.

//Send a message based on the 'channelKey' any channel with this key will receive the message
ChannelService channelService = ChannelServiceFactory.getChannelService();
channelService.sendMessage(new ChannelMessage(channelKey, message));

The ChannelMessage needs two things to send a message: One being the ‘Channel Key‘ and the other being ‘The Message‘. You can either have the client push its channel key to you, or you can keep track of them in a hash table on your server. In the example code below, the client sends its ‘Channel Key’ to the server, and then the server sends a message back to the client using the Channel Key. The ‘Message’ can be anything from xml to json, as long as it can be converted to a string.
***Updated: Production Issue Has Been Resolved. Thanks in large part to Dean Harding and Lohre.***

The Source Code & Framework can be found on GitHub: https://github.com/gvsumasl/jacc happy coding!

Tom Parker is currently working on his Masters Degree in Computer Science at Grand Valley State University. He completed his undergraduate studies in Computer Science at Michigan State University. He works as a Grad Assistant developing mobile phone applications in the GVSU Mobile Applications and Services Lab. When he’s not in the lab or busy doing homework, you can usually find him at the gym, but he also likes to dirt bike, snowboard, workout and run, as well as work on various programming projects of his own.

Echoprint (http://echoprint.me) is an open source C++ fingerprinting library that is available for desktop computers and iOS devices, however, as far as I know, there isn’t an Android-compatible version, until now

This article is divided into 3 steps, the first one is how to compile the Echoprint C++ code as an Android native library, the second one on how build a reusable Android fingerprinting library project, and finally, step 3 is how to put everything together in your own projects, so without further ado, let’s get started.

Creating the Android native library.
The codegen library does all the heavy lifting, it actually processes the audio samples and generates a set of hashes that are the actual audio fingerprint.

For this step you will need:

- Android’s NDK (r6 or better)

- Echoprint codegen source, available at https://github.com/echonest/echoprint-codegen
Once you download the project, go to the src folder and rename all .cxx files to .cpp, this is necessary as the NDK doesn’t accept files with .cxx extension. Move the downloaded folder (echonest-echoprint-codegen-v4.11-45-g7bc4764 or something similar) to a new folder (I named it jni) and rename it “codegen” or whatever makes sense to you

- Boost C++ libraries src (http://www.boost.org/) and add the boost_1_XX_X folder to the jni folder (I used version 1.47.0)

Now fire up Eclipse and create a new Android project, I used “EchoprintLib” as the name of the project, but you can use anything you like.
In order to use a native library, some jni code needs to be written, as well as glue java code to invoke the jni code. Create a new java class, I used Codegen.java and declare the following native method:

native String codegen(float data[], int numSamples);

Your class should look like this:

public class Codegen
{
	native String codegen(float data[], int numSamples);
}

Save your changes as we will need the compiled .class file.
Now, navigate to the bin folder of your project and run the javah tool, this tool will generate the required .h file needed to implement the native jni code.

alex$ javah edu.gvsu.masl.echoprint.Codegen

Once it finishes, you will see a edu_gvsu_masl_echoprint_Codegen.h file that was generated, move this file to your jni folder.

Now it’s time to write the actual jni code that will invoke the codegen library and will return the results to our java library.

Create a new C++ file AndroidCodegen.cpp and add the following code:

#include <android/log.h>
#include <string.h>
#include <jni.h>
#include "edu_gvsu_masl_echoprint_Codegen.h"
#include "codegen/src/Codegen.h"

JNIEXPORT jstring JNICALL Java_edu_gvsu_masl_echoprint_Codegen_codegen
  (JNIEnv *env, jobject thiz, jfloatArray pcmData, jint numSamples)
{
    // get the contents of the java array as native floats
	float *data = (float *)env->GetFloatArrayElements(pcmData, 0);

    // invoke the codegen
	Codegen c = Codegen(data, (unsigned int)numSamples, 0);
	const char *code = c.getCodeString().c_str();

    // release the native array as we're done with them
	env->ReleaseFloatArrayElements(pcmData, data, 0); 

    // return the fingerprint string
	return env->NewStringUTF(code);
}

The actual signature of the codegen function is machine generated by the javah tool, in the edu_gvsu_masl_echoprint_Codegen.h file.

We now have all the necessary code to compile the codegen code as an Android native library, however the NDK requires two more files in order to build the code: the Android.mk and Application.mk files.

Create a new text file and name it Android.mk, this file is the equivalent of a makefile for the NDK, it contains the list of files needed as well as other parameters used by the NDK. Add the following to the Android.mk file:

LOCAL_PATH := $(call my-dir)

include $(CLEAR_VARS)

LOCAL_MODULE    :=echoprint-jni

LOCAL_SRC_FILES :=AndroidCodegen.cpp \
			/codegen/src/Codegen.cpp \
			/codegen/src/Whitening.cpp \
			/codegen/src/SubbandAnalysis.cpp \
			/codegen/src/MatrixUtility.cpp \
			/codegen/src/Fingerprint.cpp \
			/codegen/src/Base64.cpp \
			/codegen/src/AudioStreamInput.cpp \
			/codegen/src/AudioBufferInput.cpp

LOCAL_LDLIBS    :=-llog\
		-lz
LOCAL_C_INCLUDES :=<path to your jni folder>/codegen/src \
			<path to your jni folder>/boost_1_47_0				

include $(BUILD_SHARED_LIBRARY)

In the LOCAL_C_INCLUDES line, make sure you replace the <path to your jni folder> placeholder with your actual path to the jni folder.

Create a new text file, Application.mk and add the following line:

APP_STL := gnustl_static

Basically, all this file does is enable STL support in the NDK compiler.

Your jni folder should now look similar to this:

If everything looks good, fire up a terminal window and run the ndk-build tool, from within your jni folder:

$ <path to your ndk>/ndk-build

if everything goes well you should see something similar to this:

Compile++ thumb  : echoprint-jni <= AndroidCodegen.cpp
Compile++ thumb  : echoprint-jni <= Codegen.cpp
Compile++ thumb  : echoprint-jni <= Whitening.cpp
Compile++ thumb  : echoprint-jni <= SubbandAnalysis.cpp
Compile++ thumb  : echoprint-jni <= MatrixUtility.cpp
Compile++ thumb  : echoprint-jni <= Fingerprint.cpp
Compile++ thumb  : echoprint-jni <= Base64.cpp
Compile++ thumb  : echoprint-jni <= AudioStreamInput.cpp
Compile++ thumb  : echoprint-jni <= AudioBufferInput.cpp
Prebuilt       : libstdc++.a <= <NDK>/sources/cxx-stl/gnu-libstdc++/libs/armeabi/
SharedLibrary  : libechoprint-jni.so
Install        : libechoprint-jni.so => libs/armeabi/libechoprint-jni.so

The freshly built native library now resides in a “libs” folder next to your jni folder:

Go ahead and copy the libs folder into your EchoprintLib project folder.

We can now write the rest of the glue code from the java side, open your EchoprintLib project and select the Codegen.java file.

In order to use the native library we need to load it, we just need to do this one time, when the class is loaded. To do so we add a static block that loads the library:

static
{
	System.loadLibrary("echoprint-jni");
}

As it turns out, the codegen code expects the audio samples to be floats in the [-1, 1] range, however, Android records audio samples as 16 bit shorts. The solution is simply to normalize the audio before is sent to the native library, to do so we define a normalizing value:

private final float normalizingValue = Short.MAX_VALUE;

Finally, the actual generate method will normalize the audio samples and then invoke the  codegen native method:

public String generate(short data[], int numSamples)
{
	float normalizeAudioData[] = new float[numSamples];
	for (int i = 0; i < numSamples - 1; i++)
		normalizeAudioData[i] = data[i] / normalizingValue;

	return this.codegen(normalizeAudioData, numSamples);
}

For completeness, I’m adding an overloaded method that expects already normalized values as floats:

public String generate(float data[], int numSamples)
{
	return codegen(data, numSamples);
}

This is the complete Codegen class:

package edu.gvsu.masl.echoprint;

public class Codegen
{
	private final float normalizingValue = Short.MAX_VALUE;

	native String codegen(float data[], int numSamples);

	static
	{
        	System.loadLibrary("echoprint-jni");
    	}

	/**
	 * Invoke the echoprint native library and generate the fingerprint code.<br>
	 * Echoprint REQUIRES PCM encoded audio with the following parameters:<br>
	 * Frequency: 11025 khz<br>
	 * Data: MONO - PCM enconded float array
	 *
	 * @param data PCM encoded data as floats [-1, 1]
	 * @param numSamples number of PCM samples at 11025 KHz
	 * @return The generated fingerprint as a compressed - base64 string.
	 */
	public String generate(float data[], int numSamples)
	{
		return codegen(data, numSamples);
	}

	/**
	 * Invoke the echoprint native library and generate the fingerprint code.<br>
	 * Since echoprint requires the audio data to be an array of floats in the<br>
	 * range [-1, 1] this method will normalize the data array transforming the<br>
	 * 16 bit signed shorts into floats.
	 *
	 * @param data PCM encoded data as shorts
	 * @param numSamples number of PCM samples at 11025 KHz
	 * @return The generated fingerprint as a compressed - base64 string.
	 */
	public String generate(short data[], int numSamples)
	{
		float normalizeAudioData[] = new float[numSamples];
		for (int i = 0; i < numSamples - 1; i++)
			normalizeAudioData[i] = data[i] / normalizingValue;

		return this.codegen(normalizeAudioData, numSamples);
	}
}

All the code needed to invoke the codegen native library is now in place, let’s now start step 2 and turn the project into a library and add a class that records audio from the microphone and returns its fingerprint.

Turning the project into a reusable library
This is actually pretty simple, all we need to do is go to the properties of the project, and under the Android category, check the appropriate “Is Library” checkbox:

Your project can now be used as a library for other projects to use, let’s now add some code to take care of the audio recording and returning the results back to the client code.

First, let’s define an interface to communicate the results (or problems) back to whatever code is using the fingerprinter. My background is in iOS so you’ll find that the name of the methods closely resemble iOS delegate method names, but you can rename them to whatever you like:

/**
 * Interface for the fingerprinter listener<br>
 * Contains the different delegate methods for the fingerprinting process
 * @author Alex Restrepo
 *
 */
public interface AudioFingerprinterListener
{
	/**
	 * Called when the fingerprinter process loop has finished
	 */
	public void didFinishListening();

	/**
	 * Called when a single fingerprinter pass has finished
	 */
	public void didFinishListeningPass();

	/**
	 * Called when the fingerprinter is about to start
	 */
	public void willStartListening();

	/**
	 * Called when a single listening pass is about to start
	 */
	public void willStartListeningPass();

	/**
	 * Called when the codegen libary generates a fingerprint code
	 * @param code the generated fingerprint as a zcompressed, base64 string
	 */
	public void didGenerateFingerprintCode(String code);

	/**
	 * Called if the server finds a match for the submitted fingerprint code
	 * @param table a hashtable with the metadata returned from the server
	 * @param code the submited fingerprint code
	 */
	public void didFindMatchForCode(Hashtable<String, String> table, String code);

	/**
	 * Called if the server DOES NOT find a match for the submitted fingerprint code
	 * @param code the submited fingerprint code
	 */
	public void didNotFindMatchForCode(String code);

	/**
	 * Called if there is an error / exception in the fingerprinting process
	 * @param e an exception with the error
	 */
	public void didFailWithException(Exception e);
}

Now, let’s create an AudioFingerprinter class, and define a few keys to extract the metadata returned from the echoprint server:

public final static String META_SCORE_KEY = "meta_score";
public final static String SCORE_KEY = "score";
public final static String ALBUM_KEY = "release";
public final static String TITLE_KEY = "track";
public final static String TRACK_ID_KEY = "track_id";
public final static String ARTIST_KEY = "artist";

NOTE: the metadata dictionary IS NOT included by default in the echoprint server API, to add it, just replace line 66/67 in API.py with:

return json.dumps({"ok":True,"message":response.message(), "match":response.match(), "score":response.score, \
"qtime":response.qtime, "track_id":response.TRID, "total_time":response.total_time, "metadata":response.metadata})

Then let’s declare some constants for our audio settings, as the codegen requires the audio to be uncompressed PCM, mono @ 11Khz:

private final int FREQUENCY = 11025;
private final int CHANNEL = AudioFormat.CHANNEL_IN_MONO;
private final int ENCODING = AudioFormat.ENCODING_PCM_16BIT;

With these constants in place, we can now initialize the audio recorder and start listening for audio:

// create the audio buffer
// get the minimum buffer size
int minBufferSize = AudioRecord.getMinBufferSize(FREQUENCY, CHANNEL, ENCODING);

// and the actual buffer size for the audio to record
// frequency * seconds to record.
bufferSize = Math.max(minBufferSize, this.FREQUENCY * this.secondsToRecord);

audioData = new short[bufferSize];

// start recorder
mRecordInstance = new AudioRecord(MediaRecorder.AudioSource.MIC,
					FREQUENCY, CHANNEL,
					ENCODING, minBufferSize);

mRecordInstance.startRecording();

At this point the audio recording has started, in this loop we extract the audio samples,  invoke the codegen class to extract their fingerprint and submit the fingerprint to your own echoprint server in order to get the metadata of the fingerprinted audio. The results are then forwarded to the client code, via an interface, as a hashtable.

I have defined a continuous variable to allow the class to listen for audio continuously, if it’s set to true, the loop will be repeated until the stop method is called in the AudioFingerprinter instance:

do
{
	try
	{
		// fill audio buffer with mic data.
		int samplesIn = 0;
		do
		{
			samplesIn += mRecordInstance.read(audioData, samplesIn, bufferSize - samplesIn);

			if(mRecordInstance.getRecordingState() == AudioRecord.RECORDSTATE_STOPPED)
				break;
		}
		while (samplesIn < bufferSize);				

		// create an echoprint codegen wrapper and get the code
		Codegen codegen = new Codegen();
		String code = codegen.generate(audioData, samplesIn);

		if(code.length() == 0)
		{
			// no code?
			// not enough audio data?
			continue;
		}

		// fetch data from echoprint server
		String urlstr = SERVER_URL + code;
		HttpClient client = new DefaultHttpClient();
		HttpGet get = new HttpGet(urlstr);

		// get response
		HttpResponse response = client.execute(get);
	    	// Examine the response status
		Log.d("Fingerprinter",response.getStatusLine().toString());

		// Get hold of the response entity
		HttpEntity entity = response.getEntity();
		// If the response does not enclose an entity, there is no need
		// to worry about connection release

		String result = "";
		if (entity != null)
		{
			// A Simple JSON Response Read
			InputStream instream = entity.getContent();
			result= convertStreamToString(instream);
			// now you have the string representation of the HTML request
			instream.close();
		}	     			

	    	// parse JSON
		JSONObject jobj = new JSONObject(result);
		if(jobj.has("match"))
		{
			if(jobj.getBoolean("match"))
			{
				Hashtable<String, String> match = new Hashtable<String, String>();
				match.put(SCORE_KEY, jobj.getDouble(SCORE_KEY) + "");
				match.put(TRACK_ID_KEY, jobj.getString(TRACK_ID_KEY));

				if(jobj.has("metadata"))
		    		{
					JSONObject metadata = jobj.getJSONObject("metadata");
			    		if(metadata.has(SCORE_KEY)) match.put(META_SCORE_KEY, metadata.getDouble(SCORE_KEY) + "");
					if(metadata.has(TITLE_KEY)) match.put(TITLE_KEY, metadata.getString(TITLE_KEY));
					if(metadata.has(ARTIST_KEY)) match.put(ARTIST_KEY, metadata.getString(ARTIST_KEY));
					if(metadata.has(ALBUM_KEY)) match.put(ALBUM_KEY, metadata.getString(ALBUM_KEY));
		    		}

		    		didFindMatchForCode(match, code);
			}
			else
				didNotFindMatchForCode(code);
		}
		firstRun = false;

	}
	catch(Exception e)
	{
		e.printStackTrace();
		Log.e("Fingerprinter", e.getLocalizedMessage());
	}
}
while (this.continuous);

We now have a complete library that can listen to audio, invoke the codegen native library and return the computed fingerprint back to your code.

The final step is then create a sample project that uses the completed code library.

Creating a sample project that uses the library
Create a new project in eclipse, I chose EchoprintTest as its name, and in its properties, go to the Android category and add the EchoprintLib library to the project:

In this sample project, I added a button (recordButton) and a TextView (status) to the main Activity, in order to start/stop the audio fingerprinting process and to display the results from the echoprint server.

In the main activity, we need to implement the AudioFingerprinterListener interface:

public class EchoprintTestActivity extends Activity implements AudioFingerprinterListener
...
public void didFinishListening()
{
	btn.setText("Start");

	if(!resolved)
		status.setText("Idle...");

	recording = false;
}

public void didFinishListeningPass()
{}

public void willStartListening()
{
	status.setText("Listening...");
	btn.setText("Stop");
	recording = true;
	resolved = false;
}

public void willStartListeningPass()
{}

public void didGenerateFingerprintCode(String code)
{
	status.setText("Will fetch info for code starting:\n" + code.substring(0, Math.min(50, code.length())));
}

public void didFindMatchForCode(final Hashtable<String, String> table,
		String code)
{
	resolved = true;
	status.setText("Match: \n" + table);
}

public void didNotFindMatchForCode(String code)
{
	resolved = true;
	status.setText("No match for code starting with: \n" + code.substring(0, Math.min(50, code.length())));
}

public void didFailWithException(Exception e)
{
	resolved = true;
	status.setText("Error: " + e);
}

In order to start the fingerprinting process, all we need to do is create an instance of the AudioFingerprinter class and call the fingerprint method, specifying how many seconds of audio to record per listening pass, the maximum is 30 and the minimum is 10. I’ve found out that for better results, anything above 15 should work fine:

btn = (Button) findViewById(R.id.recordButton);
btn.setOnClickListener(new View.OnClickListener()
{
    public void onClick(View v)
    {
    	if(recording)
    	{
		fingerprinter.stop();
    	}
    	else
    	{
    		if(fingerprinter == null)
    			fingerprinter = new AudioFingerprinter(EchoprintTestActivity.this);

    		fingerprinter.fingerprint(15);
    	}
    }
});

And voila, all you need to do now is do anything interesting with the results returned from the server in the didFindMatchForCode method!

One final thing though, don’t forget to add

<uses-permission android:name="android.permission.RECORD_AUDIO"/>
<uses-permission android:name="android.permission.INTERNET"/>

To your manifest file in order to enable audio recording and internet access!

Now go ahead and download the echoprint server, feed it with some audio files (check the documentation on how to do that) and start developing your own audio fingerprinting apps!

Download the complete source from GitHub repo: https://github.com/gvsumasl/EchoprintForAndroid.

Happy coding!

Andres Solano (Master’s Project Presentation)

When: 9:30 – 10:15am

Title: GVSU Art Gallery Meets iOS: How To Cram 10k+ Works of Art Into Your Pocket

Abstract: The Grand Valley State University Art Gallery has over ten thousand works of original art, including paintings, sculptures, ceramics, and more.  A large portion of this collection is displayed across GVSU’s campuses. These pieces of art are used by students for courses and research and are also simply enjoyed by people each day as they encounter them on campus.  Although each individual work has a placard describing it and its artist, these do not help people locate specific works of art around the different campuses and buildings. Furthermore, the amount of information that can be displayed on a work of art is limited to the size of the placard.  Another difficulty is that there is no way for a person to share or comment on a work of art that they really appreciate. This project addressed these problems by developing and deploying a mobile application that allows users to browse and locate works of art using their iPhone and its network connection and GPS capabilities.  Furthermore, users can use the application to go on virtual art tours, and they can interact with their friends around a piece of art via integration with Facebook, Twitter, and other social networks, all from their iPhone.

Alex Restrepo (Masters Thesis Proposal)

When: 10:30am – 11:15am

Title: Accessorized Therapeutic Game Experiences for Touch-Enabled Devices

Abstract: In the world of physical therapy, a number of consumer gaming devices have been used with various levels of success. Most commercially available video games are designed for the general population and are, in most cases, overwhelming and difficult for traumatic brain injury (TBI) or stroke patients to use.  Specialized therapeutic medical devices are not only expensive and non-portable, they also make limited use of gamification techniques to better engage and motivate the patient. This thesis aims to study the use of inexpensive, portable handheld devices, together with a custom sensor accessory in order to drive a set of therapist designed and configured, short video games. The games are intended to elicit specific therapeutic responses from the patient, and also produce clinical output for the therapists to assess the patient’s progress and refine the treatment regimen.

Preview of the soon to be released Windows Phone edition of GVSU Laker Mobile.

Title: Mango Smoothie – A Developer Overview

When: Friday, November 11, 2pm-3pm

Where: D2-168 MAK  (IT conference room)

Who: Any GVSU faculty/students who are interested in mobile technology

Abstract:

This presentation will give an overview of Microsoft’s Windows Phone Mango.  We will talk about the tools/technologies used to develop for Windows Phone, and showcase the controls available.  We will also discuss our experiences developing a Windows Phone edition of the GVSU Laker Mobile app in the mobile lab.  Our goal is to bring the audience up to date on Microsoft’s new Windows Phone platform. Attendees will gain a good overall understanding of the platform and how to get started developing for it.

Speaker:  Juan Mejia is a graduate assistant in the GVSU Mobile Applications and Services Lab.  He previously graduated with his computer science degree from EAFIT University in Columbia.  Juan is interested in .NET and Silverlight development, and is a Microsoft Certified Professional.