Smartphone Thoughts: Microsoft Research AURA - Barcode Reader that Finds Product Information on the Web

On the Web
Let's take a closer look at the details that are shown in the Web browser, on the mobile device as well as on the desktop. First let's take a look at the experience in the pocket Internet Explorer.


Figure 12: The main page shows the recently scanned item. In this case it is a DVD on my favourite cartoon character, Garfield.


Figure 13: Clicking on the item link shows the barcode information, price, manufacturer and other key details.


Figure 14: Scrolling down on the item details page brings up the links for Amazon.com, MSN Shopping, MSN Search and Usenet Newsgroup. I find the Amazon.com link the most important one.


Figure 15: This screen shot shows the product details as shown on the Amazon.com's Web page. I had clicked on the link titled "Amazon" shown in the previous screen.

Nowadays I constantly find myself researching a product on the Amazon.com's Web site before I buy it. Looking at the product ratings, user reviews and of course the price online. AURA makes this information available in a snap. I can be standing in brick and mortar store, needing help with a certain product that has caught my eye. I can instantly and accurately track down that product online. The same can be said for tagging all the products that I own. This brings us to the next set of screen. This time it is the desktop browser.


Figure 16: The desktop browser experience is similar to the mobile experience. The "My AURA" Web page shows all the scanned items. Click on the image for hi-resolution version.


Figure 17: You can either click on the item links to perform Amazon.com or MSN Shopping look up, or can click on the "+" symbol to show the product details quickly. Click on the image for hi-resolution version.


Figure 18: AURA also provides RSS feed listing all the scanned items. Click on the image for hi-resolution version.

Another Look at the Special Lens
Before I end this article I would like to show how the special lens adapter provided by the Microsoft Research team helps to capture a sharp photo of the barcode. You can also get one. All you have to do is sign up for the BETA testing and send a self-addressed envelope to the Microsoft Research at this address.


Figure 19: Without the special lens adapter, this is how the barcode photo looks like. It is blurry and the AURA client is unable to recognize it.


Figure 20: With the special lens adapter in place the photo is sharp.


Figure 21: This screen shot compares the photo through the special lens and through the regular lens. The top half shows the image through the special lens.

AURA's client does a pretty good job of recognizing the photos of the barcodes taken through the special lens. It always fails if I take the photo without the special lens, no matter how steady and close I hold the camera. AURA successfully scanned and identified a lot of books and DVDs that I had at home. It failed to identify only two products in my testing. It read the barcode but was not able to look up product details. The first item that failed was Pillsbury White Bread. The second item that it could not identify was a children's book called "Cuddles the Cow."

One thing I am not sure of, and it would be awesome if this barcode recognition technology is made available as an API for the developers. If the developers can capture the barcodes in their applications, then we can see some cool applications. For example a coupon finder application; we scan the barcode in and the software finds the coupons for that product. Also enterprise developers can use it for tracking inventory and assets. ;-)

Closing Remarks
AURA is currently a Microsoft Research project and is in public BETA. This is definitely not the final product. I am sure it will undergo many revisions and introduce more features in its final version. The biggest challenge currently for the Microsoft Research folks and for the users is the need to attach the special adapter lens. The camera on our mobile device has to evolve before this product can become a reality. Till then we can definitely play with this BETA, and dream about the possibilities that are waiting to be unlocked in the future.

Kris Kumar is a software engineer with expertise in Microsoft technologies and developer tools. He is married to a wonderful woman, and lives in Rochester, New York, USA.