The Perceptive Portable Device

The Perceptive Portable Device Started on Jul 10 2013 and finished on Dec 14 2014 Abstract: Can a smartphone perceive the environment like a human does? Portable devices are full of sensors, but they are still very limited to understand what is happening from a human perspective: Where am I inside the building? Is my user healthy? Is the baby crying? This side project is my quest to give portable devices such capabilities. Technologies:C++iOSiPhoneXcodeObjective-C Tags:Computer VisionSelf-LocalizationState estimationExperiments2 more...

As you read this, your eyes are receiving photons from all over the screen, but it is not your eyes who understand the message behind these words. It is your brain that translates that raw information into concepts, connecting them with your knowledge, with your experience, providing "meaning" to what you see. Whereas the eye senses the world, the brain perceives it. If smartphones have so many sensors, can they perceive the world as we do? I believe that they might even overpass our capacities one day and become actually smart. Until then, we can program them to mimic human perception to some degree. The goal of this project is to test how far this can be taken with current hardware and state of the art computer vision, signal processing and machine learning. Want to keep this in your radar? Follow me on Twitter or subscribe to my RSS feed for updates.

Image by Ignacio Mellado / CC BY 2.0. The smartphone image was modified from the original by  Brett Jordan / CC BY 2.0

Why portable devices? Because smartphones, tablets and wearables go with the user. They can see what he sees, they can hear what she hears, they can feel how it moves, ... They can catch the user personal experience all the time. This gives portables (or wearables) a huge potential to help people. Do not expect much of this from your desktop computer. However, unlike desktop machines, portable devices suffer from limited battery life and computing power. This constrains pretty much what you can achieve in terms of intelligent behavior, but it makes the challenge more interesting.

This project does not have specific or quantifiable goals besides the fun I am having with it, the discoveries I do and the things I learn on the way. It is more a place where I will gather, under a common topic, a set of side projects and experiments described in blog posts. These posts are listed in section "Related blog posts" below.

And, since you are still here, you might want to have a look at the amazing Project Tango. I am happy to know that Google's ATAP folks are passionate about the same things I am!