Tuesday, June 3, 2014

Sensor Data and Interactivity: Real Time vs Discrete

I need to organize some thoughts for an upcoming article. There is a lot of interest in using Internet of Things sensor data in my research group, and my work in persuasive computing is being linked to it rather heavily. There are multiple ways to categorize sensor data interactivity. I have lately been mostly considering a simple categorization: real time interactive and discrete interactivity.

In the former scenario (real time interaction), the end user application interacts with sensor data in a continuous fashion. This has the potential to create a feedback loop where a change in sensor data prompts action in the application, which in turn affects future sensor readings. For a very simple example, if a GPS navigator shows that the user is going the wrong way, the user can change their course in response, and therefore future GPS readings will be affected. This approach is technologically challenging, because it demands continuous connection between the application platform and the sensors. For instance, as of now, many physical activity meters have limited compatibility with mobile devices. If compatibility exists, it may be limited to only a certain mobile OS. One key agenda in Internet of Things research is enabling wider interoperability between sensors and user devices, but we are not there yet.

If we are able to guarantee real time interaction between sensors and applications, new opportunities open up for designing persuasive games. These opportunities arise because they give us the ability to include new kinds of activities as game actions. In the past, this has been achieved through specialized controllers and more recently with sensors built into devices (accelerometers in mobile phones, Microsoft Kinect etc.) Once sensors in everyday environments become more and more available, they also become more prominent as something to build a game on. All sorts of new physical activity games are rather obvious developments, but for creative designers all kinds of sensors can become game interfaces - whether for a persuasive purpose or simply harmless fun. Contemporary alternate reality games are just a glimpse of all sorts of crazy stuff that can be done in the future with IoT technology.

However, like I stated, we are not there yet. Still it is quite usual for persuasive applications or games to use sensor data in a more discrete fashion. Pedometer step counts are downloaded into an application periodically - and perhaps more importantly - after the fact. While performing the activity itself (walking), the user will not directly see how the application reacts. Therefore the user in turn cannot react to what the application is telling them. All interaction is therefore delayed. Furthermore, by the time the data is downloaded into an application, it has become just a number. Although it still represents activity, the activity itself took place in the past. From the perspective of using the application, we could substitute the data with a random number within the same range and observe no difference. Generating data is not part of the interaction between user and the application.

This is basically the scenario that I suggested the three axes for in my previous post. To reiterate: it highly resembles free-to-play games because something is brought into the game from the outside (i.e. real money in f2p games). No matter how much we try to dance around the issue, the fact remains that the game rewards the user based on the sensor data it receives from them. It is hard to obfuscate this fact due to the discrete nature of interaction. To alleviate the problem, in-game benefits granted by sensor data input should be carefully considered. For instance, the "rewards" should be thematically appropriate and function as gameplay or customization elements instead of direct measure of success. If the game is used to persuade players to become more fit, their in-game "fitness" should reflect improvements in their real life fitness. Powering up an avatar is one rather obvious example. This allows players to see virtual improvements in their virtual selves before real life improvements become perceptible.

In theory there is an upside to this more limited use of sensor data: because the input is just a number, a game developed for one purpose can rather effortlessly be fielded for another purpose. In reality however persuasive power is likely reduced if there is no thematic connection between activity and application. This concludes my random musings for now.