The Body Strikes Back!

This post is more of an observation and summary than anything else, but I decided to share it anyway. This is a somewhat brief summary of one discussion session we had at the summer school workshop.

Before the era of video games, play typically meant something physical, an activity that involved using the entire body. Certainly there were some board games but nevertheless, play was more physical. We can roughly say the same about work, although the shift from entirely physical labor to office jobs happened a little bit earlier than video games, and there have been professions such as scribes who worked on a desk. Nevertheless, even though we cannot point it out directly, there is some point in history where the balance tipped towards work where people mostly sit down. For games, we know that this shift started with video games, so I'm mostly going to discuss what happened there.

The arcade cabinet and especially old mechanical flippers were in fact quite physical, at least compared to home computers and consoles. At the very least, players had to stand in front of arcade cabinets. According to research, sitting is slowly killing us, so at least ye olde arcade games were marginally healthier. Eventually though, gaming moved from arcades to homes in form of consoles, and players were planted on their sofas. The ergonomy of older console controllers and computer accessories was pretty hideous. At that point, most likely nobody cared. What was important was what these wonderful new devices could do, not how comfortable they were to use. Besides, you can play from your couch! How more comfortable can you get?

If we fast-forward to the 20th century, gaming has become more widespread and the new generation of kids is practically growing up playing games. Outdoors playing and sports are not out of the picture, but as is evident from the growing obesity problem, they do not get performed as much as they used to. Technology has transformed play. What was once play of the entire body has now become play of the mind. The percentage of work that's done on a desk is on the rise. We have started to care about ergonomy and game controllers have assumed much friendlier and rounder shapes. This transformation of play happened in a relatively short time (compared to the entire history of play).

The interesting this is what happened next. The 21st century came around, and gaming technology keeps getting better and better. A lot of technology is devoted to creating better graphics, what with the HD resolution and all. However, something else is happening as well: Guitar Hero introduces special controllers to the wider audience. The already iconic plastic guitar makes gameplay a lot more physical in one sweep and becomes a hit. Not to forget dancing games, a phenomenon somewhat older than the guitar concept. Before the next console generation, there are also gadgets like EyeToy for the PlayStation 2. This particular gadget uses a camera to make the user's entire body a controller. Too bad the games were not particularly great and the tech was somewhat limited.

Then, bang, next generation. Nintendo announces Wii (initially titled Revolution) that uses a motion controller as its primary controller. The vision is clear: get people more physically involved in gaming. The console becomes an instant hit, selling to new markets, and the one killer app is the simple Wii Sports, a collection of mini games that make use of the new motion controller. This is curious because not so long ago people were drawn from sports to video games, and now a game that has the players perform mock sports motions becomes a hit. Then Nintendo releases Wii Fit with a balance board controller. Wii becomes an exercise assistant.

Now Sony and Microsoft have followed suit with their own solutions. Move is basically a higher tech Wiimote, with better accuracy and whatnot. Microsoft Kinect on the other hand uses infrared camera technology to do what the EyeToy wasn't able to fully deliver: transforming the player's body into a controller. We've gotten back to standing and waving our hands. It doesn't end here though. While home consoles are playing with motion tracking, mobile game developers are commercializing the concept of alternate reality games run on mobile phones. Running around in the real world is the name of the game. Sound familiar?

It's not like couch gaming has come to the end of its reign. Most mainstream games are still played in the traditional way, using a controller with some buttons. Regardless, the option to play more physically is available for players of video games. We have yet to see the killer game that really sells physical controls to core gamers but in the meanwhile, a lot of people can enjoy more physical activity in front of their televisions and computers. Or they can go outside where with mobile technology and pervasive games even adults can play in a socially acceptable fashion.

So, in a sense, gaming technology has taken a detour. The body lost the game for a while, but now it's back with a vengeance.

Interaction Conventions - A Local Maximum?

The desktop metaphor was introduced in the Xerox Star in 1981. This year it will be 30 years old. The desktop metaphor lives on. Of course it has evolved during these 30 years, but how much exactly? Not having used the Star, I'm still bound to guess "not much". Following conventions is one fundamental usability principle but if we optimize usability by following the same conventions, effectively iterating over same things again and again, are we bound to reach a local maximum instead of global (in terms of user experience)?

With desktop software, usability is largely operating with WIMP (Windows, Icons, Menus and Pointers). Generally graphical user interface toolkits always share certain conventions. Following conventions in itself is not a bad thing - after all, familiarity acts as great leverage for learning new things. However, it might become troublesome if the conventions themselves become outdated by new technology. WIMP is not designed to deal with tangible interfaces, voice input or gestures.

This has been noted by HCI researches in at least several papers, most likely more, using different names for things. One fundamental limitation in our current applications is that they are designed for exactly one pointer. Multitouch screens on recent smartphones and PCs are slowly paving the way for multiple pointers, allowing a user to touch several points at once. Tangibles, researched since mid 90's, will move beyond touch screens, allowing virtual objects to be controlled by multiple physical objects. Which is faster: dragging objects on the screen with a mouse or moving multiple physical objects on a flat surface?

Technologies for tangible input have been here for some time. Same can be said of voice input. Gestures are making their way, receiving a huge boost from Microsoft Kinect. Combining these technologies with the traditional keyboard (still king for typing) should result in better interaction overall. I'm not ready to kill the mouse either, it is still pretty good for pointing. Sometimes touch screen can replace it, but I don't think there are too many as effective instruments for certain tasks (especially certain games).

So, point? I don't think that our current conventions are able to adapt to these new technologies. Trying to fit these new possibilities within known conventions is more likely to hindrance improvements in overall user experience. Learning new things cannot be avoided forever. Of course in creating new interaction models it is our responsibility to make them easy to learn. Conventions from the physical world should be useful for us.

Changing the desktop world is probably too late though. That is why I've set my sights to ubicomp, where conventions don't really exist yet. It should also be easier for people to accept learning new ways of doing new things, than it is replacing their comfortable old ways of doing things with new ways. Tear down the wall, let creativity reign!

On Controllers - Is the Keyboard Evil?

In this post I am presenting just some random feelings I have about controllers - mainly gaming controllers. These days it is not uncommon for a game to be released on multiple platforms: PC, PlayStation 3 and Xbox 360. One key difference between the PC and the latter two is the way games are controlled. PC games have been using keyboard and mouse for ages already. Console games have been using gaming pads with varying number of buttons (and later analog thumb sticks).

NES controllers had a whopping four buttons and the iconic digital directional controller (aka D-pad). Back then, console games had relatively simple interaction. These days gaming pads have quite a bit more. The D-pad is still there but its use for movement is diminishing and has been largely replaced by an analog thumb stick which allows more accurate control. Another analog thumb stick has been added to the right hand side. Four digital buttons under the right thumb. One digital and one analog shoulder button on each side (for index and middle fingers). Start and select buttons in the middle. A total of 10 buttons and three directional controllers. Out of these, only the start and select buttons are not within instant reach.

On the other side of the fence, a keyboard has at least 102 keys. By using combinations of shift and alt keys, a plethora of characters can be produced with a keyboard. In gaming it is typical to use just one hand on the keyboard (assuming the game requires a mouse). If I place my hand in the classic WASD position (used by gamers for movement, although I actually use WAXD), I can reach about twenty buttons with my fingers quite easily. A typical mouse has three buttons and a wheel these days. Mouse movement typically controls a cursor, camera movement or looking (in fps games). While the difference is not as dramatic as it was in the NES era, the PC gamer still has a lot more buttons available.

So is the keyboard evil? In terms of learning, I am leaning towards yes. Figuring out what key does what is a huge task as there are many to try (remember, sometimes alt, ctrl and shift combinations do different things!) without a reference. Remembering them is another thing. On a pad it doesn't take that long to try each button to see what it does. With modern games it's not exactly as straightforward, but I'll come to that in a bit. A large amount of possibilities is also not always best for interaction design. A pad is a constrained design space - once you run out of buttons to assign actions to it's time to optimize your control scheme. On a keyboard it might be a bit too easy to just add another key.

On the other hand, PC gamers have high customization options for their control layouts. If one thing is seriously wrong with console game makers, it's this: too few games allow remapping of controls. Even if this would be almost trivial to implement. This is not a problem until we run into button combinations. 8 actions in a modern game is often not enough. On pads, developers are overcoming the problem by assigning actions to button combinations - typically combined presses of shoulder and thumb buttons. What I personally don't understand is putting actions behind combinations of two thumb buttons. Hello, I only have one thumb to press these buttons with. Pressing two buttons with one thumb ranges from okay (buttons that are on the same diagonal direction the thumb points to - for example triangle and circle on the PS3 pad) to anatomically impossible (the buttons are on opposite corners of the pad layout - for example, square and circle on the PS3 pad).

Some games handle large amount of actions on a pad gracefully (recent example: Darksiders). Some games on the other hand clearly do not (recent example: Prototype). I've heard (I don't have a modern PC, so I haven't tried) that a modern problem with multi-platform PC games is that their controls have been poorly implemented. The developers have designed console control first (because they are more restrictive) and then made as direct a port as possible to the PC, resulting in game play that is clearly not optimized for the keyboard + mouse combination. While 20 keys are reachable when using WASD, it's worthwhile to remember that at least two fingers are reserved for movement buttons most of the time. It takes some training to use all 20 reachable keys without fluidly.

So what does this random train of thought have to do with usability? For starters, design for your controller. If it easily affords only ten actions, don't try to put twelve in your application. Combinations of keys are typically bad because they make actions "invisible". This can be overcome with clever modifier design. In the console version of Assassin's Creed, thumb buttons are mapped to body parts with spatial association: top button is for eyes, left for left hand, right for right hand and bottom button for legs. When a modifier button is pressed, the action changes accordingly. If the character is in stealth mode, the actions will be subtle - if he is not, they will be more rash. The button for one hand can make the character push crowds aside gently, or throw them aside while rushing through.

Another way to go is visual feedback: when a modifier is pressed, display what buttons do on the screen. This is especially good for learning your controller and interface combination. Packing too much information into the buttons should be avoided - a lot of people might not realize that the spatial arrangement of symbols on a key on a keyboard indicate which modifier should be pressed along with that key to get a particular symbol. When the controller is relatively small, this kind of visual feedback on the display is feasible. For keyboards it would be way too much information.

Coming back to the keyboard, typing is what it affords best. If your application needs a lot of typing, using a keyboard is a necessity and coming up with a replacement that is as easy to learn and as efficient (chord keyboards are more effective but difficult to learn) might provide a rather interesting challenge. However, when binding other actions than typing to the keyboard, defining a good scheme becomes important. Shortcuts for example feel rather arbitrary in most modern applications (which is why I called for customizable shortcuts earlier) . There aren't even any standards between vendors which is a disaster.

Short version? I guess it's "pads rule!"

Features in Need of More Use

Another interlude, I know. A lot is going on but there isn't much to write home about at the moment. So I'll take this chance to discuss some great features I'd seen in some applications. This post was mostly inspired by Dia which I've been using to draw diagrams today.

The first feature is snap to grid which is present in Dia. The idea of this feature is not very radical. The editing area has a grid on it. If snap to grid is turned on, whenever objects are moved, their corners and sizes will snap to the grid. So my box for example will be exactly 6 times 4 grid squares, and is strictly lined up with the grid. This makes two things easy: creating boxes with the same size is easy, and lining up multiple boxes is super easy. Oh and drawing straight lines is almost too easy. This feature's relative is snap to object but I don't find it nearly as useful. What it does? If objects are moved close to each other, they are snapped together. Snap to grid on the other hand should be available in every program where objects can be positioned. Every. Program.

Dia also reminded me of another feature I really like: customizable keyboard shortcuts. Dia doesn't have those, but all KDE applications do. I don't think there's any limit as to how far this should go. I would consider it optimal if every single action in the application could be given a shortcut. KDE apps don't manage this, but their customization is easy. Pick an action from a list, assign a shortcut. My only gripe with them is that some actions I'd really like to assign a shortcut to are not on that list. The biggest "why?" of all: only 12 tab shortcuts available for browsers. Not quite enough. A lot of heavier programs have macros (but they can't fix everything). But: customizable shortcuts - not that hard to program. So where are they?

While on the topic of customization, how about customizable menus. Much like shortcuts, except instead of assigning actions to keys, they are assigned to user-defined menus. This is actually a feature I haven't seen much in applications and can't really name one. I have seen it in some operating systems. Most notably, my current Nokia phone. The phone itself is horrible. Can't remember the model number, but it's their current low end. It has the worst keypad I've ever used. What it has, is a "go to" menu behind one button on the main screen. It's a menu for collecting shortcuts. I don't need to navigate the stupid menu system to get to the most important things.

Sure enough, these features are mostly what could be called "power user features", especially the customizations. Actually, both of these features are very often available in games. Especially PC games. Which is yet another reason for UI designers to go and play some games. The power of these features is largely understood, so nothing revolutionary here. What I just wanted to say is: use them more! Use them everywhere. They're some of the extra features you actually can't go wrong with.

Goal Forming

The last aspect I'll explore in the flow series of posts is goals. This writing has a companion article I have written for Tiny Universes, which explores how rewarding certain kinds of behavior can change how a game is played entirely, using two games as examples. In general, goal forming is an important aspect of user experience, and on lower level also pure usability. The usability aspect is explained well enough on The Design of Everyday Things (Don Norman) - in short, the user needs to be able to understand what goals can be set when using a given interface, and how to use that interface to achieve those goals.

When discussing achievements a couple of posts back, I mentioned they have more to do with goal forming than anything else. When a player plays a game, her first goals are typically to learn the game, proceed through its levels (or whatever) and finish it. There are of course various sub-goals while playing. Achievements or trophies often come into play afterwards and collecting them is another goal. It can also be argued that good trophies are ones that give the player some clear additional goals, which will increase the longevity of the game and therefore improve its value for money ratio. Case in point, before we had these modern consoles with their trophy or achievement systems, Star Ocean 3 had built-in battle trophies. It took me about 50 hours to finish the game, and then I spent another 250 or more hours collecting trophies (I'm still missing 7 out of 300).

In general, a good trophy challenges the player to play the game differently, introducing more difficulty. In Mirror's Edge there is a trophy that requires you to not shoot a single shot during the game. BioShock rewards the player for playing without resurrection chambers turned on. And of course Star Ocean 3 rewarded so many different and interesting things that it kept me hooked for a long time. Typically the only reward is a reminder in your account that you've gotten the trophy, although some games (like Final Fantasy XIII) hand out some minor gifts like operating system themes for getting trophies. Bad trophies are the kind that just require the player to do some repetitive thing a lot of times. More of the same is not very interesting, but doing the same thing differently can be.

Now, let's exit games for a while. With interfaces, especially in interactive spaces where interfaces can be abundant, it is important to get the users to form the right goals. By this I of course mean we need to assist them in forming goals that are relevant to them, assuming they are using our interfaces at leisure. In a work situation on the other hand we might want to introduce reward schemes that support effective work. Regardless of situation though, it's useful to keep the requirements of flow in mind: the user cannot achieve flow if he cannot set goals. So even before thinking what kinds of goals our interface should support, it needs to support goal forming in general. It can then be useful to do some research on what kinds of goals a prototype inspires in people, and then think how they could be adjusted, if necessary.

When the overall goal of an activity is fixed, the system should aid users in forming sub-goals that help them move towards the overall goal and stay motivated. If the activity can be split into a plethora of short-term goals, progress is easy to measure, and if there is flexibility in task order, users can also select short-term goals that are a good match for their current skills. In general, people split activities into short-term goals all the time when working but it definitely cannot hurt to make systems that support keeping track of these goals, and that can also suggest goals.

Before achievements and trophies entered the life of gamers, people used to make up their own challenges. These were definitely no less interesting, but of course no one was certain whether a challenge would be possible at all until someone completed it. With formal trophies, game developers can design the challenges and ensure that they are indeed possible to reach (although, some of the craziest ones are almost unreachable). So for example, in order to improve my writing, I could use a text editor that has various trophies for using language in special ways, like "use 5 different synonyms for a common word in one article". Of course I can make up all these challenges myself, or search the internet, but if it is in fact built into my interface, they are constantly present and easily available for viewing.

Overall, goal forming is a highly relevant topic for user experience. I have once again just scratched the surface, but already discovered at least two important aspects of it. The first one is to gain an understanding of what goals users can form when using your interface. The second one is guiding users to form goals that are constructive towards the overall goal of a larger activity. If every phase of an activity can has its own short-term goal, the user experience is likely to improve. This is too often not the case when viewing the learning curves of more complex applications.

The Emotionality of Fuzzy Things

As you might be able to guess from the title, I've also been reading Emotional Design (Don Norman). Like with the other book, this was my second read-through. What I realized while reading this time around, is that this particular book is perhaps the most essential one for my research. Indeed, most of the things I have been writing about can be derived from what Norman presents in these pages.

Playful usability definitely builds upon the emotional side of human thought. It doesn't even pretend to be functionally optimal. Highly functional interfaces are effective. They take the shortest route. Playful interfaces on the other hand are more creative. Or, using the travel metaphor, they take the scenic route. Or maybe the route that is just more entertaining to drive. Really straight, long roads are effective. They are also really boring to drive, which is why I usually don't pick them. So today, I'll take a look at playful interfaces, through the lenses of the three levels of human thought presented in the book.

I guess before that, a really quick recap of the said levels is in order. For a more complete description, read the book. The three levels of human thought are visceral, behavioral and reflective. The first two are sub-conscious while the last one is conscious thought. Visceral level is the most basic input-output system built into us. It excels at giving out immediate responses. The behavioral level is where activity is performed. It handles any activity that doesn't require conscious thought to perform. While I'm writing this, the movements of my hands on the keyboard are controlled by the behavioral level, leaving my consciousness free to think ahead what I'll write next. And of course, that is what I'm doing on the reflective level. Here I am making decisions how to express myself and my ideas through this text.

Playful interfaces are by nature likely to be more novel than highly functional ones. Sure, functional interfaces can be decorated, but the methods of interaction follow convention. Playful or fuzzy interfaces can surprise the user with more than just visual decorations. At this point, we are clearly working on the visceral level. And I think it's really important to be affective here. It's the wow-effect, "hey, I wanna try that", that should be achieved. Applications and services need to advertise themselves, especially in interactive spaces where high personalization is not affordable (i.e. same services are generally offered for everyone). Often one glimpse at the UI is all you're ever going to get from your potential users, so that glimpse better bring them over. This kind of appeal is especially important to make people aware of services they didn't think they'd need.

As stated, playful interfaces are not even trying to compete in raw functionality. The argument goes: effectiveness can be achieved through enjoyable use, even if the actions themselves are slower. People are not machines, we get bored with uninteresting tasks and our minds wander away. This should not be big news to anyone, hopefully. Take a look at games. Games rarely provide the most effective means to reach a high level goal, which does indeed make them enjoyable. So, in the behavioral level, the aim for playful usability is to make high level goals more enjoyable to work towards. The means towards this end is to make individual tasks not necessarily optimized for speed, but optimized for fun instead.

Of course, making work-related tasks fun while retaining their original purpose is a hard one to tackle. If it was easy, we would most likely have it already. It requires a lot of lateral thinking. Especially in the domain of desktop applications, user interfaces are inside that same old ugly box. I've said this before, but it's worth repeating: that box needs to be tucked away for good. Thinking outside that box is not enough. We must find a new box altogether and then think outside that box. Or, preferably, a lot of boxes solely for the purpose of thinking outside them. Or, well, you get the idea. Just to put things in context, I have picked up the box labeled "game interfaces" and I'm trying to think outside that box.

I'm actually a bit of an optimist in the sense that I firmly believe that for playful interfaces, visceral appeal might be achieved as a side product of good behavioral appeal. At least I'd like to think that most things that are fun to do, also look so. Sure, I can come up with counter examples. However, at least in the beginning novelty is the other factor. Interfaces that are fun to use will most likely look very different from what people are used to, leading into curiosity. There lies a risk though. Control by gestures or speech for example can feel weird to some people. Especially if they have to do it in public. Talking on the phone is easy to accept. Talking to the phone is not.

Finally, we have the reflective level. I'd consider this to be even more important than the other two, but also harder to write about. Certainly, a good user experience is already achieved with enjoyable execution of tasks. On the reflective level, we want users to look back at the experience with positive feelings. Also, we need the user to feel good while using the interface. If we jump back to games, it simply doesn't matter how good playability a game has if it doesn't match the style or theme of the game. Therefore, it is important that a playful interface contributes to the overall experience of performing a higher level goal. And of course, how using the interface sits with the user's self-image.

Taking emotions into the equation raises a whole lot of things to consider when designing playful interaction. User experience is not a simple matter. Even in pure entertainment applications such as games, no one can tell exactly what needs to be done to ensure that ultimate playing experience. Sure, reading a book or ten about game design helps. Reading books on related topics helps more. But in the end, it all comes down to understanding your audience. Even then, sometimes you just have to break the rules, shake people up and bewilder them completely.

Making an experimental interface is easy. Making a successful experimental interface is not. That's what keeps researches like myself ticking.

The Design of Fuzzy Things

Another book-related post. This time I'm going to approach the topic of playful usability via The Design of Everyday Things (Don Norman). I don't think I this book needs any more recommendations, so let's just go ahead with some thoughts of my own*.

I'd like to start with the three models. There's designer's model, which is how the interface is designed to work. On the other end, the user's model, is how the user thinks the interface works. And in the middle is the system image, responsible for communicating how the interface works. In other words, the user's model is dependent on the system model. In fuzzy usability, the system image is intentionally non-obvious. The idea is not to directly communicate the workings of the system but rather present an interface that raises curiosity and allows the user to discover the workings. By obscuring knowledge we make it more desirable to attain.

My hypothesis for the benefits is that interfaces constructed this way are easier to learn. In a typical desktop interface such as the Firefox web browser I'm currently looking at, the user is immediately presented with a lot of actions. Wizards are often used to guide beginners. The downside is that wizards are separate interfaces for doing specific tasks and do not teach the user to use the program's interface. In a purposefully obscured interface, it is easy to start by presenting the most basic of tasks and reveal advanced features when the user "levels up". The application can reveal the presence of features when they become relevant.

We can think of interfaces separated into zones. Anything the user is already familiar with is in the safe zone, and the message is clear "you know these things, it is safe to work with them". On the borders of the safe zone, there is the uncertain zone. The user has an idea of what is there, and it's kind of saying "you might feel uncomfortable here, but these things can make you more powerful". Finally, there is the great unknown, beyond the uncertain zone. When the application realizes that the user is doing something that could be done better with an advanced feature, it can assign quests, sending the user to explore a particular uncertain zone or even an unknown zone. If the user wishes to find a particular feature, the help system can provide directions of how to get to that feature.

Unsurprisingly, this bears high similarity with playing openworld or sandbox games like Grand Theft Auto. The player can explore quite freely, discovering new areas progressively while learning new skills. But when the player wishes to get on with the story, there are certain things that are required, and hence the game sends the player on quests to meet these requirements. Add rewards for mastering new skills. One further way to think of this comparison is that of task. In a game, the task is to complete the story. In an application, the task is to complete some external goal. Typically the difference between these two tasks is their definer: in a game, the task is defined by its designer while in an application the task is defined by the user. The point remains the same though: why shouldn't the user/player have fun while learning the skills required to complete the task?

In terms of affordances, safe zones afford safe using of features with predictable outcomes; uncertain zones afford experimentation with features; unknown zones afford exploration. Restrictions can also be used by requiring certain features to be in the safe zone before the user is allowed to use particular advanced features. This will ensure that the user has grasped some important basic concept before trying to use a feature that heavily builds upon that concept. The restriction also comes quite naturally - the user needs to traverse through the prerequired features on his way towards the advanced feature.

Implementing an interface that conforms to this model should be an interesting task. Some key considerations in functionality are to ensure that 1) the safe zone really is safe and 2) the uncertain zone clearly supports experimentation. Predictability is key in the first consideration, while proper undo support is key in the second. Another implementation question is analysing user actions in a way that the application can decide when it is appropriate to advertise a feature.

* Similar thoughts can be encountered in several HCI articles, and I'll return to them later. Difficulty regulation is a good key word if you're looking for one.

The Future of Computing

Lately I've been traveling quite a bit, so I've had time to read some books. This time around, I'll discuss three of them: The Design of Future Things (Don Norman), Leonardo's Laptop (Ben Schneiderman) and Everyware (Adam Greenfield). I've chosen these three for discussion because in many ways they are about the same thing: the future of computing. Norman's book of course touches a whole lot of other matters about future things, whereas Greenfield's book is about ubiquitous computing specifically, but all three books send the same message. It's time for users to step up.

The machines are about to take over. Not like in the movies of course, but subtly. We are about to become servants to our machines. Clearly, it should be the other way around but without enough thought given to human-centered design, we might end up with machines that think they know us and what's best for us, restricting our actions when they see fit. Machines, that demand our attention and maintenance. Networks that share everything about us without our consent. When computing will be everywhere, using it is no longer a matter of choice. And if the interfaces suck? We are screwed, and we have to learn them. These are some of the scenarios discussed in these books.

If, for some reason, this does not scare you, then think about what user-centered design for future computing could give us. A lot of possible scenarios for the future are presented in Leonardo's Laptop and they all share one central theme: computing should support, not replace or automate, human tasks. Technology should be making our lives easier, not harder. Computing should be available, not integrated. And, contrary to some beliefs, it should never be entirely seamless or invisible. Which computing do you want?

Jumping to another topic, the idea behind the concept of interactive spaces is in many ways similar to ideas described in these books - especially The Design of Future Things. Humans and machines do not talk to each other - not today, probably not in the future - they signal. When communication is limited to signaling, it is better for one participant be completely predictable. In this case of course, it should be the machine. The user knows what he wants to do - there's no need for the machine to try and guess, because no matter how much sensor information and whatnot it has, it cannot read our thoughts (hopefully).

Norman presents a good example from human-to-human communication where this also holds true. Here's another one: sometimes when walking, you find yourself walking on a collision course with another walker and when you avoid - yup, the other guy avoids to the same direction, and probably then the both of you avoid again to the same direction. This is a scenario where neither participant is predictable. Indeed, if you had just kept walking straight, the other guy would've avoided you. This scenario becomes a lot more dangerous when bicycles or cars are involved. It's happened to me with bicycles, and if I hadn't landed on my butt, I might not be writing this.

To wrap up, here are some of the principles of interactive spaces:
- Interfaces are available to the user - not seamless but gracefully seamful.
- Interaction is initiated by the human.
- Interfaces are precitable - they don't make decisions on their own.

Out of these books, I would highly recommend Everyware for anyone even remotely interested in the future of computing, and especially defending human rights in the coming* age. The other two books are good reading for designers, and especially Norman's book is once again really easy to read and even fun. All of these books made me more confident that what I and we are doing is relevant and important.

* In fact, we are partly there. The latest of these books (Everyware) was printed in 2006, and already it was said ubiquitous technology is in part already here.

Food for Thought: Challenge-Reward Dynamics

Every once in a while I have the urge to complete all sidequests in a game I'm playing. Usually this urge happens with Japanese RPGs. But that's not the point of this post. What I would actually like to discuss today, is this topic: can we apply the challenge-reward dynamics present in games to user experience of non-game applications.

To recap briefly, challenge-reward dynamics are one of the reasons games are addictive to play (but definitely not the only reason). Obviously this should not be big news. A typical game provides challenges to the player, and rewards success. Typically the reward is provided in some form of in-game goods or information, although modern games also offer external rewards such as trophies and some digital goods (desktop themes, whatever). Most of the time, the concrete reward is tied to a sense of accomplishment, that familiar pride of overcoming a difficult challenge. Sometimes there are challenges in games that have no concrete reward, but the challenge itself is interesting enough and the sense of achievement is the only reward that's needed.

Applications on the other hand do not usually provide any internal challenges or rewards. When we think about programs like word processors, the challenge is definitely external (well, aside from the challenge of learning to use the application, which by the way is usually not very fun or rewarding). Applications are like tools. You don't want that hammer to provide any internal challenge or reward - you just want to build your house with it. Obviously applying challenge-reward dynamics to tools is probably a bad idea, but who knows? But not all applications are tools these days. Entertainment applications stand somewhere between tools and games, and are a field where ease of use might not be the obvious way to go. This is the application domain I'm primarily interested in.

While usability design principles generally agree that applications should communicate their system model to the user clearly, I think this particular principle should be challenged in entertainment applications. The value of an entertainment (or edutainment for that matter) application is mostly measured by how long can it engage its user. This roughly translates to how long the application can throw something different at the user. Games are definitely champions at engaging players; my personal record for a single player game is over 300 hours, but for online games this is more like the norm! On the other end of the spectrum, let's think of something like a virtual museum.

The idea of a virtual museum sounds good on paper - it's a museum anyone can visit from their home. But think about the concept for a while. Is there any real motivation to keep engaged to the virtual museum experience, which I assume can only be a fraction of a real museum experience. Why just simulate walking in a museum, when a virtual world allows many more options that are relatively cheap to implement. If we introduced challenge-reward dynamics, maybe part of the experience could be actually finding the attractions. The point being, if we want the users to get interested in paintings or whatever the museum is displaying, you might as well make them rewards that are challenging to get in some way. Modeling the virtual world after the real museum should not be the point.

People value things they have attained through some effort. This is something to keep in mind while thinking about entertainment and especially edutainment applications. Exploration and challenge are important keywords for this context, and are not (necessarily) in line with the user model / system model principle. Sometimes hiding meanings can make them more meaningful to the person who discovers them.

Of Word Processors and Context-Awareness

I hate word processors. In my opinion, they are an excellent example of something, that is really natural and easy for humans to do with pen and paper, done so hard that it baffles me every single time I need to do something I previously didn't need. Or indeed, do something I do rarely. Sure, the people who've been using these unnatural abominations for years and years know how to operate them, but if the casual user wants to do something as radical as putting an image somewhere in the document and wrap text around it they can be in for a world of pain. Or, let's say I want page numbers, starting from the very first page but not displaying them for the first ten pages.

It's all in the help of course, but knowing where to look is not always obvious. Open up your word processor's help and look at the size of that thing for a quick reference of what I'm talking about. This is basically understandable, as word processors have (and I guess they need to have) a plethora of features. A lot of these features are stuff needed only by selected few of us - each user group needs only a fraction of the entire feature set. The problem is, with this many features, finding the one you're looking for is difficult. Everything is organized, but before you can make any use of that, understanding the categorization logic is necessary. Simply put, they are not context-aware.

What games (good ones anyway) do effectively and word processors do not, is providing information on how to use them when you actually need that information. Sure, Microsoft tried with the Paper Clip of Mighty Annoyance, and that didn't turn out well, but is also no reason to stop trying. One problem of course is that it's quite hard for the application to understand what the user is doing, due to its monolithic design. Still, the application should at least provide a help shortcut on each and every one of its dialog windows, to open up a list of help topic relevant to that particular dialog. Help should also be action-oriented. When I want those page numbers, I don't want to know what each and every gadget on the dialog goes, I just want to know how to get my page numbers exactly where I want them.

Of course, games have the definite advantage of much better understanding about what the player is about to do, due to their predetermined nature. Games with linear progression are obvious, but also sandbox-games where the player can go where he pleases can still usually have some idea what the player might need to know next. But it's not like we can't make word processors context-aware. Perhaps it'd be best to actually break down their monolithic design, and try an object oriented approach, something programmers should be very familiar with, but which strangely doesn't find its way to user interfaces too often. I think it's better to provide an example scenario.

I start writing a document, and at the moment the word processor only provides the very basic features for writing text and positioning it intelligently. This is all the word processor itself does: manages positioning and text input. Then I realize I want a picture, so I paste in a picture component. When I click on this component, I only see tools for picture management, which is again something I don't need to see when I'm not setting properties for my picture. I write again for a while, until I need to paste in a table component. Lo and behold, there's a table, and when I select it, I have all the table-editing tools at my fingertip, and nothing else. Finally, when I'm done I realize I want to alter the positioning. So I summon up a grid, tell my components and text paragraphs to snap to it (to keep them neatly lined) and drag them around until I'm happy with the results.

See? Context-aware word processor. I actually quite recently found out that Word 2007 (been using Open Office in Linux) does a lot of this, so perhaps there is still hope for word processors as well. The last part of my example scenario on the other hand is something I can do in Excel or similar programs, but not in word processors. There could be a very good reason for this, but it's also a real possibility that there isn't, because software generally tends to follow old conventions. Of course, it is nice that all the new software works somewhat like the one's we've been using before, but if no one breaks this cycle, we're stuck with usability models that have been outdated for a long time, and bringing new people, with little experience using computers, becomes harder and harder as time goes by and feature numbers creep up.

So, perhaps it is indeed time to rethink everything from scratch.