On Controllers - Is the Keyboard Evil?

In this post I am presenting just some random feelings I have about controllers - mainly gaming controllers. These days it is not uncommon for a game to be released on multiple platforms: PC, PlayStation 3 and Xbox 360. One key difference between the PC and the latter two is the way games are controlled. PC games have been using keyboard and mouse for ages already. Console games have been using gaming pads with varying number of buttons (and later analog thumb sticks).

NES controllers had a whopping four buttons and the iconic digital directional controller (aka D-pad). Back then, console games had relatively simple interaction. These days gaming pads have quite a bit more. The D-pad is still there but its use for movement is diminishing and has been largely replaced by an analog thumb stick which allows more accurate control. Another analog thumb stick has been added to the right hand side. Four digital buttons under the right thumb. One digital and one analog shoulder button on each side (for index and middle fingers). Start and select buttons in the middle. A total of 10 buttons and three directional controllers. Out of these, only the start and select buttons are not within instant reach.

On the other side of the fence, a keyboard has at least 102 keys. By using combinations of shift and alt keys, a plethora of characters can be produced with a keyboard. In gaming it is typical to use just one hand on the keyboard (assuming the game requires a mouse). If I place my hand in the classic WASD position (used by gamers for movement, although I actually use WAXD), I can reach about twenty buttons with my fingers quite easily. A typical mouse has three buttons and a wheel these days. Mouse movement typically controls a cursor, camera movement or looking (in fps games). While the difference is not as dramatic as it was in the NES era, the PC gamer still has a lot more buttons available.

So is the keyboard evil? In terms of learning, I am leaning towards yes. Figuring out what key does what is a huge task as there are many to try (remember, sometimes alt, ctrl and shift combinations do different things!) without a reference. Remembering them is another thing. On a pad it doesn't take that long to try each button to see what it does. With modern games it's not exactly as straightforward, but I'll come to that in a bit. A large amount of possibilities is also not always best for interaction design. A pad is a constrained design space - once you run out of buttons to assign actions to it's time to optimize your control scheme. On a keyboard it might be a bit too easy to just add another key.

On the other hand, PC gamers have high customization options for their control layouts. If one thing is seriously wrong with console game makers, it's this: too few games allow remapping of controls. Even if this would be almost trivial to implement. This is not a problem until we run into button combinations. 8 actions in a modern game is often not enough. On pads, developers are overcoming the problem by assigning actions to button combinations - typically combined presses of shoulder and thumb buttons. What I personally don't understand is putting actions behind combinations of two thumb buttons. Hello, I only have one thumb to press these buttons with. Pressing two buttons with one thumb ranges from okay (buttons that are on the same diagonal direction the thumb points to - for example triangle and circle on the PS3 pad) to anatomically impossible (the buttons are on opposite corners of the pad layout - for example, square and circle on the PS3 pad).

Some games handle large amount of actions on a pad gracefully (recent example: Darksiders). Some games on the other hand clearly do not (recent example: Prototype). I've heard (I don't have a modern PC, so I haven't tried) that a modern problem with multi-platform PC games is that their controls have been poorly implemented. The developers have designed console control first (because they are more restrictive) and then made as direct a port as possible to the PC, resulting in game play that is clearly not optimized for the keyboard + mouse combination. While 20 keys are reachable when using WASD, it's worthwhile to remember that at least two fingers are reserved for movement buttons most of the time. It takes some training to use all 20 reachable keys without fluidly.

So what does this random train of thought have to do with usability? For starters, design for your controller. If it easily affords only ten actions, don't try to put twelve in your application. Combinations of keys are typically bad because they make actions "invisible". This can be overcome with clever modifier design. In the console version of Assassin's Creed, thumb buttons are mapped to body parts with spatial association: top button is for eyes, left for left hand, right for right hand and bottom button for legs. When a modifier button is pressed, the action changes accordingly. If the character is in stealth mode, the actions will be subtle - if he is not, they will be more rash. The button for one hand can make the character push crowds aside gently, or throw them aside while rushing through.

Another way to go is visual feedback: when a modifier is pressed, display what buttons do on the screen. This is especially good for learning your controller and interface combination. Packing too much information into the buttons should be avoided - a lot of people might not realize that the spatial arrangement of symbols on a key on a keyboard indicate which modifier should be pressed along with that key to get a particular symbol. When the controller is relatively small, this kind of visual feedback on the display is feasible. For keyboards it would be way too much information.

Coming back to the keyboard, typing is what it affords best. If your application needs a lot of typing, using a keyboard is a necessity and coming up with a replacement that is as easy to learn and as efficient (chord keyboards are more effective but difficult to learn) might provide a rather interesting challenge. However, when binding other actions than typing to the keyboard, defining a good scheme becomes important. Shortcuts for example feel rather arbitrary in most modern applications (which is why I called for customizable shortcuts earlier) . There aren't even any standards between vendors which is a disaster.

Short version? I guess it's "pads rule!"

Teaching Experiences - Revised

I have now given evaluations for my game programming course, and also finished my part of teaching normal elementary programming. That means I have also received feedback.

As stated earlier, the course was structured to be somewhat game-like. I had prepared one large example and taken snapshots of its development phases. It started out simple, with just drawing a ball on the screen, and expanded slowly towards a complete game. On lectures we followed the progress of this example, with each new concept adding new features to our game. Much like each new ability in a game allows a player to face new kinds of challenges. Each concept was introduced using carefully selected or made lolcats and sometimes simple diagrams. Then I proceeded to explain the concept, using a whiteboard to visualize what I was talking about. Then we looked at the cool things that can be done to our game using that concept. The final step with each topic was to give the students a new feature they should implement to the game.

This was repeated with most topics. Some topics like comments, which are not very cool, I just explained and demonstrated. The pace was set so that during any given hour there would be at least some programming for the students to do. This method of teaching was appreciated in the feedback I got for the course. Feedback in general was positive. The things I should pay more attention to in the future were the order of some concepts, and overall pacing of the course. I could have also documented the example code a bit more. For some reason the students really liked my lolcat-infested lecture slides. I don't know if the slides were very useful, but if they were memorable, there's always a chance they helped in remembering the actual concepts as well.

Like games, the course gave the students their first success early - I had them make something interesting and impressive (yet simple) on the very first lecture. Similarly, the course structure was very gamey. Students were given new "abilities" and then immediately presented a chance to use them. Need to make game objects interact? Here's collision detection to help you! This way they progressively leveled up as game programmers. Course work assignment was also one long project instead of isolated homework exercises. Relatively long anyway, 100 hours was reserved for working with it. Most of the students who passed the course reported less hours though. The work was divided into phases, but I timed them a bit poorly. The first deadline was too late into the course, and the remaining two were both after lectures had finished.

If I get the chance to teach more game development courses, I think I should look for ideas to make them even more game-like. I had one quite good interactive lecture experience while teaching elementary programming. Our topic was designing programs. I chose a fairly simple problem, one that was easy to represent and understand, and then started asking the students how they would make a computer program to solve it, step by step. So instead of telling students how programs are typically designed, I went through the process with them instead. This is something I should do with a future course. Present initial conditions and then work with the students to make something of them.

Hopefully our department sees the value of game programming in teaching programming. My current evidence suggests it can be very helpful in motivating people. I also think teaching is valuable experience towards making applications that are easy and fun to learn.

Learning Game Design - Introduction

Yesterday we had our first game design learning project meeting. The project goal is to learn the basics of game design and put them into practice. It's a project I started for members of STAGE. For me, it's also an important project to get more familiar with game design. I mean sure, I've read some books on game design, and done some designing and even prototyping myself, but getting more practice is key. I don't think I can credibly bring game design concepts into usability design if I haven't designed a lot of games myself.

The plan is to use Ian Schreiber's online course material from last year. The course is called Game Design Concepts, and should take us twenty to thirty weeks. Our study group is something like five people - a suitable size to keep meetings sane. Each of us will do reading and exercises on our own time, and then we will meet weekly to discuss the week's topics and exercises. Once the exercises start to be about actually designing something, we will include play testing into our meeting agendas. The course is about non-digital games, as these are much easier to do solo.

So, I will probably write more about the course, especially about the things I have learned, in the future.

Features in Need of More Use

Another interlude, I know. A lot is going on but there isn't much to write home about at the moment. So I'll take this chance to discuss some great features I'd seen in some applications. This post was mostly inspired by Dia which I've been using to draw diagrams today.

The first feature is snap to grid which is present in Dia. The idea of this feature is not very radical. The editing area has a grid on it. If snap to grid is turned on, whenever objects are moved, their corners and sizes will snap to the grid. So my box for example will be exactly 6 times 4 grid squares, and is strictly lined up with the grid. This makes two things easy: creating boxes with the same size is easy, and lining up multiple boxes is super easy. Oh and drawing straight lines is almost too easy. This feature's relative is snap to object but I don't find it nearly as useful. What it does? If objects are moved close to each other, they are snapped together. Snap to grid on the other hand should be available in every program where objects can be positioned. Every. Program.

Dia also reminded me of another feature I really like: customizable keyboard shortcuts. Dia doesn't have those, but all KDE applications do. I don't think there's any limit as to how far this should go. I would consider it optimal if every single action in the application could be given a shortcut. KDE apps don't manage this, but their customization is easy. Pick an action from a list, assign a shortcut. My only gripe with them is that some actions I'd really like to assign a shortcut to are not on that list. The biggest "why?" of all: only 12 tab shortcuts available for browsers. Not quite enough. A lot of heavier programs have macros (but they can't fix everything). But: customizable shortcuts - not that hard to program. So where are they?

While on the topic of customization, how about customizable menus. Much like shortcuts, except instead of assigning actions to keys, they are assigned to user-defined menus. This is actually a feature I haven't seen much in applications and can't really name one. I have seen it in some operating systems. Most notably, my current Nokia phone. The phone itself is horrible. Can't remember the model number, but it's their current low end. It has the worst keypad I've ever used. What it has, is a "go to" menu behind one button on the main screen. It's a menu for collecting shortcuts. I don't need to navigate the stupid menu system to get to the most important things.

Sure enough, these features are mostly what could be called "power user features", especially the customizations. Actually, both of these features are very often available in games. Especially PC games. Which is yet another reason for UI designers to go and play some games. The power of these features is largely understood, so nothing revolutionary here. What I just wanted to say is: use them more! Use them everywhere. They're some of the extra features you actually can't go wrong with.

Introducing STAGE

It's been a while since the last entry. There are things going on, and interesting stuff is about to happen. Results will follow. In the meanwhile, I'd like to talk about STAGE (backronym under discussion). In short, it's a game development club primarily for students in the Oulu region. The ball was set rolling by me and Ari Kangasniemi (Games Research Center, University of Oulu) earlier this Autumn.

The point of STAGE is to offer students interested in any aspect of game development a local community. We don't have that much gamedev studying options in Oulu (next to none at the uni, some at other schools) but apparently we do have lots of students interested in it. Bringing them together is a logical step, one that should have been taken some time ago. Now it's under way. We will provide information about work and study possibilities, self-study resources and a channel of communication for starting projects and getting people to work on them.

For my own research, the club is an important channel to gamedev. It is also a limited possibility to work on some game projects that might support my research, assuming I can make them interesting enough to get people participate in them. But that is definitely in the future. For now, I think the most important aspect is being able to contact other people interested in gamedev. Throw around ideas and, of course, make some games. Stuff I'm really looking forward to are game jams. First our local practice jam, then Global Game Jam next January. I think I'll focus on programming there, unless I come up with some really impressive design and decide to pitch it.

I think that's it for this interlude. Expect updates on STAGE after said jams, and possibly other stuff as well. In the meanwhile, if you are interested, go take a look at our forums (they are in Finnish though).

Cooperation in Games (and Elsewhere)

Last week on Friday I and a few fellow researchers were taking part in a cooperative game session. The purpose of that particular game is to improve communication inside teams. We tried it out for more research-oriented reasons. There is a lot we already know about playing together and cooperation but this doesn't prevent me from thinking it a bit. So here we go: another game-related blog topic. Just some general thoughts about cooperation.

Lately I have mostly cooperated with other players in board games. Recently cooperative board games have been on the rise. They are a good concept for a couple of reasons. First of all, in cooperative games it is less important for the players to be evenly matched. They are competing against the game after all, not each other (although some of them have one or two players as traitors trying to undermine the others!) This makes them more accessible to less than hardcore players. The other reason is just that thinking together is Fun (yes with a capital F). I also do believe these kinds of games are good for improving communication skills (there is most likely research done on this subject, something I'll have to dig up later).

Board games also integrate the benefit of bringing people together into the same physical space. However, cooperation in digital games over the internet is no less fun. One quite important difference worth notice is what skills each player brings to the game. In board games, players mostly bring just their brain. Manual dexterity, hand-eye coordination and so on are not really important because the games are not played in real time*. In digital games played on consoles or computers, players typically need to be familiar with appropriate input devices. Real-time games typically also require reflexes. First-person shooters require good hand-eye coordination. Put simply, they are somewhat less accessible for non-experienced players.

Digital games do have a benefit over most cooperative board games. In board games, where thinking time is theoretically infinite one player can, in theory, play the game by himself. This causes a situation where less experienced players find themselves being told exactly what they should do on their turns. This is seen as a problem among the board gaming community, largely because in board games carrying out actions is trivial. Fun is in the thinking. In digital games it is much easier for game developers to make games where one experienced player cannot play for the whole team. Experienced players can still be leaders, giving out directions or even orders (in more organized play) but overall success is much more dependent on everyone's skills.

Assuming the aim is to improve everyone's communication skills, some tricks can be employed. Many board games employ rules that somehow limit communication. For instance, in many cooperative board games players are not allowed to show their cards to others. Often it is even forbidden to talk about one's cards in detail. Another thing that can be done in board games is enforcing time limits. If time is limited, one experienced player simply doesn't have enough time to mind everyone else's playing. Basically any kind of knowledge distribution scheme works wonders.

In digital games critical resources can be distributed among the players. When everyone can do something others can not, everyone needs to be involved. In most games, location can be considered a resource. Information visible from one particular place can be valuable and therefore require one player to stay there and report said information to others. Generally speaking, dividing the game to multiple terminals (computers or consoles) allows more player specific information which is also harder to convey to others. In the spirit of time limits, the game could even enforce microphone time limits. Speak too much, and your mic goes silent leaving leading for someone else to do.

Ahem. Looks like I sidetracked quite a bit. But hey, at least now you might know more about the specifics of cooperative games. Cooperative play is motivating. Winning or losing is no longer just about yourself but the whole team. It also makes many seemingly less interesting games much more interesting (try and play some coop board games solo, they are quite boring). Cooperation is also generally important at work. Most people these days don't work solo. As usual, there are two angles when looking at games and real life cooperation. The first angle is improving communication skills via suitable cooperative games or game-like activities. The second angle is making tasks more motivating by increasing game-like cooperation.

The second angle is a matter of tools. Cooperation generally leads to better results and is therefore already desirable. Many applications on the other do not support cooperation. Means of remote communicating need more improvement. Voice and text are often insufficient. It is much more powerful to show things. Solutions have been developed and will be developed in the future. One important requirement for communication and cooperation is that it needs to be effortless. Cooperating with people on the other side of the globe should be as effective as with people in the same office. At the moment this clearly is not so. Perhaps games can help us here. That is one more question for me to think about.

* There is at least one really good exception. Space Alert is a game where players together as a team have exactly ten minutes to plan all of their actions for the game. These are then carried out afterwards to see how the plan holds together. The time limit demands quick thinking and really good coordination from players.

Goal Forming

The last aspect I'll explore in the flow series of posts is goals. This writing has a companion article I have written for Tiny Universes, which explores how rewarding certain kinds of behavior can change how a game is played entirely, using two games as examples. In general, goal forming is an important aspect of user experience, and on lower level also pure usability. The usability aspect is explained well enough on The Design of Everyday Things (Don Norman) - in short, the user needs to be able to understand what goals can be set when using a given interface, and how to use that interface to achieve those goals.

When discussing achievements a couple of posts back, I mentioned they have more to do with goal forming than anything else. When a player plays a game, her first goals are typically to learn the game, proceed through its levels (or whatever) and finish it. There are of course various sub-goals while playing. Achievements or trophies often come into play afterwards and collecting them is another goal. It can also be argued that good trophies are ones that give the player some clear additional goals, which will increase the longevity of the game and therefore improve its value for money ratio. Case in point, before we had these modern consoles with their trophy or achievement systems, Star Ocean 3 had built-in battle trophies. It took me about 50 hours to finish the game, and then I spent another 250 or more hours collecting trophies (I'm still missing 7 out of 300).

In general, a good trophy challenges the player to play the game differently, introducing more difficulty. In Mirror's Edge there is a trophy that requires you to not shoot a single shot during the game. BioShock rewards the player for playing without resurrection chambers turned on. And of course Star Ocean 3 rewarded so many different and interesting things that it kept me hooked for a long time. Typically the only reward is a reminder in your account that you've gotten the trophy, although some games (like Final Fantasy XIII) hand out some minor gifts like operating system themes for getting trophies. Bad trophies are the kind that just require the player to do some repetitive thing a lot of times. More of the same is not very interesting, but doing the same thing differently can be.

Now, let's exit games for a while. With interfaces, especially in interactive spaces where interfaces can be abundant, it is important to get the users to form the right goals. By this I of course mean we need to assist them in forming goals that are relevant to them, assuming they are using our interfaces at leisure. In a work situation on the other hand we might want to introduce reward schemes that support effective work. Regardless of situation though, it's useful to keep the requirements of flow in mind: the user cannot achieve flow if he cannot set goals. So even before thinking what kinds of goals our interface should support, it needs to support goal forming in general. It can then be useful to do some research on what kinds of goals a prototype inspires in people, and then think how they could be adjusted, if necessary.

When the overall goal of an activity is fixed, the system should aid users in forming sub-goals that help them move towards the overall goal and stay motivated. If the activity can be split into a plethora of short-term goals, progress is easy to measure, and if there is flexibility in task order, users can also select short-term goals that are a good match for their current skills. In general, people split activities into short-term goals all the time when working but it definitely cannot hurt to make systems that support keeping track of these goals, and that can also suggest goals.

Before achievements and trophies entered the life of gamers, people used to make up their own challenges. These were definitely no less interesting, but of course no one was certain whether a challenge would be possible at all until someone completed it. With formal trophies, game developers can design the challenges and ensure that they are indeed possible to reach (although, some of the craziest ones are almost unreachable). So for example, in order to improve my writing, I could use a text editor that has various trophies for using language in special ways, like "use 5 different synonyms for a common word in one article". Of course I can make up all these challenges myself, or search the internet, but if it is in fact built into my interface, they are constantly present and easily available for viewing.

Overall, goal forming is a highly relevant topic for user experience. I have once again just scratched the surface, but already discovered at least two important aspects of it. The first one is to gain an understanding of what goals users can form when using your interface. The second one is guiding users to form goals that are constructive towards the overall goal of a larger activity. If every phase of an activity can has its own short-term goal, the user experience is likely to improve. This is too often not the case when viewing the learning curves of more complex applications.

Teaching Experiences - Game Programming Course

As I mentioned briefly, this Autumn I was teaching a course called Introduction to Programming Through Game Development. It was a short short course consisting of roughly 30 hours of combined lectures and guided practice. The idea of the course was to try out a new method for teaching basic programming. Unsurprisingly it was targeted to students interested in game development. Today I had the last guided session. Final results of the course will be evaluated once the students have turned in their assignments. For now, I will share some thoughts I had about the actual teaching.

Cramming basics of programming and basics of programming into one course is a lot of things to teach. If compared to our department's normal elementary programming course, the amount of topics is most likely doubled. The saving grace is motivation. However, in retrospect I should have probably reserved more time for the basics of basics. This time I got away with it, if only because everyone actually attending the course knew at least something about programming. Although this also means we didn't exactly reach our intended target audience. The first note to self: the course needs to be longer or some topics, no matter how useful they would be, need to be dropped.

The idea that motivation is higher when the topic is truly interesting is not surprising. From some comments, and some impressions I had while teaching, the course did seem to embody this idea. Our topics were more advanced and they were introduced on a quicker pace. Most people were still able to keep up. The course implementation suggests that those who didn't should have come up to speed on their own time (around 10 hours per week were scheduled for independent learning). It is safe to say however that motivation is clearly not enough to make this really really work as an introductory course.

My teaching paradigm can be summarized into a few words: "Provide impressive results early and often". Students on a programming should get their hands dirty within the first hour. This is not unlike games - the player needs to achieve something within the first five minutes or so. Drawing objects on the screen is in fact very easy when using XNA, so this requirement was definitely met. This I found to be in general a good thing when teaching game programming. Results are highly visible. Especially in the beginning. Once they know how to make a sprite move, the students are hooked. Then it is possible to move into the nitty-gritty of programming for a while. When they start to get bored again, it's time for more impressive stuff.

I still need to work a bit on this almost game-like course structure but I do think it definitely works. Starting with the easy stuff might sound like the way to go, but I think it's better to go for the impressive first even if it is a bit harder. Another really good thing about game programming is teaching object oriented programming. It becomes a lot easier if the objects are actual sprites on the screen. You can almost touch them, that's how concrete they become. OOP in general is one of the harder programming concepts to grasp but also one of the most important. Therefore it's really good if we can make it more accessible. In game programming it's also quite impossible to get very far and stay sane without using OOP.

To summarize, I think my teaching ideas are sound in principle but I need more experience. This course was just an experiment, but we're hoping to make it into a real course in the future. This will be my chance to improve it further, and of course improve my teaching at the same time. In the meanwhile, I'm teaching normal elementary programming for the next ten weeks, which will be another chance to improve my teaching abilities.

Next month I can follow up on this after seeing the assignments and feedback from students.

Audio Mostly in Piteå

Before continuing the Flow-series of posts, I'm presenting you a conference report. Audio Mostly is a conference for interacting with sound, so what was I doing there? Well, coincidentally, the organizing party is collaborating with us in the II City project, and my work just happened to be in demonstrable condition. Of course, now that it's been demoed I guess I can also write a post describing it a bit. Later though.

While I'm no audio expert (in fact, I just know how to listen to music - but I do think I'm fairly good at it!), the conference had much to offer, mostly in the department of game audio. It was also a good glimpse into the field of audio design, something I previously had little or no idea about. So now I know a little bit, and I can put my knowledge to use in game design or elsewhere. Although I will still have to rely on someone else to actually produce the audio - I have vowed to stay off the arts. That is of course a time issue - if I had the time, sure, I'd like to be my own graphics artist and audio producer.

The more interesting papers were about researching how people perceive audio. What kind of audio is suitable for different situations, how well audio expresses something, is it possible to tell stories using non-speech audio only and so on. From game design point of view, this kind of information is highly valuable (there even was one paper about horror game sounds). From the interface point of view, I'm kind of skeptic. As I am writing this, I'm listening to music and I have turned off every single audio signal from my operating system. There is only so much audio that can fit into one environment. Audio feedback is good, but relying on it too heavily can backfire. But that is just my opinion, and I should probably check some papers on the topic.

Since it was a small conference (they're keeping it that way intentionally) the atmosphere was really relaxed and enjoyable. I met a lot of people who were doing interesting things, and many audio-related conversations were had. Overall, Audio Mostly was more fun than Foundations of Digital Games, but not as useful for my research. Also, the chances of me going there again are kind of slim, especially now that I'm slowly transferring my work to a new project from II City (which also ends before Audio Mostly is next held in Piteå, which would be 2012 - they're having it in some more remote location every other year).

Difficulty Regulation

Lumines got me again. I hadn't played it for a while, and then yesterday I decided to sink my teeth in once again. It had one mode I hadn't really tackled before: a puzzle mode where the aim is not to score points, but to form various shapes of one color from the blocks. The reason I hadn't tried it before was that I had to actually check the internet to understand how I'm supposed to form those shapes. So there's a quick lesson: sometimes instructions are simply needed. But that is not the topic today. There is a reason I mentioned Lumines and the puzzle mode though, and we'll get to it soon enough.

Difficulty regulation strategies in games are important tools for keeping the player in flow. One quite obvious strategy is to simply include multiple difficulty modes to play the game in, allowing the player to choose a mode he deems suitable for himself. Of course, one problem is: how can the player know which one to pick? Sometimes descriptions can help, such as BioShock's "Choose this if you have never played first person shooters before" description for its easy mode. This is a relatively simple strategy to implement and has the benefit of hugely improving a game's life cycle for advanced players. The only thing a game designer needs to worry about is making the correct adjustments to make the feel more difficult but not unfair. Also, players might need some encouragement that after finishing the game, their skills are on the level needed to tackle the more difficult mode.

Difficulty increase built into progression in the game is most likely the most common way. The next level is typically more difficult than the one before it. Yet it is not always easy to design a suitable difficulty ramp. In many games, some of the most challenging moments are somewhere in the middle - towards the end, player skills and character powers are becoming too big for the challenge. Another thing is that the difficulty curve should not be a ramp. I think it's important to drop in tasks that are a bit more lenient than the previous ones. This way the player, after mastering some tough challenge, can actually feel that he has gotten better, as problems that are still much harder than the first ones are becoming easy. This is exactly how the puzzle mode in Lumines got me so addicted. Every time I succeeded at a hard problem, the next ones felt like a breeze and got me to play more and more. When the next harder problem hit, I already had felt the satisfaction of success, and I was mentally prepared to face it.

Finally there's dynamic difficulty regulation. In this case, the game uses some mechanism to figure out when the player is not doing too good, and then proceeds to make the game easier. Sometimes this difficulty reduction is invisible, affecting only numbers inside the game's engine or reducing the effectiveness of artificial intelligence - other times it is more visible, when the game reduces the challenge itself by introducing fewer enemies for example. So essentially the game cheats for the player's advantage. This is quite typical in tabletop role-playing games as well when the game master makes his die rolls in secret, and can then freely adjust the result to keep the game more enjoyable. The problem with this approach is that some players might indeed feel cheated out of the challenge. Frankly, they don't appreciate your concern. Personally I think there should always be the option to turn dynamic difficulty regulation off.

Finally, multi-player games where players compete with each other are difficulty regulated by player skill. In these games, one important aspect is to try and match players with somewhat equal skill levels to keep everyone in the flow zone (i.e. not bored and not frustrated).

So, these are the most typical ways games do their best to keep players in flow. Can these strategies be used for real-life applications and tasks? It does seem to highly depend on the task. Difficulty is typically in the task itself, so we really cannot touch it. What applications can do though is to suggest a good task order for the user, based on past performance and statistics. This way users can more easily find tasks that match their skills. This approach does seem more suitable for tasks that are part of hobbies - at work the list of available tasks is often narrow, and there is not much choice in what to do and when.

One other way applications can do difficulty regulation is the level of automation, which I have already discussed in an earlier post. At first, tasks can be done in less detail, relying more on the application to work its best guess magic. When the user becomes more familiar with the application and tasks, more and more control is placed in the user's hands. Of course, the user will need full capability to override the machine's decision at any time, regardless of "difficulty mode".

I guess that's it for today.

Achieving Something... But What?

In going through the three topics I mentioned in the post on Flow, I'll start with the last one. Today, I'm going to talk about achievements - a particular kind of feedback present in both real life and games. Truth be told, I'm mostly going to talk about games, but you probably already guessed that anyway.

These days, games feature quite literal achievements in form of, well, achievements (XBox Live) or trophies (PlayStation Network) etc. These are built-in goals attached to the games, which can be pursued in addition to simply finishing the game. Some are quite trivial to get and usually gotten without any extra effort. These are not of much interest, as they do not motivate the player to do any extra work. The interesting ones are those that give players more reason to play the game beyond finishing it such as "Finish the game without firing a single shot", "Collect all secret items", "Defeat ultrahardbossmonsterofthemonth" and so on. However, it does seem that these kinds of achievements have more to do with setting goals than feedback. They are of limited use in measuring progress, but it's strictly on/off.

For the purposes of this topic, high scores and time attacks are of more interest. Both are ways of getting feedback immediately after the fact, and even during playing (if score/time is displayed). Hitting a better score or time is a clear sign of improvement - especially if you are improving your average result over time. Another quite similar way to measure one's own progress in a game is to see how far you get before game over. This is especially true in games that never really end, and the only real goal is to get better/faster/further (like Lumines Supernova, discussed in detail earlier). Similar measuring of real world tasks is possible as well, and at least one example was even mentioned in Flow. Of course, this kind of progress measurement works best with tasks that are repeated more than once. My job, for example, has little repetition. I don't write the same software twice or write the same article twice (unless I miserably fail with backups). I can optimize my software though.

Another quite similar method is measuring success in relation to others. Ranking lists, or in activities that support competition, win rates against different opponents work similarly. Of course, this way you are measuring your progress in relation to others and theoretically it's possible not to notice any improvement, assuming everyone progresses at the same rate (but of course in real life this is never the case). Competition also has some possible negative side effects (not everyone likes to compete), which is not the case when using scores or times to simply measure your own progress (of course, if you start comparing results with others, enter competition). I definitely do think everyone should have the benefit of privacy in measuring their progress.

The first step to take advantage of scoring-based feedback is to apply it to interfaces that are used for repeated tasks. The key is to make feedback easily accessible for those who want it. It is once again important to recognize that not everyone wants to be measured in this way. Another important point is to create a scoring system that encourages the best ways of doing the task. For other tasks that are not so easy to score (or time), providing statistics that can help the user make up his own scoring system could still be a useful way of providing more feedback. I for example am among people who are interested in the statistics of their own actions. Which word I use most in my blog? Do I have some preferred programming habits? Maybe I'd like to improve my vocabulary and start using more synonyms - getting statistics could empower me to do this.

So extending feedback beyond what the system is using, including statistics or measurements of user behavior into the application, is one direction I think should be explored more widely than it currently is. I also do think it is one step towards applications that are more supportive of flow activities.

A Look at Games - Flower

Before moving to more serious matters considering flow and user interfaces, let's take a look at a one particular flow experience. In other words a game, and more precisely, Flower. The game has received high acclaim all around, so I too had to see for myself what it's all about.

In Flower, you control a single petal, and your objective is to collect more petals by guiding yourself into flowers, at the same time bringing life back to the world. Some flowers open the way forward, allowing you to move into new areas. The basic concept might not sound very impressive but playing the game is a highly immerse experience for many reasons. First of all, audiovisual presentation of the game is really artistic and well-thought. Go on, find a game play video from YouTube and come back here. Disagree in the comments if you feel like it. Either way, for me the key point is how lifelike everything feels when you, as a stream of petals, are flying through the landscape, parting tall grass as you go.

Much like Lumines, which I discussed in my previous look at games, Flower also combines background music and sound effects as a for of composition. Guiding yourself through a line of flowers successfully creates a pleasant musical piece which further enhances the thrill from visual effects. Flower rewards success instantly - and not with points, better weapons or anything like that - but rather with a powerful feeling of satisfaction. This is a good sign that flow is happening. Of course, it's not just the feedback that makes immersion in Flower so strong. So, what is the key?

The fact I've been withholding is the way Flower is controlled. It uses just one button in combination with the PlayStation 3 controller's motion detecting capabilities. To change direction, just tilt the controller. Of course it's not a new idea, the Wii has been around for quite a long time already, but in Flower it really really works. You see, it's actually not very easy to control a flying stream of petals using motion detection, especially when they are flying quite fast. There is challenge, and the beauty here is that much of the challenge comes from the controls; the game challenges the player in a very physical way. And of course, the levels in the game do get harder.

Harder in a sense at least. I'm not sure if you can actually fail in Flower. No matter how long it takes to get that winding flower path collected, whether you get it on a single pass or twenty passes, the game itself does neither reward or penalize you. The player can set his own goals, getting better and better all the time, and there is almost always a longer path of flowers to collect on a single pass. That is, if you want to do that. It's entirely possible to just enjoy Flower as an experience, and sense of achievement can always be found. When you do a sharp turn, you can see the tail of your petal stream, in all its colorful glory, and realize that it is your creation.

Flower does a good job of inspiring flow. The challenge always fits the player's skills, as long as you're willing to set your goals (which happens almost automatically anyway). The game provides instant feedback and sense of achievement. Finally, the game allows you to marvel at what you've just created. That said, it's not without flaws. I didn't like the second half nearly as much as the first half, as it becomes more challenge-oriented. But it's not a long game, so go ahead and just play it now, if you own a PlayStation 3.

But wait! What can interface designers learn from it? Everything. If I could make a user interface as elegant and beautiful as Flower, I think I could stop my research right there.

And That's how I Flow

Guess which book I've been reading lately. I've accumulated some topics for blog posts lately, so I'll try and put them into writing in the near future. Starting with this one obviously. So the book I just finished was, if you didn't guess already, Flow - The Psychology of Optimal Experience (Mihaly Csikszentmihalyi) which is a book that gets mentioned constantly when discussing game design. There is, of course, a very good reason for that. In Flow, the fundamental guidelines to enjoyable game play are laid out, although it's not a book about games. The book in itself is not particularly lengthy, and if you just want the fundamentals, reading the first three or so chapters should do the trick (the rest is for you who want to know how to enjoy life to the fullest), but I'll go through the fundamentals very quickly.

Flow is the most enjoyable state of the mind, and flow is a very descriptive word for this state of mind as well. You remember those hours you spent solving that really hard problem at work? They did seem to go past awfully quick, right? That's flow, basically. When we are really focusing on a task that is challenging enough for our skills, that's when we reach flow. While in flow, we can forget about everything unrelated to the task at hand and avoid what is called psychic entropy, or disorder in consciousness. The activity and the person become one entity. Flow exists between boredom and frustration; if the activity becomes too easy, it's boring, and if it becomes too difficult, enter frustration. To stay in flow, the challenge of the activity needs to grow along with the person's skills.

Of course, some people can turn any activity into flow by designing their own goals within the activity while some people can't find flow even in the best circumstances. However, if the activity itself is designed with flow in mind, the state should be far easier to reach. This is why the concept of flow is extremely important for game designers. Games provide a continuous series of challenges and associated long (e.g. finish the game) and short term goals (e.g. defeat the boss). I will go over goals and goal forming in a later entry. In the optimal situation, a game can continuously provide interesting challenges to the player as her skills develop. Reality of course typically falls a bit short of the mark because player skills develop at a different pace. To compensate, difficulty regulation strategies are typically present in games.

The interesting question here is can we use interface or application design to improve the chances of turning tasks into flow activities. I have actually visited this topic earlier in several blog entries, although I didn't use the term flow activity as I hadn't read the book yet. One really simple idea that has been evaluated in at least one research paper* by researchers at Nokia Research and University of Tampere is to add achievements as an additional way to track one's progress. What makes games like Tetris or pinball machines highly addictive is the ability to compare your results to previous achievements of yourself or others. Score keeping is a really powerful feedback machine. When you get more points or break your time record you immediately know that you have improved.

Unsurprisingly, proper feedback is mentioned as one requirement of a flow activity. If we take all the requirements as inputs into a design process of an application and interface, what will come out? Will the outcome make it easier for users to get into flow state when working with that application? I'm thinking these could be among my key research questions. One challenge is to find a way to integrate flow activity requirements into a design process, and another one is to evaluate the results. In this entry I have been merely scratching the surface of things like goal forming, difficulty regulation and achievements. I will delve deeper into these subjects later on and see what interesting things I can dig out. In the meanwhile, I suggest everyone to check out this book. It really is important.

* The paper I'm referring to is "Applying Game Achievement Systems to Enhance User Experience in a Photo Sharing Service" (Markus Montola, Timo Nummenmaa, Andrés Lucero, Marion Boberg, Hannu Korhonen).

A Look at Games - Lumines Supernova

I've been back from vacation for a week now, so I guess it's time to get back to this blog as well. Last weekend I finally got a router and was able to share my internet connection to the PlayStation 3, granting me access to PSN store and some games I've been wanting to play. Some of these are quite relevant to my research, so you will be most likely seeing posts like this one in the future as well. So let's get this series started with Lumines Supernova.

I took initial interest to Lumines when I was doing research about games where audio plays a bigger role than usual. Lumines is a relatively simple block dropping game, where you as the player need to form squares of blocks with the same color. The playing field is constantly swept over by a moving beat line, and full squares are eliminated only when the line passes over them. The more squares you can eliminate in a single sweep, the more points you get. Sounds simple, right? In other words, it belongs to the category of highly addictive games like Tetris and Bejeweled. But Lumines might be even more successful in this. Why?

Audio in the game is not simply just separated into background music and sound effects. Every skin (kind of a level) in the game has its own music and effects, which are in fact part of the background music. The music being played is a relatively simple loop but when things start happening on the screen, the player, through his block manipulating actions, becomes the composer. Rotating blocks creates small sounds, forming squares bring forth a little more dramatic effect, and of course the most notable sounds are generated when the beat line erases complete squares. On the paper it might not sound very impressive, but the gaming experience is from an entirely another dimension.

The game pace changes over time and the blocks start dropping faster and faster. In the beginning everything is quite relaxed, but sooner or later things start to happen so fast that complete planning becomes impossible and towards the end it's all just hectic panic. In the beginning, when the game itself is relaxing, the player can pay more attention to the complete audiovisual experience that is Lumines, enjoying the results of his actions portrayed in the background music. This occupies the player right off the bat. Of course, towards the end, the brain becomes so occupied with dropping blocks that no attention is paid to the music. For me it gets hazy after the fifth or sixth skin, and I have no recollection of what kind of music is played from that point on. Another neat thing is that when the game becomes more hectic, things start to automatically happen more quickly - because those blocks are dropping fast - and the music becomes increasingly more intense.

And now the interesting question: what can we learn from it? It would be interesting to try out something similar in user interfaces. Play background music and tie part of it to user actions. Produce small sounds when typing letters in a word processor, complete with a longer sound when a full line is written. Or a paragraph. Whatever the implementation, the key idea would be to use this kind of music generation to increase motivation. I think it would be most suitable for an application domain where user actions are relatively simple and doing them quickly is possible and desirable. In a way, something like this would make the rhythm and flow of work quite concrete.

While waiting for someone to come up with a working prototype, I recommend you to try Lumines (any version) or another game that uses the same kind of background music system such as Rez or Chime (neither of which I have been able to play yet, but will in the future as soon as I get an Xbox 360).

Vacation & Looking Back

I'm going off on vacation for about two and half weeks. Most likely there will be no updates until I'm back. Unless I feel some strange urge. Anyway, now's a good time to recap what's been going on since I started.

A lot of progress has been done in the reading department. 8 trade books on usability or game design related topics, and I'm now going through Rules of Play (Salen & Zimmerman) I've also been digging through any articles I was able to find about combining games and HCI. There were some, but not that many. So definitely there is still work to be done in this area.

In this blog, I've been outlining things about playful usability, and bits and pieces about interactive spaces as well. The ideas are at this point still all over the place and I haven't landed on an actual topic for my doctoral thesis. But I have time, and I think I should explore a bit of everything just to get a feel of things. I do have a much stronger idea now than I did three months ago, so clearly there's been some progress.

Experimental interfaces are raising their heads. After vacation, I'll try to find some time to work with prototypes using various kinds of controllers. I'm not going to unveil which ones at this point, but I'll be sure to make some noise here once I get to work. At the moment though, these are mostly something I have to do on my own time, and on my own computer at home which is a bit lacking. Probably I'll need to get a new one sooner than later.

At my actual job we're going to finish our prototype in August, and during Autumn there will be user tests and paper writing. And of course, there's the XNA programming course looming ahead, which I'm really looking forward to. If that experiment goes well, we can try and introduce other game related courses. And I do like teaching as well, although I don't have that much experience, yet.

So that's it for the recap. More posts will follow after vacation.

The Emotionality of Fuzzy Things

As you might be able to guess from the title, I've also been reading Emotional Design (Don Norman). Like with the other book, this was my second read-through. What I realized while reading this time around, is that this particular book is perhaps the most essential one for my research. Indeed, most of the things I have been writing about can be derived from what Norman presents in these pages.

Playful usability definitely builds upon the emotional side of human thought. It doesn't even pretend to be functionally optimal. Highly functional interfaces are effective. They take the shortest route. Playful interfaces on the other hand are more creative. Or, using the travel metaphor, they take the scenic route. Or maybe the route that is just more entertaining to drive. Really straight, long roads are effective. They are also really boring to drive, which is why I usually don't pick them. So today, I'll take a look at playful interfaces, through the lenses of the three levels of human thought presented in the book.

I guess before that, a really quick recap of the said levels is in order. For a more complete description, read the book. The three levels of human thought are visceral, behavioral and reflective. The first two are sub-conscious while the last one is conscious thought. Visceral level is the most basic input-output system built into us. It excels at giving out immediate responses. The behavioral level is where activity is performed. It handles any activity that doesn't require conscious thought to perform. While I'm writing this, the movements of my hands on the keyboard are controlled by the behavioral level, leaving my consciousness free to think ahead what I'll write next. And of course, that is what I'm doing on the reflective level. Here I am making decisions how to express myself and my ideas through this text.

Playful interfaces are by nature likely to be more novel than highly functional ones. Sure, functional interfaces can be decorated, but the methods of interaction follow convention. Playful or fuzzy interfaces can surprise the user with more than just visual decorations. At this point, we are clearly working on the visceral level. And I think it's really important to be affective here. It's the wow-effect, "hey, I wanna try that", that should be achieved. Applications and services need to advertise themselves, especially in interactive spaces where high personalization is not affordable (i.e. same services are generally offered for everyone). Often one glimpse at the UI is all you're ever going to get from your potential users, so that glimpse better bring them over. This kind of appeal is especially important to make people aware of services they didn't think they'd need.

As stated, playful interfaces are not even trying to compete in raw functionality. The argument goes: effectiveness can be achieved through enjoyable use, even if the actions themselves are slower. People are not machines, we get bored with uninteresting tasks and our minds wander away. This should not be big news to anyone, hopefully. Take a look at games. Games rarely provide the most effective means to reach a high level goal, which does indeed make them enjoyable. So, in the behavioral level, the aim for playful usability is to make high level goals more enjoyable to work towards. The means towards this end is to make individual tasks not necessarily optimized for speed, but optimized for fun instead.

Of course, making work-related tasks fun while retaining their original purpose is a hard one to tackle. If it was easy, we would most likely have it already. It requires a lot of lateral thinking. Especially in the domain of desktop applications, user interfaces are inside that same old ugly box. I've said this before, but it's worth repeating: that box needs to be tucked away for good. Thinking outside that box is not enough. We must find a new box altogether and then think outside that box. Or, preferably, a lot of boxes solely for the purpose of thinking outside them. Or, well, you get the idea. Just to put things in context, I have picked up the box labeled "game interfaces" and I'm trying to think outside that box.

I'm actually a bit of an optimist in the sense that I firmly believe that for playful interfaces, visceral appeal might be achieved as a side product of good behavioral appeal. At least I'd like to think that most things that are fun to do, also look so. Sure, I can come up with counter examples. However, at least in the beginning novelty is the other factor. Interfaces that are fun to use will most likely look very different from what people are used to, leading into curiosity. There lies a risk though. Control by gestures or speech for example can feel weird to some people. Especially if they have to do it in public. Talking on the phone is easy to accept. Talking to the phone is not.

Finally, we have the reflective level. I'd consider this to be even more important than the other two, but also harder to write about. Certainly, a good user experience is already achieved with enjoyable execution of tasks. On the reflective level, we want users to look back at the experience with positive feelings. Also, we need the user to feel good while using the interface. If we jump back to games, it simply doesn't matter how good playability a game has if it doesn't match the style or theme of the game. Therefore, it is important that a playful interface contributes to the overall experience of performing a higher level goal. And of course, how using the interface sits with the user's self-image.

Taking emotions into the equation raises a whole lot of things to consider when designing playful interaction. User experience is not a simple matter. Even in pure entertainment applications such as games, no one can tell exactly what needs to be done to ensure that ultimate playing experience. Sure, reading a book or ten about game design helps. Reading books on related topics helps more. But in the end, it all comes down to understanding your audience. Even then, sometimes you just have to break the rules, shake people up and bewilder them completely.

Making an experimental interface is easy. Making a successful experimental interface is not. That's what keeps researches like myself ticking.

The Design of Fuzzy Things

Another book-related post. This time I'm going to approach the topic of playful usability via The Design of Everyday Things (Don Norman). I don't think I this book needs any more recommendations, so let's just go ahead with some thoughts of my own*.

I'd like to start with the three models. There's designer's model, which is how the interface is designed to work. On the other end, the user's model, is how the user thinks the interface works. And in the middle is the system image, responsible for communicating how the interface works. In other words, the user's model is dependent on the system model. In fuzzy usability, the system image is intentionally non-obvious. The idea is not to directly communicate the workings of the system but rather present an interface that raises curiosity and allows the user to discover the workings. By obscuring knowledge we make it more desirable to attain.

My hypothesis for the benefits is that interfaces constructed this way are easier to learn. In a typical desktop interface such as the Firefox web browser I'm currently looking at, the user is immediately presented with a lot of actions. Wizards are often used to guide beginners. The downside is that wizards are separate interfaces for doing specific tasks and do not teach the user to use the program's interface. In a purposefully obscured interface, it is easy to start by presenting the most basic of tasks and reveal advanced features when the user "levels up". The application can reveal the presence of features when they become relevant.

We can think of interfaces separated into zones. Anything the user is already familiar with is in the safe zone, and the message is clear "you know these things, it is safe to work with them". On the borders of the safe zone, there is the uncertain zone. The user has an idea of what is there, and it's kind of saying "you might feel uncomfortable here, but these things can make you more powerful". Finally, there is the great unknown, beyond the uncertain zone. When the application realizes that the user is doing something that could be done better with an advanced feature, it can assign quests, sending the user to explore a particular uncertain zone or even an unknown zone. If the user wishes to find a particular feature, the help system can provide directions of how to get to that feature.

Unsurprisingly, this bears high similarity with playing openworld or sandbox games like Grand Theft Auto. The player can explore quite freely, discovering new areas progressively while learning new skills. But when the player wishes to get on with the story, there are certain things that are required, and hence the game sends the player on quests to meet these requirements. Add rewards for mastering new skills. One further way to think of this comparison is that of task. In a game, the task is to complete the story. In an application, the task is to complete some external goal. Typically the difference between these two tasks is their definer: in a game, the task is defined by its designer while in an application the task is defined by the user. The point remains the same though: why shouldn't the user/player have fun while learning the skills required to complete the task?

In terms of affordances, safe zones afford safe using of features with predictable outcomes; uncertain zones afford experimentation with features; unknown zones afford exploration. Restrictions can also be used by requiring certain features to be in the safe zone before the user is allowed to use particular advanced features. This will ensure that the user has grasped some important basic concept before trying to use a feature that heavily builds upon that concept. The restriction also comes quite naturally - the user needs to traverse through the prerequired features on his way towards the advanced feature.

Implementing an interface that conforms to this model should be an interesting task. Some key considerations in functionality are to ensure that 1) the safe zone really is safe and 2) the uncertain zone clearly supports experimentation. Predictability is key in the first consideration, while proper undo support is key in the second. Another implementation question is analysing user actions in a way that the application can decide when it is appropriate to advertise a feature.

* Similar thoughts can be encountered in several HCI articles, and I'll return to them later. Difficulty regulation is a good key word if you're looking for one.

The Future of Computing

Lately I've been traveling quite a bit, so I've had time to read some books. This time around, I'll discuss three of them: The Design of Future Things (Don Norman), Leonardo's Laptop (Ben Schneiderman) and Everyware (Adam Greenfield). I've chosen these three for discussion because in many ways they are about the same thing: the future of computing. Norman's book of course touches a whole lot of other matters about future things, whereas Greenfield's book is about ubiquitous computing specifically, but all three books send the same message. It's time for users to step up.

The machines are about to take over. Not like in the movies of course, but subtly. We are about to become servants to our machines. Clearly, it should be the other way around but without enough thought given to human-centered design, we might end up with machines that think they know us and what's best for us, restricting our actions when they see fit. Machines, that demand our attention and maintenance. Networks that share everything about us without our consent. When computing will be everywhere, using it is no longer a matter of choice. And if the interfaces suck? We are screwed, and we have to learn them. These are some of the scenarios discussed in these books.

If, for some reason, this does not scare you, then think about what user-centered design for future computing could give us. A lot of possible scenarios for the future are presented in Leonardo's Laptop and they all share one central theme: computing should support, not replace or automate, human tasks. Technology should be making our lives easier, not harder. Computing should be available, not integrated. And, contrary to some beliefs, it should never be entirely seamless or invisible. Which computing do you want?

Jumping to another topic, the idea behind the concept of interactive spaces is in many ways similar to ideas described in these books - especially The Design of Future Things. Humans and machines do not talk to each other - not today, probably not in the future - they signal. When communication is limited to signaling, it is better for one participant be completely predictable. In this case of course, it should be the machine. The user knows what he wants to do - there's no need for the machine to try and guess, because no matter how much sensor information and whatnot it has, it cannot read our thoughts (hopefully).

Norman presents a good example from human-to-human communication where this also holds true. Here's another one: sometimes when walking, you find yourself walking on a collision course with another walker and when you avoid - yup, the other guy avoids to the same direction, and probably then the both of you avoid again to the same direction. This is a scenario where neither participant is predictable. Indeed, if you had just kept walking straight, the other guy would've avoided you. This scenario becomes a lot more dangerous when bicycles or cars are involved. It's happened to me with bicycles, and if I hadn't landed on my butt, I might not be writing this.

To wrap up, here are some of the principles of interactive spaces:
- Interfaces are available to the user - not seamless but gracefully seamful.
- Interaction is initiated by the human.
- Interfaces are precitable - they don't make decisions on their own.

Out of these books, I would highly recommend Everyware for anyone even remotely interested in the future of computing, and especially defending human rights in the coming* age. The other two books are good reading for designers, and especially Norman's book is once again really easy to read and even fun. All of these books made me more confident that what I and we are doing is relevant and important.

* In fact, we are partly there. The latest of these books (Everyware) was printed in 2006, and already it was said ubiquitous technology is in part already here.

Foundations of Digital Games 2010

Another busy week behind. Exactly a week ago I was on my way to California, and more precisely the Foundations of Digital Games 2010 conference in Monterey. Here comes the aftermath.

Considering my own research, FDG2010 was mostly useful for the game design point of view but not that much actually. There were some interesting presentations and conversations with people about experimental game control methods, and finally some proper information about Kineckt. However, considering teaching game-related anything or game-aided education, the conference was much more useful for me. The details about the conference will be available later somewhere in the internet, so I'll just post some collected thoughts.

There was a plethora of sessions about inspiring more interest for computer science by using games and game design as part of CS education as well as to make potential students familiar with game development before college. I'm not exactly sure as to what the situation is here in Finland but I guess we can always use more interest. Probably the single most interesting session was about Microsoft Kodu, which is a visual game programming language, easy enough for children to learn. In the II City project, we are interested in using focus groups to create ideas for games, so you can probably see where this might be going. Yes, we can use Kodu to familiarize non-programmers with game dev.

Another thing I picked up were a lot of ideas for teaching a more serious game development course for post-grad students. This topic was presented in several sessions, and also discussed randomly throughout the conference. I'm mostly interested in doing this as a project-based course, and if possible, as a cooperation between our university and the department of industrial design in the University of Lapland. Bringing in artists from another university can get around the problem that CS students are not expected to have artistic abilities (and therefore we can't require them). This was how I had thought to do it to begin with, but discussions in the conference made me more confident. We can do this.

It was also nice to see a good glimpse of all the work being done in educational games, the Holy Grail of e-learning. There were quite a lot of projects presented that taught students computer science concepts in ways more interesting than plain programming practice. Using these kinds of games for teaching the basics of programming is an idea worth contemplation, especially if people are having a hard time grasping some of the concepts in our introductory course (I haven't asked yet, but in August I'll probably see for myself anyway). Of course, many a presented game taught other concepts than CS, such as how cats see the world.

Finally, game design was definitely not a forgotten topic. However, since there were always two sessions in parallel, I had to make choices, and because of my job, I ended up selecting all the education related sessions. So, like other people, I will have to check the interesting game design presentations when the presentation videos become available.

Jetlag and all that jazz aside, it was a good experience for a first conference trip. Although I definitely didn't expect that it would actually be cold in California. The wind was a real killer, especially on Saturday. In addition to huge amount of inspiration, I got to meet a lot of awesome people. Hopefully I'll be able to visit FDG2011 if it gets organized, wherever it might be. Oh, and the conference center had some serious Norman Doors - I took one pic, which I'll try to post later when I get it on my computer.

(During the trip I also had a lot of time to read books, so book reading report is up next)

Moving to Education - First Steps

Lately I've been working on course material, including sample code and lecture slides. I will be teaching my first course in august, and the topic is Introduction to Programming Through Game Development. It's a trial, an intensive course with the goal to teach students basics of programming and basics of game programming. We're hoping to inspire greater motivation for students to learn programming.

In my research, I'm taking part in rethinking usability. In education, I feel I should take part in rethinking teaching. I haven't fully thought out my teaching plan yet, but I will pursue active learning by having the students work on their own (simple) game projects throughout the course, instead of artificial exercise assignments. Of course, I'll have to do some lecturing since it's a basics course, and I'm specifically required to teach them the topics covered in our standard Introduction to Programming course. My initial inspiration for how I might do this comes from Ian Schreiber's blog Teaching Game Design, and for more inspiration, I think I'll revisit What Video Games Have to Teach Us About Learning and Literacy (by Paul James Gee).

I'm trying to avoid traditional lecture material. Instead of factual slides, I've made a relatively small amount, using mostly pictures that are fun and reflect the topic I'm talking about. More specifically, I'm using a lot of lolcats - we all have our personal quirks, so get over it. Slides aside, I have created an example project and taken snapshots from its source code throughout all phases of development so I can throw my students some practical examples to explore and play around with. I pretty much think that's all the material I'll need to pull this off (of course, we have a textbook as well, so students can look there if they prefer more reading).

My thoughts for this one are not exactly radical but it's a start. If the course goes well, we can probably try some more advanced courses in game programming, and then I can scrap the lecturing part altogether. Programming is fun. Game programming more so. Teaching game programming should also be fun. Looks like the course will be Introduction to Programming, Game Programming and Obscure Internet Humor. All in the name of science and creativity.

An interesting sidenote: Using pictures that are far from obvious is (sometimes) better than using ones that explain a thing really well. When I was leaning Japanese Kanji, the ones I found easiest to remember were the ones where I had to think for myself how to interpret the character. Humans are proud of achievements - if we make sense of something non-obvious, we want to remember our explanation and spread it to others. So maybe if students understand how this picture reflects doing operations to elements of arrays, the whole concept of arrays sticks better.

Irrationality and What Should We Do with it

Time to discuss some stuff I've read from books recently. The first book is Dan Ariely's Predictably Irrational. It's actually a book for behavioral economics, but can tell a lot about how we humans are wired for anyone in any field. Because it's also quite fun, not too long and well-written, I can recommend it to anyone even remotely interested in human behavior.

In the book, Ariely discusses a lot of different types of circumstances where humans repeatedly fail in rational thinking. Hence the name. A lot of the stuff is actually really familiar for most of us, because we've been down those roads. The book does a good job of pointing out the circumstances where people make bad decisions. If people became more aware of these things, maybe they could make better decisions in the future. Who knows, right? For someone in the HCI field, and game design as well, every tidbit of knowledge about human behavior is valuable. Here are some thoughts I gathered while reading:

Actually my first thoughts were whether we could actually use this kind of irrational behavior to purposefully "deceive" the users. For game design, it's also helpful to know how people form their decisions in different circumstances. This way players could be guided implicitly instead of explicitly when the designer would really like the them to pick a certain option. Most players would likely end up picking the intended option, but they would still think they are firmly on the driver's seat. Actually I think it would be an interesting for game researches to analyze how decision making situations are staged in games (although they may have also done this kind of research already).

And what about interfaces? It's kind of a harder case. Here the designer doesn't want to guide the user into doing something. The idea is for users to get what they want, which is something we really cannot predict. I don't have any particular ideas yet, but I'd like to explore the concept of purposeful deception in usability to produce outcomes that make for a better user experience. Maybe it's a dead end but you never know before you try.

Then again, should we exploit irrationality? After all, it causes people to make bad decisions that can be really damaging to themselves in the long run. Perhaps it would actually be better to make games and applications that highlight our irrationality and make us aware of how it affects us even in situations we consider "too important to fail". Some games do require players to abandon certain types of irrational behavior. A lot of board games for example encourage players to make hard decisions instead of keeping all the options open. In order to win, players need to commit to a strategy sooner or later. It promotes simple folklore wisdom; "if you run after two hares you will catch neither".

This I think is another interesting topic where games could be used for teaching people, more effectively than most learning methods. After all, games are good for experimentation, because players don't lose anything permanently (in most games anyway). Of course therein lies a problem as well: we think differently when we know there is nothing to lose. So even if we learn to avoid our irrational behavior in a virtual world, does this wisdom transfer into real life, where losses are also real? It might be a tough challenge to come up with something that produces real benefit, but something that should be considered.

We are already coming up with games and applications that encourage people to exercise and look after their health etc. Should improving our thinking be the next step? After all, it is our thinking that's the root of everything else.

(I've also re-read Donald Norman's The Design of Everyday Things but it's something I'll get back to later.)

1st International UBI Summer School - The Aftermath

I haven't been writing anything for a while, and here comes the explanation. I was attending to the 1st Internation UBI Summer School, held here in Oulu by the UBI project (UrBan Interactions). Basically it took all of my time last week: seminar and workshops every day between 9:00 and 16:00, and social events in the evenings. Of course it was totally worth it - there were lots of interesting people from all around the globe, and the workshop I was in was cool and really relevant to my post-graduate studies.

I was in a workshop called Urban Social Networks Analysis, instructed by professor Vassilis Kostakos from the University of Madeira. Our task during three days was to gather bluetooth data from the city of Oulu, by scanning for bluetooth devices in various locations, and to use that data to analyze how people (carrying the devices) were moving around the city. My workgroup was trying to figure out groups from the data, and some flocking behavior. Sadly it turned out that two and half hours worth of scanning by five groups was not enough for finding any groups of devices that would appear together in more than one location, so we had to to just do some group analysis.

It also turned out that analyzing groups computationally was far from trivial, and turned out to be quite a challenging task to do in about two days of programming. More work needs to be done on the algorithm later on. But anyway, the really useful stuff I took home from the workshop was how can we utilize simple data that has just time stamp, location and identifier to derive quite impressive representations of how people move around. Figuring out this kind of information can be seen as essential when seeking to understand the flow of people in a physical space. It answers the question "where should we put interfaces?", which is a key question if thinking about the difference between typical HCI and HCI for interactive spaces.

On Friday we had the final presentations for all the workshops (six in total). I did have a lot of difficulty choosing the workshop to attend to beforehand, and after seeing the presentations I really wished I could've been in all of the workshops (except one, which was very technically oriented). The other topics included "Real World Context-Aware Applications", "Urban Informatics and Sustainable Cities", "Creating and Sharing Artistic Experiences with Ubiquitous Technology" and "Interactive Textures - Rethinking Materiality". Let's just say that the results for each workshop were pretty amazing, especially considering they were done in just three days.

If any of the guys I met are reading this, thanks again for being there - great times were had! For those who weren't there, I'll try to post some links later if they release any interesting aftermath material about the summer school.

Food for Thought: Challenge-Reward Dynamics

Every once in a while I have the urge to complete all sidequests in a game I'm playing. Usually this urge happens with Japanese RPGs. But that's not the point of this post. What I would actually like to discuss today, is this topic: can we apply the challenge-reward dynamics present in games to user experience of non-game applications.

To recap briefly, challenge-reward dynamics are one of the reasons games are addictive to play (but definitely not the only reason). Obviously this should not be big news. A typical game provides challenges to the player, and rewards success. Typically the reward is provided in some form of in-game goods or information, although modern games also offer external rewards such as trophies and some digital goods (desktop themes, whatever). Most of the time, the concrete reward is tied to a sense of accomplishment, that familiar pride of overcoming a difficult challenge. Sometimes there are challenges in games that have no concrete reward, but the challenge itself is interesting enough and the sense of achievement is the only reward that's needed.

Applications on the other hand do not usually provide any internal challenges or rewards. When we think about programs like word processors, the challenge is definitely external (well, aside from the challenge of learning to use the application, which by the way is usually not very fun or rewarding). Applications are like tools. You don't want that hammer to provide any internal challenge or reward - you just want to build your house with it. Obviously applying challenge-reward dynamics to tools is probably a bad idea, but who knows? But not all applications are tools these days. Entertainment applications stand somewhere between tools and games, and are a field where ease of use might not be the obvious way to go. This is the application domain I'm primarily interested in.

While usability design principles generally agree that applications should communicate their system model to the user clearly, I think this particular principle should be challenged in entertainment applications. The value of an entertainment (or edutainment for that matter) application is mostly measured by how long can it engage its user. This roughly translates to how long the application can throw something different at the user. Games are definitely champions at engaging players; my personal record for a single player game is over 300 hours, but for online games this is more like the norm! On the other end of the spectrum, let's think of something like a virtual museum.

The idea of a virtual museum sounds good on paper - it's a museum anyone can visit from their home. But think about the concept for a while. Is there any real motivation to keep engaged to the virtual museum experience, which I assume can only be a fraction of a real museum experience. Why just simulate walking in a museum, when a virtual world allows many more options that are relatively cheap to implement. If we introduced challenge-reward dynamics, maybe part of the experience could be actually finding the attractions. The point being, if we want the users to get interested in paintings or whatever the museum is displaying, you might as well make them rewards that are challenging to get in some way. Modeling the virtual world after the real museum should not be the point.

People value things they have attained through some effort. This is something to keep in mind while thinking about entertainment and especially edutainment applications. Exploration and challenge are important keywords for this context, and are not (necessarily) in line with the user model / system model principle. Sometimes hiding meanings can make them more meaningful to the person who discovers them.

Interactive vs. Intelligent

At the very beginning I mentioned I will be working with interactive spaces, and briefly noted that they are basically in the same ballpark as intelligent spaces, but have a different emphasis. Now I'll proceed to open up that topic a bit more.

Judging by what I've seen, the general goal of intelligent spaces is that the *system* understands the user. Emotions and such can be detected from voice attributes, facial expressions and posture (among other things) using cameras and microphones (and possibly other kinds of sensors). The idea is that the computer will adapt its behavior to suit the current situation, automatically. It clearly acknowledges the user's presence, which is a good thing. This is, at least to my knowledge, a relatively technological approach, although there are of course challenges in understanding human behavior as well. This area has a lot of research going on, and I'd take a guess and say we can start seeing some impressive results in the near future.

If we skip all the technological problems related to intelligent spaces, I still think there's an additional problem: they might be seen as intrusive, because whatever the technology, the system needs various surveillance techniques to detect the user. Another thing is that people are different. Really, really different. For the system to understand any given user properly, personalization seems almost necessary.

The goal of interactive spaces on the other hand is that the *designer* understands the user. This is of course a very typical approach in usability design. We don't call it user-centered design for nothing after all (although the trend seems to be toward action-centered design, which is slightly different). An interactive space is a space containing many simple interfaces. The design consists of two primary layers: placement of interfaces in the environment, and design of individual interfaces. Each interface is placed in a relevant context. The key word here is interactive. Everything that happens, is directly dependent on how the user interacts with an interface. Basically, we like to keep the user in control of all the strings.

So instead of focusing on machine vision, voice analysis, artificial intelligence and so on, in interactive spaces design the focus is on designing interfaces that are highly context-aware by design, and are fun and interesting to use. The interfaces are not intrusive but rather blend into the environment, making themselves readily available when the user needs them.

In the end, like I said, we play in the same park. Be it intelligent or interactive spaces, both design and technology are necessary, as is putting the user first. But as you can see (at least I hope so) the emphasis is different. Maybe in the future we have just intelliactive spaces, combining the best of both worlds. That should be the goal, and to get there, the problem needs to be approached from both perspectives.

Of Word Processors and Context-Awareness

I hate word processors. In my opinion, they are an excellent example of something, that is really natural and easy for humans to do with pen and paper, done so hard that it baffles me every single time I need to do something I previously didn't need. Or indeed, do something I do rarely. Sure, the people who've been using these unnatural abominations for years and years know how to operate them, but if the casual user wants to do something as radical as putting an image somewhere in the document and wrap text around it they can be in for a world of pain. Or, let's say I want page numbers, starting from the very first page but not displaying them for the first ten pages.

It's all in the help of course, but knowing where to look is not always obvious. Open up your word processor's help and look at the size of that thing for a quick reference of what I'm talking about. This is basically understandable, as word processors have (and I guess they need to have) a plethora of features. A lot of these features are stuff needed only by selected few of us - each user group needs only a fraction of the entire feature set. The problem is, with this many features, finding the one you're looking for is difficult. Everything is organized, but before you can make any use of that, understanding the categorization logic is necessary. Simply put, they are not context-aware.

What games (good ones anyway) do effectively and word processors do not, is providing information on how to use them when you actually need that information. Sure, Microsoft tried with the Paper Clip of Mighty Annoyance, and that didn't turn out well, but is also no reason to stop trying. One problem of course is that it's quite hard for the application to understand what the user is doing, due to its monolithic design. Still, the application should at least provide a help shortcut on each and every one of its dialog windows, to open up a list of help topic relevant to that particular dialog. Help should also be action-oriented. When I want those page numbers, I don't want to know what each and every gadget on the dialog goes, I just want to know how to get my page numbers exactly where I want them.

Of course, games have the definite advantage of much better understanding about what the player is about to do, due to their predetermined nature. Games with linear progression are obvious, but also sandbox-games where the player can go where he pleases can still usually have some idea what the player might need to know next. But it's not like we can't make word processors context-aware. Perhaps it'd be best to actually break down their monolithic design, and try an object oriented approach, something programmers should be very familiar with, but which strangely doesn't find its way to user interfaces too often. I think it's better to provide an example scenario.

I start writing a document, and at the moment the word processor only provides the very basic features for writing text and positioning it intelligently. This is all the word processor itself does: manages positioning and text input. Then I realize I want a picture, so I paste in a picture component. When I click on this component, I only see tools for picture management, which is again something I don't need to see when I'm not setting properties for my picture. I write again for a while, until I need to paste in a table component. Lo and behold, there's a table, and when I select it, I have all the table-editing tools at my fingertip, and nothing else. Finally, when I'm done I realize I want to alter the positioning. So I summon up a grid, tell my components and text paragraphs to snap to it (to keep them neatly lined) and drag them around until I'm happy with the results.

See? Context-aware word processor. I actually quite recently found out that Word 2007 (been using Open Office in Linux) does a lot of this, so perhaps there is still hope for word processors as well. The last part of my example scenario on the other hand is something I can do in Excel or similar programs, but not in word processors. There could be a very good reason for this, but it's also a real possibility that there isn't, because software generally tends to follow old conventions. Of course, it is nice that all the new software works somewhat like the one's we've been using before, but if no one breaks this cycle, we're stuck with usability models that have been outdated for a long time, and bringing new people, with little experience using computers, becomes harder and harder as time goes by and feature numbers creep up.

So, perhaps it is indeed time to rethink everything from scratch.

Games are Complex - A Case in Point

The purpose of this post is to just take a moment and consider how much complexity can there be in a game that looks quite simple from the outside. Of course I could always take a game that looks and is complex, like a simulator or heavy strategy game, but for those you can really tell their complexity by just looking at them (or their manuals). Besides, I have relatively little experience with them, especially recently. Instead, I'll tackle something I'm quite familiar with, and something that might surprise at least some readers. Let's just call it the Monday morning shock effect.

I'm going to talk about fighting games, a genre which includes quite popular titles such as Street Fighter, Tekken and Soul Calibur. On the outside, these games look relatively simple and straightforward. Each player controls a character on the screen, moving and performing attacks using their controllers and trying to deplete the other player's life bar before losing their own. But let's take a look under the hood. Just poking around inside the game (using Tekken 6 as an example) we can see that each character has in fact more than forty different entries on their command list. These are the button combinations the player needs to press to perform a particular move or canned series of moves. Before you can even consider becoming a good player with one character, you need to know these combinations.

This is hardly mastery though. Moves in fighting games have lots of properties that are not listed in the game. One of the things serious players quickly need to become familiar with is frame data. First of all, frame data tells how many frames (1 / 60th of a second) a particular move needs to come out. Second, it tells frame advantage or disadvantage in three situations: the move is blocked by the opponent, the move hits the opponent or the move hits the opponent who is also performing a move (think of it is interrupting). Just so you have the basic idea, if I do a move that puts me at a disadvantage of 12 frames if it's blocked, then I cannot do anything during those 12 frames, meaning you can get a free hit with a move that comes out in 12 frames or less. So that's four additional numbers for each of the 40+ moves (in reality, you don't need to know the exact numbers for all moves) to learn.

Of course, all this stuff is learned not by reading and memorizing, but by reading, applying in practice, then reading some more and applying, until all the relevant information sticks. Oh, and in addition, you also need to figure out how far a given move reaches and is it circular or semi-circular (i.e. can the opponent avoid it by moving sideways). For argument's sake, let's assume you have learned all this stuff for one character. Then what? Well, there are 39 other characters in the game, and if you play competitively, you can run into any one of them. And yes, in order to be a competitive player, you need to know their moves, frame data and all, as well. Granted, you don't need to know everything about a character to fight against him effectively, but knowing at least the most important moves really helps.

The best way to do this is to first play against someone, then read, then play again, and read some more, until you get the hang of it. This can take a lot of hours for just one character, and the only way to do it is to play against a human opponent. These days you can play online, but as a reminder, when talking about games where fractions of seconds are of importance, even small amount of network lag can make a big difference in how the game is played. So in order to really play the game, traveling becomes a necessity. All this for what gain? Well, most of us just gain the thrill of competition out of it. Of course if you are really, really good you can even make a living, at least if you live in South Korea, the world capital of eSports.

To summarize, to fully understand the system in Tekken 6 (yes, just one game), you need to know the properties of about 20 moves or more for each of the 40 characters - that's around 800 datasets to learn - and of course you still need to figure out how to make use of all this information. Oh yeah, when they update the game to the next version, this information changes, so you need to keep up. So, how hard did you say your college math course was again? Naturally not that many players want to achieve this level of mastery - they just want to mash some buttons and never understand what's going on in the game. Then again, some people just memorize the facts the night before a test, to forget everything in the next week.

This kind of complexity and infinite learning curve is in fact pretty typical for really competitive multiplayer games (for a classic example, think Chess). Their learning curve is in fact pretty exemplary: starting to play is easy, and new things are learned when you need them - when you lose to someone, you need to either figure out new tricks for yourself, or research their tricks and learn some counter techniques (best do both!). Finally, playing competitive games like this is a strongly social activity. Once I have a hard copy of What Video Games Have to Teach Us About Learning and Literacy (I've read it via ebrary, which I must say is a good example of horrible usability) I might run an even more through analysis of how the learning principles in that book match with competitive playing of fighting games.