A Jacaard two-way agreement score is a measure of the similarity or differences within a set of data. It ranges from 0 to 1. If all the items within a dataset differ, the score will be a 0. If all the items within a dataset are exactly the same, the score will be a 1. Thus, higher scores indicated greater similarity of items in the dataset, and thus higher agreement.

Table 1. Agreement scores for novices and experts

As can be seen in this dataset, Novices and Experts had very similar agreement scores, with a few exceptions (outlined in blue).

We also conducted a visual inspection of the data itself . We looked at the detailed results for both Novices and Experts, as shown in Data Analysis (Part 2: Winners and Tails), and looked for differences between the two groups that were explainable based on experience. We found no strong pattern to any of the differences we saw that was explainable based on experience (aside from the pattern already noted above for Zoom In and Zoom Out).

As mentioned in Data Analysis (Part 3: Cultural Differences), however, we investigated the cause of the seemingly cultural influence on Scroll Up, Scroll Down, Back, and Forward. These actions are directional in nature (people tended to view the Back Action as retrieving a page to the left of the visible page and the Forward action as retrieving a page to the right of the visible page). In each of these, roughly half the participants scrolled one way while the other half scrolled the other way. In looking at this data a little more closely, it shows France and Spain tending to go one direction with many of the other countries going the other direction. At first, we thought this may be a cultural difference. However, when looking at the data closely, we discovered it was actually an experiential difference.

Swiping up to scroll down is a behavior found in most touchscreen devices that do not show scroll bars and that are “finger-driven” touchscreen interfaces. Swiping down to scroll down is a behavior found in most non-touchscreen devices (those that use navigation keys) and in touchscreen interfaces that do show a scroll bar, most commonly found on “stylus-driven” touchscreen interfaces. We hypothesized that experience with finger-driven interfaces may provide a better explanation of the differences that we saw in these actions

So, we classified every user’s phone as a “finger-driven” touchscreen phone, a “stylus-driven” touchscreen phone, or not (“none”) a touchscreen phone at all (made up of the Novices in this study). We then looked at the number of people who own a finger-drive touchscreen phone and the direction that they scrolled, and the number of people who own a stylus-driven touchscreen phone or didn’t own a touchscreen phone at all and the direction that they scrolled. In the end, we found that those who own a finger-driven touchscreen phone were far more likely, in general, to swipe up to Scroll Down, swipe down to Scroll Up, swipe left to go Forward, and swipe right to go Back. Thus, it seems what was once thought to be a cultural difference was in actuality an experiential difference.

Tables 1-4 below show the results of this analysis.

Table 1. Back

Table 2. Forward

Table 3. Scroll Down

Table 4. Scroll Up

The first analysis we ran is: did one country use certain gestures more than any other country? To answer this question, we analyzed the top 38 gestures (gestures that had more than 40 people use them) and ran a chi-squared analysis to identify if one country used that gesture more than another. There were no significant differences found in that analysis. Thus, this analysis did not identify any significant differences between countries in their use of the top gestures.

The second analysis we ran was to assess whether one country used certain types of gestures more than another. We classified all the gestures into 2 categories:

1. Direct manipulation
2. Symbolic

A direct manipulation gesture is a gesture where the participant using gestures that relate at least loosely to the real world. So, to move an object, the participant touches the object and then physically drags it to a new location. Or, to scroll, the user will touch the screen and push it one direction or another.

A symbolic gesture is a gesture where the participant creates a symbol that conveys a desired action, such as drawing a question mark for help or making an “x” over an object to delete it.

We ran an Analysis of Variance (ANOVA) to evaluate the use of direct manipulation versus symbolic gestures. We ran an 8 x 2 x 2 ANOVA to analyze the differences shown by 8 Countries (China, Finland, France, Germany, Finland, India, Spain, UK, USA),  2 Genders (Male, Female), and 2 levels of Experience (Expert, Novice). This analysis showed the main effect of Country was significant (p <0.01), as was the interaction between Country and Gender (p < 0.05). The main effect of Country was very interesting and we looked at it in more detail. Subsequent analyses showed the following differences depicted in Figure 1.

Figure 1:

The third analysis we conducted was actually more of a visual inspection of the data itself. We looked at the detailed results from each country, as shown in Data Analysis (Part 2: Winners and Tails), and looked for differences between countries that were explainable based on culture. We found no pattern to any of the differences we saw that was explainable as based on culture (aside from the pattern already noted above that Chinese participants used more symbolic gestures than any other country). We did find a pattern that at first looked like a cultural difference (see the results for Scroll Up, Scroll Down, Back, and Forward), but in further analyses discovered that it was actually an experiential difference, which we will discuss next.

To find out the answer to this question, we first sorted the data from most frequent to least frequent. This resulted in a data table that shows the gestures created, the number of people in each country that created that gesture, the number of experts and novices who created that gesture, and the total number of participants overall who created that gesture, all sorted from most frequent to least frequent.

When looking at the data, we wanted to know, statistically, which gesture(s) were created the most often (which we termed the “winner”) and which gestures were created the least often (which we termed the “tail”).

To identify the statistical winners, we ran a chi square test and compared the top gesture to the next top gesture and if there was no statistical difference between them, we grouped them together in the winner group. We then compared the top gesture to the third-top gesture and if there was no statistical difference between them, we added the third-top gesture to the winner group. We continued in this way until we found a statistical difference and then we stopped. Figure 1 describes this process.

Figure 1. Chi-Square Testing

To identify the tail, we did the same thing only we used a Fisher’s exact test (due to the low number of observations) and compared the lowest gesture to the next lowest gesture. If there was no statistical difference, we grouped them together into the “tail.” We then compared the lowest gesture to the third-lowest gesture and again if there was no statistical difference, we added the third-lowest gesture to the tail. We continued in this way until we found a statistical difference and then we stopped. Figure 2 shows this process:

Figure 2. Fisher Exact Testing

Below are all the results for all 30 actions, with the winners at the top and tail at the bottom in gray. The top winners are highlighted in yellow.

[ Accept/Verify | Back | Close Folder | Continuous Scroll | Copy | Cut |Delete | Forward | Help | Home | Magnify | Minimize | Move | Multi-Select | Open Folder | Open Menu | Paste | Pitch | Print | Redo | Rotate| Save | Scroll Down | Scroll Up | Stop Scroll | Undo | Zoom In | Zoom Out | Place Focus | Select Text ]

Accept/Verify

Back

Close Folder

Continuous Scroll

Copy

Cut

Delete

Forward

Help

Home

Magnify

Minimize

Move

Multi-Select

Open Folder

Open Menu

Paste

Pitch

Print

Redo

Rotate

Save

Scroll Down

Scroll Up

Stop Scroll

Undo

Zoom In

Zoom Out

Place Focus

Select Text

The result of the data collection yielded a detailed spreadsheet of all unique gestures (Gesture Glossary) and a results spreadsheet of all the gestures each participant made for each action in every country.

In all, we had 40 participants in 8 countries and 20 participants in 1 country create gestures for 28 actions, yielding over 9,500 gestures in total. Each of these gestures were catalogued according to the lexicon defined by the research team.

As noted in Data Structure, the lexicon collected very detailed information about each gesture.  This detail will prove useful in many analyses. However, for certain types of data analyses, not all of the detail is needed. At times, the detail of the information resulted in gesture differences that were not meaningful. For example, when someone selects a word to cut or copy, it isn’t significant which direction they swipe to highlight that word. Another example: when someone is performing a page-level command (like scrolling), it doesn’t necessarily matter if they perform that gesture over an object or over whitespace. So, we undertook an effort to combine gestures when the differences between them are not meaningful for a particular action.

To combine gestures where the differences between them are not meaningful, we first defined a series of guideline definitions of what is meaningful and what is not. We then had 3 independent researchers apply those guidelines to the unique gestures in the glossary. We then reconciled the different results of those 3 people into a final combination set.

We defined 2 sets of guideline principles, based on the type of action. There are 2 types of actions: 1) those that require action on an individual object on the page (termed Object Level Actions) like delete and close folder, and 2) those that require an action on the page itself (termed Page Level Actions), like scroll and print.

For Object Level Actions, we decided to apply the following guidelines:

• Swipe direction is not meaningful, so combine all swipe actions regardless of direction
• Location of action is meaningful, so leave all detail regarding location
• Combine when the participant performs an action outside the scope of the task, and removing that action leaves an identical gesture to an existing gesture
• Combine repeated gestures into one group regardless of the number of times it was repeated (e.g. “tap”)
• Combine letter/word symbols into two categories: 1) Letter/word other than “X”, and 2) “X”

The actions classified as “Object Level Actions” are:

• Delete
• Open Menu
• Close Folder
• Open Folder
• Accept/Verify
• Paste
• Move
• Place Focus
• Multi Select
• Select Text

For Page Level Actions, we decided to apply the following guidelines:

• Swipe direction is meaningful, so leave all detail regarding direction of swipe
• Location of action is not meaningful, so combine all gestures where location was the only difference.
• Combine when the participant performs an action outside the scope of the task, and removing that action leaves an identical gesture to an existing gesture
• Combine repeated gestures into one group regardless of the number of times it was repeated (e.g. “tap”)
• Combine letter/word symbols into two categories: 1) Letter/word other than “X”, and 2) “X”

The actions classified as “Page Level Actions” are:

• Forward
• Undo
• Cut
• Copy
• Redo
• Back
• Home
• Save
• Print
• Pitch
• Rotate
• Scroll Down
• Scroll up
• Zoom Out
• Magnify
• Zoom In
• Continuous Scroll
• Help
• Minimize
• Stop Scroll

After the gestures were combined, we took one additional step. For five actions (cut, copy, paste, magnify, and open menu), there were actually two steps. For cut, copy, and magnify, the user first had to select text and then perform a command that would cut, copy or magnifiy. Some users combined the two steps into one gesture, which is fine, but many took two distinct steps. We decided the step of selecting the text (cut, copy, magnify) should be split from the command that performed the action, so we split them. We created a new gesture category, called “select text” and put all of the gestures intended to select text into that category, leaving only the gestures that were intended to perform the original action (cut, copy, magnify) in the action’s category.

For paste and open menu, the user first had to place focus on a particular spot on the screen and then give the command that would paste or open the menu. Again, some users combined the two steps into one gesture, but many took two distinct steps. We again decided to split the step of placing focus from the step of performing the command (paste and open menu) and moved the gestures intended to place focus into a “place focus” category, leaving only the gestures that were intended to perform the original action (paste, open menu) in the action’s category. These steps together resulted in a cleaner view of those 5 actions.

Our next step was to analyze the data.

It’s safe to say that Apple is loving their neo-minimalism right now. From the sleek Dieter Rams inspired iMac, all the way to the sleek Hara-esque iPod Shuffle, every bell and whistle on any object has been melted off to leave only the core functionality visible, removing all visual clutter and distraction. In fact, at first, the idea of having a multitouch almost non-tactile mouse may seem ludicrous, seeing as most of us use our mice without looking at them but we’d probably not be able to use an iPhone without looking at it. OK, unfair comparison, as a mouse has far fewer interactions than the esteemed smartphone. Nevertheless, mice have evolved a little, you might say, since they had little rubber balls in them (remember them?) so the selection of interactions to be performed blindly need to be well thought out.

Unpack it and set it up!

Let’s start from the top. The mouse ships in a very tight plastic container thats only marginally bigger than the mouse itself. Unboxing shots ahoy puh-leeease!

Looks lovely doesn't it? AND IT'S ALL OURS!

One thing that Apple does brilliantly is marketing. Remember how you knew how the iPhone worked before you got it? That’s because for one afternoon, Steve Jobs became a talking instruction manual and showed everything you needed to know for the phone a few months before it was released. Apple’s marketing is the instruction manual. And by having either intuitive interactions or a few key ones at least, they can shove them on as “features” on their site, adverts and the packaging… which brings me to….

Oh hai.....

All the instructions are at the underside of the packaging, which tells you pretty much everything you need to know.It’s safe to say that multitouch gestures seem to be on the card here. Let’s open it and get our hands dirty.

Take off the top....

Off the old plinth.....

... and compare it side by side with the old mouse, on the moon.

Comparing it to the old bluetooth mouse, it’s obvious that there’s been a lot of streamlining. The removal of most seams and joins is the obvious thing, but the nice metal chassis on the underside reveals a few hidden secrets. From a technical viewpoint, this is all bluetooth, which I’m not sure I’m happy about. Sure, there’s less cables, but there’s also that “OMGZ I’m up working super late and the battery has run out and I have no shop thats 24 hours near me” moments too.

Looking at the device, it’s apparent that there’s only one big button across the top, which is a little disorienting at first. Those who had Apple’s previous mouse, may miss the roller ball and the squeeze buttons at the side and that’s all you can do, because they’re GONE. For good. If you used the middle mouse press, like I did, for accessing Dashboard or opening new tabs in a browser, then sorry, that’s no longer possible. Likewise, if you used the side buttons to use Expose, then sorry, that’s gone as well. It’s super annoying. While I can understand the lack of side buttons, the middle button is such a standard, I can’t see why it couldn’t have been replicated so easily given the accuracy of the touchpad. What feels worse, is that some basic usability testing probably would have exposed this instantly.

Let’s have a look underneath for fun.

On the underside are two key features of the mouse. The first is the fact that we’re now using laser tracking over the traditional optical tracking. The other item is the power button and herein lies my first basic usability error – there is no label that this is the power switch. A simple 0/1 would suffice here. I noticed how to switch it on straight away, because I’ve been using Apple stuff before, but basic usability indicates that a child, or a man with the mind of a child, should be able to work out the entire device intuitively. In the previous mouse, the on/off switch acted as a guard to the optical sensor, which not only doubled it’s usefulness but made sense as well. In this, while the switch still shows green to indicate its on, it lacks the guard for the bulb that made it a hit in the first place. I might be nitpicking here, but it really seems a step back from what it was.

From a usability point of view, setting up is easy or slightly more difficult depending on your setup. If you have no bluetooth mouse connected, then the mouse will work when you switch it on. Your multi touch features won’t work instantly though and if you’re on the internet, your Software Update should spot you have the new mouse and mention you need to download the new software.

If you have a bluetooth mouse already, then you need to set up your new mouse through the Bluetooth Device Setup on System Settings in OSX and then you can safely disconnect your old one. Talking of OSX, I did notice that this wasn’t for Windows, so sorry for 90% of users out there in the dark. I think you can use it as a standard two button bluetooth mouse in BootCamp. With a bit of luck, Apple will release Windows drivers to allow the multitouch gestures. It’s a weird thing really, because hardware is one of those few things you can’t download off the internet, so you’d think that Apple would be kind enough just to release it for both platforms.

So what happens when you use it?

In use, the mouse is pleasant to use for tracking and moving around the page. The weird feeling initially comes from the recognition that the whole surface is a massive touchpad. However, it’s not simply a moveable touchpad; that would be ridiculously confusing. Instead, it has touchpad elements on a traditional mouse.

Look at it. LOOK AT IT!

Tracking is brilliant. Trying it on the carpet, jeans and a shiny surface proved to have no problems at all. Bizarrely, the only surface it didn’t work on was the iMac monitor on which I am typing this. But that’s fine really, because at that point, I was just trying to find a surface it didn’t work on and anyone who tries to use a mouse like that should not be allowed anywhere near a computer. Or society.

Onto multitouch. It’s a mixed bag really. On one hand, I want to praise Apple for incorporating these gestures that we may be familiar with from iPhones or the new Macbooks, but I also want to hit them for removing things I used to use. We’ve already talked about the lack of mousewheel/ball and that’s not something that’s too major. The overall feeling I get from using a single finger to scroll anywhere on the page is great and it reminds me of a trackpad. Of course, two finger scrolling (why doesn’t EVERY computer with a trackpad have this?) works as well. Trying to use the whole surface as a trackpad doesn’t work though (i.e. keeping the mouse static, but using a single digit to move the pointer around the screen). This may not seem like an obvious interaction, but could be an interesting implementation in the future.

Two finger swiping works well for going back and forth in Safari, but not much else really. It’s easy enough to do but ultimately I never felt that it saved much time. It’s somewhat annoying that all the customisation features that the Mighty Mouse used to have are missing. The ability to map all buttons to whatever you wanted is lost. I’d like swipe to act like Expose and clear windows out to my desktop, or swiping to the left to show all my windows, but I can’t.

The other annoyance that plagues mac users is that the right button isn’t activated by default. You have to go into System Preferences to switch it on. It just makes jealous PC users stuck in 1987 say “Mac mice don’t have two buttons.”

The mouse is satisfying to click and really tricks you into thinking that just the part you’ve pressed has clicked. This is a great implementation and I really think it works in practice. I even knew where the middle mouse scrolling wheel part would be without having to look at the mouse at any point. Bizarrely, multi touch gestures like pinch and zoom and single finger swiping are absent.

All my arguments at this stage could be remedied with software updates, but they are, as a released product, valid for now. Until Apple sort it out, power users who were used to interactions with their Magic Mice should wait for a bit.

One area that I can praise is the new Control Panel setup.

Now, that's a lovely picture.....

Moving over every single description of actions available brings up a beautiful little instruction video (silent) that shows exactly what each term means. It’s a really nice way of presenting things and clears up questions for the users.

Overall

It’s hard not to love the Magic Mouse, really. It’s simplicity of use and superb tracking/scrolling make it a winner. However, it’s lack of middle mouse button action, it’s fairly high price tag and apparent lack of Windows support will just fuel arguments against it and I really wish these had been rectified.

Scrolling feels great on the device and adds an interactive element to a mouse making it feel like you really are in control of the screen. It’s sleek, light and easy to move around and for this Apple should be praised. Just hurry up with that software update, OK?

To help maintain consistency across researchers, all unique gestures were catalogued and uploaded to an online gesture glossary. Tables 1 and 2 below show the data collected for each unique gesture in the gesture glossary.

Early researchers uploaded many unique gestures to the online gesture glossary, while later researchers were able to compare the gesture made by their participants to the gesture pictures in the glossary and re-use the same gesture ID and description.

All gestures were reviewed and entered into the results spreadsheet, the structure of which is shown in Table 3. Because most of the detail describing the gesture is contained in the online gesture glossary, the values entered into the results spreadsheet were limited to the minimum number of items to classify the gesture and enable the data analyses.

Table 1. Lexicon for describing unique gestures.

Table 2.  Lexicon values.

Table 3. Gesture results spreadsheet

Here are a few key instructions to all moderators who ran this study:

- The moderator needs to make sure that the participant performs an actual “gesture”! If participants say that they would want to tap a softkey for a particular action then we have to document this finding. But we would then inform the user that there is no such softkey and encourage them to find a gesture, with which they would trigger this action.

- The moderator should encourage the participant to use more than one finger if they think that is best for the action.

- It is important that the participants hold the rig inline with their line of sight. If participants hold the rig at an angle or twisted in one direction, the marks may not be accurate.

- If participants want to perform more than one gesture for a certain action, the moderator asks them to choose the most adequate one.

Action: Multiselect

“Please perform a gesture, with which you would select different items at once.”

Action: Move object

“Please perform a gesture, with which you would move an object (drag & drop).”

Action: Delete

“Please perform a gesture, with which you would delete a file.”

Action: Scroll down

“Please perform a gesture, with which you would scroll down to the next page.”

Action: Scroll up

“Please perform a gesture, with which you would scroll up to the previous page.”

Action: Continuous scroll

“Please perform a gesture, with which you would continuously scroll down a very long list of items.”

“For example, if you were looking at the first item in a list of 1,000 items, how would you scroll to the 500^th item without repeating a gesture?”

Action: Stop scroll

“Please perform a gesture, with which you would stop to continuously scroll a page.”

Action: Open menu

“Please perform a gesture, with which you would open up a menu; e.g. on the computer, you would be using the right click of your mouse and a context menu opens.”

Action: Open folder

“Please perform a gesture, with which you would open a folder to see what files are inside.”

Action: Close folder

“Please perform a gesture, with which you would close a folder.”

Action: Accept / Verify

“Please perform a gesture, with which you would accept /verify a certain action, e.g. you would like to accept that a text message can be deleted.”

Action: Home

“Please perform a gesture, with which you would return to your home screen.”

Action: Help

“Please perform a gesture which you would use to access help.”

Action: Zoom in

“Please perform a gesture, with which you would zoom into a map.”

Action: Zoom out

“Please perform a gesture, with which you would zoom out of a map.

Action: Magnify

“Please perform a gesture, with which you would magnify a text fragment.

Action: Rotate image

“Please perform a gesture, with which you would rotate an image.”

Action: Pitch

“Please perform a gesture, with which you would tilt the view forward and backward when looking at Google Maps.”

Action: Save

“Please perform a gesture, with which you would save changes that you have made on a screen.”

Action: Print

“Please perform a gesture, with which you would print a certain document.”

Action: Minimize

“Please perform a gesture, with which you would minimize an application.”

Action: Back

“Next we will be looking at things you might do in a Web Browser. Please perform a gesture, with which you would move back a page in your browser window.”

Action: Forward

“Please perform a gesture, with which you would move forward a page in your browser window.”

Action: Cut

“Please perform a gesture, with which you would cut out a text fragment”.

Action: Paste

“Please perform a gesture, with which you would paste a text fragment into an existing text”

Action: Undo

“Please perform a gesture, with which you would undo a certain action.”

Action: Redo

“Please perform a gesture, with which you would redo / repeat a certain action.”

Action: Copy

“Please perform a gesture, with which you would copy a text fragment.”

As mentioned in the Gesture Research – Protocol (Part 1: Equipment) blog, participants were asked to define their own gesture for 28 actions. The 28 actions were:

• Multiselect
• Move object
• Delete
• Scroll down
• Scroll up
• Continuous scroll
• Stop scroll
• Open menu
• Open folder
• Closer folder
• Accept/Verify
• Home
• Help
• Zoom in
• Zoom out
• Magnify
• Rotate Image
• Pitch
• Save
• Print
• Minimize
• Back
• Forward
• Cut
• Paste
• Undo
• Redo
• Copy

These actions were described pictorially by showing participants a “before” screen and an “after” screen.

Figure 1. The Before screen and After screen for the multiselect gesture

The “before” screen was placed in the gesture rig we created. Participants were asked to dip their fingers in charcoal and make the gesture that would result in the “after” screen. The charcoal left a trace that was recorded by the webcam

Figure 2. The gesture rig

40 people participated in this research from 8 different countries. 20 of the participants were experienced with multitouch interfaces, and 20 of the participants were not experienced with them. The 8 countries were:

• China
• Finland
• France
• Germany
• India
• Spain
• UK

In addition to 40 people from each of these countries, 20 people from Italy participated.

The study concept: Ask 40 people (20 experienced in using multitouch touchscreens, 20 inexperienced) to create a gesture that will elicit software to perform a particular action (e.g. scroll, rotate, multiselect, cut, copy, paste). Then, see what gestures people commonly make and if there is a difference across cultures in those common gestures.

The challenge: Ensure repeatability of the study protocol across 9 different teams.

The solution: A pictorial-based protocol where participants were shown a before screen and an after screen and asked to perform a gesture that would result in the after screen. This protocol relied very little on verbal communication for trying to describe an action where the mere description itself could influence the participant’s gesture and translation of that description across multiple languages would increase the chance of that occurring.

Figure 1. Example Before and After screen for Multiselect Action

The next challenge: How to record the gesture generated. Also, how can the gesture be made on the before picture so we can identify location-specific gestures? And, how can the gesture be recorded and viewed later?

The solution: We created a gesture rig where a webcam was mounted on a neck over a base where the gesture would be performed. The before screen was placed on the base itself. Every country used a gesture rig for recording the gestures made to ensure a consistent video output.

Figure 2. Gesture rig

The gesture rig consists of three primary components:

1. Base
2. Neck
3. Camera

Base

The base is designed to simulate a handheld touchscreen device. Each base held a printed image of the before screen and a protective layer of transparent plastic (i.e., Plexiglas). The Plexiglas fit securely within the base so gestures can be captured on its surface. Users performed the gestures on the Plexiglas after dipping their finger(s) in charcoal.  The charcoal left a trace of the gesture which could then be referenced in pictures and video in later analyses. The Plexiglas was then wiped clean after the gesture was made and a new before screen placed underneath it.

The base was constructed from sturdy lightweight plastic. The base can be held or placed on a surface to provide balance for the attached webcam or better simulate an electronic handheld device.

Figure 3. Base

Neck

The neck, which was made of 3mm thick aluminum, secured the Webcam to the base, managed the Webcam’s cable and maintained a consistent camera position to record the gestures. The neck can be adjusted by carefully bending it toward or away from the base to allow for alternate camera distances.

Figure 4. Neck

Camera

The camera used was any readily-available USB Webcam capable of recording video in 640/480 resolution at 10 frames per second or better.

Figure 5. Sample screen capture

A lot.

Some believe we have entered a new era in interaction design.  For many years we’ve been using the mouse, pointstick, touchpad, and of course, the keyboard.  A new interaction convention is gaining popularity now that makes use of the whole human body.

This is the first in a series of blogs about gesture research conducted by the International Usability Partners (IUP). As this research is conducted, analyses and conclusions will be posted here as they are completed. The first couple blogs will provide the background, protocol, and data collection methods. Future blogs will contain the analyses and results.

Touchscreen user interfaces are gaining popularity seemingly exponentially.  Many touchscreens are incorporating gestures, and they are rising just as rapidly.  Touchscreens are most popular on mobile phones, but many products are under development and on the market using touchscreens such as ovens, refrigerators, combines, laptops, automobiles, insulin pumps and many others.  Touchscreens are appearing on an increasing number of consumer products.  And the desire to enable gestures will follow.

Dan Saffer states in his book Designing Gestural Interfaces that “the next several years will be seminal years for interaction designers and engineers who will create the next generation of interaction design inputs, possibly defining them for decades to come.  We will design new ways of interaction with our devices, environment, and even each other. We have an opportunity that comes along only once in a generation, and we should seize it.”

Very few studies appear in the literature on the identification and understandability of gestures on touchscreens.  It is truly an emerging convention. Even fewer, and perhaps even none, have studied the identification and understandability of gestures across cultures.  Yet, many of the products that enable gestures will be intended for a global audience.  Designers have a definite need to know how understandable gestures are, and how universal they are, while the field is still young.

The International Usability Partners (IUP) has run a global study aimed at identifying the most common user-generated gestures for control of a touchscreen within a given country and across cultures.  The (IUP) is an established network of 12 independent usability companies based in 12 different countries (including Belgium, China, Chile, Finland, France, Germany, India, Italy, Spain, Sweden, UK, and USA) who have joined  to provide user experience services worldwide. This research involved collecting user-defined gestures for 28 common commands (like scroll, open, rotate, zoom in, cut, copy, paste) across 9 different countries and analyzing the resultant data for consistency of responses and cultural influence. 40 participants in each country were presented with a “before” screen image and an “after” screen image and requested to make a gesture that will result in that “after” screen image. The gesture made was recorded and catalogued.

The analysis begins soon.

About the International Usability Partners:
IUP is a network of equals, run by all partners for their clients’ benefit. Each partner is an expert in global user-experience research and design, specializing in the cultural and linguistic characteristics of their home country. “IUP quality” stands for companies that offer world-class user experience services of consistently high quality in a reliable and efficient manner. All IUP members have a common understanding of the methods, processes and scientific background of the field of user experience (human-computer interaction).

IUP members have effective quality assurance and process improvement programs in place as well as regular professional training for their staff. Members also have state of the art equipment and tools that facilitate in-house and mobile user research. Facilities are conveniently located around the world in 14 cities including: Chicago, Helsinki, Hyderabad, London, Paris, Torino, Munich, and Beijing.