Gesture Research – Data Analysis (Part 1: Data Reduction)

WARNING: This blog is a detailed description of the data reduction steps and is not suitable for all audiences. Parental discretion is advised. These are professionals; do not try this at home.

The result of the data collection yielded a detailed spreadsheet of all unique gestures (Gesture Glossary) and a results spreadsheet of all the gestures each participant made for each action in every country.

In all, we had 40 participants in 8 countries and 20 participants in 1 country create gestures for 28 actions, yielding over 9,500 gestures in total. Each of these gestures were catalogued according to the lexicon defined by the research team.

As noted in Data Structure, the lexicon collected very detailed information about each gesture.  This detail will prove useful in many analyses. However, for certain types of data analyses, not all of the detail is needed. At times, the detail of the information resulted in gesture differences that were not meaningful. For example, when someone selects a word to cut or copy, it isn’t significant which direction they swipe to highlight that word. Another example: when someone is performing a page-level command (like scrolling), it doesn’t necessarily matter if they perform that gesture over an object or over whitespace. So, we undertook an effort to combine gestures when the differences between them are not meaningful for a particular action.

To combine gestures where the differences between them are not meaningful, we first defined a series of guideline definitions of what is meaningful and what is not. We then had 3 independent researchers apply those guidelines to the unique gestures in the glossary. We then reconciled the different results of those 3 people into a final combination set.

We defined 2 sets of guideline principles, based on the type of action. There are 2 types of actions: 1) those that require action on an individual object on the page (termed Object Level Actions) like delete and close folder, and 2) those that require an action on the page itself (termed Page Level Actions), like scroll and print.

For Object Level Actions, we decided to apply the following guidelines:

• Swipe direction is not meaningful, so combine all swipe actions regardless of direction
• Location of action is meaningful, so leave all detail regarding location
• Combine when the participant performs an action outside the scope of the task, and removing that action leaves an identical gesture to an existing gesture
• Combine repeated gestures into one group regardless of the number of times it was repeated (e.g. “tap”)
• Combine letter/word symbols into two categories: 1) Letter/word other than “X”, and 2) “X”

The actions classified as “Object Level Actions” are:

• Delete
• Open Menu
• Close Folder
• Open Folder
• Accept/Verify
• Paste
• Move
• Place Focus
• Multi Select
• Select Text

For Page Level Actions, we decided to apply the following guidelines:

• Swipe direction is meaningful, so leave all detail regarding direction of swipe
• Location of action is not meaningful, so combine all gestures where location was the only difference.
• Combine when the participant performs an action outside the scope of the task, and removing that action leaves an identical gesture to an existing gesture
• Combine repeated gestures into one group regardless of the number of times it was repeated (e.g. “tap”)
• Combine letter/word symbols into two categories: 1) Letter/word other than “X”, and 2) “X”

The actions classified as “Page Level Actions” are:

• Forward
• Undo
• Cut
• Copy
• Redo
• Back
• Home
• Save
• Print
• Pitch
• Rotate
• Scroll Down
• Scroll up
• Zoom Out
• Magnify
• Zoom In
• Continuous Scroll
• Help
• Minimize
• Stop Scroll

After the gestures were combined, we took one additional step. For five actions (cut, copy, paste, magnify, and open menu), there were actually two steps. For cut, copy, and magnify, the user first had to select text and then perform a command that would cut, copy or magnifiy. Some users combined the two steps into one gesture, which is fine, but many took two distinct steps. We decided the step of selecting the text (cut, copy, magnify) should be split from the command that performed the action, so we split them. We created a new gesture category, called “select text” and put all of the gestures intended to select text into that category, leaving only the gestures that were intended to perform the original action (cut, copy, magnify) in the action’s category.

For paste and open menu, the user first had to place focus on a particular spot on the screen and then give the command that would paste or open the menu. Again, some users combined the two steps into one gesture, but many took two distinct steps. We again decided to split the step of placing focus from the step of performing the command (paste and open menu) and moved the gestures intended to place focus into a “place focus” category, leaving only the gestures that were intended to perform the original action (paste, open menu) in the action’s category. These steps together resulted in a cleaner view of those 5 actions.

Our next step was to analyze the data.