Back in 1997 people started questioning the applicability of the current WIMP (Windows Icons Menus and Pointers) paradigm to the upcoming NUI generation of HCI. When we think of interfaces such as speech, hand gestures or touch it is very hard to think in terms of the standard GUI WIMP toolkit. So recently Ron George has suggested we think in terms of these things:
- Objects
- Containers
- Gestures
- Manipulations
You can read his original post here.
I'm not going to go into the complete explanation of each of these, I will leave this up to you. I will however talk about my feelings in regards to OCGM. For one I think the acronym is brilliant. Its managed to stay in my head with barely any effort, and in the heads of the people I've told about it. But thats not the important part. Lets start with the things I like about OCGM.
The important thing I like about OCGM is the natural way it links to the idea of metaphors. One thing that keeps popping up in my research of UI design is the importance of metaphors. When a user first sees an interface they are much more likely to comprehend the interface and be able to easily pick it up if it has obvious metaphors they are used to. On a computer this might be the way that "Folders" are used to collect "Files" these are obvious metaphors to an office situation. Or how the buttons on my media player are the same as those on my video player. Metaphors allow the user to move from the real world to the virtual world with ease. OCGM can also be used to describe the real world. Things we use in every day life are Objects and of course Containers are obvious. Manipulations are the way we physically interact with things. Then we come to gestures. This one might be a little harder to pin down. But in the real world I see gestures relating to communication. When I wave goodbye to someone, this is a very simple gesture. When I say "How's your day been Bill?" this is a slightly more complex one.
Now, moving back to computers. Imagine an interface for sorting contacts. Let's imagine this is an application on a multitouch fridge. A standard WIMP approach would be to have a list of people going down a page. Click to show each person. Open a menu to edit, save etc. Imagine giving this interface to a 4 year old. They wouldn't know what to do. Now lets think in terms of metaphors. In the real world people represent themselves as Business cards. We can store business cards in books. So these become our Objects and Collections. Manipulations are obvious drags. Our gestures might be a double tap to edit, or maybe a hold.
So at this point we have an idea of why I like OCGM, when we remove ourselves from the restrictions of WIMP we are much more free to design interfaces in a natural way. This is good. By imposing only vague restrictions in forms that are familiar to people from the real world we only restrict ourselves to things that make natural sense.
I guess this is the part where I go into criticisms. My main criticism of OCGM is in Gestures. I understand exactly what the difference is between Gestures and Manipulations but I feel that in a Tabletop or Multitouch environment (as Ron uses for an explanation) Gestures are something which do not come naturally. I'm currently torn up between the idea of one global gesture, which brings up menus. Similar to the "menu" button on android. Or individual gestures which users learn over time to perform actions. At the moment I much prefer the idea of objects which serve as tools for commands. For example imagine back to our contact organisation system, if a naive user wanted to edit the information the interface does not show an obvious way to do this. But if we had some sort of "Editing Pen" we could drag this object onto the contact and suddenly it becomes editable. Of course my problem with this approach is the clutter that could come about.
At this point I'm mostly sold on OCGM. I don't think finding a magical WIMP replacement is a huge issue, but I think OCGM sets a nice foundation. A foundation that allows UI designers to remove themselves from their restrictive environments and let their minds free. I'll of course be doing more research into this and will be thinking things through a bit myself.
If you have any thoughts of your own, the comments here exist for a reason.
The problem with using design metaphors (actually there is HCI term for that which I can't remember... you should probably find out what it is and use it) is that you get stuck with old, shitty interfaces which are constrained in ways that they don't have to be anymore.
ReplyDeleteIf we constantly copy other interfaces people know, we'll never move forward in interface design.
STOP LIMITING YOURSELF
STOP LIMITING YOURSELF
Thats a fair point. However metaphors are a good place to start. From there we can build more abstract manipulations and actions that cannot be done in the real world.
ReplyDeleteAs you said, a huge problem with gestures is that often there is no real world equivalent and thus no transfer to be had. Look at the difficulty cruiser had trying to find a "copy" gesture. As you know, they eventually went with he metaphor of the copy machine, when of course a gesture would have saved unnecessary interface clutter, however just wasn't natural or apparent to users.
ReplyDeleteI do like the work I saw in Chi 09 where they did a whole lot of user studies and created a user-defined set of gestures, but their process was fairly preliminary.
I read the article and I think they did a fairly good job of succinctly articulating current trends in interface design.
I look forward to future blog posts :)
Mmmm I think these things are interesting. Of course at some point people will have to learn new things about an interface. If you read Ron's blog he has a new post about gesture icons. It would take a little while to learn these, but they do seem to apply quite well.
ReplyDeleteA good idea that 10/GUI had was making each added finger mean something more. Kind of like ctrl,shift,alt,etc work on a keyboard. For example two finger (or two hand, one finger each) pull (like zoom out on iPhone) is as expected, but make it a four finger gesture (two hands two fingers each) and it could become copy...
Of course adding more and more complex manipulations isn't really the solution.
As you said ben as new systems and interface are developed people are going to just have to deal with it at some point and learn the new way to do things. You can't hold the users hand forever eventually people just have to learn.
ReplyDeleteComplex gestures are eventually going to have to be used if you want a large number of tool at the users fingertips in a surface computing environment. There are only so many simple gestures you can do and you (generally) can only map one thing or tool to a gesture.
Hi Ben,
ReplyDeleteGreat post about OCGM. I'm Josh Blake, one of Ron's colleagues and a co-conspirator about this whole OCGM thing. I'm actually who thought up the acronym order and link to Occam's Razor, based upon Ron's idea. I have a few posts on my blog that you and the other commenters might be interested.
http://nui.joshland.org/search/label/OCGM
I elaborate on OCGM, metaphors and their role in human cognition and interfaces, and also posted a scholarly paper about OCGM with a lot of references and academic justification for OCGM.
You may also be interested in the book I'm currently writing called "Multitouch on Windows." http://www.manning.com/blake Chapter 2 is all about OCGM. There's some really interesting (IMHO) stuff in there. Chapter 1 is a free download, all about NUI. If you pre-order the book you can read the chapters as I write them, and chapter 2 is next to be posted.
Ben, Alex, and Cat,
ReplyDeleteYou guys have the right idea about favoring manipulations over gestures and keeping gestures simple.
One thing to keep in mind as we talk about gestures is that they are any body motion that contains a message in the movement itself. As you mentioned Ben, double tap and hold are both gestures, as is single tap. Gestures such as tracing out complex shapes or others that require learning should be minimized, or if needed, be limited to advanced users. The novice users can get by with the manipulations and very basic gestures like tap.
Another thing to think about is complex gestures can be shortcuts for things you could otherwise do with just manipulations. Just like Control-C is a shortcut for Copy, which you could also do by moving the mouse cursor up to a toolbar or menu and clicking the right places. This also enables novice users to use the interface and learn the complex but quicker interactions later.
Hi Josh,
ReplyDeleteThanks for the intriguing comments. I actually first got on to OCGM via your blog, but of course lost it in my computing history. Since then I found it again on Ron's. I'll make sure I bookmark yours this time.
Cool. Are you doing any multi-touch or NUI projects?
ReplyDeleteI'm currently doing Honours in HCI at the University of Sydney in Australia. My Honours project involves exploring the implementation of software for multi-touch tabletop systems. Something obviously relevant to your own research.
ReplyDeleteI just started last week and I've got around a year to finish, so I'm hitting the ground running. I figure if I write enough now, it'll be much easier to write later.
I've also done some other NUI projects in the past, the most recent was a multitouch music visualisation app that we were running on a wall at a party. Was very cool, people quite enjoyed it. I'll put details up later this week.