Smart Things: Chapter 3, Interaction Metaphors, Part 2

This is Part 2 of a pre-print draft of Chapter 3 from Smart Things: Ubiquitous Computing User Experience Design, my upcoming book. (Part 1) (Part 3) (Part 4) The final book will be different and this is no substitute for it, but it's a taste of what the book is about.

Last month I posted a draft of Chapter 1

Citations to references can be found here.

Chapter 3: Interaction metaphors

Part 2: A Catalog of Ubicomp UX Metaphors

Metaphors already form the conceptual scaffolding for many prominent ubiquitous computing projects and products. Though rarely labeled as such in the projects' description, user experience metaphors guide many assumptions about the projects' value, desirability, and how people will use it.

The following metaphors are ones that I have observed in ubicomp user experience design. This list is certainly not exhaustive and there are many natural overlaps in the concepts involved. I include it here to prompt creative thinking about how people relate to new technologies and how to design for them.

[Footnote: See Barr et al. (2002) for a different taxonomy of user interface metaphors.]

(Image from PICOL)

Organizational Metaphors

These metaphors describe ideas that include how systems of interacting ubiquitous computing technologies relate to each other and to the people who use them.

1. THE FACTORY The world is a factory.

   Domotics can be defined as the set of elements that, when installed, interconnected and automatically controlled at home, release the user from the routine of intervening in everyday actions and, at the same time, provide optimised control of comfort, energy consumption, security and communications.

   - Bravo et al. (2006)

   Our kitchen supports the automatic generation of web-ready recipe pages, with other possible applications including actual cooking assistance, and communication or education across distances, cultures and generations.

   - Siio et al. (2004)

   The field of "home automation," for example, treats the house as a kind of factory by implying that repetitive activities should be automated to maximize production of leisure. In this metaphor, people own the factory. Their role is to act as overseers, organizing an efficient assembly line of appliances to produce ever more free time. The implication is that automation will automatically make more time for pleasurable activities.

   Most "labor-saving" products fall into this category. Their core assumption is that technology should and will eliminate repetitive actions.

   [Footnote: As documented in Cowan (1983), one of the unintended consequences of such technologies is that they tend to raise expectations, rather than eliminate labor. From an experience design perspective this could be because they do not try to address how that time could be filled, and whether the repetitive work is drudgery in the first place.]

2. PUBLIC UTILITY
   Information processing is a utility, like electricity.

   If computers of the kind I have advocated become the computers of the future, then computing may someday be organized as a public utility just as the telephone system is a public utility.

   John McCarthy (1961), quoted in Garfinkel and Abelson (1999)

   From the earliest days, computing has been characterized as a new kind of electrification. Ubiquitous computing likewise continued to embrace it.6. The metaphor's implication is that information processing will be as accessible, in as many places, as electricity. In this view, the movement of computing out of personal computers is analogous to early twentieth century electrification of ordinary homes and workplaces, and will create applications as pervasive as electric lighting. In this metaphor, the walls and floors of the spaces we inhabit house conduits filled with information processing that enlivens devices as electricity powers lamps.

   Many telecommunications networks implicitly rely on this model of information processing by treating the traffic they carry with a neutral distance. The idea of "network neutrality" in effect demands that the Internet backbone treat packets as the electrical infrastructure treats electrons. Some electrons may heat a lamp filament, while others turn a motor. The electrical infrastructure does not differentiate in supplying them — it raises prices based only on aggregate network demand.

3. BACK TO NATURE
   Ubiquitous computing returns us to a pre-industrial life.

   There is more information available at our fingertips during a walk in the woods than in any computer system, yet people find a walk among trees relaxing and computers frustrating.

   - Weiser (1991)

   This metaphor implies that technology and nature can coexist in a way that de-emphasizes technology to the point that it becomes indistinguishable from nature.

   [Footnote: Without, it should be noted, defining what "nature" is. See Marx's (1964) description about how American notions of pastoral coexistence with Industrial Revolution technology, the "Machine in the Garden," resulted in the disruption of American pastoral life.]

   Projects that use this metaphor emphasize the "organic" nature of interaction (such as Rekimoto, 2008) and the way that computers can free people to leave their offices to sit on the beach.

   [Footnote: It should be noted that one project that literally put computers into the forest. Ambient Wood (Rogers et al., 2004) focused on the educational possibilities of ubicomp technology outside the classroom, rather than attempting to create a pre-industrial landscape.]

4. THE VAPOR
   Computation is a cloud that surrounds us.

   This metaphor compares the permanent availability of information and information processing to a vapor enveloping (and possible penetrating) every person and every object. Like Pigpen’s cloud of dust in the Peanuts comic strip by Charles Schulz, everyone perpetually moves within a cloud of digital information. The clouds extend beyond our reach and do not have a defined shape or boundary. Events that happen in the cloud may be outside of the control of any one person in it.

   Projects such as Urban Atmospheres (Paulos and Jenkins, 2005), Digital Aura (Ferscha et al., 2004) and Personal InfoCloud (Vander Wal, 2005) evoke this idea. The terms "pervasive computing" and "ambient intelligence" derive from this metaphor: in pervading an area, vapors process information ambiently.

5. PARALLEL UNIVERSES
   Technology gives us access to a parallel universe with different laws.

   It [appears] to the user that the virtual and real objects coexist in the same space, similar to the effects achieved in the film Who Framed Roger Rabbit?

   - Azuma (1997)

   

   A vision of "cyberspace" underlies many views of Internet-based information access. In this vision, computers create, reveal, or possibly contain a parallel universe. Implicit in this vision is the notion that while technology may provide access to another universe, the universe exists independently of the technology. For example, the term "physical computing" implies that other kinds of computing are not physical, they are not here, which means that they happen elsewhere, perhaps a parallel universe. In this vision, the device creates a window, and beyond that window lies a radically different world with its own physics and geography. Science fiction gives us the canonical examples of this alternate space, from William Gibson’s original notion of cyberspace to Neal Stephenson’s Street in Snow Crash (Stephenson, 1992).

   Augmented reality applications often rely on this metaphor as well. When the screen of a mobile device superimposes digital images on a live camera feed, it is presented as a window on another world where, like Roger Rabbit's Toon Town, data and humans cohabit. Similarly, Hertzian space (Dunne and Raby, 2001) is presented as a parallel dimension created by devices using the radio frequency spectrum, and the term "physical computing" implies that other kinds of computing are not physical, they are not here, which means that they happen elsewhere, perhaps a parallel universe.

6. THE INESCAPABLE PRISON
   Omnipresent technology imprisons us.

   When every action is recorded for perpetuity, in a seemingly objective manner, and there is a likelihood that the consequences will be realised, then Bentham's Panopticon becomes fully realised.

   - Dodge and Kitchin (2007)

   This dystopian metaphor implies that information technology will imprison its users, empowering governments, organizations, and individuals to create novel forms of tyranny. As people increase their dependence on information technology and share personal information, they decrease their ability to act (and possibly think) freely. If taken too far, or left unchecked, proponents of this metaphor suggest a coercive environment might emerge. For example, Albrecht and McIntyre (2005) argue that governments and corporations will inevitably use RFIDs to uniquely identify and track devices and individuals to force people to behave in certain ways. Others suggest that omnipresent sensing and recording devices will create a situation of sousveillance (Mann et al., 2003): a world in which people are constantly spying on each other and being spied upon.

Interaction Metaphors

The second class of ubiquitous computing metaphors concerns how people will interact with these technologies.

1. TERMINALS EVERYWHERE Every surface is a display.

   

   This metaphor takes the laptop as its point of departure and imagines the functionality of laptops embedded into a variety of flat surfaces. General purpose, network-connected displays expand or contract and become embedded in furniture, architecture, or clothing, but their imagined functionality remains basically the same. The metaphor implies that interaction with these displays may differ in some details, but that we will use them for much the same purposes as we do today.

   The Microsoft Surface is an example of a device that follows this interaction metaphor by imagining a coffee table as a data terminal.

2. INVISIBILITY
   Computers are ever present, but transparent.

   It is invisible, everywhere computing that does not live on a personal device of any sort, but is in the woodwork everywhere.

   -Weiser (1994)

   The implication of Weiser's quote, the title of Don Norman's The Invisible Computer (Norman, 1998) and the European "Disappearing Computer" initiative (Streitz and Nixon, 2005) implied that computers can vanish from human perception while continuing somehow to exist. The metaphor establishes a powerful image: everyday life stays exactly the same as today, but is simultaneously altered by invisible computing everywhere.

   How do we interact with these invisible computers? Since Weiser's original statement, scientists, engineers, and designers have struggled to implement invisibility. One model is the car. Though contemporary automobiles contain dozens of microprocessors, most drivers will never recognize many of these tiny computers as such. [Footnote: Cars have included multiple computers in the form of dedicated microprocessors since the early 1980s. Yakal (1983) listed five in the 1984 Lincoln Continental alone.] Instead, they just feel the brakes react to road conditions, turn on satellite radio, and cheerfully ignore the automatic gear adjustment.

3. ANIMISM
   Digital devices become animals or people.

   I don't want to argue with my car about where I want to go.

   - Mark Weiser, quoted in The Plenitude (Gold, 2002)

   Any uniquely detectable physical object may become a Passenger.

   Streitz et al. (1999)

   

   One broad definition of animism is the belief that objects have will, intelligence, and memory, and that they interact with and affect our lives in a deliberate, intelligent, and somehow conscious way (Kuniavsky, 2003). Animist metaphors for devices illustrate that we might interact with them as we interact with friends, pets, or pests.

   Virtually every project that emphasizes how devices react to people is in a sense animist. The Nabaztag is a rabbit. Roomba robotic vacuum cleaners were designed to emulate insect behavior (Kurt, 2006). And one widely cited scientific project is described as an "aware home" and "living laboratory" (Kidd et al., 1999).

4. PROSTHETICS
   Ubicomp extends our bodies.

   To me, the primary motivation behind the information appliance is clear: simplicity. Design the tool to fit the task so well that the tool becomes a part of the task, feeling like a natural extension of the person.

   - Norman (1998)

   The computer is a more sophisticated extension of the central nervous system than ordinary electric relays and circuits.

   - McLuhan (1968)

   
   Figure 3-4: Motorola WT4070/90 wearable terminal.
   (Copyright 2008 Motorola)

   Technology attached to human bodies that is not biological acts as a kind of prosthetic (Figure 3-4). In this view, ubiquitous computing replaces human biological functions with silicon ones, or gives people a kind of super power by amplifying their natural senses. The comic book superhero Batman is an archetypal prosthetic technology user. Special crime-fighting technologies enhance Batman's natural abilities, but they do not remove the need to engage with the world directly. Batman fights crime in person, with his hands, instead of sending a remote-controlled robot to do his fighting for him.

   

   From the notorious Lovegety dating device (CNN, 1998) to the adidas_1 running shoe, many wearable computing products want to extend natural senses and physical capabilities. With industrialized countries facing a wave of aging population, cognitive abilities are also up for augmentation as well. The Forget-me-not (Lamming and Flynn, 1994) is one of the earliest computer-based attempts to aid memory.

5. ENCHANTED OBJECTS
   Electronic devices behave like enchanted objects.

   
   Figure 3-5: Ambient Orb by Ambient Devices.
   (Courtesy Ambient Devices)

   Some devices add computational functionality to well-known — if often fictional — objects. By referencing the name, form, or behavior of magical objects in myths and fairy tales, their makers ask users to ignore the technical details and focus on the resulting experience.

   One waves around a Nintendo Wiimote, for example, much like a magic wand, while the Ambient Orb (Figure 3-5) resembles a fortune teller's crystal ball. Accenture's Magic Medicine Cabinet (Wan, 1999) "takes advantage of the situated nature of the medicine cabinet but extends it from a passive storage space into an interactive appliance." Enchantment — the addition of a magical spell to an existing object — implies that the object is mostly like its mundane counterpart, but with key behavioral differences created by information processing technology.

Mixed Metaphors

Sun Microsystems Unveils Open Cloud Platform.

- Press release headline, March 18, 2009

Since explicit metaphor design is not a typical part of experience design, it is not surprising that many, possibly even most, ubicomp projects mix metaphors freely. For example, Starner et al. (1997) described an augmented reality system that acts as a "butler/confidante" that enhances memory by overlaying the results on a video image of the current surroundings projected through a worn video display. Parsing the description carefully, it seems that the system is a human-like prosthetic that presents a parallel universe of personal memory data. Mixed metaphors might not make systems any less functional or valuable in practice, but looking for inconsistencies and contradictions in metaphors does identify potential design issues to resolve. Does the system represent itself as a single butler or perhaps as multiple butler–agents? Is the butler's reasoning expected to appear human? Does the butler disappear when the heads-up display is removed, or could you call him (it?) on the phone?

Tomorrow: Chapter 3, Part 3: Designing with metaphors