August 2005 Archives

At the recent NPUC there was a demo of the IBM SHARK gesture recognition system (pages with videos and a download of a software demo). As presented by Shumin Zhai, this is an alternate handwriting-based text input methodology. It works roughly like this: you draw a line between all of the letters in a word on a grid and the system recognizes the shape of the line you drew as the word, even if you didn't land exactly on all of the letters exactly. There's enough identifying information in the line shape to produce high accuracy of uniquely identifying the word. This similar to how Xerox's Graffiti method works for individual letters, but it's for whole words, and is significantly more efficient, apparently rivaling typing. Since the shapes of words on the grid are relatively unique, the grid arrangement is fairly arbitrary (and IBM is trying to find an optimal one for English). Although initially taking up more space than the Graffiti box, ultimately, the grid become unnecessary as people learn the shapes of the word-lines. Once that happens, the system can recognize the shape wherever it's drawn, and even recognize it despite severe distortions. Text entry ergonomics are generally a geeky thing, but this is way cool.

It's also seems like a much better way of introducing people to gestures command systems than having them memorize arbitrary gestures (even simple ones). First, people make the arbitrary shapes on a grid. Then, as they get good, they can put the grid away and the gestures still work. More importantly (for me, anyway), is that the shape of the lines is arbitrary. It could mean a word, or it could be a sequence of pictures, or a direction on a map. The basic recognition engine doesn't care.

As I was watching the demo, this thought came to me and I couldn't watch the rest of the demo without thinking it: magic wand. With an accelerometer embedded in a wand, or an IR LED and camera tracking system (like the PS2 Eyetoy), suddenly the waving of the wand, the patterns of loops, lines and intersections in the air, become meaningful. Commands can be issued by waving in specific patterns. Now-defunct Neurosmith made one for their Musini toy, and it did a lot less, but it may have been an early sign (like the Samba de Amigo maracas):

This dovetails well with what I've been thinking about the long-term effects of Harry Potter on the generation of kids growing up with it, the generation of ubiquitous computing consumers (call 'em Generation U or Generation H--maybe not, that's too close to Preparation H...but I digress). It seems clear where this is going. Low-power wireless networking, distributed computing, accelerometers combined with SHARK-like gesture recognition means action at a distance with a wave of the wand. Magic.

Now consider the pick-and-drop research from Jun Rekimoto's lab at Sony. Pick-and-drop is an interaction metaphor in which people "pick" up a virtual object (such as a window) on one device with a special pen, "carry" it to another device, then "drop" it there. This works by giving every pen a unique identity and having each device query a central pen server and file server before allowing the item to display on the local machine. However, all of this happens quickly, so the effect is that of tearing off a window from one screen and plopping it on another.

If we add this functionality to the device I described above, we get a thing that works using a paradigm that closely resembles a traditional magic device. And wands are devices that a significant portion of Generation U has grown up in comfort with. As Liz points out, one of the best things about Harry Potter is how mundane, comfortable and everyday magic is (to the wizard class). In reality, in the ubicomp version the magic is the interaction between networking hardware, displays, gesture recognition, window servers, access control lists and encryption (a public key may be the only data actually stored in the wand), but that's not how it feels.

Call it heresy--user-centered design is probably not supposed to make computers seem magical, just metaphorically appropriate--but maybe Harry Potter and its sisters (such as the His Dark Materials) provides a new model of ubiquitous computer experience design. After all, it's a distillation and update of classical metaphors for a new generation, which means that it's both (excuse me the marketing speak, but maybe this one time it's appropriate) timeless and contemporary. Maybe it's time to make magic objects.

[Before anyone mentions it: yes, I remember when, in 1994, Bill Gross said that if someone wanted to read his business plan for Knowledge Adventure Worlds, all they had to do was read Snow Crash.]

One of my ongoing projects is (as described in this Slate article by Paul Boutin). Since I have a fairly large corpus (to use the linquistic geek term) to play with, I occasionally do a little analysis on it. Here are the top 250 words used by the 6000 authors in 2004, in frequency order. You can probably figure out the gist of the subject of many of the letters from this list:

1. tired
2. i
3. and
4. the
5. of
6. a
7. my
8. am
9. it
10. in
11. you
12. because
13. is
14. that
15. for
16. this
17. have
18. me
19. not
20. so
21. at
22. up
23. im
24. sleep
25. all
26. do
27. on
28. with
29. but
30. or
31. just
32. get
33. no
34. be
35. why
36. are
37. was
38. can
39. work
40. like
41. go
42. if
43. out
44. about
45. night
46. may
47. time
48. what
49. now
50. don
51. we
52. day
53. mail
54. really
55. know
56. your
57. too
58. they
59. people
60. any
61. as
62. had
63. by
64. then
65. much
66. life
67. want
68. when
69. been
70. who
71. being
72. he
73. e
74. an
75. one
76. she
77. school
78. more
79. there
80. her
81. will
82. part
83. its
84. hours
85. think
86. only
87. last
88. would
89. got
90. has
91. dont
92. well
93. some
94. going
95. new
96. email
97. how
98. back
99. even
100. please
101. us
102. good
103. ve
104. u
105. enough
106. other
107. very
108. love
109. free
110. bed
111. which
112. them
113. feel
114. home
115. late
116. need
117. our
118. every
119. way
120. job
121. never
122. here
123. things
124. make
125. bored
126. still
127. their
128. morning
129. always
130. could
131. also
132. than
133. today
134. d
135. right
136. information
137. over
138. help
139. old
140. off
141. intended
142. after
143. around
144. take
145. image
146. friends
147. gif
148. site
149. tell
150. having
151. went
152. getting
153. see
154. org
155. should
156. cause
157. two
158. ll
159. long
160. something
161. his
162. years
163. week
164. him
165. use
166. year
167. into
168. little
169. recipient
170. thanks
171. didn
172. unable
173. print
174. myself
175. doing
176. find
177. nothing
178. working
179. hard
180. read
181. maybe
182. where
183. again
184. makes
185. cuz
186. world
187. anything
188. early
189. until
190. house
191. money
192. cant
193. did
194. these
195. wake
196. down
197. best
198. days
199. everything
200. trying

201. friend
202. lot
203. many
204. computer
205. live
206. ever
207. thing
208. before
209. stupid
210. better
211. most
212. say
213. those
214. same
215. yes
216. confidential
217. family
218. keep
219. since
220. person
221. care
222. ru
223. thank
224. college
225. come
226. sick
227. stay
228. virus
229. next
230. shit
231. bad
232. does
233. fucking
234. website
235. were
236. while
237. months
238. hate
239. though
240. able
241. oh
242. hour
243. hi
244. thats
245. girl
246. reason
247. web
248. let
249. first
250. kids

Whoops, I thought I had taken out most of the non-content words, but it looks like "confidential", "recipient" and "information" slipped through. These usually come from disclaimers like this, which makes them especially amusing:

CONFIDENTIALITY NOTICE: This electronic message transmission is intended only for the person or the entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. If you have received this transmission, but are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this information is strictly prohibited. If you have received this e-mail in error, please contact the sender of the e-mail and destroy the original message and all copies.

Just noticed that Vonage is sponsoring a phone casemod contest (called "Pimp that Phone"). Seems a bit half-baked and jumping on the flavor of the moment, but it's an interesting experiment and another example of casemod culture moving into the mainstream.




A device studio that lives at the intersections of ubiquitous computing, ambient intelligence, industrial design and materials science.

The Smart Furniture Manifesto

Giant poster, suitable for framing! (300K PDF)
Full text and explanation

Recent Photos (from Flickr)

Smart Things: Ubiquitous Computing User Experience Design

By me!
ISBN: 0123748992
Published in September 2010
Available from Amazon

Observing the User Experience: a practitioner's guide to user research

By me!
ISBN: 1558609237
Published April 2003
Available from Amazon

Recent Comments

  • Katherina: Information not just material. In our days it is a read more
  • Hi Mike, totally agree on building the IoT in a read more
  • Mutuelle: Man is the reflections of his thought, some name it read more
  • Amanda Carter: You obviously placed a great deal of work into that read more
  • Molly: You might find it interesting to connect with return of read more
  • George: You might want to change "Size" to "form" for terminal. read more
  • Mike: Thanks for the reminder, Robin. I'm aware of that article, read more
  • Robin: It's a slightly different argument (it predates most work in read more
  • Tim: This reminded me of the Pleo video Mark posted awhile read more
  • michael studli: i was wonting to know is the game fun to read more

About this Archive

This page is an archive of entries from August 2005 listed from newest to oldest.

July 2005 is the previous archive.

September 2005 is the next archive.

Find recent content on the main index or look in the archives to find all content.