Tod pointed me to this excellent article by the late Jef Raskin, Macintosh catalyst, designer and author of The Humane Interface.
He rightly identifies a lack of ways of comparing outcomes produced by technologies as supporting th creation of superstitious beliefs. He uses fishing lures and stereo cables as examples of technologies that have lots of superstition surrounding them.
When out angling for rock fish, you generally use the same lure as everybody else. There is not much technique to it, so the number of fish you catch is proportional to the time your lure is in the water. Those who spend time fiddling with the equipment beforehand catch fewer fish. It's a mathematical certainty.[...]
Superstitions grow rampant when testing is subjective, difficult, and (usually) not performed at all. There is a purely magical belief in the idea that you can hear the difference between different brands of audio cables, for example.
He then relates an experiment he conducted that showed that people's preconceptions affect their perception. This is a point explored in detail by Nass and Reeves in The Media Equation. He concludes by noting that
Computer systems exhibit all the behaviors best suited to create superstitious responses. You will try something, it won't work, so you try it again—the exact same way—and this time it works, or not. That's random reinforcement.[...]
We rarely understand, in any detail, the processes going on behind the tasks we do with computers. We're using megabytes of code written by others, code that is indifferently documented and inadequately tested, and which is being used in ways and in combinations unforeseen by its creators.
No wonder we tend to act as if computers are run by magic.
I agree with him entirely, but I'm not sure what his recommendations would have been about how to avoid this reaction; clearly faulty, superstitious models based on incomplete information have existed for a long time (as one of commentors to Raskin's piece says, it's the basis of religion). But we will never have complete information and we can't expect the users of technology to go out and get it. Without going too far into philosophy, everything we know is an incomplete model. As technology becomes more complex, our models will grow ever more distant from the reality of what's going on, so Raskin's Complaint is totally valid, yet it's not clear (for me, anyway) how we can use it to determine where to go next.
NOTE: I've updated my Partial Bibliography of Magic in UX Design with Raskin's paper and the excerpt from Steven Levy's iPod story I had blogged about earlier.
[Tangentially, it's interesting to note the seemingly opposite directions that our understanding of technology and science are going: over the last couple of hundred years we've continually improved our models of how things--life, physics, society--work, while our tools have grown proportionally less comprehensible. Will there be a meeting at some point, where our study of our tools becomes as involved and complex as our study of our world? Will there be a crossover, where the study of our tools becomes a more relevant point of scientific inquiry than the study of the "natural" universe?]
