jnd.org

“]]],[1,”blockquote”,[[0,[2,5],1,”FOOTNOTE”],[0,[],0,”1. Copyright © 1992 by Donald A. Norman. All rights reserved. Originally published by Addison Wesley. Now out of print. [“],[0,[6],1,”Return to Text”],[0,[],1,”]”]]]]}’>

CHAPTER SIX OF TURN SIGNALS ARE THE FACIAL EXPRESSION OF AUTOMOBILES¹

The delights of having information ever-present are amazingly seductive. Wouldn’t it be nice to have a personal assistant, small and unobtrusive, that could remember the details of life for us, so that we could always have them available on demand? It would take care of the daily trivia of life, things like telephone, passport and drivers license numbers, as well as the important things:

“What was the name of that wonderful restaurant we had dinner at two years ago?”

“How late does the library stay open during the summer? Do I have time to get there?”

“Is today Michael’s birthday? I forgot all about it. Quick, I’ll buy him something. Umm, what size does he wear?”

“Remember good old what’s-his-name, you know, the person we had dinner with after that party at what’s-her-name’s house?”

Or even new information:

“I have a free evening: what’s happening in town tonight? Any tickets available?”

We are particularly bad at remembering, especially when it comes to details. In fact, we aren’t very good at anything that requires great precision and accuracy. We remember the major experiences of life, but less accurately than we might like to believe: the details fade quickly. Our strengths lie in other areas: in aesthetics, beauty, humor, and imagination. We are excellent at doing meaningful things, at understanding, at leaps of creativity. In fact, I believe that the very brain mechanisms that make us so good at creativity and aesthetics result in poor capability for the memory of details.

You would think that if we were good at one thing, bad at another, we would structure the world to emphasize the things we excel at and minimize those we aren’t so good at. Nope. We did create a world that appreciates creativity and beauty, humor and insight, but we also managed to create one that requires precision and accuracy. In today’s world of technology, of machines, clocks, and computers, details are essential. Arbitrary numbers rule our lives: phone numbers, mail codes, street addresses, government identification numbers, drivers license numbers, bank accounts, passports. Somebody seems always wanting us to provide a number, some new form seems always to want precisely the details we cannot remember.

The problem is that the same machines that we invented to aid us also demands precise, accurate information from us. As a result, we often end up the slaves of the very machines we have invented to serve us, slaves to their relentless demand for precision, accuracy, and continual supervision.

But, thinks the ever-optimistic technologist, the problems created by machines can also be overcome by machines. Suppose we could make machines to aid us in providing these details, the minutiae of life. Suppose we could give every one of us a little personal assistant, one that we would carried with us everywhere and that could continually give us access to the information we need to go about our lives quietly and efficiently.

You can, I am sure, make up your own list of things you wish you could have remembered at the time you needed them. Suppose you could somehow record them readily and easily with a device that was ever present, always at your service, but small enough not to be a bother. Suppose we could invent the perfect mixture of artifact and human, each serving the other well.We humans, as I have said, are good at the creative things, no so good with the details. Computers are just the opposite. Why not marry the two properties: create a computer-like device that was small, portable, and unfailingly accurate, retentive, and precise. Imagine, if you will, a science-fiction scenario. You may view it with delight or horror, but whichever, it is also reasonably plausible. So bear with me for a while.

The Teddy

Assume the time when the power of information technology has increased enormously, with the whole country – nay, the whole world – wired so that anyone anywhere can connect to the huge communication network. As a result, society has evolved to the point where everyone always carries a portable computer with them, except it isn’t thought of as a computer, it is thought of as a personal, confidential assistant.

Everyone would have their own portable device, all the time. In fact, suppose we started out with our personal assistant at a very early age, two or perhaps three years of age. It could help us learn to read and write, draw and sing, spell. Because the devices would be handed out early in life, the version for young children should be soft and cuddly so children will always want it to be by their side: soft, and furry like a Teddy bear — hence the name: “The Teddy.”

By starting so young, the Teddy could store within itself all the information and experiences of a lifetime. People would become quite intimate with their Teddys. It would know all about them, while also giving them complete access to the world’s databases of knowledge. I assume that by the time such devices are possible the speech recognition problem will certainly be licked, so we could communicate with Teddy by talking. We talk to it, it talks to us.

Teddys would be with their user for their entire lives. They would change in shape and form to match the growing sophistication and interests of the person, but each time someone got a new model Teddy, the information from the earlier version would be transferred to the new. As a result, Teddy would always retain a complete record of all the person’s personal experiences and knowledge for an entire lifetime even as it changed in physical form.

Eventually, as people came to rely upon their Teddys, they would reach a point where they would be disoriented without them. After all, with a Teddy, you would never be alone. You could always talk to Teddy. It would never desert its owner. It could be programmed to give reassurances, to follow progress on a task and to make appropriate suggestions. It could serve as a continual reminder of names and dates: time to do exercise, to phone home, buy gifts, etc.

With a permanent Teddy, memorization would no longer be needed: just tell Teddy all you want to remember and then have it do the remembering.

Of course, with all our thoughts, ideas, and memories within an artificial device, we would have to make sure it was always with us. Without Teddy, we could no longer function. People today can no longer memorize lengthy poems and orations as in times past, for the art of memorization has all but disappeared as writing, books, and recorders have proliferated. Skill at arithmetic deteriorates as we rely more and more on calculators. Teddy would accelerate this reliance upon technology. The technological solution is to ensure that we could never be without our Teddy: attach it to the body, much as today we attach our watches to our bodies. Maybe Teddy could even be surgically implanted inside the body so you could never misplace it. It would interact by voice, a small microphone surgically implanted near the throat, a small loudspeaker or earphone surgically implanted in the head.

Implications

The Teddy is the stuff of Science Fiction. It is easy to get carried away with the theme, to imagine results either wondrous or horrid. The problem is that it really isn’t fiction, it is very likely to take place.

Can you imagine the future? People walking around with little lights glowing in their foreheads indicating whether their Teddy was on or not. People going around apparently mumbling to themselves but actually conversing with their Teddy. New forms of mental disturbances: is that voice talking to you at night a manifestation of schizophrenia, or simply an overactive Teddy? At parties, some people would cuddle up to their Teddys and ignore the other people. Others might be openly defiant, making a show of turning off their Teddy, defying it to complain, insisting that anyone they talked with do the same. Parents might have to instruct their children “It isn’t polite to talk with Teddy at the dinner table.” All this from a device intended to help you remember numbers and dates.

We could imagine a debate about the necessity of being able to turn a Teddy off. After all, if Teddy recorded everything you said to it, it could also record everything said to you. I could imagine one scenario in which Congress might pass a law saying that to protect civil liberties, all Teddys must be able to be turned off, with some indicator so that others would know whether or not your Teddy was listening. Then again, I can also imagine a different scenario, one in which failure to have a Teddy on at critical moments led to unfortunate incidents. In this scenario, Congress might pass a law forbidding it to be possible to turn Teddys off.

Many people seem to have a need for artificial memories or for a quiet, non-judgmental confidante. The written personal diary serves somewhat the same purpose. Diaries have problems, however. They are hard work to create: many more diaries are started than are continued. Worse, because they often contain private thoughts and information, they dare not be left where others might gain access to them.

The Teddy would solve all these problems by making it easier to record thoughts, allowing voice, keyboard, handwriting, and perhaps even photography. It would be small – as small as you wish to imagine – and private, with the most advanced cryptographic techniques rendering the information useless for anyone but its owner (although I can imagine that the legal arguments about when it was proper to force an owner to reveal the contents could keep thousands of lawyers occupied for many years).

How likely is this? Very. Even today we see people carrying around their primitive Teddys. Pocket appointment books, address books, notepads. Some are relatively big and bulky, notebook sized, but too big for the pocket. More and more are electronic, tiny little devices made for entering names, phone numbers, appointments, and even memos. The one I am experimenting with is small enough to fit into most of my pockets (except then I can’t sit down), is much smaller than the address books and calendar I used to use that contained the same information, and even comes with an adapter that makes it capable of sending faxes previously typed on its tiny keyboard.

Portable computers already can fit in the shirt pocket. Voice recognition systems are primitive, but they do exist. Each year these new tiny artifacts become ever just so much more powerful. The progress may appear slow to us, whose lives span less than a century, but for the pace of human history, where a hundred years is barely noticeable, the pace is rapid indeed.

I am intrigued by the fact that we like some artifacts so much that we strap them to our bodies, such as the wristwatch. We need a watch only in a society that synchronizes all activities by reference to time.

Do we need entertainment so much that we must strap the devices that provide it to our bodies? There are some who seem never to be without their plugged-in earphones. And soon, never without their plugged in eye-set, glued to a video image of this or that, whether fact, fiction, or fantasy. Readers too fall prey to this disease. Why always with book in hand, book to the eyes? Reading while waiting, reading while a passenger, reading even at the dinner table with others present. Is it better to be with someone else’s thoughts than one’s own or one’s companions?

Why is it that some people never wish to be alone, never allow themselves a quiet, reflective moment? The popularity of portable entertainment systems seems overwhelming. Books, magazines, comics. Wearable radios and television sets. Why aren’t some people comfortable with their own thoughts, alone, in privacy.

One reason might be that isolation breeds paranoia. It is easy, when alone in the midst of night, to think of some past or future event, imagining as it was or will be, but going beyond what was there. One imagines the comments of colleagues, and the comments to those comments, comments, of course, that were never even thought, let alone said. One imagines events taking their own courses, events that never happened. Wondrous and horrid things take place in the solitary mind. It is too easy to develop a subconscious fantasy in which you are the butt of all others. A fearful spectacle in which you think the worst of all. Supportive colleagues withdraw their support. Casual enemies become dreaded villainy. The slight mishap of yesterday becomes a major disaster of tomorrow.

Or the other way around, of course. The unaided mind can just as easily fly into euphoria. A slight advance of yesterday makes you the hero of tomorrow. Slightly supportive friends become heroic champions. Disciples even. You wander through your own fantasy land as superhero, supergod.

Whichever way it is, it has lost all touch with reality. The quick cure is once again to encounter reality, to substitute the real world for that fantasized one.

What has all this to do with the Teddy? Actually, it seems an argument for a constant companion, an argument against isolation. That depends upon the nature of Teddy.

In everyday life, positive means good, negative bad. In the engineering of systems with feedback, it is often the reverse. Negative is stabilizing, calming. Positive is encouraging, reckless.

Negative feedback is what lets you drive a car at a steady pace, keep the room’s temperature at a constant value, control an airplane to maintain its altitude, speed, and heading. Set an automobile’s cruise control at some fixed speed and it sets up a negative feedback loop: Whenever the behavior of the car deviates from the desired setting, the feedback loop corrects by making it do the negative action, which translates into the opposite. Is the car starting to go too fast? The controller makes it go slower. Is the car starting to go too slow? The controller makes it go faster.

If the controller worked by positive feedback, it would be a disaster. As long as the car stayed at its assigned pace everything would work fine. This is what we call equilibrium. But whereas the negative feedback circuit yields stable equilibrium with every deviation automatically recovered, the positive feedback one is an unstable equilibrium, where every deviation leads to worse deviation.

Suppose the car controlled by a positive feedback loop starts to slow down. The positive feedback says to do even more of the same: go even slower. Having gone even slower, the positive feedback circuit notices and says to go yet slower. Eventually the car will stop. At least this is reasonably safe, even if it isn’t what was wanted.

What if the car goes slightly above its set speed, slightly too fast. Well, the positive feedback circuit will make it go even faster. And that even faster speed, then being noticed by the positive feedback circuit will lead to an ever faster one, until finally the automobile goes out of control.

Dreams and fantasies can feed upon themselves through a positive feedback loop, each turn reinforcing the hopes or the fears of the previous, until the dreamer has distorted reality into a foreign existence, far removed from what is possible or even likely. Positive feedback can lead the dreamer to lose touch with reality.

How should Teddy respond to one’s dreams and fantasies? Should it follow the route of positive feedback, always being supportive, always encouraging? Or should it follow negative feedback, always being critical, corrective.

Consider the supportive Teddy. Tell it your fears and it will confirm them, which drives you to even worst fears. Tell your Teddy your most wonderful fantasies and it will confirm them, moving you further from reality. A positive feedback Teddy could be harmful.

But a critical Teddy would not help either. If every time you dreamed of something wonderful, Teddy voiced its doubts, well, it would be like living with the mythical nagging parent-in law. It might work better on the downside, so that every time we voiced doubts and suspicions, we would be met with a reassuring response. Except that some doubts are necessary and useful. Just as some wild fantasies keep us excited, dreaming of and even accomplishing events that once were only dreams, doubts keep us realistic, forcing us to consider alternative courses of action, forcing to plan for the unexpected.

To get the correct balance of support and criticism will be difficult. Many people have never managed to find it in their relationships with others, or even themselves. How can we expect the anonymous programmers and engineers of the future who devise the Teddy to do better?

There is another argument against a continuous presence of a companion. If we are never alone, never in quiet, when would we think? There is a difference between the mental stimulation created by signals from the outside and that which is self-created. In the first case the mind can be passive, simply responding to and enjoying whatever is offered it. The scientific term for this mode of operation is data-driven processing, where all that goes on is driven by the arrival of sensory data. It is a necessary part of brain functioning, but it is externally-driven, which runs the danger of also meaning externally controlled. In the other case, the mind has to drive itself, to develop and invent new concepts and thoughts. Now the mind is active, creating for itself the images and thoughts that will occupy it. The scientific term for this mode of operation is conceptually-driven processing. This is the inventive, creative part of life. This is the mode in which new thoughts and ideas can arise. Normal processing requires both modes of operations. Excesses of one mode over the other have different implications.

Excessive stimulation – too much data-driven processing – leads to an externally-driven existence, passively accepting the guidance and information from others. A complete lack of stimulation – too much conceptually driven processing – can lead a person away from reality. The extremes of this situation are the drug-induced experiences and hallucinations created by mind-altering drugs, including alcohol. Or perhaps the extreme paranoia or euphoria of the positive-feedback driven dream state. The experiences may be enjoyable, but they are also completely out of touch with the world, with actual events. A normal life consists of a healthy balance between the two modes.

There is something to be said, however, for the quiet, reflective mode, working within the private thoughts created from within, without outside interference. This is where creativity lies. The solitary mind is often the inventive, ingenious mind. If we were never alone, we might never create. If we are never left alone, we might lose our ability to think unaided. To dream. To fantasize. And thereby to create and invent.

Perhaps sometimes we should do without specialized machines like the Teddy. Sometimes we should leave things as nature made them be. Millions of years of evolution have led us to a very special niche. The human mind is very specially adapted to its environment. It is dangerous to tinker, whether by drugs, by physical manipulation, or by mind-control. It is too complex to be understood, and the effects of the tinkering are more apt to lead to harm than good.

A Teddy could be a wondrous thing. Maybe. But will solitary thinking disappear, along, perhaps, with great creativity and invention?

Moravec’s Robotic Vision

Chapter Note: Hans Moravec’s dreams are described in his book Mind children: The future of robot and human intelligence (Moravec, 1988). A more engaging treatment is given in Chapter 5 (“Postbiological Man”) in Ed Regis’s Great mambo chicken and the transhuman condition (1990). I recommend you start with Regis.

It is easy to make a case for the synergy of human and machine, each left to do what it does best, each complementing the skills of the other. The problem comes when the machines take over from the human, taking away initiative, forcing the person to serve as slave to the ends of the machine.

The extreme case of takeover is that advocated by Hans Moravec, he of robotics fame. In his book Mind children: (subtitled The future of robot and human intelligence) Moravec predicted the day when machine intelligence would equal or surpass that of people. Moreover, Moravec pointed out, a machine intelligence should never wear out, for it is based upon information, not mechanics or biology. Machines do break of course, but when they do, they can simply transfer their knowledge to new machines. The knowledge can thereby stay around forever, for knowledge leads an existence quite removed from the machines in which it resides.

That would be alright if Moravec were wiling to stick to machines. But Moravec is after immortality: he wants human knowledge to exist forever. Not just knowledge, minds. Although Teddy may have started out as trusted aid and confidant, over time, he could become you, at least if Moravec’s vision comes true.

Think of it this way. Project yourself into the future to a time when mechanical parts have reached a skill undreamed of today. If you damage a leg or arm, why it can be replaced with a mechanical one, just as effective and functional as the original. In fact, each biological part of the body can be replaced by a mechanical one: arthritis, rheumatism, broken limbs, failing hearing or eyesight – all things of the past, for as each biological structure fails, it can be replaced with its mechanical equal. Imagine growing old, replacing each body part as it fails, resulting, perhaps in a completely mechanical body. Hard to imagine, maybe, but possible.

But what of the brain, what happens there? Why we replace it too. Now this is tricky. If you simply remove the brain and pop in a new one, whether biological or artificial, what happened to you, your mind, your experience, your very existence? Gone. Your feeling of self is woven into the structure of the brain. You can’t just pop in a new brain and continue to function. The new brain wouldn’t have an identity, or at least, it wouldn’t have your identity.

Moravec suggests we overcome this problem through a steady replacement of cells: why not simply replace each brain cell with a mechanical equivalent, carefully adjusting it so the knowledge and structural information in the biological cell is exactly duplicated by the artificial one. You would never know the difference.

See how it works? I connect a computer up beside you in the operating room. Then I peer into your skull and replace one cell with a computer circuit. I carefully find all the connections of the cell and replace them with wires to the computer, and then I adjust the computer program so that it exactly duplicates the functions of that cell. You can do the judging because, fortunately, the brain cells themselves cannot feel pain, so it is possible (and common) to do surgery on the brain while the patient is alert, conscious, and talking with the surgeons. So, I replace your cell with the computer and then switch between the two: first to the real cell then to the artificial. “Notice a difference? Yes? OK, just a moment.” I tinker with the program some more, then switch back and forth again. We simply keep doing this until you can’t distinguish between the real cell and the artificial one.

Then, zap the real one — it isn’t needed anymore — and do the same for the next. Eventually you would be in the computer, not in the brain, and you couldn’t tell the difference. The procedure guarantees that.

Of course, at some point, it would not be clear who “you” is. Which is exactly Moravec’s point. If you can’t tell the difference, why would you care? Your mind and all your experiences can live forever, moving to newer computers each time the old one wears out. Except that, well, he doesn’t really think much of emotions, so those got left out. And actually, he doesn’t think much of the human mind either, so as long as he has it in information form and inside a computer which we all know can do arithmetic and remember properly without error, why not take advantage of these capabilities and use them to improve upon the mind? Make it remember better, do arithmetic better, think more logically.

Moravec’s brain transplant scheme is very clever. Teddy takes over. Fortunately, the plan has many drawbacks.

1. It assumes that a single cell is independent enough that we could replace it, then go on to the next one. But its operation may be tightly linked with that of thousands of others (it is not unusual for one brain cell to make 10,000 connections to other cells), and the fact that it seemed replaceable under rather special circumstances — the quiet of the operating room — does not mean that it will work correctly in other situations.
2. Single cells seldom provide essential information. Remove a single cell and there will be no effect on the normal operation of the brain. That, in fact is how the brain maintains its reliability and ruggedness: when cells die it seldom matters. It is cell assemblies that matter, and these may consist of hundreds of thousands or millions of cells. Replacement is no longer such a do-able task.
3. The scheme ignores the chemical operation of cells. Basically, Moravec assumes that a cell is a cell, working in isolation, operating solely on information electrically conducted through its fibers and junctions. He completely neglects the chemical operations: the proper functioning of a cell depends upon the makeup of the chemical fluids in which it bathes. Not only that, but there is a vast chemical information processing system, huge ducts in the brain that convey important information through the composition of its fluids. Each duct bathes millions of cells. When you look at a photograph or drawing of the brain, you will see large dark areas, empty space. These are the ventricles, ducts that deliver fluids to large areas of the brain, subtle, complex mixtures of chemicals that have major impact on the operation of the brain cells. Get agitated and the hormonal glands squirt their output into the ducts, which rapidly carries them to large areas of the brain. The chemical signals can be remarkably selective, affecting only cells that have specific receptors for the particular chemical. But those cells then change their operation in a manner that depends upon the nature of the chemicals and the circumstances, in ways that are not yet understood. Moravec’s notion of replacing the brain cells one by one will miss this most important part of brain functioning. In fact, because the new brain that Moravec is creating does not have these chemical pathways, it will never work the same as the old. On top of that, even if Moravec suddenly begins to understand the chemical structure of the brain, he will have difficulty reproducing it. His scheme works because it assumes that each element works independently of the otherwise, so that if you replace it with a different device that has the same characteristics and the same connections, the system will not notice any change. But that isn’t how the system works at all: the cells do not work independently of one another, especially when chemical communication is taken into account.
4. Finally, replacing the brain cell by cell would simply take too long. That is like saying that it is possible to win at chess simply by thinking through the implications of each move. True, but there are so many implications that even the world’s fastest computers could never manage it in many lifetimes. That has been known for decades. There are at least 10¹¹ neural cells in the brain, or 100 billion (some estimates are ten times more, 10¹²). If each cell has to be is replaced by asking the person if any difference can be noticed, it will take some time to allow for a question, a reflection, and an answer, but let’s assume it can be done at the average rate of one second per cell. One hundred billion cells means one hundred billion seconds. That’s 1 2/3 billion minutes, almost 28 million hours, over a million days, or more than three thousand years. All in the operating room —- after all, you wouldn’t want to stop with half your brain in the head, half in a computer.

So the scheme won’t work. Thank goodness.

The Future

What I want is all of the virtues of machines and none of the disadvantages — the scientific version of eating my cake and having it too. After all, if I carried my own information bank, my own Teddy with me at all times, with me in control of the on-off switch, what are the deficits?

Alas, technology is always a two-edged sword, always showing two faces to the world. Every benefit has its accompanying drawback. One simple possible deficit is the danger of the tuned-out world. Look around you today and you can see the early beginnings. All those people, earphones strapped to the ears walking about, wandering through the world. Tuned in to their own sensorium, tuned out of ordinary human discourse or interaction with the environment.

I have mixed feelings about this Teddy. I can imagine the good things. I can fear the bad. But in actuality, I have little choice in the matter. It is coming into being.

Yes, some form of Teddy will be with us, but the exact shape or form cannot yet be predicted. But whatever form it takes, I predict it will be both a boon and a loss. It will lead to reasonable and sensible fears and hopes. And if we humans are not sufficiently intelligent about its design, functionality, and use, it will forever alter our lives in ways we do not want.< previous page | next page >

FOOTNOTE1. Copyright © 1992 by Donald A. Norman. All rights reserved. Originally published by Addison Wesley. Now out of print. [Return to Text]