jnd.org

Donald A. NormanCopyright © 2001 Donald A. Norman. All rights reserved.

Prologue & Abstract

People trained in the Behavioral, Cognitive, and Social Sciences (BCSS) seldom play a critical role in the development of new products. Yeah, they do user testing and sometimes take part in the design, but seldom take part in specifying the product in the first place. Moreover, when economic times get tight, they are among the first to be let go.

Why the failure? I place the blame squarely upon BCSS itself: students are badly prepared for the demands of a product-driven industry. Faculty are equally ill-prepared, and therefore unable to make a difference — assuming they would even be interested in doing so.

Current academia is ill-suited to the task. Industrial needs differ greatly from the interests and talents of the BCSS scientific community. The BCSS student is taught analysis: industry needs synthesis — design. The scientific community seeks truth and perfection. Industry needs doers, not analyzers. It needs quick, approximate answers that are “good enough” for the purpose: Engineering approximations rule the day, not precision. Mind you, the talents of the BCSS are badly needed. Most products benefit and interact with human beings: most are used directly by people. With the advent of ubiquitous communication links, social interaction, and group work independent of time and distance are a part of the work scene. All these applications will fail without appropriate attention to the human needs. But the skills currently taught within the Universities are simply inappropriate to the needs of industry.

The needs of industry are orthogonal to the needs and interests of a research-based academia. To make an impact, we need to develop an applied discipline. But an applied discipline needs applied content, much of which does not exist. We have lots of analytical techniques, but few for synthesis, for design. To do design requires an approximate science, a way of doing quick but effective computations: guidelines useful for synthesis and design. The field of BCSS applications is different from that of BCSS science. This does not mean low-quality work, it means different work, with different skills and different goals.

APPLYING SCIENCE TO PRODUCTS: FORGET WHAT YOU LEARNED ABOUT EXPERIMENTAL DESIGN

It all seems so simple. Anyone with a background in any of the Behavioral, Cognitive, or Social sciences (BCSS) can take one look at many products and find the flaws resulting from a disregard of human skills, attributes, and behavior. “Ugh,” you say, “they need help.” And so, freshly armed with a good scientific and experimental knowledge, off you go to industry, going to improve the world.

Nope. You don’t really have a clue. Could you truly design something? Try it: take your favorite badly-designed remote control and redesign it. Draw a picture of the new device, to scale, with all the new controls and buttons. Make sure you cover all the controls that people around the world will require (issue: how would you know what these needs are?). Make sure that the result is actually manufacturable, at a reasonable cost. (Issue: target cost should be about $5, but if you insist you could go to $10 for the luxury market. Issues: How would you estimate cost? Do you understand why the number has to be so low?) When you finish, test your design, using the paper prototype you have drawn to make sure that it really is more usable than the one you hate. (Issue: Do you know how to do fast, inexpensive tests, and — most importantly — to have the results within minutes or hours of completion?)

I once complained about a horrid remote control that was to ship with a product for a company. I was told, “OK, lead a team and fix it.” Wow! I assembled some of our best people from industrial design and human interface. We had ambitious plans — I am still convinced we would have produced a brilliant new design. Oops, I was then told “Oh by the way, the design must be completed in two weeks because the factory in Spain is going goes on summer vacation, and after that it will be too late to make the Christmas season.” Two weeks to do a detailed design, complete with mechanical specs and guaranteed delivery of the components to the factory. We managed to change some of the wording on the buttons. When the product was released, the product itself got a glowing review, but the remote control was severely — and appropriately — panned. That was our sole consolation.

Science is mostly analyzing. Engineering is doing. The difference between these two is profound, and it affects not only the way that your knowledge is applied, but even the culture of the workplace. Working for a company is very different than working for a university: the reward system is different, the role of criticism is different, and the value of group efforts is different.

In academia, we teach how to critique: to analyze a published study and find the flaws. We are superb at finding flaws. We are also superb at doing carefully controlled studies that minimize biases of all sorts, that are able to detect subtle differences. Problem is, these studies take months to perform, and they are designed for subtleties. Individual creativity is valued, and the reward system encourages publications, with concern about the order in which a person’s name is listed as author.

In industry, we need to provide products that offer value to consumers. Speed and cost are critical parameters. Competition with other companies is intense. On the whole, the subtleties that interest academics are of little interest to consumers: if you like, they are beneath the threshold of perception. Group work is encouraged. Criticism is frowned upon. Publication isn’t relevant: shipping a product is. There is little time for writing — each product is a rush job, and when it is nearing completion, well, the next ones are already underway.

Design is very different than analysis, and the training in traditional departments of BCSS are not very relevant. Worse, there is no place to get the relevance. One course simply is not enough. Engineering schools do have Design Curricula, but usually limited, usually in Mechanical Engineering, and even there, not with much stature. Design, if taught in universities, is often a part of the Architecture or Art curriculum. Most computer science and many psychology or cognitive science departments now do include a course on Human-Computer Interaction, but although these are often excellent, they are one optional course out of an overfilled curriculum, often taught by faculty who themselves have had no practical experience designing real products. Yes, there are exceptions: Carnegie Mellon University’s HCI department, or the program at Georgia Tech, but these are exceptions. And yes, there are departments and programs in Applied Psychology, Engineering Psychology, and Human Factors and Ergonomics, but I have seen little evidence that these programs teach the skills I am advocating.

Applied design is taught in Design Schools, but these seldom have a practical component, seldom emphasize the human side of design. There are exceptions, such as the Institute of Design (Chicago), the Computer-Related Design program at the Royal College of Art (London), and the newly formed Interaction Design Institute in Ivrea (Italy).

Two Conferences, Two Cultures

A few years ago I went to two conferences, back-to-back. First I flew into the Rocky Mountains, to a YMCA camp 10,000 feet in the Colorado Rockies. I spent four days in a productive, deep analysis of distance learning, joint work, and other important topics. There were around 40 people there — the leading academic researchers and their graduate students and leading researchers from industrial research labs, such as ATT Labs, Bell Labs, IBM and Lotus Labs, US West, Microsoft. The conference sessions were deep and rewarding. Each paper was lengthy, followed by a prepared, lengthy commentary, and a lot of time for discussion. Because we would spend from one to two hours per paper, we only discussed four papers per day.

At the conclusion of this conference I drove to Denver and flew to Palm Springs to a luxury resort hotel. Now I was at the annual “Demo” conference, attended by product developers, marketing leaders, and executives of leading technology companies, the press, and the VC community. For 2 1/2 days we heard presentations — demos — of all the latest, hottest new technology. Each presentation got 8 minutes. There was seldom any discussion, and when there was, it was about the business proposition, funding, and ship dates. A number of the presentations had to do with the same topics discussed at the YMCA camp, but the differences were profound. The one was deep and studied: abstract discussions of fundamental issues. The other was shallow and business-like: what are the markets, the costs, the distribution channels? The first conference agonized over fundamental limitations and weaknesses of the technology, the second couldn’t be bothered: “will it sell?” was the dominating theme.

Footnote: This was Demo 99 conference, but the lessons have not changed: for typical reports on the conference, see Demo-99; Demo-2000; Demo-2001; or the main Demo organization.

I was the only connection between these two meetings, and even I provided no bridge: my participation at one had nothing to do with my presence at the other. The YMCA camp discussions will lead publications in journals and will spawn further research. Future generations of graduate students will be asked to read these papers. The Demo conferences have already led to numerous newspaper articles and discussions, to numerous business deals and funding opportunities. A few of the products shown at Demo will succeed and lead to numerous copies of the concept. Most will fail, but that’s OK: that’s the way the game is played. Think of it as research.

Remember that I pointed out that the YMCA camp and many of the Demo products were on the same topic. Lessons from one were very relevant to the other. Yet there was no communication between the camps. I once observed that in academia there is much talk, little action. In industry, there is much action, little thought.

Resources and Leaders

I recently consulted for a company, where a friend described the company politics in terms of “resources” and “leaders.” A “resource” is someone hired to do a specific task. Resources are replaceable. Hire one today, another tomorrow. The lower tiers of a company are populated by resources.

A “leader” on the other hand, is an irreplaceable asset. A leader is someone who adds a special mix of talents, who makes it all happen, who galvanizes the company.

In industry today, the job categories populated by people from BCSS are thought of as resources, never as leaders. HCI, psychologists, anthropologists, social scientists, human factors and ergonomicists: all are resources.

“They,” those who matter, the “leaders,” design the system, then call in us, the user interface designers, the user testers, and the human factors groups to verify the work that has already been done. And, of course, the technical writers are brought in, at the end, to explain it. Notice the “they” — them versus “us.”

We will never make progress as long we are resources and not leaders. Resources don’t discuss the business plan, the marketing strategy. Resources don’t help decide what the product or service will be in the first place. Resources are called in when the leaders think they are needed. They do their job and then get out of the way.

How do we get respect? By becoming leaders. We have to speak the language of business — schedules, profits and loss, sales, and margins. We need to understand why products sell, who buys them, what the distribution channels are. In other words, we need to learn the language of business, of finance, and of marketing. We have to put the system first, not our special skills.

When engineers do this, they get promoted. They move up to management. MBAs are taught to do this. Those from the BCSS community must do the same. We will not have impact until it is no longer “them” and “us” but just “us.”

WORKING IN INDUSTRY

On the whole, it’s fun, challenging rewarding.

Pluses:

• The joy of seeing one’s products in stores, reviewed in popular magazines and newspapers, and used by people.
• The feeling of accomplishment.
• Working as a group, where everyone is pulling toward a common goal.
• The financial rewards are pretty good, as well.
• Travel, although frequent, is made less onerous than in academia because it is considered part of work, so it can be scheduled during daytime hours (although see the story under “minuses”). Weekends are considered private, so meetings are usually scheduled during weekdays, and often so arranged so that travel on Saturday or Sunday is not required.
• Opportunities to meet a wide variety of people, other companies, other cultures.
• Managing teams. Being a manager, in industry is a plus. It allows one to be in more control over the work, to have a greater scope of influence. The more senior the position, the more interesting, for strategy is done at the higher levels. Higher is better, at least for me.

Minuses:

• No job security. Economic or business conditions could cause your job to be terminated at any time, sometimes with no warning.
• Politics do exist — everywhere: people are people.
• Bosses really do boss. Some are wonderful, some horrid.
• Less freedom to do the tasks that truly interest you: company needs dictate.
• Your life is not your own: you may be asked to travel around the world on a moment’s notice. (“By the way,” I was once told at a meeting I was suddenly called into, “you have a meeting tomorrow morning in New Jersey.” (I was in California). “Your tickets have already been arranged.” I had to dash home, pack a suitcase, and get to the airport. I managed a few hours sleep on the airplane because I also had to read a thick pile of briefing material to prepare for the meeting.)
• No deep intellectual discussions. Who has time?
• Managing teams. Being a manager takes you away from the fun part of work — from hands-on design and test. It means more meetings, more concerns over budgets and schedules. And management means dealing with the personalities and disagreements of the team: management is mostly applied clinical psychology, which is not what you might have thought you were getting into.
• You think professors work hard in Academia? You think they travel a lot? Hah.

There is a large difference in the working place. Academics emphasize individual work. Intense criticism is encouraged — required even. One separates criticism of an idea from criticism of the person. The pay is relatively low by industrial standards, but the research time is controlled by the individual. Universities have more committee work for the entry-level worker (Assistant Professor) than does industry. In industry, committees are mainly delegated to management. On the other hand, in industry one reports to management, who specifies the work to be done. If the manager kills a project or transfers you to a new project, you go along. Not so in academia.

Industry rewards joint work. Whereas in academia, working for the good of the university can get in the way of publication, and therefore promotion, the opposite is true in industry. A side-effect is that intellectual criticism is muted: criticizing an idea is equated with criticizing the person. So the intellectual give-and-take of academics is missing.

The job security is industry is much less than that of academia. Even an assistant professor is given 3 to 5 years to prove themselves. In industry, a job layoff can come unexpectedly, effective immediately, with a few months salary as compensation. In academics, more senior members have tenure.

In Academia, there is a continual struggle to get research funds: sometimes it feels like writing proposals is a never-ending effort. Moreover, there are multiple jobs: Teaching, publishing, advising, mentoring, committees, publishing, research, publishing, attending conferences, writing grant proposals, publishing. Industry does not have these burdens: funding is relatively straightforward. Yes, a justification is required, but it is almost always granted, sometimes with appropriate negotiation. But funding is assured: after all, if they hired you, they expect to have to pay for you to get your work done. When they lose confidence in you, or when funding dries up, well, you are simply fired. (Firing is usually called “laid-off,” with a layoff package that includes some salary and assistance in finding other jobs. In good times, being laid off can be a monetary coup. In bad times, well, it is bad.)

AN APPLIED CURRICULUM FOR THE BEHAVIORAL, COGNITIVE AND SOCIAL SCIENCES

Caveat: Curricula are controversial objects. This one is neither perfect nor intended to be: The purpose of this section is to illustrate the sorts of issues that a practitioner needs to know. It needs to be modified in numerous ways. Take this for the spirit of the enterprise, not the actual form.

What do you need to know? An amazing amount of stuff. First, there is the information from BCSS itself — even though you may have specialized in some narrow topic in the university, once at the company, you are assumed to be the expert on absolutely anything dealing with people. Here is an example of things I have been asked to give instant, expert advice on.

• Human memory.
• Human error.
• Learning — learnability. Pedagogical styles.
• Technical manuals, help systems, and error messages. How best to explain things.
• Design of keypads and buttons.
• Individual differences.
• Cultural differences — the technical term is “localization” — making something developed in the United States work across all the languages and cultures of the world. I am supposed to be an expert on whether or not this product will work in Farsi, or with workers in India, or Brazil.
• Personality tests. Suddenly I am supposed to pass judgment on clinical and personality assessments that I shunned in the university — I was, after all, a cognitive psychologist, not a clinical one, and we didn’t look at individual differences, or clinical assessments.
• User testing methods, from think-aloud protocols, to focus groups, to questionnaires. Ethnographic field research. Video technology for recording and scoring.
• Speech recognition and text-to-speech output.
• Input devices for Chinese and Japanese, both keyboard and spoken.
• Pattern recognition — fingerprints, iris scans, facial recognition.
• Group work. Bulletin board and discussion board design. Chat tools.
• Layout of panels, design of the power switch for the Macintosh.
• Typewriter and numerical keyboards.
• Accessibility issues for the handicapped — or better — universal design.
• HTML and XML coding.
• APIs (Application Programming Interfaces, the tools given to developers).
• The telecommunications industry.
• Cable TV. Set-top boxes. Spectrum allocation. The High Definition TV battles (I testified before the FCC on these issues).
• Distribution channels for products.
• Business plans.
• Dealing with legacy problems.

No curriculum can hope to cover all these issues, but it can provide a background of skills that let the person grow the required skills. At the least, it must cover these areas: BCSS principles, Methods, Design, Business, and Marketing.

BCSS Principles

Methods for applying the scientific knowledge of BCSS to the needs of industry. The point is to transform the scientific knowledge and the observational and experimental paradigms into an applied, engineering discipline. Thus, the practical implications of the findings are stressed, including approximate methods for making use of them. This is difficult, because the relevant scientific fields have made no attempt to package their findings in this way: new advances, new compilations, and new approaches will have to be devised.

The range of fields is large: anthropology, cognitive science, education, psychology, and sociology. Each discipline includes a number of sub-disciplines — e.g., in psychology there is clinical, developmental, cognitive, experimental, perceptual, psycholinguistics, social, etc., and different areas are relevant for different industries. I am certain I have left out areas.

What is needed is good overviews of the fields and deep understanding of the major findings and approaches. Detailed, specialist knowledge is not required.

Toward an Approximate Science

I have long advocated the creation of an approximate science, one that is truly useful in design (e.g., Norman, 1998, p. 193-201). There are several examples, including the pioneering work of Card, Moran, and Newell (1983) on GOMS (Goals, Operators, Methods and Selection rules), a set of useful engineering approximations for some aspects of human-machine interaction . GOMS is a valuable first step, but its rules are at a micro-level, useful for those occasions where one wishes to minimize time to do a task, but not appropriate to determining what tasks to do in the first place, what human behaviors to support, and how to structure the system. All engineering disciplines have useful approximate methods, usually linear models of the system under design that allow quick computations to be performed, even at the risk of some inaccuracy.

Design is a complex business, requiring a mix of skills and talents. More importantly, successful product requires good design skills coupled with analytical abilities and a good business sense — knowing the factors that cause people to buy a product, understanding the technological strengths and limitations, and being aware of the economic climate. One of the hardest jobs in a company is that of product manager, balancing the conflicting demands of the marketplace, of engineering, of marketing, of usability, aesthetics, cost, and time. The successful worker understands these conflicts and knows how to overcome them. But this requires a larger product view of the product and the business than is offered from the vantage of just one discipline. The person who focuses upon one dimension, be it usability, aesthetics, cost, technology, or marketing will fail: the successful product requires a balance: the successful worker understands this.

Methods

Every discipline has its own collection of methods, and the practitioner must know them all. The least valuable are the traditional experimental designs of experimental psychology. These are large, complex studies, aimed at teasing out relatively small effects. Industry needs fast, inexpensive studies that can find large effects quickly and avoid major mistakes. (Minor mistakes are often acceptable.) As a result, the standard balanced designs, with large numbers of subjects, and detailed statistical analyses are not only seldom needed, they are counter-productive. In the product industry where I work, if the result isn’t big enough to show up with a few subjects and minimal statistical analysis (i.e., means, standard deviations, medians, trend lines), it probably isn’t worth worrying about.

Observational methods are critical. I have advocated a kind of Rapid Ethnography, borrowing the term from Anthropology, but trying to get the time frame down to days rather than months or years, and allowing interaction and manipulation of the environment. After all, in product companies, the goal is to understand how best to make changes that will have positive impact upon people’s lives (Norman, 1998, p. 194-5).

I advocate observation over interviewing, but interviewing skills are also be necessary. One must know how to develop questionnaires, how to interview, and how to moderate a focus group.

I do not advocate much video or audio recording. Video is really valuable in convincing those who were not present of the validity of your findings

User testing requires a mix of skills from all the social sciences, modified to meet the time demands of industry.

Rapid prototyping, with continual user testing is yet another skill that needs to be developed.

The applied field has developed a number of useful analysis tools, including heuristic methods, expert walkthroughs, and discount engineering. Contextual Design has proven to be a valuable method for analyzing user behavior (Beyer and Holtzblatt, 1998). Many card-sorting, categorizing, and clustering tools are very valuable.

Basically, knowledge of a wide variety of data collection, analysis, and observational methods, together with the skills to improvise and modify as needed are critical.

Design

Design skills allow one to take the need (either handed to you by the marketing department or — ideally — discovered through your field observations and ethnographic studies) and design a product or service to satisfy the need, taking into account all the business considerations such as marketability, cost, time to market, etc. A Design curriculum includes numerous topics.

The Stanford University product Design program (in Mechanical Engineering), describes its program like this:

“It is a program offered jointly with the Art Department and has graduated over 250 graduate students and many more undergraduates. Product Design concerns itself with conceiving and designing products for the benefit of society. This process requires resolution of constraints arising from technical, aesthetic, human, and business concerns. A product designer uses his or her creativity, imagination, and technical knowledge to satisfy these requirements and create products to satisfy human needs. This program offers both a BS and an MS degree. Applicants to this program are required to provide a strong art and design portfolio in addition to their technical qualifications.” (Quotations from The Stanford website on Product Design)

Our needs overlap these issues, but are not necessarily identical.

Business

The most important thing to understand in business is the need for profitable products. It doesn’t matter how great the product if it doesn’t make a profit: an unprofitable company cannot survive. This means that attention to profit has to be a major consideration in all one’s work.

It is important to be able to speak the language of business. You should learn about finance and accounting, about business plans, and about product introductions, marketing, and distribution. All are essential components of a successful product.

My basic list includes:

• Product Cycles
• The Product Process
• Patterns of Adoption (Diffusion of Innovation)
• Managerial Finance
• Managerial Accounting
• The Business Plan
• Cost structures
• Learning curves and pricing
• Competition
• Competitive strategy
• The Sales Cycle
• Distributors, sales force, direct marketing. Channels
• Ethics and Law
• Sales and Marketing

Although sales and marketing are subsets of business, they are important enough to deserve special attention. Marketing usually believes it “owns” the customer (a point of contention with the sales force). This often brings the group into conflict with the BCSS contingent, who also think they “own” the customer. Worse, the tools of marketing are those of surveys, questionnaires, and focus groups, almost always the techniques that are derided by the BCSS community as lacking in validity.

The goal is to transform the relationship with marketing into a positive, supportive one. Marketing does understand why customers purchase items, which may have remarkably little relationship to their actual needs or their actual behavior with the products.

The BCSS community is best equipped to fulfill real customer needs and to understand exactly how customers use the product. This sometimes gets in the way of marketing, who wants extra features and promotions that the BCSS community can show are detrimental to use, or who downplay design aspects that BCSS thinks are critical.

The trick is to learn the concerns and goals of marketing, the better to work cooperatively rather than to fight. Marketing is right about one thing: if a product exactly meets consumer needs, but is not being purchased, it has failed. So making sure that the product is actually purchased is a number one priority, even if it means degrading its usefulness and usability.

Principles of Marketing:

• The four Ps (Price, Product, Promotion, Place)
• Consumer lock-in
• Switching costs
• Bundling
• Competitive strategies
• Advertising
• BBrand

ENDING COMMENT

Industry and Academia: Which is better? Neither: each is different. I’ve done both, and I’m glad I did.

I know academics who have gone into industry and hated it — no job security: people are not treated well — out you go, with hardly any notice, even if you have done a good job.

I know industry people who have moved to academia and hated it: Too much politics. No group support. Too much emphasis on trivia, on publication, on credit. Too many committees. The tension of passing the tenure hurdle, plus the never-ending fight to get — and keep — research funding.And I have known people who have moved back and forth between the two — I’m an example — liking both.

REFERENCES

Card, S. K., Moran, T. P., & Newell, A. (1983). The psychology of human-computer interaction. Hillsdale, NJ: Lawrence Erlbaum Associates.

Beyer, H. & Holtzblatt, K. (1998). Contextual Design: Defining Customer-Centered Systems. San Francisco: Morgan-Kaufmann.

Norman, D. A. (1998). The Invisible Computer: Why good products can fail, the Personal Computer is so complex, and information appliances are the answer. Cambridge, MA: MIT Press.