jnd.org

I agreed to give a keynote address at the “21st Century Transmedia Innovation Symposium“. Normal dictionaries do not have the word “transmedia,” but Wikipedia does. That definition introduced me to many other words that neither I nor my dictionaries had never before heard (for example, narratological). Strange jargon aside, I do believe that there is an important idea here, which I explore in this column.(Intelligible discussions can be found in the books and articles of Henry Jenkins (2003, 2006).) This article is published in ACM’s Interactions, volume 17, issue 1_._

We live in exciting times. Finally, we are beginning to understand that pleasure and fun are important components of life, that emotion is not a bad thing, and that learning, education and work can all benefit through encouraging pleasure and fun. Up to now, a primary goal of product and service design has been to provide useful functions and results. We should not lose track of these goals, but now that we are well on our way to doing that for an amazing variety of goods and services, it is time to make sure that they are pleasurable as well. Not only does this require emotions to be a major component of design thinking, but we must incorporate action as well, actions that use the whole body in movement, rhythm, and purpose.

In the bad old days we learned that thinking – cognition – was king. Emotion was bad. We were encouraged to memorize, to study, to think abstractly in words: reading, writing, and arithmetic prevailed.

But that is not how people have evolved. We are living animals, creatures with bodies, with legs and arms, eyes and ears, taste and odor sensors, vestibular and feeling systems. We use our bodies to understand the world: we learn from concrete experiences, not from abstractions: abstraction comes last. If cognition is about understanding the world, emotion is about interacting with it: judging, evaluating, and preparing to engage.

Games are the natural way we explore the world. Modern games are engaging, entertaining, and filled with learning experiences. They require thinking and acting, cognition and emotion, body motion and mental creativity. Games ought to be how we learn in school. Teachers should learn along with students. The key term here is “Engagement.”

Transmedia is a strange beast. It comes from the world of commerce, where different people and companies used to own different parts of our experience. Transmedia talks of the new emergence of multiple media in common pursuit of a story or experience. Alas, it still focuses upon corporations, companies, profit making, and ownership. It mainly speaks of how companies tie together movie releases with videos, games, books, and websites. Blogs and tweets, social networking and telephone calls. Yes, this is a clever use of multiple media, but it is still based upon a distorted view of commerce: We make it, you consume it. The media moguls think of this as a one-way transmission: they would have their companies producing, with us everyday people consuming. Why the asymmetry? We should all be producers. We should all have a say in what we experience.

Let transmedia stand for those multi-sensory natural experiences: trans-action, trans-sensory. Let it stand for the mix of modalities: reading and writing, speaking and seeing, listening and touching, feeling and tasting. Let it stand for actions and behavior, thought and emotion. My form of transmedia has nothing to do with companies and formal media channels. It has everything to do with free, natural powerful expression.

There is another side of this new transmedia: co-development, co-creation, co-ownership. In this new world, we all produce, we all share, we all enjoy. Teacher and student learn together achieving new understanding. Reader and writer create together. Game player and game developer work together. This is the age of creativity, where everyone can participate. Everyone can be a designer. Everyone can be involved.

The personal computer revolution has been both liberating and restricting. We have gained access to powerful technologies for communicating with one another, for creating art, music, and literature. Everyday people could do extraordinary things. At the same time, we were trapped by the confines of a keyboard, mouse, and screen. Instead of actively engaging the world, we spent our days in front of keyboards and screens, typing and pointing.

Today, we are moving beyond the constraints of the mouse, screen, and keyboard. Now we can merge all the benefits of the information revolution with the benefits of movement and activity. We can post notes on buildings where only the intended receiver can see them, or we can let everyone see them, whatever we wish. We can play games or hold meetings with people all over the world, moving, gesturing, and acting.

Products used to be designed for the functions they performed. But when all companies can make products that perform their functions equally well, the distinctive advantage goes to those who provide pleasure and enjoyment while maintaining the power. If functions are equated with cognition: pleasure is equated with emotion: today we want products that appeal to both cognition and emotion.

CONSUMING VERSUS PRODUCING: SPECTATOR VERSUS CREATOR.

There is a major difference between the experience of consuming versus producing, or if you will, between being a spectator and being a creator. In the traditional view of media, most of us are consumers. Artists and companies produce, the rest of us consume. We are spectators.

There is nothing the matter with being an audience, a consumer, or a spectator. It is how we have come to enjoy the great works of art and literature. We go to galleries and view, theatres and watch, libraries and read. We can be casual or engaged, watching from a distance or becoming deeply embedded in the events of the painting, music, opera, video, or book. We can become emotionally involved, weeping or laughing as the scenes unfold.

But there is a great difference when we are actually engaged in the activity, whether as producer, participant, or creator. When playing a musical instrument, I am producing and all the senses are involved. I am engaged with the music and the playing. I feel the sound pulsating through my body. My mind is completely engaged with the music, not only with the emotional aspects and the sound, but also with the physical and cognitive complexities of the mechanics of playing. To me it is simultaneously frustrating and pleasurable. To the listeners, it is probably awful, but I am not playing for them, I am playing for myself.

The same holds true for the objects of our lives. We can purchase them in stores, bring them home and either display or use them. They may give pleasure. But contrast this with objects that we ourselves have created or, perhaps, co-created.

Consider the old story so beloved in introductory marketing courses about the introduction of cake mix. When the Better Crocker Company first introduced a cake mix, so the story goes, it was supposed to revolutionize the making of cakes. Instead of hours of toil, one only had to open the package of cake mix, add water, and bake. The result was a simple, satisfying cake. But the product was not a success. Housewives (which at the time was the target audience – college students and single people were not then considered a market) rejected it. After a bit of market research, the Betty Crocker Company realized that they had made the mix too simple: there was no pride of ownership. The cake could have been purchased at a store. It tasted fine, but it wasn’t truly made at home, even if it was baked at home.

The solution was to modify the recipe to require the addition of an egg. This worked: sales soared. Requiring a bit of extra labor gave the cook some feeling of accomplishment, a feeling of being the producer.

Today, a reasonable number of products are designed to require work and effort on the part of their possessor. IKEA furniture has to be assembled by the recipient. Harley-Davidson motorcycles are customized by their owners: many take their bikes straight from the dealer to the custom house, and even though they themselves do not do the customization, they spend considerable time and thought specifying just how the finished bike shall look and behave. Similarly, many home electronics devices are customizable, with personalizable “skins,” adjustable features, add-on components, and hand-painted exteriors. So too with automobiles. One could argue that part of the popularity of social sites is that they are personal: one is sharing personal ideas and thoughts.

But how much of this is creative? How much requires commitment and concern, deep thought and effort? Most of this is the simple following o instructions, whether for a cake or a chair. Or customizing an automobile by choosing among predefined options such as color and fabric. None of this is truly creative, none of this is truly meaningful.Adding an egg to a mix that didn’t really need one makes use of the clever psychology, but it is not what I call being truly creative. The cake mix, with egg or without, is mindless. Read the instructions and follow them: everyone’s mix produces the same result. Following instructions to assemble furniture does not qualify, but mixing and matching furniture parts to create something personal, something special does. So too with the customization of the Harley bike. Even though the customization is actually done by the specialists in the shop, the specification and design relate to the specific needs and aspirations of the bike owner.

Music mashups qualify. Here, one takes samples of existing music and mixes them to create a truly novel experience. The result may sound awful or wonderful, but that is not nearly so important as the act of creation that is invoked. The world of “Do it yourself” or “make” relishes in creativity and imagination. Mashups work across all media, sometimes producing spoofs and satire, sometimes truly useful and valuable results.

Here is a simple example of a mashup that, although not deep and profound, does reveal cleverness and a sense of humor, creating a clever spoof of two very different events. The first event occurred during the televised presentation of an MTV Video award. Just after one award had been announced, someone (Kanye West) jumped on to the stage to complain that a better performer, Beyonce, had been passed over. The second event was a major speech on healthcare by President Obama to the United States Congress. Obama’s speech was interrupted by a congressman who shouted “you lie.” An enterprising mashuper recognized the similarities of the two interruptions and quickly combined components of the two videos so that the complaint about Beyonce was inserted into President Obama’s speech. As a result, now one can watch president Obama delivering a speech on healthcare with a heckler interrupting to say “I’m a let you finish, but Beyonce had one of the best videos of all time,” to which Obama calmly responds, “not true.” That is mashup as satire. Mashups don’t have to be satirical, of course: when someone takes census data, overlaps it with police reports, and enters all on to a city map, that is mashup as meaningful and important.

Good games can also create meaningful participation, meaningful experiences. Whatever the form of game – athletics and sports, cards, board games, video or computer – the players are simultaneously creating the experience. Perhaps this is why they are so engrossing. They provide a transmedia experience where people are simultaneously spectator and performer, and in the case of many games, using all of the senses, all of the body.

New technologies allow creativity to blossom, whether for reasons silly or sublime. Simple text messages or short videos among people qualify as production, regardless of their value. This new movement is about participating and creating, invoking the creative spirit. This is what the transmedia experience should be about. All of these experiences are allowing people to feel more like producers and creators rather than passive consumers or spectators.

THE DESIGN CHALLENGE: ACTIVE, PARTICIPATORY TRANSMEDIA

Transmedia experiences are not particularly new. Consider an opera, a musical comedy, a Hollywood (or better, a Bollywood) extravaganza, or an amusement park. All of these are experiences that cut across the media: sight and sound, motion and emotion. But all of these involve a transmitter of the experience and a passive audience. Creation is not new. Artists and craftspeople create. Amateurs artists and musicians create. Game players create. But in all of these activities, there are still creators and viewers. Moreover, the creativity is often limited, much as it is limited in so-called “personalization” of software or IKEA furniture: it is limited by the desires of the manufacturer. What is needed is meaningful, thoughtful creation and participation.

Jon Kolko examined this point in a thoughtful essay in Interactions Magazine. Assembling IKEA furniture is not a display of creativity, nor are any of the standard selections of items from a menu that go along with simple personalization or customization choices offered by manufacturers or websites. A simple, mindless twitter is not creative. True creativity requires some thought, some work, some effort. It has to be reflective, even if only after the fact. Mindless creativity has its place, but the real challenge before us is to unleash the substantive creativity inside most people.

The new design challenge is to create true participatory designs coupled with true multi-media immersion that reveal new insights and create true novel experiences. We all participate, we all experience. We all design, we all partake. But much of this is meaningless: how do we provide richness and depth, enhanced through the active engagement of all, whether they be the originators or the recipients of the experience?

How will this come to pass? What is the role in everyday life? Will this be a small portion or will it dominate? Will it even be permitted within the confines of contemporary commercialism? Those are the significant design challenges.

Don Norman wears many hats, including co-founder of the Nielsen Norman group, Professor at Northwestern University, Visiting Professor at KAIST (South Korea), and author, his latest book being The Design of Future Things. He lives at jnd.org.

• Jenkins, H. (2003). Transmedia storytelling: Moving characters from books to films to video games can make them stronger and more compelling. Technology Review. http://www.technologyreview.com/Biotech/13052/
• Jenkins, H. (2006). Convergence culture: where old and new media collide. New York: New York University Press.
• Kolko, J. (2009). On creation and consumption. Interactions, 16(5), 80-80.

Column written for Interactions. © CACM. This is the author’s version of the work. It is posted here by permission of ACM for your personal use. It may be redistributed for non-commercial use only, provided this paragraph is included. The definitive version will be published in Interactions.