Course

Continuing my infatuation with the method of loci, I presented a walk around panorama that introduces the image schema for user interface design of videogames. Here are the slides for the 14-screen panorama. Compared to my previous panoramas, I think the parsominious and iconic content and black background avoided the problem of overloading the user with multimedia. As you might guess, all slides were composed of copy-and-pasted shapes from Pac-man and Ms Pac-man.



If you are curious (and contact or comment), then I'll decipher the images by writing the verbal content of my talk. In any case, below are links that I found illuminating.

The term "image schema" itself makes more sense when an applicable etymology (or at least a mnemonic device) is teased out. Schema in psychology refers to plans derived from patterns that will lead to behavior. I suspect, but am not certain, that "image" in image schema most closely matches the meaning of image in mathematics, and has no other meaningful association with pictorial images. An image is the target of a function for mapping one domain to another. Domain mapping is one popular frame for discussing conceptual metaphors.



Last summer, while researching theoretical foundations for my thesis, I became introduced to image schemas in Jerome Feldman's excellent summary of the state of research in embodied cognitive linguistics, From Molecule to Metaphor. The notion of an image schema first became popular among cognitive linguists, for its nearly comprehensive ability to decipher how humans think about words. In the linguistic context an image schema explains how abstract concepts are mapped onto a template scenario that may be physically simulated. A handful of image schemas can explain how your mind processes much of the conversation that you listen to and generate.

But image schemas explain even more than language; they explain many kinds of symbolic behavior. Image schemas may be observed in the interface design of virtual reality and has framed the design of some tangible interaction. The artful application of appropriate image schemas can enhance concept communication in graphic design. Since I design games, I noticed a spooky parallel between image schemas and user interface design, such as the primacy of the source-path-goal in the level design of Super Mario Bros, the in-out schema in Go's territories and Diablo's health display and inventory system. I suspect that many effective user interfaces leverage image schemas to communicate efficiently to the user's subconscious mind.

Yet I'm a designer and not a cognitive scientist, so would appreciate your comments and corrections that lead to a more accurate understanding of how to apply image schemas to the design of interactive media.

Today while playing Puzzle Bobble around the corner from me, I noticed that my loss felt a little bit random, even though playing it convinces me it is deterministic. The sense of randomness, yet with exposed mechanics, reminded me of RJ Layton's gumball pachinko prototype, which I played with and was wondering what about its design inclined me to play. I saw the buckets below the pegs, wheel, and bumpers. I speculated on how I could land a gumball in the highest scoring bucket (in the center, flanked by the losing buckets). I could choose which horizontal position to roll the ball down, and I had a little influence on making a low risk play on the side or high risk in the center (which Masahiro Sakura discussed much better than I can).



To speculate as conjecture, I'll hypothetically suppose that part of the motivation to interact with RJ's pachinko prototype was an illusory belief about the control over the outcome. From events where control really does exist (such as tossing a ball in beer pong at the beginning of an undergrad party), to where it doesn't (such as tossing the ball at the end of the party), there seems to be a feeling that if you can observe the results of your input, then there is a subconscious assumption of control.



The phenomena of illusory control through perceptible dynamics seems to have plenty of ubiquitous examples, from the flip of a coin (when its result is called in the air) to a fair game of rock, paper, scissors. In dice games, such as Craps or Dungeons & Dragons, the players have a superstitious belief in the user's influence on the die roll. In card games and even non-game card rituals (such as Tarot card divination), there is a subconscious sense of either control or destiny, too, but it is not as strong as the feeling that particular gesture on shaken fair dice (barring barrel rolling techniques) will have a significant influence on the result.


Whereas in basketball, beer pong, and Worms Armageddon, there is control.

If this hypothesis were broadly applicable, one would expect that systems of chance, where the mechanisms of chance were exposed as a visually and aurally consonant dynamics with perceptible origins with the user's tactile input (such as the roll of dice or bounce of a pachinko ball) then the user would be more invested in the outcome than if the mechanism were unexposed. In casino slot machines, the spinning of the wheels might inject a sense of control. In roulette, the bouncing of the ball (unless you're Hiro Nakamura) is outside of control and designed to be unpredictable.



Psychologists have posited that random reinforcement schedules elicit trained responses. Some, such as John Hopson have suggested the application of the theory of reinforcement schedules to designing reward mechanisms. Psychologists might say that such superstitious belief in control is due to the neural mechanics of learning, whereby results are causally associated with behavior, even when the results are chaotic.



While visiting my son in Kentucky, his little sisters of ages 5 to 8 would engross themselves for several minutes with Cascade on my Psion Revo. The game can be played skillfully, but only the oldest (Nature) would play it that way sometimes. The youngest (Eudora) would stipple the screen with the stylus until some adjacent balls popped. She kept at it earnestly for several minutes, in hopes of a high score, and yet poking almost randomly, game after game. Her and her sister Haven would apply such sincere determination to turning over the cards in Sorry Junior, giving the act of turning more attention than the shuffle. The behavior of such children, I believe, illustrates principles of the subconscious psychology of adults; in this case, focusing on interactive dynamics, deterministic or not.



It's ironic that one company, such as Crow River, services parts for pachinko, pinball, vending machines, and trade simulators. What could be closer to laboratory operant conditioning than a candy vending machine?

Around the block, I often see children and young adults manipulating a crane to to fetch a stuffed animal. As a function of the crane's starting horizontal position, I've thought the outcomes were close to chance. But there seems to be a commercial connection. As an example, Birmingham Vending is the same company that offers slot machines, popcorn machines, and pool tables.



It seems that not only is the apparent nature of the mechanisms--or more precisely the methods by which the user interface represents the simulation--it seems using a mental model that humans can subconsciously process with high fidelity (such as kinetics), that then even random results can mistakenly feel controllable. This is not news to those familiar with superstitions of boxers, baseball players, businesspeople, gamblers, and medieval medicine. But it suggests to game designers that visualization of quasi-stochastic simulation mechanisms through the user interface can impact the player's belief of control, and therefore their investment in monitoring the result of their input. If the user feels an illusory sense of control over the outcome through visual, aural, and tactile perception of the dynamics of seemingly deterministic system (such as the toss of a die), then the user is rewarded by observing the procedure that creates the outcome.



The Price is Right (of which my grandmother was a devotee and audience member of, even though she lived in North Carolina) offers other examples, from Plinko, a derivative of Pachinko, to its big wheel. Speaking of wheels, Wheel of Fortune offers its own example of perceptible dynamics for a stochastic simulation. From the spinning wheels of slot machines to the Mario Kart 64 power-up spinner.

In WarioWare Smooth Moves Darts, there is a bit of randomness to the dart toss (displayed as a pulsing circle), which is mostly skill based, yet with some perceptible randomness. This randomness is more of an edge case. What is a closer to a confirming example is cutting the yellow or red strings at the end of the lifeline race, when they are all the same color.



While considering the gulf between aesthetic requirements for educational versus entertaining software, I considered that Guitar Hero is effective and entertaining because of the illusion that the user is controlling the song, that the user is responsible for more of the music than she has control over. It is this illusion of control that both encourages play through rapid rewards, and short-circuits the difficulties and precision-tuning exercises of guitar playing. Rock Band extends this thesis from the guitar to drums and vocals: make believe is enhanced through perceptible interactive dynamics of tracks and forgiving feedback.

What are some more telling examples in videogames, and even better, what are some edge cases or counter examples?

In the USC Game Innovation Lab, there are Game Developers Conference audio recordings for 2003, 2004, and 2005. Around November, I happened to be showing someone around and looking on the shelf and found this treasure trove. Over the course of the next three months, I listened to all of the game design lectures and a few of the production, programming, visual arts, audio, and business lectures.

There were many good lectures from GDC 03-05. Out of about 120 hours of game design and 30 hours from other tracks that I've listened to so far, here are a dozen of the cream of the crop, which I would recommend listening to, if you don't have time to listen to them all. It's my personal take, but these are the lectures that educated me the most about design of interactive media; not just games, but interactive media for a variety of aesthetic requirements that lie outside the domain of videogames.

A few of the lectures were not audible, so if your favorite is not here, I might not have been able to hear the recording.

It was tough to pick just twelve; there were twice as many that have changed my opinion on an aspect of design. In terms of mastering the fine art of designing interactive media, I found a lot of gems in the following, listed roughly in ascending order of elucidation:

What Makes Music for Games "Music for Games"?
Chuck Doud, Clint Bajakian, Jack Wall, Jared Emerson-Johnson, Peter McConnell
Game Developers Conference 2005 Audio Track
How to map music for interactivity, such as layering, recombining, sequencing, and the genres of music prevalent during play.

Interface Design: Don't Make Them Think or Read
Brent Fox
Game Developers Conference 2005 Visual Arts Track
Usability for beginner and expert users, menu design, applications of cognitive load, how to acquire attention, and button design.

The Early Months of Spider-Man 2: Using Preproduction to Invent Gameplay
Jamie Fristrom, Tomo Moriwaki
Game Developers Conference 2005 Production Track
Great designer is probably right 2 times out of 5, so is probably wrong 3 times out of 5; therefore, prototype. Prototype risky features first. Mark Cerny production method helped.

Orthogonal Unit Differentiation
Harvey Smith
Game Developers Conference 2003 Game Design Track
Units, or actors in a game should have qualitative, non-substitutable differences. To deepen tactics, alternate units should be independent variables.

Killzone's AI: Dynamic Procedural Combat Tactics
Arjen Beij, William van der Sterren
Game Developers Conference 2005 Programming Track
Evaluating cover points, waypoints, potential paths, based on cover, shot availability, and risk.

Intuition and Intellect: Deconstructing the Design of Oasis
Marc LeBlanc
Game Developers Conference 2005 Game Design Track
Select a thesis of design. mechanics, dynamics, aesthetics. drama decomposed into uncertainty and inevitability.

Dynamics for Designers
Will Wright
Game Developers Conference 2003 Game Design Track
Stocks and flows, cellular automata, and solving the right aesthetic problems.

Leker 20-20 Design Vision
Game Developers Conference 2003 Game Design Track
Fail fast. Fail often. Construct a hypothesis and draw conclusions in order to make changes that evolve the design.

Mano a Mano: A Primer on Melee Combat in Videogames
Luis Barriga, Rob Gallerani
Game Developers Conference 2005 Game Design Track
Time stun and animation sequence. Chain interruptible combos together.

Game Design: Risk and Return
Masahiro Sakurai
Game Developers Conference 2005 Game Design Track
The aesthetically effective mechanics of gameplay balance risk and return. Salient example of Smash bros knockout and fireball versus jump on turtle and goopa in Super Mario Bros.

Anatomy of a 2D Side-Scroller
Luis Barriga
Game Developers Conference 2004 Game Design Track
Conventions of damage models, such as collision and damage over time. Speculations into psychology of 2D versus 3D.

Bossy Behavior: Patterns and Techniques in Boss Design
Luis Barriga
Game Developers Conference 2003 Game Design Track
Function of bosses as climactic antagonists and checkpoint of user skill. Make vulnerability feedback obvious. Tune waves of attack. Avoid dominant strategies in safe spots. When observing test, assume boss is broken.

There you have 'em. And no, I'm not sleeping with Luis Barriga.

There is one last one from GDC 2006, which was one of the most relevant to all forms of interactive media. The Game Innovation Lab doesn't have GDC 2006 or 2007 audio recordings yet, but I hope this post echoes the other students who have pointed out their cost-effective educational value for the program.

Advanced Prototyping
Chris Hecker, Chaim Gimgold
Game Developers Conference 2006 Game Design Track
Pick a precise, testable question. Avoid making things. Do in content. Code little. Prototype to change opinions. Prototype the risky items first. Avoid frameworks and infrastructure. Getting feedback frequently increases the designer's autonomy.

I started this post as a short resource report on videogame design from the GDC audio CDs that the interactive media department has, but in review, about one-quarter of the content is just videogame design. Most of the content above applies to interactive media for a variety of aesthetic requirements, not just games. So if your focus is more fine art or serious than commercial play, I still suspect some of these lectures will help you understand how to design interactive media.

For my taste, these are exemplary lectures. I'd be happily educated if I hear more of their caliber (especially while attending graduate classes in interactive media at USC). So, while you're reading this, I'd like to learn from you. Whether GDC or elsewhere, why don't you comment on what lectures or other media that taught you the most about designing interactive media?