Weblog

This Spring, the Interactive Media Division and the USC Game Pipe will be offering classes where students will get hands-on experience working with and developing for the new XBOX 360 Natal device. Project Natal is a new, full-body interaction system for use with the XBox 360 game console. It allows players to interact with games in an entirely new way, by using their bodies as the controller. USC is the only school in the world that will be receiving advance Natal development kits.

The class will focus on creating games and experimenting with this novel interaction device. Students will be in bi-weekly communication with professionals at Microsoft who are on the Natal development team as well as mentors from throughout Microsoft.

The class will be taught by Jeremy Gibson from the Interactive Media Division and Michael Zyda of the Viterbi School's GamePipe. It will fulfill 2 units of the CTIN Elective A requirement.

Registration is open now, and instructor permission is required to register. For more info, contact Jeremy Gibson.


    Click here to download and read the flier

Here's a video and description of the prototype I whipped up yesterday.

Collaborative creativity using Twitter and Microsoft's Multi-Touch Surface.

This is a one-day prototype exploring the possibilities of connecting the ubiquity of Twitter with the single-location Microsoft Surface.

It's basically collaborative magnetic poetry. Any user can add words to be used by replying to @touchPoetry on Twitter. Users on the Surface drag words around to form poems and encircle them to create an outgoing "tweet" on the touchPoetry twitter account.

I was just catching up on my Reuters Tech News, and found this video, while I'm sure I'm behind the curve compared to our VR researchers, and innovators, I thought it worthy of a post nonetheless. I'm wondering what people like Mark, Scott, and Michael might think about the viability of a such a system. I found it also interesting that they tried to claim in the Reuters story that there are no VR simulators for ground troops; having seen some of what goes on at ICT I found it a seeming unfounded statement. You can learn more about Vitrusphere @ http://www.virtusphere.com/

Last Friday, a funny-looking guy from Oblong mentioned during his introductory remarks at one of the UCLA Mobile Media symposium's panels a strange bent he'd been noticing among students and practitioners to accept as UI gospel designs emitted by the two major OS manufacturers. What he had no time to go on to say (but should have anyway): the phenomenon isn't ascribable to the individual; it feels, rather, like a pandemic of ingrained assumption -- at the aggregate tech-humanity level -- in which we all of us can't help wallowing.

That assumption is to do with where good UI ideas come from and who gets to have them (the ideas).

The sole dominant (universal, actually) GUI is twenty-five years old, was introduced by one of the incumbents, and was copied by the other as slavishly as law, talent, and pride would allow. Already a problem, no? A monosetup doesn't promote a mental model in which there's evolution, an ecosystem, a dialogue. Because there was no GUI before that, we haven't even had a pattern that suggests "you get a new one every three decades". And yet we should make it our job to assume that, and to assure that, and while we're at it to get itchy every decade. We're currently coming up on twenty years late.

Touchscreens that accommodate multiple points of contact got rediscovered recently. Products involving these modalities have been released -- some of them to commercial success, some of them to PR success -- and this in turn has suddenly launched large-scale public interest in novel interface. The interest is welcome.

The shame, really, is in presupposing that the incumbents have some advantage in designing radically new interfaces. There's certainly reason to expect they'd have an advantage in asserting new interfaces, but that's hardly the same thing. Yet even there it's unsimple: an incumbent has a responsibility (read: fiscal incentive) not to alienate or confuse or discomfit existing customers. So in fact that translates to a formidable disincentive to offer anything too radical.

Enough about them. What we want is encouragement against being discouraged from designing and building freely. Here's an axiom: a new UI that's a vast leap forward must also be sweepingly different. That's not to say that an incrementally different UI couldn't be a swollen commercial triumph. But such swelling is not what we should be after (not all of us, at least). Here's something between an axiom and an assertion: a vast leap forward is possible. The leap's vast, so it won't look like a traditional GUI with some touch stuff layered on. Therefore, we haven't seen it yet.

That ought be all we need to know.

A last part is that a giant incumbent company has about the same chance of being the one to find it as International Business Machines had of inventing and popularizing the GUI: which is to say: greater than zero: and it is also to say: less that one.

October 20 download pdf
October 27 download pdf

Here are Kate and my gnotes from the following classes:

August 25 download pdf
September 01 download pdf
September 08 download pdf
September 22 download pdf
October 06 download pdf


The rest are coming soon.

These guys use an an HTC Magic running the built in Google Street View and a box to make themselves a primitive pair of VR goggles....minus the whole 3D part.

Editorial: Craft & Tools, Craft v. Tools.
James Haygood



James Haygood, one of the pre- and post-millenial decades' most respected editors, will discuss editing process, the way in which contemporary tools both influence and impede editorial choices, and the possibility of a new workflow that first enables and then depends on a more explicit integration of editorial activity into the broader production effort.

Lifelogging has been a constant interest for many of us in IMD and there are a few posts around the blog such as here and here. More recently, Professor Mark Bolas did some research with Microsoft on their prototype Sensecam. Here's an update on a new commercial version by UK company Vicon (via New Scientist):

New camera promises to capture your whole life

13:10 16 October 2009 by Kurt Kleiner

A camera you can wear as a pendant to record every moment of your life will soon be launched by a UK-based firm.
Originally invented to help jog the memories of people with Alzheimer's disease, it might one day be used by consumers to create "lifelogs" that archive their entire lives.
Worn on a cord around the neck, the camera takes pictures automatically as often as once every 30 seconds. It also uses an accelerometer and light sensors to snap an image when a person enters a new environment, and an infrared sensor to take one when it detects the body heat of a person in front of the wearer. It can fit 30,000 images onto its 1-gigabyte memory.
The ViconRevue was originally developed as the SenseCam by Microsoft Research Cambridge, UK, for researchers studying Alzheimer's and other dementias. Studies showed that reviewing the events of the day using SenseCam photos could help some people improve long-term recall.

Sonic Forest is a mesmerizing light and sound installation created by acclaimed artist, architect and composer Christopher Janney. The installation is composed of sixteen eight-foot columns, each containing audio speakers, lights and photo-electric sensors that enable up to four people to interact with each column at once. Students will immerse themselves in the multisensory installation as they pass between the electronic trees, “playing” the forest as they go by, triggering sensors by touch or movement, and creating an ever-changing score of melodic tones, environmental sounds and spoken or whispered texts, with varying effects of light. Christopher Janney has created public interactive sound-light artworks and performances all over the world. Trained as an architect and jazz musician, Janney aims to have music “wrap around you like a blanket.”

OPENING EVENTS:
Monday, October 12
6:45 p.m.: Performance on Hahn Plaza near Tommy Trojan
7:30 p.m.: Panel Discussion at Annenberg Auditorium

For the official opening of Sonic Forest at USC, Christopher Janney has composed a "truly immersive sound experience" to be performed within the installation. The piece will be performed by Stan Strickland on electronic flute together with the percussion section of the Trojan Marching Band. The performance will begin at dusk, and will be followed by the panel discussion, "Public Space, Public Art and Public Life." The panel will be held at 7:30 p.m. in Annenberg Auditorium. For more information on the panel, please click here.
Organized by Scott Fisher (Cinematic Arts), Martin Kaplan (Communication) and Patrick Morganelli (Music).

Speaker: Christopher Janney, PhenomenArts, Inc.
Time: Wednesday, October 14, 6-8 pm
Location: USC's Robert Zemeckis Center for Digital Arts (RZC)
Room 201 Zemeckis Media Lab (ZML)


Title: "Architecture of the Air: From Urban Musical Instruments to Physical Music"

Trained as an architect and a jazz musician, Christopher Janney has combined these two disciplines in a number of projects. Sometimes he has tried to make architecture more like music as in his "Urban Musical Instruments" series. These include a number of large-scale interactive sound/light installations. Projects completed include "Harmonic Runway", a 200 ft. long interactive light/sound corridor in the Miami International Airport; "Chromatic Oasis", a 30 ft. diameter colored glass and steel mobile, at the Sacramento International Airport; "Touch My Building, an interactive light/sound piece for the entire facade of a new nine-story Bank of America building in Charlotte, NC; “Rainbow Cove,” two nine-story colored glass pedestrian towers at Logan International Airport; “Whistle Grove: The National Steamboat Monument”, a 2500 square foot interactive light, sound, steam environment on the banks of the Ohio River; and “Turn Up the Heat” a 30-ft. diameter interactive scoreboard for the American Airlines Arena in Miami, FL. At other times, Janney has tried to make music more like architecture- more physical, more visual. Projects in this vein include his "HeartBeat," a dance/performance piece where the performer wears a modified heartbeat monitor and moves to the sounds of his/her own heartbeat while other musicians perform in counterpoint.

Backchannel log of the presentation after the jump:

(following Peter Brinson's example, I am shamelessly promoting my class)

Preliminary Syllabus Outline*


draft syllabus

CTIN492L - Experimental Game Topics (TTh 5-6.50pm - with serious coffee break...)
Instructor: Marientina Gotsis

Overview. This course will prepare students for designing interactive media-based health “interventions”, such as sensor-based games on mobile platforms. We will review priority health areas in the US health system, as well as emerging markets and technologies, key topics in public health, behavior, neuroscience and social networks. Students will be mentored through project case studies in focused areas (e.g., obesity, autism, cancer, rehabilitation). Students will be given the opportunity to generate ideas, design and playtest paper and digital prototypes in their area of interest, as well as randomly picked topics during lab exercises. Assignments will be individual and group-based.

Goal. The goal of the course is to increase student awareness of healthcare challenges from individuals to groups and communities, cultivate design empathy from multiple perspectives (consumer, patient, health professional, designer, engineer) and foster a spirit of understanding toward collaborative design. Whereas pure entertainment may focus more on the player experience rather than the message, health interactives require equal consideration to both user/player and message. Students will be challenged to think beyond pure entertainment and understand holistic issues of design that take into account the user/players’ overall mental and physical health, environmental factors, social networks, and scalability issues, as well as the economics that govern healthcare issues. Students will be required to communicate issues based on their diverse perspectives in order to help identify stakeholder interests in health interactives.

Structure. The class will spend the initial part of the semester understanding key concepts, reviewing current literature and evaluating prior art in this area. The class will also cover basic skills in prototyping, playtesting, study design and evaluation. The class will create a series of small prototype exercises from which to draw ideas and experience for individual and group projects. During the final half of the semester, the class will work together in groups to design, playtest, and produce, one or more class projects depending on class size and expertise.

NO PREREQUISITES REQUIRED.
BOTH GRADUATE AND UNDERGRADUATES ELIGIBLE FOR CREDIT.

* The course will be quite different from last year - focused more on hands on design and critique/discussion with an eclectic readings list reviewing special topics in design, health, behavior, technology, and interactive media including games (no book). Many thanks to the students who helped us pilot this class last semester. Suggestions very welcome so long as you're honest but nice :)

My fiancee and I have some quite random conversations. The other night, we walked by an ad for the Toy Story 3D movies, and we started wondering about how the 3-eyed alien's 3D glasses would work if he was real. My brother later joined in on the conversation and he brought up the cyclops-monster Bob from Monsters vs. Aliens. We came up with some ideas, but I wanted to first pose the question to the IM department:

Assuming each creature is in a HUMAN 3D movie theater (i.e. 2 projectors in stereoscopic 3D or red-blue stereo), how do each creature's 3D glasses work? (Or do they work at all?)

VFX: Pipeline, Bottlenecks, Workflow, Process.
John Nelson




John Nelson, accomplished visual effects supervisor of such monumental efforts as Iron Man, Gladiator, and I, Robot, speaks about the role of VFX in modern filmmaking, how the discipline has been integrated into the larger production process, and opportunities for refining elements of the effects workflow.

I have uploaded my powerpoint presentation as a PDF for everyone's reference. Enjoy!

Flatbed Presentation

Gestural Interface Questions:

1) The gspeak gripe sheet indicates finger angle. Can finger angle in orientation to other fingers be a variable for commands? Additionally, an the thumb be normal to the palm as a command?

2) Is there a character input system designed for gspeak? Is it intuitive? Is there any feedback when you enter a character?

3) Gspeak is an optical system. How far beyond the central space can it see? Does it lose sensitivity outside the immediate area? If so, how much?

4) Can the angle and orientation of the (Ikea) table be modified to fit different purposes? How difficult would it be to modify (adjust the table, adjust the projector, etc.)?

5) Dare I ask? How much does the gspeak system cost? Can it be replicated commercially and/or economically?

Cinema Production Questions

6) How do visual effects artists replicate lighting in shots with live action? Do they consult the gaffer or cinematographer's notes or simply execute by trial and error?

7) Random and specific: what is the most effective and accurate method for creating shots with turbulence (earthquakes, spaceships, etc.)? How do you coordinate performance, camera, and post effects to achieve this dynamic?

8) Has anyone figured out how to effectively keep sound edits in sync with picture cut revisions? Can collaboration between picture and sound editors occur in a live simultaneous environment?

9) How common are pickup shoots for large-budget features? And to what complexity are these pickups permitted? How much original content is scheduled beyond principal photography?

10) What roll, if any, does the production designer play in post? Is there anything a production designer should be able or would like to do with footage after it is shot?

g-Speak Questions
1. Has a formal database been developed for g-Speak?
2. How is excess data currently handled within the system?
3. How easy is it to enter text?
4. What other external technologies can be used?
5. Is there more sample code than what we already have? Is there any current database code with the Java application?

Production Questions
1. On average, how many people are involved in the editing process? How many of these people play an integral, hands-on role?
2. What types of notes are kept during the editing process?
3. How is versioning done?
4. How common is it to do multiple edits of the same scene and show all to gather feedback? Multiple iterations at once vs. single iterations continuously.
5. What happens to all the leftover footage?

Project Pitch Documents
I would like to focus on the Bin structure and capabilities within an editing environment. I think it would be beneficial to tag clips with notes, key phrases, inspiration pieces, pre-vis element, etc, aggregating any elements accumulated during the pre and post production phases in one location.

Download Pitch Outline

5 Questions regarding the interface:
1. Can the interface distinguish readily between right and left hands?
2. How much processing power is required simply for gestural recognition / interface elements (projector screens, logistics)
3. How many physical tags can be tracked at any given time?
4. Can physical tags track orientation similar to the finger tags?
5. How is text input handled at the moment? Is it handled?

5 Filmmaking Questions:
1. What kinds of custom hardware are being utilized among key depts. in the modern workflow?
2. Software? Is most of it custom?
3. How much of the film is director oriented - and how much is primarily constructed and managed by splinter groups (animators, effects artists, etc) working entirely on their own volition?
4. How important are aesthetic concerns when designing a program for any given dept. versus straight functionality?
5. How have the role of traditional storyboards changed (or been antiquated) by modern live action film techniques - are they still necessary within the spectrum of previs?

Both of these may be out in time for our class to compare them to our own techniques-
PS3 EyePet Trailer
360 Project Natal Interface

DimP Prototype Video Player is free and downloadable (Windows and an OSX port) for an alternative control schema regarding timeline control with "gestural" motions using the mouse. You can download it and try it out for yourself