Course

Some fun photos from the one week project!

Presenters: 2nd Year MFA students in CTIN 542, 544, 548
Time: Wednesday, March 12, 6pm-8pm
Location: USC's Robert Zemeckis Center for Digital Arts (RZC),
Room 201 Zemeckis Media Lab (ZML)

Title: "Flying Off The Handle - CTIN 542, 544, 548 Second Year MFA Combined One-Week Interactive Design Project, Spring 2008"

Instructors/Jurors: Mark Bolas, Perry Hoberman, Michael Naimark, Peggy Weil

Project Description:
"In the center of the table in front of you is a pile of handles, knobs and pulls. These objects, specifically shaped for the human hand, can be attached to other objects or surfaces, allowing them to be pushed, pulled, grabbed, closed, opened, turned, twisted, switched, lifted, shifted, operated, poured, tossed, etc. Handles can be attached to either movable and immobile objects. Handles can have both symbolic and/or practical functions - a handle is a kind of affordance. Affordances provide clues to how an object can or should be used (clues that can be useful, but also misleading). Your assignment is to conceive of, design, and produce an interactive experience in which the operation of one or more of these handles by a human user is a central component. This experience should be one or more of the following: surprising, shocking, bewildering, addictive, amazing, exquisite, subtle, provocative."

Perry and I (and maybe you too?) are starting to develop test cases for a Wii Media Lab. Toward that end, here is a proposal for some uses, risks, and plans for the lab. Your comments, especially CRITICAL problems are most welcome. Thanks already to Mark Bolas, Marientina Gotsis, Perry Hoberman, and Lily Cheng, for their advice.

Wii Media Lab

• utilize the room
  
  
  • to present interactive media
    
  • to navigate an virtual environment objects
    
  • to model objects
    
• low cost
  
  
  • $50 to $100 per station.
    
• available hardware
  
  
  • Wiimote
    
  • bluetooth port
    
  • infrared LED
    
• multipurpose
  
  
  • easy to recreate for future users
    
  • easy to apply to other artists

2D pointer + 6 DOF device

2D multipointer

2D multitouch whiteboard

3D multipoint tracker

Risks

• Engineers (such as I) are unfamiliar with programming input devices.
  
• Several Bluetooth connections interfere with each other and slow down data transfer.
  
• Cameras can be confused by multiple points of light from neighboring input stations.
  
• User may be confused by six degrees of freedom by moving in 3D space.
  
• User becomes fatigued from maintaining limbs away from body and supporting devices after several seconds.
  
• User may be confused by ubiquitous shape of wii mote.
  
• Adapting device drivers and user interface control maps to software applications may be time consuming for each application.
  

Precautions

1. Test Wii remote on a Mac and Windows machine.
   
2. Test operation of standard Wii system in Zemeckis Media Lab.
   
3. Test previous implementations of Wii input device for PC that have similar objectives.
   
4. Scan heat sources in Zemeckis Media Lab. Test range and field of view and field of emission of camera and infrared LED. Direct camera away from heat sources in Zemeckis Media Lab.
   
5. Minimize number of bluetooth devices in use, by starting with a single wii remote.
   
6. Employ a real-time introspective development language that I'm familiar with, Python.
   
7. Start development in an OS that I have access to at school and home and the most experience with: Windows XP.
   
8. Establish unit tests to quickly confirm operation of camera, wii remote, device drivers, and other units that are developed.
   
9. Roleplay rehearsal of input device manipulation to detect fatigue, disorientation, confusion, and lack of affordances.
   
10. Test simple cases of system operation and functionality for usability.
    
11. Cite adaptation required to modify system to control novel software.
    

System Architecture

1. Raw image in physical space.
   
2. Blobs of infrared light on photocell array of wii remote.
   
3. Two dimensional points within wii remote.
   
4. Bluetooth encoded 2D points from wii remote transmitter.
   
5. Bluetooth encoded 2D points at Bluetooth-USB adapter.
   
6. Bluetooth encoded 2D points in Windows PC.
   
7. Wii 2D points decoded by pyBluez, Python binding to Bluetooth drivers.
   
8. Wii data decoded by wii.py script, adapted from Wiiewer.
   
9. Matrix transformation of 2D input to normalize plane of 2D points.
   
10. Simple gesture recognition, as jestur.py.
    
11. Inject input to mouse movement through windll.py.
    
12. Display device shows mouse pointer moving.
    

Test case requirements

• Renders simple 3D scene.
  
• The input device enables the user to translate, rotate, and scale an object in the environment.
  
• With 15 minutes training, a graduate designer is faster at performing these manipulations with the input device than with a mouse.
  
• The prototype can be developed by a couple full-time graduate designers within their technical knowledge set in a couple weeks.
  
• Prototype costs less than $200 in hardware and software.
  

Test case: Chopsticks

Test case: Model manipulator

Test case: Free paint

Technical Requirements

Last semester, I had made models for the panorama presentation of the story bible for my thesis in Google Sketchup. By the way, this week, Perry asked us to produce a panoramic image. I found this ZML panorama template for Photoshop (CS2+) very convenient. This template made the two panorama presentations I did for CTIN 532 (Method of Loci, and Monkey Monastery). With this template, Photoshop automatically slices a large image into 14 images for the 14 continguous projectors in ZML. And creates an HTML file, too.

In the course of making the models, I fell into an infatuation with Google SketchUp, but using a mouse to model was like modeling with a bar of soap. So I began imagining a one-to-one correspondence of physical pointer position with a virtual cursor. My first sketch was for webcams, but then I remembered Ken's post.

Ken Leung posted a link to a novel Wii input device probject, by Johnny Lee. Johnny Chung Lee at Carnegie Mellon University prototyped a low-cost whiteboard and finger-controlled user interface using a Wii mote as the camera.

and so an idea began for a...

Wii Media Lab

Use infrared camera and infrared LED to intuitively control the cursor of a variety of applications while being free from the keyboard and mouse.
These are input device systems for the Zemeckis Media Lab. My requirements for these input devices were:

• utilize the room
  
  
  • to present interactive media
    
  • to navigate an virtual environment objects
    
  • to model objects
    
• low cost
  
  
  • $50 to $100 per station.
    
• available hardware
  
  
  • Wiimote
    
  • bluetooth port
    
  • infrared LED
    
• multipurpose
  
  
  • easy to recreate for future users
    
  • easy to apply to other artists

2D multipointer (wii mote, infared light pen, by Johnny Chung Lee). $50 hardware. Application: Control multiple points for scaling or rotating on 2D plane.

2D multitouch whiteboard (wii mote camera). $100 hardware. Applications: Draw on the wall. Control multiple points.

3D multipoint tracker (2 wii motes, $150 hardware. Applications: Model 3D image. Navigate cursor in 3D environment.

According to Peter Preuss, who had also mentioned Johnny Chung Lee's prototypes in the same class, each station requires bluetooth to talk to wii mote.

I had such promising results from the second-phase design chat for the second gameplay prototype of my thesis today, that I'd thought it would be most useful not to ask for support, but instead to brainstorm the problems in these ideas. To criticize on the grounds of not meeting the requirements put forth in the preamble. So: What's wrong with any of these ideas? What are the problems? What will cause this experiment to fail?

Will you help me by pointing out the defects in regards to meeting the above requirements?