May 09, 2005
February 25, 2005
February 24, 2005
A Look at the Pepper's Ghost Projection Technique
Pepper's Ghost is a projection technique of sorts as it involves a light source that projects an image on a large piece of glass at a 45 degree angle that "reflects brightly lit objects offstage, superimposing them with the on-stage set and actors."1
The trick is called Pepper's Ghost after John Henry Pepper, a professor at London's Royal Polytechnic Institution, the man who first used it in a stage performance in 1863. Some sources argue that Henry Dircks should receive the credit, since he came up with the idea a year earlier. However, it is noted that Dircks' ghost was only a small-scale prototype, while Pepper's was a full-sized stage show. (As an aside, I think this points to the value of having good content to support a new entertainment technology, rather than just a prototype, because people tend to remember content. It's an important issue to consider as we set out to create our thesis projects).
Pepper's Ghost was, in a sense, a natural evolution of the "magic lantern show," already popular in England at the time, and it went on to be featured in various "ghost shows" (or "bogey shows") at British fairgrounds. Its popularity lasted until the early 1900s, when cinema replaced it as the latest "trick."
Despite its disappearance as a show in itself, the Pepper's Ghost illusion continued to be used in stage productions that required a semi-transparent ghost, including productions of Hamlet, Macbeth and the Christmas Carol. However since the large glass blocked a lot of the sound coming from the stage, the trick was used only sparingly, until the advent of microphone/speaker sound systems.
The effect, and several evolutions of it, is still used today. The Shuftan Process uses an angled, partially mirrored sheet of glass to allow miniatures to be "rear projected" into a scene with an actor. The miniatures appear to be full-size, and reduce the requirements of set-building. The effect is also used in modern haunted houses, including the famous Disney Haunted Mansion, which features the ghostly images of animatronic ghosts dancing in a ballroom. A variation of the Pepper's Ghost technique is also being used in special effects theaters, including the Holavision Theater developed by BRC Imagination Arts.
This theater uses "computer-controlled combination of special effects, theatrical lighting, audio, projection and live performance."
One non-entertainment use of the effect is the heads-up display on modern fighter jets, which allows pilots to see vital information without looking away from the cockpit window.
Sources:
1 http://www.dafe.org/misc/peppers/peppers/htm
2 http://www.brcweb.com/products/special-effects-theaters
3 www.wikipedia.org - "Pepper's Ghost"
4 "Adventures in Cybersound", http://www.acmi.net.au/AIC/PEPPER_BIO.html
5 "Giving Up the Ghost", http://www.thegalloper.com/backstories/0702ghost.html
February 19, 2005
real-time compositing for mixed reality
As titled, my paper centers on techniques for mixed reality, and real-time compositing, and concludes with a proposed project. An example of a stage performance that uses these techniques is the Big Art Group out of New York. The stage setup generally consists of a screen with cameras mounted on top that point at performers who are visible from the waist up on a stage behind the screen. Video of the performers is then projected onto the screen. While their show Flicker doesn’t go far beyond this simple (but effective and highly choreographed) setup, House of No More takes it to its logical next step where actors perform in front of green screens, and computer generated imagery is composited into the image.
While the use of green screen can be an integrated aspect of a well-designed show, for my own work I would find it to be too limiting. Visually, the green surface would need to be lit uniformly, and be fairly bright, which would likely turn it into a large and glaring visual focus and could easily become overly distracting. On top of this, the color spectrum of objects and performers would need to be restricted away from green. While this could offer some creative potentials (such as the green shoes used in House of No More), over all it’s too limiting.
Academic research by Paul Debevec at the Institute for Creative Technologies at USC takes another approach utilizing near-infrared illumination in place of green screen technology. Using an IR sensitive camera with an IR pass filter, Debevec was able to identify a fabric that absorbed most visible light (thus appearing black to the eye), while reflected most IR light (appearing white to the IR camera).
This was then used in place of a green screen material behind the performers and lit with IR LED lamps. Two cameras were employed for compositing purposes: one being sensitive to only visible light (and not IR), and the other being sensitive to IR (and visible light) with an IR pass filter placed over the lens. The IR camera sees the IR illuminated background screen as mostly white, while the figure is dark. This grayscale feed is then used to key out the figure in the RGB feed from the visible light camera.
While this helps with the problem of spill (where the green screen reflects green light back onto the performer), the background is far from completely white, and in a more complicated lighting setup, could cause the system to confuse the figure and background. This lack of robustness is likely due to the fact that the system was not designed for real-time, but rather a complicated post-production process of compositing.
A system that has many interesting possibilities is one developed at the BBC’s R&D division. Instead of a green screen, or a black IR reflective material, they use a retro-reflective material that is highly reflective (similar to the function of materials in street signs). The material is hundreds of times more reflective than white paper, and can be lit with a small amount of blue light emanating from around the camera lens.
This material in general is not new and has been used in special effects for some time, however the specific material developed by the BBC in conjunction with Reflecmedia is much more robust than it’s predecessor. Normally, retro-reflective material must be lit and viewed at near 90 degree of incidence. This new material however has a much wider reflectivity angle allowing it to be more flexibly lit and viewed.
In addition, the BBC developed a camera tracking system for orientation and translation that uses a series of circular markers with different printed patterns of concentric circles placed on the ceiling. Termed the Free-d system, the production camera has a second small camera mounted on top of it with a LED lamp pointed at the ceiling. The lamp illuminates the markers, and the camera tracks them to calculate the position and orientation of the larger camera in real-time.
This retro-reflective material and camera tracking system were employed by Industrial Light and Magic (ILM) to create a system for real-time compositing of CGI elements on the set of the movie AI: Artificial Intelligence.
On set Steven Spielberg was able to see a low quality virtual set composited in with the live action actors and set, and thus was able to more precisely control the movement and placement of camera, actors, and sets in relation to the future CGI.
A project idea I have is to line the interior surfaces of a room with Reflectmedia’s special retro-reflective material for use in an augmented reality performance installation, and then light the material with a series of non-visible IR lamps. I would then attach IR sensitive cameras to a conventional augmented reality setup (HMD + visible light cameras) to make it possible to use the IR camera feed to key out the retroreflective screens in the visible light camera feed and composite in real-time graphics into a feed to the HMD screens.
Camera tracking would likely take the form of an inertial tracker with a cord for orientation, and an IR LED on the top of the HMD tracked by a camera in the ceiling for translation (although a setup with ceiling markers would be interesting to attempt with some software engineering help, perhaps from IMSC). The interior would be lit with visible light as well, and allow physical objects to be interacted with on top of a virtual background. Determining the z-depth of real objects to aid in compositing real-time graphics in front and in back of physical objects would be ideal, and perhaps something possible with the Canesta system (to hopefully improve upon previous multi-video camera z-depth work).
February 18, 2005
Projection Systems for Special Venues & Theme Parks:
The majority of special venue and theme park projection systems are designed to enhance the viewer’s sense of immersion. This is typically done through the use of broader and brighter imagery. Visual impact is further enhanced by quality sound systems that create a “larger than life” sound. Other enhancers include seating movement, ultra low frequency speakers, smoke, smells, lasers, and other physical effects.
Filling the viewers’ field of view can greatly enhance the sense of visual immersion. A human’s maximum limits of color discrimination are approximately 60 degrees vertically and horizontally. This “cone of vision” varies with hue, saturation, light intensity, age of viewer, etc. Field of view can be increased with comfortable head movements of approximately 30 degrees vertically and 45 degrees horizontally (see diagram below).
The most common specialty venue projection system is IMAX®. These specialized theaters are characterized by steeply raked seating and large (up to 100’x74’), and slightly curved, vertical screens. They can seat up to 1000 people and are often attached to Museums to increase membership and ticket sales. Visuals occupy approximately 100% of the viewer’s field of view. These theaters also have a proprietary “rolling loop” vacuum projector system that moves a 70mm horizontal film stock through the gate at 24 frames/sec. The new IMAX®HD system increases the frame rate to 48 frames/sec. for an even more realistic image quality.
IMAX® “Magic Carpet” theaters experimented with a second screen positioned below the audience and viewed through sections of glazed floor. This was most effective for aerial, aerospace, or underwater content. However, the cost of building such theaters was prohibitive. OMNIMAX® theaters use a similar system as IMAX® with a 75’ hemispherical-dome screen. A fisheye projector lens is situated at the center of a 30 degree sloped, 350 seat theater. These are well suited for planetarium and laser shows. Both OMNIMAX® and IMAX® can feature an optional 3D SOLIDO® dual synchronized projection system with IR signal, polarized viewer goggles.
Showscan® is another large format projection system used in special venues like theme parks and World’s Fairs. These theaters tend to be a little smaller than IMAX® but offer a standard 60 fps (frames per second) flickerless high definition image. Showscan screens are a wide aspect ratio, closer to the wide-angle projector, and curved to produce a brighter image than most theaters. Films typically last 20 minutes and many offer Dynamic Motion ® moving seating for synchronized motion simulation.
Showscan also offers a 3-D system utilizing Twin left and right 70mm projectors @ 60 f.p.s. Some say this is the most effective 3-D system on the market. Additional 3-D venues host Omni Films’ ESI-3D® (a good polarized 3D system for standard theater screens) and Stereovision 70 (split 70mm frames off one negative).
Another popular special venue and theme park projection is Disney Circle-Vision 360. This proprietary system capitalizes on a cylindrical 360-degree image. 9 curved screens are placed side by side with synchronized projectors placed at the opposite seams. Content is shot utilizing specially constructed rigs with nine 35mm cameras. Most of these theaters have the audience stand so they are free to turn completely around during the films. Handrails are essential as the slightest tilt in horizon convincingly suggest the entire theater is tilting. Other similar systems include Vision 360, Superscan Cinesystem, Cirkinovision, and Ultra Toruscope 3 and 5 screen systems.
Iwerks Imagine 360 and Swissorama 360 are similar from the audience’s perspective. However, both of these systems have a seamless projection. Shooting 70mm film with a specialized fisheye lens through a doughnut shaped mask provides each image frame. Utilizing a matching fisheye projector, mounted in the center of the round theater, completes the illusion. Although the image is less vibrant that 9 separate projectors, the camera itself is much less clumsy to use than the previous 360 rigs.
Other less common special venue projection systems include Cinema 180 featuring a quadrispherical screen, Motion Master with its hydraulically activated seats, and motion based simulators with standard projectors housed on the moving platform.
Disney’s California Adventure hosts a unique set of theaters in the “Soarin’ Over California” attraction. Two OMNIMAX domes are set on their sides and the viewers are lifted in bench-like seats to a four-tier vertical stack theater. Each bench tilts and pitches in conjunction with the scenic fly-over of California’s greatest landscapes. A grandiose score by Jerry Goldsmith and familiar smells pumped into the theater, like pine and citrus, enhance the epic dangling foot journey.
On a completely different scale, Disneyland’s “Haunted Mansion” showcases several specialty projection systems. These include Flying ghosts projected on three story scrims, singing faces projected on gravestone busts, and a proprietary gypsy’s head fiber optic projection on the interior of a floating crystal ball. These and more Imagineering tricks make the Disneyland’s Haunted Mansion one of the best collections of specialized projection systems.
Sources/Bibliography:
BKSTS - The Moving Image Society (http://www.bksts.com/)
IMAX Corporation (http://www.imax.com)
iWerks Entertainment (http://www.iwerks.com)
Building A Dream – the Art of Disney Architecture by Beth Dunlop
http://www.doombuggies.com/
(I was not able to upload related images to this page)
February 03, 2005
Idea about my "Two weeks project"
So my project is related to projection. My original idea is using camera mounted behind the wall of IML, or window of ZML to project what's outside the room. I'll adjust to make it looks like a hole or window on the wall. The purpose of this project is to bring in sunshine and communication between the two sides of the barriers.
But I've other idea about a performance piece. I think painting process is somewhat interesting to watch. I'd like to do some 3D projection painting. We can have a sculpture with no texture set up in the room. And I'll paint metal, wood & human skin on top of it to create this illusion of the object. Or we can have a participater make a pose and take a photo of him from the webcam. And I can paint tattoo or body armor onto his/her body. When I'm done, ask the participater to repeat the same pose, and we can see the body armor or tattoo there.
Which one would you like? If we all need to do some kind of performance, I can do the second one. Otherwise I think the first one is more useful.
October 26, 2004
The McVoting Booth
Here are the BS2 and the Processing codes I used to execute my "McVoting Booth." The voting booth allowed the voter to choose between Bush or Kerry. There was a button for each candidate. The Bush button always worked (to add votes to the Bush side), but the Kerry button worked some of the time, sometimes added votes for Bush instead, and sometimes did nothing at all.
THE BS2 CODE:
********************************************************************************************
SerInPin PIN 0 'serial IN pin (RXD - at the other end: TXD)
SerOutPin PIN 1 'serial OUT pin (TXD - at the other end: RXD)
SerInData VAR BYTE 'number to match to incoming data
SerWait CON 50 'ms to wait for input before timing out
N9600 CON 16468 'BS2 baudmode values: 16468/9600, 16416/19200, 16390/38400
'See SEROUT in the Basic Stamp manual for details.
test3:
PAUSE 100
IF IN2 THEN
SEROUT SEROUTPin, N9600,[DEC 1]
DEBUG "1"
ENDIF
IF IN4 THEN
SEROUT SerOutPin, N9600,[DEC 2]
DEBUG "2"
ENDIF
GOTO test3
********************************************************************************************
PROCESSING CODE:
********************************************************************************************
// McVoting Booth
// by Kellee Santiago
// A phony voting booth that reads off of two buttons.
// votes for Bush all of the time, the other button votes for Kerry sometimes,
// Bush at others, and sometimes does nothing.
// Created 20 Oct 2004
int bushVotes = 0; //declares the variable number of bushVotes and starts it at 0
int kerryVotes = 0; //declares the variable number of kerryVotes and starts it at 0
BFont theFont; //declared the variable of the font as "theFont"
int val; //reads the value off the serial port
void setup() {
beginSerial();//starts reading the serial port
size(400,200);//screensize
theFont = loadFont("Bauhaus.vlw.gz");//declared font - make sure to copy it into project folder!
fill(255,0,0);
stroke(255);
textFont(theFont, 48);
}
void loop() {
background(255);
BImage elephant;//declare image of rep. elephant
elephant = loadImage("tease.republican.gif");//loads image - have copy in project folder!
BImage donkey;
donkey = loadImage("tease.democrat.gif");
image(elephant, 120, 50); //sets the place of the image
image(donkey, 200, 50);
smooth();//smooths out font
fill(255,0,0);
text("bush", 40,75);
text(bushVotes,65,125);
fill(0,0,255);
text("kerry", 260,75);
text(kerryVotes,285,125);
}
void serialEvent() {
val = serial;
println(val);
if(val==49) {
bushVotes++; //reads value off serial port. Processing reads in ASCII, so when the
} //serial port sends out a 1, processing sees a 49.
else
if(val==50) {
if(random(10) < 3) {
kerryVotes++;
if(random(10) > 6){
bushVotes++;
//this is how i set up a basic random generator. When processing gets a 50 (ASCII for 2)
//it randomly generates a number from 1-10. If it's less than 3, Kerry gets a vote. If it's greater
//than 6, Bush gets a vote. Nothing happens if it's 3-6.
}
}
}
}
********************************************************************************************
February 28, 2004
patch
patch_ for reading a single movie into multiple videoplanes.
October 26, 2003
john oswald
John Oswald makes music. he makes music from the music of other people (for the most part). John Oswald has been in trouble w/ the law for this (see the Plunderphonics website). His stuff is great. He is able to capture the essence of his subjects (james brown, public enemy, metallica, the beatles...) construct pieces from that core. He is a Lawrence Lessig example, and should be championed by the EFF and Creative Commons. He has been forced to take his stuff offline (or at least, it's not where it was before), and since I'm doing a more automated oswaldian system for my simulation project, I figured it might be good to get a taste of him. So I've temporarily put some stuff of his up on my server:
Net is my favorite. I have more, so if anyone wants, email me and I will put them up.
October 25, 2003
October 21, 2003
CTIN532 assignments
These are the six assignments from the original syllabus for CTIN 532.
• Assignment 1: Feedback system. Design and implement something (but not mouse/keyboard/screen-based) that responds to user input in an interesting way (ie., in an unexpected or unpredictable way)
• Assignment 2: Interactive structures. Choose a narrative (fiction, news story, personal experience, etc) and define an interactive structure (based on navigation or choice or something else) for it. Make a diagram of this structure.
• Assignment 3: Simulation. Make a simulation of a real-world object. Bring both the object and the simulation to present and compare in class.
• Assignment 4: Superimposition. Design an alternative reality and come up with a method to superimpose it onto the lab. Coordinate your project with the rest of the class so that all superimpositions are able to coexist simultaneously.
• Assignment 5: Blind Interactivity. Design an interactive project that is entirely non-visual. The project can use any of the other senses for input and output.
• Assignment 6: Group Interaction. Design a project that simultaneously engages three or more users.
By this point in the semester, we're supposed to be on the fourth assignment, but we're just finishing up with the second one.
We're running out of time. Here are some ideas for getting through the rest of the semester.
A. Each student completes 3 of the 4 possible assignments. You can each choose which one to drop.
B. Whenever possible, assignments should be documented and presented via blog (rather than in class). Everyone would be responsible for viewing the projects outside of class so that class time could be devoted mostly to discussion. Obviously, some projects would be difficult or impossible to present this way, and these projects would still be shown in class.
Other ideas?