I’m using the midterm as an opportunity to work on techniques for my final project, so I’ll be doing IR motion tracking, in conjunction with multi-tracked synchronous audio.
The basic idea is to allow players to use their hands to wipe away a virtual wall of wax, which will be projected on a wall. By doing so, they can uncover a scene; for example, an argument between two people. Uncovering more of the image of a given person will cause the corresponding audio tracks of their side of the argument to become louder and more clear.
The wax will “heal up” over time, so the player will have to work hard to fully uncover either side of the scene. Dong so, however, will yield the reward of opening up the corresponding door, through which will be physical items (letters, photos, etc.) that explain the backstory of that person (i.e., what they’re actually angry about). These doors will also close if enough time passes.
——————-
To see a basic demo of the motion tracking I’m working on, download this. There are no visuals; just wave your hand in front of your webcam, and try to change the pitch, volume, and pan of the sounds.