nime 05

well, got a paper for my thesis accepted to the nime 05 conference. I'm excited to go to vancouver. we'll see about the workshop on 3/25...

but this post isn't about that, really -- more about posting some awesome reviewer comments. either people were just skimming my paper and either got certain aspects of the project or didn't, or there is some serious 1/30 action.

reviewer 1:

The topic itself is not really new, no new ideas are outlined in the paper.

reviewer 2:

This work is original and very interesting. Very good work, very good paper!

man, I'm getting so very dubious about this research paper world...

bullets

kill people

.pdf

time travel...

...is working

( hasselhoff recursion )

content

Wow, so this is what my thesis project ends up looking like... 386 database entries, each row linked to a specific audio file and corresponding to a physical space.

1, 1_1_2.wav, 118.39338, 34.02393, 15, 1, 2; 2, 1_2_3.wav, 118.39338, 34.02393, 15, 2, 3; 3, 1_2_2.wav, 118.39338, 34.02393, 15, 2, 2; 4, 1_2_1.wav, 118.39338, 34.02393, 15, 2, 1; 5, 1_1_1.wav, 118.39338, 34.02393, 15, 1, 1; 6, 1_1_3.wav, 118.39338, 34.02393, 15, 1, 3; 7, 1_3_1.wav, 118.39338, 34.02393, 15, 3, 1; 8, 1_3_3.wav, 118.39338, 34.02393, 15, 3, 3; 9, 1_3_2.wav, 118.39338, 34.02393, 15, 3, 2; 10, 1_4_1.wav, 118.39338, 34.02393, 15, 4, 1; 11, 1_4_2.wav, 118.39338, 34.02393, 15, 4, 2; 12, 1_4_3.wav, 118.39338, 34.02393, 15, 4, 3; 13, 1_5_1.wav, 118.39338, 34.02393, 15, 5, 1;

more maps, progress (the non-visible kind)

finished authoring the space, and started creating the database. I've got one entire day done, and about 200 db rows at this point, which is frightening.

so the phone version is basically all working, except that I haven't completed uploading all the lo-fi mobile-optimized filez.

it's been a complete pain creating all these rows manually, and keeping track of not only the file names, etc., but also the time of day and day of week, along with the node that each file is attached to. So I've been having to be very careful that my naming scheme is working, because it's the only way I can make any sense of any of this.

the map stuff is coming along ok. I screwed up our printer trying to print on card stock tonight, but I'm going to try again tomorrow.

Mackbot Johnson

map it

here's the mas o menos finalized map for the culver city project.

basically, this images reflects how each day of the week is layered upon a base map via transparency.

so each day will show a suggested path through the space. I think the current idea is to print out about 20-30 of these fancy transparency maps, and provide downloadable, cheaper ones for mass distribution, depending on how the event goes.

In bad news, my engineer is driving me nuts and stressing me out.

itunes mix of location33 music

Just let iTunes mix together 20 minutes (ok, 18.5) of music from my "forthcoming" thesis project.

Get It Now (to use verizon-lingo)

this is part of what will amount to about 2.5-3.0 hours of music to be embedded in the downtown culver city area.

in other news, had a good meeting with John Richa, the head of the IT dept. at Culver City Hall. Looks like we're going to do some sort of event in early-ish May, pre-graduation, so I think that will be fun.

In other news, the daily show last night was the funniest I've seen in weeks. But when did blogging become a journalistic thing only... it's all in the news these days, but the definition has seemed to stray far from the justin hall (a student at this very dept. who you might have seen or met) version.

hm.

nime paper

damn, I totally have the sick. I was making ridiculously awesome progress this whole week until bam, on friday/saturday I start in with the cough/runny nose/etc. I hate being sick. I think the last time I had a cold was 2 years ago. So fuck man, I'm pissed off.

being sick did allow me to focus on writing this paper for NIME 2005, which I was putting off because singing folk songs in a robot voice is way more fun.

my hair also makes me look like Syndrome from the Incredibles.

anyway, this is more of a personal archiving post, since I'm sure no one will read it.

soundscape project

cool fm soundscape project.

http://berlin.soundscape-fm.net/

environmental sound -- server -- radio

read more here:

docs

Production Document

Potential Venues

Current Prior Art

instant replay

Marientina made a comment last semester on my final thesis presentation, with the idea that perhaps I should consider the notion of replay-ability in my project. The way things currently stand, locations hold different music files depending on what time of day, and what day of week, you are at a given location. This creates a sense of the space as a shifting, living thing, and fits more nicely into my narrative framework. However, as marientina pointed out, and glenn III also noted, such a scheme will prevent people from, for example, walking the same path everyday and hearing the same music.

While I certainly respect that idea, and relate to the desire for consistency between location and music, I'm going to try and relate now why I think this is not ideal for this specific encarnation of location-based audio.

a) You can do this with your iPod, and it will be a better experience. This is what normal portable music devices allow you to do: to be in control. What I'm trying to do is build a new type of music architecture, in a way, and allowing this element of control runs contrary to advancing that goal. I think it becomes too much like trying to pump a model of music onto a new technology, instead of thinking about how a new technology could change the experiences we have with music.

That being said, certainly there are a number of different applications that would greatly benefit from a consistency, and I'm more than certain that I've thought of, and plan to think about these applications in the future. I think the idea with this specific project, though, is to have the music space dynamic and flexible, and I'm afraid of what the static thing would do to it. I guess I'm thinking of reaching for the sweet spot of the below matrix we keep talking about, the new technology, new media. I think I'm in the "new technology, old media" category, and I'm really trying to push into that "new media, new technology" category.

b) The best way to implement such a feature would require people logging into the system and setting up a preference that would circumvent the time-based quality of the music. The wireless area of culver city is not such that it could support having all 7 songs active at unique nodes each day.

c) you can replay the music. In the space, you have to know when the section of music you like is active, and then you can reexperience it. Otherwise, you can unlock an mp3 of the stuff you like and listen to it at home whenever you want.

Kurt made a great analogy today, relating the experience people have when they walk out of a movie or musical theater or opera or something, and are humming it outside of the offical space of the original music content (the theater, etc). Then if you want more, you can go back to the theater, or you can buy the soundtrack.

I'm not trying to strike down suggestions here. this specific idea was a good one -- in fact I encourage more because it helps me elucidate my own intentions.

Any additional comments would be appreciated.

project website

url is loction33.net

The page is basically what I hope to be a good 2nd or so draft of how I want the web presence of the project to be (other than the research aspects of it, which live in this space). I guess this site would be sort of the public face. Instructions abound, and I've more or less got the entire backend working, so people can log in and unlock mp3s and stuff from hints and clues they get while experiencing the in-space version of the project.

but I've finally been crystallizing my concept, as well as continuing the production and technology, and I wanted to get all of this up.

325 unique files

Just calculated the number of files I am creating / will create for my thesis project, location33.

the number scares me a little bit. The math breaks down like this:

1 song X 7 days per week = 7 songs
7 songs X 2 characters = 14 songs
3 versions of each song = 42 songs

each song is dispersed throughout 7 locations = 294 song fragments

each day has 3 unique hints = 21 hint fragments

each day has 1 hidden song fragment = 7 hidden song fragments

there are 3 guides that remain fixed each day = 3 guide songs

294 song fragments + 21 hint fragments + 7 hidden song fragments + 3 guide songs = 325 unique audio files

I think I'm about halfway there, although I haven't counted exactly how many I have. It's time to start taking a bit more of a brute force, checklist type approach to things now that I've got everything straight in my head.

file reference

internal

tech schedule for break, 2nd semester

Download file

jacked from PDA

HEY!

here's a file that I recorded last night (~2 minutes) of me walking around near zemekis with my little PDA and having it play me some songs. The points where each file played was picked rather randomly, for testing purposes, so the length, etc., of how long it takes to walk from place to place, etc. is totally off. This is just to get an idea of the experience, how the music sounds out of the pda, etc. Warning: I recorded through the line in of my laptop using Audiocorder OSX, and that process is actually adding a lot of the noise. While the current files are 8Bit Mono at 32khz, they actually end up sounding much better with headphones and minus the line noise (surprise).

here's the file:

FILE [.mp3]

phone version

after a week or so I've got a basic port of my thesis to the phone.

Flow is as follows:

See some sort of code in the environment, get phone number from a sticker or flier or something... --> call the phone number --> get instructions --> enter the code --> hit webserver to check date, time and retrieve appropriate music file --> stream that music file --> loop if necessary --> allow user to interrupt with another code they see as they are walking around listening --> awesomely generic fade out sound --> awesomely spectacular fade-in sound --> hit php server, etc., get new file from the new code they just entered --> play that file --> loop this process.

the image is an example of a possible tag, although part of me wants to integrate all that into the environment. giving people the phone number is the tricky part. getting people to make a damn phone call these days is even trickier.

bastards.

just some strollin'

Ok, so this is pretty promo happy. But basically the video is just a vehicle for me to throw up a montage of some sci-folk fragments.

click on image to view. I think it's about 2.5 MB

(excuse the potentially illegible text...)

sheets of paper on the web

Sci-Folk

I think that the term Sci-Folk accurately describes my project location33. I was all proud of myself for thinking of that term, but unfortunately (and perhaps predictably) there was some dude in San Diego that had dubbed his music sci-folk. Crap.

In other news, today was a wonderful day of hacking. I finally fixed a bug in my code so now adding the PDA map interface to my project should be a relative snap. These presentations coming up monday and we're supposed to be like, 75% done with the technical aspect and 50% done with the content. I feel like with the amount of man hours from both myself and my wonderful engineer Leslie, the tech part should probably have been done by now. But basically at this point, I think 75% is accurate. The GPS and sound stuff is all working, the visual interface is ready to be plopped in, and the thing now is just going to be to plug away at the wireless component, getting all that stuff worked out. But I feel good about it.

Ok, ramble mode off.

man and robots

So, my location33 is primarily based in the realm of music, but I'm trying to compile visuals for it as well, partly for presentation/promotion, but also to further develop the space. So I've harassed by friend into doing some concept art for the man, and his robot counterpart. I like the way he made these sketches the same scale, so they can be easily composited. After all, you may turn into a robot in the future, but the robot might still end up looking a little bit like you, right? Movies tell me yes.

Also, I like the way the brain emerges from the robot head. Sort of like Krang, but you have to give Krang props for being a brain that lives in a robot's torso. Viva Krang!

more organizational fun

simple flow...click on image for a larger view...

xml schema

so, also worked out the basic xml schema for the data... it is as follows... excitement is in the air in my apartment tonight!

back-end

above is a snap from a simple web interface tool I built as a helper app for keeping tabs on project-x (a.k.a. heather, location33, etHEAReal, pgnome, thesis, etc.). Basically, it's a simple p5 app that pulls nodes from a db, and displays them overlayed on a map of culver city's downtown area. That's the simple visualization part, but the real purpose of this is app is for mining the persistent space, so for example, when you click within the established radius of a given node, a script is run that checks the current day of the week and the time, remembers the node you clicked, then pulls the file path of the appropriate audio file (as it was laid out in the db). So yeah, this is boring, but it helps me figure out how much more content I need to fill up. Currently, I've got 1 of the 7 songs more or less worked out and the main parts recorded, and will try and get that all on some server in the next week, scattered about the 7 nodes that make up song one: the theme.

But for now, that db isn't all that populated, but when it is, you will be able to see what the current state of the audio layer is at any time of day, or for any given day.

Sorry for those of you I have confused... if you are interested, the applet is here for now.

another kind of map

culver city -- update

Spent 2 hours in Culver City downtown today. Only driven through Iowa, but if Iowa is in fact like Culver City's downtown, I think I'd quite like it there.

So I'm pretty pleased with the tests I ran. While pinging interactive, I walked the loop of the culver city hotspot area, and learned that it is much bigger than I originally thought, in terms of walking time. This is wonderful news. It took me about 20-25 minutes to walk the loop, give or take the route + traffic signals. And this was the straight loop, on one side of culver to washington, then down Hughes, and back down culver. 20 minutes, minimum, I would guess, and that's not including any journeys down side streets, or that little area where there is a pacific movieplex. As I learned in a previous test, getting on the network was really easy, and it was very reliable. It didn't crap out on me once, even when I was pushing the boundaries, and only lost the connection after I was quite a ways down Ince Blvd. past the Trader Joe's. So all in all, good test. And a good strawberry shortcake scone from *bucks.

culver city

So, I'm gung-ho about using culver city as my site for project-x. After the midterm-thesis-review I've sort of accepted the fact that a) USC's wireless net is going continue to suck and b) culver city's downtown area is a much more intriguing space to author on top of, and potentially one that will allow my project to establish more of a rapport between itself and it's host environment.

music

so, here's a little 2 minute prototype of what an ideal sort of transition between locations would sound like.

what you are hearing (if you click the link...) :

a) the type of acoustic --> electronic transitions I want to emphasize in the location mix. e.g. you are opening and closing doors between 2 distinct audio worlds, instrumentation-wise, and between 2 worlds, narrative/theme wise.

b) a rough version with some placeholder lyrics / instrumentatation

c) the basic main theme, musically

d) a very rough mixdown

so, I realize that I've not posted much about the state of project-x recently, mainly because I've been super busy scribbling notes, drawing lines on maps, making spreadsheets with like, 1000 cells needed to be filled (gasp!), and recording lots of music with my little synthesizers and my little guitars. And to be honest, that's ok -- because there are lots of new video games to play, and I'm such a small part of your lives. I accept that -- I love you anyway. Me and master chief.

project-x theme [.MP3]

robot songs

3/4

I (D) dispatched myself into the future
to (C) see what the world had in (D) store
in (D) 50 years would I be remembered
or (C) slowly fade after the 5th world (D) war

But I (D) guess in the future I'm a robot
And I sing to my(G)self every(D)day
about (am)all of the (C) messes I've (em) gotten me (am) into
and about (C) everything I'd (G) forgotten to (D) say

a, b, a, a, b (c)

another shard...

download a, b, a, a, b (c) [.mp3 ~1.7mb]

yes/

fragment

another of many song fragments I've been creating. This one is sort of a 'go between' -- a piece designed to be entered or left at any given time.

fragment: MP3

authoring tool

experimenting with an authoring tool for creating multiple sound nodes + layers for when those nodes can be activated. for example, the nodes will remain in a static place (except the mobile ones...) but the sounds linked to it will differ depending on time of day you are accessing it.

rss weather

hrm. trying to figure out if uncovering certain layers of audio based on weather is possible. perhaps, as perry noted, impractical in Los Angeles, but worth investigating nonetheless.

RssWeather.com is one weather feed.

RainGuage is another. But that one is broken now.

here is a link to what this feed looks like via a simple php script I wrote:

Link via me

interface

new version of the web interface for my thesis project is up. it lives: here.

pls. check it out, any comments appreciated. additions: simulation of a random walk, preview sounds from nodes, more defined positions for each node (compositions embedded in paths...)

moving

Will post a short snip of some of the test audio (wireframe audio, as I heard it referred to today) that I've been working on for my thesis.

I'm liking the way, just going through this mock-up process, of the way that compositions can be experienced as someone moves through space. If you can get the play/stop functions to work well, without abrupt pauses, etc., then you get this really amazing experience, this fluid control of the sound that I'm really enjoying. The difficultly is in dealing with gps accuracy and wireless connectivity. I've dealt with these problems in previous posts -- solutions which I hypothetically like a lot -- but they still are worth mentioning.

By the end of this week I should have my 2nd development unit running (so I don't have to pry the other one away from my engineer).

This means that I'd like to try and start seeing if anyone is interested in trying it out (with the caveat that this this is still in early development).

I will post more about said trial runs later this week, or early next...

dancing shoes

in the context of an earlier conversation I had with Kurt about the coolness of Get Smart (and the cooresponding cool-osity of his shoe phone), comes the Sensor Shoe, designed by Professor Joe Paradiso, MIT Media Lab.

Electronic sensors are built into this ordinary shoe to pick up foot movement and changes in pressure. The short antenna at the back of the shoe transmits data from each sensor to a nearby computer. The computer is programmed with a set of rules that map the data to particular sounds. A digital music synthesizer produces the sounds, which are then heard through speakers.

here is a link of NYC dancer Mark Haim.

Speaking of Get Smart -- that bastard isn't out on DVD yet. Damn. All night I've been wondering: How can I help? The answer: sign an online petition!!! to get these things produced in stunning digitality.

Side note #1 - Carl, your complete Get Smart page is wicked bad.

Side note #2 - this post quickly turned into about how I think Get Smart is totally wicked bad.

timeline

a little flash app built as a timeline tool for my thesis project. Will be up soon...

view

for now, here's my weekly task timeline for my thesis project in xml format.

schedule.xml

picture in picture at MSRDesignExpo

So Much Anger! Who Knew!

Kurt/Scott -- saw lots of pics on the wallop link, but do you have any other pics to post from the expo? Like ones of me getting yelled at by Joi, or perhaps that big red X that decided to make it's home in our powerpoint presentation?

thesis directions

toys + sounds + mobile platforms + social interaction + embedded media

this is the direction I'm heading for my thesis project. looking at that string above, it's perhaps not so clear as one direction. [side note: I hate the way I write on blogs']. But the goal is to make it one direction. So in order to achieve the equation above, I guess I'm going for a game [toy] that relies heavily on sounds and game elements embedded in the physical world, is used on a mobile platform, and is collaborative (in some sort of socially beneficial way). (side note: read this cnn article about geeks and the social characteristics of). Tangible stuff is also tempting...but a large wrench. Fusing these elements may prove difficult, but important to show the similarities/cohesiveness of not only of game/mobile/social/audio technologies, but also of game/mobile/social/audio applications, or content. All this stuff can be weaved together in a really nice way, I think, that goes beyond the singular strengths of each individual format--into something new and engaging.

Listed below is a set of links pointing to some projects / ideas I find relevant to this general area. I will try and consistently update and expand this section.

embedded media (location based)
Environmental Media: Linking Virtual Environments to the Real World [an article by Scott]
WEM project at Keio U.
Urban Tapestries

toys and games (mobile/networked/bits)
It's Alive
Can you See me Now?
Blast Theory
media X (bolas)
I-C-IT Smart Toys
Tangible Bits (almost all projects)

toys and games (traditional)
majestic
mario party
Ambrosia
Lego
Froebel's gifts (kindergarten).

sound
Glasbead
Gakkimon Planet
SMDK SimilationSoundMosaic
Augmented Composer (Rodney Brooks)
Sound Toys for the non-musical (Sony CSL) [.PDF]

this list is a never ending work in progress. I want to look at more traditional games too, so anyone, please feel free to point me in any direction...

embedded music / sound

new project:

- embed authored sounds in the physical world
- track gps user & continuously upload gps data to mySQL server using wireless/JDBC
- play appropriate sound back to user

this is relatively simple, once you get around the hurdles of maintaining a strong wireless signal. test device for this would clearly be a tablet, as my abilities to code jdbc stuff are pretty frail anyway, and would be particularly meek in a j2me vs. a full java environment. but this could be super cool. 2 specific content ideas are a) remote musical (and you could add video or whatever) performance using location as an instrument or b) I like the idea of embedding tracks from an album into the space, and having people interface with the music through the environment. so this was inspired by the fact that I'm trying to think of cool ways to release music (from my nascent label sao bento music), and it would be cool to release albums into the environment, provided that it would be someday feasible for people to easily access that content. Yeah, and I guess you wouldn't need gps for this either -- you could easily do a similar thing using other stuff like bluetooth or glyph stuff. I hope that some big record company steals this idea and makes it work. or better yet, hires me to make it work for them (in which case I would contract out other workers to make it work better). that would probably be the end of the hiring.

anyway, going to start building this...

open house project

tripp and I are thinking of building an open-house specific project. here is the proposal I've been scheming:

- video camera hooked up somewhere near the door to zml. maybe hooked up actually in the entrance room (where the racks are kept)

- camera tracks movement in two ways -- left to right, and right to left. if the movement detected is right to left, it is assumed someone is leaving the room, if the movement is left to right, then someone is entering the room.

- depending on these actions, software running on a machine in the zml keeps a virtual representation of bodies in the room -- one virtual object for each real person in the room. The camera tracking -- checking if people enter or leave the room, will inform this tally.

- another camera hooked up in zml tracks the movement within a particular space. by interacting with this area, open-house goers can alter the movement of the the avatars around the virtual space (which is projected)

- the sound of the piece will change depending on the number of people in the room

- the camera tracking will be done in softVNS, which will output a binary textfile (0, 1) whenever someone enters or leaves the room (0-leaves, 1-enters). Software then parses that textfile and updates the room avatar number based on the movement. svns also creates a textfile that determines the specific movements in front of the lab camera, which is also read in by the software.

so, this is basically highly ambient / exploratory / simple in nature. but I think it'd be relatively easy to pull off within a few weeks upon getting back.

sound community

started writing this test as an example of what a possible sound community might sound like -- part of my mobile sound communities project. in the project, users would create the yellow rectangles above, but the system would animate them, as this system does. when the shapes run into each other, they play whatever sound is attached to them. as I was writing this, I started think of iterations of this that place the user in a more active position as they are experiencing the sound. my moving an avatar (the red square, or using a tracking device, yourself) throughout the sound community. You physically engage each individual object, pulling sounds from each. In the thing I pulled the above image from, each object is simply a sine wave of a different frequency, but clearly that could be cooler). So in this iteration, the soundscape becomes more personal (almost a performance), and less communially experienced, which I'm not sure I like philosophically, but which I like from a game / toy / entertainment perspective. So I'm still focusing on the community engagement with the system, but this is an interesting strand. [written in p5, with amit pitaru's sonia externals to jSyn.

I would post it, but I don't want to bother anyone with the downloading of jSyn. the code is here: .pde

Gelson's Log

check it:

Gelson's Log

if someone could post some comments or something, it would aide my final presentation. thanks.

feedback instrument

Written in max/msp, the feedback instrument does the following: take camera input and generate musical sequences based upon total presence within the frame. Based on those specific sequences, download google images with search strings stored in a list. Then randomly choose one of those downloaded files and scan through the rgb values, outputing frequencies based upon the values of each scanned pixel.

Needed Files:
jit.uldl update
Matrix Processer
Main Object
Partial~

abstract|outline for 590 research

Mobile Sound Communities:
Sonifying a Shared Landscape
William Carter
Division of Interactive Media | CNTV
University of Southern California

ABSTRACT:
Mobile Devices create the ability for users to form new relationships with the environment by allowing them access to embedded virtual layers of information. With this new environmental awareness, users forge new and unique experiences with the physical world. However, the discourse surrounding mobile media is typically centered upon a single user, documenting the individual’s emerging awareness of these new virtual layers. What is often overlooked in this rhetoric is the importance of community within the environment, the existence of discrete, idiosyncratic spaces forged through community and shaped by community members. Users should be able to interact with their environment, but they should also be aware of the communities that exist within the space they are engaging.

Oftentimes, we recognize communities by ear, listening to the music emanating from an open apartment window, or overhearing a conversation in a foreign tongue. Sound and music are therefore important means by which we establish relationships with specific communities.

The goal of this research is to leverage the power of mobile technologies to grow sonic ecosystems that reflect the fundamental nature of a community. By generating individual sounds on a mobile device and planting them in a location-embedded virtual layer, users are given the ability to generate emergent soundscapes that reflect upon the surrounding community.

534 Proposals

I have a few projects I'm going to try and work on for the 534 final, since 2 of them are very similar and are group projects. All such proposals have already been posted, but you can track them with this post.

mobile media projects: w/ Kurt, Tripp, Todd

Location based Zombie Game

Patholog

Another project, to be completed in all the time I have...right...
Environmental Feedback Instrument

that is all.

anaglyph recycling

Anaglyph Stereographic Image of a decidedly Yuppie Product Sculpture that I am more than reasonably addicted to. (on sale now at gelson's, must go buy)

320X240 image is here

past | current projects

so I've been doing a little research on generative music, particularly with simple synthesized tones being instantiated in interesting ways. so I created some sounds using this additive synth engine I built with MSP and popped them into a flash script I was writing that uses really basic sine and cosine waves to scale, tint, and rotate a series of rectangles moving along a fixed horizontal (2d, soon to be 3d hopefully) plane. Anyway, the sounds a pulled from an array based upon frame rate, and a few simple rules which the user can alter using [sndVar]. Other changable aspects are the xscale_max variable and the rotate boolean. That project lives [here], or click on the thumbnail below.

I've done a bunch of other stuff that lives on the web, including my thesis project from last year [Main Travelled Roads], other flash experiments, and video clips from some selected installations I've done.

All of these media files have descriptions of the piece, and can be seen [here]. I also put [music] I've written up on the site, if any of you are interested, you can download mp3s there.

I hope that some of you will be interested in this stuff, and I hope that people will post examples of their previous work. I'd hate it if all this lip-service people are giving to *wanting to see previous work* will prove to be more than that, because while I know most of our backgrounds to some extent, I'd like to see some work.