a looking-glass book

iam updated

spent some time over the last week and a half tweaking code. i listened to the feedback offered at the show and managed to incorporate some of it into my project.

its nothing earth-shattering, but i wanted to take what i learned from having so many people use the system and try to improve it the best i could.

that said, here are the 3 major changes i have made:

1. the date of the currently viewed movie now appears in the blue stripe section. this way, you have feedback of what movie you are looking at. watching people use the system, they would skip the weekly timeline -- they wanted the calendar to pull up a movie. now, if i was ready to move towards a v2, i might try to do something cooler than i did before and allow that. (the reason for it not working like that is because many days have more than 1 movie and i wanted to display the relationship of a movie in terms of an entire day.)

2. on the timeline, the currently selected film is highlighted. this means when you select something, you get the visual feedback in this menu as well. when people were using the timeline and two movies were up against each other, you couldn't tell what piece you were viewing. this way, the selected movie is highlighted, making it easy to step through movies next to each other on the timeline.

3. finally, i added a javascript function that keeps track of where the quicktime movie is on the image timeline. the function is simple in theory - every 5 seconds, it checks where the movie is, rounds to the nearest minute and then highlights the correct image in the timeline. (in reality, it gets hairy and i had to cheat a little - i was already using the 'id' attribute, so i had to use the 'name' one for this. and name is not supposed to be a unique identifier. ah well, that was easier than making a pseudo-attribute. i wont tell if you wont.)
this is kinda cool, cause as you watch, scrub or skip around in the quicktime file, the outline of the image below will change. its a nice indicater that helps keep the relationship between the movie and images synced. plus its just fun to watch it move automagically as you play a movie.

like i said, none of these are earth-shattering, but it makes me feel like the project is more complete. there is always more to be done, but i feel glad to have taken the show and come out of it with constructive elements i could then use. i learned something and was able to make my project better as a result.

werd.

g4 appearance

big thanks to dave for commenting on the fact that my thesis was mentioned on g4 last night.

it airs next on sun, may 8 at 12am (est), 9pm (wt).

theres a nice blurb/recap here.

very exciting. and glad someone let me know about it; i would have totally missed it. (and he even blogged about it as well. god bless the internets.)

new recording device

thanks to erin for pointing this out to me. its a new camera from samsung whic sounds and looks rad. the downside is the battery life - amazon reviewers say its clocks in around an hour. way too short to be that useful. once we can get up to the 2, 3 hour mark, one of these will be in my hands.

linky link

press release

well...

spent pretty much the entire day rewriting the page. it isnt any prettier but it works.

got mentioned on we make money not art again. itll be nice to be showing the project soon.

got a whole bunch of links i need to throw up here too.
no rest for the wicked.

update

so i havent posted in a while. most of this is due to preparing deliverables that werent made for the web (like the defense and the papers).

but im back in dev mode. which is awesome because what i thought was going to be css and wrapup has turned into a full-fledged nightmare.

somehow, the code for the timeline just refuses to work, although it was fine before. i get it working for one week of movies and the rest seem to fall apart magically. im going nuts.

the biggest joy is the thought of rewriting the page this close to being done. but im starting to think there isnt much of an alternative. joy of joys.

but really, when you end up with if statements like this:

if ((($sec_counter >= $daymovs[$r][0]) && ($sec_counter >= $daymovs[$r][2]) && (($sec_counter - 3600) < $daymovs[$r][2])) || (($sec_counter >= $daymovs[$r][0]) && ($sec_counter <= $daymovs[$r][2])) && (($daymovs[$r][4]==($daymovs[$r-1][4] || 0)) || $r==0)){

what choice do you really have?

first thoughts on ffmpeg-php

1. this, esp as a php extension, is not made to pull 300+ images out of a video file on the file. it takes upwards of 6+ minutes to get through those 300.

2. it seems to slow the entire server to a crawl when it runs through the process. this obviously bears more investigation (and im sure marientina will let me know if im already destroying the speed).

3. this is all after compressing the movie to a smaller size - taking a 90 minute / 130 meg movie down to 30 megs. this was done by compressing the quality, the size and dumping the audio.

4. a solution would be the following: get the frame number. see if there is a file existing already for said frame. if so, output the existing frame. if not, create a new file and output the frame into it. then display that new file. the problem with this solution is this: its no better in the long run than dumping every frame out into a directory to start with. this new way requires some file system checks which might be faster than outputting the frame. but they system is predicated on the weighted good moments coming to the surface before you save out every frame of the movie. if we do it this way even for 1 frame a minute, the entire movie could be saved out in as few as 900 reloads. if we do it for 20 second intervals, then its 300 reloads. i am not convinced this is worth it. esp when you end up with 80,000 images in the folder. filesystems dont seem to like this.

so what do i do? well, for now my design shifts slightly and i see what happens with 1 frame per minute and letting the page take a little longer ot load. its a balance and if everyone complains, then i keep trying to find a solution. but this is one of those 'in 5 years...' kind of issues that doesnt seem truly within my scope this year to solve. i have bigger fish to fry right now than making lightning fast webpages. unless everyone starts complaining.

in the meantime, back to rewriting code....

success!

this might not look like much.

but it is frames of a movie, a qt file, pulled out dynamically on the server.

this is the culmination of over two months of trying to get this working on a server. this could not have been done with help from the developer, todd kirby, and marientina.

this bodes very well for much of what i want to do.

in looking at my production schedule, i am a little behind in coding (about a week) but i think that is made up for in movies. i have missed only 2 days since about the start of this month. i have 81 movie files, 99% of which will be published in the coming weeks. i do not know how many hours this is, but i do not fear hitting the magic 200 hour mark.

much of my backend code needs to be rewritten in the next week to allow for these new thumbnails. this will speed publishing and once this new code is in, things will move very very quickly - ill be able to publish a day after filming with very little post (basically compressing the movie and uploading it to the server - a much easier process than the current one).

even if i lag behind by a week, with spring break and gdc approaching, i will catch right back up to my production schedule with no worries. i am glad i gave myself some production wiggle room then. of course, there is still much to be done. but isnt that always the case?

db updates and onunload

bear with me as i sidestep this absurd ffmpeg-php installation and all its woes and think about 1 more piece. im just throwing this up so i can find these notes later; avoid this if you dont care about me thinking through a small little block of (complex) code.

so.

what i want is to keep track of when people click a thumbnail on the page. because if they click it, they want to watch that segment. which means they found it interesting somehow. so i can create an array in javascript of timecodes. actually, it can be a string. everytime you click an image, the 'onclick' is fired, calls a javascript function that adds that timecode to the string.

then, when 'onunload' is fired, another javascript function is called. this one loads a php file in a hidden iframe that updates the database. all we want is for each timecode to include a counter. (the db will have a primary key, the movie id, the time code and a counter. this could get pretty large.)

so why?

because then we can display the thumbnails based on interest. say a scene has 5 clicks registered with it. then we want to display that thumbnail over one with 1. though this sounds dangerous over time if weighted unfairly, i think thats just a balancing act and a set of ratios.

this way, we can track overall habits/interests of the viewers and turn around use that data invisibly to provide automated 'highlights'.

(between this bit and the ffmpeg-php, looks like the entire system will require a rewrite in a week or so. or basically as soon as ffmpeg-php is running correctly. whee iteration!)

no rest for the wicked

or the healing.

picked up the repaired sensecam down in long beach this morning. props to my sister for driving my pain-killered butt down there to get it. expect multitudes of pictures now - jim said in his test run last week, he took over 1400 in a day.

working with marientina on getting ffmpeg-php working on the server. this has been a much more frustrating experience than i would have guessed. she and i spent a couple of hours on it yesterday and then more time today. and this was after spending a good deal of time on it before i got sick.

does look like itll be fully functional in a few days at most, so thats good. i just hate the fact that even though im spending a lot of time on thesis work right now, its just installing some software. its one of those victories where you dont have a lot to show for the time spent. though once it is 100%, ill be able to pull out movie frames on the fly. this should be cool enough to warrent the time spent.

now i just need to figure out how to:
update a database on a click or on page unload.

im not sure this is actually possible.

e.r.

so tonight's episode of e.r. has a ton of pov video footage in it. i was hoping for something a bit more experimental, where we didnt leave the character's pov for the entire episode, but within the first 4 minutes we have already spent more time outside the character and jumped back and forth several times.

cant win them all, but it is interesting to see a recent example of pov video in such a commercial program.

back

im out of the hospital and back on my feet. mostly. if you want more info about the last week and half, check my site (roxy and carter posted a lot about it).

the upside is that i filmed every day i was in save one. not every movie is great, but at least they are unique. at least i feel like i was somewhat productive, even while stuck on my back, having morphine with a tube up my nose pumping my stomach.

bad engineering

the last couple of weeks i have been recording a lot.

or, rather, trying to.

problem #1 was the sensecam. it just didnt like me. so it is on its way back to england and i should have another one (or this one repaired) in the next week.

i have 1 days worth of pictures from it, though it didn't really do what it should have. i need to write a php file that will allow you to see all the images from that folder. thats on the table for this weekend hopefully - depends on how long the blog is down.

problem #2. this one was much larger. my powerbook was acting up. i would slide it into my bag to film a movie and *zoink*. out like a light. not asleep. not on. not on to the point it would reset my system clock. thats some bad news.

the failure rate for making movies was going through the floor. i usually can hit at about 70% success rate. meaning i get the file. sometimes the system dies, sometimes the camera gets fudtzy and loses connection. but i would say i get about 70% of what im trying to get. sadly, i usually lose it when im moving, which is far and away the most interesting stuff.

i went to the apple store today to see what they could tell me.

long story short, turns out it was the firewire cord for the isight.

note to everyone:
1. apples firewire cords blow. out of the two i have owned, this is the second that has gone bad on me.

2. thanks to some wise engineering, when that wire cuts out, the camera doesnt skip frames. it doesnt disconnect. it doesnt even crash the app using the cam. it doesnt put the computer to sleep. no, it cuts all power to the machine. all power. wiping the time/date and everything.

who was the ad wizard that came up with that campaign?

but, that said, im back in business. lets make some movies this weekend.

hp working on digital blur for faces in photos

now i understand why kurt said i should have this feature in my thesis down the road. he just wants to be the one to sell it to me. it all makes sense to me now. way to go hp!

link from /.

queue system

i envision this a bit like a playlist, a way to navigate through the movies, a way to make custom 'edits' (a tricky word, since you arent actually making a 'cut' but rather jumping from movie to movie).

this obviously isnt going to be a pretty version, as load times will make it tough. (see previous post about segmenting video for thoughts on that. its something i dont mind acknowledging and, if time allows, tackling. but for now i want to pretend it is a perfect world and we can get files in a flash.)

so you add things to the queue and can navigate like that. i dont have a good sense on a way to make it like a playlist yet - where you add these items, hit play and it does the work of flipping through them for you.

in fact, i am not sure that is a path i want to follow yet. if at all. it makes some assumptions - namely where the outpoint is to jump to the next movie. that should be your decision to make.

so i like the queue as more of a navigation aid.

and its mostly built now.

you can click on an item and add it to the queue. you can delete items and clear the queue. you can specify items down to the frame to go into the queue. when you click on an item in the queue, the browser heads there and starts the movie from that point.

its an ugly implementation right now. but functionally, it exists.

the obvious next step with it is the downloading/cookie-ing of the queue. does it get saved from visit to visit? or can you save it out as a querystring to send to friends? (though you hit the limit of 255 pretty quickly then i think probably. esp if you have a long queue. assume 8-10 characters an item...queue maxes out at about 15-20 items then.)

opinions from anyone?

(i have movies ready to post, ill get them up real soon. and the queue, as soon as i can make it look a little less ugly...if thats possible.)

Thesis Project Production

(thank you kurt for allowing me to steal your template of dates for this. this explains the non-tripp formatting for the week by week headers.)

------

Jan 25 - class 3 - Production Document Due

Project in alpha. Published to the web.
10 hours published to the web.

Feb 1 - class 4 - Production

Finish queue system and release.
20 hours published to the web.

Feb 8 - class 5 - Production

Have ffmpeg-php running on server.
30 hours on web.

Feb 15 - class 6 - Production

Wrap up backend of application.
40 hours on web.

Feb 22 - class 7 - Production

Redesign layout and surrounding pages.
60 hours on web.

Mar 1 - class 8 - Base Production Completed

Move into beta testing. Launch new designs and have complete package.
80 hours on web.

Mar 8 - class 9 - [GDC San Francisco]

Solicit users and feedback.
100 hours on web.

[Spring Break Mar 14 –19]

Mar 22 - class 10 - Lock Down Final Details

Tweak UI and recode backend as necessary.
140 hours on web.

Mar 29 - class 11 - Setup

Dress rehearsal
Install project in Zemeckis 201 (or alternate performance venue)

Apr 5 - class 12 - Internal Presentation & Exhibition Opening

First official showing and performance

Apr 12 - class 13 – Feedback/Tweak

Apr 19 - class 14 – Feedback/Tweak

200 hours on web.

Apr 26 - class 15 (last class) - Papers Due

[Study Days Apr 30 – May 2]

May 12 - Thursday (day before Commencement) - Public Presentation

segmenting qt files

jim gemmell suggested to me the other day some ideas about segmenting my video files. this is a cool idea, because then you kids wouldnt be downloading 200 meg video files.

it would work technically because you can link qt files to each other in the browser. so smaller bits to download. everyone wins. except:

windows movie maker will do this.

but there isnt anything for quicktime. a little digging finds this google answer. qt pro is the best program for this? i would get to do it by hand for each movie?

yuck.
so for now, this is a great idea for dealing with current bandwidth. but i cant spend the time chopping up these movies right now. the payoff isnt worth the time.

but its worth noting as a nice idea that makes a lot of sense.

now if anyone wants to write a auto-segmenter for qt files...

3 minutes later and here is an applescript that might work...

dorm life in pictures

an interesting 'public vs private' story about a photjournalist freshman getting kicked out after taking too many pics of a private nature. i guess i need to wear the cam around soliciting sex with it on to get kicked out of my dorm. oh wait, i dont live in a dorm.

story

google video has launched

http://video.google.com/

havent played with it too much.
now kurt and i have launched into a fight over whether the implementation is good or bad. well, its really neither but it just makes me a little insane.

like video games, video on the web is the hot thing right now. but its all the same - its all thumbnails and a few keywords. ive said it over nad over so i dont have to again.

but i suppose all of this activity is better than nothing.
this is to prove to kurt i am not just a huge whiner.

sensecam pics

the first sensecam pictures. sadly, after taking these 3, it has decided it doesnt want to take any more. so here comes the support call to ms. but at least ive gotten this far.

sensecam

there is one in my hands as i type this. should be posting stuff with it in a few days at the latest.

im very excited to have the opportunity to test out this item.

ourmedia set to launch

it goes live this weekend.

what is it? from the link:
1. an open-source platform and registry to bring personal media to the desktop;
2. a destination website at www.Ourmedia.org -- preliminary docs are
up on the site, with a preliminary launch scheduled for January 2005;
3. eventually, it will evolve into a not-for-profit organization
dedicated to advancing amateur, hobbyist, semi-professional, and
professional visual works and other media.

this is cool. it nails home that disk space is becoming less a problem and video over the internet is within reach.

the problem is this: its still just a front-end for a ftp server. (well, ym fingers are crossed that this isnt the case, but im not going to be surprised when it is.)

we still need some work actually getting into the files themselves. not only in interpretation of the content, but also in the representation. do we really need screens full of the first frame of a movie file? how much does this really tell you about what is there? why do i have to download then entire movie and scrub it to see what is in there?

surely i cant be the only one thinking about it in these terms.
nonetheless, a step in the right direction.

couple of articles

this creates some sort of weird feedback loop i feel like. i hope this isnt too tasteless (and that if it is, that justin can forgive me). but the comments made below relate so well to things about my thesis, i cant not put them up...

from grand text auto:

"A more likely explanation is somewhere in the middle — since Justin knows he’s going to be putting this footage online as he lives it, it turns the experience of his real life into a simultaneous performance, that he’s surely conscious of as he lives/performs it. Which exactly seems to be the problem he’s dealing with.

Clever! And impressively gutsy. And pretentious, but that’s okay. It’s another flavor of fusing of fiction and reality within the computer medium, I suppose. Reminds me to write another blog post on the similarity between reality TV and interactive drama."

----

blinkx does tv. looks like more and more we will be delivering tv through the internet. sad to me that this is the next killer app.

but what is interesting is this:

these video searches arent real video searches. they are still just searching on title, date, metadata. we havent gotten to the point where we can find actual content inside of video easily.

and thats no good.

(and im not even going to go into the whole "having more content jsut means we have more to sort through." thing. or the "having everything as content means we can burrow into our narrow subcultures more easily.")

iam is a go

the prototype shown at the end of last semester is live and online now.

it will obviously change a lot over the next several months - already several new pieces have been built and id like to get some new bits in by the end of the weekend.

for now though, the basic stuff is there. right now, there are only a few movies up - a few from oct when i was in nyc. again, i want to get a whole bunch more up in the next week or two. i have about 35 different ones sitting on my drive here. not too exciting though - most are of me reading or watching tv. these should expand in the next several weeks, which is another reason i want to get caught up quickly in what i am publishing. the newer stuff should get better and better as i get used to the system.

in the meantime, enjoy what is there and please feel free to comment on the system as a whole, feel free to leave comments within the system itself (using the small white bullets above the images) and enjoy watching these movies (they are on the large side - i think the longest is about 130 megs, so give them time to load).

link

m:robe 500

interesting new device passed to me by a friend.
looks like it take the photo ipod to a slightly more logical conclusion - it will make movies of the music and photos on the device.

could be cool - even if the device is bad, the idea of making these mini-movies is pretty cool. kurt, erin, will - any thoughts compared to all the stuff we are doing?

plrs

another old link. such good things happen when you clean off the desktop i suppose.

this one is just a few bits about the possibility of a 'personal life recorder', which seems like a silly post. we all know its going to happen and soon. we can stop talking about it as 'theory'.

link off smart mobs

2,000 cameras to watch city

this is several months old, i just found it on my desktop here at imsc.

looks like there is a new reason to (not) move to chicago.

"Mayor Daley officially opened a new city operations center Tuesday that will include a dramatic increase in camera surveillance on Chicago’s streets. The new addition to the 9-1-1 center will help manage and coordinate in response to emergencies such as a terrorist attack...The new system also has the ability to instantly report the sound of gun shots within hearing distance of the microphones planned around the city." [emphasis mine]

http://cbs2chicago.com/topstories/local_story_321123518.html

urls

so im going to buy a url to point to the site 'proper' when it goes live (hopefully in the next week). anyone have any domain ideas?

iam is taken in every version. and im not sold on iam-lifelog. or iam-blog or anything.

anyone have any strokes of genius here?

dev thoughts

there arent any markers for the white board in the lab, so im reduced to making dev notes here. i suppose this will do. please skip if you care not about database dev, html, javascript, etc.

so.

two things. 1 is the timeline, the other is comment weighting.

1. the timeline. pretty easy to do - a php loop writing out a span for each hour. pull the movie(s) data from the db - start time and length. you can find a start and end time then, round to the hour and assign a class to each span based on that. thats the basic code. then the placement on the page is somewhat tricky. thatll really have to be dealt with later as i refine the layout. for now, if i put it in the top right of the screen, then its sitting in the top iframe with the dropdown. which is ok. it just means that as you pick from the dropdown, then you need to reload that top iframe as well. and we need to tweak the drop to only show per day instead of per movie.

whew.

2. the comment weighting. without dealing with all the math yet, i just want to deal with the number of times a thumbnail is clicked on. i can keep track of that with a javascript array i suppose and then write that data out to the db on page unload. ill have to test this and make a new db table - itll hold 3 different counters: comment count, author count (which is really a boolean flag) and the clicker count. there will be a primary key, a timecode and a main id which links back to the main movie file. whew. thatll be some fun juggling too.

time to start on #1 in earnest and see where we land.

sorry, just had to reason some of this out so i knew where i was about to head.

equipment from the ces

i'm telling you - a year from now, hell, six months from now, this project would look very different. announced at the ces last week were two different cameras that would rule:
samsung announced one that looks a little like the isight.
toshiba announced one based on a tiny harddrive that can shoot 2 hours of footage.

sadly, apple didnt announce an isight2 yesterday. (note to apple: everyone would love to pulg an isight into a ipod and have it record to the harddrive. do it.)

(and failing that, why not make the isight available as a wireless bluetooth camera?)

ffmpeg-php fun

well, the good news first: this thing is designed to be built on a linux box. this means that it should be pretty simple to get it up and running on the server here at school in a week or so with some help.

the bad news is that it isnt made for osx. so im running into all sorts of problems getting it running on my powerbook. im enlisting help from people trying to figure out how i can make this thing compile but it isnt looking too good right now.

the current issue: what the flag '-d' means for gcc compared to cc. the makefile isnt cooperating obviously and im working what little c knowledge i have to try and figure out the different switches in compilers.

ug.

any thoughts or help would be greatly appreciated, those of you smarter in this area than i am.

how to pull images from a qt movie with php

for weeks i have been looking for a way to dynamically pull images out of qt movies. i couldnt dump every frame from a 90 minute qt file - i dont care what you say about filesystems, they arent truly designed to handle thousands and thousands of files in a folder.

i have sent out emails, set up meetings and talked to people about solving this particular hurdle. the best response i had gotten thus far was 'you could use java wrapped in python called from php'. i was to have a meeting on tues about getting this written.

and then, the clouds parted. the new year began with a bang.

it turns out there has been written (as i felt there must have been but i couldnt find it) an extension to php that allows you to pull images out of movie files dynamically on the fly.

i havent installed this on my server yet to play. but from reading the docs it sounds like itll do exactly what i want. bam!

so if you want to pull out images, frame captures or stills from a movie file for the web, on the fly and dynamically, ffmpeg-php might just be the thing you need.

big big thanks to kevin and meg for pointing me to this. (im amazed at how i havent met kevin yet still, but we manage to pick brains over web stuff coast-to-coast every 6 months or so. this time he really came through. big ups.)

expect a more complete review soon. (along with news of my other developments. i just couldnt wait to get this up and online.)

javascript controls of a quicktime movie (update again)

more things learned about the stupid workings of quicktime.
im adding a few more movies into the db for the presentation tomorrow.

what has this taught me?

1. the formular i described yesterday holds up for a movie that is 1:26 long. so youre safe at least for movies of an hour and a half.

2. that compressor is a funky thing. i used to try and squeeze these long qt files (1.78 gig) down to something more usable. it looks like mpg4, 160x120, stereo, 16kHz is passable. it took said movie and shrunk it to 131 meg. add in that it made it progressive and suddenly the wait isnt too bad for an hour and a half of video - esp since you arent reloading it all the time now.

3. but the funkiness of compressor? oh yes.
thats when you encode these files (or save them from mp4 to mov inside of qt) that this magical 'rate' changes. normally, the movie has this magical rate of 600. (see yesterdays post for info on that.)
but then, during some of this conversion, it seems the rate jumps up to 1500. i cant find any setting for this, i cant change it or access it at all. but yet, my movies were back to misbehaving until i changed the rate from 600 to 1500. thanks again qt - this is a really great feature.

javascript controls of a quicktime movie (update)

so at first i thought i was just dumb. this is common to me on some level (see: will telling me i inhabit a special place in stupid space today).

but even after seeing that video was controlled by setting an int of second times framerate, i couldnt make the movie jump to the correct point.

this was a problem. for wed, i could put the iframe back in, but thats no good. i dont want to reload the movie every time. and i especially want to be able to scrub. (starttime being set in the embed tag literally makes the movie start from there, no matter where you drag the slider.)

so its an issue.

after poking around for hours and hours, i found a page on apple (who has convienently removed the section on scripting quicktime controls from their site) that states this is not how you set the time int.

instead, movies are set by seconds times 600. this constant rate of 600 seems totally arbitrary. and i had computed this already last night by trial and error.

the great part is that seconds times 600 works great for small movies. but on one of my test movies, which is 26 minutes long, it starts slipping frames around 8 or 10 minutes in.

this isnt good.

so i keep digging all day, trying to figure out whats going on.

i never find anything, so i go back to the drawing board and decide to compute the error into my function. another nice chunk of time and i have figured out the function you really want to jump in a movie.

this has been tested up to a movie 56 minutes long, it might change once i start testing longer files.

but the function looks like this (where timer is seconds, in an int, of the distance from the beginning you want to skip to):

function startmovie(timer) {
timer = timer * 600;

//to correct for error, add 1 frame every 400 seconds of video
timer = Math.round(timer + ((timer / 240000)*60));

document.screen.SetTime(timer);
}

now dont tell me that the math line isnt hot. i mean, thats some sexy math to tweak something that is just horribly designed. well done.

javascript controls of a quicktime movie

there might be nothing better in the world than finding little documented features in programs.

several weeks ago, i found some links and info for the embed tag with quicktime. ive kept looking for info as i have been developing.

i have iframes right now nested three deep and every time you wanted to skip around in the movie, you had to reload the iframe to set a new 'starttime' in the embed tag.

until today that is.

using google for a long time, i found several pages that continued to spoonfeed info on controlling qt through js. most of these (for some reason) referenced qt4.1, so that was encouraging. maybe.

at any rate, i found one page on ken villines site and one on another site.

a quick page showed that you can control jumping around a movie file with js. so i get to rip out that second iframe and hold my movie in the same frame as my stills. this is great news programatically, but even better it means you only have to load the movie once. thats a fantastic performance boost and exactly the solution i wanted.

i just cant believe this kind of thing isnt better documented.

(oh, the function i wanted looks like this: javascript:document.nameofmovie.SetTime(22000) where 22000 somehow translates to 22 seconds. since im not dealing with frames, this is cakewalk for me to implement. in fact, i was already passing the timecode to the iframe, so its just a matter of pointing it within the document instead of through a querystring.)

***

and if i could read (from the second link above, but way up on the page):

movname.SetTime(time) For .mov files time is indicated by number of frames- for example for a 6 frame per second movie setTime(30) would take it to 30th frame or 5 seconds. For sound the time is indicated in miliseconds - for example, 22000 (22x1000) would take it to 22th second. ** note this setting is different than the hour:minute:second:frame notation used in the embed tag.

image dump

a 'quick' 30 minutes of image dumping and i have learned the following:

a 54 minute movie, with every frame dumped is a mess. it took over 30 minutes, when i killed it.

i suppose i could have let it finish, but it was unhappy. it had dumped over 65,000 images at that point, taking up almost 700 megs.

this is for less than 1 hour of footage, so obviously this isn't the ideal path to pursue. not that the math i did previously didnt support this. but now i have tried it and have found it to be something to avoid.

i have other things to get done by thurs for the presentation, but this is one of those 'this could be a cool thing' if i could work out the details. so i will keep playing on the side and seeing what i come up with.

im still trying to figure out a solution.

dynamic imaging

had a few talks about this in the last few days. i think i nailed the pieces im building right now, so im starting to think about plugins.

go with this for a moment:

i want to display thumnbails in some different dynamic ways. the ways dont matter.

but if quickly gets complicated.
because when you shoot 30 frames a second, thats a little over 100,000 still frames an hour. which is great, because i want to shoot a good 4, 6, 8 hours of footage a day. so thats somewhere around half a million shots a day.

and im still talking about at least 50 days of shooting.

that would be 25 million images.

thats a lot of images. thats too many images.
storage is an issue. filesystem support is an issue.

does anyone have any ideas how to overcome this?
do i dump 15 frames a second and let the other half disappear [as stills, they will still obviously be in the video]? do i find more clever ways to pull them out using user feedback? do i hope for sensors in the future (ala sensecam) that will recognize the good ones (but what makes good)?

25 million images.
50 million images.

i cant even imagine.

further sketches

im not sure what kurt and i have about drawing on napkins, but we had a good talk right before thanksgiving about uis. this was right on the heels of a good talk with todd as well.

both of these conversations, along with my own thoughts have led me to several interesting new ideas.

all i have been thinking about the last 48 hours is how to architect them; how to make them flow.

i need to sit down at a white board for 30 minutes real soon to capture it all.

in the meantime, ill ask to the air: is there a(n easy) way to dynamically create an element (either an option in a select box or a div of content) on a page, called from a child iframe?

i think the answer to this must be no, but id like to avoid the reload if possible. cause that will be ugly. but for now, it might not be worth worrying about.

video searching by the big 3

news article outlining how google, msn and yahoo are all working on making video searchable over the web.

well, ill let these guys work this one out for me. ill just keep on making the content for them to search on.

though it does open up another issue - if google is using the closed captioning for the searchable topics, it creates all kinds of hassle for the little guys, like me. what? now i have to not only transcribe my videos but i have to add them in as a close captioned track somehow?

dilemma

(yes, im working on thanksgiving.)

is it better to store all info in the format of seconds? this means that the backend will all be in the same format, but will require some math every time i want to spit it out. (or just once and stored in a variable.)

or...i store in a more human readable format for things (which is the way i have been doing it) but it means that i have to do do conversions the other way to be able to compare timecode.

i guess in the end it doesnt really matter - i have to convert one way to the other regardless. and i suppose since i already have most of the timecode stuff in a human readable format, there isnt any reason to shift it from that. (although, in the db, times would be stored then as largeints, as opposed to 'time'. but i dont think that truly matters either.)

i suppose i could dig around to see if i could find an argument, but im just not caring right now.

i just wanted to reason through it.

also, the page i am writing is getting annoying. i (fear i) am going to have to split it up, though im not sure how yet.

the architecture depends greatly on the functionality. and since that isnt completely defined yet, i imagine itll mean one of two things: 1. lots o rewrites or 2. lots of little pieces that i can plug and replug. (which i think comes down to lots o iframes. oi.)

happy thanksgiving.

movie sync

a little messy perhaps, but ive synced up the thumbnails with the quicktime movie.

here.

the movie is large (50meg), so it might take a few minutes to download. but then you hop back and forth, using some quicktime embed commands and an iframe. (on top of the other javascript and server-side stuff.)

to get the file down, i had to recompress it a bunch and drop sound from it. but then 50 meg for 20 minutes isnt awful i suppose, at least for demo purposes.

this will probably be the last demo i post this semester, since im converting everything over to php/apache/mysql now and the server isnt ready for posting.

'important moments'/timecode patch for max (v1)

i swear, you get rusty with max and it takes some refreshing. but here i have version 1 of my 'important moments' patcher.

right now, it only recognizes a single button press. in the future, i plan to have it allowing for several different types of button presses, to account for different types of moments. its also hardwired for the spacebar and to write to a text file on my desktop. youll have to modify the patch if you want it to do something different.

the max patch

so how is this all wired up?

well, i take the patch and make it an application. i take the powertrip and plug it in. i use the pref pane it comes with to make the global setting single click to send a space bar command.

this means every time i press the thing, a space bar command is sent. i make sure the max app has focus and im off and running.

when i finish my marking/filming, i am left with a text file of timecodes from the time the patch was open. this time should correlate (give or take a second or 2) with the quicktime movie i just filmed.

add the timecode into the database for the movie and bam! moments marked and synced up.

database tables

will, im half-looking in your direction on this one.
ive gotten mysql, php and apache installed on my laptop now.
(i have not yet begun to port my asp code to php though, for the moment that has to wait.)

i have, however, begun to design the tables i am going to need. right now, ive got 3.

major (movies):

-id
-date
-day of the week
-start time (of movie, 24 hour clock)
-length (of movie)
-interval between still frames
-height of movie
-width of movie
-framerate of movie
-root folder for movie and images (different folder for each movie)
-name of the movie file

minor (moments):

-id
-timestamp of moment
-alt tag/notation
-majorid (to connect back to the main table)

comments:

-id
-user email
-user ip
-date
-timestamp of comment
-comment
-majorid (to connect back to the main table)

can anyone think of any other data/metadata that i might want to capture? i think that covers most of it and i think i can extract the rest with some fairly simple math with a lot of the timestamping stuff.

but i'm sure there is something there i have overlooked (anything?) that might be useful to capture.

this isnt set in stone, but i would like to deal with it effeciently.

witnessing

i was on melrose briefly this afternoon. as i walked out of the store where i was picking up a few things, i saw a police car across the street. it had been there when i walked in, 15 minutes earlier. but now, walking out, i noticed the officers were behind their doors, guns drawn and pointed at another car.

people were standing on the sidewalk, on both sides watching. i stopped and decided to gawk as well.

and nothing happened. for about 10 minutes, it was a standoff. i was mildly nervous standing just across the street - if guns started firing, surely i was in a prime position to be hit.

and yet, i didnt move.

after about 10 minutes, another car pulled up. blocking the traffic, he hopped out, covered the officers and began unloading the car. 1 after another, teenage girls piled out - 4 in all. there was no driver by the time i got there (or else the driver had moved to the backseat).

then another car pulled up, right after the last person got out. the driver pulled out a shotgun and cocked it. i thought the car was empty, so this was an interesting turn of events.

they evenually moved in on the car, cleared it and the trunk. (the trunk was a bit nerve-racking, watching them tap and move slowly. i was wondering what would happen if there was a bomb in it. but there wasnt.)

all the girls in handcuffs and being loaded in, i left.

so why post the story here?

well, it was very patholog-y - i would love to know the story behind all this. its not often you get to witness something like this, even in la. so id like to hear the rest of the story.

and then, id like to share what i saw. ive had some dicussions with kurt about the narrative end of my thesis. he thinks i need scripts. i disagree.

and this sort of cinched it for me - not every moment is going to be precious. but without recording all the time, you wont have the stuff like this. and the ability to pick what is important to you (the viewer) is significant. to decide you want to watch me eat dinner vs witness a police showdown - well, thats just personal preference.

it also proves (to me) that you do need several ways to mark moments. i knew this before, but there is a difference between a moment and a scene. between something that has happened and something that will.

just worth noting how the everyday (and not so everyday) changes when you have the data.

quicktime timecode filenames

growl.

as smart as quicktime might be, it seems as if there is no easy way to dump an image sequence from a movie with the filenames containing the timecode.

ive looked at max/jitter, processing, qt itself, applescripts, software programs....

and nothing.

since im dumping at a continious rate, i can get that info back fairly easily. but having it dumped automagically into the filenames (or even a text doc) would be so much easier.

anyone have any clever ideas?

slight mockup

fleshing out the whiteboard from yesterday, this is a clearer basic picture.

yes, it resembles a lot of other programs (video editing). but it is missing a lot of other pieces - comments, queue, etc. but i wanted to get this down. its important to note that you have 3 distinct areas - the 'timeline' at the bottom, the 'preview/mouseover' on the left and the 'movie screen/ video playback' on the right.

gah. screens arent made fo so much concurrent visual data.

quicktime dev links

posting them here so i dont lose them.

qt dev reference

qt embed tag reference

what is sweet here is that the embed tag allows 4 nice things:
1. 'starttime'
2. 'endtime'

these are timecodes to set the pay times within a larger file. no chapter creation for me - all i have to do is output the timecode of the stills and then link with that start time to the actual qt file.

3. 'qtnext'

which lets you specify the next movie to be shown after the one playing. apparently you can link up to 256 movies in a row.

this is cool for several reasons: if i split files up for playback, itll let you hop from one 'chapter' to the next without any fancy wrangling.
also, if we do start talking some kind of composite reel or queue system for watching the best bits, it can be done pretty dynamically. i think. maybe.

4. 'enablejavascript'

which says it lets you control a movie through javascript. could be very very useful and i need to find some more info on this.

(kurt and i have been talking about my project a lot. time-code based comments is a no brainer. but then it could be pretty trivial to get the 'greatest hits' based on comments/views and link those chapters together. [the trick there maybe is to figure out a way to branch - to stay with the clip and see more or to branch to the next interesting bit.] 'greatest hits' could be a number of things - most commented, important moments, most viewed...im all about skipping around, as long as it remains easy to get to the original material. i dont want to end up with a system where the data is less accessible once you apply these types of filters.)

iam teaser trailer

Download file (quicktime, 3.5 meg, 2 minutes)

another mention

just got an email from rudy rucker.
it seems he mentioned me in an interview for the italian version of 'vanity fair'. i am not clear right now which issue/if it is being published. but really cool.

this is what he sent (emphasis mine):

For: Italian edition of Vanity Fair magazine.

Q 161. Uploading our mind, are we able to reach a sort of immortality?

A 161. Uploading your mind into any kind of more permanent form would be a kind of immortality. The big questions about this is how much of my mind can I actually upload. Sights, memories, feelings, thoughts, personality, soul...?
The memories are like a data base, while the feelings and personality is more like the operating system. But I don't think the operating system is very complicated. So I think that, if we could upload enough memories and (big if) we could build a "human conscoiusness operating system," then the upload would be a good model of your mind.
The feelings and personality are quite closely related to the physical connections and biochemistry of the brain. You might think you'd almost need to make a model of the actual brain. But my guess is that in fact there are only a few basic kinds of personality types, akin to the phlegmatic, sanguine, bilious,and choleric categories of the Middle Ages. So then, you just add a person’s memories to, say, a standard choleric operating system, and they’re back. A lot of what we view as our "self" really just has to do with the brain observing a model of itself observing itself. And this can be emulated.

Q 162. Do you think it will be possible? In this case, when?

A 162. To a limited degree its already possible to upload your ideas to society’s information network --- this is what we do in creating books or paintings. In the next ten years it will be possible for even non-artistic people to record fairly exhaustive memoirs about themselves by using a little device that I call a lifebox. The lifebox is like a cell phone that asks you lots of questions.

Q 163. Which are the most interesting experiments in this field?

A 163. This is a time-relative answer.
As of Fall, 2004. Microsoft has conducted an experiment called MyLifeBits
http://research.microsoft.com/barc/mediapresence/MyLifeBits.aspx
in which a researcher named Gordon Bell has digitally stored all of the paper memoirs accumulated during his life.
The cell phone company Nokia is preparing to market a system called Lifeblog http://www.nokia.com/nokia/0,1522,,00.html?orig=/lifeblog
in which a person can link and record all of their daily activities by using a cell phone. This is quite similar to what I call a lifebox.
A student named Tripp Millican at USC is writing a thesis about an interface for a filmed life blog. http://interactive.usc.edu/members/tripp/

Q 164. If our mind could be uploaded, what could happen? We could store it forever (but, why?) or transplant it into another body. Or what?

A 164. I like that you ask “why?” Why indeed! There’s enough people anyway!
But, still, most people would enjoy being able to talk to some kind of simulacrum of their ancestors. This would be a useful thing for ones sense of personal identity and continuity.
And, yes, the ultimate dream is that someone might provide your stored mind with a fresh robot body to use. You’d be back in action then.
I think this can happen. I would suggest, however, that machines are going to vanish and that biotechnology will take over, as in my novel Frek and the Elixir . So you would be copying your stored mind not onto a brittle robot, but onto a tank-grown clone of your (or perhaps of someone else's) body. This could happen in about a hundred years. It'll be like getting new clothes. Fashion magazines can market new-body styles.

Q 165. Maybe, could we learn a lot of things just downloading files like a computer?

A 165. I think you’re suggesting that it might be nice to reverse the flow and simply download files directly into the brain without having to laboriously read them. Like the old dream of learning something by putting the book under your pillow. Or by playing a recording of the book while you sleep.
Once we understand the biochemical basis of memory it could be possible to implant memories. The bad thing is that some powerful person might become a disease that other people catch. Like, a Republican sneezes on me, and I start thinking I'm George Bush?!?

ui v3

in my newest attempt to stay at the top of the blogroll, i give you version 3 of my ui, the version i have been thinking about for the past couple of weeks.

link

i think this one has some advantages to previous versions - you mouse horizontally now, not diagonally. pictures are not blocked by other pictures. the big thumbnails are a minute apart, the small slices are 20 seconds of time. you cant make them out, but by mousing over the stream, you can get a nice timelapse thing.

next up is to see if i can figure out how to jump to a point in a quicktime movie with a link/querystring/somehow.

thanks to aaron for hitting me up on im and offering feedback. always appreciated.

ui with nyc stills

again, not the most beautiful thing in the world (yet), but ive got a slightly improved ui with stills from a nyc subway ride with my sister up and posted.

i want to tweak this one a bit, having gone from the movie to the stills, i know the missing information here. and i know its not exactly the correct interface. yet.

the problem with it is this:
no matter what interval you use, longer movies are going to force you to drop data. there will be info missing. so i need to figure out a way to minimize this. ive got a plan.

taking #3 ui from the sketches posted a couple of weeks ago. use these pics as the boxes that show up on the drawing. take the 'missing' pictures and put them in as 2x2 boxes (or something) along the line. when you mouseover the timeline, you get more info displaying without having to show every picture at once.

i hope i can knock that out this week at least. (along with finishing this short demo movie and digitizing sounds for mobile class.)

if (big if), i have time, ill see if i can knock out some sort of version of #7 too. that would be ideal. but not likely by monday.

oh and here is the link to the nyc images. (again, its not overly exciting yet.)

dumper

got a max patch together (finally) that dumps out frames from quicktime movies. it isnt pretty (yet), which is why im not posting it.

but ive got it dumping out images based on intervals (thank goodness the input allowed for less than 1 frame a second when dumping).

so now, i can take the movies ive made, get image dumps and really start in on some ui work.

i know some of you out there are really into max - if you want to tackle some interesting but small patches, let me know.

id like to clean this one up, make one to recognize lighting changes using matrixes, make one to out put timestamps and then one to read/flag those timestamps back into the completed movie.

none of them are super complicated, i just havent had time to get to them all yet and would love to work with some people if you have time/energy.

thesis points for bolas

sharing/publishing

no one has dealt with a ui that effectively allows quick and simple review of the data glut created by wearing a camera for a long amount of time. in order to truly share and expect yourself or others to enjoy and review the material one captures, reviewing it must be quick and effecient.

content

no one is thinking about first-person recorded video as a narrative construct. the ability to create movies without sets or actors, from a first person view, without editing or cutting the video creates a completely different language of film than we have previously known. coupled with the serialization of day-to-day life, the dramatic/soap opera elements are undeniable and could provoke a new genre of personal cinema.

front ends

it seems that flash has to load an entire movie into itself befor eit can deal with frames within the movie. and it doesnt handle movies very well. as in, 500 meg, 1 gig movies are out of the question.

so no flash front end it looks like. at least for any sort of manipulation of the movies. maybe as some sort of interface to flip through.

so now im back to digging through max/jitter to pull scene stills, then all that will go up on the server as stills. so it looks like dhtml for the stills might still be the best display ive got.

i know something sexier can be done, but right now i really just need a front end.

figures

lessons for today:
1. when zemekis, director of such notable films as 'back to the future', 'polar express', 'castaway' and 'contact', stops in with cbs to film some work _and_ then singles your project out of the group to talk about the work you are doing: try not to sound like an idiot.

2. also, try to make sure your equipment is actually working. as you have a camera crew, lights and zemekis all standing over your desk as you sweat bullets trying to to get everything working again.

i dont know if ill be cut out of not (though word has just come up that they are coming back up for some pickups, so maybe i can redeem myself).

more in a bit.

releases

it occurs to me that by posting videos of my life on here, by releasing them from any sort of true copyright, im giving up the rights of my life.

kurt doesnt find this as odd as i do. but then it isnts his life.

griffen powertrip

ten minutes later, i have thrown down 55 dollars to have one of these in my hands by tomorrow.

i got it in silver. and will you blieve you can set it to pulse when the computer is asleep? now ive got visual indication if im not recording. sweetness.

hardware

there have been a number of interesting articles on /. recently relating to my thesis.

first up is the tapeless digital camcorder for your pocket. the review isnt great but it does bring up the whole 'one piece of hardware' idea again.

the other one just ran - thinking about the snitch cam. the article referenced is actually disappointing; it rehashes the same things that have been said 100 times. i suppose i really need to start working on a better version.

it does though link to a tiny camcorder which can be picked up for about 130 bucks. it runs on a flash card and aa batteries. its about the size of a pack of cigarettes.

i couple of weeks ago, i would have picked one up. now i can think of several reasons not to: windows only codec, files must be transferred to computer later, flash card fills up, crappy/uncontrollable conpression and quality.

though im not in love with having to carry my powerbook around with the isight, at least 1. the quality is mine to set and 2. it records straight to the drive in quicktime.

the last article was today and focused mainly on hardware again - the joypad that became a rotary controller. this is interesting because of a link over to griffen's powermate. this seems to be a nice little dial/button that goes in via usb and can be set to do 'something' on the machine.

this might be the best/most attractive button idea ive seen yet. if i can configure it to mark moments down (timecode/clock), this looks better than using a mouse or some other joystick to do the same thing. at 40 bucks it isnt outrageous, maybe i can find it cheaper.

iam uis

some sketches for possible uis for dealing with the data glut im working on creating.

for now, ill prob be trying to build a few of these in max/msp/jitter. would like to move it to flash frontend with php/asp backend at some point. just dont want to spend the time ramping up on flash unless i have to.

back to developing.

#2 is the one from last semster i have already built in max with urt. #4 is the one i thought i would use. #3 or #7 look like the most likely ones in some form right now.

thoughts or ideas?

re-blogged

woah.

here and here. weird.

i hate the reblogging thing personally. but at least im getting exposure. now if i could just find a flash developer.

bueller? bueller?

pics of iam in nyc

i walked around after the conference with the camera on. my friend meg and i went to a street fair near times square. then my sister and i walked around the dumbo festival out in brooklyn.

meg just sent me some pictures of my lowtech setup, so im passing them on. as soon as i get this video compressed, ill be posting it as well.

also

i am the 'camera on head' guy today. other than mann. we are the only ones recording with mounted cameras. and jim was nice enough to point me out to the group.

granted there are 2 sensecams, marc davis is using his cell phone for video and other people are using video on their digi-cams.

but im the video kid.

the end, its over.

more mann

he states that the new cyborg age will force honesty. that two-faced people, dishonest people, liars will not be able to cope in the new age of recording everything.

one more from carpe

no one here, not a single person other than me, is thinking about using pov 24/7 video for entertainment. no one is thinking much about narratives even. or even really that other people would want to see them. that has been brought up at least. but its mentioned in a family sense, not as 'wow, i wish i could see the videos of lincoln's life' kind of thing.

thats insane to me. every use listed is very practical, very...selfish.

this is a huge strength for my project.
this is a great reason to be in the film school.

(also, the question just asked is how to access the data created. and amazingly, no one has a solid answer. one more strength for me. least i have picked good areas of research for contributions.)

random carpe notes

. looks like i might get a deja view to experiment with after all
. spoke with a recent phd grad from media lab. interesting to hear his thoughts and experience
. big props to dr drucker, who is sitting next to me as i write this, for walking me around when the conference started.
. steve mann, who actually offered to be on my thesis committee, invited me to his conference next year (hopefully to present) and offered whatever help he could.
. ive got video of almost all of the sessions, but its small and im not sure of the sound quality. i have to shrink it further to get it on the blog, so im not sure it will end up being worth sharing. once i get cleaner installed, ill see what i can eek out.
. met the eyeblog guys and have some media of myself from them. was cool to talk about their work, their experiences and their publicity (they left early to go get interviewed by 'good morning america'.)
. it looks very very likely i will get my hands on a sensecam in the future to play with narratives on it. exciting to know that i will be able to work with a variety of hardware systems this year. being able to explore video/audio vs. video vs. stills is great.
. this was an amazing day. so nerdy but so compelling to me. i feel caught up, educated and linked up. i have felt behind since i got sick, but just this one day here had made me feel on top of my game. i am now part of the community. i am ready.

session 4

Memory Cues for Meeting Video Retrieval - Alejandro Jaimes, Kengo Omura, Takeshi Nagamine, Kazutaka Hirata

Total Recall: Are Privacy Changes Inevitable? a position paper - William Cheng, Leana Golubchik, David Kay

meeting review tools, how can we leverage video to get info out of it? video is never used now to review what happens in meetings (events).

this first presentation seems very geared, like the business one in the previous session, focused on making video and content and computers into personal assistants.

there seems to be a drive to push the desktop computer away from desktop and more towards a shoebox. a collection of data that you pull up. memex memex memex. i wonder if this is to be the future of computing.

again, this goes back to the 2 uses for this stuff - personal assistant vs sharing experiences.

the front end for this demo is in flash. and it seems mostly graphical, to be used for business meetings. employee faces and layouts of rooms are shown. you drag images into the 'search box' to get info back. an interesting way to search on a limited dataset.

it also shows the camera location when you play the video back and you can pick scenes and speakers. and seating arrangements.

pretty cool, but somewhat niche.

they are doing basic motion tracking for a lot of the actions. also cool, but then there will be someone complaining that we are being taught to behave a certain way to get the system to function. i think its needed on this first step, but id like to see if it could be gotten away from down the road.

again, these guys are ignoring audio. its weird that audio and video are being worked on so independently. be nice when they each get to a good point and someone matches them up. i like the idea of searching the video of my life for quotes people said. now thats some good stuff.

---

holy crap.

this lecture, the second one, is from usc/imsc/isi/cs.
their project - total recall
ill have to talk to her after her presentation. this is incredible.
i might have found my lawyers and info. wow.

i had to fly across the country, to find out someone is working on this literally in my own backyard. and they want to record everything in someone's life.

this could be a match made in heaven; they still seem to be lacking the entertainment content.

excuse me while i try not to make loud, happy noises in my seat.

and scott (or anyone) - did you know about this project?

mann brings up nonwillful blindness. (encrypting video with a key that is held in part by numerous people who hand it over as long as mann is not under duress. forces the system to subpenoa the entire universe to unlock a video)

didnt post much about this one, mainly because its local to us and ill be speaking to them soon i am sure.

session 3

Personal Chronicling Tools for Enhancing Information Archival and Collaboration in Enterprise - Pilho Kim, Mark Podlaseck, and Gopal Pingali

Uniscript: a model for persistent and incremental knowledge storage - Adorjan Kiss, Joel Quinqueton

thanks to firefox crashing, i lost my post for these two sessions.
im a little lucky, as these two drifted away form my work and talked about organizing data/text in business and philosophical ways.

increased productivity and relational database organization...
it was decent and interesting but not my cup of tea.

sorry this seems short, ive lost the notes forever and am frustrated by this meager portion.

session 2

Minimal-Impact Audio-Based Personal Archives - Daniel P.W. Ellis & Keansub Lee

Passive Capture and Ensuing Issues for a Personal Lifetime Store - Jim Gemmell, Lyndsay Williams, Ken Wood, Gordon Bell and Roger Lueder

bolas, this audio lecture is for you. pretty cool, but he just said he still doesnt have the killer app for it yet. true that. it also seems like that darpa proposal is all over the place. i would be surprised at this point if there was anyone in the room who didnt apply for it. mr. fisher, i want to hear more.

ive got jitter installed now, so i think it might be time to look at making a patch and then exporting that out as an exe for systems. its not webbased, but that might be ok. esp as i dont have a flash developer yet.

anyone know of one who wants to work on a cool video ui for free? i cant promise money, but it is a cool cool thing they would be helping create. i can almost promise some level of exposure, based on the people's hands i was just shaking in regards to my project.

so he is just recording audio on an archos 20 gig drive. so he analyzed the audio at 1 minute intervals instead os something like 10ms. for the 'diary app', he is looking more at background noise instead of foreground.

i ought to talk to rachael and see if i can draw up a survey for people once they wear the system to see if/how their behavior changed. thats going to be important.

im also worried that the isight's mic isnt good enough. and it sucks in low light. but still.

also, the huge cup of coffee i just took in is kicking in. wow. no more sleepy tripp.

---

sensecam wants to filter at capture time. so you get the flipbook thing. hm. drucker really thinks the isight is good, as opposed to the stills sensecam does. i think the sensecam can be a basic, bad framerate video camera. but if im not getting one to play with, its moot.

as a side note, the device is small. think gba sp size and shape. not that bad. no way to tag important moments though.

davis agreed i had to have it. i dont disagree, but to be honest, i keep forgetting this isight is on my head, given that, i dont know ill remember to hit a button. i want to wave my hand for important moments, but so far, it hasnt crossed my mind.

there are 12 sensecams right now. guess ill go beg during lunch. cant hurt. (oh and ken wood has said that he will put me in touch with the ex-producer from seacrest. so between that and bridget's help at ca, i think ill be able to explore reality tv a fair amount.)

sensecam people are doing a cross-cultural privacy study. he says that other cultures are just now getting used to the idea of privacy. important to remember that we arent the only culture and other ones function much differently.

rich, the coder of mylifebits, suggested looking for 'web recorders' for the mlb function i like of saving out every page you visit. unlike history hound, google desktop, etc etc, it doesnt use the cache. thats the real important part. so ill have to dig around later.

man, mylifebits is so hot. i love it. but its almost too much. i hope that a more useful/lightweight version can be built. itll make me want a pc again.

mlb thinks the killer app for pic and video is the screensaver. boy oh boy, i hope i can convince them they can do more. cause its neat, but so limited. esp when you dont have segmented video.

breakout

5 minutes after that post, dr. drucker walks me around a bit. meet the sensecam folks, walk to get coffee and actually meet and speak with marc davis (weird timing after me mentioning him in my last post) and then speak to the deja view folks.

looks like, all around, that the isight/powerbook combo is the way to go. i need to find a way to tag important moments, but it doesnt seem like there is a better device. ive got some more people to talk to today, but after today, im going to feel mostly committed to my hardware.

thats good news. i figure if all these people, the top people in the world looking at what i am looking into cant suggest better things than this, it prob isnt available. deja view says again, in 8 months, they will have the system i want. stupid stupid stupid. i wish i could get a promo unit now. but they seem intent on being annoying.

now the next lecture has begun.
this is already a freaking amazing day.

session 1

Efficient Retrieval of Life Log Based on Context and Content - Kiyoharu Aizawa, Datchakorn Tancharoen, Shinya Kawasaki, Toshihiko Yamasaki

A Layered Interpretation of Human Interaction Captured by Ubiquitous Sensors - Masashi Takahashi, Sadanori Ito, Yasuyuki Sumi, Megumu Tsuchikawa, Kiyoshi Kogure, Kenji Mase, and Toyoaki Nishida

wow.
everything that is being talked about is right on target with what im doing. the gps, annotation, i even just saw a drawing that was exactly what kurt and i drew last semester for perry. whats amazing is no one is talking about ui or entertainment.

things seem to fall into 2 categories - memory/personal assistant and then more open, shared content delivery systems. most people seem to see this as the former, not the later. i just dont think that is how it is going to play out - the networked life is too strong now.

---

a video summarization system was just suggested.
its the wrong approach, nothing against the guys talking about it. but why would you want to throw data out? isnt a better interface one that allows you to set the level of detail? go for the powers of 10 - where you can zoom in and out. simply creating a highlight reel is nothing but auto-editing. hello, marc davis. but no thanks, its not the way to go i think.

ive got to find out what some of these people are using, hardware-wise. many are using off the shelf stuff.

interesting - they synced up 4 videos by timestamp and are playing them all at once. they are all pointed at each other, so its nice. very time-codish. but i dont know what the real use of it is.

hm. maybe there is something important to think about the camera position with pov. no, not position, identity. ground. mounting a camera on a child is different than a baskeyball player. how does that type of thing inform the narrative and the character?

seems people are using units like this to help reconstruct space. either by using multiple cameras or (as mann did)) composite a tracking shot into one image.

mann pt 2

boy, this is fun - post after post.
interesting, mann is suggesting a pair of usable digital glasses, which could donm 'anything' - prescription glasses all the way to ar, with facial recognition in between.

he tested his eyetaps at gambling places and got away with it. 'if youre electric eyeglasses are working well, you should be able yo wear them without being discrimanted against.'
thats just rad.

the video that is playing not is work he doing with a phd. he can use the glasses to block out billboards and augment them with other stuff.

mann has been been involved with 'possibility of copyright infringement.' cyborg law is actually being taught as a class. and so now he wants to wear the camera for safety, as opposed to partial. streaming the content has 'saved his life' he thinks.

the talk just finished. holy crap it was good.
ill post the video in a few minutes.

questions:

wow, he claims he must wear this - he has 'wrecked' part of his brain and needs the system to function properly. and thats how he gets around legal issues with his lawyer.

someone just asked about the 1:1 map question. he kinda glossed over the answer, just that he has done some work and its an issue.

his nice eyetap glasses cost him about 500,000 to build. so while he just kinda slammed 'lipstick cameras mounted to a pair of eyeglasses', the odds of me getting one is more than slim.

he explains the mediated world and how his eyetap is removing the advertising, etc.

mann's talk

im recording it, but the size is small and the file is still going to be large.

ill post it later, but its funny to have mann reference joi ito's moblog. kids, it really is a small world after all.

carpe 2004

im at the carpe conference today in nyc. sweetly, i got a book of all the white papers being presented today. some really sweet stuff. at first, i was overwhelmed and was worried my project was going to retread ground, but there seems to be nothing being done yet with a good ui and entertainment. so i win on both counts. ill update through the day; i also plan on video recording a bunch of it. there are going to be people around wearing sensecams, ill be sporting the isight and steve mann is presenting. so wearable computing/lifelogging is in full effect today. nerd on.

(also, i saw roger zimmerman from imsc here for the immersive media talk that is happening today in this same building. again - its a small world. i dont think he saw me though. or he did, but i dont think he realized it was me. because really, why would he expect a student worked to be at this conference on the other side of the country? still fun.)

/. roundup

another piece on video surveillance and privacy here. this one is a bit less weird - a roommate installed a hidden camera in the bathroom. (the comments range from 'nothing you can do about it' to 'def illegal.' not sure where it really falls.) nonetheless, for me, it raises a bigger issue - how do we opt out of things like video recording?

and

'possible spectator mode for xbox 2'
to which id like to say congrats to dr. drucker on this one, finally.

u2 slideshow

similar to some of the stuff im working on/towards is this little flash piece.

its a set of stills, but let the pics load and then give the flash piece focus and hold the space bar, itll do a neat little slideshow kind of thing that ive been working on implementing in javascript.

cough.

ive been _trying_ to implement in javascript.

right now, the page is a bit of a mess, its in progress. but im taking the 640 pics from ea (which ill post in a better format/way soon), splitting them into chapters based on content/color/location. finding a ratio to apply across each chapter and then spitting out the right number, separated by that ratio.

when you mouse over, the image at the top swaps. hopefully by thurs, when you mouse over, it starts a slideshow, switching that picture with another in the equence (the full 640 image sequence, not just the 60 or so displayed on the page).

sadly, setTimeout is a fickle fickle beast and doesnt seem to work like it should. so im still wrestling.

once i get it working right, ill post and would love to hear your thoughts on the experience.

whew.

new thesis woe

well bolas, here go your thoughts on my thesis as porn.

from this article:
"Calling video voyeurism the new frontier of stalking, the House on Tuesday approved legislation to make it a crime to secretly photograph or videotape people, often for lascivious purposes.

Under the legislation passed by voice vote, video voyeurism on federal lands would be punishable by a fine of not more than $100,000 or imprisonment for up to one year, or both.

The bill would make it a crime to videotape or photograph the naked or underwear-covered private parts of a person without consent when the person has a reasonable expectation of privacy."

anyone have any thoughts on this?
i was worried til i read that last paragraph. even so though, seems worryingly vagueish.

dhtml sketch

this isnt exactly right on yet and it probably wont stay at this url forever, but heres a rough draft of the ui im working on for my thesis.

the mouseovers arent behaving correctly yet, but thats obvious.

the 'cool' part of this thing is that its being done dynamically in asp. the images for each 'chapter' are in separate directories. im reading through the directories and writing out the images with changes in the style for each one.

whew.

'the final cut' trailer

thanks to mark bolas for finding this - it seems the movie 'the final cut' is my thesis in a hollywood movie. im not sure whether i should be happy or sad.

Thesis Proposal

elevator pitch

iam is:
24/7 point-of-view video,
published to the web as serialized metafictional video blogs,
in a layered/drill down UI,
exploring narrative possiblities and new types of personal filmmaking.

2 page overview (pdf)

abstract

iam is a research project to explore and catalogue user created point-of-view video. When recording media today, one pulls out imaging equipment and records the scene. But when the equipment becomes visible, the scene itself changes. People behave differently knowing their actions are being recorded. To this end, the iam system helps the user create personal stories while they are occurring, without the author (or the audience) being conscious that recording is happening. Not only does the author become the director, editor and actor in his or her own movie, the point-of-view video allows the author to review (and relive) his or her experiences directly from his or her vantage point. Others are able to experience life through your eyes, able to see your life story in front of them to experience through your eyes. Once this large amount of video data has been recorded, software is needed in order to navigate through it. Using data analysis, scene extraction and meta-data, iam provides an interface that allows the user to quickly absorb an overview of the entire video. Using the system, users are able to create personal content, share it with others and easily navigate and browse the videos they have created.

thesis proposal (PDF)

Excerpts from PDF:

Project Description

Twenty four/seven video recording is becoming a reality as technology advances. Storage space continues to grow, ensuring that data is no longer worth destroying. Camera technology also continues to improve, creating smaller, better recording devices. These cameras can be easily mounted on a person, recording what a person sees: point-of-view video. The combination of these two factors leads to one inevitable conclusion – in the future, everything will be recorded and saved.

The iam system creates a personal hybrid of television's reality show and scripted drama. It will naturally change the way authors live their lives, while opening their experiences up others on the Internet. It is not difficult to imagine the system to be used as a memory aid, as a personal nostalgia instrument, an extended personal diary or photograph collection. These creations allow others to see life from your point of view. The movies will change the way one looks at the world and the way one moves through it.

These are all valid, but simple, uses. Iam gets more interesting when one begins imagining creative content beyond these scenarios. Amateur serialized movies, much like television soap operas, are possible and can be easily produced. Staged events and friends playing characters blur the line between reality and scripted drama, not unlike present-day reality shows.

Additional usages for the system become apparent when one looks beyond personal use into community and team-based situations. Training, reconnaissance, even anthropological community studies offer new content opportunities to a variety of authors and viewers.

But the creation of this much video creates a huge data glut. Unlike traditional media, time-based media requires the audience to spend a set amount of time with the creation. While a painting can be viewed for as long (or as little) as one wishes, video (and other time-based creations) requires one to stay with the entire piece in order to understand it fully. When the piece is not only shot in real-time but is also continuously being produced, the ability to watch everything that is shot becomes an impossibility. From the creation standpoint, trying to edit this much video is formidable. And with storage capable of holding all of the video, there is no reason destroy any of it.

What is needed is not to present the viewer with less video, what is need is better summation tools and a UI suited to handle large amounts of video data. The iam system auto-summarizes the video for the viewer, saving time and helping them understand the full video without having to watch it.

By incorporating a button with the camera, the author is able to tag moments during his/her day as 'important'. Software can analyze video for changes in scenery and lighting which further breaks apart video into separate scenes. By offering a stackable interface, the audience is able to quickly understand the complete content of a video in a way that was previously impossible.

Unobtrusive recording from a personal viewpoint offers a good deal of personal recording opportunities, allowing you to review your actions and reactions throughout the recording period. This alone is an interesting proposition, but when coupled with the ability to share these same events with others so that they can see what you have seen, it literally offers a new way to view the world.

Scenerio

The ability to watch another person's life through their eyes would be amazing. To see what they have seen and to understand the events that shape their perception of the world is something that has never before been available. While content creation could begin in this way, it could quickly jump into more sensational video creation – reality television and Jackass movies will spring forward. From these movies, it is another simple jump to more serialized dramatic fare, mimicking soap operas.

Jimmy puts on camera, complete with empty memory card (for recording) and button for marking important moments. Turning the system on begins the recording, straight to the memory card. Because the camera is mounted to the glasses Jimmy wears, it records what he sees. (Future versions hope to actually be able to follow the focus of the eye, not simply the field of vision.) Jimmy goes about his day, or, in the case of the movie maker, he records his fictional movie (hyper-real/meta-fictional cinema verte). * As Jimmy sees or experiences interesting moments during his recording session, he hits the button on the camera. This sets a flag on the video, noting the timecode. These moments can be anything from seeing a billboard as he drives to work to a humorous conversation over the water cooler. It is these moments which will help the viewer relate to Jimmy's experience later, while the video in its entirety will offer a complete record.

Jimmy has decided to record a meta-fictional narrative. He has spent time 'offline' of the iam system, scripting out the next chapter in the ongoing film of his life. He begins recording for the day, having let his friends and actors know what parts he wants them to fulfill during the day. He has scouted locations for the day's filming as well.

Jimmy gets a call from Frank, one of his friends, telling him to head up to the Hollywood sign around 10am. This friend owes him money and, although the location is odd, Jimmy is curious. He drives there and then hikes the rest of way to the sign. As he makes his way to the sign, he takes scenic routes, trying to compose interesting shots from his point-of-view. He tags several of the shots he composes.

Frank is waiting for Jimmy when he shows up. Previously, Frank had gotten himself in trouble he never clearly explained to Jimmy. They are friends and Jimmy loaned Frank 750 dollars to help him. The money didn't get Frank out of his problems; while Jimmy thinks he is there to get repaid, Frank is intent on asking Jimmy for more money.

While they are meeting, Jimmy gets a call on his cell phone. He can feel it vibrating in his pocket, but not wanting to interrupt the current scene in his movie (and knowing he can't reshoot), he ignores the call. Later, during a lull in the narrative or when he has wrapped for the day, Jimmy can check to see who was calling him.

He and Frank get into a fight over money and during the argument, Jimmy makes sure to press the 'important moment' button on the recorder. Frank threatens Jimmy and then storms off. Jimmy follows Frank down the trail to the cars, prompting another fight (which Jimmy also tags) before Frank pulls off in his car. Jimmy checks his phone to see who was calling.

It was Jimmy's girlfriend, Jennifer. She wants to have lunch with Jimmy on her break from work. He drives near her workplace, again taking a strategically planned route through the city. He picks her up several blocks from where she works, attempting to provide some privacy for her. He doesn’t want viewers to figure out exactly where she works, for fear of her safety.

He picks her up for lunch and they drive to their favorite Mexican restaurant. He doesn't mention Frank, not wanting to bore his viewers with redundant information. In fact, as he records things, he makes sure to move his head often, offering new information and vantage points to the camera so his audience has new content continuously. This is important as Jimmy doesn't have the luxury of editing his shots – everything occurs in a single, real-time shot.

In the restaurant, Jimmy spots one of his friends. Unfortunately, Jimmy hasn't scripted his friend into the narrative recently and wants to avoid ad-libbing right now. He ignores the friend, actually going through elaborate movements to avoid being seen and causing an awkward moment. Jimmy and Jennifer discuss matters over lunch; Jimmy steers the conversation in specific ways, fully aware of the items he is speaking aloud for his viewers (and intentionally leaving other pieces of the conversation vague and unexplained for another episode).

When Jimmy is finished recording, he turns the camera off. He removes the memory card from the camera and connects it to his computer. He uploads the recorded movie to his web server and publishes it. Jimmy can tweak, add and delete flagged moments if he wishes at this point.

The server processes the file for other users, creating stills and flagging the moments Jimmy marked. Jimmy is done authoring. The system provides the scene breaks for his day (his apartment, the drive to the sign, the encounter at the sign, the drive to lunch and lunch) as well as the tagged moments within each scene.

The system parses the video for scene changes and flagged moments. These are saved out as stills. The longest scene is then split into 20 chapters, with the first frame from each chapter saved as a still. The space between each chapter is then used on the rest of the chapters. This produces a stack of frames for each chapter, all separated by the same amount of time. Important moments are read back in and also saved as stills.

Jenny visits Jimmy's site. She selects the day/movie she wants to catch up on. The computer returns to her a horizontal line of scenes, with all stills for that scene layered behind the first. Important moments are indicated by an arrow above the still. Mousing over any of the stills raises it out of the pile. Clicking on any of the stills opens the movie to that location and begins playing.

Jenny is able to quickly scan the chapter list and visually tell the length of each scene within the movie, as well as being able to quickly browse (and watch if she chooses) the interesting moments of Jimmy's day. She has the option to spend as much or as little time with his movie as she desires, while still being able to absorb the overall story quickly and easily. She will also return regularly, becoming more and more involved in the ongoing narrative of Jimmy's life.

Images, References and Schedule all are within the PDF.

* Several important features differentiate Jimmy's fictional movie from more traditional films: 1. Although the movie features Jimmy, Jimmy does not actively star in the movie. Because the film is Jimmy's POV, Jimmy's face is only seen in reflections, much like real life. 2. The movie is one continuous take. Much like "Time Code" or "Rope", events occur in real time. Jimmy does not have the option of retakes or pickups. 3. He is living the movie. Jimmy isn't creating movie magic – he is experiencing it. The narrative he is telling is also the story of his life, whether it is fictional or not.

thesis scenerio

here is an explanation of how a system like the one i am building could be used, how the system might function. sorry the screenshots are so small, ill see if i can post larger ones up. i don think youre missing anything though - the important stuff should be fairly easy to grasp in them.

Download file (pdf)

bullet list

so heres a bulleted list of a lot of the stuff im working towards. this should at least give an overview of all the notes and thoughts ive been collecting the last couple of weeks regarding my thesis. more later im sure.

• Main pieces of project
• A system with POV 24/7 video, publishable to the web.
• A UI for the end user, as the time and energy required to sort through such a glut of data/video is overwhelming.
• In addition to the system, the narrative possibilities for such a system are extremely strong and I want to focus on them as well.
• See Applications, I plan to focus on the Bloggers and Hyper Real Fiction Sections
• System
• Treat movies as photos, grouped by time/location
• By incorporating a button with the camera, the author is able to tag moments during his/her day as 'important'. Software can analyze video for changes in scenery and lighting which further breaks apart video into separate scenes.
• Can display pieces of movies as stills on the web within a DHTML interface.
• Easily tweaked
• Requires no plugins or special code
• Use MyLifeBits codebase for publishing, UI and movie analysis/processing.
• Must confirm camera format and ability to process frames within movies. (Requires non-web coding. Red flag.)
• Applications
• Bloggers
• Travel diaries>
• Family History
• Time Capsules
• Entertainment
• Hyper Real Fiction
• Non-edited
• Continuous Shot
• Meta-Fiction – reality and fiction overlapping
• Pornography
• Training Videos
• Surgery
• Flight Training
• Any repetitive action that is dangerous to learn in the real world
• Soldier commands
• Remote leadership
• Anthropological Studies
• Artists/Notable personalities
• Location experience
• Community studies
• Replay Events where perception and POV impact outcome
• Other Research Areas
• Social networking and interlinking videos
• Privacy issues
• Permissions and editing
• Personal Cinema
• Hyper Real Story-telling
• Linking Electronic Documents To Video (MyLifeBits)
• Meta-data analysis (SenseCam and pattern-matching)
• Audio Analysis
• GPS/location mapping
• Research Links
• Lifelog on interactive
• MyLifeBits
• Sensecam links
• Anil Dash's Personal Panopticon
• EyeBlog
• EyeBlog on /.
• National Geographic Crittercam
• DejaView
• Camera News
• Nokia LifeLog on interactive
• HP Wearable Camera
• HP Wearable Cam on BBC
• HP Wearable Cam on PC Mag
• History of Wearable Computing
• VBlogging Central
• Vog Manifest
• Uploading with the soul catcher
• Uploading
• Nokia Lifelog info
• Rudy Rucker (guest blog on boingboing)
• Rudy rucker's homepage
• Rucker's transrealist manifesto (pdf)
• Committee


• Scott Fisher


• Mark Bolas


• Steve Drucker


• Rudy Rucker

simple steps

1. investigate interface, sensecam and mylifebits
2. strip down mylifebits to something more manageable, add publish button, deal with links
3. add sensecam, ui for pictures (directX/drucker ui), links and metadata for pics
4. (time permitting) look into video, treating it as stills with user defined 'x' frames between stills, using the same ui as the sensecam, ignoring audio.

during all this, develop matching content.
This looks more like a digital life auto-blogging tool which could easily lead to video blogging once these steps have been met. (audio is purposefully avoided here.)

(Get enough info on manipulating mpgs and its simply pulling out frames and then displaying dhtml divs that open the file up to a specific point in it. some processing on each end to be sure, but doesnt require an insane activeX plugin or anything.)

updated google search

i did a google search for "video blogging" on march 22.

3,560 entries then.

searching today, the same term has about 15,700 now.

i think i picked the right topic.
now if i can just get it done before the wave breaks.

there will be a flood of thoughts, notes and information posted about my thesis project in this next week. as always, comments will be more than welcome.

thesis_r

slim pickings for now, yes. but most of my links for video blooging are on my desktop at home.

buts here a google search for the term "video blogging" (3,560 on march 22)