I’m familiar with the researchers in Microsoft China. Before Steven Drucker & Curtis Wong come, I had already made a picture of how amazing are they researching now. But when I saw their works, I was still astonished.

To me, Microsoft is not like the other entertainment company which is seeking awesome graphics and Hi-fi hardware, people in Microsoft are more aware of the users. From these projects, I get a good feeling of the future.

Media Variations is a simple concept. We can’t say how wonderful it is. But the purpose is to design a more friendly interface, which is a very essential part today and in the future. But in my opinion, this could only be a partial application in certain area. It’s not a general resolution. And its prototype is similar to the hypertext like wikipedia.com

Media Frame Browser is one of my favorite among these projects. Having gotten used to the windows interface, I lose my desire to better service. I’ve confronted with the troublesome management of my own picture database, but I didn’t ever think about improving it. But Curtis and his guys made a lot on this particular area. Although there is still problem nowadays, but their work is intriguing. Picture filters, human head detectors, 3D manipulating, temporary bin and audio annotation, all these points are great works.

Smart Skip and Deep News are also small idea and practical attempts. What I want to mention here is the Spectator and the cinematic camera for Mech Warrior 4. Microsoft considered both game player and the game audiences. When a game player is intrigued by the games, Most of the audiences are having troubles to get the same feeling. Instead of looking for a better angle for both the player and the audiences, why not give them different angels? As we can see, the result did be dramatic. Although it’s not human cinematic, it generated great entertainment for the audiences.

Finally I gonna say, when we are ambitious about some revolutionary designs, can we make it better from such subtle and detailed parts? Have we put human and users into our consideration?

This was a very interesting presentation yet also a very double edged one at the same time. I found myself both heartily agreeing and disagreeing with many of the topics brought up by Steven Drucker. The first presentation of the photo catalogue system was very informative. I felt partially blinded by the increased number of options however, and felt the software would work well with those who are maticulous with their photos but fall short with those who don't want to spend so much time organizing their collections. I personally felt that adding information in about the photos was cumbersome and confusing, to much work for my tastes. I also didn't like the distinction between video and photos, as their is is too much going on in a video clip to organize it like a photo.

The other projects that stood out to me that I really wanted to voice an opinion about was both the Smart Skip feature and the arena audience prototype for involving TV watchers/gamers and the internet. I felt Smart Skip had merits, but was too much WORK! I am lazy, and I don't want to sort through more frames just to get through a commercial break or find the weather. Call me super lazy, but the idea of sitting in front of TV for me is just that, sitting there and vegging out. I would rather not have to worry about finding the info I want to see. I just want to watch, that's it, regardless of mindless commercials or annoying ads. The video mock up and proto-type really had me going, as I was totally shocked by the interuption factor in my TV going expierence. I remember watching X-files back in its heyday, and I wanted NOTHING to do with the outside world during my hour of bliss. Much like watching a movie, I wanted as few interuptions as possible, even if I could hit the pause button. This just seemed to ruin the idea of cinema/television by being transported into the gameworld just to be a passive onlooker. I REALLY wanted to scream, NO INTERUPTIONS PLEASE! eeek.

The fundamentals behind the design concepts are very sound though, and it is very nice to see this coming from Microsoft.

good to know that microsoft is funding other things other than bulky, insecure operating systems. in response to mike's comments, I'd have to say that I really found the work the group was doing highly relevant and interesting in the context of research. I didn't necessarily think that these exact projects would be wonderful products, but they exposed a lot of interesting research topics. Digital media is going to make our data overwhelming in most cases, even for a more average user. Between digital photos, music, and video alone, we need to start thinking about interfaces that allow us to make sense of this vast store of digital data. This could mean organization (the photo system), productivity/efficiency (the smartskip project) or entertainment (the gaming thing). Yeah, clearly there were some issues with all of these - privacy would be a big factor in the game/tv scenario, etc., but I think that it's important to study these things without having to necessarily make specific products.

Steven & Curtis are extremely interesting and have some great research in the works for Microsoft. I’m glad they could give us an hour of their time and present some of their projects. Making the UI a pleasant experience, unto itself, is an exciting objective. The notion that the UI becomes a character, or personifies certain emotions and/or characteristics, is something I’m very interested in pursuing myself (see my Joy Mountford Blog posting on 2/8/04).

The Multimedia Browsing projects dealt with the user’s interface in imaginative ways. The movie “one sheet” and/or box cover lends itself to instant identification of content. Organizing those in “clusters” that circled like a Rolodex was a familiar and accessible way to access the content. It was also frightfully similar to an early proposal I made to Jerry Bruckheimer for his Business to Business website.

I also liked the personal photo organizations and the robust potential of a database driven file structure. Hearing more about the next generation of Windows would have helped clarify some of my questions regarding the infrastructure. Organizing a visual representation of a complex database is an exciting challenge. The graphic organization of personal photos seemed to get a little cumbersome and confusing (to the average computer user). I would have like to seen more iconography to represent methods of organization (i.e. name, word, date, size, etc.)

The objective and/or approach for “Smart Skip” didn’t seem as clear. I’m not sure why someone would be so actively engaged in skipping around a passive medium, like TV. I can understand the Tivo model analogy on a show-to-show scale. But I wasn’t drawn to the idea of skipping through small segments of a soap opera or game show. I also prefer a “headline” approach to news rather than fast forward and reverse. In addition, users may be offended by the false promise of skipping past commercials.

Sharing the experience is a key to the socialization of the media culture. “Spectator” is a huge idea and something I discussed at great lengths while developing the GameWorks concept for Spielberg. It would be exciting to have control over the camera position and be the cinematographer! Another interesting approach would be to give the user the “Producer’s Chair” and allow them to switch between a half a dozen cameras. I would enjoy contributing to these and other “active viewer” possibilities.

I enjoyed the presentation of some of the Next media's research projects. I appreciated Curtis's insight to rapid prototyping and churning out a project rather quickly, and I think the fact that they work on these projects at Microsoft even though they'll probably never see "the light of day" is very interesting.

Since I have strong interests in the enhanced/interactive television realm, the smart skip and spectator projects, as well as Curtis's work highlighted some interesting ideas/conundrums about just what to do with that passive thing called a television.

Curtis's work in broadband iTV presented a new side to my interests. I am aware of the problem with the set-top box, and my personal focuses on ETV/ITV have never ventured into the realm of television for broadband. Why this may not work/never occurred to me may have something to do with the fact that television is still a passive activity, and I think it also ties into creating interesting interactive narrative. Interactivity does not really lend itself to traditional form of narrative, including sitcoms or dramas, so its important to look at what parts of television do lend themselves to interactivity. Beyond news and gameshows, since those are the only ones that work, but are constanstly being done and re-examined.

The Gameshow Network, http://www.gameshownetwork.com/, utilizes both a computer and a television for their interactive gameshows. They provide a VUP the chance to watch the show and play along on their computer. Unfortunately, some of the shows--like trivia--work better then others--like Lingo, a spelling based game.

Eventually someone will have to come up with the perfect formula, right now there are problems with distributing interactive content on a television because of the set-top box issue, broadband telvision seems less fulfilling and the combo of the two can work, but can be annoying if your computer and television are in different rooms.

I like the work they're doing at Microsoft, because I think they are looking at ways to resolve this issue, and that is definitely intriguing to me, I'd sure like to help them! For now, I'd like to do some research into the different forms of televisions shows and how each can become interactive. Because we all know clicking on Jennifer Aniston's shirt (http://www.wired.com/news/business/0,1367,59765,00.html) in order to buy it really wouldn't appease too many people.

So, thanks to Steven and Curtis for their insightful presentation, and I hope they can return soon and have a little more time for presentations and discussion.

the area of presentation they covered is more and more what i spend my days thinking about. i was familar with a lot of their projects, having looked into the lab for thesis stuff. filtering and organizing has become a huge part of the work i am doing - my midterm for perry, leading into my thesis (hopefully) with 'vlogging' (you heard it here first. though im not sure its anything brag about). it was interesting to hear them talk through the projects they presented. the structure of the team was also interesting - what was it - 4 programmers for one researcher? scary, only in that we seem to be very lucky if the ratio works the other way around for us (right now at least). time to be picking up some c# books and going to town myself i suppose.

really glad to see these demos - these are the types of tools that we have to build to be able to apply interesting content to them. iterative cycles of development.

they didnt talk too much about content, thats where we come in...

Metadata is the word of the day (if not the decade, century or even millennium). The most interesting innovation that Curtis and Steve showed, was a system that allowed a user to manage and control the metadata attached to her media. No one likes to act as a file clerk for her own documents, but with most modern OS GUI file managers, a moment of relaxed vigilance can result in a cascade of organizational anarchy. With the price of memory cheapening, the amount of storage space for data is growing at a faster rate than most users can fill it. This leads inevitably to a pack-rat behavior, why throw it away when I have plenty of space left? Of course the nightmare happens when you need to find that one item you saved three months ago and it has been lost in the dense foliage of a directory tree.

While the underpinnings of a metadata focused file structure are important infrastructurally, the user-interface for access to the power that such a structure grants is far more crucial to the adoption of metadata for the masses. I found that the ability to apply annotations to multiple files at once was an invaluable time-saving technique when working with thousands of digital images. By providing the user with an interface that reveals multiple perspectives on organization and ordering, it seems far more likely that the user will engage with their own content, giving a sense of control that is mostly absent when linear time is the only view of the data. I believe that there are three classes of metadata that will emerge as useful:

1. Capture state metadata- everything about the time, space, device and author at the moment of origin. Looks like statistics.

2. Annotative metadata- everything added by authors and users after. Looks like media (capable of containing its own metadata) or links or groupings.

3. Algorithmic analysis metadata- everything that drops out of running an analysis on the media. Looks like media or links or groupings.

The second and third classes of metadata may not resemble each other much right now, but soon they may be indistinguishable except for the source. But I assume that we will care deeply about the origin of annotations.

I found this presentation to be rather entertaining, and was inspired by several of the products they showed, my favorite being the Media Frame Browser. The interface was very clean, and of all the innovations we've been exposed to lately, I think this is the product I would most like to use--I hope they integrate it into future versions of Windows. The Browser is an intriguing approach to organizing files. The traditional way of doing this is with a simple directory tree structure, but the database approach that they took here actually makes more sense. It sort of takes personal computing to the next level, in that the computer is programmed to better help you find the data you're looking for.

This presentation was both very informative and visually interesting. I was not aware of the types of projects that the Next Media Research Group and Microsoft Research were involved in before the talk, and I feel it was a very nice introduction.

The focus on the end user experience was very evident in all of the pieces that were shown, and further solidified the idea in my head that the end user experience is something that is a necessity in any product/idea development.

Of the projects that were shown, I was the most impressed by the photo media browser. While I agree with Mike that the options available were a little blinding, the idea that you can organize your digital photo collection in such a specific way was very appealing. Creating such a large database of memories and being able to organize it through such specific categories/clusters enables the user to experience looking at a photo album in such a different way. Instead of being forced to click through the images of a boring family vacation in iPhoto or on a CD-ROM, the user can browse the database and decide what areas they want to focus on. Through the inclusion of various forms of metadata, the experience is further enhanced and various levels of immersion are possible. All in all, the user has a richer media experience and all of the additional options could lead to a more compelling photo album.

While the other projects that were presented were interesting, they did not capture my attention the way the photo media browser did. Again, I have to agree with Mike that the Smart Skip seemed like too much work, and Spectator seemed too intrusive. The privacy issue that Will brought up was also a concern for me. When I sit down to watch TV, I want to watch TV. I don’t want to be interrupted by instant messages or watch someone else play a video game. Additionally, I would not want the programs I watch to be accessible to a large community of users (I do not remember if it was a small select community or anyone who is online at the time). While the underlying concept is interesting, I do not see the need for such a complicated and disruptive interface. Instead of sitting on your couch and watching from afar, why not just go to your friend’s house and watch/play with them? Why should I be an omniscient spectator through an isolated technological interface, when I can be an active participant with human interaction?

All in all, it was an interesting talk and introduction to the ideas and concepts the Next Media Research Group and Microsoft Research are working on.

One of the biggest things I took from the lecture was the concept of user endurance. A lot of the applications shown didn't do anything that couldn't be done with existing programs, but the main difference was key: Not only were the programs designed specifically for the task (such as photo organization), but also provided an interface that allowed the user to accomplish their task easily.

"Easily" does not always mean "quickest", despite some prejudices of classical technocracy. The Smart Skip gave the most obvious example: While one could perhaps use a shuttle more quickly, such activity can exhaust the mind...processing 90 frames per second, while building a mental model of what you're looking for gets taxing after a while. The photo organization works the same way. This dispelled some of my reservations, for at first glance I thought the interfaces were simply pretty faces, akin to overused flash web sites that actually get in the way of functionality. The idea behind these is different: Imagine a mental sprint versus a marathon.

This idea of "ease of use", or, perhaps "fun to use", although I find that phrase a little too specific, should be applied to a wider range of applications and tools. Indeed, a lot of my programming acts along similar lines. One game I've been working on uses color coded bitmaps for various pieces of data, instead of solid values. A fair amount of effort is done by the computer to translate color values into usable number values, but with the way my mind works, I'd rather spend scores of hours playing with bitmaps in photoshop that slightly fewer hours looking at row upon row of comma delimited text.

It's all about the mental endurance of the user. I get exhausted after hours of programming, and my brain decides it'd rather sit in front of one of teh built in screen-savers so it can decompress. Interface, if used wisely, can lengthen the span of time before this happens.

I, too, was most interested in the Media Frame Browser, or whatever it's called. It lends itself to so many possibilities, not only for organizing media on a surface level, but being able to group by similarities with thresholds, etc. But it seemed to me to have a bit of that "Microsoft touch" where too many of the controls were based on what Microsoft thinks you might want, decreasing the amount of specific control that the user might have. At least it doesn't have a talking paperclip.

Also, like Julie, I found it fascinating that many of these projects will never see full development and distribution, but are for research only. It is encouraging to see Microsoft working on developing new ways of thinking with computers in order to create better products later.

Kurt, I agree that analytics will be an importance force-multiplier for assisting tagging. Automated clustering, emergent taxonomies, and network scaling w/ collaborative filtering, webs of trust will all factor in. As will good interfaces (personally, I find navigating long, nested contextual menus a PITA).

We're seeing a lot of areas converging right now. It's all quite exciting.

some other intersting MSR stuff:
http://research.microsoft.com/~jplatt/
http://research.microsoft.com/research/downloads/

A few disparate (but related) links that may be of interest:
http://users.bestweb.net/~sowa/peirce/ontometa.htm
http://facetmap.com/
http://www.iuiconf.org/pastiui.html
http://www.idlewords.com/pages/emerging_tech_talk.htm
http://collaboratory.planetwork.net/linktank_whitepaper
http://www.dynamicobjects.com/d2r/archives/002564.html
http://www.neci.nec.com/~lawrence/papers.html
http://www.nuclearelephant.com/projects/dspam/dobly.html
http://web.umr.edu/~tauritzd/art/
http://bonsai.ims.u-tokyo.ac.jp/~mdehoon/software/cluster/software.htm

I was happy to see that both Curtis and Steven were very enthusiastic about their work with Microsoft Research and Next Media Research Group. I rather enjoyed their talk, but I felt that there really wasn't enough time to give any of these projects their due. I think the one project which seemed to resonate not only with me, but with Tracy as well, was "Commanding Heights" Perhaps the fact that it was a sucessful interactive documentary, based both in web and video on WGBH Boston, was what really got to me, seeing as how I am doing an interactive doc. for my midterm with Perry.

I also thought that the "Visual Decision Maker" was a great thing. Giving users the option to 'thumbs up' or 'thumbs down' media was seemingly inane, but it would be something to sit in front of for hours, just to see what may pop up next...it's kinda like that new DVD Board Game "SCENE IT?" Inane, yet fun and also informative for futher social interaction. I could also see it as a small add-on to iTunes, one where you would have to sit and rate all the songs in your library with a thumbs up or down. Could be cool....!

Curtis and Steve gave a damn good demo for the time they had. It’s great to see Microsoft doing some innovation in terms of gui and metadata. I’ve been mystified as to why win95—win2000 was so static and boring. XP makes some interesting inroads for gui’s and usability…but not that much. When you have so much power and money to do innovation, why remain so static? Perhaps they’re worried about making radical changes because it will confuse grandmothers?

The thing I found most interesting were the metadata issues of organization, browsing and annotation. Data is not just exploding in volume, but also in disparity and disposability. I don’t know details on the Longhorn fs, but I’m very excited to see where all this research leads and if any of it will see the light of day soon.

I found the media browser gui to be very visually pleasing and ostensibly highly usable. Highly relative and customizable ways to view and organize data is exciting. I’d like to see the same degree of relative custom ability within the gui itself. Skins are a start.

I’m wondering how much of this motion graphic gui innovation is being fueled internally and how much is being inspired by the success of OSX?

It was great to visit IMD and listen to each of the members of the class talk about their projects. I also really enjoyed reading the comments above discussing the pros and cons for each of the projects that we have done. Some of the common threads are worth elaborating on to give you some insight into what we were thinking for each project.

Mike's comments about TV are right on in that we all just want to watch and not do a lot of work.

Smartskip was initially developed from living with PVR's both Replays and TiVo's over the past 5 years. Replay allows you to skip 30 seconds (mostly disabled) and both tivo and Replay allow variable speed fast forwarding. It is really tedious to get to another place in a movie. Even at 20x, 3 minutes to get to the end of a movie is too long for me. What you saw with smart skip in zooming up and down in time resolution still strikes me as too much work. I'd like to get to the end of a movie in just a handful of clicks. I do like the idea of thumbnails giving me clues as to how many clicks over I would need to get to the part I want. Additional user testing would probably reveal some sweet spot of time which would be relative to the program ie 10% to 20% of total time for each thumbnail so the entire program could be represented on one screen with a series of 5 to 10 thumbnails. Anyway I think you've made some good observations about that project and we had the same opinions about it that you did.

Many of you had some very good comments about the media browsing and annotation prototype. The comment about annotation (tagging) being a lot of work is absolutely valid and for many people they would never want to do it. I would be in that group but I now have 6,000+ photos and I have to do something in order to find anything. ( I used year/month/descrip folders and that's pretty crude)

We had a number of goals for this project and it was based on the fact that existing systems to tag are all pretty poor. My original goal (2 years ago)for this project was in thinking about the time when wireless display devices would be affordable and I was guessing that would be around 2006. In that timeframe I expected people to have 5,000 to 20,000 digital photos. I imagined that with those volumes of photos on your PC, you would print a few and probably share many of those same ones but the vast majority would be "hidden" on your pc never to be seen.

I felt that ideal photo experience (not slide shows or screen savers) would be primarily be via these frames constantly cycling through all of the thousands of photos that you have on your pc and if something caught your eye, you could pause and pivot to see related photos, perhaps record an audio comment which is then stored with the photo.

To make that scenario happen, we would have to come up with a way to rapidly and simply tag groups of photos as easily as a single photo. The media browsing prototype you saw has many simple tools, some analytic and some visual UI, which can be combined to simplify the tagging process to be as quick as just a few seconds but still provide greater value in terms of making that material accessible later (via the frame or other purposes). We didn't set out to create an annotation prototype for its own purposes but really as a means to an end.

I'd answer some other questions mentioned: we have a team of 5 of which 2 are developers, 1 reasearcher, 1 designer and myself (idea guy & producer).

We started doing motion gui stuff around 2000 because we felt the video cards were largely not being used outside of games and that the UI could benefit from fluid animation, transparency and 2.5d. OSX came out later but they clearly felt the same way.

Both Stephen and I look forward to coming back, seeing your projects and hearing your thinking process about what you felt worked and didn't work.

I really enjoyed the interaction with the class. The last time I did a talk at USC was in 1992 when I was at Voyager. I met a student in that class who I thought was pretty talented and I brought him out to Voyager and we offered him a job. Now Eric Loyer is on your faculty at IMD. It's a small world.

On mike's comment about smartskip - TiVo handles the same way, but without the visual indexes. The user skips through a timeline that looks very similar to SmartSkip's. At first, this interface was weird, but you get used to it, and probably the growing popularity of TiVo would allow for an easier transition into SmartSkip.

My favortie concept behind the photo annotation program was the idea of using it so that others can more easily look through your photos. this is something that "scrapbooking" is only now starting to utilize in the analog version of photo albums, so it will be very interesting to see how we can use digital technology to make our photos more accessible to others. Because isn't that usually the point?

I was most frustrated that traffic held me up and I was unable to talk to Curtis about Multimedia Beethoven. If you're reading this, Curtis, that CD-ROM was highly influential in my interest in interactive media, and I look forward to seeing your other projects in this same vein.