I’ve recently been asked to give a lecture on Digital Technology, Design and the Body and I found the research for this lecture to be so enjoyable that I thought I’d share it with you as well. I went back into history a little and delved into technologies that seem so old-fashioned now, but I remember being blown away by them when I was growing up. I’ve also looked at very current consumer and movie technologies but the one linking factor is the way that our human bodies are linked to this technology. As a human interface, our legs, arms, hands and heads make great communication tools. Why should we be stuck with a mouse, trackpad and keyboard?
Max Headroom
It all started in my memory in the late 80s with a talking head character on a music show called Max Headroom. Actor Matt Frewer played the character of Max, and I thought at the time they had somehow digitized his talking head, but I’ve since realized it’s a lot of latex make-up and jerky video editing. Max would cynically rant about the music he was introducing and impart snippets of comedic wisdom. I remember being mesmerized by the endless swirling digital background and his jerky movements and digital feedback. Now that I watch his videos again over twenty years later, I realize he gave Jim Carrey most of his best material.
Matt Frewer
His pinnacle of fame was probably appearing in the Art of noise film clip for Paranomia.
He also made a few Coke commercials, a movie and a TV series and then disappeared, but it marked the beginning of people wanting to use digital technology to alter their appearance and subsequently the things they were able to say. In much the same way as the Simpson’s cartoon can get away with things that a “real” family sitcom couldn’t, Max Headroom was able to say things that Matt Frewer couldn’t.
Virtual Reality
The term Virtual Reality applies to any technology where an environment is simulated, and this can apply to the simulation of real places and events, such as for pilot testing, or the simulation of fantasy places, such as gaming environments. When I think of VR, I flash back to the 90s where people would wear sensor-ridden gloves and futuristic goggles. Unfortunately, nothing much seems to have changed. The graphics and experiences have got considerably better, but the wired gloves and head-mounted displays still remain the best way to deliver these virtual worlds and experiences.
Ames developed (Pop Optics) goggles now at Dulles Annex of Natl. Air and Space Museum/Smithsonian Inst.
Virtual Reality 1: simulating actual reality
One use of Virtual Reality environments is in training scenarios. Pilots, fireman, space walks, nuclear plant operators. These types of simulations mean that people can experience different scenarios without waiting for a real emergency to occur, and can repeat the process multiple times without endangering life or property. It also usually means a much lower training cost. Building the simulations is usually expensive to start with, but over time, the cost reduces, whereas actual training costs money in fuel, equipment and personnel every time.
Parachute simulation
Walk-through / Fly-through
Another use of realistic VR is the “walk-through” where an environment doesn’t exist yet (such as a building development), or no longer exists (such as an archeological site), computer modeling and simulation can be used to map something in 3D space, and we the user can pilot our way through the environment.
The first “virtual tour” was of Dudley Castle in 1993. The castle itself was built in 1550 and not much remains of the building, but digital artists rebuilt the interiors.
http://www.exrenda.net/dudley/sharrington01.htm
These types of computer modeling applications are commonplace today, and highly sophisticated examples form part of just about every 3D modelers portfolio.
http://www.archiform3d.com/3d-showreel/index.php
Archiform 3D portfolio showreel
Spa fly through
In these instances, the “fly-through” as it’s known, is controlled by an imagined “camera operator”. We are replacing our “bodies” with an unseen “camera”. The camera doesn’t actually exist, but the “real world” metaphor for our bodies is used and understood by a movie-watching audience.
The Google Art Project
Google is currently piloting a program where they “map” the insides of the world’s art galleries and so our experience of the London National Art Gallery or the Metropolitan Museum of Art is more intimate and personal than if we were there in person and with the addition of meta-data, we are able to enrich and personalize the experience.
http://www.googleartproject.com/
In this case, when you “explore the museum” it is you the viewer, controlling how and where you go. It uses the same technology as Google “street view” but with much more detail.
Virtual Reality 2: simulating an imagined reality
Multi-player games such as World of Warcraft and Second Life allow users to interact with each other over the Internet and they inhabit imagined worlds. Lots of people now use the World of Warcraft engine to make comedy videos, music videos and so on, because you have a ready made world with actors, costumes and sets.
World of Warcraft BigBlueDress
Second Life
This is one of the most customizable of all the virtual worlds. People can actually build and create in this world and the use of an avatar means you genuinely interact with people, even if you are hiding behind a constructed appearance.
This video is quite old but still explains the principles of Second Life quite clearly.
Holograms
The other use for virtual reality is in a game space, or for entertainment purposes. Star Trek viewers will be familiar with the Holodeck.
Touchable holography
;
People have been fascinated by holograms and the Star Wars talking hologram of Princess Leia was such a fantastic idea at the time but is a reality today.
Dancing window girl
Kate Moss Hologram
Kate Moss was shown in this quite magical hologram at an Alexander McQueen fashion show
Movies and Technology
Lord of the Rings: Gollum
The Lord of the Rings Trilogy pioneered the use of motion capture technology to capture the actor’s performance and map it onto the digital CGI creature of Gollum, who then interacted with live action characters.
In this video from the extended DVD, the actors and crew explain how they brought the character of Gollum to life as an actual acting character, rather than light relief as Jar Jar Binks was viewed in Star Wars.
Avatar
The most obvious example of a movie that integrates digital technology with the human body is James Cameron’s Avatar. The entire plot of the movie is based around the idea of a human going into the body of an alien creature and the interface between human reality and the alien reality is generated entirely by computer.
The word avatar is often used in computer games or for online personas as a graphical image that represents a person.
This is the scene where the disabled human character Jake first inhabits his avatar’s body. The human character lost the use of his legs and so is very excited to have the technology to move around again. It was a brave move to combine live action and 3D animation in such a way but the result was very convincing.
http://www.youtube.com/watch?v=1QEFrI-D_3c
The way this action was captured in an animation setting uses motion capture technology, where the actor is filmed carrying out the moves and they are then mapped onto the digital CGI character to perform on-screen. In this video, director James Cameron discusses the way the emotions were captured and also shows the actual technology as it was used. As you can see, it is a more sophisticated step forward from the ping pong balls of light used in Lord of the Rings.
The Matrix
Again, this movie pioneered some visual technologies that announced a stylistic leap forward in special effects and visual sophistication.
This is the scene from the movie, showing actor Keanu Reeves bending backwards in slow motion as the CGI bullet whooshes over him and the camera cycles around him in 360 degrees.
This video shows the blue screen set-up and the multi-camera rig that captured the motion.
Minority Report
There are several fantastic technology and body concepts in the Steven Spielberg film Minority Report, but this User Interface sequence really showcases how the technology interfaces with the human body as user.
This film pioneered several technology and marketing techniques that are now in use today, or have telegraphed future technologies.
Consumer Applications
As movies and games pioneer motion control and capture technologies, they become available for our use at home. The console market has boomed in the last ten years, mainly due to the improvement of graphics and the introduction of new technologies that allow us to have more interaction with our avatars in the game space.
WII
The Nintendo WII controller was the first of these hand-held consoles that went beyond the joystick, steering wheel or game console and this additional interactivity has proved very popular. The WII controller uses a similar accelerometer technology as an iPhone. This gravity and speed response allows the user to have “real” control over the game on the screen, such as hitting a tennis ball or hitting a home run.
Kinect
Microsoft Kinect took this interactivity one stage futher and removed the need for a controller, making the human body the controller. The device uses a scanner which responds to the motion in front of it.
The Sixth Sense
In closing, I’d like to share with you a TED video from 2009 that inspired me at the time and a lot of people since. The idea of using the human body as a portable internet device and camera, using the finger tip interface idea from Minority Report and gesture technology from Kinect, Pranav Mistry telegraphed some of these ideas with his research in the MIT Media Lab several years ago.
