Mirror’s Edge; “proprioception” hack
Mind Hacks has an interesting piece about the game “Mirror’s Edge” and a story in Wired that talks about the realism of the game’s simulation of free-running and the physical effects on the player;
Only 15 minutes into the game, my mouth began overproducing saliva, and I had to pause the action for a few seconds to avoid carsickness. I would feel like a total lamer, but apparently even the Penny Arcade guys wrestled with nausea.
The Wired writer thinks that the game fools the body’s sense of where it’s limbs are. Also known as “proprioception”. Mind Hacks adds;
Research into illusions of proprioception — your sense of where you body is in space — has shown that our body map is surprisingly flexible. It is possible to mislocate your hand, for example, coming to believe that it is directly in front of you when in fact it’s out at the side, or behind you;
Jaron Lanier has reported on an early virtual reality experience he had that made him feel like he had the body of a lobster, with 6 extra limbs. The important feature of all these illusions is that they rely on precisely timed visual feedback. Although visual input can reprogramme our body image, it only does so when there is a tight coupling between what we see and feel. The importance is not the level of detail in what we see, but in the fluidity of the interaction. If Mirror’s Edge makes you feel like you are really are doing Parkour then it is because it has the correct kind of visual feedback (your limbs, in a fully interactive world) with the correct timing.
A final thought: if a computer game really is immersive for something as visceral as free-running, isn’t that kind of surprising, given how complex free running is physically, and how simple the commands used to control a computer game are? Perhaps what this is because when we automatise an action such as a run, a jump or a roll part of the process of making it automatic is losing the experience of the component parts. So, when a computer game feels like real, it is because real feels like nothing — we just ask our brains ‘jump’ and the motor system sorts out the details without our any deep experience of how the jump is performed.
I often find that myself. As I’m leaping from rooftop to rooftop at night; that it feels like nothing. Same with when I’m gunning shoppers down in the supermarket… Ha ha.
It’s an interesting thought; that a video game is simulating the experience of competence- the feeling that your body has learned something. Anyone who’s ever trained at something athletic or demanding of huge coordination- something physical- knows the pain and the perseverance required to get to the point where it feels effortless.
But in so many ways the world of the video game is the world of sensation. Much more so than movies ever could be. This is the characteristic of simulation- it’s an active representation. The processes of feeling and experiencing are represented. The research going into first person shooters, for example, is amazing. Right down to levels of excitement experienced whilst using individual weapon types, in different games.
Alva Noe has an interesting talk up on Edge about our relationship with pictures;
One experience that I’ve been especially interested in is our understanding and experience of pictures. If I show you a picture from a newspaper—for example, a photo of Hillary Clinton—there is a sense in which, when you look at that picture, you see Hillary. There she is, in the picture. Of course, Hillary is not there, so there is an obvious sense in which you don’t see Hillary when you look at the picture. There is a sense in which you see her; and a sense in which you don’t. She shows up for you, in the picture, even though she is not there. She shows up as not there. Getting clear about this phenomenon is the central empirical and conceptual problem about depiction.
One idea might be to say, well, seeing a picture of Hillary is just like seeing Hillary. Seeing a picture of Hillary produces in you, the perceiver, just the same effects that actually seeing Hillary would produce. The problem with that suggestion is that if that’s right then we lose our sense of the difference between seeing Hillary and seeing a picture of Hillary. The distinctive thing about seeing Hillary in a picture is that she is there but not there. She is there but visually absent. She is manifestly absent in her visual presence. It’s a kind of a paradoxical thing. There is something paradoxical about pictures.
My view is that traditional philosophy and cognitive science has been asking the wrong question when it comes to pictures. They ask, how does the picture affect us and give rise to an experience in our heads? Instead, what they should ask is how do we achieve a kind of access to Hillary, to properties of Hillary, such as her visual appearance, by exploration of something which is not Hillary, namely, a picture?
The critical thing is the relation between this model, this picture, and that which is absent, such that we can gain access to what is absent in the picture. Once again we are thrown back to this idea that the perceiving is an achievement of access by making use of skills, knowledge. I need to know what Hillary looks like in order to recognize Hillary in her picture.
A striking feature of pictures is their immediacy. A picture of Hillary doesn’t seem to be a symbolic representation of Hillary. There seems to be the sense in which merely knowing how to recognize Hillary or how to recognize a human form, a figure, is enough to recognize a picture of Hillary. There is this idea that we don’t need any further knowledge or further skills in order to perceive something in the picture.
That is a very interesting idea. But, in fact, there is a nice comparison we can make to help us see that pictures don’t really have this sort of immediacy. Think about something like the Macintosh operating system. No promotional endorsement intended, but the Mac OS is user friendly. If you understand a few basic metaphors, about the desktop, clicking, open files, closing files, a few basic metaphors allow you to unpack just about any program that you might be working with.
So there is a sense in which the functionality of the graphical user interface is straightforward and immediate. But, of course, that is precisely because the engineers have built the program with our particular predilections and capacities in mind. They built it to be easy for us. It’s not as though it just happens to be easy. Technological evolution made it transparent for us. And pictures are just the same. You encounter pictures in a newspaper, say, and we find it easy to see Hillary Clinton in the picture. We don’t need any further training. But that is not because you don’t require training to see Hillary Clinton in a picture. It’s because that technology was devised to be easy for us. The technology was designed for people with the training we already had.
OK, what does that mean? Pictorial technologies, both painting and photography, have been designed to be straightforward for people that already know how to recognize things by using their eyes. Certain background visual skills are all that is presupposed. But then seeing itself requires tremendous background knowledge.
How do we achieve access to properties of Hillary? Such that we can gain access to that which is absent in the picture? Not that you’d want to. It’s interesting to think about in relation to, say, war in video games. Probably none or very few of the people who make those games have experience of real war. Neither do most of the people that play them. But the war in the game is really just an aesthetic, derived from other representations of war in movies or pictures. The images of war in a video game exist only to give us access to the sensation of shooting. The relationship to war is very tenuous. Even more so when you consider that the movies and pictures that the images in the video game is based on, have their own rules of abstraction. The background knowledge required to successfully access what you are intended to access, is not about the thing that’s represented.
