Lowering the threshold of immersion in embodied interactive art


Virtual Reality in the form of cumbersome head mounted displays, wired data gloves sprouting cables and exorbitantly expensive multi-projector cave environments was the obsession of the digital culture communities in the 90s. Theorists, writers, journalist, electronic artists and general IT pundits fed off the hype being generated both by hardware manufacturers and their own imaginations. Cyber sex, virtual worlds, walk-through architecture and telepresence were concepts that spiced our hazy vision of the future with a tingling sensation, hinting at the possibilities just around the corner. The reality, as we discovered a decade or so later, was much more banal and practical than we might have hoped. Virtual reality in its traditional form is still with us in much the same incarnation as before – universities and hospitals among others employ VR for volumetric rendering of 3D objects, making it available to the likes of chemists and brain-surgeons for the visualisations of complex molecular structures and the insides of people's heads respectively. Mining companies utilise heavy machinery 3D simulations to train their staff in the use of mechanised shovels. However for the most part we have moved on, relinquishing traditional VR to the graveyard of half-baked techno-deterministic ideas that never caught on. Mention VR in a conversation and you might even provoke an arch look and a raised eyebrow, because these days it feels so....so....version 1.0. Virtual reality did leave an important legacy of course, one that has since been subsumed by an ever increasing torrent of new technologies, but one that nevertheless haunts on-line forums, MMORPGs and even webcam internet telephony. The historical precursors to VR rigs are even more significant and date back centuries. Oliver Grau has written on the subject of pre-VR immersive environments which hark back to the 18th century1 or even earlier. While it might be stretching the imagination a little to suggest that cave paintings were a precursor to modern VR, the fact remains that people have been wanting to immerse themselves in alternate worlds for as long as we can remember and there is certainly no indication that this inclination will ever cease to exist.

Flawed Reality

One of the most fundamental flaws of Virtual Reality was its equation of immersion with life-like environments. The human eye is so attuned to the features of the human body and the subtle details of realistic movement that it is very difficult to recreate them without drawing attention to their artificiality. This partly relates to the often cited concept of the "uncanny valley" first posited by roboticist Dr. Masahiro Mori. The Uncanny Valley is a phenomenon whereby increasingly realistic representations of humans eventually evoke repulsion from an audience due to being too realistic yet obviously not human. This revulsion stems from our tendency to project ourselves onto human-like forms or at least anthropomorphise them. As human representation increases in realism we reach a point where we no longer need to use our imagination in order to complete the picture and consequently become much more critical towards the subject as we compare it to a real human being and not some abstract form. Scott McCloud's Understanding Comics touches on this issue in a number of tangential ways. One example he describes is the technique of "masking"2 in comics which employs cartoon-like protagonists drawn over quite highly realistically drawn backgrounds. The idea is that less realistic characters are often more easier to empathise with and identify with, enabling readers to superimpose themselves into the comic through the character and enter the comic book world. Highly realistic characters tend to evoke more critical reactions from readers and therefore have a more narrow appeal.

Perhaps one day when processing power dwarfs our current machines and real-time ray-tracing is child's play we might then be able to return to the pursuit of "realistic" interactive virtual environments. By that stage we will probably already have anthropomorphic androids, quantum personal computing and be in the midst of a massive ecological disaster so devastating that it spurs a boom in escapist virtual environments to offset the guilt of our planetary destruction. The other possibility is that we will eventually encounter an "uncanny valley" syndrome where the simulated realism feels so real that it becomes disconcerting and alienating instead. In the meantime, I can't help but feel that all this technology has fogged our ability to appreciate our own susceptibility to immersion, which requires far less computing power and just a few understated perceptual and visual cues. Interactive installation art – art which requires the viewer to close the semantic/aesthetic loop, has been slowly chipping away at the phenomenon of immersion making it more ubiquitous, more accessible and more liminal than ever before.

The Threshold of Immersion

What I am proposing is a notion of immersion that focusses less on the all-encompassing, multi-sensory variety and introduces the possibility of a more subliminal alternative. I also believe there are ways in which interactive art is able to tap into this more accessible and inconspicuous immersive experience. Oliver Grau succinctly encapsulates the problematic of VR's mission to immerse us in dazzling, ultra-real environments:

'It is not possible for any form of art to reproduce reality completely and we must remain aware that there is no objective appropriation of reality Plato's metaphor of the cave demonstrates this. Only interpretations are decisive.'3

In fact subjective, concise interpretations of reality could well describe a significant proportion of contemporary art production. However the question that interests me is not so much whether simulated environments will ever attain such an immersive quality as to confuse reality from illusion, the question is how little perceptual stimulation is required in order to immerse an individual – the threshold of immersion so to speak.

Immersion, while primarily an experience overwhelming the mental faculties, also has a physical, bodily component. When we are engrossed in a book or film, the physicality of that immersion is very much an important part of the immersive state. Taking this even further, I would suggest that contemplation, while potentially leading to an immersive state, is not a prerequisite to such an event. Our physicality, consisting of nerve endings, hormones and sensory input can trigger an experience of immersion without any prior circumspection or reflection. The physicality of sport, for example, while far from the artistic impulse is a good example of a non-contemplative immersive state...a "being there with a purpose" that is not reflexive in any intellectual way.

A useful tool to further understand this concept of physical immersion is Heidegger's notion of Ready-to-hand. Ready-to-hand is a status generally applied to objects in a person's environment which afford some sort of specific purpose or use. A classic example of this is a hammer4, which we can immediately use without theorising it's purpose – a hammer is intuitive. Ready-to-hand implies proximity, familiarity and the primordial, an instinctive relationship between a person and their environment. The complement to Ready-to-hand is the idea of Present-at -hand. The latter describes a more factual, intellectual, theorising engagement with objects and the environment. It also suggests a certain neutrality in the relationship. Somewhere between the poles of these states there must exist a transition between the two, and it is this transition which I feel could well describe the threshold between an immersive state and a non-immersive one in the context of interaction.

Marshall McLuhan also speaks of physicality as an important aspect of immersion. He proposed the idea that we think of media and objects as extensions of ourselves. Just as a wheel becomes the extension of a leg, a book can become an extension of our eyes. While some media are more insubstantial they still become extensions of ourselves as we immerse ourselves in them. An effective interactive artwork should become an extension of the viewer/interactor in Ready-to-hand fashion. McLuhan also describes media as being classified as either "hot" or "cold"5. A hot medium is one containing large amounts of data and is well defined, while a cool medium is minimally defined and of low resolution. From the point of view of interactivity, a cold medium involves more participating to complete its meaning. This is not unlike the concept of closure in comics that McCloud refers to6 - the semantic gaps between frames in a comic book that a reader must fill through their imagination. Interactive artworks can be found in both hot and cold categories; the ones of interest to me are more of the luke-warm or cooler variety that are both data-rich but leave enough room for the audience to exercise their fantasy.

Immersive Works

Working principally as an artist and not a theoretician or curator, I tend not to have a comprehensive overview of interactive art as such, being a little myopic in my own creative pursuits. However I do come in regular contact with works either through exhibitions or project documentation. Here's a description of some works from the last few years that somehow relate to this concept of liminal immersion. I make no excuse for the fact that several of these pieces emerged from the Interactivos? workshop series organised by the Medialab-Prado in Madrid. The Interactivos? events and exhibitions have been critical in stretching the limits of the interactive arts imagination.

Chris Sugrue's piece Delicate Boundaries (pictured above) is a clear example of the concept of liminal immersion. In fact Chris describes her own piece with the following text:

'As digital technologies become more embedded in everyday life, the line between the virtual and real is increasingly blurred. Delicate Boundaries imagines a space in which the worlds inside our digital devices can move into the physical world.'7

Delicate Boundaries is a very accessible yet technically cunning piece which depicts a swarming mass of luminescent bugs on a computer screen. As a visitor touches the screen, the bugs swim out onto the person's hand and along their arm, breaching the traditional divide of screen and physical reality. This effect is achieved through a ceiling based vertical projection combined with computer vision tracking of the person making contact. The technology involved is not made known to the audience and the darkness required for the projections and tracking cloaks any secrets the technical set-up might otherwise reveal. The point at which an audience member makes contact with the screen defines the threshold of real/unreal and opens up the possibility of immersion.

A product of the same Interactivos? workshop, Julian Oliver's levelHead augmented reality game (pictured above) creates a similar portal between the virtual and real, with immersion a by-product of the interaction process. LevelHead is a game comprised of small cubes adorned with augmented reality markers that are used to track the movement of each cube face by a camera. As each cube is physically rotated, a corresponding virtual labyrinth overlaid over a live video stream of the cube is tilted, allowing a small animated character to move through various virtual rooms in search of an exit. Described as "A spatial memory game" the aesthetic is of overlapping realities accessed through the interface of the cube.

'While nowadays most artists generally go for full body interfaces and giant screens, Julian built a fascinating, tiny little world in a box that is a pleasure to play with. Game art doesn't get any bigger than this.'

This quote from a Ludology.org post on the piece makes a salient point regarding the physical space taken up by the installation – that in the world of immersive interfaces, despite general trends, size does not always matter. While I have seen the piece connected to a large video projection screen, which nevertheless aids in drawing people into the cube world, the game works equally with the video of the cube maze displayed on a regular computer monitor.