by David F. Bello (for Digital Rhetoric class)
Representation has been one of the most fundamental problems of human communication and rhetoric since the dawn of both and the dawn of the studies of both. Plato's Rhetoric examines the representation of his ideal forms through a "divided line," which contains four states: Imagination, Confidence, Thought, and Intelligence (Irwin 274). Imagination deals with sensory perception; confidence with the recognition of a distinction between representation and the real; thought as a development of causal relationships between these signifiers and signified; and, finally, the fourth, unreachable stage of this line is characterized as "complete knowledge" (Irwin 274). Of course, the reason that this complete knowledge of the realm of Plato's Forms cannot be realized is that it does not exist. There is, in his work, an assumption toward the ideal signified, which cannot be represented; except, perhaps, by the ultimate, superhuman philosopher in his paradigm.
Augmented Reality, or AR, has become a trendy field of computer science due to its rather noteworthy applications to marketing and geolocation that have come about recently. It has often been an element in science fiction (for example, the computerized vision of the eponymous character in the Terminator series), and is seen as a somewhat more attainable extension of research into complete virtual environments research. Rather than immersing the user into a simulated environment, AR proposes to "augment" existing reality with relevant data. However, there are a number of issues with the claim of augmentation and realistic addition. Using a broad range of work on the nature of representation, I will examine how AR systems adapt the notion of reality into televisual streams that incorporate extra-visual data, and whether or not the current implementations of AR technology deserve the name.
Do Computers Experience the Real?
AR implies the augmentation of reality, that is, when considered within the bounds of digital media and computer science, the placement of additional sensory information ("augmented") into the physical domain ("reality"). However, its application and the rhetoric of its use is limited to the placement of additional sensory information into a priori virtual domains, primarily two-dimensional display devices. Implementation of AR can often involve the computation of multiple real-world data types as a means of inserting visual information: precise location-awareness, machine-readable data representation ("barcodes" of many kinds), and the systematic combination of perspective information. However, while these elements construe a more holistic set of source material, they do not implement representations of data within reality or accurately concatenate the real world elements and virtual data elements; they in fact serve the construction of a hybrid virtuality created in the merging of two information systems within a single visual display.
Therefore, AR is an illusion in both its name and functionality, for it is only in an entirely metaphorical sense that reality is in any way augmented; and it is in this metaphorical manner through which a hybrid reality is created as an illusion which serves both the further depreciation of any signified reality and the power of digital media to interact with our world on anything but a symbolic and sensory level. The computer does not in any way understand "reality" and supplement it, merely it concatenates two divergent flows of data, that of direct visual representation through a camera lens and extravisual data being processed in realtime and coordinated to match visually relevant data points. In considering this distinction between the two sets of visual data being represented, an AR implementation exists as a circumstance of extended simulacra.
"The closer one gets to the perfection of the simulacrum... the more evident is becomes... how everything escapes representation, escapes its own double and its resemblance... Escalation in the production of a real that is more and more real through the addition of successive dimensions" (Baudrillard 107).
This set of statements in the chapter on holograms summarizes Baudrillard's view of representation: that the increased "realism" of multidimensional representation "exalt[s] the opposite movement: only what plays with one less dimension is true, is truly seductive" (Baudrillard 107). Therefore, in making the goal not one of full-on virtual space in real time, but that of AR, with its two-dimensional (though with the fourth dimension of realtime interaction) display limitation, the seduction manifests more successfully. A tech-savvy audience is not expecting a Matrix-like experience of infinitely rich data visualization surrounding them in actual space; they are presented with the impressive combination of multiple computer vision and location-aware data technologies.
What is occurring is in no way an augmentation of reality; for reality can be understood as the whole of our experience; rather, encompassing the whole of an experience within a context. In this more specific use of the term, reality must then be subsumed within the idea that a computer may exist much like a human does. For the reality being augmented in the sense of AR is not that of human reality, but that of a computer's limited, based on its "sensory" (geolocative, filmic, auditory, and programmatic) input devices, "perception." Reality, as it is commonly understood, is typified by its commonality to non-schizophrenic living beings. Thus AR must be reconciled as carving out a much wider space for the definition of reality to include the metaphorical sensation, perception, and cognitive reconstruction of computing machinery.
On the other hand, if the intended object of augmentation is not that of the computer's reality, but that of the human, it must also be understood that reality is not being represented in any given AR system. The object of augmentation in AR is, of course, not reality itself. If this were the case, there would be floating objects around us at all times: the products of technology which implements within reality the programmed responses to the physical and cultural data surrounding us at all times. What occurs instead, is that the televisual representation of reality, achieved with a webcam or iPhone camera, perhaps, is augmented with two-dimensional visualizations of data coordinated through complex computer vision technology to appear as if is, in fact, tethered within the physical realm. This is augmentation of filmic representation, not reality in the first order. Reality is encapsulated as a set of objects which the software can understand, and augmented from there.
Hand from Above from Chris O'Shea on Vimeo.
The Shadows and the Prison
Not only is reality in AR applications relegated to the two-dimensional visual display, but its role for the viewer can no longer be primary. The figure-ground relationship is supplanted by a data-reality binary. When holding an iPhone in the air, pointed at a landscape, the landscape itself becomes background, and the visualized data (depending on the given software, this could be measures of distance, names of places, relevant encyclopedic references, or any data deemed by the programmer to be "tethered" "within" this particular physical space) achieves primacy. Derrida, in his exploration of the work of painter Francis Bacon, describes the formation of "haptic vision," which embodies physical sensation in two-dimensional representation (Derrida 155). This notion is reversed in an AR context; the actual, tactile world becomes intangible and illusory, while touchscreen displays allow for the simulated haptic manipulation of the augmentative data. The programmed information becomes the primary figure amongst a background of the represented world.
Being that we exist first and foremost in what I shall call reality, there is no reason to assume that users of AR applications are exactly like those of Plato's allegory of the cave. The presumption there is that knowledge of the real world is predicated by a lifetime of existence under a ruse; that of understanding a representation of reality as the reality in and of itself. In considering AR applications in this scenario, one must admit the foreknowledge of a first-order reality to that of the represented reality in AR applications. However, it would be helpful to consider the allegory for the degrees of representation it presents. In looking solely at shadows, there is the abstraction of a real world into two-dimensional figures which exist on the flat surface of the cave wall. Here, we have the cave wall taken over by a two-dimensional view of our actual reality, but these shadows remain: in the form of the visualized data of AR programs. When we, for instance, view our selves in the midst of objects which do not, for all intents and purposes, exist (such as the three-dimensional model of a car imposed onto the barcode design as part of the German Mini print ad campaign, viewable at http:// www.mini.de/de/de/webcam/index.jsp), we become shadows of ourselves inside a further shadowed world, in which the computer's eye; the computer's perceptive organ of visual input, is the all-knowing philosopher in Plato's model: that which sees reality at a higher level. In other words, the augmented objects (a class of "virtual holograms," which appear computer-generated, yet appear to be set within our tangible world) are a type of Formal ideal; that which our inglorious, earthly minds cannot perceive, but is shown to us in the form of a shadow. Using AR applications allows us to become the prisoner who sees the shadows of a higher, ideal realm of programmed vision. It is without this set of chains (the webcam setup; the iPhone and the app; the VR goggles and the treadmill) that we experience only the actual reality, and not reality in the sense of AR: encapsulated, two-dimensional (therefore, according to Baudrillard, "seductive") reality of simulacra.
Conclusion
Baudrillard's worry was that the real would disappear under a blanket of hyperreal objects. If gullible users are to accept the shift of reality by AR systems into the background, there is the potential for the hyperreal (that is, the replacement of our human modes of perception with televisual ones) to encompass all that we encounter; leaving reality to culturally disintegrate.
This is not to malign the technological significance of AR technology in society and computational development as a whole. AR applications have the potential to provide a great deal of assistance to those with disabilities, and anyone existing in a highly complex environment such as the internal mechanisms of a motor vehicle, the physiological body, or navigable territories. On the other hand, AR rhetoric portrays the ultimate embodiment of technology in reality (through its name) and the complete embodiment of reality in technological representation (through its functionality); two capacities of digital media which, if at all, are only achieved through the highly mediated combination of visual and extra-visual data. ! Much of the hype of AR is in the proposal of practical applications. While many are available already (an application which visually directs you to the nearest subway stop in London and New York are already available for Apple's iPhone), the majority of AR systems have been in prototype form for a number of years, and are demonstrated on YouTube by many design groups. This is a relatively new direction for computer science, and it will be interesting to see how it will engage with practical computer use, at home and in mobile devices, in the future. As a new avenue of research, it is important to recognize the role of theoretical discourse in situating these implementations as part of our lives and actual reality. Will we become willful prisoners who worship the realm of digital shadows? Unlikely. However, it may become more common to see individuals interacting with the physical realm through the filter of camera and software. It is important to understand the separation of computer vision form that of the human experience. If, one day, Paul Virilio's ideal prediction of the "vision machine" were to come to pass, would we voluntarily insert that layer of software representation between ourselves and the reality we so currently encounter?
WORKS CITED
Baudrillard, Jean. "Holograms." Simulacra and Simulation. Ann Arbor: University of Michigan Press, 1994. 105-110.
Deleuze, Gilles and Francis Bacon. Francis Bacon: The Logic of Sensation. London: Continuum, 2003.
Irwin, Terence. Plato's Ethics. Oxford Oxfordshire: Oxford University Press, 1995.
"MINI.de - Das Neue MINI Cabrio." 2008. Bavarian Motor Works. 17 Oct. 2009. .
Virilio, Paul. The Vision Machine. Bloomington: Indiana University Press, 1994.