Abstract
Recently availed modes of mediated communication and perception, generally categorised as 'virtual experiences', have offered the designer of space a new array of spatial conditions to contend with. Each variety of virtual experience, from text-based discussion forum to immersive virtual reality environment presents challenges to traditional assumptions about space and its inhabitation. These challenges require design theorisation which extends beyond the notions of design within the virtual, or projected, space (the layout of the chat forum, the appearance of the virtual realm etc), and require a reconsideration of the entire apparatus of the mediating devices (the physical spaces which facilitate the interaction, the connection between them and the virtual space etc). This paper presents both an experimental framework for understanding the space of the virtual and outlines a current research project addressing these theoretical challenges through the spatial implementation of a synthetic environment.
Original language | English (US) |
---|---|
Pages (from-to) | 61-77 |
Number of pages | 17 |
Journal | Convergence |
Volume | 8 |
Issue number | 3 |
DOIs | |
State | Published - 2002 |
Externally published | Yes |
ASJC Scopus subject areas
- Communication
- Arts and Humanities (miscellaneous)
Access to Document
Other files and links
Fingerprint
Dive into the research topics of 'Testing the space of the virtual'. Together they form a unique fingerprint.Cite this
- APA
- Standard
- Harvard
- Vancouver
- Author
- BIBTEX
- RIS
In: Convergence, Vol. 8, No. 3, 2002, p. 61-77.
Research output: Contribution to journal › Article › peer-review
}
TY - JOUR
T1 - Testing the space of the virtual
AU - Lonsway, Brian
N1 - Funding Information: Lonsway Brian 09 2002 8 3 61 77 Recently availed modes of mediated communication and perception, generally categorised as 'virtual experiences', have offered the designer of space a new array of spatial conditions to contend with. Each variety of virtual experience, from text-based discussion forum to immersive virtual reality environment presents challenges to traditional assumptions about space and its inhabitation. These challenges require design theorisation which extends beyond the notions of design within the virtual, or projected, space (the layout of the chat forum, the appearance of the virtual realm etc), and require a reconsideration of the entire apparatus of the mediating devices (the physical spaces which facilitate the interaction, the connection between them and the virtual space etc). This paper presents both an experimental framework for understanding the space of the virtual and outlines a current research project addressing these theoretical challenges through the spatial implementation of a synthetic environment. sagemeta-type Journal Article search-text 61 Testing the Space of the Virtual SAGE Publications, Inc.2002DOI: 10.1177/135485650200800306 Brian Lonsway Recently availed modes of mediated communication and perception, generally categorised as 'virtual experiences', have offered the designer of space a new array of spatial conditions to contend with. Each variety of virtual experience, from text-based discussion forum to immersive virtual reality environment presents challenges to traditional assumptions about space and its inhabitation. These challenges require design theorisation which extends beyond the notions of design within the virtual, or projected, space (the layout of the chat forum, the appearance of the virtual realm etc), and require a reconsideration of the entire apparatus of the mediating devices (the physical spaces which facilitate the interaction, the connection between them and the virtual space etc). This paper presents both an experimental framework for understanding the space of the virtual and outlines a current research project addressing these theoretical challenges through the spatial implementation of a synthetic environment. Proposition Like most binary oppositions, that formulated between virtual reality and that other reality we call real life does little to advance our understanding of the subject at hand - in this case, of the complex spatial and cultural implications of new immersive technologies. If designers are to competently address the systemic shifts that may be induced by these technologies, it is imperative that they have the capacity to understand their explicit and implicit modes of operation, and in particular, their modes of spatial operation. The commonly accepted but reductive framework which categorically separates the space of virtual experience from our larger spatial context privileges technological apparatus over socio-spatial context, perception, and inhabitation. This attitude is echoed in most cultural representations of virtual reality (VR): product advertising promotes the ability of users to leave their real environments and enter fantastic ones with the aid of certain devices, developmental research of VR systems focuses almost entirely on technical implementation and discusses social or psychological issues only where these are a result of prior technological work, and popular fiction invents myriad mechanisms and devices which provide novel ways to travel from reality into cyberspace. Not all research in VR technologies, of course, so naively fails to recognise the significant amount of interconnectedness between the 'virtual' part 6462 of an experience and the other (the existence of the body, the physical manifestation of the technological apparatus, etc).' However, even a perfunctory analysis of VR research concerned with spatial implications reveals that the opposition between the real and the virtual is quite manufactured. The commonly accepted space of VR is that representation of space which is presented perceptually to the user. This conceptualisation of space privileges, through technological capability, the representational aspects of space - the conception of space as a geometric (and due to dominant representational systems, Cartesian) realm accommodating placement, movement, and proximity. This geometrically instrumentalised conception of space fails to recognise space's material or social aspects, and therefore excludes the space of immersive operation - that space within which the VR experience occurs. I would like to argue for an expansion of this narrow spatial conceptualisation as a way of transcending the merely technological manner in which VR technologies are currently being developed. This expanded notion positions the space of the VR experience as a mediated (or more specifically, 'informated') realm which includes not only the domain of representation, but the physical domain of the apparatus and the physical and psychological domains of those who interact with it. Central to my argument is a kind of defence of space, or at least a defence of its importance as a framing mechanism for critical thought. The understanding of space is part of a long philosophical tradition, and my brief analysis here is no attempt to categorise this history or even to succinctly address its significant moments. Nor is this an attempt to offer any kind of totalising conception of space. It is an attempt, nevertheless, to offer a multiscalar reading of the spatial operations which we seem to most regularly encounter implicitly or explicitly in our contemporary actions and discourses. Much of this outline is grounded in the post-Marxist tradition of writing on space in the last three decades - notably that of Henri Lefebvre, Michel Foucault, Manuel Castells, and Edward Soja - and the work in reaction to this which has critiqued its phallocentrism, essentialism, or cultural elitism - notably, the writings of Rosalyn Deutche, Mary Macleod, and Elizabeth Wilson.2 This particular strain of the evolving arguments on the construction of space and its reciprocal effects on social development situates space at the intersection of representational phenomenon (space is primarily an invented mode of discourse which facilitates the conceptualisation of certain kinds of organisation), social phenomenon (space serves as a conduit of communication systems, and thus informs identities and subjectivities), and material phenomenon (space, perhaps more than money or goods, embodies social, cultural, and political power structures). Common to these arguments, the field of actions and discourses in which we all participate forms an array of operations constituting three scales of spatial practices: Cartesian, scapic, and - organisational. 6563 Cartesian spatial practices are formed from essentially geometric readings of space; consisting entirely of quantitative spatial relationships, spatial activity is limited to location, extension, and proximity. Movement is implicitly addressed but not facilitated (ie vectors and curves can be represented within a Cartesian system, but the rules of Cartesian ordering do not provide the tools to construct them). The scapic mode of spatial practice introduces a social arena which is produced or constructed in certain arrangements to facilitate cultural sustenance or human desire.3 Landscape is perhaps the most obvious such scape, but any domain of social engagement manifesting its forms of production or construction can be understood in this way. Where the Cartesian framing of space introduces location, distance, and direction as spatial operatives, the scapic introduces inhabitation, navigation, and movement. Within the realm of landscape, for example, one can speak of a landscape of power, or an urban or suburban landscape, or a landscape of thought. Spatially, in these examples, the landscape is a framing device, collecting temporally, socially, and culturally diverse phenomena into a collective realm of discursive interaction. Finally, organisational practices, as the third spatial formation, extend the spatial framework into the tropic realm of conceptualisation. Here, the metaphors of Cartesian and scapic spatial practices are used to form spatial domains for any phenomena which could benefit from the vocabulary of spatial relationships. When we speak of navigating the World Wide Web, or refer to the proximity of one piece of information to another, we are spatialising these phenomena through our rhetorical modes of operation. The defence of space, a practice I share with other authors working in this field of inquiry, arises from the observation that most cultural, political, economic, and historical thought through the better part of the twentieth century suffers from a temporal bias, examining human development along a temporally historical axis and ignoring the impact of our spatial formation. While the details of this argument are well beyond the scope of this essay, the bias remains explicitly visible in the technological project of VR. The hyperbole of the information technology volution (I hesitate to call it a revolution; the decorative characteristic of a volute seems a more appropriate descriptor) has positioned obsolescence and 'versioning' (a word which, tellingly, my spell checker did not reject) at the core of its progress. The object, service, or function, let alone the spatial existence of a particular technological product is subsumed by its temporality. The fact that our soon-to-be-disposed digital-incapable televisions will overwhelm our landfills in 2007 when broadcasters are required to transmit only digital signals is, in this regard, an inconsequential byproduct of technological development. It is a spatial effect which, if taken seriously by television's manufacturers, would impede the technological advances required in a time-obsessed consumer culture. Equipment is not 6664 designed to be upgraded, but rather replaced. By publicly negating technology's spatiality, its developers are able to offer a rhetoric of progress which is unhindered by material constraint. The assertion of the significance of a spatial framework to technological development allows one to reveal these material constraints, empowering the ideas and individuals which rely on space for their survival. Virtuality Yet, within the specific context of virtual systems, traditional concepts of space are challenged, making a clear articulation of the nature of space difficult. At the same time that the spatiality of a virtual environment may seem to be acknowledged in its very nature - that the imagery on screen or on a head-mounted display represents a space with its own spatial logic - it is nonetheless a space tied explicitly to the state of the enabling technology. While the space of virtual environments is most certainly a new spatial formation, it is not the space inside the system that is new, but rather the space within which the system operates. Inasmuch as space is simultaneously geometric, scapic and tropic, it cannot be explicitly implemented within the technologies of a virtual system. This technology does modify spatial behaviour in each of these three domains, but it is the resultant space of human interaction which is the new space of its making. In exploring similar topics, although less explicitly focused on spatial phenomena, Margaret Morse introduces the concept of virtuality to conceptualise technological interaction as a socially engaged phenomenon. This notion, embraced most clearly in her book of the same title, is used to examine a wide range of technological and spatial constructs highly relevant to the understanding of 'the virtual'. Morse explains the virtuality by tracing elements of Heideggerian phenomenology through the viewpoint of a contemporary cultural anthropologist: 'cultural forms from television graphics and shopping malls to the apparatus of virtual reality, as well as practices from driving to conducting war to making art employ various forms of engagement to construct a virtual relationship between subjects in a here-and-now.'4 She later suggests that 'monitor-human relations are thus bubbles or pockets of virtuality in the midst of the material world', a framework which can help in conceptualising the virtuality as a spatial phenomenon.5 5 By generalising the notion of a virtuality to include all conditions of in- effect-but-not-in-fact (the seldom understood denotation of'virtual'), Morse is able to contextualise the concept of VR and virtual experience within a social framework: a virtualscape, if you will. Morse begins her argument with an analysis of the subjectivity- constructing effects of television. The television, she argues, continually positions its viewers as a you through the single-sided dissemination of information. The act of broadcasting, most apparently in news and advertising, is an act of subjectivity construction through the emphasis of the 'you', the receiver of televisual utterances. Advertisements speak to 6765 a valued consumer and are placed during programmes which have been proven to be popular with specific demographic groups. Their transmissions are directed to a you as you are supposed to be. Viewers form subjective representations of their selves through their identification with this you, even if, as Morse suggests, 'you may not actually be in that position', having actually flipped channels or left the room. This temporal you as a subject of televisual utterance exists as a kind of virtuality: a realm which appears in effect, although not in actual fact. On one hand, you are still the you you were before the utterance, but you are also now the you who should buy detergent, tune in later, wear fashionable clothing, etc. Effectively, your subjectivity has multiplied. With regard to interactive experiences, including interactive VR systems, Morse argues that there has been a shift from the subjectivity discourse of the you to the subjectivity discourse of the I, where technological interaction is used to construct an / similar to the you of television. With the special case of the immersive VR experience, the user is (in actual fact) located in physical space within the apparatus of the technology. The computer-mediated environment suggests (in effect) a trans-location outside of this domain, but only through the construction of a subject centred on the self (I), controlling an abstract position in a graphic database of spatial coordinates. The individual, of which this newly positioned subject is but one component, is participant in a virtuality: a spatio-temporal moment of immersion, virtualised travel, physical fixity, and perhaps, depending on the technologies employed, electro-magnetic frequency exposure, lag- induced nausea, etc. Morse's notion is hardly new; similar notions of subjectivity formation influenced by Lacanian psychoanalysis proliferate in late twentieth century cultural theory. What is potent about Morse's observation, however, is the way in which she engages the term virtuality to refer to this moment of subjective multiplicity as a moment of technological engagement.b If the notion of virtuality is expanded to include any such moment of multiplied identification or subjectivity formation, then the virtuality becomes a way of conceptualising these mediated moments of subjectivity identification. There is, by this argument, no substantial distinction between the mediation of our perception by VR apparatuses and by our genetic, cultural, physical, or other technological mediations. All such mediations, technologically manifest or not, form our senses of reality and subjective positions within it. VR, therein, is merely another one of these moments, taking place in the space of our lives. Its space is not the instrumentalised space of its presentation devices, but the subjectivity-forming space of its physical, cultural, and social manifestation. The failure to recognise the actual spatial context of a so-called VR experience in effect places a faith in technological systems to address the complexities of human social experience. If, in other words, the dominant development framework for VR remains 6866 exclusively technological, VR systems will remain unable to account for complex social and human behavioural conditions. But they will be unable to do so not because of technological limitations, but because of the restrictive means of conceptualising their very nature. Moreover, even as virtual systems may ostensibly extend their technical functionality, their conceptual constraints will continue to delimit human behaviour ever more severely. Presence This is not to say that technical researchers have completely failed to recognise that their immersive creations have a significant impact on human perception, cognition, and sociality. However, even their primary measure to establish a sense of human engagement with virtual systems - the metric of presence - fails to address any of the spatial criteria that the virtuality offers to connect mediating technologies to human socio-spatial existence.' Presence evaluates the success of a virtual environment system as a measure of an individual user's sense of feeling present in a simulated environment. While rooted in the proprioceptive reactions of the human cognitive system, this measure exhibits a bias toward understanding technological apparati rather than human spatiality. The evaluative question basically asks 'does the technology of the virtual system interfere in any way with your ability to believe that you are located in a space different than the one your body is in?' The technical literature concerning presence enumerates those things which obstruct or hinder its perception, including obstructive aspects of gear, lag time in image updating, lack of realism in visual, aural, or tactile presentation, or lack of a complete multi-sensory environment. Philosophically, it is untenable to evaluate one's perception of presence without first establishing the socio-spatial context within which presence is measured. But I have yet to come across any such validation. In spite of this, presence has been adopted as the foremost evaluative measure of virtual systems, and as a consequence has lead to a complete neglect of any significant spatial development in the research domain. In particular, I see three significant problems inherent in the assumption that presence can operate as a reasonable terrain for evaluation. First, the measure of presence in a virtual environment above all assumes that the sensation of presence is a natural means of human interaction with and perception of our social and physical environments. This privileges the already-present or assumed present within a given cultural context. N. Katherine Hayles and Alluquere Stone among others have written extensively on the cultured associations of such assumptions, arguing that the privileging of that which is present merely privileges predominant cultural types: the male, the 'normal', etc.' Successful presence privileges certain abilities: the ability to afford (financially or temporally) to interact with a VR environment, the knowledge of how to navigate within it, the level of comfort with the cultural environments 6967 represented (often warlike or gendered), the lack of fear of technology (in particular when it has to be worn or applied to the body). Those without these abilities are, by the VR industry's measure of presence, already (and perhaps always) absent. Second, aside from these social and psychological constraints, it has not yet been established that those of us who are privileged to be present exist capably in the world by measuring our successful interaction with our surroundings via the question 'do I feel present here?' An empirical assumption derived from early existentialist thought, this belief fails to address a myriad of other possible models for spatial interaction which extend beyond the visceral and psychological concept of spatial interaction into the socio-cultural. And finally, there is an irony in even conceptualising the need to have such a measure as presence. If we take a less technologically mediated condition of virtual experience - immersion in the plot of a printed book - it is impossible to conceive of one's immersion in the book and one's spatial immersion in a physical environment independently. With the imperative to sensorially immerse a participant completely in another realm with advanced technologies, however, the problem of the electronic replacement of the physical comes to the fore. The technology itself - by its design and naive conception - draws a distinction between the technologically mediated and the non-technological. The immersive technology of virtual environments, a direct manifestation of the flawed philosophical underpinning of VR research, itself raises the problem of presence. In addressing these limitations of the presence model for evaluating virtual environments, Morse's notion of the virtuality is highly instructive. If the moment of interaction with the virtual environment is conceived to be akin to the moment of interaction with a television or a telephone, then we can more easily see how this immersion is no different than any other technological engagement. The engagement is an event of mediated socialisation (socially and spatially immersive) rather than technologised escape from the everyday (socially and spatially divisive). The space of this engagement is multi-scalar, encompassing the space of computation as well as the space of subjectivity inducement and the space of cultural association. The space of the virtual, as introduced above, is no different than any other form of space. The problem is that this argument will neither sell products nor attract the Department of Defence to a research proposal. The cult of the new, well established before the fetish of information technology, has become the modus operandi of VR research, producing a cycle of limited research work which will continue to be unable to achieve the social, communicative, and truly radical aspirations the researchers propose. Extension In order to attempt to escape these limitations on technical research, I have been involved in the development of a virtual system which strives to account for the subjective devices of the virtuality through spatial 7068 design and theorisation. On one hand, this is a relatively straightforward enterprise: we (the research team and I) have established that our system must acknowledge that the embodied social inhabitation of space is essential to a workable solution.9 In other words, we reject solutions which obstruct the kind of spatial interaction we expect in non-technologically mediated spaces. On the other hand, this spatial drive of our research work requires us to constantly parallel technical research (like visual processing algorithms or acoustic wave diffraction calculations) with cultural theory (eg the interpretive theorisation of mediated subjectivity) and design inquiry (the considerations of architectural materials for human comfort, for example). While we constantly risk instrumentalising the theoretical inquiries and assumptions, we believe that this paralleled development of technical implementation and cultural/social/spatial research is required for any significant contributions to the future of virtual systems. In particular, understanding the virtuality - a socio-spatial construct that includes, but does not privilege the technologically mediated experience of engagement with a specific set of interactive devices - requires that its tenants be tested in practice. Our challenge specifically attempts to engage the development and implementation of a synthetic reality environment" on socio-spatial terms (ie as a virtuality), and argues that this means of conceptualising the domain of virtual reality environments may more substantially address current limitations of VR research than typical technology-focused means. All of our projects, collected under the title 'Synthetic Space Environment Research', explicitly engage a synthesis of technological and non-technological approaches to spatial design and are currently focused on the realisation of a multi-sited tele-collaboration environment for performative activities. We have chosen to examine the notion of performative action as a subset of spatial activity in order to locate types of human activity in space which are excluded from existing tele-collaboration systems. In particular, we define performance to include any sort of activity where bodily action plays a substantial role in the successful communication of information between individuals. This includes what is generally understood to be performance: musical presentation, theatre, dance, etc, as well as other performative activities which typically fall outside this realm: team-based design project reviews, lectures by exuberant speakers, spatially interactive distance education classes, etc. Performers move around actively and perhaps rapidly, they do not necessarily congregate in clearly discernable groups, and they may gesticulate with their entire body. Their bodies and the space their bodies occupy are essential aspects of their communication. Within this performative framework, it is important that user obstruction by head-mounted displays, tracking devices, and the like be eliminated, 7169 and that the spatial mode of user interaction be highly intuitive and as close to full-scale as is possible. From our theoretical interests, we also wanted to avoid any qualitative predeterminations about the virtual condition, and to put into question the fundamental separation of real and virtual spaces which existing technologies reify. Finally, we take seriously the limitations of digital processes for achieving the virtual, and are thus employing systems and methods which operate at the intersection of the analogue and the digital. To understand the implications of these challenges, first consider a typical (and well- researched) immersive tele-collaboration configuration: users don tethered head-mounted displays which provide stereo imagery and audio of a computer-generated environment in which collaborators are represented by avatars (iconographic representations). Users navigate this representational space by manipulating a three-dimensional mouse or some such device, and interact with each other through their voices and through graphic modifications of their avatars. User interaction is far from intuitive: a gesture might have to have been pre-written into an avatar behaviour and called from a library of possible gestures; movement, which is actually only a movement of one's point of view, is accomplished with movement of a user's thumb on a mouse. While visuals may be presented at full scale, the disconnection between the movement of one's bodily position and the shift of a user's point of view breaks down the cognitive understanding of the visual environment as a full-scale environment. Immersive systems which require users to disengage from their physical environment (by blocking their sight and hearing) preconceive a distinction between the 'reality' of the physical world and the 'virtuality' of the computer-generated one; a distinction which, as I've pointed out, is wholly artificial. We avoid (or at least are proposing to avoid) these problems through the construction of a system which allows inhabitants to move freely about a space unconstrained by cables, which facilitates interaction with others in a remote location across a spatialised video and audio link with their own bodies and gestures, and which uses spatially inventive video processing and acoustic simulation techniques to provide the necessary signals to the remote site. Few of the components (architectural design, acoustic simulation, or video processing) are our own; we are compiling a complex array of existing work, incorporating some of our own unique research, and developing a new integration of these to carry out our objectives. Implementation The space of collaboration in the Synthetic Space Environment consists of two geographically separated rooms connected across a high bandwidth network with one full wall of each room presenting live video of the remote inhabitants and with an array of speakers in each room presenting spatialised audio of the joined space (figure 1). The video signal is perspective-corrected for each space such that the image 7270 Figure 1. Diagram of the coupled spaced: (a) illustrates the configuration in each studio, (bJ illustrates the combined, or coupled effect. In (b), the straight lines represent the physical studios and the curved lines represent the computer-generated space in which remote collaborators may appear. The coupled space of studio 1 in (a) is indicated in (bJ with double-lines, and that of studios 2 is indicated with single- lines. appears as a spatial extension of each room. Using computer-vision methods, we are able to extract the video of individuals from the video signal and, if we have a need to, composite the inhabitants into a computer-generated environment (figure 2). Whether the remote participants are viewed in the room they physically occupy or in a computer-generated space, the space of collaboration expands from the single (local) environment to the coupled space of both environments. For this to be convincing acoustically, we cannot merely transmit the audio from one room to the other; it should appear (acoustically) to the inhabitants as if they are occupying a space that has the dimension and geometry of the coupled space. In other words, if I am working in a local room that is 10 metres square with remote collaborators that visually appear to be in a room that is 30 metres deep by 10 metres across, I would expect that the acoustic environment would be of the coupled space, 40 metres deep and 10 metres across. By creating a visual and acoustic environment that operates in this way, and in particular by explicitly connecting the visual and acoustic parameters in 7371 Figure 2. Demonstration of the keying process which composes two video signals together to insert individuals into a composite image, taken from an early prototype of the synthetic environment. Our current research engages techniques which do not require the mono-chrome background shown in the upper left. the data transmission, we are able to establish at this meta-level of the project a substantially more spatial environment for collaboration than that provided by existing tele-collaborative systems. With our goal to allow - across a network - as much spatial freedom as can be expected in a single architectural environment, even the (theoretically albeit not technically) simple phenomenon of correlating aural and visual perception can increase the sense of spatial participation. This doubled coupling, the coupling both of two physically and/or geographically discrete spaces and the coupling of two sensory phenomena (the latter not at all seriously addressed in existing tele-collaboration research) can most importantly help us escape the evaluative framework of presence by placing the focus of inquiry on the extant, self-evident spatiality of the participant's environment rather than on the wholly mediated so-called immersive environment that exists outside it. As inhabitants never leave their material, socio-spatial environment to enter into a mediated one, their concept of presence and absence remains in the realm of subjective spatial perception rather than in the realm of technological determination. In other words, presence and absence are accommodated as psycho-perceptual phenomena rather than tied explicitly to the instrument of technical mediation. It is thus possible to feel absent within the space of the SSE while still an active participant in the construction of the virtuality. In fact, if we were to reduce our research objectives to a direct parry against virtual environment investigations of presence, we could claim that we encourage the feeling of absence as much as the feeling of presence. Both are not only constructive for an inhabitation of space, but are required psycho-social perceptions for our survival. Our technological accommodation of the virtuality, however, extends - beyond the full-scale manifestation of visual and acoustic representations. 7472 if the virtual, as Brian Massumi suggests, is a condition of vagueness transcending not only the quantitative condition of possibility but even the complex 'multiple-vague' condition of potential, then a technological project addressing the virtual must accommodate its complicated 'singular multiple vagueness'. Where the vagueness of potential is 'the presence of something more than any pre-established array of possibility, [...] the even-more-vague of the virtual, the in-itself of in-folded transformation, can be neither felt nor thought."' This topological state of virtual's absence can only exist, according to Massumi, in the analogue; digital computation is not only not vague, but is explicitly non-virtual. 'Nothing is more destructive for the thinking of the virtual than equating it with the digital.''2 Based on discrete binary calculations, digital systems can merely represent the realm of possibility. A bit may be on or off; it never exists in a vague condition where it is off but potentially~n. Within digital systems, even the smallest unit of organisation is discrete, whereas in analogue systems (eg time and space) there is no smallest unit. Every state is explicitly full of potential, as the entropic evolution of our universe continually reorganises its energy. While it seems that even the 'pre- virtual' condition of potential is an unrealisable temporal condition, the suggestion of the analogue as virtual's medium places it into a spatio-temporal framework. By complicating the vagueness of potential, the virtual creates a point in space-time about which the effects of potential revolve, creating a condition where the virtual, while remaining inaccessible, is conceivable in its effects. 'The virtual that cannot be felt cannot but be felt, in want of potential, outside possibility: in its effects.''3 Figure 3. Example of the use of the SSE for geographically dispersed performance. The two actors are in different studios (both at Rensselaer Polytechnic Institute at this stage), and each represents a character acting in a different computer-generated environment. One is waking up at home, the other, arriving at the office. The diagrams at the bottom of the image are plan views of the extents of the home l'h 'J and office ('0') environments. By engaging the body as its own full-scale visual and acoustic representation, we position the concept of embodied, multiple-vague subjective spatial formation at the core of our system. Furthermore, we have positioned one theoretical pursuit at the core of our future work which we expect will allow a unique integration of digital manipulation with subjective inhabitation and perception. Our intention is that the digital will operate to compliment rather than limit the field of potential within the tele-collaborative space. In so doing, we are attempting to engage the logics of analogue space within our digital manipulations, providing an opportunity for the effects of digital possibility and analogue potential to fold out into a virtual condition. Undoubtedly this is a substantial challenge, and I won't yet pretend to defend our technical position against the complex challenges of our theoretical and 7573 philosophical foundation. Nonetheless, I would like to at least offer our response to the challenge in an experimental framework as we commence its undertaking, and see it as an opportunity to continue the parallel investigation of spatial theory and technical research. To date, we have established a live full-scale video connection between two sites and have completed significant experimentation with the composition of inhabitants in these sites into a common computer-generated environment. (See figure 3.) This work has borrowed heavily from the field of Virtual Set research; in fact, what we have accomplished most significantly is the ground-up construction of a highly open-ended virtual set system that operates much like those of major television studios, but which does so explicitly across a network to join collaborators in remote geographic locations (figure 4). We have used this virtual set configuration primarily because it has best met the challenges of spatial engagement that I have set out above; participants Figure 4. Diagram of the remote composition process, compositing video of individuals from two geographical locations into a single computer-generated space. are free to walk about a large space, interacting visually and aurally through a full-scale video projection. In addition, it is a relatively simple resource to configure spatially, and offers great flexibility for experimentation. However, we are currently limited by a number of factors which in fact prevent us from reaching the more challenging goals of the virtuality. We are currently operating with a single camera in each space, which makes eye-contact across the large projection wall impossible (and thus wholly within the domain of the possible, inaccessible to potential or the virtual), we are unable to ascertain the locations of participants within each space unless we use tethered tracking devices, and we are limited to operating in a room painted a garish shade of blue. These constraints demonstrate the difficulties in - overcoming the quantitative realm of digital technologies. A single 7674 camera, while not itself digital, operates as a digital constraint To make eye contact across a video link, one must look directly at the camera, and unless this camera were ro be positioned mthm the video image of the remote collaborator precisely where their eyes appear, it IS not possible to make eye contact This has proven difficult to accomplish physically because of material limits (a camera m front of a video image will block the image, a hole in a video screen cannot dynam~caily move to the position of a user's eyes such that a camera could see through the screen, etc) Similarly, tracking devices tethered to an IndivIdual severely restrict their movement, setting up a binary division of space mto trackable and non-trackable areas And finally, at a larger scale, the specific spatial restriction of our research to a highly configured and specifically outfitted studio environment is setting up a similar limit of spatial possibility, prohibiting applications of the system because of the impossibility of transporting it Figure 5 Illustration of the virtual video techniques The local and remote studios are shown on the left, with hvo participants in each space zip7 ond P2 Pl' and P2' remotely) 14 Our primary research objective in this next phase off work is to overcome these three constraints with computational techniques of who> might besf be called subjective spatial modelling In the area of computer vision we are employing a set of 'virtual video' techniques which allow us to synthesise views from two or more actjol cameras and simulate virtual camera locations at any point in between These techniques also allow us to create an approximate spatial model of the physical objects and inhabitants which are captured on video Finally they allow us to create a complex multi-view/point projection image rho allows multiple inhabitants in each space to make direct eye contact with each other across the projection surface Figure 5 demonstrates the combination of virtual video techniques Acoustically, we are using tv,-- methods - active sound field synthesis and wave diffraction modelling - to simulate a highly accurate sound field (a real time three-d~menstona~ representation of a space's acoustic properties) which can accommodate an indeterminate array of arbitrary spatial geometries In this way, we are able to make each inhabitant sound as if they ore speaking from the precise point they visually appear to be standing at 7775 and as if they are speaking in the space of the coupled room. These acoustic and visual techniques require each other; to be able to create an accurate sound model of a sound source in a given room, for example, the sound source - that is, the individual making noise - needs to be located, which our virtual video techniques can determine. When implemented in this coupled fashion, these techniques, we have argued, will be able to accommodate multi-user eye contact, untethered movement about a space, and system portability. This proposed system requires only a projection screen, a small set of speakers, two video cameras, and two computers at each location. Because of this, the specific room requirements are lessened (although some geometric constraints remain due to camera field of view limitations), the technology itself is more portable, those engaged with the system do not need to wear any special gear to interact with the system, and - should we desire to place the occupants in a computer-generated environment - the requirements for a virtual set studio are eliminated. In other words, by moving to techniques which rely on space (the spatial separation of the cameras and speakers, the use of the space between the camera and the objects in the scene as a medium to track objects, the spatial properties of acoustic wave behaviour) we are able to increase the analogic nature of the system. The computer models we employ are not merely geometric representations of space, but analogues to a subjective interpretation of spatial perception. In both the aural and visual techniques, the space modelled is not the 'global' space of interaction, but rather a space of combined individual perception. The model is literally virtual. It is not a possible model of the space in that there is no clear transformation between the model and the physical space. And it is more than a potential model of the space in that its vagaries are both multiple (as described in figure 5, the composed image is comprised of multiple arbitrary points of view) and singular (the composed image is itself a folding-in, a complication of, the multiple images into a vague image surface). If one were to perceive the visual and acoustic space of the system from an untracked position, this virtual condition would be immediately apparent. This subjective representation of space operates specifically in reaction to the individuals in the physical environment, establishing a direct relationship between the analogue subject and the digitised model. By allowing one's social body to mediate the spatiality of the computational model rather than one's digitally discretised avatar to merely operate within a preconceived Cartesian model, we thus increase the project's support of the hyper-potential of the virtual over the possibility of the digital. While we do believe that we can address certain of our limitations with respect to the theory of the virtuality espoused, we do understand that any such instrumental application will still fail to answer all the challenges such theory offers. Yet, by conceptualising our technological goals in terms of these premises, we believe that we are able to better contend with the more complex requirements of humans - bodily, 7876 cognitively, and subjectively - than are systems which either rely on reductively simple concepts of space, or worse, which fail at all to critically address what the virtual (actually) implies. Above all, we are concerned with the meaningful socio-spatial development of new forms of intensely mediated environments. Our process is one of a forced ambiguity between space and technological instrument, a conjunction which is not exactly simple to maintain when working so closely with systems which prefer expected, enumerated, and consistent input. However, we do believe it to be a process worth maintaining, lest we return to uncritical oppositions between the computational and the theoretical. If the real is, in fact, inseparable from the virtual, then so is the theoretical, in effect, from the applied. Notes 1 From the online Mirriam-Webster Collegiate Dictionary, virtual is defined as 'being such in essence or effect though not formally recognized or admitted' (Mirriam-Webster's Collegiate Dictionary 2000. 15 April, 2000) The online Cambridge Dictionary of American English defines it to be 'almost, but not exactly or in every way' (Cambridge Dictionary of American English 2000. 15 April 2000) And finally, a 1985 American Heritage defines it to be 'existing or resulting in essence or effect, though not in actual fact, form, or name' (American Heritage Dictionary. Boston: Houghton Mifflin Company, 2000). 2 Henri Lefebvre, The Production of Space, trans., Donald Nicholson-Smith (Oxford: Blackwell, 1974). Michel Foucault, The Order of Things (New York: Vintage Books, 1970). Manuel Castells, The Informational City: information technology, economic restructuring, and the urban-regional process, Oxford: Blackwell, 1989). Edward Soja, Postmodern Geographies: the reassertion of space in critical social theory (London: Verso, 1989). Rosalyn Deutsche, Evictions: Art and Spatial Politics (Cambridge: MIT Press, 1986). Mary McLeod, 'Henri Lefebvre's Critique of Everyday Life: An Introduction,' in Architecture of the Everyday, eds. Steven Harris and Deborah Berke (New York, Princeton Architectural Press, 1987). Elizabeth Wilson, The Spinx in The City: Urban Life, the Control of Disorder, and Women (Berkeley, University of California Press, 1991). 3 Arjun Appadurai, in Modernity at Large: Cultural Dimensions of Globalization (Minneapolis: University of Minnesota, 1996), articulates a theory of cultural differences and interactions through a notion of scapes (eg financescape, mediascape). For Appadurai, this framework helps spatialise these phenomena without grounding them in a particular instance of space (national boundaries, neighbourhoods, etc.). 4 Margaret Morse Virtualities: Television, Media Art, and Cyberculture (Bloomington: Indiana University Press, 1998), p. 4. I use Margaret Morse's explicit concept of virtuality because of its clarity and relevance to my arguments herein. However, it is important to note that this notion, whether in name or not, has formed the basis of many writings on the virtual realm. In particular, I would like to acknowledge Michael Heim's The Metaphysics of Cyberspace (New York: Oxford University Press, 1993), Avital Ronell's The Telephone Book (Lincoln: University of Nebraska Press, 1989), and N. Katherine's Hayle's How We Became Posthuman: Virtual Bodies in 7977 Cybernetics, Literature, and Informatics (Chicago: University of Chicago Press, 1999) for influencing my work on the subject. 5 Morse, p. 7. 6 A possible weakness of her suggestion is that it privileges the self - the self before the transmission-imposed you or the interactivity-imposed I - as a self without subjectivity, where the self is somehow seen as actual fact; the subject as in-effect-but-not-in-fact. I accept this as a rhetorical weakness of the text, however, as Morse is clear to articulate that, within the complex realm of subjectivity formation, notions of original have little significance. 7 The concept of presence is fundamental to current discussions of virtual environment design. Among the important contributions are the following: David Zeltzer,' Autonomy, Interaction, and Presence' Presence, 1 no. 1 (1992), pp 127-132; Woodrow Barfield, David Zeltzer, T. Sheridan, and Mel Slater, 'Presence and Performance Within Virtual Environments' in Virtual Environments and Advanced Interface Design, eds. Woodrow Barfield and Thomas A. Furness (New York: Oxford University Press, 1995); David Schloerb. 'A Quantitative Measure of Telepresence', Presence, 4, no.1 (1995), pp. 64-80; Rudolph P. Darken, Terry Allard, and Lisa B. Achille, 'Spatial Orientation and Wayfinding in Large-Scale Virtual Spaces: An Introduction', Presence, 7 no. 2 (1998), pp. 101-107. 8 N. Katherine Hayles, How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics (op. cit.). Alluquere Stone, The War of Desire and Technology at the Close of the Mechanical Age (Cambridge: MIT Press, 1999). 9 My collaborators on this project are: Kathleen Brandt, independent media artist; Karel Dudesek, media artist, VanGoghTV; Richard Radke, Assistant Professor of Electrical and Computer Systems Engineering, Rensselaer; Rendell Torres, Assistant Professor of Architectural Acoustics, Rensselaer; and Yasushi Shimizu, Research Professor of Architectural Acoustics, Rensselaer. 10 Synthetic reality is defined as a general term by the National Research Council which includes virtual reality, augmented reality, and telepresence environments. I prefer the generic nature of this term and use it perhaps most importantly in our research to avoid problematic associations with the adjective 'virtual' (National Research Council, Virtual Reality: Scientific and Technological Challenge [Washington D.C.: National Academy Press, 1995]). 11 Brian Massumi, 'Line Parable for the Virtual (On the Superiority of the Analog)' in The Virtual Dimension: Architecture, Representation, and Crash Culture, ed. John Beckman (New York: Princeton Architectural Press, 1998), p. 313. 12 Ibid, p. 309. 13 Ibid, p. 305. 14 Two conversations are taking place, one between P1 and P1' and another between P2 and P2'. Lines of sight and cones of vision are indicated for each of these conversations. By manipulating the video signals from cam_A and cam_B, we are able to simulate the view from a virtual camera in any position in between. In order to facilitate eye contact for multiple people, we position a virtual video camera at each person's point of view (Vcam_P1 and Vcam_P2 are shown for the local site). We use cam_A view and cam_B view to calculate Vcam_P1 view and Vcam_P2 view. The image that is displayed on the projection screen in the local site (shown above in translucent grey), is a morph between these two resultant views. 1 From the online Mirriam-Webster Collegiate Dictionary, virtual is defined as 'being such in essence or effect though not formally recognized or admitted' (Mirriam-Webster's Collegiate Dictionary 2000. 15 April, 2000) The online Cambridge Dictionary of American English defines it to be 'almost, but not exactly or in every way' (Cambridge Dictionary of American English 2000. 15 April 2000) And finally, a 1985 American Heritage defines it to be 'existing or resulting in essence or effect, though not in actual fact, form, or name' (American Heritage Dictionary. Boston: Houghton Mifflin Company, 2000). 2 Henri Lefebvre, The Production of Space, trans., Donald Nicholson-Smith (Oxford: Blackwell, 1974). Michel Foucault, The Order of Things (New York: Vintage Books, 1970). Manuel Castells, The Informational City: information technology, economic restructuring, and the urban-regional process, Oxford: Blackwell, 1989). Edward Soja, Postmodern Geographies: the reassertion of space in critical social theory (London: Verso, 1989). Rosalyn Deutsche, Evictions: Art and Spatial Politics (Cambridge: MIT Press, 1986). Mary McLeod, 'Henri Lefebvre's Critique of Everyday Life: An Introduction,' in Architecture of the Everyday, eds. Steven Harris and Deborah Berke (New York, Princeton Architectural Press, 1987). Elizabeth Wilson, The Spinx in The City: Urban Life, the Control of Disorder, and Women (Berkeley, University of California Press, 1991). 3 Arjun Appadurai, in Modernity at Large: Cultural Dimensions of Globalization (Minneapolis: University of Minnesota, 1996), articulates a theory of cultural differences and interactions through a notion of scapes (eg financescape, mediascape). For Appadurai, this framework helps spatialise these phenomena without grounding them in a particular instance of space (national boundaries, neighbourhoods, etc.). 4 Margaret Morse Virtualities: Television, Media Art, and Cyberculture (Bloomington: Indiana University Press, 1998), p. 4. I use Margaret Morse's explicit concept of virtuality because of its clarity and relevance to my arguments herein. However, it is important to note that this notion, whether in name or not, has formed the basis of many writings on the virtual realm. In particular, I would like to acknowledge Michael Heim's The Metaphysics of Cyberspace (New York: Oxford University Press, 1993), Avital Ronell's The Telephone Book (Lincoln: University of Nebraska Press, 1989), and N. Katherine's Hayle's How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics (Chicago: University of Chicago Press, 1999) for influencing my work on the subject. 5 Morse, p. 7. 6 A possible weakness of her suggestion is that it privileges the self - the self before the transmission-imposed you or the interactivity-imposed I - as a self without subjectivity, where the self is somehow seen as actual fact; the subject as in-effect-but-not-in-fact. I accept this as a rhetorical weakness of the text, however, as Morse is clear to articulate that, within the complex realm of subjectivity formation, notions of original have little significance. 7 The concept of presence is fundamental to current discussions of virtual environment design. Among the important contributions are the following: David Zeltzer,' Autonomy, Interaction, and Presence' Presence, 1 no. 1 (1992), pp 127-132; Woodrow Barfield, David Zeltzer, T. Sheridan, and Mel Slater, 'Presence and Performance Within Virtual Environments' in Virtual Environments and Advanced Interface Design, eds. Woodrow Barfield and Thomas A. Furness (New York: Oxford University Press, 1995); David Schloerb. 'A Quantitative Measure of Telepresence', Presence, 4, no.1 (1995), pp. 64-80; Rudolph P. Darken, Terry Allard, and Lisa B. Achille, 'Spatial Orientation and Wayfinding in Large-Scale Virtual Spaces: An Introduction', Presence, 7 no. 2 (1998), pp. 101-107. 8 N. Katherine Hayles, How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics (op. cit.). Alluquere Stone, The War of Desire and Technology at the Close of the Mechanical Age (Cambridge: MIT Press, 1999). 9 My collaborators on this project are: Kathleen Brandt, independent media artist; Karel Dudesek, media artist, VanGoghTV; Richard Radke, Assistant Professor of Electrical and Computer Systems Engineering, Rensselaer; Rendell Torres, Assistant Professor of Architectural Acoustics, Rensselaer; and Yasushi Shimizu, Research Professor of Architectural Acoustics, Rensselaer. 10 Synthetic reality is defined as a general term by the National Research Council which includes virtual reality, augmented reality, and telepresence environments. I prefer the generic nature of this term and use it perhaps most importantly in our research to avoid problematic associations with the adjective 'virtual' (National Research Council, Virtual Reality: Scientific and Technological Challenge [Washington D.C.: National Academy Press, 1995]). 11 Brian Massumi, 'Line Parable for the Virtual (On the Superiority of the Analog)' in The Virtual Dimension: Architecture, Representation, and Crash Culture, ed. John Beckman (New York: Princeton Architectural Press, 1998), p. 313. 12 Ibid, p. 309. 13 Ibid, p. 305. 14 Two conversations are taking place, one between P1 and P1' and another between P2 and P2'. Lines of sight and cones of vision are indicated for each of these conversations. By manipulating the video signals from cam_A and cam_B, we are able to simulate the view from a virtual camera in any position in between. In order to facilitate eye contact for multiple people, we position a virtual video camera at each person's point of view (Vcam_P1 and Vcam_P2 are shown for the local site). We use cam_A view and cam_B view to calculate Vcam_P1 view and Vcam_P2 view. The image that is displayed on the projection screen in the local site (shown above in translucent grey), is a morph between these two resultant views.
PY - 2002
Y1 - 2002
N2 - Recently availed modes of mediated communication and perception, generally categorised as 'virtual experiences', have offered the designer of space a new array of spatial conditions to contend with. Each variety of virtual experience, from text-based discussion forum to immersive virtual reality environment presents challenges to traditional assumptions about space and its inhabitation. These challenges require design theorisation which extends beyond the notions of design within the virtual, or projected, space (the layout of the chat forum, the appearance of the virtual realm etc), and require a reconsideration of the entire apparatus of the mediating devices (the physical spaces which facilitate the interaction, the connection between them and the virtual space etc). This paper presents both an experimental framework for understanding the space of the virtual and outlines a current research project addressing these theoretical challenges through the spatial implementation of a synthetic environment.
AB - Recently availed modes of mediated communication and perception, generally categorised as 'virtual experiences', have offered the designer of space a new array of spatial conditions to contend with. Each variety of virtual experience, from text-based discussion forum to immersive virtual reality environment presents challenges to traditional assumptions about space and its inhabitation. These challenges require design theorisation which extends beyond the notions of design within the virtual, or projected, space (the layout of the chat forum, the appearance of the virtual realm etc), and require a reconsideration of the entire apparatus of the mediating devices (the physical spaces which facilitate the interaction, the connection between them and the virtual space etc). This paper presents both an experimental framework for understanding the space of the virtual and outlines a current research project addressing these theoretical challenges through the spatial implementation of a synthetic environment.
UR - http://www.scopus.com/inward/record.url?scp=30744461970&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=30744461970&partnerID=8YFLogxK
U2 - 10.1177/135485650200800306
DO - 10.1177/135485650200800306
M3 - Article
AN - SCOPUS:30744461970
SN - 1354-8565
VL - 8
SP - 61
EP - 77
JO - Convergence
JF - Convergence
IS - 3
ER -