Project Description

IRHA is a non-profit research network comprised of the CCA, McGill University, University of Montréal, and Concordia University to investigate issues related to the history and theory of architecture, design and urban practices. For the year 2009-2010, IRHA will focus on the theme of the Ephemeral City – how contemporary urban space is increasingly shaped by new dynamic and temporary forces from economics to design and new technologies.

The modernist city that was formerly dictated and constructed chiefly by architecture and planning models is increasingly being confronted daily by temporal forces: the dynamics of unstable financial markets and fluctuating economic patterns of consumption and leisure, the rise of ecological processes and practices, the transformation of public space by the methods of branding, interaction and multi-sensory design and last, but certainly not least, the dissemination of new ubiquitous technologies of surveillance and monitoring that are rapidly revising our concepts of urban construction, fabrication, orientation and experience.

The Ephemeral City thus aims to grapple with issues in the urban context of Montréal related to temporal, performative phenomena that go beyond programs, plans, models and other static representations of the urban environment. Research proposed will focus on broader concepts such as performativity in urban space, the role of transience, liminality and improvisation in the city and its citizenry, the different interpolations of ecology and sustainability, urban experience economies generated by marketing in collaboration with design practices perception and sensation/sentience that goes beyond visual perception and co-structuration between inhabitants and the urban environment as a result of new technologies.

The IRHA project for 2009-2010 will be comprised of the following activities:

(1) A series of public forums to be held one Thursday each month at the CCA, each focused on one of the sub-themes under the ephemeral city umbrella These public forums aim to bring together faculty researchers, graduate students and architects/designers/artists from the four universities as well as the greater Montréal artistic and design community and the general public. The themes and forum dates will be announced on this website as well as the CCA site and other appropriate online dissemination contexts.

(2) A concluding discussion/symposium in Fall 2010 with past participants and national/international guests will mark the end of the IRHA activities for 2009-2010. This symposium will also be hosted at the CCA and take a broader look at the repercussions of the overall theme on the question of practice. The symposium will involve a mix of panels and round tables as well as a potential “poster session” like demo of projects and concepts developed over the course of the year.

(3) An online website for both public dissemination of IRHA activities and a networking tool for the broader Montréal research and artistic/architectural/design communities.

(4) A series of seven commissioned essays highlighting the seven core IRHA themes. These online essays will complement the monthly forums at the CCA by going into further depth on the pertinent topic. An open call for proposals will go out at the end of September to graduate students in the Montréal architecture, design and new media communities. The essays can be either in English or French.

(5). A series of small commissioned prototypes, “proof of concepts,” models, sketches or concepts that highlight specific thematics during the year. These works will be featured in the context of the IRHA forums at the CCA and/or shown on the project website. An open call to graduate students in the Montréal architecture, design and new media communities will go out end of September.

Chris Salter is an artist based in Montréal, Canada and Berlin, Germany. His artistic and research interests revolve around the development and production of real time, computationally-augmented responsive performance environments fusing space, sound, image, architectural material and sensor-based technologies. Such projects range from small and large scale, public driven installations where the line between spectators and performers is blurred to traditional performance environments with trained performers that are augmented with computational and media systems.

Salter studied economics and philosophy at Emory University and received his Ph.D. in the areas of theater and computer-generated sound at Stanford University where he worked with former Brecht assistant Carl Weber as well as pioneers of digital synthesis John Chowning, Max Matthews and Chris Chafe at the Center for Research in Computer Music and Acoustics (CCRMA). He was awarded the Fulbright and Alexander von Humboldt Chancellor grants for research/work in Germany from 1993-1995.

After collaborating with Peter Sellars and William Forsythe/Ballett Frankfurt, he co-founded the art and research organization Sponge, whose works have stretched between the arenas of performance, installation, scientific research and publications and have toured internationally to festivals, exhibitions and venues. His work with Sponge as well as solo projects has been seen in such venues as the Venice Architecture Biennale (Venice), Ars Electronica (Linz), Villette Numerique (Paris), Transmediale (Berlin), EXIT Festival (Maison des Arts, Creteil-Paris), Place des Arts (Montréal), Venice Biennale, Elektra (Montréal), Shanghai Dance Festival (Shanghai), Yerba Buena Center for the Arts (San Francisco), the Banff Center (Banff), Dance Theater Workshop (New York), V2 (Rotterdam), SIGGRAPH 2001 (New Orleans), Mediaterra (Athens) and the Exploratorium (San Francisco), among others.

Salter’s projects have been written about most recently in the New York Times, ID Magazine and Leonardo magazine and received grants from the Rockefeller Foundation, the Daniel Langlois Foundation, the Creative Work Fund/San Francisco, the Social Sciences and Humanities Research Council of Canada, FQRSC and the LEF Foundation, among others. He has given invited talks at such venues as the Banff Center for the Arts, Ars Electronica, Deutsches Architektur Museum, HKW Berlin, ZKM, Interactive Telecommunications Program (ITP) at NYU, Queens College/CUNY, Deutsches Architektur Museum, Brown University, Stanford University, Goldsmiths College, Elektra festival, Amherst College, the Rhode Island School of Design, Transmediale, HfG Karlsruhe, Concordia University, Subtle Technologies-Toronto, SLS-Paris, NIME 03-Montréal, UdK-Berlin and e-Culture, Amsterdam, among many others. He has sat on numerous juries including NIME, ISEA and the Prix Ars Electronica.

Salter has published in the areas of technology and performance, real time responsive environments, mobile real time media and cultural politics. His book Entangled: Technology and the Transformation of Performance will be published by MIT Press in 2010. He has been visiting professor in music, graduate studies and digital media at Brown University and the Rhode Island School of Design (RISD) and is currently Assistant Professor in the Department of Design and Computation Arts at Concordia University in Montréal, Canada where he teaches in the areas of real time digital audio, immersive environments and critical theories of media, performativity and technology.

SENSE/STAGE will develop new wireless sensor and software tools that can be used to create innovative and compelling forms of interactive real time performance. This term encompasses live performances, from traditional theatre to new gaming environments that use sensors and computers to create dynamic and interactive scenography. Scenography is the creation of an immersive and total theatrical stage environment through scenery, lighting and sound.

Traditionally, the control of lighting, sound and mechanical scenery in live performance has been achieved by pre-programmed, cue-based software: all choices have to be planned out beforehand. Our main objective is to develop innovative tools to create a new kind of scenographic ecology where performer and stage environment mutually influence and impact each other. By understanding what is taking place in the stage environment by way of sensors, our software will be used to create interesting and surprising behaviours for light, sound and mechanical movement of scenery rather than controlling these in a predetermined way. The human performer can directly influence the system’s behaviour from his/her actions and movements. The performer’s movements or gestures can also impact the system indirectly over a longer time period causing the behaviour of the scenography to be surprising and unpredictable to performer and audience alike.

The project includes interrelated technical and creative objectives. The technical objectives include the development of new wireless sensor hardware and software tools to enable the creation of an interactive, scenographic ecology where performer and lighting, sound and mechanical scenery can mutually influence each other in unpredictable and surprising ways. The technical work in the project will be guided by the team’s extensive experience understanding the potentials and limitations of new interactive technologies used in live performance. Specifically, the objectives include three aspects:

1) Develop Wireless Sensing Devices:

Following an initial and extensive analysis of existing sensor technologies and their potential for live performance which will include collaborative experimentation sessions with team members and professional artists, we will develop an innovative wireless sensor infrastructure specifically designed for live performance conditions.

A series of small electronic circuit boards the size of large postage stamp will be developed which can be easily mounted on actors or dancers, scenery and props (see Figure 1). Each sensor processing board will be “plug and play:” directors, choreographers and scenographers will be able to easily hook up multiple sensors to the boards. These sensor boards will sense the movement, position and acceleration of an actor or dancer and relay this data between each of the boards wirelessly to “understand” and monitor what is happening around them. The custom electronics will be energy efficient: low power, able to run over extended periods of time (6-9 hours) and use rechargeable batteries.

In order to choose appropriate sensing devices for the wireless platform, we will work with professional performance artists in extensive experimentation rehearsals to explore a wide range of sensory information that can be used as input to our system. From this experimentation we will also specify a range of low cost sensing devices for the wireless boards as well as accompanying documentation with “scenarios for use.”

2) Develop Interactive System Software:

The second tool is open source/cross platform software that can intelligently control the behaviour of lighting, audio and physical scenery in live performance. To create such interaction, the software will implement current scientific research in dynamical systems, i.e. systems that change continuously over time, depending on the current input, past input and the internal state of the system. The sensors will be used to understand what is happening on stage. This sensor information will then be processed by the software and “fed” to lighting, sound and other devices to create potentially surprising behaviours. Interaction will take place at the level of performer and scenographic environment, with the software facilitating both direct and indirect interaction. Direct interaction will allow performers to play the stage environment like a musical instrument, while indirect interaction will allow the scenographic environment to evolve and behave on its own.

Artistic Results of Research:

Chronotopia – dance-theater work in collaboration with Attakkalari Centre for Movement Arts

Bangalore, India

Interactive Scenography

Chronotopia is a collaboration with the Bangalore, India-based movement arts center Attakkalari. A specially designed, wirelessly controlled lighting system utilizing technologies developed on the Sense Stage project is influenced by the rhythms and motion of the dancers, creating an abstract, pulsing visual stage environment for Attakkalari’s choreographic re-imagining of the ancient Tamil epic poem Silappatikaram (The Tale of the Anklet).

A series of 9 meter tall columns holds a series of 6 Cold Cathode Florescent Lights encased in acrylic cylinders which form a matrix of 36 individually controllable lighting elements. Additionally, three wirelessly controlled CCFL’s are mounted into pedestals which are carried around the stage by the performers and function as sacred objects in the work’s dramaturgical structure.

The stage scenographic environment consists of the lighting system in front of which is suspended a large scale, semi translucent projection screen upon which is projected a series of five iconic yet, visually abstract textures. Top and side cameras analyze the flow of motion from the dancers and are used to provide simultaneous input to both the lighting and projection systems. The lighting responds to the performers’ motion by either triggering set patterns, directly coupling with the dancers or creating abstract traces based on different rhythmic and temporal patterns.

The lighting scenography is inspired by both the abstract images of bodily gestures that are prevalent in traditional Indian dance forms such as Bharatnatyam as well as by the ubiquitous appearance of florescent lighting fixtures in everyday life in India.

Documentation:

Location:

Attakkalari India Biennial 2009, Ranga Shankara Theater, Bangalore, India, February 2009

Music Academy, Chennai, India, February 2009

Niraswaram, Heggodu, India, February 2009

European venues in 2010 in Goteborg and Frankfurt

Just Noticeable Difference (JND) – interactive sensory environment

JND is a 1.2 m wide x 2.5 m long x 3 m high portable sensory reduction environment combining near darkness with extraordinary low levels of tactile vibration, flicker-based, color changing light and sound at the thresholds of human sense perception. The installation is for one person at a time that lies down inside the space for 15 minutes on a custom developed floor. Micro-movements ranging from twitches of the muscles to larger body motions are measured in real time and affect the patterns and intensity of a composition of touch, light and sound. The installation explores the fluctuation between noise and order at the level of body perception and the resulting shifts of experience of self that might occur.

EMPAC (Experimental Media and Performing Arts Center), Renssalaer Polytechnic Institute (RPI), Troy, New York, March 2010
Other European venues TBA

Other projects utilizing Sense Stage technologies:

M.A.R.I.N.

Website: marin.cc

Arctic Perspective

Website: arcticperspective.org

Pneuma

Website: www.pneumata.net

Sense/Stage workshop

Website: sensestage.hexagram.ca/workshop

Documentation:

Publications:

“Sharing Data in Collective Interactive Performances: The SenseWorldDataNetwork:” Submitted for 9th Annual Conference in New Interfaces for Musical Expression (NIME 2009), Pittsburg, Pa, USA

Other publications in developmemt

The proposed project focuses on the research and creation of interactive, dynamically changing auditory “scenography” for the stage. Movement and gesture data captured by a network of wireless sensors located on both the bodies of performers as well as within objects and material within a stage environment will be used to synthesize and sculpt audio in real time, creating the possibility of a continually evolving acoustic scenic design. While the concept of scenography in theatrical performance contexts has traditionally been associated with visual-based design and construction, we will develop both technologies and new dramaturgical scenarios deploying such technologies that will enable the creation of entirely interactive sonic stage environments.

Technical research can be broken down into four principle components: (1) Augmentation of performers, objects and physical architecture with wireless, remote sensors to enable real time capture of continuous data streams generated by interaction of performers with the stage environment, (2) exploration of unique forms of multi-modal sensor input such as sound, touch, breath, acceleration, motion and spatial proximity that can be used to control sonic output, (3) development of software-based audio synthesis instruments that parameterize continuous gesture and movement-based sensor data and render it into audio and (4) novel exploitation and combination of existing sound diffusion technologies (multi-channel speaker arrays, parabolic and ultrasonic beam-based loudspeakers, infrasonic (ultra low) frequency projection techniques, acoustic actuators )to explore new techniques for sound projection in theatrical stage space.

Creative application and dissemination of the research will take three principle forms: (1) the development of a completely responsive acoustic stage environment for an original media performance work entitled Schwelle, (2) the building of a local, national and international network of student and professional collaborators through workshops held in Montreal with partners in the academic and professional artistic community and (3) the dissemination of technical/ creative research through participation in conferences and publications.

The first application of the research will be towards the development of an original media theater work entitled Schwelle. Schwelle (German for threshold or transitory state) is a three part technology-augmented performance project whose narrative structure follows three of the six bardo states (Tibetan for transition or threshold) described in the Tibetan Book of the Dead: the moment of death (Part I), the forty-nine day transition between death and rebirth (Part II) and the moment of rebirth (Part III). Application results of the research will feed direct presentations of the Schwelle project over the next three years within a network of high profile international art and technology/media performance venues and festivals.

Artistic results of research:

Schwelle, Part II

Schwelle II is a 45 minute dance-theater work. As the second part of a three act media performance project, the piece is a solo work for one dancer performed in a 6 x 13 meter long space with audience seating on both sides. The performance explores what occurs to the body in the transition period after death. A room filled with light sensors continually modulates an ever changing, spatialized acoustic and lighting environment as the performer watches his own body shift from one state to another.

Website: www.chrissalter.com/projects.php#Schwelle2

Artistic Performances:

Schwelle, 2, EXIT Festival, Creteil, France, March 2008

Schwelle 2, Elektra Festival and Cinquieme Salle, Place des Arts, Montréal, Canada, May 2007

Schwelle 1 and 2, Premiere, Tesla and Transmediale, February 2007, Berlin, Germany

Publications:

“Dramaturgies of Sound: Interactive Sound Design in Live Performance,” Proceedings for the 1st International Workshop in Sonic Interaction Design, CHI, Florence, April 2008

“Between Mapping, Sonification and Composition: Responsive Audio Environments in Live Performance,” Proceedings for the 2007 International Computer Music Conference (ICMC), Copenhagen, September 2007

“Schwelle: Sensor Augmented, Adaptive Sound Design for Live Theatrical Performance,” Proceedings for the 7th Annual Conference in New Interfaces for Musical Expression (NIME 2007), New York, NY, April 2007

Presentations:

Live Sound Design and Real Time Performance. Theater et Son research group. CRI. Monument Nationale, Montreal, March 2009.

“Dramaturgies of Sound: Interactive Sound Design in Live Performance-Two Case Studies,” Sound in Interaction Design, CHI 2008 Workshop, April 2008

“Schwelle and Responsive Performance Environments,” Attakkalari Center for Movement Arts, Bangalore, India, February 2008

“Sensor driven Responsive Audio Environments,” Hexagram research presentation at Cirque du Soleil Headquarters, Montreal, October 2007

“Between Mapping, Sonification and Composition: Responsive Audio Environments in Live Performance,” Proceedings for the 2007 International Computer Music Conference (ICMC), Copenhagen, September 2007

Schwelle, Roundtable Discussion, Place des Arts, May 2007

“Schwelle: Sensor Augmented, Adaptive Sound Design for Live Theatrical Performance,” NIME 2007, New York, NY, April 2007

Marije Baalman studied Applied Physics at the Technical University in Delft and graduated in February 2002 on the topic of Perceptual Acoustics. In 2001/2002 she followed the Sonology Course at the Royal Conservatory in The Hague. She completed her Ph.D. on Wave Field Synthesis and electro-acoustic music in 2007 at the Electronic Studio of the Technical University of Berlin. Currently she is a post-doctoral researcher in Computation Arts at Concordia University in Montreal. Her research goes into the use of wireless networks for live performance (such as dance and music), and installations. She has performed and exhibited work across Europe (STEIM, WORM (NL), EXIT festival (F), Club Transmediale (D)) and beyond (Electrofringe (AU)). She is a contributor to “The SuperCollider Book” (MIT Press, coming up in 2010).

PAPYRUS is a collaborative research project to develop home made sensors using readily available industrial materials such as paper, pigments and conductive inks. The research aims to show that home made solutions are efficient and robust alternatives to industrial sensors and that, by using such solutions, one can produce interfaces not possible from existing, off the shelf industrial sensors and significantly reduce development cost. Based on these objectives, the research will: (1) develop these new sensing technologies, (2) support the sensor research needs of a new dance-theater project entitled City of Paper from the United States/China, (3) examine and document with the artists the effectiveness of the sensors under the real world conditions of live performance, and (4) provide future researchers and artists with information on the development of such sensors through a tool kit and a dedicated project dissemination website.

PAPYRUS brings together new technologies in sensing, materials and electronics with one of the oldest materials known to man: paper. Unlike industrial sensors, paper as a material is recyclable, flexible and almost infinitely variable in terms of shaping, folding, cutting, fastening and other methods. The manufacturing process utilizes ecologically more sustainable techniques through the use of water and natural fibers. Furthermore, off the shelf industrial sensors such as FSR’s (force sensitive resistors) which are used to sense multiple points of pressure, are expensive, have pre-defined shapes which make them difficult to scale in size for unusual artistic applications (like sensing oddly shaped objects) and have pre-defined electrical characteristics which again makes sensing over large surfaces difficult. The logistical challenge involved in the use and repair of industrial sensors in non-Western or remote contexts/locations where one does not have immediate access to electronics suppliers or stores also suggests a strong motivation for the home made creation of sensors using materials like paper which are ready at hand and can be produced without expensive fabrication equipment.

In artistic collaboration with co-PI Christopher Salter, New York based, Chinese born choreographer Yin Mei will research a new dance theater work entitled City of Paper: a duet between the choreographer and former William Forsythe dancer Sang Jijia, a star performer in contemporary Chinese dance. Throughout City of Paper, the elements of ink (which the dancer’s soak themselves in), video, light, sound and paper and architecture interact and converge with each other, resulting in a form of “danced calligraphy” between performers and environment.

In order to explore paper as a new, sensate scenographic element, the project demands low cost, robust, scalable, easily produced and repaired sensors that can measure such factors as pressure, bend and position of touch across the large paper constructed surfaces (floor elements, sculptural forms, window shade-like “pull down” screens) with accurate resolution within a few millimeters. Thus, the research will draw on and extend three different but complementary sources of previous work: (1) development of paper and traction sensors at the IDMIL, McGill University, (2) development of embedded sensing and electronics inside paper pulp begun at Concordia University and now being continued under the Pulp-based Computing initiative at the MIT Media Lab’s Ambient Intelligence research group and (3) various artistic explorations with active paper sensing and actuation technologies arising from the 2006 Hexagram project Excitable Sites with Joey Berzowska, Christopher Salter, Barbara Layne and Sha Xin Wei at Concordia University. In addition, the project will uniquely extend new and ongoing work between the PI’s funded by a SSHRC research/creation grant to develop wireless distributed sensor networks for live stage performance contexts, opening up the possibility of making the proposed paper sensors wireless.

Artistic Results of the Research:

City of Paper Workshops

Hexagram, Montréal (August 2008)

City of Paper Workshops

Asia Society, New York, City (October 2008)

Excitable Sites is a collaborative research project to develop interactive textiles that enable and augment dynamic performance environments: textiles that occupy and inhabit a space (or form it) rather than just decorate it. We will produce four textile-based artifacts for interactive performance, which explore the body as a mobile entity in constant communicative interaction with its environment. The prototypes will explore the following categories: (a) architectural textiles for sensing (woven fiber substrates that allow simple gesture recognition), (b) body-worn textiles for gestural input (sensor-enabled wearable artifacts), (c) architectural textiles for data visualization, and (d) body-worn display textiles. These four pieces will be developed through a series of workshops involving an iterative and collaborative design process.

Research Specifics included:

Experimentation with sensing/physical animation of materials: We have examined low cost, minimal technology-based solutions to physically animate and actuate materials. Based on the workshop led by co-PI Barbara Layne in September 2005, we studied more in depth the process of papermaking, both at small and large-scale sheet capacity. Having learned the basic process, experimentation is beginning with embedding sensing and small actuators into the paper substrates, to see what kinds of physical movement can be obtained from the actuators. We are currently looking at using conductive ink to power the actuators and sensors in order to avoid the problem of fragile wiring inside the paper itself. It is hoped that the actuation will be of a sufficient degree to be seen by a seated audience at a distance within a performance context that the size of the actuated paper can be enlarged. It is also desired that multiple copies of the actuated paper can be put together so that a larger actuated surface could be constructed that would give the effect of moving textures that can be either floor or table top mounted. Additionally, we plan to introduce sensing into the paper substrate that can monitor the amount of actuator vibration and thus provide sensor data that can be synthesized and sonified in the audio domain.

Audio synthesis for active materials: The third stream of research being undertaken is the audio synthesis and sonification of sensor data that will be derived from the active materials research. As part of this, Chris Salter has hired two RA’s (Philip Viel and Daniel Grigsby) who are experienced in audio production and programming to begin work on developing real time synthesis models for synthesizing sensor data by way of audio. Work in this area encompasses two directions: (1) exploration of HFC and LFC content (high and low frequency) content that will be used for auditorily augmenting responsive/active materials and (2) real time, direct sound synthesis of body-based sensor data (breath and acceleration) and room-based sensor data (from active materials positioned in the room). Work is commencing in developing synthesis and signal processing “modules” in the high level, open source real time software synthesis environment Supercollider for direct (raw) data coming from the active materials. It is planned that this work will be combined with the work of the other PI’s to handle the auditory segment of the Excitable Sites project in early 2006.

Documentation: