interdisciplinary research studio-lab

Interactive, Real Time Acoustic Scenography for Live Stage Environments

Project Description:

The proposed project focuses on the research and creation of interactive, dynamically changing auditory “scenography” for the stage. Movement and gesture data captured by a network of wireless sensors located on both the bodies of performers as well as within objects and material within a stage environment will be used to synthesize and sculpt audio in real time, creating the possibility of a continually evolving acoustic scenic design. While the concept of scenography in theatrical performance contexts has traditionally been associated with visual-based design and construction, we will develop both technologies and new dramaturgical scenarios deploying such technologies that will enable the creation of entirely interactive sonic stage environments.

Technical research can be broken down into four principle components: (1) Augmentation of performers, objects and physical architecture with wireless, remote sensors to enable real time capture of continuous data streams generated by interaction of performers with the stage environment, (2) exploration of unique forms of multi-modal sensor input such as sound, touch, breath, acceleration, motion and spatial proximity that can be used to control sonic output, (3) development of software-based audio synthesis instruments that parameterize continuous gesture and movement-based sensor data and render it into audio and (4) novel exploitation and combination of existing sound diffusion technologies (multi-channel speaker arrays, parabolic and ultrasonic beam-based loudspeakers, infrasonic (ultra low) frequency projection techniques, acoustic actuators )to explore new techniques for sound projection in theatrical stage space.

Creative application and dissemination of the research will take three principle forms: (1) the development of a completely responsive acoustic stage environment for an original media performance work entitled Schwelle, (2) the building of a local, national and international network of student and professional collaborators through workshops held in Montreal with partners in the academic and professional artistic community and (3) the dissemination of technical/ creative research through participation in conferences and publications.

The first application of the research will be towards the development of an original media theater work entitled Schwelle. Schwelle (German for threshold or transitory state) is a three part technology-augmented performance project whose narrative structure follows three of the six bardo states (Tibetan for transition or threshold) described in the Tibetan Book of the Dead: the moment of death (Part I), the forty-nine day transition between death and rebirth (Part II) and the moment of rebirth (Part III). Application results of the research will feed direct presentations of the Schwelle project over the next three years within a network of high profile international art and technology/media performance venues and festivals.

Artistic results of research:

Schwelle, Part II

Schwelle II is a 45 minute dance-theater work. As the second part of a three act media performance project, the piece is a solo work for one dancer performed in a 6 x 13 meter long space with audience seating on both sides. The performance explores what occurs to the body in the transition period after death. A room filled with light sensors continually modulates an ever changing, spatialized acoustic and lighting environment as the performer watches his own body shift from one state to another.


Artistic Performances:

Schwelle, 2, EXIT Festival, Creteil, France, March 2008

Schwelle 2, Elektra Festival and Cinquieme Salle, Place des Arts, Montréal, Canada, May 2007

Schwelle 1 and 2, Premiere, Tesla and Transmediale, February 2007, Berlin, Germany


“Dramaturgies of Sound: Interactive Sound Design in Live Performance,” Proceedings for the 1st International Workshop in Sonic Interaction Design, CHI, Florence, April 2008

“Between Mapping, Sonification and Composition: Responsive Audio Environments in Live Performance,” Proceedings for the 2007 International Computer Music Conference (ICMC), Copenhagen, September 2007

“Schwelle: Sensor Augmented, Adaptive Sound Design for Live Theatrical Performance,” Proceedings for the 7th Annual Conference in New Interfaces for Musical Expression (NIME 2007), New York, NY, April 2007


Live Sound Design and Real Time Performance. Theater et Son research group. CRI. Monument Nationale, Montreal, March 2009.

“Dramaturgies of Sound: Interactive Sound Design in Live Performance-Two Case Studies,” Sound in Interaction Design, CHI 2008 Workshop, April 2008

“Schwelle and Responsive Performance Environments,” Attakkalari Center for Movement Arts, Bangalore, India, February 2008

“Sensor driven Responsive Audio Environments,” Hexagram research presentation at Cirque du Soleil Headquarters, Montreal, October 2007

“Between Mapping, Sonification and Composition: Responsive Audio Environments in Live Performance,” Proceedings for the 2007 International Computer Music Conference (ICMC), Copenhagen, September 2007

Schwelle, Roundtable Discussion, Place des Arts, May 2007

“Schwelle: Sensor Augmented, Adaptive Sound Design for Live Theatrical Performance,” NIME 2007, New York, NY, April 2007

Funder: FQRSC (Le Fonds québécois de la recherche sur la société et la culture)

Program: Établissement de nouveaux professeurs-chercheurs-créateurs

Dates: April 2006-April 2009

Collaborators: Chris Salter (PI), Marije Baalman, Philip Viel, Daniel Grigsby, Daniel Wessolek, Michael Schumacher, Brett Bergmann, Heidi Gilpin, Thomas Beguin, Thomas Spier

Status: Finished