barcode
interdisciplinary research studio-lab

EMERGE: Artistic Applications Using Wireless Sensor Networks

Project Description:

E[MERGE]is a collaborative research/creation project between labs at Concordia and McGill Universities to develop new software tools for artists to better understand and compose with large amounts of real time data generated by many sensing and computation devices spread throughout a physical environment. The innovative artistic project N-Polytope that relies on such distributed, wireless multi-channel real time sensor data is at the forefront of this research. N-Polytope is a real time composition for sound, light and architecture by Chris Salter in collaboration with Netherlands-based artist/researcher Dr. Marije Baalman inspired by Iannis Xenakis’ Polytope de Montreal composition from the 1967 Montreal Expo.

The complexity of dealing with hundreds of channels of real time sensor data in this creative work requires software tools that allow artists to make sense of this data and compose with it in an intuitive and aesthetically compelling way. Our research thus seeks to address this issue by developing a unified open source software framework that will:

(1). Enable the connection, processing, merging and organization of many streams of sensor data in real time (sensor fusion).
(2). Detect correlations and behaviours across many streams of data (real time sensor data analysis).
(3). Facilitate rapid prototyping/creation of complex behaviours for media that are directly driven by this real time sensor data (sensor data modelling).

Artistic Results of Research:

N-Polytope: Behaviors in Light and Sound After Iannis Xenakis

N-Polytope is a light and sound performance-installation combining lighting, lasers, sound, sensing and machine learning software inspired by composer Iannis Xenakis’s Polytopes works from the 1960s and 70s: large scale, immersive architectural environments that made the indeterminate and chaotic patterns and behaviour of natural phenomena experiential through the temporal dynamics of light and the spatial dynamics of sound. The Polytopes still to this day are relatively unknown, but were far ahead of their time.

N-Polytope is an attempt to both re-imagine Xenakis’ work with probabilistic/stochastic systems with new techniques as well as to explore how these techniques can exemplify our own historical moment of extreme instability.

The re-imagining runs in both 15 minute performance and continuously evolving installation modes. The system is steered through a custom developed wireless sensor network utilizing machine learning techniques, specifically from the research area of reinforcement learning in which software agents interact with their environment in order to achieve a goal. The agent seeks to achieve its goal despite the fact that there is a high degree of uncertainty about the environment – in other words, the agent doesn’t know until it does something and is then “rewarded” in either a positive or negative manner. In N-Polytope, agents receive sensor-actuator information from the environment (the brightness of an LED and the frequency/amplitude of a sound) and can either turn the LED or the sound on or off, receiving a reward for it. However, the environment around the agent (and the sensor) is continually changing, which makes it challenging to determine what steps the agent will take and what such actions will result in. The agent’s actions thus influence not only the state of the environment in the present but also can affect the environment’s state in the future.

The overall course of N-Polytope is that of an event that is partially scripted and partially indeterminate thus enabling the performance to continually move between order and disorder, tranquility and thundering chaos. The audience “feels” these algorithms as two forms of “life” – the behaviours in light and sound and the “life” of experiencing such a system evolving in the present.

N-Polytope was awarded an honourable mention in the 14th VIDA Art and Artificial Life Award from Fundacion Telefonica.

Documentation:







Location:

“Open Codes.” ZKM, Karlsruhe, Germany, February-August 2018

“In Search of Expo ‘67” Musée d’art Contemporain, Montreal, Canada, June-October 2017

Halle Pujol/Jardin Rosa Luxembourg, Nuit Blanche, Paris, France, October 2015

BIAN 2014, Darling Foundry, Montreal Canada, May 2014

Stadtbad Wedding, CTM, Berlin, Germany, February 2014

Buckminster Fuller Dome/Vitra Design Museum, Weil am Rhein, Germany, January 2014

LABoral Centro de Arte y Creación Industrial Gijón, Spain, July-August 2012

Publications:

Salter, Chris and Sofian Audry. 2017. “Towards Probabalistic Worldmaking: Xenakis, N-Polytope, and the Cybernetic Path to Chaos” Worldmaking as Techne: Exploring Worlds of Participatory Art, Architecture, and Music. Eds. Alberto de Campo, Mark David Hosale and Sana Murrani. Riverside Architectural Press. 25 pages.

Salter, Chris. 2014. “Iannis Xenakis, the Polytopes and Musics of Otherness.” Discontinuity: Select Trajectories in Experimental and Electronic Music. CTM – Festival for Adventurous Music and Art. 15th Edition Anniversary Catalog. Berlin, Germany. Februrary 2014.

Salter, Chris. 2013. “N_Polytope: Behaviors in Light and Sound after Iannis Xenakis.” Xenakis Matters. Ed. Sharon Kanach. New York: Pendragon Press.

PRESENTATIONS:

Artist Talk. Musée d’art Contemporain. Montreal, QC. September 2017.

Artist Presentation. International Digital Arts Market. Romaeuropa Festival. Rome, Italy. October 2015.

Artist Presentation. Quebec Digital Art @ NYC. Made in New York Media Centre. New York, NY. October 2015.

“N-Polytope.” Roundtable. Fondarie Darling. Montreal, QC. May 2014.

“Xenakis, Polytopes and the Actions of Perception.” Vitra Design Museum. Weil am Rhein, Germany. January 2014.

Visiting Artist Talk. Holst Centre. High Tech Campus. Eindhoven, Netherlands. August 2013.

N-Polytope. Book Launch – Xenakis Matters. CIRMMT. Mc Gill University, Montreal, Canada, January 2013.

“Scenography in the 21st Century.” Keynote Address. “Stage Digital.” Department of Performing Arts and Scenography. ZhDK (Zurich University of the Arts). Zürich, Switzerland. November 2012.

Artist Presentation. LABoral. Gijon, Spain. July 2012.
PRESS

“S’inspirer des icônes: A la Recherche D’Expo ‘67” Jérôme Delgado. Le Devoir. April 22, 2017.

“Un menu engage et poétique au Musée d’art contemporain.” Eric Clement. La Presse. January 19, 2017.

“La Nuit Blanche,” caisse de resonance artistique de la COP 21. Le Monde. October 3, 2015.

“5 Sorties Architecturales Pour La Nuit Blanche.” Tema.archi. October 3, 2015.

“Chris Salter: N-Polytope” October 3, 2015. Telerama.fr.

“2nd International Digital Arts Biennale (BIAN).” The Collector Tribune: The Daily for Art Collectors. May 10, 2014.

“Must Sees: May 2014.” Canadian Art. May 1-7, 2014.

“Digital Art Invasion with the BIAN: Picks and Highlights.” Montreal Rampage. April 25, 2014.

“Architecture Installation Makes Infinite Sound Loop Based on it Own Data.” Benoit Palop. The Creators Project. May 7, 2014.

“N-Polytope” preview. Tagespiegel. Berlin. January 27, 2014.

“CTM 2014: N-Polytope Nach Iannis Xenakis.” Spex: Magazin für Popkultur. January 27, 2014.

“Licht erhellt und inspiriert Designkünstler.” Badische Zeitung. October 9, 2013.

“Luz y Sonido.” Dxi Magazine. September 2012.

Interview. N-Polytope. We Make Money Not Art. July 2012.

N_Polytope. Baunetzwoche # 280. July 2012

“Architectura para los sentidos.” La Nueva Espana. July 2012

“Lo Dijo Xenakis.” ABC Cultural (Espana). August 2012

FUNDER:

Fonds de Recherche du Québec-Sociéte et Culture, Programme d’appui aux arts et technologies médiatiques

DATES:

November 2010-November 2012

COLLABORATORS

Labxmodal (Concordia University)– Chris Salter (PI), Marije Baalman, Harry Smoak, Sofian Audry, Elio Bidinost, Daniel Wessolek, Adam Basanta, Samuel Tissot, Owen Coolidge, Garrett Lockhart, Joseph Plazak and Alexandre Saunier

IDMIL (McGill University)
Marcelo Wanderley (Co-PI – Scientific Investigator), Joseph Malloch, Steven Sinclair

Electronics Assembly: Simon Claessen, Rene Wassenburg (Schrikdraad Ontwerp) and Stan Verberkt

Production: xmodal/Montréal/LABoral

With the original support of:

Fonds de Recherche du Québec Sociéte et Culture, Hexagram Concordia, Schlaich Bergemann und Partner, STEIM (Amsterdam)

STATUS:

Completed