Sensory Response Systems

Sensory Response Systems is an exploration into realtime DIY FLOSSbased interactive performance systems. Using an array of sensors and DIY electronics responsive to physical movements in order to control the audiovisual output in Pd, CMOS technology, strobe lights and smoke, it reshapes and replicates the human body and forces the user and audience into an immersing hypnotic experience.

The performance also alters the perspective of the performer and performance space through noise, code, strobe lighting and hacked electronics. Flickering shadows of the performers jerking movements and cables attached to the body are cast onto the walls of the performance space as the performers movements control screaming and shrieking hacked electronics and computer programmes, spewing out their internal circuitry.


On the subject of the mask Wilsher states that masks are folk art, instantly connecting with the audiences imagination, crossing cultural boundaries and leaping the centuries (Wilsher, 2007). They act as an antidote to technological excess and bland realism, requiring the audience to watch as if they were children again, suspending their disbelief and adult sensibilities; believing. Masks allow us to enter a different mind state, to be taken outside of ourselves and to loose ourselves temporarily in a world of imagination. The use of the mask in System 1 is to transform. Wilsher breaks down the use of the mask into two worlds, the sacred and the secular. The sacred being a belief that the mask is magic and holds this power not only over the audience but over the wearer as well, who is transformed mentally by wearing a mask, into a state of trance. In a secular society, all the above is not true. In a secular society the understanding of the mask has become wrapped up in superstition. Wilsher goes on to say:

In a sacred society, a mask is totemic, shamanic, an object linked to deity, a liminal tool that stands at the threshold of the world they know and the world they don't. Masks in this society are unequivocal, unambiguous and never metaphoric. (Wilsher, 2007).

Masks were also worn around those with the plague.


Arduino, ADXL3xx 3axisaccelerometer (x3), push buttons (x8), LDR (Light Dependant Resistor) (x1), multiplexer (x1), Pd, Laptop, a strobe light, stretchy black fabric.

A wearable interactive performance system based on sensing discrete movements of the performer. The data generated by the movements is then used to control various parameters within a Pd patch which has been specifically created for this project.

The physical part of the system is the sensors. The accelerometers are attached to the backs of each hand and the top of the head. The push buttons are a attached to the finger tips of the right hand and each have a number, and the LDR is attached to the palm of the left hand. Wires are left to fall freely over the body and are attached to a circuit board via ten meter long cables which connect it to the Arduino. The Arduino has a multiplexer to allow for more sensors to be connected to it. The Arduino code reads the incoming sensor data and sends it via a serial port into Pd.

The Pd patch has been hacked together from various help files and other patches. The data generated from the performers movements are filtered and scaled inside the Pd patch so that the data flow is manageable and useful. It is then sent to various sound devices. The maximum parameters of the system developed are not used in the performances as they are not necessary. Simple interfaces can be used to great affect to control and create complex sound in live performance. In total, this system has ten potential analogue sensors, however only four are utilised. Future performances may require more parameters to be used. The parameters used are as follows:

● Accelerometer, left hand – Xaxis controls the procession speed and direction of an audio sample in hundredths of a second inside a Phase Vocoder.

● Accelerometer, right hand – Xaxis controls the amount an audio sample is stretched inside a Timestretch effect (from 1 to +1).

● Accelerometer, head – Xaxis controls the flash rate on an external strobe light and a phasor~ object inside of the Timestretch effect. Yaxis controls a phasor~ object inside of the Timestretch effect.

● Push button 1 – triggers an audio sample in the Phase Vocoder to play. The button is often triggered numerous times during a performance to build up rhythms and tensions.

● Push button 2 – triggers a pitch shift of the audio sample in the Phase Vocoder and turns on a delay effect.

● Push button 3 – triggers a distortion effect and a reverberation effect to turn on.

● Push button 4 – triggers the the Timestretch effect to turn on.

● Push button 5 – Turns all the sound off.

The very nature of developing this system was performative as you can see from the way the sensors and buttons are used. As Richards (2008) details when he is talking about dirty electronics, “process and performance in dirty electronics are inseparably bound. The performance begins in the search for found objects and on the workbench, where sound generating devices are conceived”. The system also provides a score for the performance (e.g. button 1 = begin, button 2 = development, button 3 = middle bit, button 4 = recapitulation, button 5 = stop). The idea of having the system as a score is also supported from the previous works of people such as David Tudor who's home built electronics would serve as the score for the piece.

There is nothing ground breaking or unique in the technology used in System 1 and it is not very advanced in terms of computer programming or electronics, however, what is of interest and uniqueness here is how it is used. A simple system can give a complex and aesthetically powerful output.

The photograph is taken from the second ever performance of Sensory Response Systems at Piksel 08 in Bergen, Norway. The strong aesthetic and symbolic significance of the shadow, mask, noise and stroboscope used in the performances is of the most importance, not the technical expertise and sophistication of the programming behind it.

sensory_response_systems.txt · Last modified: 2012/09/03 10:53 by r
Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki