adriana  sá
trans-disciplinary   music


INSTRUMENTS audio-visual instrument # 2 sounding light instrument bodiless instrument
audio-visual instrument # 3 audio-visual instrument # 1 sounding [video]light instrument light percussion instrument

AG#3 is a recent version of the 3D software from my audio-visual instrument. Like the previous versions, it operates based on the audio input of a zither with aged strings, a personal tuning system, and specific playing techniques. Similarly to AG#2 (see bellow), it places a sound emitter in the digital 3D world everytime it detects audio input. The emitted sounds are pre-recorded, and spatialized depending on their position relatively to the screen. The difference between the detected pitch from the zither and the closest tone/ half tone is applied to the processing of digital sound and image, and to the audio-visual mapping.

For AG#3, John Klima adapted the code a 3D engine called Irlicht, and the FMOD library. The digital sound and image became very different, as I created a new digital 3D world, new images, and new sounds/ mappings/ musical forms. AG#3.1 draws from the urban environment, both sonically and grapphically. AG#3.2 the audio draws from the countryside and the images applied to the 3D world result from graphic manipulations of my scores.

Screen capture from AG#3.9
Screen capture from AG#3.0

Related research

AG#2 is the second 3D software version from my audio-visual instrument. Like AG#1, it operates based on amplitude and pitch detection from my zither input. Accordingly, it creates elevations in the terrain of a 3D-world, and it moves the virtual camera to change the view over its deserted landscape.

The difference between the detected pitch from the zither and the closest tone/ half tone is applied to the processing of digital sound and image, as well as to the audio-visual mapping. The digital sound material consists of pre-recorded sounds of nature, string instrument sounds and synthesised sounds.

AG#1 is the first version of 3D software made for the specific input of my custom zither - the first digital component an audio-visual instrument that outputs acoustic sound, digital sound and digital image. AG#1 was developed along with the WINDOWMATTER project series. It is a modification of software originally written by John Klima. I modified digital functions, configurations, mappings, audio, and 3D world design. My modifications circumvented assumptions embedded in videogame technologies, so as to convey my musical perspective.

AG#1 articulated audio processing (FMOD audio library) and 3D-technologies (Torque videogame engine). It operated based on amplitude and pitch detection. Accordingly, it placed audio-visual particles in a 3D-world, and it moved the virtual camera to change the view over its deserted landscape. The audio-visual particles were pre-recorded audio samples that also appeared as lights. The virtual camera movements also affected the audio spatialisation.

The SOUNDING LIGHT INSTRUMENT includes light sensors, i-cube and Lisa software. It unfolds into a big range of performative possibilities - it was the basis of my project series PARALLAX, METAPARALLAX , PARALLAX DEVIATION and MOMEMT IN MOVEMENT. As the architecturally-scaled instrument it is, it can playfully (re)establish the relationships between a site and its environmental context.

Natural and artificial light, body shadow or video may affect the processing of audio samples, or the modulating of audio frequencies, or the processing of a live-input. The inconstant behavior of light guarantees certain unpredictability.

The physical and visual aspects of the instrument are of core importance. Sensor cables are very long; they are hung from the ceiling as volumetric drawings.

The SOUNDING [VIDEO]LIGHT INSTRUMENT is a special mode of the sounding light instrument. Sound recordings are modulated and processed through video-beamed images, e.g. colors or words. Images juxtapose to affect each sensor independently, and each sensor controls multiple audio parameters. Sensor cables are in vertical pipes placed in front of the projection.

Most performances with this instrument involved a collaboration with Hugo Barbosa, for video with Modul8.

The BODILESS INSTRUMENT is a set of long piezzo-amplified wires stretched in space and attached to the walls. It acquires a different body - e.g. different length e.g. different timbre - each time it is newly installed in a room. Adriana plays it with hands, bow and many other tools attached to a tool necklace named R2D2.

The LIGHT PERCUSSION INSTRUMENT combines a stroboscope, a light sensor circuit and an audio generator. It produces audiovisual flashes. Timbre and dynamics are live-controlled parameters.

Prepared ZITHERS are used with sampler and delay modulator. Attached are custom-made pressure sensors and rotation sensors connected to I-cube and Lisa software.


OTHER INSTRUMENTAL COMPONENTS used are: vocals, samplers and various electronics including sensors.