WINDOW MATTER / AUDIOGRAPHICS 2008
Adriana Sa & John Klima
<BGSOUND src="first.wav">

window matter (with Adriana Sa) 2008 portuguese guitar hero 2007


The "Window Matter" project is a series of live performances with Adriana Sa  and John Klima  involving traditional acoustic musical instruments in combination with live audio and visual processing within a networked multi-user 3d environment. The musical conversation between the performers, their modes of play, instrumentation, the audio software analysis and visual generation form a complete environmental audio-visual experience.

The “Audiographics” software, begun in 2007 (as a proof of concept) under the title “Portuguese Guitar Hero,” has evolved considerably through the collaboration with Adriana Sa, an experimental musician from Lisbon, Portugal. This collaborative effort is titled "Window Matter."

In the performance setting for “Window Matter” Adriana Sa plays the zither and John Klima plays the Portuguese guitar.


zither

portuguese guitar

These acoustic instruments (equipped with microphones) become the joysticks navigating through a networked 3d environment. While moving through space, carefully created audio samples are audibly spatialized and visually placed within the environment, in response to specific acoustic frequency ranges played by the performers. The software analysis of the microphone input thus results in clearly recognizable "in-world" events, while specific musical notes control navigation in exactly the same manner as would a joystick in a video game.

first graphical iteration (John Klima)
second graphical iteration (Adriana Sa)

The initial phase thus complete has included software programming (John Klima), world design (Adriana Sa), and audio production (both) and is resolved adequately to have engaged audiences in several previous performances. However, the project is far from complete - these performances have served to identify necessary enhancements to both the audio system and its expression visually within the live performance context. The next phase of development includes three primary areas:


1.Further development of the audio analysis - As each performer plays a different instrument in a different manner, more individualization in the manner of analysis is needed. For example, Klima's Portuguese guitar tends to produce “straight,” clearly recognizable notes (resulting in clearly recognizable actions) where as Sa's zither often creates a “wall of sound” with countless harmonic frequencies and resonances. Certain customization already exists between each player's version, but more development in this area would greatly enhance individual expression.

2.Continued refinement of audio samples and visual/graphical components – Thus far we have each assembled four sets of samples that we interchange in the course of the performance, and a single form of visual representation within the environment. In conjunction with audio analysis enhancements, enhancements to the variability and control of the audio and visual elements would further complement individual expression.

3.Creation and programming of customized input/control devices – Much as an electric guitarist often requires an array of foot-pedals and switches, it would greatly enhance the performance experience if the musicians were not required to focus on the computer keyboard to affect specific changes in the environment, such as selecting sets of samples, or other alterations not feasible through normal play.


PROJECT TIME-LINE
First Phase
sept/oct 2008 		1. refine frequency ranges and establish
    non-tonal event relationships

2. detection of attack and decay frequency states
    and corresponding graphical modulations
Second Phase
nov/dec 2008 		3. neural network processing of acoustic input
    for complex pattern recognition and audio triggering.

4. enhancement of graphical complexity and variance trough
    the implementation of established movement algorithms.

Final Phase
jan/feb 2009 		5. circuit design and programming of foot switches, pedals,
    and light-sensor triggers and modulators.

6. fabrication of robust "road-ready" enclosures for all
    custom electronics.

7. final enhancement of spatialization characteristics
    and graphics.
March 2009 Project Complete


PROJECT BUDGET
Logistics (performances in New York and Lisbon):
  transportation
  accomodation
  shipping
 		$1100
Physical materials:
  foot switches
  light-sensors
  enclosures
 		$300
Rehearsal and recording studio rental $1000
Artist Fee (software programming) $600
Artist Fee (audio production) $600
Artist Fee (graphics/visuals) $600
TOTAL $4200



CIRICULUM VITAE

  John Klima

  Adriana Sa


RELEVANT WORK SAMPLES

John Klima

  glasbead  (audio-visual multi-user musical interface)
  terrain  (analog circuit design/fabrication)
  ecosystm  (game engine programming, real-time audio and graphics)
  complete works  (cityarts website)


Adriana Sa

  meta(local)morphosis  (Moment in Movement project series)
  portablellax #1  (Moment in Movement project series)
  lightandsoundscape  (Parallax project series)
  complete works   (artist's website)





PERFOMANCE HISTORY - PGH / AUDIOGRAPHICS / WINDOW MATTER

Bang Festival apr 3rd 2008, adriana sa & john klima, faculdade de belas artes da universidade de lisboa, lisbon
Musicbox jan 30th 2008, adriana sa & john klima, lisbon
15 Anos Bolsa Ernesto de Sousa fellowship performance jan 18th 2008, adriana sa & john klima, espaço avenida, lisbon
Creative Fest #1 december 2007, adriana sa & john klima, bomba suicida, lisbon
Knock on Woods @ Lg. do Seculo nov 3,4,5 2007, largo do seculo, lisbon
PGH debut thurs oct 25 at OUVE in o seculo, lisbon