The Work "Navigating and Negotiating Sound Architectures of the Night" is a major evocative multi-media black box theatre work. Its hybrid form includes 3d architectonic models that are both virtual and physical; digital video; a generative visual system including 4 real-time projections; 10 live musical improvisers; a generative soundtrack; a recorded spoken poetic text; and interactive interface table enabling multiple interactants to explore the work simultaneously.
The work engages with the poetics of night through an interactive structure of 84 abstract architectural models that are explored over time through multiple sonic / architectonic movements; through placement and spatial movement / displacement; as well through architectural abstraction. A specially designed glass interface table enables participants from the general public to interact with the work and become part of the performance via this intuitive multi-user interface. A computational sensing system and specially authored computer code enables one to juxtapose projection-like texture maps of night-related architectural / city-scape imagery, forming large format virtual / architectural landscapes presented as 4 large scale projections in the space. Each projection contains the view visible from a particular side of the table. A seven movement generative/interactive musical score is also driven via the interactive choices of the participants; and a series of 10 live audio improvisers also add additional layers to the interactive musical score. Multiple architectonic sculptures become part of the "set". A poetic text related to the night theme is also enfolded as part of the environment. The work is highly interdisciplinary and students from over 5 departments at Rensselaer Polytechnic Institute have been participating over the course of the year.
Each of the 84 physical interface architectural primitives (physical architectural model shapes) are robust in nature — many people interacted with them during the performance; each piece is recognized by the system through a unique symbol on the bottom of the architectonic form. An additional set of objects can be placed next to the models and create architectural abstractions in real time on the display screens. This makes the work extremely flexible in terms of real-time exploration of architectonic forms and related texture maps. The architectural models have been made of many different materials – wood, metal, plastic (3D printed) etc. Usability tests were done related to which materials would work Best.
For each physical piece an isomorphic form was made with 3D modelling
software. The programmer facilitated “parallel” spatial moves in the “projected
space” to that of the movement of the object on the interface table. A “Night”
video landscape was also projected behind the 3d Models/texture maps. Thus a
kind of 1-1 relation was formed and brought to life via computer code. For each
piece a set of texture maps was developed.
A series of music loops were developed by Seaman before the performance,
working in tandem with student members contributing sonic materials that were
folded into the mix. These loops were “attached” by the programmer to each
object. Each time an architectural model was put down on the table that loop was
played back in the room. The room reproduced the spatial positioning of the
objects on the table and as participants moved the objects, their loop moved in
physical space forming a dynamic real time spatial mix. Each of the 7
movements had 12 differing loop/objects and functioned as a generative
soundtrack. Docents delivered the new objects and cleared the table for each
new movement.
When the participant/interactant placed the object on the glass interface surface,
the image appeared in real time on one of the screens from a particular
“perspective” that the programmer had authored. Texture maps also appear on
the surface of the object at that time as presented on the screen. As discussed
above the sound had a positionality in real space. When the participant moved
the piece the sound actually moved to a corresponding location in physical
space. A dynamic relationality of the pieces was thus achieved though both sonic
and visual spatial means. The goal was to make a working system where many
pieces could be put on the table and interacted with via multiple interactants
simultaneously enabling quick dynamic architectural design capabilities — thus
the work functioned in part as a real-time multi-user “sketching” tool as well as
artwork generator.
Once this goal was achieved, we authored higher level functionalities that related
to the proximity of one object to another, and/or the functionality of one set of
objects that could functionally abstract the architectonic qualities of the original
object – e.g. make differing algorithmic abstractions of the objects including the
following— making the selected objects bend; multiply; deform or twist; as well
as make them more or less transparent; change the scale of the objects that are
projected; and/or change the height of the objects.
The interface table had a sensing system that read the symbols affixed to each object via an infrared camera. The table was robust and inspired in part by the
initial open source specifications provided by Todd Berreth (Seaman’s
collaborator at Duke). The table was later fully re-designed by the team. The
room had 4 major screens as one component of the work each providing a
differing view of the architectural models covered in texture maps, with an
additional slow moving/pulsing video texture map as background. This physical
interface table empowered dialogue between participants standing adjacent to
the table who in real time manipulated architectural models/projections/sound
positioning. Over the course of the evening literally hundreds of participants
explored the system!
Being the main focus, the table was placed in the center of the room. The 12’x16’
screens were placed 8’ above the ground, tilted at an angle that was comfortable
for audience members to see. The luminous screens together formed a large
central space that with the table created a space that was the focus of the
performance, though the participants were allowed to move freely within the
room. The projectors were placed in the corners of the room, rear projecting onto
the screens.
Pauline Oliveros worked with a group of improvisers who were distributed around the edge of the room. This group provided some audio materials to Seaman for the construction of the loops. These audio materials consisted of improvisations recorded alone and/or in groups; as well as field recordings at night – drones (the sound of lights or transformers; cars over the surface of a bridge creating pitches, instruments and sung aspects recorded from a distance; Seaman made a generative loop system with additional instruments that was designed to work together in any generative combination of the architectonic movements. These loops were attached to the architectonic “model pieces” described above. As the pieces were manipulated on the interface tables individual performers improvised with these loops distributed across the space. Thus both the loops and the performers had a spatial position in the room. Performers positioned themselves in relation to each other – e.g. working closely together to make a certain kinds of sound --- and also functioning at a distributed distance to each other. To give the performers a defined space, we created canopy structures that hung at various heights over the performers. The musicians needed to be able to see each other in order to give cues while improvising, and most also needed power sources. They also couldn’t be too close, crowding the central table. We spaced the performers out around the perimeter to fulfill these requirements. Small architectonic enclosures were designed. These canopies were dimly lit to have an ambient glow. The table also had a light over it that would come on in between each set to signal a new movement.
I made 12 sonic submixes of these tuned loops for each of the 7 movements – one for each object. Thus one series of these loops/objects were brought out and placed on the table. These were then completely removed and a new set were brought out. Thus new sonic keys and subtle rhythms could be provided over the course of the seven movements lasting approximately one hour. In this way we could achieve movements in the work – one for each night of the week. The goal was to get one set of objects/sounds functioning that produce a selforganizing system with variation between the audio work produced via the sonic spatial mixing of performers and loops. Once this had been facilitated 6 additional “sets” were devised and a series of ambient plateau-like sonic states functioning at different levels of intensity were authored.
Video images of night both still and moving were recorded by both myself and the students. These were reduced to HD for playback (and saved at even lower resolutions to function in conjuction with the programmer’s needs in relation to the specifications of the numerous texture maps.
Seaman wrote and recorded a poem which was presented in the space in two locations via highly focused speakers from above. Thus, one “found” the poetic text while walking around the room. Only when directly under the speaker did the poem become part of the mix.
The programming was built using the Unity engine. Use of both open source and new code was explored. A central machine would read the IR video image via a camera which was housed in the bottom of the interface table pointing upward; and the system processed the symbol token for identity, location and orientation. Identity refers to a database that contains, in this case, 3D models, texture maps, and sound. Most tokens refered to 3D models but some were modifiers that transformed objects that they were adjacent to. The database for modifiers was a different one. A central machine communicated the locations of the objects and orientations to four other machines, each running a copy of the unity environment. Objects were then placed and rendered in each separate system, this enabled the unique camera locations to be designated in each case with the coordinated yet unique perspectives shown on the large displays above the tables. This process also resulted in a system that was robust to glitches/failures during the performance. Sound files all run continuously in the background in an initial silent mode. Placement and removal of objects on the tables turned up or down volume with fast controlled attack and decay envelopes. This allowed the sound files to remain coordinated. I have enclosed the coordinated movements 1-7 —this generative coordination is achieved by playing back all of the 12 loops for a particular movement in sinc. The object’s placement on the table also determines the apparent location of the sound within the black-box theatre using a novel sound rendering technique by Samuel Chabot/Jonas Braasch detailed in the MS Thesis: SPATIALIZED SOUND REPRODUCTION FOR TELEMATIC MUSIC PERFORMANCES IN AN IMMERSIVE VIRTUAL ENVIRONMENT.
Both still and video documentation of each of the architectonic movements were recorded from multiple positions in the space at EMPAC where the work was performed. I have enclosed the first 3 minutes of movement 1 as documentation. Each movement was similar in structure where the models were delivered, and put on the table one at a time. The actual duration of the 7 movements is variable but at the moment we have a 54 minute version. The improvisers that played along with my generative audio files were students studying under Pauline Oliveros. The performance was well attended. Sadly, Pauline Oliveros died shortly after this project was completed.
Swimming in the night
In the sound
Across the lake I hear a loon
A light shimmers a line of abstraction in reflection
One can hear the shape of the landscape
As sounds echo across the night from shore to shore
Cove to cove
A sound architecture, a landscape reflection
The waves are refracted in the wind of thought
Transparent light walls speak in muted voices
I am a child, blindfolding myself in darkness
Approaching blindness with intention
The thought of blindness
To understand
To learn my reach, my grasp of space
My bodily limits
I learn the room, felt meanings on memories
A history of relations
bumping over the glass and the water spills
So different
The sound of the heater releasing steam
I grasp the architecture as I fall in relation to it
In a Paris hotel room, I forget the steps having not learned the space
And returning back in the blackness
I fall
I feel the pain of my leg bending back, broken, laying now on the floor
In the deep black depth of the night
Waiting patiently for the light
The window forms in the distance
A figure moves across a room
Color perception diminished
A perfectly lit room framed by the dusk
The clarity of the air, a disturbance,
transmits the light in a crystalline manner
clearly the dead of winter, extreme cold
Light illuminates the dance
Steam forms shape shadows
as two workman cast a perception of their frames
Unknown noises abstracted by distance and wind
Move in and out of the audible range
In resonance with the shape of doubt,
my sense projections filtered
in the light of the evening’s noise
at first it appears to be a figure
I project onto the landscape
a history of observations
Into the darkness thought moves
proximity, a relationality, a false projection
An illusion of the night
A quiet disillusionment unfolds
A poem of light and space
Speaking of the night
The light states are captured in this instance,
Housed, transformed, and released
With sound architecture framing these dark transparent meshes
projected sketches, made of mutable light motions
These quietly moving forms, form a space of negotiation, of thought passages
As they play out across the night’s figures and architectural domains
Where will we end up after these navigations?
A new place in thought.
The chess-like moves have had the rules shifted,
or removed --- they are open
So negotiations of light reflect the navigations
the night sounds are triggered and released, each unfolding a spatial relation
setting in motion living responses
reactions flowing
patterns of resonance transgress the dissonances of darkness
Feeling the way around the darkened room
picturing your face
From memories of touch
Eyes closed
One remembers this touching moment
The limits of a body of thought – a body of memories
The light arc of experience
I have found my way here navigating negotiations
Memories of the movements within
The dark spaces of my nature
These multiple lives
in thought and action
In a vast architecture of darkness,
Navigating meshed virtualities
the losses, with sound enterings,
the back and forth, turning around,
turning in on itself
The quiet internal conversation.
The night is a veil masking true detail
Again, mistaken in our projections
overlays
doubts made palpable
The architectonic forms of thought
Are here and gone
On the tip of the tongue
behaviors are rendered, remembered.
The skew embrace
The body’s positionings in space
The movements and proximities are navigated
In the dark room
Focused rays render the moment
Light abstractions, memories and realizations
Are rendered palpable
After years of being locked away
The space of an association is released and meanders
Open to this negotiation
This speaking across boundaries
Triggers
as the world unfolds in the moment
Out of these spaces of dark silence.
These sound architectures
The architecture of thought and remembrance.