Into the Shell
Into the Shell is an interactive installation that combines advanced manufacturing, music, and applied mathematics to evoke the power and beauty of the ocean.
Into the Shell is sponsored by Fractured Atlas and Voodoo Manufacturing.
THE CONCEPT
Into the Shell is an interactive art installation designed to reawaken the wonderment of hearing the ocean in a seashell. Audiences enter a giant, 3D-printed seashell crown to manipulate in real-time an original oceanic lullaby. Drawing from the artists’ relationship to the ocean, Into the Shell offers audiences a journey into seashells, across oceans, and through memory.
Into the Shell at the Imagine Science Festival
ISSUE Project Room, October 2018
Team
Smita Sen | Creative Director
Joo Won Park | Sound Designer
Angela Wang | Producer
Siena Sofia-Bergt | Producer EMERITA
Aramael Peña-Alcantara | Set Designer
Jaycee Holmes | Creative Engineer
Christina Tang | Lighting Designer
ROLE
As Creative Director and Producer, I created the Bivalvia crown and worked with the team to develop the USER EXPERIENCE and interaction concept. i oversee creative, financial, and technological development, as well as organizational partnerships.
FROM CONCEPT TO PRODUCT
Concept
Our concept was simple – to reawaken the childhood wonder of hearing the ocean in a seashell. We discussed our own relationships to that memory and to the ocean. Could we use technology to evoke feelings that are at once fragile and universal? What approach would we use?
Approach
For a memory that is primarily auditory and tactile in nature, we thought it would be more powerful to create an installation that didn’t use a Graphical User Interface (GUI). I had designed the Bivalvia Crown and developed a rough idea for an interactive experience: Users would wear or sit below the giant, 3D-printed shell crown and live-compose an oceanic lullaby. . When I brought it to Joo Won Park, a friend and long-time collaborator, we fleshed it our further. At the time, Joo Won, a musician and applied mathematician, had been working on spatialized audio for Virtual Reality (VR). We discussed the possibility of using VR spatialized audio to drive the audio interaction.
To reawaken the magic of hearing the ocean in a seashell,
we invited users to enter a giant, 3D-printed seashell crown and manipulate an original oceanic lullaby.
Above: From Audio Localization Method for VR Application (Joo Won Park, Audio Engineering Society Journal). Below: Renderings of the Bivalvia Crown (Smita Sen).
UX CHALLENGES AND Proof of concept
Key Questions
Will users respond to interactive 3D audio localization without a GUI to guide them? How can we bring together the 3D printed crown, the environmental design, and the lighting design to influence the user’s approach to manipulating the sound and interacting with the story?
Technology Challenges and Proof of Concept
Isolating spatialized audio from the VR headset is an enormous undertaking. In the first iteration, we used microcontrollers to recreate the action of spatialized audio in a stereo sound system. This serves us well as a Proof of Concept in testing how users approach interacting with sound and light without a Voice User Interface (VUI) or a GUI. We use clear and dramatic sound and light changes to create positive feedback loops and increase user engagement. Jaycee Holmes explored and implemented these technology challenges.
STORY DEVELOPMENT
Story development and design exercise led by Set Designer, Aramael Peña-Alcantara.
Story and Design Challenges
”To reawaken the childhood wonder of hearing the ocean in a seashell.”
At the core of this story are experiences of wonderment and discovery. In an exercise led by Aramael, the Set Designer, we came to the conclusion that to ensure users had these core experiences, our design required ownership as a vehicle of wonder; structural spiraling to control visibility; and discovery as play. These conclusions would drive environmental design choices.
USER EXPERIENCE
I sketched out different potential layouts for the team to discuss different potential visitor pathways. Each would change the pace of the user journey – the pace at which users would discover that as they move through the space, they are manipulating the music and the light cues.
The Evolution of Into the Shell
Designing the Bivalvia Crown, June 2017.
Printing the 16” scale sample in September 2017.
Demo desk setup with audio sample, 16” 3D-printed scale sample, and VR audio algorithms for NYC Media Lab Summit 2017. We went on to present the work at Advertising Week TechX, attended by ~100,000, and Future Reality, all in Q4 of 2017.
Lighting design developed by Lighting Designer, Christina Tang.
SOUND DESIGN
Joo Won and I talked about how standing beside the ocean can bring up feelings of both wonder and fear. We also talked about how the music could be evocative of the ocean without just being sounds of ocean waves. She developed a stunning oceanic lullaby that “evokes the ocean with a deep, even, base line that ebbs and flows like the tide.” (Merritt Ver Steeg for Imagine Science Festival)
LIGHTING DESIGN
Christina, the Lighting Designer, and I talked extensively about the mystique of the ocean and the feelings of awe it can inspire. She developed the ‘mylar pool’ lighting design with LED lights and mylar, creating ethereal reflections and refractions that evoked water and waves.
Into the Shell at the Imagine Science Festival, ISSUE Project Room
Brooklyn, NY
12 October 2018
Our vision for the large-scale iteration of Into the Shell includes four Bivalvia Crowns and a 30’-diameter shell-inspired housing structure.
Large-scale design by Aramael Peña-Alcantara.
Renderings by Smita Sen.
Sen Studio, founded in 2017, is a multimedia collaborative specializing in interactive exhibits which play with the junction of art and STEM to increase interdisciplinary collaboration as well as spread public knowledge of new technology. This interdisciplinary and international team has brought together leading graduates and graduate students from Columbia University, MIT, and NYU, as well as professionals working in the arts and technology at companies like Microsoft, Sony Music Entertainment, and Disney Imagineering.
