Form

MFA Thesis Project

Form is the result of a life-long passion for music, communication, accessibility, and sharing the power of the creative arts for self-reflection and healing.

I am presently working with a music therapist at the Elizabeth Seton Pediatric Center in Yonkers, NY to build a haptic blanket to augment music therapy’s benefits for their patients who are deaf or hearing-impaired.

As a former classical singer and person with altered hearing after an acoustic trauma, Form is close to my heart. I’ll be presenting Form at the MFA Interaction Design Thesis Festival in May 2019, but am always looking for collaborators to bring the project forward.

 
 
The most recent prototype of  form.

The most recent prototype of form.

 
 

How might we augment the benefits of music therapy for pediatric patients who are deaf or hearing-impaired?

 

The Problem
Deaf and profoundly hearing impaired patients rely on the tangible aspects of music to gain benefit from music therapy, a creative and clinical practice that can play a key role in their multi-faceted growth and development. However, traditional instruments were largely built for audio, not tactile, sound experience. They were also built for body shapes, sizes, and abilities that fall towards the center of the bell curve.

Duration
Eight Months (In Progress)

Role
This is a “solo” thesis project, but as with many things of scale and complexity, it’s been built and tested with the love and commitment of many (see below)

Collaborators
The pediatric patients and music therapists of Elizabeth Seton Children’s in Yonkers, NY, along with many musicians, technologists, composers, and makers who are deaf and hearing-impaired.

Advisors
James Maxson, MM, LCAT, MT-BC, Music Therapist
Brian House, MS, PhD Computer Music
Eric Forman, New Lab, The New School

 
 
Testing a prototype with music therapists at Elizabeth Seton Children’s

Testing a prototype with music therapists at Elizabeth Seton Children’s

 

Form is a blanket and music-therapy tool that translates audio music into varied, haptic feedback.


Target Audience
The music therapists of Elizabeth Seton Pediatric Center and their pediatric patients who are deaf and hearing-impaired.

Solution (in-progress)
Form takes a fluid and familiar shape, a blanket, to meet the varied needs of ESPC patients. It allows music therapists to use their own familiar interface, traditional instruments, but gives patients more nuanced haptic feedback than a traditional instrument is able to. Form is, at its heart, an exercise in exploring what music feels like when the only way it’s experienced is through tactile means. More theoretically, it looks to shift our perspective on music, sound, and ability.

 
 
 

Leveraging essential musical elements to translate for clear communication.


Form’s Translation
Form uses rhythm, an aspect of music that exists for every instrument, so patients can feel each note being played all over their bodies. Form also uses volume to distribute the density of vibration over the blanket’s surface. These measures give the patient a more dynamic experience of the vibrations of the music, and the therapist a clear language and ability to gauge the patient’s response as they play. Below is a video showing interactions at each volume threshold.

 
 

 
 

Under the hood

The hardworking process under the solution.

This is a very light overview of some process points and findings of my thesis work. If you’d like to learn more as I build, please say hi.

 
 
The sensory abilities of some of the patients (names changed) at ESPC.

The sensory abilities of some of the patients (names changed) at ESPC.

 
 

Vibrotactile musical interfaces exist for people who are deaf and hearing-impaired, but they don’t work for all people, and not for the patients at ESPC.


The Problem Up Close
Patients at ESPC, some of whom are deaf and hearing impaired, also have cognitive and developmental delays, live connected to medical equipment (e.g. ventilators), and have varying degrees of motor control. For them, existing solutions can be alarming or confining. They all have a high degree of vibrotactile sensitivity, and I wanted build something with them that felt native to their tactile fluency.

 
 
music is multi-sensory

music is multi-sensory

 

Music is multi-sensory, but it is thought of and explored primarily as an audial experience.

Key Insights
Music is powerful, think of the last time you were at a concert and felt the bass, the sensation you get when you hear the chords of your favorite song, or see a violinist’s face as they play a passionate piece of a score. Music can make us feel passionate, powerful, angry, and calm. It is multi-sensory — we hear, feel, and see it. And when words aren’t enough, it can let us speak in ways we might not otherwise be able to.

I wanted to build a solution for the patients and music therapists of ESPC. But I also wanted to shift the conversation on ability, music, and how we perceive sound so others might be inspired to build new interfaces.

 
 
 

How, and how much do we need to “translate” music for it to move between audial and haptic experiences?

Research & Synthesis
Building, learning from, and observing patients, therapists, and their interactions has been the center of my thesis work. All of us as exploring this both practical and theoretic space together. We’ve shared journal articles on rhythm and child development, to the way the brain processes audio and vibrational cues. I’ve spent many hours working alongside patients and ESPC’s therapists to learn about their existing interactions.

Ideation & Prototyping
The patients I’m working with are not always able to communicate verbally or physically. Their cues can be subtle and it’s not always possible to explain to them that a prototype is a prototype. This has meant having to build more quickly at a higher fidelity to learn. It’s also meant that Form relies heavily on the collaboration and passion of therapists, sound artists, and technologists who have leant their expertise and time.

Iteration & User Testing
I’m right in the middle of this phase, collaboratively building with my audience. It’s thrilling to see therapists gathered round, and it’s been enlightening to see how hearing individuals react to feeling music in a new way.

 
 

 
 

What Lies Ahead

Looking back to move forward

Hurdles & Learnings
The biggest hurdles of this project were unexpected. They came not in the form of communication amongst stakeholders or technical hiccups, but rather in the limitations of our own way of thinking of sound, music, and communicating multi-sensory perception.

When presenting my work to a large group, I’ve found myself struggling to communicate the meaning of a haptic, tactile-native interfaces with only visual and audial means of communication. We live in a highly tactile world, but for those who see and hear, it is our tertiary means of communication. Learning to build around and expose those limitations has been an motivating experience.

Next Steps
I have several months left to go with my thesis work, but I’ll keep updating here as I go. If you have questions, interest, or want to collaborate, feel free to reach out!