MFA Thesis Project (in-progress)
Form is the in-progress result of a life-long passion for music, communication, accessibility, and sharing the power of the creative arts for self-reflection and healing.
I am presently working with a music therapist at the Elizabeth Seton Pediatric Center in Yonkers, NY to build a haptic blanket to augment music therapy’s benefits for their patients who are deaf or hearing-impaired.
As a former classical singer and person with altered hearing after an acoustic trauma, Form is close to my heart. I’ll be presenting it at our MFA Interaction Design Thesis Festival in May 2019, so stay tuned.
How might we augment the benefits of music therapy for pediatric patients who are deaf or hearing-impaired?
Deaf and profoundly hearing impaired patients rely on the tangible aspects of music to gain benefit from music therapy, a creative and clinical practice that can play a key role in their multi-faceted growth and development. However, traditional instruments were largely built for audio, not tactile, sound experience. They were also built for body shapes, sizes, and abilities that fall towards the center of the bell curve.
Eight Months (In Progress)
This is a “solo” thesis project, but as with many things of scale and complexity, it’s been built and tested with the love and commitment of many (see below)
The pediatric patients and music therapists of Elizabeth Seton Pediatric Center in Yonkers, NY, along with many musicians, technologists, composers, and makers who are deaf and hearing-impaired.
Form is a blanket and music-therapy tool that translates audio music into varied, haptic feedback.
The music therapists of Elizabeth Seton Pediatric Center and their pediatric patients who are deaf and hearing-impaired.
Form takes a fluid and familiar shape, a blanket, to meet the varied needs of ESPC patients. It allows music therapists to use their own familiar interface, traditional instruments, but gives patients more nuanced haptic feedback than a traditional instrument is able to. Form is, at its heart, an exercise in exploring what music feels like when the only way it’s experienced is through tactile means. More theoretically, it looks to shift our perspective on music, sound, and ability.
Leveraging essential musical elements to translate for clear communication.
Form uses rhythm, an aspect of music that exists for every instrument, so patients can feel each note being played all over their bodies. Form also uses volume to distribute the density of vibration over the blanket’s surface. These measures give the patient a more dynamic experience of the vibrations of the music, and the therapist a clear language and ability to gauge the patient’s response as they play. Below is a video showing interactions at each volume threshold.
Under the hood
The hardworking process under the solution.
Vibrotactile musical interfaces exist for people who are deaf and hearing-impaired, but they don’t work for all people, and not for the patients at ESPC.
The Problem Up Close
Patients at ESPC, some of whom are deaf and hearing impaired, also have cognitive and developmental delays, live connected to medical equipment (e.g. ventilators), and have varying degrees of motor control. For them, existing solutions can be alarming or confining. They all have a high degree of vibrotactile sensitivity, and I wanted build something with them that felt native to their tactile fluency.
Music is multi-sensory, but it is thought of and explored primarily as an audial experience.
Music is powerful, think of the last time you were at a concert and felt the bass, the sensation you get when you hear the chords of your favorite song, or see a violinist’s face as they play a passionate piece of a score. Music can make us feel passionate, powerful, angry, and calm. It is multi-sensory — we hear, feel, and see it. And when words aren’t enough, it can let us speak in ways we might not otherwise be able to.
I wanted to build a solution for the patients and music therapists of ESPC. But I also wanted to shift the conversation on ability, music, and how we perceive sound so others might be inspired to build new interfaces.
How, and how much do we need to “translate” music for it to move between audial and haptic experiences?
Research & Synthesis
Building, learning from, and observing patients, therapists, and their interactions has been the center of my thesis work. All of us as exploring this both practical and theoretic space together. We’ve shared journal articles on rhythm and child development, to the way the brain processes audio and vibrational cues. I’ve spent many hours working alongside patients and ESPC’s therapists to learn about their existing interactions.
Ideation & Prototyping
The patients I’m working with are not always able to communicate verbally or physically. Their cues can be subtle and it’s not always possible to explain to them that a prototype is a prototype. This has meant having to build more quickly at a higher fidelity to learn. It’s also meant that Form relies heavily on the collaboration and passion of therapists, sound artists, and technologists who have leant their expertise and time.
Iteration & User Testing
I’m right in the middle of this phase, collaboratively building with my audience. It’s thrilling to see therapists gathered round, and it’s been enlightening to see how hearing individuals react to feeling music in a new way.
What Lies Ahead
Looking back to move forward
Hurdles & Learnings
The biggest hurdles of this project were unexpected. They came not in the form of communication amongst stakeholders or technical hiccups, but rather in the limitations of our own way of thinking of sound, music, and communicating multi-sensory perception.
When presenting my work to a large group, I’ve found myself struggling to communicate the meaning of a haptic, tactile-native interfaces with only visual and audial means of communication. We live in a highly tactile world, but for those who see and hear, it is our tertiary means of communication. Learning to build around and expose those limitations has been an motivating experience.
I have several months left to go with my thesis work, but I’ll keep updating here as I go. If you have questions, interest, or want to collaborate, feel free to reach out!