Telephone booths installed in the show enable visitors to experience a digital simulation of what it would be like to lose their hearing and be forced to communicate using a cochlear implant.
Communicating using spoken language is arguably one of the main daily activities of people all over the world. Interpersonal communication is part of almost every social interaction and it is inexhaustible in its complexity and fields of application. Since the global coronavirus pandemic, such communication has increasingly been digital. We talk online via microphones, see each other on a screen, and often only hear each other through headphones.
Find the differences together with a partner! It's actually not that easy if you don't understand each other well. (Yadav, Manuj, Schuetze, Julia, & Wagener, Kirsten C. (2020). PDF and Photoshop files of DiapixGE picture materials - adapted German version. Zenodo. https://doi.org/10.5281/zenodo...)
In the exhibition you can become part of this pervasive type of daily communication by debating with someone online on a specific issue. You and your interlocutor will take a seat in different booths that serve as something akin to modern telephone booths. Using a microphone and headphones, you will speak to and hear the other person, while you will be able to see each other on a screen.
After a short time in the booth, the situation will change dramatically. Both voices will suddenly be markedly distorted. You will have the impression that you are constantly losing the signal or having to communicate despite a very poor connection. Communication will become difficult. In such situations, we invariably look for ways to continue communicating – for example by trying to adjust the way we talk to make our speech more readily comprehensible to the other person, or by paying more attention to the screen, because lip movements and facial expression may provide clues to decipher the distorted sound of the other person’s voice.
At some point, though, the image will also disappear. When that happens, you will be trapped in a situation where you can only communicate via rudimentary acoustic signals. You will be transported back to the early days of telephone communication or even to the last two months before your birth, when you first perceived linguistic signals from the outside world while still in your mother’s womb. Is communication still possible in this set-up? Give it a try! It will only work if you cooperate with your interlocutor. Open your mind to new ways of thinking about communication in the real and digital world.
Inside the telephone booth. (© Frank Brüderli)
You might be interested to discover that the voices are generated by a digital noise vocoder here. It works by almost completely removing the spectral (frequency) features of the voice, leaving only the rough temporal sequence of the signal. The auditory impression you receive is comparable to what someone with a cochlear implant hears. Such implants can be used to regenerate hearing performance in cases of hearing loss caused by a dysfunctional inner ear. In the booth, you will have a chance to experience what it might be like to communicate when fitted with a cochlear implant. However, the auditory impression generated by today’s cochlear implants is usually much better than the type of simulation chosen for the exhibition.
Numerous scientific studies have examined how people adapt their voice when talking to others in conditions that make communication difficult. They offer scope to gain important insights into the acoustical structure of speech signals and how information is encoded in them. Digital systems can also be trained using such data. After participating in Listen up!, you will have an opportunity to make your voice available anonymously to scientific research.
Prof. Dr. Volker Dellwo, Department of Computational Linguistics & Linguistic Research Infrastructure, UZH
Andrew Clark Linguistic Research Infrastructure, UZH
Dr. Claudia Roswandowitz, Department of Computational Linguistics, UZH
Dr. Thayabaran Kathiresan, Telepathy Labs, Zurich