Highly complex algorithms enable video- and voice-controlled “deep fakes” to create fictitious realities almost in real time, steering our perception without leaving a trace.
All the Lives, Trailer, 2022 © Nadine Cocina, Björn Franke, Paulina Zybinska
These technical algorithms work with the very features that give us our unique and individual human identity: our voice or our face. That is a remarkable feat given that science is still in its infancy when it comes to a deeper understanding of these identity-forming characteristics. Deep fakes are often associated with cases of misuse for nefarious purposes. But, while the technology certainly harbors considerable potential for spreading misinformation in the socio-political realm, in the fields of art and entertainment it is often used instead to put words in the mouths of celebrities or swap out peoples’ faces.
The immersive installation All the Lives, by contrast, allows viewers to use deep fakes to enter into alternative living situations that may be imminent in the future or that have already been experienced in the past. First, an image and a voice sample of the visitor is recorded, and then these are integrated using a deep fake algorithm into short films in which viewers find themselves in a synthesized life reality. Visitors see themselves, for example, as seniors on the terrace of their sunny beach house or sitting alone at a table, trapped in a dreary office or as movie stars besieged by flashing cameras, or as athletes pummeling a punching bag.
The experience of seeing your own deep fake alter-ego in a new role, situation, or identity that you had not previously experienced encourages reflection on conventional roles and the life plan that you have been pursuing. In this sense, All the Lives can be understood as a philosophical machine that brings opting for a certain way of life, with all its limitations, into sharp focus while at the same time offering a projection screen for the self and potential alternate identities. In the process, the immersive experience enables a deeper understanding of the social and political dimensions of these new technologies. This innovative art installation thus paves the way for novel scientific approaches to researching the human identity code.
All the Lives, 2022 © Nadine Cocina, Björn Franke, Paulina Zybinska
All the Lives, Making of, 2022 © Nadine Cocina, Björn Franke, Paulina Zybinska
Jeffery DelViscio, “A Nixon Deepfake, a ‘Moon Disaster’ Speech and an Information Ecosystem at Risk,” Scientific American, July 20, 2020
Katerina Cizek, William Uricchio, and Sarah Wolozin, “Media Co-creation with Non-human Systems,” in Collective Wisdom: Co-Creating Media within Communities, across Disciplines and with Algorithms, 2019
Jordan Peele, “Obama Deep Fake,” Ars Electronica
Luke O’Neil, “Doctored Video of Sinister Mark Zuckerberg puts Facebook to the Test,” The Guardian, June 12, 2019
EJ Dickson, “TikTok Stars Are Being Turned Into Deepfake Porn Without Their Consent,” Rolling Stone, October 26, 2020
Supasorn Suwajanakorn, Steven M. Seitz, Ira Kemelmacher-Shlizerman, “Synthesizing Obama: Learning Lip Sync from Audio,” ACM Transactions on Graphics 36, no. 4 (2017): 95:1–95:13.
Lin Cao, Wenjun Sheng, Fan Zhang, Kangning Du, Chong Fu and Peiran Song, “Face Manipulation Detection Based on Supervised Multi-Feature Fusion Attention Network,” Sensors 21, no. 24 (2021): 8181.
Yisroel Mirsky and Wenke Lee, “The Creation and Detection of Deepfakes: A Survey,” ACM Computing Surveys 54, no. 1 (2022): 1–41.
Sherry Turkle, The Second Self: Computers and the Human Spirit (New York: Simon & Schuster, 1984).
Sherry Turkle, Life on the Screen: Identity in the Age of the Internet (New York: Weidenfeld and Nicolson, 1995).
Donna Haraway, “A Cyborg Manifesto: Science, Technology, and Socialist-Feminism in the Late Twentieth Century,” in Simians, Cyborgs and Women: The Reinvention of Nature (London: Routledge, 1991).
James Glass, Shattered Selves: Multiple Personality in a Postmodern World (Ithaca, NY: Cornell University Press, 1993).
Erving Goffman, The Presentation of Self in Everyday Life (London: Penguin Books, 1990).
Jacques Lacan, “The Mirror-Stage as Formative of the I as Revealed in Psychoanalytic Experience,” in Écrits: A Selection (New York: W. W. Norton, 1977).
Paul Ricoeur, Oneself as Another (Chicago: University of Chicago Press, 1994).
Concept: Dr. Björn Franke, Interaction Design & Visual Communication, ZHdK
Programming: Paulina Zybinska, Interaction Design, ZHdK
Video: Nadine Cocina, Interaction Design, ZHdK
Casting: Isabela Gygax
Actors: Tina Schmid, Christian Leugger, Teresa Matusadila Leuzinger, Rahim Lascandri, Astrid Kehl, Kurt Herzog, Kemal Dempster, Christoph Gross, Susann Klossek, Walesca Frank, Melisa Sari Arslan, Ramona Sprenger, Jemsith Regan Raveendran, June Donkor, Oliver Meier, Clifford Seidmann, Elijah Knight, Kate Tsui