This VR game aims to visualize the challenges of communicating theoretical concepts in a second language, especially during a fast-paced presentation within a limited timeframe.The game is a satirical representation of the mental state of someone navigating an unintuitive talk, where the player experiences an exaggerated physical interaction with some academic jargon. The player is situated inside a mouth, controlling the movements of an exposed tongue in outside the mouth by leaning and jumping. Words are launched from a gigantic brain in the distance, and player’s objective is to hit as many words as possible within a given time limit to achieve a high score. The distance between the tongue and the brain is filled with large ears, symbolizes the distractions and pressure of being at the center of attention, making it even harder to focus on understanding and communicating the concepts being presented. This setup mirrors the difficulty of expressing complex ideas in a non-native language under stressful conditions.
Based on my personal experience, I have realized that several factors intensify the challenge and turn people into what I’ll call “People with Helpless Tongues” (PHT). First and foremost, intuitive conversation and real-time improvisation — both essential tools for effective presentations — are almost entirely beyond the reach of PHTs. This issue becomes even more pronounced during moments of anxiety and fatigue, which are often unavoidable in public speaking. Additionally, I observed that when presenting creative projects, the need to constantly switch between technical and academic language further complicates communication. Finally, having an introverted personality can exacerbate anxiety, adding yet another layer of difficulty.
Theoretical Framework
David Rokeby‘s Transforming Mirrors: Subjectivity and Control in Interactive Media (1995) served as the primary theoretical inspiration for this experiment. Rokeby describes interactive technology as a “medium through which we communicate with ourselves; a mirror” (133). The experiment aimed to create an enlarged and altered representation of neural communication between the conscious and unconscious selves of a PHT (Person with a Helpless Tongue), specifically between the brain and the tongue. This concept takes shape as a delayed echo of the player’s movements and a distorted reflection of their self-embodiment. The central challenge of the experiment lay in determining how to represent this echo and distortion, and to what extent.
Inspired by Rokeby’s paper, I started to relate the notion of “medium as a mirror” to my general understanding of the differences between films and games, art and technology.
Rokeby’s case-studies in the paper were also quite inspirational; His work, “Very Nervous System” (1982-1991), led me to consider how gestures could be translated into meaningful yet satirical sounds. For instance, the player constantly hears a “aaa” sound effect from a mouth, with its pitch fluctuating as the player stretches out to reach words at greater distances. This sound represents the continuous effort PHTs (People with Helpless Tongues) invest in conversation while not hearing themselves in a pleasant way. Rokeby’s other example, “Tumbling Man” (by Chico Macmurtrie), also significantly inspired this experiment; here, the tongue, like the robots in Macmurtrie’s piece, becomes a subject of domination by the participants (147). The dichotomy of control (the illusion of power) and subjectivity, the central theme of Rokeby’s work, aligns closely with the core message of this experience. Viewing language as a technology, or an interface, “Ping Tongue” reminds us that in our attempts to fully control language, we often lose control over our own embodied actions, which are largely dictated by the language itself.
You can download the executable file. It was tested on Vive, Windows Mixed Reality and Quest 2 (using the link cable) (SteamVR is needed)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Technical Steps
Ping Tongue was developed using Unity and Samsung Windows Mixed Reality (WMR). The primary interaction in this project relies on the movement of the head-mounted display (HMD) rather than VR controllers, for two main reasons: First, head tracking is a unique feature of VR that sets it apart from flat-screen mediums, which typically use touchscreens or mouse-like controllers for interaction. Second, restricting the player to control a gigantic tongue primarily through head movements aligns with the experience’s goal of highlighting the difficulty of mastering a non-native language.To convey the exhaustion associated with this challenge, I initially designed the tongue’s movements to be very slow, with even some responses occurring in the opposite direction. However, I realized that this created an overwhelming and confusing experience, making it harder for players to engage quickly. As Rokeby aptly notes, “there is a threshold of distortion and complexity beyond which an interactor loses sight of himself or herself in the mirror” (148). One of the key design questions was whether the tongue should be positioned horizontally or vertically. The vertical orientation was ultimately chosen because it better represents the body’s upright posture and more clearly illustrates the tongue’s movement. Initially, I intended to use an Inverse Kinematic (IK) rigging system, allowing the VR camera to lead the IK target that drives other bones. However, I decided to individually control the orientation of each bone object in Unity using independent scripts. This approach enabled me to introduce custom transformation delays for each bone, each corresponding slightly differently to the VR HMD movement.
Takeaway
During the presentation and playtest, one of the points raised was the solitary nature of the experience, which actually aligns well with the intended atmosphere. Although PHTs outwardly engage in communication with the audience, there is a certain loneliness in the physiological experience they undergo. Another concern I had was that the excessive physical effort required from the players, while they remained unaware of the reactions of others, might create discomfort. However, not only was this issue not raised, but it also seemed to encourage everyone to try the experience. This may be because, as Rokeby suggests, the familiar and ordinary method of interaction reduces the pressure of feeling judged for one’s behavior.
There are additional difficulty elements that could make the experience more challenging. For example, requiring the player to keep the giant mouth open by holding one of the controllers up throughout the session, and closing it to reset the speed of the words when they begin to overwhelm the player. These features still need to be tested to determine whether they would disrupt or enhance the overall experience. Ping Tongue has the potential to evolve into a full VR game with varying difficulty levels, all contributing to the underlying message.