Ping Tongue

Objective

The main objective of this VR project was to symbolically visualize how challenging it could be to communicate theoretical concepts in a second language and particularly in a fast pace type of presentation within a limited timeframe. Ping Tongue is an attempt to satirically translate the mental state of someone who is going through an unintuitive talk in the form of an exaggerated physical interaction with vocabularies. The player is situated inside a mouth and by leaning towards left and right or jumping up they control an exposed tongue. The goal is to hit as many as possible words that are being thrown out towards you from a distant brain within a certain time frame. The more words you manage to beat, the higher your score will be. The distance from the brain which is filled with large ears represents the distractions that are created as a result of being at the center of attention.

Ping Tongue

From a personal experience, I came to realize that there are a number of issues that amplify the intensity of the challenge and turn people into let us call People with Helpless Tongues (PHT). First and foremost is the fact that the intuitive conversation and real-time improvisations that act as vital instruments of presentation are almost entirely out of the PHT’s reach. This becomes even more critical in times of anxiety and fatigue that usually are inevitable parts of the public presentations. Second, I noticed that especially for presentations of creation projects, a back and forth switch between technical and academic language adds to the complexity of the communication. Third, having an introverted personality trait could also boost the anxiety.

Theoretical Framework

Transforming Mirrors: Subjectivity and Control in Interactive Media (1995) by David Rokeby was the key theoretical inspiration behind this experiment. He defines the interactive technology as a “medium through which we communicate with ourselves; a mirror” (133). The idea was to create  an enlarged and transformed representation of neural communication between PHT’s conscious and unconscious self, between the brain and the tongue. It is a delayed echo of the player's movement and a distorted mirror of their self-embodiment. The whole challenge of this experiment was the way and the extent by which this echo and distortion could be represented.

Transforming Mirrors
Inspired by Rokeby's paper, I started to relate the notion of “medium as a mirror” to my general understanding of the differences between films and games, art and technology.

Rokeby's examples were also quite inspirational; his own work, Very Nervous System (1982-1991) made me take into consideration the translation of gestures into meaningful yet satiric sounds. There is a mouth’s “aaa” sound effect that the player hears constantly, whose pitch fluctuates when the player stretches out to reach the words at further distances. This is to denote the constant energy PHTs put into the conversation while not hearing themselves pleasantly. Rokeby's other example Tumbling Man (Chico Macmurtrie), was also a significant inspiration to this experiment; The tongue, too, like the robots in this example is a subject of domination by the participants (147). The dichotomy of control (illusion of power) and subjectivity as the key topic of this paper overlays well with to the core message of this experience; if we look at the language as a technology (an interface), Ping Tongue reminds us that while being immersed in attempting to be in a full control of the language, we almost lose the control of our embodied actions that is being almost totally dictated by the language itself.

Downladable File

You can download the executable file (Icon). SteamVR is needed.

It was tested on Vive, Windows Mixed Reality and Quest 2 (using the link cable).

Technical Steps

Ping Tongue was developed with Unity and Samsung Windows Mixed Reality (WMR). The main interaction of this project is centered on the HMD movement instead of the VR controllers, for two main reasons: first, interaction through head tracking is fairly unique to VR in comparison with that of flat screen mediums which typically suggest interaction via touchscreens or a mouse-like controller. Second, being restricted to control a gigantic tongue, mainly with your head’s movement well suited the aim of this experience, which was to emphasize the burden of mastering a non mother tongue. To echo the exhaustion as a result of such a burden, I started with the very slow bones which even responded in the reverse directions. I realized that it is getting overwhelming and at the same time confusing and players required a much longer time to engage. As Rokeby rightly mentions “there is a threshold of distortion and complexity beyond which an interactor loses sight of himself or herself in the mirror” (148).

One of the key questions was whether the tongue should be placed horizontally or vertically, but soon the latter was chosen as it better represented the verticality of the body and more clearly displayed the movement of the tongue. At the beginning, the aim was to use an Inverse Kinematic (IK) rigging system and let the VR camera to simply lead the IK target which drives other bones. But then I decided to control the orientation of each bone object individually inside the Unity via independent scripts; this allowed me to apply custom transformation delays for each bone, each corresponding to the VR HMD movement slightly different.

Takeaway

One of the points raised during the presentation and playtest was the solitary of the experience, which in fact well matches with the intended atmosphere; although PHTs do engage in communication with the audience outwardly, there is some level of loneliness in what they are physiologically going through. The other point I was concerned about was the possibility that exposing such excessive physical effort of the players to other viewers (while being blind to their reactions) might result in an uncomfortable experience. Not only this issue was not raised, but somehow it provoked everybody to try the experience. This is perhaps due to the familiar and banal method of interaction that as Rokeby suggests reduces the pressure of their behaviour being judged. There are other difficulty measures that could intensify the challenge; for instance, keeping the huge mouth open by holding one of the controllers upwards throughout the experience and closing it to reset the speed of the words when they start to flood. Such hinders are yet to be tested if and how they might disturb or enhance the overall experience. Ping Tongue could certainly be evolved into a full VR game with different difficulty levels that all contribute to the underlying message.