Lost in Translation
2020A project in collaboration with Ravin Raori at dfpi. A study on using language as a way of seeing.
Lost in Translation is an interactive game called findX that explores language, as a way of seeing. It pays homage to early text based adventure games, from the 1970s (onset of home computing) by situating a physical robot (Bob) within its world, an embodied version of the player themselves. The objective is to find something hidden, the nature of which is unknown. The player has no idea where the robot is. The robot provides limited textual and visual feedback to the player, as extractions of its environment. While incomplete, they allow the player to use their imagination and weave a physical world through the descriptive power of language.
You can visit our website to keep track of the development of the game.(Desktop Only)
https://aliceleon.github.io/findX/︎︎︎
The robot sends visual information to Runway ML, a software that allows you to train and use pre-existing machine learning models, that can be run on remote servers. We are using three of these models to read the environment that the robot is situated in. The models help us read the objects, textures, materials and spaces in the environment, based off of the visual information collected on site, by the robot (Bob).
Once processed, the information is presented to the player, sitting in front of the computer screen in a separate location. The processing of information follows the following steps.
