Game engines with grounded language

Most video games are programmed with a game engine. The game engine stores the objects on the screen and renders them to the monitor. Such a setup ensures that the game can be implemented in Python with the constraint that a human player interacts with the game engine.

To make sure, that the game can also be played with artificial intelligence and additional module is needed which is a textual buffer. The buffer converts the output of the game egine into textual sentences e.g. "player is left, red obstacle ahead". The assumption of the textual buffer is, that the game should be rendered to a mini text terminal which has only 16x2 characters. The challenge is that a complex game gets compressed into such a small text buffer. This is only possible with a vocabulary of words. Converting the normal videoscreen of 800x600 pixels into a list of words can be realized with a computer program in realtime.