October 02, 2026

Creating chatty robots

 

The Micromouse challenge is a well known robot competition which is mostly a hardware building plus programming challenge. The rules of the challenge can be modified so that the ability of a robot is determined to generate grounded language. In the screenshot, the robot has a text widget at the bottom for showing the text.

The human user controls the robot and at the same time the textual output of the robot gets updated. This textual output allows a rule based AI to control the robot automatically which wasn't implemented yet. The only difference to the normal micromouse challenge is, that in the example the mentioned text widget is visible.

The text gets produced with a dictionary of words. Possible words like [move, goal, blocked] are describing objects and events of the game. The program detects situations and uses the words to generate the output. Because there is a mapping between the game state and the textual output its called grounded language.

import math, sys, pygame

# 1. SETUP & CONFIGURATION
WIN_W, WIN_H, MAZE_H = 800, 700, 580
WHITE, BLACK, GREEN, BLUE, GRAY = (255, 255, 255), (0, 0, 0), (40, 180, 40), (40, 100, 220), (200, 200, 200)

# Grounded language vocabulary for rule-based AI parsing
DICT = {
    "verbs": ["detected", "reached", "turn", "move"],
    "nouns": ["wall", "path", "goal", "junction", "distance"],
    "adj": ["ahead", "left", "right", "clear", "blocked", "near"]
}

# Solvable Maze Walls [x, y, w, h] & Goal
WALLS = [
    pygame.Rect(0, 0, 800, 10), pygame.Rect(0, 0, 10, 580),
    pygame.Rect(790, 0, 10, 580), pygame.Rect(0, 570, 800, 10),
    pygame.Rect(150, 0, 10, 420), pygame.Rect(300, 160, 10, 420),
    pygame.Rect(450, 0, 10, 420), pygame.Rect(600, 160, 10, 420),
    pygame.Rect(150, 250, 80, 10), pygame.Rect(450, 350, 80, 10)
]
GOAL = pygame.Rect(680, 480, 80, 80)

# 2. ROBOT CLASS
class Micromouse:
    def __init__(self, x, y):
        self.x, self.y, self.angle, self.radius, self.sensor_len = x, y, 0, 14, 55

    def update(self, keys):
        speed = (2.5 if keys[pygame.K_UP] else 0) - (1.5 if keys[pygame.K_DOWN] else 0)
        self.angle = (self.angle + (3.5 if keys[pygame.K_RIGHT] else 0) - (3.5 if keys[pygame.K_LEFT] else 0)) % 360
        rad = math.radians(self.angle)
        nx, ny = self.x + math.cos(rad) * speed, self.y + math.sin(rad) * speed
        
        # Collision detection with walls
        nrect = pygame.Rect(nx - self.radius, ny - self.radius, self.radius * 2, self.radius * 2)
        collided = any(nrect.colliderect(w) for w in WALLS)
        if not collided:
            self.x, self.y = nx, ny
        return collided

    def sense(self):
        """Scans left (-45 deg), ahead (0 deg), right (+45 deg) for obstacles."""
        dirs, res = {"left": -45, "ahead": 0, "right": 45}, {}
        for d, off in dirs.items():
            rad = math.radians(self.angle + off)
            ex, ey = self.x + math.cos(rad) * self.sensor_len, self.y + math.sin(rad) * self.sensor_len
            res[d] = any(w.clipline((self.x, self.y), (ex, ey)) for w in WALLS)
        return res

    def draw(self, surface):
        pygame.draw.circle(surface, BLUE, (int(self.x), int(self.y)), self.radius)
        rad = math.radians(self.angle)
        pygame.draw.line(surface, WHITE, (self.x, self.y), (self.x + math.cos(rad)*self.radius, self.y + math.sin(rad)*self.radius), 3)

# 3. HELPER FOR WORD WRAPPING TEXT
def render_wrapped_text(surface, text, font, rect, color=BLACK):
    words = text.split(" ")
    lines, current_line = [], ""
    for word in words:
        test_line = f"{current_line} {word}".strip()
        if font.size(test_line)[0] <= rect.width:
            current_line = test_line
        else:
            lines.append(current_line)
            current_line = word
    lines.append(current_line)
    
    y = rect.y
    for line in lines:
        if y + font.get_height() <= rect.bottom:
            surface.blit(font.render(line, True, color), (rect.x, y))
            y += font.get_height() + 2

# 4. MAIN LOOP
def main():
    pygame.init()
    screen = pygame.display.set_mode((WIN_W, WIN_H))
    pygame.display.set_caption("Grounded Language Micromouse")
    clock, font = pygame.time.Clock(), pygame.font.SysFont("Arial", 30, bold=False)
    mouse = Micromouse(50, 50)

    while True:
        for event in pygame.event.get():
            if event.type == pygame.QUIT:
                pygame.quit(); sys.exit()

        collided = mouse.update(pygame.key.get_pressed())
        sensors = mouse.sense()
        dist = int(math.hypot(mouse.x - GOAL.centerx, mouse.y - GOAL.centery))

        # Build rule-oriented grounded language output
        tokens = []
        if GOAL.collidepoint(mouse.x, mouse.y):
            tokens.append(f"{DICT['nouns'][2]} {DICT['verbs'][1]}")  # "goal reached"
        else:
            tokens.append(f"{DICT['nouns'][4]} to {DICT['nouns'][2]}: {dist}px")  # "distance to goal: Xpx"

        if collided:
            tokens.append(f"{DICT['nouns'][0]} collision {DICT['adj'][4]}")  # "wall collision blocked"
        
        # Tactical language for rule-based navigation AI
        blocked_dirs = [d for d, is_blocked in sensors.items() if is_blocked]
        if not blocked_dirs:
            tokens.append(f"{DICT['nouns'][1]} {DICT['adj'][3]}: {DICT['verbs'][3]} {DICT['adj'][0]}") # "path clear: move ahead"
        else:
            tokens.append(f"{DICT['nouns'][0]} {DICT['verbs'][0]} on " + " ".join(blocked_dirs))
            if sensors["ahead"]:
                suggested_turn = "right" if not sensors["right"] else ("left" if not sensors["left"] else "back")
                tokens.append(f"recommendation: {DICT['verbs'][2]} {suggested_turn}")

        # Render Scene
        screen.fill(WHITE)
        pygame.draw.rect(screen, GREEN, GOAL)
        screen.blit(font.render("GOAL", True, WHITE), (GOAL.x + 18, GOAL.y + 30))
        for w in WALLS: pygame.draw.rect(screen, BLACK, w)
        mouse.draw(screen)

        # Bottom Text Panel (White Background, Black Text, Word Wrapped)
        panel = pygame.Rect(0, MAZE_H, WIN_W, WIN_H - MAZE_H)
        pygame.draw.rect(screen, WHITE, panel)
        pygame.draw.line(screen, BLACK, (0, MAZE_H), (WIN_W, MAZE_H), 2)

        output_str = f"STATUS: [ {' | '.join(tokens)} ] | CONTROLS: Arrow Keys"
        render_wrapped_text(screen, output_str, font, pygame.Rect(10, MAZE_H + 8, WIN_W - 20, WIN_H - MAZE_H - 12))

        pygame.display.flip()
        clock.tick(60)

if __name__ == "__main__":
    main()

October 01, 2026

Understanding robotics in the past

Until around the year 2010, robotics projects were started with a certain bias which wasn't discussed but seen as mandatory. The philosophy was to see a robot as a closed system similar to a meccano windmill or a steam engine. All these systems are described by its internal working. In case of a windmill there is a mechanical mechanism and in case of a robot there is a computer program available. The assumption was that the power of a robot is located inside the machine encoded in an AI algorithm.

The only debate available was about the details of a closed system, for example which programming language is useful and which algorithm should be preferred. The AI engineers were interested in improving the robot itself because this principle was successful for mechanical machines and its also relevant for classic computer software. A certain software e.g. a graphics routine is optimized by improving the code. The code can become faster in terms of RAM consumption and CPU cycles. Also the lines of codes can be reduced so the machine is working more efficient.

Unfortunately, this paradigm failed for robotics applications. Its not possible to develop a robot with this bias. This is not a theoretical criticism but can be verified with a closer look into robotics software written before 2010. Many examples projects are available at github, some of them consists of 100k lines of code written in C++ with advanced algorithms, but the resulting robot can do nothing. It fails for simple tasks.

In a more harder language, robotics projects in the past were similar to non sense machines. It was cargo cult science. Even real software code was written in C/C++ and executed on a microcontroller, the robot wasn't able to do something. This problem was recognized by researchers in the past, but they didn't know how to improve the robots.

The criticism can be summarized to a single requirement: Robots in the past were not able to communicate with the environment. Even the robot was equipped with a microphone and sensors these hardware parts were not important for the software. Even more, the goal of robotics projects in the past was to avoid any communication with the environment because such a behavior has much in common with teleoperation which was seen as dead end.

AI before the year 2010 was trapped into a philosophy based on closed system plus computionalism. The thought model was a Turing machine without any external sensors. Such a closed system was recognized as best practice method for creating Artificial intelligence because it was fully understood by the software engineers.

One possible explanation for this self created closed system trap was the inability of computer science to describe the reality. Computer science is great in creating computer hardware and software but struggles in capture the environment of a robot. The reality is a highly complex, unspecified and non mathematical system which can't be compressed in algorithm or mathematical equation which creates a gap between the internal model of a robot and the external reality. This gap was first described by Rodney Brooks in 1990 but Brooks didn't mentioned how to overcome the gap.