Open systems for robotics

 Robotics in the past was organized with a closed system paradigm. A robot was described as a machine which consists of hardware, software and algorithms and the task for the programmer was to improve the internal mechanism of the robot. It was ignored that robots are communicating with the outside world. For example a robot might receives commmands by teleoperation and submits a status code to the operator. Such kind of interaction was mostly described as wrong path towards robotics because such a machine isn't autonomous anymore. There decision making isn't determined by the internal algorithm but from the outside which was seen as anti pattern in Artificial intelligence.

It takes decades until computer science has questioned the self created bias. Modern robotics is working as open system which means, that the robot gets information from sensors and from remote control. Also the robot interacts with human operators and is able to answer questions like "What object is visible in the camera?".

The transition from closed to open systems in robotics can be seen as an important innovation. In contrast to invent yet another path planning algorithm or program a robot control software in C/C++ the open system paradigm reformulates the goals of a robot system. It puts a higher importance on the robot's environment and allows the enviornment to take influence on the robot. There are many examples available in the history of robotics with this background, e.g. Braitenberg vehicle, kismet social robot and SHRDLU. These projects have demonstrated interactive robotics. There is always a robot and a human operator who interacts with the robot.

From a technical perspectives, interactive robotics is equal to teleoperation. Teleoperation was recognized by computer science as opposite to artificial intelligence, because the machine doesn't decide by itself but is guided by external human wisdom. So the maschine can't be called a robot anymore but has more in common with a RC Car.

The rejection of teleoperation makes sense on the first look. If a human operator is in charge to control the RC car, then no artificial intelligence is needed. Therefor it has nothing to do with thinking machines and is located outside of robotics. Only autonomous robots are intelligent robots.

With a modern perspective, Artificial intelligence isn't located inside of a robot but its the interface between a robot and its environment. Such an interface can become smart in the sense that the interface understands natural language.

Trash collecting robot with head up display

 

Robot control system with grounded language

There is no single AI algorithm available but its a pipeline with many substeps until a robot can do something usefull. Each single step is well understood by computer science in the past and the only thing missing is to combine all the steps into a single pipeline.

The staarting point is usually a teleoperated robot. The movements are converted into numerical sensor information called a mocap recording.  Then the motion capture information are converted into a text adventure. This translation step is perhaps the core element in artificial intelligence and has to do with grounded language. After a text adventure is available, the game is solved by a computer program which decides which step is executed next.

None of these substeps can be called advenced computer science. Even the automatical gameplay of text adentures like Zork can be mastered with mainstream software for example with a reinforcement learning algorithm. Also the translation from motion capture recording into a text adventure can't be called a demanding project. Nevertheless the entire pipeline is something new not realized before. A modern term for the entire system is "vision language action model" which is state of the art in robotics in the year 2026.

In general the described pipeline is an abstraction mechanism. It converts a large state space into a small state space. This small state space can be solved with a computer. In the past it was unclear how to do so, and therefor the assumption was the robotics problems are np hard. 

Line following roboter

 

Üblicherweise ist die Programmierung eines Line following roboters auf den Sourcecode fokussiert der in Java, Python oder C++ erstellt wird. Im folgende wird der Fokus auf die Mensch Maschine Kommunikation gelegt. Es wird ein semantischer Tagging space verwendet um die Linie auf dem Boden zu klassifizieren. Der Software für diesen Liniengenerator findet sich weiter unten und wurde in der Sprache python erstellt.

Die Hauptaufgabe der Software besteht darin, die dargestellte visuelle Szene innerhalb der 200x200 Pixel Karte in eine semantische Tagging beschreibung zu überführen. Diese Tagging beschreibung dient der Mensch Maschine interaktion. Der Roboter meldet die Tags zurück an den Menschen und dieser entscheidet was in der jeweiligen Situation zu tun ist. 

import pygame
import random

# --- Konfiguration ---
WIDTH, HEIGHT = 800, 600
BOX_SIZE = 200
WHITE = (255, 255, 255)
BLACK = (0, 0, 0)
RED = (200, 0, 0)  # Hindernis
GRAY = (200, 200, 200) # Box-Rahmen

# --- Moegliche Parameter ---
paths = ["vorwärts", "links", "rechts", "sackgasse", "Kreuzung"]
thicknesses = {"normal": 10, "dick": 20}
breaks = [False, True]
obstacles = [False, True]

# --- Pygame Setup ---
pygame.init()
screen = pygame.display.set_mode((WIDTH, HEIGHT))
pygame.display.set_caption("Roboter Linien-Generator (Space für neue Karte)")
font_small = pygame.font.Font(None, 24)
font_large = pygame.font.Font(None, 36)

def generate_random_config():
    return {
        "path": random.choice(paths),
        "thickness_name": random.choice(list(thicknesses.keys())),
        "break": random.choice(breaks),
        "obstacle": random.choice(obstacles)
    }

def draw_text(screen, text, font, color, x, y, align_center=False):
    text_obj = font.render(text, True, color)
    text_rect = text_obj.get_rect()
    if align_center:
        text_rect.center = (x, y)
    else:
        text_rect.topleft = (x, y)
    screen.blit(text_obj, text_rect)

def draw_card(config):
    screen.fill(WHITE)
    
    # 1. Box berechnen (Zentrum des Bildschirms)
    box_rect = pygame.Rect((WIDTH//2 - BOX_SIZE//2, HEIGHT//2 - BOX_SIZE//2), (BOX_SIZE, BOX_SIZE))
    pygame.draw.rect(screen, GRAY, box_rect, 1) # Rahmen der Box
    
    center_x, center_y = box_rect.center
    half = BOX_SIZE // 2
    
    # Parameter
    path = config["path"]
    thickness_val = thicknesses[config["thickness_name"]]
    has_break = config["break"]
    has_obstacle = config["obstacle"]

    # 2. Pfade definieren (Relativ zur Box)
    # Startpunkt ist immer unten in der Mitte der Box
    start_p = (center_x, center_y + half)
    mid_p = (center_x, center_y)
    
    lines = [] # Liste von (Start, Ende) Paaren
    
    if path == "vorwärts":
        lines.append((start_p, (center_x, center_y - half)))
    elif path == "links":
        lines.append((start_p, mid_p))
        lines.append((mid_p, (center_x - half, center_y)))
    elif path == "rechts":
        lines.append((start_p, mid_p))
        lines.append((mid_p, (center_x + half, center_y)))
    elif path == "sackgasse":
        lines.append((start_p, (center_x, center_y + 10))) # Kurzes Stück
    elif path == "Kreuzung":
        lines.append((start_p, (center_x, center_y - half)))
        lines.append(((center_x - half, center_y), (center_x + half, center_y)))

    # 3. Zeichnen mit optionaler Unterbrechung
    for s, e in lines:
        if has_break:
            # Zeichne nur das erste und letzte Drittel der Teil-Linie
            m1 = (s[0] + (e[0]-s[0])//3, s[1] + (e[1]-s[1])//3)
            m2 = (s[0] + 2*(e[0]-s[0])//3, s[1] + 2*(e[1]-s[1])//3)
            pygame.draw.line(screen, BLACK, s, m1, thickness_val)
            pygame.draw.line(screen, BLACK, m2, e, thickness_val)
        else:
            pygame.draw.line(screen, BLACK, s, e, thickness_val)

    # 4. Hindernis (Falls ja, immer am Mittelpunkt der Box)
    if has_obstacle:
        pygame.draw.rect(screen, RED, (center_x - 15, center_y - 15, 30, 30))

    # 5. Semantische Beschreibung
    desc_str = f"Linienweg: {path} | Liniendicke: {config['thickness_name']}"
    extra_str = f"Unterbrechung: {'ja' if has_break else 'nein'} | Hindernis: {'ja' if has_obstacle else 'nein'}"
    
    draw_text(screen, desc_str, font_small, BLACK, WIDTH//2, HEIGHT//2 + half + 40, True)
    draw_text(screen, extra_str, font_small, BLACK, WIDTH//2, HEIGHT//2 + half + 65, True)
    draw_text(screen, "Drücke LEERTASTE für neue Karte", font_large, BLACK, WIDTH//2, 50, True)

# --- Hauptschleife ---
current_config = generate_random_config()
draw_card(current_config)

running = True
while running:
    for event in pygame.event.get():
        if event.type == pygame.QUIT:
            running = False
        if event.type == pygame.KEYDOWN:
            if event.key == pygame.K_SPACE:
                current_config = generate_random_config()
                draw_card(current_config)

    pygame.display.flip()

pygame.quit()
 

Line following robot with grounded language

 

Mögliche Worte sind nach Kategorien sortiert. Darüber erfolgt die Kommunikation zwischen Mensch und Maschine. Die Software hat lediglich die Aufgabe die Kommunikation sicherzustellen:

- navigation (folgen, suchen, andocken, ignorieren, u_turn)
- line (junction, curve, end)
- attribute_speed (slow, fast)
- attribute_color (black, white)
- attribute_thick (thin, thick)
- feedback (lost_line, multiple_paths, pathblocked)

Making robots more chatty with minsky frames

Autonomous robots in the past were mostly silent systems which aren't talking but processing information. This makes it hard to debug the AI software.

The screenshot shows an alternative which is a very chatty maze robot. His task is to move around and recharge its battery when its empty. The AI brain consists of mulitiple Minsky frames with key/value information. Technically it was realized as python dictionary shown in text overlay windows.

The surprising situation is, that even such a minimal robot game consists of huge amount of information. There are raw sensor data itself like the position but there are also semantic information like the location in the map and the planned actions.

Creating a Minsky frame itself is not very complicated because its a normal python dictionary. What makes the datastructure powerful is, that frames are written information. They are stored in a database and this allows to add information and translate the existing Minsky frames into new information. For example the planned actions of the robot can only be determined if the existing frames are available which are analyzed.

In other word, the AI isn't a list of algorithms, but the AI is a database distributed over hierarchical key/value data.

Robot control with a DIKW pyramid

Symbol grounding is about moving down and moving up along a dikw pyramid. This allows to hide the details and expand the details of a subject. For the example of a warehouse robot the dikw pyramid can be implemented as a python dictionary which shows only the upper layer and the bottom layer:

dikw_pyramid={
  "wisdom": {
    "Go to the loading bay and clear the blockage.",
  },
  "data": { 
    "lidar_dist": 0.5, "weight_kg": 25.0, "coords": (12.4, 45.8)
  },
}

The raw sensor data are feed into the data layer and are formmated as numerical values. In contrast the wisdom layer of the pyramid stores the voice commands formulated in English sentences. The task for the symbol grounding engine is to translate between these layers. This is realized by instruction following (from top to bottom) and activitity recognition (from bottom to top).

Robot swarm builds a house

 

The picture shows multiple robots on a construction site who are controlled by a large language model over a longer time span on the same goal. The AI technology is based on natural language for reducing the state space drastically. The Large language model describes the project in English nouns and verbs and the single robots are converting the commands into physical action.

Improved chatbot for a kitchen robot

In addition to the previous post, the python script was improved a bit. There are more entries in the database, the amount of informaiton is higher, and very important a telemetry mapping function is available. This allows to monitor a teleoperation robot. The amount of codelines was increased to 80 but the software remains easy to understand.

The core element is a database with words. Every word is described with additional key-value informaiton for example a picture or a position. The AI takes the current sensory data and searches for a match in the database and the AI also searches for a text input from a user. If the AI has found an entry in the database its equal to understand a situation. In short, the AI is a database lookup algorithm. Here is an example interaction and of course the source code written in Python3.

----
gathering telemetry ...
attention near apple
robotpos near table
user: lookat table
search database ...
lookat action inspect object
table {'pos': (0, 0), 'desc': 'place for storing objects', 'word': 'noun'}

gathering telemetry ...
attention near apple
robotpos near table
user: grasp apple
search database ...
grasp action take an object
apple {'pos': (10, 3), 'word': 'noun', 'category': 'fruit', 'desc': 'is food to eat', 'filename': 'apple.jpg'}
----

"""
chatbot kitchen robot
a wordlist is stored as python dictionary, user enters command which is searched in the wordlist
application: Teleoperation monitoring
"""
class Chatbot:
  def __init__(self):
    self.data={
      # verb
      "open": "action open something",
      "grasp": "action take an object",
      "ungrasp": "action place object from hand to world",
      "eat": "action eat food",
      "lookat": "action inspect object",
      "walkto": { 
        "word": "verb",
        "category": "action",
        "desc": "move towards location",
        "motor": "legs",
      },
      # noun
      "apple": {
        "pos": (10,3),
        "word": "noun",
        "category": "fruit",
        "desc": "is food to eat",
        "filename": "apple.jpg",
      },
      "banana": {
        "desc": "noun food",
      },
      "table": { 
        "pos": (0,0),
        "desc": "place for storing objects",
        "word": "noun",
      },
      "fridge": {
        "pos": (1,0),
        "word": "noun",
        "status": "closed",
        "category": "furniture",
      },
      "plate": "noun food is served there",
      "door": "noun entrance to room",
    }
    self.telemetry()
    self.parser()
  def getdist(self,p1,p2): # return: manhattan_distance
    result=abs(p1[0]-p2[0])+abs(p1[1]-p2[1])
    return result
  def telemetry(self):
    self.sensor={
      "robotpos": (0,1),
      "camera": "cam02.jpg",
      "attention": (10,3),
    }
    # search robotpos and attention
    print("gathering telemetry ...")
    for i in self.data:
      if "pos" in self.data[i]:
        dist=self.getdist(self.sensor["robotpos"],self.data[i]["pos"])
        if dist<=1:
          print("robotpos near",i)
        dist=self.getdist(self.sensor["attention"],self.data[i]["pos"])
        if dist<=1:
          print("attention near",i)
  def parser(self):
    line=input("user: ") # manuel input
    line=line.split()
    print("search database ...")
    for i in line:
      if i in self.data:
        print(i,self.data[i])
      else:
        print(i,"not found")
    

if __name__ == '__main__':
  c=Chatbot()

Programming a symbol grounding engine

The core element is a database which consists of flashcards. Entries in the database are natural language words for [apple, banana, plate, table, ...] and also for verbs like [grasp, walkto, use, ungrasp, ...]. The symbol grounding engine works like a parser for a text adventure: a certain input on the terminal like "grasp apple" is matched with the database. The found entries in the database are extracted and converted into action signals for the robot hand.

In other words, there is no AI algorithm needed, but there is a word database. The database ensures that the computer understands natural language commands like "walkto table, locate apple, grasp banana".

Python for event detection of a warehouse robot

 The Python programming language is recognized as a high level language which allows to program complex software in a low amount of code. The reason is, that python has many builtin libraries and has an easy to understand syntax for parse. Its much easier to create a prototype in Python than in other programming languages like C or Assembly.

Unfurtunately, the Python language remains a classical imperative language which means, that the programmer has to define functions, classes and variables. This formal syntax ensures, that a computer can execute the statements. To increase the abstraction level further another syntax is needed which is presented next. The goal is to program an event recognition system for a warehouse robot. Instead of implementing a GUI prototype the goal is to write only a list of [tags] which can be detected by the robot. The tags are described in a python dictionary:

warehouse_robot_taxonomy = {
    # --- Navigation & Localization ---
    "NAV_WAYPOINT_REACHED": "Robot base has arrived at the specific coordinate destination.",
    "NAV_PATH_OBSTRUCTED": "LIDAR/Depth sensors detect an unexpected object in the path.",
    "NAV_RELOCALIZATION_REQ": "Robot uncertainty in pose exceeds threshold; seeking landmarks.",
    "NAV_FLOOR_HAZARD": "Detection of liquid spills, debris, or uneven surfaces.",

    # --- Manipulation & Payload ---
    "MAN_SHELF_ALIGNED": "End-effector is centered and parallel to the targeted rack slot.",
    "MAN_GRIP_SUCCESS": "Tactile/Force sensors confirm object acquisition.",
    "MAN_GRIP_SLIP": "Loss of contact or shifting weight detected during transport.",
    "MAN_LOAD_SHIFT": "Internal IMU detects payload instability while the robot is moving.",

    # --- Perception & Identification ---
    "PRC_SKU_VALIDATED": "Barcode, QR code, or RFID successfully read and matched to manifest.",
    "PRC_MISPLACED_ITEM": "Vision system identifies an object where the database expects a void.",
    "PRC_SHELF_FULL": "The destination bin has no available volume for placement.",

    # --- Safety & Human-Robot Interaction ---
    "SAF_ESTOP_ACTIVE": "Physical or software-based emergency stop has been engaged.",
    "SAF_HUMAN_NEAR": "Safety scanners detect a human worker within the 'Slowdown' zone.",
    "SAF_COLLISION_IMMINENT": "Time-to-collision calculation triggers immediate braking.",

    # --- System & Maintenance ---
    "SYS_BATTERY_LOW": "Charge level requires return to docking station.",
    "SYS_COMMS_LOST": "Loss of heartbeat or high latency with the Warehouse Management System (WMS)."
}

Such a list of tags has not much in common with a software project, but its similar to a database. The table stores two columns: name, description. The dictionry stores the items in the table.

The idea behind the project is, to annotate the sensory perception with one of the tags. For example, if the battery is low the tag [SYS_BATTERY_LOW] gets activated or if the robot has scanned an object the tag [PRC_SKU_VALIDATED] gets activated. The entire game state is projected towards a tag vector with 16 entries, each of the tags can be true or false so the game state is encoded in 16 bits, which is a very compact representation. The python dictionary ensures, that the human operator has a better understanding of each of the tags. There is a name plus a descripotion given.

language games in recent robotics

 Every machine contains of an internal mechanism which can be explained from a scientific perspective. A steam engine is driven by combustion, a computer works with electricity and a robot also have an internal driving force. In the science fiction world, the core mechanism of a robot is sometimes an AI chip which enables the robot to think. A rough estimation is, that robots in the reality also have a chip or a graphics processing unit which is very expensive and makes the robot move and think. Unfurtunately, this hardware oriented explanation is wrong. Modern robotics has no AI chip.

The next possible explanation is, that a robot is driven by a software architecture, for example an algorithm or a robot control software. Such a computer program would be the core element and enables the robot to take decision. Unfurtunately, the hint with a software architecture is also wrong. Modern robotics doesn't require a dedicated firmware nor an operating system.

If hardware and software both is not the explanation for the artificial intelligence inside a robot there are not much alternative explanations available. At the same time, recent robotics has demonstrated remarkable skills like biped walking anb dexterous object grasping so there must be a mechanism available which explains the internal working. The mechanism is a bit hidden. Its not located inside the robot torso but its outside of the robot. Or to be more specific, the driving force behind modern robotics are language games.

A language game is an activity played with a set of rules. Typical games might be chess or 4 in a row. A language game is a certain category of a game which is working interactively and by using words. Language games are the driving force behind recent robotics. A certain robot for example a biped robot, implements a language game. The language game defines also the limitation of a robot. For example if the game is about navigating in a warehouse scenario, the robot can do only this single task.

The problem with games and especially with language games is, that they can't located in the reality very well. A game doesn't need a cpu and no certain software program, but a game is an abstract idea described in a document. For example the game of chess can be implemented in different physical chess boards which might have 10cm width, 14 cm width or it can be implemented in a video game. The same situation is available for language games. A certain speaker to hearer dialogue game can be implemented on different computer hardware with different algorithms. THe only fixed element is the game itself.

Even if games can't be located physically inside a robot, they are part of the reality. Abstract ideas are usually described in books, and books are located in a library. In a library there are many books available about board games, card games, word puzzles and language games for robots. These books are the single explanation why modern robotics is working.

Rückblick auf die World Robot Conference 2025 in China

Vom 8. Aug  bis 12. Aug 2025 wurde die World Robot Conference 2025 in China Beijing durchgeführt. Wie auch die Veranstaltung im Jahr 2024, welche ebenfalls in China durchgeführt wurde, kamen sehr viele Besucher zum weltweit größten Robotik-Event. Der Veranstalter spricht von 1.3 Millionen Besuchern die sich über 5 Tage lang die neueste Technologie angeschaut haben. Ausgehend aus den publizierten Youtube Videos, die allesamt sehr viele Menschen zeigen, könnte diese Zahl halbwegs realistisch sein.

Der Unterschied zwischen der chinesischen Robotik Veranstaltung und ähnlichen Veranstaltungen aus den USA liegt vor allem in der Quantität. Finden sich bei der Robocup Verstaltung nur eine Handvoll von Robotern, sind die Hallen in Beijing vollgestopft mit hunderten wenn nicht sogar tausenden von Robotern aus allen möglichen Bereichen. So gibt es Industrieroboter, die Gegenstände von einem Fließband aufnehmen, es gibt Roboterhunde auf 4 Beinen, humanoide Roboter auf 2 Beinen, es gibt Roboter die Popcorn servieren, andere die Eiscreme oder Kaffee zubereiten und Roboterarme die Bilder zeichnen. Es gibt Roboter die boxen und welche, die Produkte aus einem Regal herausnehmen. Was es jedoch nicht zu sehen gibt (jedenfalls nicht in den youtube videos) sind UAVs, also fliegende Roboter die in der Halle herumflattern.

Der Publikumsmagnet waren eindeutig zweibeinige balancierende Roboter. Insgeheim wartet das Publikum wohl darauf, dass einer der metallischen Laufmaschinen das Gleichgewicht verliert und dann jemand aus dem Backstage Bereich hervoreilt, um die Vorführung zu unterbrechen. Tatsächlich kam es zu seltenen Vorfällen dieser Art, aber anders als beim berühmten Treppensturz von Asimo in 2006 verfügen heutige Roboter über eingebaute Fehlerroutinen. Wenn ein Roboter umfällt wird dies durch die Elektronik erkannt und es wird eine Aufsteh-Routine gestartet. Gesteuert werden die meisten dieser fortschrittlichen Roboter übrigens mittels Fernbedienung die Ähnlichkeit hat mit einem Gamepad einer Playstation Konsole.

Zum Schluss ein Vergleich mit anderen bekannten Roboterveranstaltungen:

- Robocup Veranstaltung wird von ca. 100k Besuchern frequentiert
- die IROS Robotik Konferenz hat 10k Besucher
- Automatica München hat 49k Besucher

Impact of AI towards society

 Instead of predicting future development, the following essay describes only the past which is well documented. Most attempts in the past to build robots and develop artificial Intelligence has failed. A famous large scale project was realized by Cycorp in the mid 1980s [1]. The impact for the society was nearly zero. No valuable product was generated by Cyc. Another larger project was company "Helpmate robotics" from the mid 1990s. The goal was to build hospital robots which deliver food. Even some of these robots were used in reality, the company went bankrupt and the customers were not satisfied.

Last but not least, the "Rethink Robotics" company should be mentioned which was founded in 2008. The most famous product of the company was Baxter which was an industrial robot with a smiling face. The amount of sold units were low and the company went bankrupt after a while.

So we can say, that the attempts to build robots in the mid 1980s until 2010s were not very successful. Not a single human workers were replaced by these AI systems and the amount of sales was very low. There are many reasons available for this failure, for example the immature technology, a public how doesn't understand robotics, and management failures by the founders of the AI companies. 

[1] Cycorp https://yuxi.ml/cyc/

Wie die Kommerzialisierung die Seele der Amateur-Robotik fraß

 Erinnern Sie sich noch an die Zeit, als die Robotik-Szene ein lebendiges Biotop von Enthusiasten war? In staubigen Garagen und muffigen Kellerräumen werkelten Tüftler an ihren selbstgebauten Arduino-Robotern. Stolz wurden die neuesten Kreationen in lokalen Robotik-Clubs präsentiert – einfache Linienverfolger, wackelige Greifarme, gesteuert mit ein paar Zeilen selbstgeschriebenem Code. Hier ging es um die pure Freude am Entdecken, am Ausprobieren, am gemeinsamen Lernen. Und oft gab es spannende Vorträge, die auch mal in die aufregenden Gefilde der Künstlichen Intelligenz abdrifteten, präsentiert von Gleichgesinnten, die ihre neuesten Erkenntnisse teilten.

Diese Ära der unkommerziellen, gemeinschaftsgetriebenen Robotik scheint jedoch langsam zu verblassen. Was ist passiert? Die Antwort ist, wie so oft, die Kommerzialisierung.

Heute dominieren Giganten den Markt. Multimilliarden-Dollar-Unternehmen entwickeln hochkomplexe Roboter für industrielle Anwendungen, für die Logistik oder sogar für den persönlichen Gebrauch – Produkte, die die bescheidenen Arduino-Kreationen von einst wie Spielzeug aussehen lassen. Diese Unternehmen verlegen Hochglanz-Bücher über Robotik und KI, produzieren aufwendige Online-Vorlesungen über neuronale Netze, Motion Planning und die neuesten multimodalen Roboter. Die Informationsflut ist enorm, die Qualität oft exzellent.

Doch wo bleibt dabei der Raum für den Amateur, für denjenigen, der einfach nur aus Leidenschaft an Robotern basteln und sich mit anderen austauschen möchte? Die einstigen Robotik-Clubs, die von Idealismus und dem Wunsch nach Wissensaustausch getragen wurden, scheinen im Schatten der kommerziellen Angebote zu verkümmern.

Die Gründe dafür sind vielfältig:

* Überwältigende Professionalität: Die schiere Komplexität und Perfektion der kommerziellen Roboter kann entmutigend wirken. Der Sprung vom einfachen Arduino-Projekt zum Verständnis und zur Nachahmung modernster Robotik scheint riesig.
* Fokus auf Konsum statt Kreation: Die Kommerzialisierung fördert den Konsum. Statt selbst zu entwickeln, wird das fertige Produkt gekauft. Die Freude am eigenen Schaffen, am Lösen von Problemen mit begrenzten Mitteln, geht verloren.
* Dominanz der Online-Lehre: Während Online-Vorlesungen und -Kurse zweifellos wertvoll sind, fehlt ihnen oft die persönliche Interaktion, der informelle Austausch und die Möglichkeit, gemeinsam an realen Projekten zu arbeiten, die die lokalen Clubs auszeichneten.
* Verlagerung des Interesses: Das Interesse an Robotik und KI ist zwar enorm gewachsen, doch der Fokus hat sich oft von der praktischen, hands-on Erfahrung hin zum theoretischen Verständnis und der Bewunderung der kommerziellen Produkte verschoben.

Das ist nicht per se schlecht. Der Fortschritt in der Robotik und KI ist beeindruckend und die breite Verfügbarkeit von hochwertigen Informationen ist ein Segen. Dennoch geht mit dem Verschwinden der unkommerziellen Robotik-Clubs etwas Wertvolles verloren:

* Die niedrigschwellige Zugänglichkeit: Hier konnten Anfänger ohne Vorkenntnisse und mit geringem Budget erste Schritte in die Welt der Robotik wagen.
* Die Gemeinschaft und der Austausch: Das gemeinsame Tüfteln, das Teilen von Misserfolgen und Erfolgen, die gegenseitige Inspiration – all das schuf eine einzigartige Lernumgebung.
* Die spielerische Entdeckung: Ohne den Druck kommerzieller Verwertbarkeit stand die reine Neugier und der Spaß am Experimentieren im Vordergrund.

Es ist an der Zeit, sich zu fragen, wie wir diese Kultur der Amateur-Robotik wiederbeleben können. Brauchen wir neue Formen von Gemeinschaften, die sowohl die Möglichkeiten der modernen Technologie nutzen als auch den Geist der ursprünglichen Robotik-Clubs bewahren? Vielleicht Online-Plattformen, die lokale Treffen und gemeinsame Projekte fördern? Oder Initiativen, die den Zugang zu Ressourcen und Wissen für Hobbybastler erleichtern?

Die Kommerzialisierung hat zweifellos die Robotik und KI revolutioniert. Aber wir sollten nicht vergessen, wo diese Reise ihren bescheidenen Anfang nahm: in den Werkstätten und Kellern von passionierten Amateuren. Es wäre schade, wenn dieser Geist der Entdeckung und des gemeinschaftlichen Schaffens ganz verloren ginge.

Was denken Sie? Haben Sie ähnliche Erfahrungen gemacht? Gibt es noch lebendige Beispiele für unkommerzielle Robotik-Initiativen? Teilen Sie Ihre Gedanken in den Kommentaren!

A chatbot which can grasp an apple

[2025-02-18 13:45:00] Human: Hello robot, can you help me with a task in the kitchen?

[2025-02-18 13:45:05] Robot: Hello! Of course, I'd be happy to assist you with a task in the kitchen. What would you like me to do?

[2025-02-18 13:45:15] Human: I need you to grasp an apple for me.

[2025-02-18 13:45:20] Robot: Certainly, I can help you grasp an apple. Let me process that request:
1. Locating the kitchen
2. Identifying an apple
3. Planning the grasping motion

Could you please confirm if there's a specific location in the kitchen where I should look for the apple?

[2025-02-18 13:45:35] Human: The apple should be in the fruit bowl on the counter.

[2025-02-18 13:45:40] Robot: Thank you for the additional information. I'll proceed with the task:
1. Navigating to the kitchen counter
2. Locating the fruit bowl
3. Identifying the apple among other fruits
4. Planning and executing the grasping motion

I'll narrate my actions as I perform them. Please let me know if you need me to do anything differently.

[2025-02-18 13:45:50] Human: That sounds good. Please proceed.

[2025-02-18 13:45:55] Robot: Understood. I'm starting the task now.

1. Navigating to the kitchen counter.
2. I've located the fruit bowl on the counter.
3. Scanning the contents of the bowl... I've identified several fruits, including an apple.
4. Planning the optimal grasping motion for the apple.
5. Extending my arm and adjusting my hand position.
6. Gently grasping the apple.
7. Lifting the apple from the bowl.

I've successfully grasped the apple. Would you like me to hand it to you or place it somewhere specific?

[2025-02-18 13:46:25] Human: Please hand it to me.

[2025-02-18 13:46:30] Robot: Certainly. I'm now:

1. Turning towards your location.
2. Extending my arm to offer you the apple.
3. Waiting for you to take it.

Please take the apple whenever you're ready. Once you've taken it, I'll release my grip.

[2025-02-18 13:46:45] Human: I've taken the apple. Thank you!

[2025-02-18 13:46:50] Robot: You're welcome! I'm glad I could assist you. I've released my grip on the apple now that you've taken it. Is there anything else you'd like me to help you with in the kitchen or elsewhere?

[2025-02-18 13:47:00] Human: No, that's all for now. Thank you for your help.

[2025-02-18 13:47:05] Robot: You're welcome! I'm happy I could help you with grasping the apple. If you need any assistance in the future, don't hesitate to ask. Have a great day!

Von pseudo Robotern zu echten Robotern

 

Wie man einen Roboterarm fernsteuert ist seit mindestens den 1980er Jahren bekannt. Ein menschlicher Bediener bewegt einen Joystick, dessen Signale werden über ein Kabel an einen Robterarm übertragen und dieser bewegt sich dann. Der Grund warum diese technische Aparatur nicht als echter Roboter gilt ist weil der menschliche Bediener die ganze Zeit über anwesend sein muss und auch nur den einen Roboterarm zeitgleich steuern kann, aber nicht 2 oder noch mehr.

Um echte Robotik zu realisieren, die ohne menschlichen Bediener auskommt, muss man etwas tiefer in den Methodenkoffer der Informatik hineingreifen und dort nach Sprachinterface Ausschau halten. Insbesondere in ihrer Schriftlichen Form kann damit Automatisierung erzielen. Man notiert die Befehlssequenz in einer Textdatei ähnlich wie ein Computer program und führt diese dann aus. Während der Ausführung muss kein menschlicher Bediener anwesend sein. Je abstrakter die Sprache gewählt wurde, desto einfacher ist die Steuerung der Robotern. Man notiert in dem Program beispielsweise: “greife den Apfel, das wiederhole dann 100x” und schon hat man einen Subtask automatisiert.

Der Grund warum dieses Prinzip bis ungefähr 2010 nicht in der Robotik eingesetzt wurde hat einen simplen Grund. Es gab früher keine Sprachinterfaces die leistungsfähig genug waren. Es gab zwar einige Experimente wo über Sprachkommandos Roboter gesteuert wurden, aber es war unklar wozu das nützlich sein könnte.

Die Sprachsteuerung von Robotern hat einen winzigen Nachteil: und zwar werden Roboter dadurch sehr menschlich. Es sind keine simplen Automaten oder Rechenmaschinen mehr die elektrische Impulse weiterleiten und Zahlen aufaddieren sondern künftige Roboter werden die solben Worte verstehen wie Menschen auch, also Vokabeln wie “schnell, stop, links, rechts, greife, Apfel, Banane, Tisch” usw. Dadurch geht der Unterschied zwischen Mensch und seiner Technologie verloren. Anders als die Fähigkeit eine mathematische Rechnung auszuführen ist die Fähigkeit Substantive und Verben zu verstehen ein sicheres Zeichen für intelligenz. Computer wie sie früher verbreiteet waren waren dazu nicht imstande. Es waren schnellere Taschenrechner die über eingebaute Gleitkommaarithmetik verfügten aber nicht über Sprachprozessoren verfügten.

Es gibt einen großen Unterschied zwischen menschlicher und maschineller Sprachverarbeitung. Wenn Computer Sätze parsen geht das tausendmal schneller. Ein Computer muss nicht 2 Wochen einplanen um einen Roman zu lesen sondern Computer erledigen diese Aufgabe in unter 1 Minute. Diese messbare Beschleunigung ist ein sicheres Anzeichen für technischen Fortschritt weil es bedeutet, dass Menschen ihre wichtigste Fähigkeit automatisiert haben.

Pipeline for developing chatbot based robotcs

 Since the introduction of large language models in the year 2023, the term "artificial intelligence" was defined very clearly. AI means simply that the user interacts with a chatbot in natural language. The chatbot is able to answer questions, can draw pictures and also the chatbot controls a robot.

From the user perspective, the software works suprisingly easy to explain. The user enters a sentence like "open left hand" and submits the text to the chatbot. The chatbot is parsing the sentence and executes the command, which means that the robot will open indeed the hand. More complex actions like moveing to a table and wash all the dishes are the result of more advanced text prompts which contains a list of sub actions.

The unsolved issue is how to program such advanced chatbots in software. Before a user can talk to a robot this way, somebody has to program the chatbot first which can be realized in C++, Java, and so on. The basic element of any chatbot isn't a certain programming library like nltk and its not a certain operating system like Windows or Unix, but the needed building block is a dataset. Or to be more specific, a dataset which maps language to perception, and language to action.

Such a mapping is realized with multiple column because the dataset is always a table. In the easiest example the table consists of a images in the first column and the nouns in the second column:

[picture1.jpg], apple
[picture2.jpg], banana
[picture3.jpg], table
[picture4.jpg], spoon

The chatbot software is using such a dataset to understand a text prompt from the user. For example if the user types into the textbox "take apple". The word apple is converted into [picture1.jpg] and this picture allows to find the apple with the camera sensor.

More complex interactions like generating entire motion data are provided the same way. A verb like "grasp" is converted into motion capture trajectory with the following table:

[trajectory1.traj], open
[trajectory2.traj], grasp
[trajectory3.traj], standup
[trajectory4.traj], sitdown
[trajectory5.traj], moveto

Let me give a longer example to make the point clear. Suppose the user enters the command "moveto table. grasp apple". This command sequence is converted into:
1. [trajectory5.traj], moveto
2. [picture3.jpg], table
3. [trajectory2.traj], grasp
4. [picture1.jpg], apple

In the next step of the parsing pipeline the given jpeg images and .traj data are converted into search patterns and motion pipelines. This allows to convert a sentence into robot actions.

There are mutiple techniques available how to program a chatbot in detail. Its possible to use ordinary programming languages or more advanced deep neural networks. What these methods have in common is, that they are requiring always a dataset in the background. Somebody has to create a table with pictures with objects, and annotate the pictures. Also a dataset with mocap data is needed. Such a dataset allows a chatbot to convert a short sentence into something meaningful. Meaning is equal to a translation task from a word into a picture, and from a word into a trajectory.

So we can say, that a chatbot is the frontend of an AI while the dataset is the backend.

There is no AI singularity yet

The main criteria for an upraising of robotics is mass production of intelligent machines. If advanced biped robots are created with only 1 units it is a boring research-only robot. Such a single unit robot has no relevance to the pubiic but it is a demonstration only project to write a paper about it.

True robotics is equal to increase to amount of units drastically. If millions of customer around the world are motivated to buy a certain product, that it is competitive. In case of robotics the situation is relaxed. The amount of cobots shipped for automation reason is only 45k units worldwide. The amount of miles driven by self driving cars is also very low.

Other household robots like vacuum cleaners and kitchen robots are not produced on a mass scale. Despite its attention on computer fairs the public demand for such technology not there.

Let us imagine who an AI takeover will look like without any real robots. It is a paradox situation because it is the opposite. An AI takeover described in the literature means usually that millions of human robots are produced on a mass scale so every household own at least one. Such a scenario is unrealistic. What is available instead are early prototypes similar to mechanical automaton in the 18th century. These robots are demonstrating the technical skills of university and research institutions but they are created with only 1 unit. This is similar to the famous fictional robot in Star Trek TNG which was also produced in a low amount of units.

What is available today are mass produced classical electronics devices lie smartphone, laptops and watches. But this technology can't be described as intelligent but it is normal pre-singularity innovation.

The opeq question is where robots are not mass produced yet? To answer the question lets take a look at the previously mentioned cobots. Cobots are industrial robots which are more powerful than normal automation technology. In most cases cobots are used for intermediate pick&place tasks at the conveyer. in contrast to a famous myth such a task has a low priority in the industry. Even if cobots are working great they are not used frequently. The amount of annually shipped cobots is 45k worldwide whcih includes all brands and all sort of cobots. so it ic compared to the importance of the industry similar to nothing.

If cobots are not used in the reality, how exactly are the conveyor belts automated? Right it is a rhetorical question. In most cases normal automation technique is used which doesn't need Artificial Intelligence but it is working purely mechanically. And the remaining tasks are highly complex and can't be automated soon. So we can say that the industry struggles even with easy to built cobots. It is unlikely that more advanced human robots are produced on a mass scale soon. Let us make a prediction in numbers who the year 2030 will look like.

the amount of cobots will grow by 5% annually. That means around 65k cobits are getting produced in the future per year. Not a single human worker is getting replaced by this little amount of technology. The chance is high that many of these cobots not even installed on a production facility. The same prediction can be made for self driving cars. The chance is high that because of regulation problems existing autopilots will remain offline and new cars are produced without any sort of AI. The only automation technique which is produced on mass-scale is an automatic door opener. That means, the owner can press a button and the car will unlock the door even if the owner is 1 meter away.

Such an easy to describe outlook sounds a bit boring for an audience which is fear that the robot revolution has already started. Most of the so called Singularity is only available in science fiction literature. The reality is much more conservative in introducing AI technology.

Programming exercises to understand robotics

 AI and robotics is mostly described as programming technique or as an algorithm. The question is which sort of program provides AI, or which sort of AI library is available? The surprising situation is, that AI is located in a different position. It has to do with with a programming exercise.

Some typical Non AI programming exercises are:
- "Write a software in python which prints out 6 randomly generated numbers"
- "Write a python program which plots a line on the screen"
- "write a java program which adds two numbers and prints the result on the screen"

These exercises are used to a teach programming and a certain programming language at the university. It depends on the student how to solve it. AI and robotics is some sort of advanced programming exercises which can be labeled as ultra-hard. The question is which sort of AI programming exercise is available? A possible challenge is given next.

There is a robot in a maze which can be controlled with the keyboard. The task is to write a grounded sensor for the robot which contains of 6 elements:
[xpos,ypos,distancefront, distanceleft,distanceright,distancetoenergy]

The sensor array should be printed to the screen all the time.

Such a programming exercise fits into the same category like "write a program which prints out all the prime numbers" because it formulates a problem which can be solved with an algorithm. The task for the student is to understand the problem, write a short program in Python or Java and then it can be determined if the software solves the task.

From an abstract point of view it is important to ask which sort of programming exercises are needed to explain the subject of robotics. What all these challenges have in common is, that not a certain algorithm is needed but a certain exercise. The formulated problem with the sensor array doesn't contain of program code nor an algorithm. But is a figure plus a text which formulates a problem. It is up to the opponent (the student) to provide an answer to the problem. The answer is written in a certain proramming language and will contain of an algorithm. In the concrete example, the typical answer will take the requested 6 elements from the underlying physics engine and in case of the distance value it has to be calculated from scratch. Then a print routine is needed to show the result on the screen.