Robotics and Artificial Intelligence
July 22, 2026
Grounded language in open systems
The box on the left is the human who describes the reality with natural language. The box on the right is the environment which can be perceived with sensors. Symbol grounding is the connection between both boxes.
From a system perspective the 2 box system is an open system because both boxes are connected to each other. Natural language from the left box is referencing to physical objects in the right box, while perceived reality in the right box gets described with English words in the left box.
The assumption is, that there are 2 different systems available which are working with different internal logic. The language layer consists of nouns, verbs, adjectives and grammars which is the symbolic layer. In contrast, the environment has no natural language but it consists of sensor perception, motor actions and 3d objects. The 2 box paradigm describes in a simplified format what natural language is about. Its an abstraction mechanism for the reality. Physical objects like a table or a banana are labeled with words. The ability to label objects is the key element in grounded language and allows to build intelligent robots.
Natural Language as AI technology
Languages likes English or French are discussed by Linguists not by computer scientists. A language is located in the humanities but not within the mathematics department. So its logical that most Linguists have no idea about computer science and vice versa. This might explain why AI research wasn't succesful over decades, because Natural langauge is the missing puzzle piece to make machines intelligent.
From a birds eye perspective, a language like English consists of verbs, nouns and adjectives. The grammar consists of rules how to connect the words to sentences. And language also is directed towards the reality. A word like "a flying bird" or "green flower" is referencing to objects which are available in front of the speaker.
The ability to name every object from the reality with a word and describe activities also with words makes natural language a powerful technology which can be used for human to human communication and machine to human communication both. In case of Artificial Intelligence a computer needs to parse natural language which is the bottleneck in modern AI research. Suppose a computer program asks the user to enter a word. The user enters "red box", then the computer stores the input in a variable but it has no consequences. So the computer isn't able to understand the meaning.
This parsing problem can be solved by inventing a language game. A language game is similar to a 2d arcade game a rule book which explain who to react to a certain input pattern. Typical language oriented games are translation games, the board game Scrabble or the "guess what" game. After implementing these games on a computer, the input of a human will have consequences. The consequences are given by the rules of a certain game, for example in a translation game the user needs to enter the correct translation for a word from another language. The computer verifies if the answer is correct.
Strictly spoken, not the computer decides about the meaning but the rules of the language games are providing the meaning. Modern AI research after the year 2020 is mostly focused on natural language and language games like "Visual question answering", "instruction following" and "question answering chatbots". All these games are formalizing the human to machine interaction in a sense that the computer can determine a score. This score allows to train artificial neural networks. The result of the training process is recognized as Artificial Intelligence in the modern sense. It allows to control robots with language instructions, and generate text with Large language models.
A common assumption in the past was that its very complicated to parse natural language with a computer. This assumption is only correct if the entire corpus of English should be understand by a computer which is around 1 million words and stored in endless amount of books. Parsing this written language with a computer is indeed a hard problem for computer science. What is possible instead is to reduce the task to a subset of English which consists of a dozens words from a restricted domain which are used to play a language game. In the minimal case, there are 6 picture cards and 6 word cards and the task is to match the correct pairs. Such a language game can be implemented in a short computer program and can be played by an automated AI algorithm. The AI algorithm has access to a database with the correct answers. This allows the computer to connect the picture of a banana with the word "banana".
July 21, 2026
Head up display for a kitchen robot
The picture shows an artist version of a head up display. It contains of:
- camera picture of a kitchen
- text box with inner voice
- bounding boxes
- labels for the bounding boxes
Surprisingly, the information in the picture can solve the symbol grounding problem because the head up display connects visual perception with textual information. The text from the inner voice like "I need to find 200g of flour" can be converted into meaning with the help of the bounding boxes. There is a box available with such an ingredient. The task for the robot is not to plan actions but the main problem is to connect language from the inner voice with detected objects in the camera.
Such a link of visual objects with textual labels is the core element in grounded language. If the robot is able to identify objects from the text box, its possible to generate all sort of inner voice. For example, the robot can say that he needs to peel the banana or "open the oven". All these nouns and verbs are translated into position of the bounding box on the screen which allows to execute the action physically.
July 20, 2026
How important is mathematics to understand Artificial intelligence?
On the first look, computer science has its root in mathematics and physics, therefor the assumption is, that AI is based on mathematics too. The precondition, according to the claim, to understand modern robotics is located in analysis, algebra, statistics and boolean algebra.
A closer look will show that mathematics isn't needed in AI research at all or to be more precise, mathematics didn't enable advanced AI in the past. What researchers from 1960 until around 2000s tried was to describe artifciial intelligence in terms of mathematics and logic and they failed. The problem is that none of the mentioned disciplines like statistics, algebra and so on contain a method how to enable artificial intelligence. Even if mathematics is a great science disciplines its complete useless to control a robot. Some attempts were made to solve robotics problems with mathematical description of trajectories including model predictive control, but even these advanced math subjects are not powerful enough to enable autonomous robots.
Artificial intelligence as a science discipline is working different than classic sciences like physics, math and computer science. From a pessimistic perspective there is no such thing like Artificial intelligence. Most research around thinking machines and autonomous robots comes to the conclusion that the promising new technique isn't working. That means, a large scale robot project with hundreds of man year effort isn't working at all, the applied techniques are useless and the researchers have no idea about the cause.
Such a kind of pessimistic situation is not an exception in AI research but it was the default situation from 1960 to 2000s. So the best analogy is to compare AI with a complex puzzle which is impossible to solve and no matter how well the researchers are familiar with mathematics, philosophy or psychology they had no idea how to make a machine intelligent.
AI research in the past was mostly a trial by error meta disciplines which wasn't able to solve any of the goals. It was impossible to build intelligent robots, it was hard to program AI agents for computer games, and speech recognition with a computer was another unsolved topic.
The good news is, that its possible to list techniques who are not leading to AI. These techniques are:
- neural networks
- maschine learning
- reinforcement learning
- mathematical optimiziation
- heuristics
- number crunching
- genetic algorithms
- state space search
- case based reasoning
- expert system
In other words, the entire AIMA book (Russle/Norvig: AI a modern approach) is an anti pattern. It doesn't contain recipes to program robots but it describes the struggle of AI researchers for doing so.
After listing all these techniques who are not leading to AI there is a need to find the shared bias. THis bias is the computational paradigm which means, to use a computer to provide intelligence. This shared bias hasn't worked in the past because its an anti pattern in AI research.
On the first look it makes sense to assume, that AI has to do with computation because the computer or the robot should calculate something which leads to intelligent decision making. Therefor AI has to do with number crunching, programming and mathematics. The problem is that in the reality this paradigm isn't able to control robots but it blocks the progress in technology.
It seems, that AI aka intelligence isn't located inside a robot but outside of the machine. On the first look, such an assumption sounds like blasphemy because outside of a computer there is nothing which can calculate or make decisions. At least until the 2000s such a claim would be rejected by mainstream AI research for sure. With more recent understanding of AI there are some new results available which show that AI might located indeed outside of a robot.
Suppose the source of intelligence is not located in the CPU and not inside of a robot hardware, then intelligence has nothing to do with mathematics or physics. This doesn't mean that intelligence is equal to a magic force but it implies that AI has to do with communication. Communication is the science of how to connect things, communication puts a focus on the air gap between two systems.
The transition from former computational paradigm which locates AI Inside of a robot, towards modern communication paradigm which locates AI between two systems is the major paradigm shift in AI research which took place after the year 2000. The revised understanding of intelligence is strongly connected with communication, linguistics and man to machine interaction. In contrast, former focus on computation, mathematics, algorithms and programming have been discarded.
July 19, 2026
Robot control with head up displays
In contrast to a famous assumption, modern robotics isn't working with algorithms or neural networks but the basic building block is graphical user interface, namely a head up display (HuD). The HuD solves the symbol grounding problem. Typical elements are: bounding boxes around detected objects, text labels for describing the content of a bounding box, another text box for showing the inner voice of rhe robot.
These ingredients are enough to program an advanced artificial intelligence which can solve complex problems. The HuD including the mentioned bounding boxes acts as a communication layer. It ensures that the computer understands basic commands like "move to shelf and grasp the box". A certain high level command is converted into a visual pictures in the HuD, e.g. the word "shelf" is referencing to a bounding box with the label "shelf" which has a 2d position on the screen.
Programming a Head up display for an existing video game is a demanding task but can be solved with standard programming techniques. Most videogames created since the 1980s have a built in debug mode which comes close to a head up display. In the debut mode, all the sprites on the screen are highlighted with frames and sometimes the name of the objects are shown as textual overlay. The combination of graphical display plus textual overlay is the main principle of a head up display and also the main principle of grounded language. So the HuD itself acts as technology for enabling artificial intelligence.
Let me give another example to demonstrate the advantages: Suppose the head up display for a warehouse robot videogame was activated. The user sees some bounding boxes on the screen for highlighting objects in the map like charging station, corridor, shelf A, shelf B, green box, red box. Also the inner voice of the robot is shown a text frame and contains:
"I'm standing at position (3,2). My battery level is 80%, my goal is to fetch the red box from shelf A, the planned trajectory is shown as arrows in the map".
So the initial situation for the robot is, that an annotated HuD is visible which labels objects and mentions the current goal. These information can be translated into actions for the robot. All what the AI of the robot has to do is to compile these information and decide what to do next. From an AI perspective its an instruction following task with an aciivated head up display.
A head up display provides a cognitive space. The shown bouding boxes and labels are creating a symbolic representation of the world. The world of the robot can be described in terms from the head up display. Its no longer a mathematical space and not a 3d space but the reality introduced by the HuD consists of words, locations of items and goals from the inner voice. Such a high level space can be processed by a computer because the amount of possible states is small. There are not millions of possible objects but the HuD shows only 6 different objects in a map. and the inner voice doesn't display millions of possible actions, but the inner voice describes clearly what the current situation is, and what the desired goal state is, similar to a text adventure.
July 18, 2026
Introduction into typst typeseetting
The amount of tutorials for typst is very low, because the software is new and works different from LaTeX. The following tutorial should explain the basics.
At first, a new file is created in the working directory which gets compiled into a pdf document with: "typst compile main.typ". The software itself is available as a binary file for all operating systems and needs around 60 MB on the SSD storage.
----------------
mainsimple.typ
----------------
#align(center)[
#text(24pt, weight:"bold", "title of paper")
#text(16pt, "Manuel Rodriguez\n 1 July 2026")
]
= 1 Introduction
#lorem(100)
= 2 Literature
- #lorem(10)
- #lorem(10)
If the typst user is reducing its demands to a minimum, the academic paper is ready for submission. Most authors have a need for more advanced layout so the file can be modified a bit.
----------------
maincomplex.typ
----------------
#set par(
justify: true,
spacing: 0.65em,
first-line-indent: 2em,
)
#set text(
font: "Liberation Sans",
size: 9pt,
lang: "en",
)
#align(center)[
#text(24pt, weight:"bold", "title of paper")
#text(16pt, "Manuel Rodriguez\n 1 July 2026")
]
#outline()
= 1 Introduction
#lorem(100)
#lorem(100)
= 2 Topic
== 2.1 Subtopic
#table(
columns: 2,
table.header[date][event],
[May 2, 2026], [hello],
[Jun 3, 2026], [world],
)
== 2.2 Subtopic
#figure(
image("drawing2.jpg", width: 4cm),
caption: [Drawing with pencil],
)
= 3 Literature
- #lorem(10)
- #lorem(10)
July 13, 2026
AI as open system
The last AI winter during the early 1990s was caused by the ignorance towards open systems namely Teleoperation for robot control. What the AI researchers have prefered instead were autonomous algorithm controlled AI systems. The goal was to program a large scale software similar to an operating system or a word processing software and make the software higly intelligent. Such bloat AI projects have failed, even a program written in 200k lines of code in C/C++ code isn't able to control a toy car in an obstacle course.
The reason why closed system failed is because existing programming languages like C/C++ can't grasp reality outside of a robot, existing algorithms like RRT pathplanning are too slow for realtime planning and the possible amount of trajectories for a robot in the reality is too large. This mixtures of challenges prevents that robot projects in the 1990s have become succesful. The written sourcecode was useless and the project doesn't make any sense.
The term AI winter is referencing to a situation in which the problems are known but no answer is available. This answer is maybe the transition from closed systems towards open systems. Open systems in robotics are equal to teleoperation which means that the human operator controls the robot. So its not longer an algorithm controlled robot but its an RC car. The main advantage is that such an open system can be realized easier with existing technology. The needed software is minimal and no true Artificial Intelligence is required.
What is used instead for remote controlled robots is a sender/receiver device which is an interface between human and machine. Such a device has multiple tasks:
1. it receives radio waves over the air
2. it parses natural language commands
3. it receives sensor signals
4. it transmits radio waves to the remove control
5. it transmits natural language status information to the human operator
In one word, the transceiver connects the robot with the environment.
The picture on the left shows the older paradigm. A robot in enclosed by a box and forms a closed system. The robot's AI is a turing machine executed on the CPU and the goal is to invent a sophisticated algorithm which makes the robot intelligent.
The picture on the right shows the modern paradigm which assumes two different systems connected by a sender/receiver. THese two systems are the robot and the environment around the robot. Both systems need to communicate. Communication doesn't require an autonomous algorithm but a protocol.
The transition from older closed systems into modern open systems is equal to discard algorithm oriented AI in favor of a communication perspective.
July 12, 2026
Die KI Blase ist geplatzt ...
Schauplatz: Ein Besprechungsraum am Rande der Endmontage in einem süddeutschen Automobilwerk.
Die Beteiligten:
Dr. Matthias Vogt (48), Leiter der Innovations- und Automatisierungsabteilung.
Elena Rostova (34), leitende Projektingenieurin für Robotik.
Dr. Julian Arndt (41), Key Account Manager von „Apex Robotics“ (Hersteller des Roboters).
Auf dem Tisch stehen drei unberührte Kaffeetassen. Durch die Glasscheibe sieht man die Werkshalle, in der ein leerer Stellplatz markiert ist. Die Testwoche des humanoiden Prototyps „Apex-One“ ist vorbei.
Arndt: (bemüht optimistisch) Erst einmal vielen Dank, Herr Dr. Vogt, Frau Rostova, dass wir unseren Apex-One unter echten Linienbedingungen testen durften. Ein Vision-Language-Action-Model, kurz VLAM, direkt in der Aggregate-Montage einzusetzen, das ist Pionierarbeit. Ich habe mir die Logdaten angesehen – die semantische Erfassung der Werkzeuge war phänomenal, oder nicht?
Vogt: (seufzt, reibt sich die Schläfen) Herr Arndt, ich mache es kurz. Der Roboter ist bereits verpackt. Er steht auf einer Palette im Wareneingang und wartet auf Ihren Spediteur. Wir treten von der Kaufoption zurück und werden das Projekt plangemäß beenden.
Arndt: (konsterniert) Bitte? Nach nur einer Woche? Gab es Hardware-Ausfälle? Wir können das Modell sofort gegen die Revision 1.4 austauschen, die hat verstärkte Aktuatoren in den Handgelenken…
Rostova: Es liegt nicht an den Gelenken, Herr Arndt. Es liegt am Gehirn. Genauer gesagt: an der Latenz und der mangelnden Deterministik dieses KI-Ansatzes.
Arndt: Aber das VLAM ist die Zukunft! Sie steuern die Maschine mit natürlicher Sprache. Keine Zeile Code. Der Roboter sieht die Werkstücke, versteht den Befehl und handelt.
Rostova: Ja, in der Theorie. In der Praxis sah das so aus: Am Dienstag sollte der Roboter Getriebeölkühler aus der Kiste nehmen und am Chassis fixieren. Der Befehl lautete: „Nimm den Kühler, überprüfe die Dichtung und setze ihn an Position B.“ Wissen Sie, was passiert ist?
Arndt: Er hat die Position gesucht?
Rostova: Er hat elf Sekunden lang „nachgedacht“. Elf Sekunden Standzeit, in denen sein neuronales Netz die visuelle Szene mit dem Sprachbefehl abgeglichen hat. In der Taktzeit unserer Produktion sind elf Sekunden eine Ewigkeit. Und als die Spätschicht am Mittwoch den Befehl leicht abwandelte – „Kühler greifen, Dichtring checken, ran an B“ – hat das Modell halluziniert. Es hat den Kühler gegriffen und ihn mit achtzig Newtonmetern gegen die Windschutzscheibe gedrückt, weil es „ran an B“ als „Scheibe einschlagen“ interpretiert hat.
Arndt: Oh. Gab es einen Personenschaden?
Vogt: Gott sei Dank nein, die Lichtgitter haben ausgelöst. Aber die Windschutzscheibe war Schrott und das Band stand für zwanzig Minuten. Herr Arndt, wir bauen hier achthundert Fahrzeuge am Tag. Wir können uns keine Maschine leisten, die auf denselben Befehl dreimal unterschiedlich reagiert, nur weil sich das Umgebungslicht ändert oder der Werker einen Dialekt spricht.
Arndt: Das sind Feinheiten im Prompt-Engineering! Wir können das Modell feintunen. Wir füttern es mit spezifischen Daten aus Ihrer Halle. Mit einem Ersatzmodell und zwei Wochen Datenkorrektur kriegen wir die Fehlerquote unter ein Prozent.
Vogt: Ein Prozent? Das ist im Automobilbau eine Katastrophe. Ein herkömmlicher Knickarmroboter von Kuka oder Fanuc arbeitet mit einer Wiederholgenauigkeit von weniger als einem Zehntel Millimeter, stundenlang, fehlerfrei, deterministisch. Er denkt nicht nach, er tut es einfach.
Arndt: Aber ein Knickarmroboter kann nicht flexibel auf unstrukturierte Kisten reagieren oder per Sprache umprogrammiert werden! Humanoiden sind die Zukunft für die flexible Montage.
Rostova: Flexibilität nützt uns nichts, wenn sie auf Kosten der Prozesssicherheit geht. Ihr Apex-One hat versucht, einen Schlagschrauber wie eine Kaffeetasse zu greifen, weil am Donnerstag jemand eine Mate-Flasche neben der Station vergessen hatte und das Vision-Modell die Geometrien verwechselt hat. Die Multimodalität ist für komplexe, dynamische Industrieanwendungen einfach noch nicht reif. Es fehlen die harten Sicherheitsgarantien. Ein neuronales Netz ist eine Blackbox. Wir können nicht zertifizieren, was wir nicht mathematisch beweisen können.
Arndt: (schaut auf seine Notizen) Ich verstehe Ihre Frustration. Aber bedenken Sie den Imagegewinn. Ein humanoider Roboter an der Linie…
Vogt: (unterbricht ihn kühl) …ist teures Theater für die Aktionärshauptversammlung, aber kein Werkzeug für die Werkshalle. Wir brauchen keine Roboter, die wie Menschen aussehen und versuchen, wie Menschen zu denken, nur um Aufgaben zu erledigen, die eine starre Automatisierungslösung in einem Zehntel der Zeit für ein Fünftel der Kosten erledigt.
Arndt: Also kein Ersatzmodell? Auch kein kostenloser Folgetest mit unserer neuesten Software-Generation im Herbst?
Vogt: Nein. Das Thema Humanoiden ist für uns vorerst gestorben. Wir investieren das Budget wieder in klassische Portalroboter und smarte Kamerasysteme. Die sprechen zwar nicht mit uns, aber sie halten den Takt.
Rostova: (steht auf) Ich begleite Sie zum Wareneingang, Herr Arndt. Die Papiere für die Rückgabe liegen beim Meister.
Arndt: (packt enttäuscht sein Tablet ein) Schade. Sie verpassen den Anschluss an die nächste industrielle Revolution.
Vogt: Mag sein. Aber dafür steht mein Band morgen früh um sechs nicht still. Auf Wiedersehen, Herr Arndt.


