Wie man Robotik vereinfacht

 Robotik und Künstliche Intelligenz ist ein multidisziplinäres Fachgebiet was mehrere hochkomplexe Wissenschaften umfasst wie Naturwissenschaften, Geisteswissensschaften, Sozialwissenschaften und Informatik als verbindenes Element. Dadurch entsteht eine Komplexität und es ist unmöglich in einfachen Worten zu erklären wie ein Roboter funktioniert.

Um all die Wissenschaftsdisziplinen gemeinsam darzustellen im Kontext Robotik, bietet sich ein Head up display an. Ein head up display zeigt die Welt aus sicht eines Roboters und umfasst Kamerafenster und ein textuelles Fenster mit der inner voice des Roboters. Die Softwareentwicklung reduziert sich darauf das Layout und die Funktionsweise des head up displays zu optimieren, also zu bestimmen wo die fenster auf dem bildschirm angezeigt werden, welche Informationen darin zu sehen sind und sicherzustellen dass die daten aktuell sind.

Zum Beispiel werden visuelle Sensoren mit einer hohen Frequenz von 60fps aktualisiert, während die inner voice des Roboters in einer langsamen Frequenz von 1fps aktualisiert wird.

Selbst Laien außerhalb der Künstlichen Intelligenz können anhand eines Head up displays nachvollziehen wie Roboter funktionieren. Sie sehen auf dem Bildschirm was der Roboter sieht und können lesen was der Roboter denkt. Das inner voice textfenster enthält z.B. "Auf dem Tisch steht eine Tasse. Ziel ist die Tasee zu greifen".

Ein Head up display ist das Frontend, im Hintergrund wird weitere Software benötigt wie large language modelle, Vision language action modelle, datasets und bilderkennungssoftware. Diese Softwarebestandteile sind zwar für die funktionsweise nötig haben aber lediglich unterstütztende Funktion. Wichtig ist dass im Head up display alle Softwarekomponenten gemeinsam angezeigt werden.

Das Grundprinzip hinter jedem Head up display ist Bilder und Texte gemeinsam anzuzeigen. Charakterische Elemente sind bounding boxes, textuelle Beschreibungen von erkannten Objekten sowie kurzen Stichworten für Planungsaufgaben. Die Mischung aus Text und Bild ist die Antwort auf das Symbol grounding problem. Es ermöglicht einer Künstlicher Intelligenz Informationen zu speichern. Anstatt das komplette Kamerasignal im speicher abzulegen was sehr viel RAM benötigt, werden ledigliche Textualle Annotationen zu den Kamerabildern abgelegt. Jede Entscheidung, Planung und Ausführung von Aktionen hat etwas mit Verknpfüung von Bild und Text zu tun.

Birds eye perspective towards AI

 The AI revolution started in 2023 is different to the PC revolution in the 1990er because its more complex to understand. There is not a single technology available like a desktop PC but multiple abstract technologies which are resulting into surprising result. To become an overview over the development there is a need to identify key patterns which are common for all possible large language3 models created in the past and in the future.

What these AI systems have in common is, that they working with natural language and they were optimized for benchmarks. The ability of language understanding and the complexity of the benchmarks has been improved over the months. That means, that current LLMs are more powerful than former counterparts.

The surprising situation is, that even before the year 2010 early attempts were made to measure Artificial intelligence. In computer chess the ELO score is used to measure the propability to win a game of chess played by humans or machines. The lastest iteration of chess engines have an ELO score much higher than the smartest human chess player available. Modern AI systems were benchmarked not only with chess elo score but with question answering quizs, image generation benchmarks and öcoding benchmarks. The limitation of an AI is given by the benchmark. If the benchmark isn't able to measure correctly, the underlying AI system which scores in the benchmark has a low quality.

The general pathway towards AI is to create first a benchmark, for example a VQA benchmark, then test existing AI systems in this benchmark, and in the third step, the original benchmark gets improved. Such a development workflow produces a self evolving ecosystem which consists of more complex benchmarks, in combination with more powerful Large langhuage models.

The main difference between former chess playing AI in the 1990s and current LLMs like chatgpt is, that the benchmark is more difficult. Playing and winning a game of chess or tictactoe isn't recognized as a serious challenge. All the existing AI system are able to do so. The newly invented obstacle is to understand documents, generate prose text and generate videos. Such kind of challenge is a serious obstacle for current AI systems.

What makes the situation a bit unusual is, that the newly created / discovered benchmarks are closely related to human's problem solving skills. Recognizing objects in a symbol and answering questions for a document is similar to what humans are doing. And doing physical tasks, like pick&place objects or assembling a car is also closely related to human's daily life. If robots and AI software is able to do these tasks with the same and even a higher precision this is perceived as a technological singularity.

Its hard or even impossible to find a benchmark which can be solved by humans, but not by machines. Only very advanced tasks like driving a car or writing academic papers are not solved by machines yet. Instead of analyzing how a certain LLMs is working internally, the more important question is which sort of benchmarks can be solved by this AI. This allows to get a better picture about the current situation in AI technology.

Human Level AI for industrial robots

Industrial robots were never successful because all the tasks in reality are asking for human level AI. A human level AI is a robot which is on the same level like a human worker. That means, he understands normal English, is able to learn new tasks and is able to fulfill complex tasks by it's own. What autonomous robots can provide is reduced form of Artificial Intelligence. The typical AI control program is able to steer a robot on a line or can do simple pick&place tasks which are preprogrammed by the algorithm. In robotics challenges like micromouse and robocup such minimal AI is enough to solve the task. The problem is, that in reality the robot needs more skills to become highly productive.

From a technical point of view, it's not possible to program a human level AI in software. Even advanced research projects in the universities are not providing such features. All the existing robots have only sub-human level AI implemented. Because of this reason, they failed in real life applications. The better alternative is to use a teleoperated robot. Teleoperation means, that a human operator controls the robot which is connected with the robot with an internet connection. Teleoperation itself is not able to increase the productivity. The human operator will need the same time until the task is finished, and he has to be payed like the normal worker. The advantage of teleoperation is, that the distance between the robot and the human operator can be increased.

The normal Internet connection is remarkable fast. It allows to control a robot in realtime, similar to what a multi-player online game is about. That means the latency in games is enough for a robot control problem. In contrast to autonomous robots, a teleoperated robot is on the same cognitive level like a human. That means, it's possible to talk to the machine like “hello robot”. And the robot will answer in normal English, because on the other side there is normal human.

This kind of human level skills is required for solving real tasks. For example, the crane on a construction site is doing a complex task and there is a need to talk to the crane operator. If the crane operator is a software which was programmed for pick&place actions, it's not possible to talk to the crane. As a result, autonomous cranes are not used in reality. But a teleoperated crane is useful tool. The same is true for delivery robots which transports a box from a to b. A normal robot which is working with a computer program doesnt provide human level capabilities. A simple request like “put the box down” won't be understood by the robot, because the software has no speech recognition module. But if the same delivery drone is controlled a by a human operator it will understand each single word. And much better, the human operator will understand even sign language without extra commands so that the interaction make sense.

The work hypothesis is, that teleoperated robots are useful for commercial applications while autonomous robots are not. The only task which can be solved by software controlled robots are synthetici challenges like Micromouse, but these challenges are different from practical applications.

Is there a need for human level AI?

Perhaps it make sense to go a step backword and describe the precondition for normal robotics. The common idea of robot programming is, that at first the robot is equipped with piece of software, and then the software is able to solve the task. A typical example is a pick&place robot which moves an object from A to B. The assumption is, that the pick&place software is enough for solving problems in reality.

The problem is, that the engineers are not able to increase the skills of the software but what they are doing in reality is to modify the requirements of the tasks. In case of the pick&place robot the engineers are inventing a robot challenge in which a box needs to be moved from A to B. If the robot is able to do so, he has won the challenge. This kind of task is very different from real applications. In reality, a pick&place task is more complicated. This sort of real tasks can't be solved by the initial software. That is the reason, why a pick&place robots works great in the laboratory but fails in the reality. Let us imagine a real pick&place task which is required in the factory. Solving this task with a robot is not possible. What the companies are doing is to utilize human workers for this task. So the question is: which kind of software is needed to replace a human worker with an AI?

The answer is a bit complicated. It has to do with the task. Or let me reformulate the question: how much Artificial Intelligence is needed to solve pick&place tasks from the reality? The answer is, that only human level AI is capable of doing so. Even if the task looks easy to solve a normal algorithm isn't able to do so. That is the true reason why robotics were never used in the factory. Because the gap what robots have to offer and the requirement of the factory is, is too large.

The problem is not located in the domain of Artificial Intelligence. But it has to do with the human work in reality. All the jobs in the service industry, on the construction site, in the supermarket and for driving trucks to a destination are highly complex. They look easy only for humans, but they too complicated for robots. The reason why these tasks are so demanding is because most of the work was automated already. For example the engine in the truck moves the vehicle forward and the engine is driven by fuel. The only thing what is not automated is the steering task, which means to operate the truck and decide in which moment the brake is needed. The same is true for the crane on a construction site. The crane itself is driven by an electric motor. What the human operator is doing is to control the crane. That means, he is doing a high level task which needs a lot of domain specific knowledge.

This kind of human level knowledge isn't provided by simple path planning algorithm. The minimum requirement for a human worker is, that he understands normal English. Nearly all existing robots are not able to do so, only humans can understand a sentence like “please stop the engine”. If a robot doesn't even understand a simple sentence, how is he able to replace the human worker? Right, there is no way and as a result the automation project will fail.´