Automatisierung von Teleoperation in der Robotik

 Es gibt einen rationalen Grund warum ferngesteuerte Robotik über Jahrzehnte ein Schattendasein führte. Und zwar benötigt teleoperation einen menschlichen Bediener und lässt sich nicht automatisieren. Robotik versucht jedoch explizit Prozesse zu automatisieren ohne dass dafür menschliche Bediener nötig sind. Ergo wurde Teleoperation als Sackgasse definiert.

Seit ca. 2010 erlebten Ferngesteuerte Roboter ein Revival was auf ein besseres Verständnis für teilautonome Systeme zurückzuführen sind. Frühere Bedenken gegenüber ferngesteuerten Systemen konnten entkräftet werden. Teleoperation kann sehr wohl automatisiert werden und das soll kurz erläutert werden.

Angenommen ein ferngesteuerte Lagerroboter versteht einen Befehl wie "fahre in Raum B und bring mir die Box #4". Streng genommen handelt es sich nicht um einen autonomen roboter sondern wegen des Befehls ist es nur ein ferngesteuerter Roboter. Die Interaktion lässt sich aber mittels Script leit automatisieren. Im SCript steht folgende Sequenz:

1. "fahre in Raum B und bring mir die Box #4"
2. "fahre in Raum A und bring mir die Box #2"
3. "fahre in Raum B und bring mir die Box #8"
4. "fahre in Raum D und bring mir die Box #12"
5. "fahre in Raum C und bring mir die Box #1"

Nach Ausführung dieser Sequenz hat der Roboter ohne weitere Interaktion immerhin 5 unterschiedliche Boxen von verschiedenen Standorten geholt. Er war dazu mehrere Minuten im Einsatz und verhielt sich fast wie ein autonomes System. Wenn die Kopmmandos auf einer hohen Abstraktionsebene formuliert werden, wie im obigen Beispiel zu sehen, ist es keine klassische Joystick Teleoperation mehr sondern solche Systeme haben starke Ähnlichkeit mit autonomer Roboter.

Zwar erhält der Roboter technisch gesehen Befehle von einem menschlichen Bediener, denkt also nicht selber, nur kann die o.g. Befehlssystsem auch in einem SCript oder Makro gespeichert sein, es wird also gar kein menschlicher Bediener benötigt sondern der mensch erstellt einmalig ein Script und damit wird der Roboter autonom.

Die Frage ist weniger ob ein System autonom, teilautonom oder ferngesteuert funktioniort sondern die eigentliche Frage ist auf welchem Abstraktionslevel die Fernsteuerung erfolgt. Wenn über einen joystick lowlevel steuersignale gesendet werden sind diese tatsächlich nur schwer zu scripten. Man kann die Joystick kommandos nicht aufzeichnen und erneut abspielen weil der Roboter in einer anderen Ausgangsposition startet. Wenn jedoch die fernsteuerung mittels text interface erfolgt lassen sich die kommandos leicht aufnehmen und erneut einsetzen. Es lassen sich sogar Programme schreiben, um die Scripte automatisiert zu erzeugen.

Moderne Ferngesteuerte Robotik ab ca. 2010 versucht primär über Text interfaces ein hohes Abstraktionsniveau zu erreichen bei der Mensch maschine Kommunikation. Damit kommt man dem Ziel einer künstlichen Intelligenz sehr nahe.

The need for human to robot communication

Layered communication with a DIKW pyramid solves a simple problem: how human can submit commands to a robot. For example the human operator submits "bring me the red box" to the robot and the robot will fetch the box.

The unsolved question is why such a human to robot interaction is needed? In the classical understanding of AI until the 2000s, such kind of pattern was interpreted as a dead end. Artificial intelligence was described as autonomous system which doesn't need to communicate with humans. If a robot doesn't communicates with humans the robot is described as a closed system which is able to run a computer program written in C or runs a neural network algorithm but the robot isn't using human language because there is no need for doing so.

Grounded language realized in AI systems like SHRDLU (1968), Vitra (1987) and M.I.T. Ripley robot (2003) is only needed if human to machine interaction is intended. A possible explanation for this paradigm shift has to do with the weakness of closed AI systems. Existing attempts to build autonomous robots have failed because closed systems are overwhelmed by a complex environment. Even if the robot's software consists of 100k lines of code in the C/C++ language this code won't match to a warehouse robot task because ambiguity and vague goals. Classical computer programming language are working only in a predicted environment like sorting an array or showing pixels on a monitor. Computer programs are the internal language of machines but the code can't store the knowledge for robots task.

Before the advent of human to machine interaction there was another available to build more powerful software for robots based on ontologies. Instead of storing the world knowledge in computer code the goal was to capture knowledge in a Cyc like mental map. Unfortunately, this concept has failed too. Even OWL ontologies are not powerful enough to store domain knowledge. Only teleoperation matches to high complexity. A teleoperated robot arm can do any demanding task including dexterous grasping and novel trajectories never seen before.

During teleoperation the AI problem gets outsourced from the robot itself towards an external source which is the human operator. The teleoperation interface allows a man to machine interaction which translates the external knowledge into robot movements.

Instruction following for videogames

The most advanced AI model yet is "Google Sima 2" from the year 2025.[1] The most remarable element isn't the inner working of the agent but its interaction with a human. Its basically a teleoperated video game based on natural language. The human operator enters a command and the AI is translating the text into action.

In other words, the AI itself isn't running a sophisticated algorithm and tries to win any game, but the AI is a text to action converter known as "instruction following". Similar to a large language model the working thesis is, that natural language is the key element for abstract thinking. Describing the next action of character in a videogame in English provides an abstraction mechanism. Instead of mathematical state space used for early chess games based on alpha beta pruning, the "Sima 2" agent utilizes natural language as communication tool.

Of course the input command can be generated by human operators and by large language models both. In the second case the agent will be controlled autonomously like a fully working ingame AI. So the human operator enters only the general command like "win the game" and the large language model will generate all the sub commands and submits them to the Sina 2 agent.

Let me explain the mechanism from a different point of view. Suppose the parser gets removed from the Sina 2 agent so he won't no longer understand natural language. In such a case, the AI can't do anything. It is not able to play the game or solve very simple puzzles. In other words, the intelligence of the AI depends entirely on natural language understanding. The agent needs to know what the term "jump" is about and needs to localize objects like "wall" and "coin" in the game. The software is utilizing the power of English which consists of verbs, adjectives and nouns to interact with videogames.

[1] Bolton, Adrian, et al. "Sima 2: A generalist embodied agent for virtual worlds." arXiv preprint arXiv:2512.04797 (2025).

How to scale up artificial intelligence

 In the past it was mostly unknown how to create Artificial intelligence, even restricted problems like the game of chess or robot control in a warehouse were recognized as hard to realize in software. The cause was a missing understanding of the domain and missing tools to implement AI domains on a computer.

What is available today is a pathway how to realize Artificial intelligence in a step by step fashion. It has to do with splitting the task between human and computer. In case of computer chess the situation can be described. Automating the entire process of playing the game is difficult, but the computer can be used to only count the pieces on the board and print out the number to the screen. This information helps the human player to decide what for the best move. So the human sees the chessboard itself and a dashboard with important information generated by a computer.

The next task is to improve the dashboard, in a sense that more information are recognized by the computer software like the value for each piece, similar board situation found in the database, and the allowed possible moves. All the information are shown on the same dashboard and reduce the workload for the human operator. The resulting AI system can't be called a true AI but its only a semi-autonomous system.

Instead of answering the question how to play chess with a computer the modified task is to communicate between a human and a computer program about the domain of chess. This communication is realized with an algorithm. The human enters high level actions like "move pawn forward" or "protect queen" and the AI software is in charge to realize this command on the board.

Die Erfindung der Teleoperation / Fernsteuerung

Ferngesteuerte Roboter und technische Anlagen sind ein sehr altes Prinzip. Das Telekino war ein ferngesteuertes Boot welche um 1905 von Leonardo Torres Quevedo vorgestellt wurde, seit den 1950er gibt es ferngesteuerte Spielzeugautos die meist über Kabel funktionierten. Und seit den 1980er sind funkferngesteuerte Spielzeugautos im Mainstream bekannt.

Allerdings wurde das Konzept von der Informatik und speziell der Robotikforschung ignoriert. Stattdessen fokussierte man sich über jahrzehnte auf autonome algorithmen gesteuerte Roboter, was man als Dogma bezeichnen könnte.

Es gab in der Geschichte der Robotik zumindest früher keinen Grund "remote control" näher zu untersuchen, zumal technisch gesehen das Problem gelöst ist also dort nichts neues mehr entwickelt werden kann. Um die Vorteile von fenrgesteuerten Robotern in Bezug zur künstlichen Intelligenz näher zu untersuchen muss man die Transformaion von einem geschlossenen in ein offenes System beschreiben. Klassicherweise wurden Roboter als geschlossene Systeme betrachtet. Die Frage war immer wie sieht die Maschinen innen aus, also welche Hardware, welche Programmiersprache und welcher Algorithmus steuert den Roboter. Sobald der Roboter jedoch von einem operator ferngesteuert wird, wird ein wichtiges Modul ausgelagert und zwar das Modul der Entscheidungsfindung.

Laut Definition gibt bei einem ferngesteuerten Roboter der Bediener am Joystick vor, was der Roboter als nächtes tut. Der Bediener ist nicht länger Teil des Roboters sondern befindet sich außerhalb. Dadurch steigt die Komplexität massiv an, und es bedarf eines Kommunikationsprotokoll. Also einer Sprache die definiert wie Signale vom joystick zum Roboter gelangen. Dieses Kommunikationsprotokoll ist der Schlüssel zum Verständnis von künstlicher Intelligenz als signalverarbeitende Zwischenschicht.

Ein offenes System wie es mittels Fernsteuerung erzielt wird, folgt nicht länger den bekannten Gesetzen einer Turing Maschine. Der Roboter wird also nicht per algorithmus gesteuert sondern über Input außerhalb des Roboters. Man kann den Roboter länger programmieren sondern man kann lediglich einen Parser programmieren der die äußeren Kommandos in Aktionen übersetzt.

Ferngesteuerte Roboter sind automatisch offene Systeme während algorithmengesteurte Roboter geschlossene Systeme sind. Ein Algorithmus benötigt keine Kommunikation mit der außenwelt weil die Entscheidungen intern getroffen werden.

Teleoperation with natural language

 A good starting points for programming a robot is a teleoperated simulation. A possible implementation would be a python video game in which a human controls a robot gripper with the mouse. Such a system simulates a real world sceneario, in which the human also has control over a robot arm and grasps objects with a joystick.

The main disadvantage of teleoperation in the reality and in a simulation is, that the human operator is needed all the time. Even if its technically easy to implement, the missing ability to run the system autonomously are a great problem. So the question is how to increase the autonomy of the robot slightly without using very advanced AI techniques like vision language action (VLA) models.

The idea is to introduce two constraints, first the communication from the robot to the human is improved only but not the other way around and secondly the robot doesn't need to verbalize the scene in an elaborated style but its enough if the robot only annotates the scene with [tags] like [gripper_open], [collision_gripper_box] and [box_isfalling]. Each tag is a boolean value and the entire tag space is stored in a binary feature vector.

The task for the programmer is to convert the existing numerical information from the physics engine like the position and the rotation of the Box2d objects into the semantic tag space which consists of 3 or more different tags. In other words, the translation process is equal to climbing upwards in the DIKW pyramid.

The resulting system remains a teleoperated robot, but the improved software gives textual feedback to the human operator. The human operator is doing a task, e.g. stacking two boxes on top  and the robot annotates the activities with a tagging mechanism.

The slow transition from teleoperation towards grounded language

Over decades, teleoperation was imagined as joystick based control. The human operator is moving the joystick forward and this will move the RC car also forward. Such a system has no builtin Artificial intelligence but can be described in mechanical and electrical terms. The only technical requirement is, that the control signal from the remote device will reach the RC car and this allows the human to control the machine.

Implementing an artificial intelligence doesn't mean to decide for a different control system, but artificial intelligence is only a small improvement over existing numerical teleoperation. What is called AI is technically a voice based teleoperation. Instead of submitting a numerical signal to the rc car, a sentence is submitted like "move 30 cm ahead and then stop". Decoding such a signal is more demanding than building a classical RC car but its located within engineering. Its possible to imagine a text-to-servocontrol parser realized in software.

Even a voice based teleoperation remains an example for teleoperation. The rc car won't act autonomously but the RC car reacts to the input of the human operator. The difference is, that the human input is given on a higher abstraction level. Instead of pressing a joystick button during a task, the human operator formulates the task only once and then the robot is executing it.

Such kind of interaction can only be realized with natural language. Natural language acts as an abstraction mechanism which replaces low level servo control. An abstract command needs to be translated first into low level signals, e.g. the command "move until waypoint D and rotate left" can't be parsed directly by a RC car electronics but needs to be translated first. This translation takes place within the DIKW pyramid from top to bottom and its called symbol grounding.

It should be mentioned, that technically its a bit tricky to realize such a grounding algorithm in software. The initial situation is, that computers only understands numerical information but can't interpret natural language. That the reason why a programming language is used to instruct a computer to do a task. converting an English sentence direct into computer instruction is a demanding task and its no surprise that it took decades until the task was realized by computer scientists.

There are two notable projects available with the goal of voice controlled robots. Both projects were developed late in the timeline of computing. In 2003 the Ripley robot developed by Deb roy. Its a robot arm controlled by natural language and can grasp simple objects on table. The second project is the M.I.T. forklift from 2010, developed by Stefanie Tellex, which is also late in history of computer science. The forklift understands basic commands like "move the pallet to the truck" and executes the desired trajectory.

In addition the SHRDLU project from 1968 should be mentioned. In contrast to the MIT robots, SHDRLU was limited to a virtual world. It was a computer program without access to physical sensors and actuators. All the mentioned project can be alled advanced demonstrations because it was realized at a research university with a high amount of codelines.

So we can say, that technically its possible to program a voice controlled robot, but its a demanding task which requires experts knowledge in computer science. With the advent of deep learning new ideas were implemented. Instead of programming a parser algorithm, the software is based on neural network architecture, trained on a dataset. This allows to scale up the approach to more words and more robotics domains. The goal of a modern vision language action model is the same as for the ripley robot from 2003, to control a machine with natural language.

Teleoperation with joystick and natural language

In the past, teleoperation was realized with a joystick. The human operator is navigating a robot by moving the joystick forward and backward. This allows a precise movement and the robot can do very complex tasks. The same principle is available for a construction crane and for joystick controlled UAV.

Even if joystick based teleoperation works great there is a bottleneck available because a human operator is needed all the time. A single human can control a single robot, controlling two UAV at the same time by a single operator is difficult or even impossible. From a technical perspective, a drone can receive signals with a higher frequency, the problem is that the human operator isn't able to generate the signals fast enough. To address this bottleneck a different sort of teleoperation is needed which is located on a higher level.

A slightly improvement over joystick based teleop is waypoint navigation. The human operator selects waypoints on a map and the robot is moving along the trajectory. This allows the human operator to reduce its workload. If the robot knows the next waypoint it is able to navgiate to the target by itself.

The next logical step after waypoint navigation is "grounded language control". The human operator communicates with the robot in natural language and gives a command like "move ahead, then rotate left, the move ahead for 10 meter, then stop". Such kind of language based communication reduces the workload for the human operator further. On the other hand, its a demanding task to program such an interface in a software.

Language based communication with robots is the answer to the teleoperation problem. It allows to control robots remotely with a reduced mental workload. Language has a higher abstraction level compared to a joystick control. This higher abstraction level must be translated for a robot into low level servo commands which known as "Symbol grounding". Let me explain it from a different perspective.

In classical joystick based teleoperation there is no grounding problem. The robot doesn't know terms like obstacle, shelf, move_ahead or stop. The robot understands only voltage signals transmitted from a remote control device. Such a robot can*t parse natural language but its a classical analog receiver. Of course, the human operator knows the words, he is aware that the robot enters a room and moves towards a shelf with a box. But this information is not relevant for the robot. its enough to move the joystick forward to navigate in a warehouse.

In contrast, a language based teleoperation requires that the robot understands natural language. The robot parses natural language commands and the robot gives feedback also in English.

The first electric RC toy cars were available since the 1960s. The build and operate such a car, a certain amount of knowledge in mechanics and electronics is needed. What isn't require is linguistic knowledge, because an RC car is not an English dictionary. It is a technical machine working with a battery and analog circuits. It took many decades until more advanced language controlled machines were available. One landmark project was the Ripley project in 2003 at the M.I.T, and also the voice controlled forklift at the same M.I.T. from 2010. Since the advent of vision language models in 2023, humanoid robots can be controlled with natural language.

The myth of autonomous robotics

In the past of computer science philosophy until around the year 2010 a certain paradigm was widespread available about the inner working of a robot. The idea was derived from science fiction novels written by Isaac Asimov and were based on the idea of an independent robot who is not in control of a human operator but takes its own decisions. In most or even all science fiction stores about humanoid robots, the robots have their own brain which allows them to take decision, analyze a situation and take actions. These fictional robots have much in common with animals in nature who are also independent beeings with their own will.

Engineers in the past were trying to realize this idea in technology, namely in hardware and software. The goal was to program a closed system which takes decisions by its own. The concrete realization can be seen in early self driving cars and early maze robots who are working in the autonmous mode.

Despite the large amount of effort to realize these robots, the concept of autonomous robotics has failed. The typical autonomous car programmed before the year 2010 was powered by millions lines of code but wasn't able to solve simple navigation tasks. The bottleneck is not located in a certain software architecture but it has to do with the idea of autonomy. This idea prevents the development of advanced artificial intelligence which is not working independent from a human operator but assumes teleoperation and especially text based teleoperation.

Solving the so called "instruction followin" task in robotics is much easier than implementing autonomouos robots. instruction following means basically, that gets instruction from a human. For example, the robot is grasping the ball because the human operator is pressing the button for "grasp the ball".

Such a remote controlled robot can't be called intelligent anymore, but its a tool similar to a crane which also operates by levers pressed by a human. The goal of building autonomous robots makes only sense for science fiction novels but its a bad advice for implementing robots in the reality. real robotis is based on teleoperation and voice commands.

The beginning of modern teleoperated robotics can be traced back to a single talk, held by Edwin Olson in 2010.[1] He explained to the perplexed audience that his robots doesn't working with software nor algorithms, but they are teleoperated with a joystick. Olsen claims, that such a control paradigm is harder to realize than classical algorithm based robot control.

To understand why the audience during this 2010 talk was upset, we have to listen was Olsen said exactly. In the introduction he made a joke about former attempts in realizing robotics, especially the idea of writing large amount of software for implementing algorithms. These large scale software based robots were seen as the here to stay paradigm for most of computer scientists and it was blasphemy to question this paradigm in the year 2010. In simpler words Olsen said basically, that all the sophisticated motion planning algorithms developed in thousands lines of code with endless amount of man hours are useless, and his robots are controlled by a joystick which is more efficient. Some people in the audience assumed, that Edwin Olsen is not a computer scientist but a comedian and perhaps they are right.

Edwin Olsen didn't mention in his talk natural language as source for robot control, but he is focussing only on joystick control. His talk is focusson the difference of autonomous robots vs teleoperated robots.

[1] Winning the MAGIC 2010 Autonomous Robotics Competition https://www.youtube.com/watch?v=OuOQ--CyBwc

Data driven model for robotics control

 

Abstract

Mocap data are stored in a pose database. The human operator retrieves the database and the found keyframe is shown on the screen.

TOC

6a Sprite sheet
▸ 6a1 From a model to a robot
▸ 6a2 Abstraction mechanism
▸ 6a3 Rapid prototyping
  ▸ 6a3a Problem grounding
6b data driven models
        Animation sequences
▸ 6b1 Data vs programming
▸ 6b2 examples for data models
▸ 6b3 data driven task models
6c more modeling tools
▸ 6c1 Model based tracking
  ▸ 6c1a Interactive animation
  ▸ 6c1b Motion retrival
▸ 6c2 Activity recognition but why?
  ▸ 6c2a Game log recording
6d Bibliography

6a Sprite sheet

The paradox situation is, that robotics control is perceived as a complicated task, because no problem at all is available. This understanding contradicts itself: if no problem is there, why should it be hard to solve this problem? The answer is, that computers can solve only well defined tasks. For example, a computer can determine what “78.0/3.14" is. The found solution is very precise and a computer can do so much faster than any human.
The only real obstacle for computer control is, if the computer should execute a task like “move the robot out of the maze”. Such problem is defined on a high level and for executing the task the computer needs to know background knowledge about how to parse english sentence, about spatial navigation and about robotics control. It is hard or even impossible to program computers in such a way, and because of this reason, robotics remains an unsolved problem.
The good news is, that we can formulate a thesis why exactly computers have failed in solving tasks. The reason is because of a missing model. A model abstracts from a problem and without such an abstraction mechanism the problem remains vague.
Let us go a step backward and ask the simple but provocative question: what is a model? From a programming perspective a model is a simulation realized in an object oriented programming language. It allows to predict future situations and helps to understand a domain. So a model is the same as well programmed source code in the python programming language which contains of differential equations, a list of possible actions and a cost function.
Unfortunately, it is hard to program such a model for a concrete task. In case of a simple line following robot it is maybe within the reach to create a model aka a simulator in python but for more complex domains like human animation such models will become very complicated.
An alternative approach in model design works with the data driven paradigm. The idea is to create first a database with information about the domain and then in the second step such a database is converted into an executable model which can predict something. For reason of convenience we can focus on how to create a data-driven model.
The perhaps most obvious technique in doing is motion capture with markers. The raw data are stored into a SQL database and can be analyzed later. A less common but also very powerful technique is to use a sprite sheet as input data. A sprite sheet is an often used tool within game programming. It contains according to the name of a sheet which holds pictures. So it is basically a .PNG file. Such a file can't executed on a computer but it is a data format.
Sprite sheets are able to model an entire domain. They hold information how a walk animation will look like and which other sort of actions are possible, for example, idle and jump. From an animation perspective, a sprite sheet has much in common with a body pose taxonomy. A taxonomy is some sort of table which holds also information about keyframes.
The interesting situation is, that the sprite sheet doesn't solve an existing problem but it creates a new one. If the sprite sheet is there the programmer has to implement a slicing algorithm. Slicing means to extract single images and paint them on the screen.
The main reason why sprite sheets are used frequently by game programmers is because they are helping to understand a domain. According to the sprite sheet the world aka the game looks in a certain way. There are some actions, and some keyframes and this information is everything what is important for a domain. A sprite sheet can be used to model very different problems like sport games, jump'n'run games and even racing games.
see also: 6c more modeling tools

6a1 From a model to a robot

Sprite sheets are used by game programmers since decades. Even the old NES games were using this technique. It remains an open problem how to use this information to create an Artificial Intelligence.
The answer to the issue is, that a sprite sheet is a certain form of a model and if a model is available a computer can utilize this model for solving a problem. Let me give an example. Suppose there is a graph and the computer has to find the shortest path in the graph. Such a problem can be solved easily with an algorithm. There are hundreds of tutorials available which are explaining the details.
The only problem within Artificial Intelligence is a situation which doesn't contain of a graph nor a model. Then it is impossible to solve the domain. All what a programmer has to ensure is, that at least a data driven model is there. In the easiest case this is equal to a sprite sheet in a more advanced setting an additional motion graph is available.
So the question is not how to solve a certain problem, but the question is how to invent a problem. If a problem is there then an algorithm can find the optimal actions within the problem.
Let us go a step backward to understand the situation. A sprite sheet itself won't help to implement an Artificial Intelligence. The reason is, that both things are opposite things. The common understanding is, that a robot aka an artificial agent is able to do something meaningful, for example to find a path in the maze, while sprite sheets are used to create such games. The source of confusion is the algorithm perspective. Computer programmers and AI engineers are usually searching for algorithm. Because an algorithm allows to solve problem. But, AI problems are different from this approach. AI is about defining the problem first, this is called modelling or grounding. Only if the problem is grounded it can be solved.
So the idea is to use sprite sheets, motion graphs, rules and cost functions for inventing a game. And only if the game is there a computer program can solve it. This results into Artificial Intelligence. Perhaps an example make the situation easier to grasp:
Suppose there is a simple sprite sheet which contains of two actions: walk and jump. Also a level map is provided. In addition the rule is, that the robot in the game has to reach the exit of the level. All these ingredients can be combined into a model which is realized with a physics engine. If the game is available it can be solved with model predictive control or reinforcement learning. That means, the computer takes the physics engine as a simulator and tries to figure out the optimal trajectory within this simulator.
 

6a2 Abstraction mechanism

A common understanding is, that a sprite sheet is a simple .png file which is used to animate characters in a game. This description is correct from a technical perspective, but the inner working goes beyond this understanding. From a general perspective, sprite sheets are used as an abstraction mechanism. The idea is to convert a problem into a model.
The main advantage of such a sheet of sprites is, that each sprite gets a unique ID. The sprite with id #1 stands for a certain picture, while the sprite with id #4 stands for a different one. Instead of describing a picture by it's details which means with the colors, the paintings and the size, the programmer can reference only to the number. That means, there are nodes from id #1 to #20 and each number represents a small graphic.
Such abstract description allows to program very advanced games. For example, the id can be used in an animation routine, or it can be used to react to keyboard input actions. For example, if the user is pressing the left key, a certain sprite id is shown on the screen.
 

6a3 Rapid prototyping

The reason why sprite sheets, level maps and motion graphs are used frequently in game design is because they are reduce the time until the game is created. If someone takes an existing sprite sheet which contains of 200 keyframes he won't need to paint the animation manual. And if some one is using a map created by someone else in the past, there is need to reinvent the property from scratch.
If in addition advanced GUI tools and game engines are used it is possible to create an entire game only by drag and drop the resources into the main window. The precondition which allows this highly efficient workflow is to divide complex games into sub elements which are the mentioned sprite sheet, level map, game engine and so on.
Game development doesn't mean to program something for example in the assembly or C language but game development is about aggregate existing content into something which is new.

6a3a Problem grounding

Within the robotics and AI community there is some sort of unsolved mystery. It is about possible algorithms or libraries how to solve AI problems. What computer programmers are doing frequently is to ask for an algorithm. Because an algorithm is a tool to solve a problem. Therefor it sounds logical to ask which sort of software or algorithm will provide artificial intelligence. The sad answer is that no software and no algorithm is available which fits into this category.
But there must be a way available how to create robots otherwise the subject AI at all can be dismissed. The answer to this complicated problem is not to search for algorithm but to describe the process from a design perspective. Design something means, that ideas from the outer perspective are converted into software. This is sometimes called the grounding problem.
To understand who design works different from algorithmic thinking we have to observe how new games are created. What the average game designer is doing is to start a level editor which is of course tile based and in the level editor he drags and drops icons into the map. After a while the user will save the result and this forms the basis of a new game.
The interesting situation is, that the process of creating a new map isn't based on an algorithm, but it is a creative process. What the game designer is doing is use a tool for rapid prototyping a game. That means, level design has nothing to do with game programming in a sense that someone opens an IDE and then programs the game in C++ against a game library, but game design has to do with converting external knowledge into the game. A source of inspiration for a new level might be a good map in another game, or older projects from the game designer. That means, he comes with his personal knowledge to the level editor and creates the map for a certain objective.[Engstrom2018] 6d Bibliography
Formalizing such process is not possible. The reason is, that game design doesn't solve an optimization problem and it has nothing to do with programming at all. Even the beginner friendly python language is offtopic for level design. Instead the computer is used only as a tool similar to a pen to write something down.
The chance is high, that programming robotics works with the same paradigm. It is not about writing source code in python nor C++ but it has to do with level design and GUI interfaces. This makes it hard to use certain programming language or algorithms, because in the design process such tools are not useful at all.
What makes sense instead are rapid prototyping tool. These programs are supporting the design process much better.

6b data driven models

A model is an abstract description of a problem. There are two ways for creating models:
1. Data driven models
2. source code driven models
Programming a model in source code is the goal in robotics development. It is equal to create a forward model which has much in common with a physics engine. Such a model is realized in Python or in C/C++ and can be used by a solver to determine the optimal actions.
The disadvantage of source code driven models is, that somebody has to program it. It is known that writing source code is a complicated task. Especially if the code contains of differential equations to simulate physical systems it is very demanding to write realistic models.
The alternative is to focus on data driven models. According to the name, data driven means the opposite of programming source code but it is equal to paint a picture, write a text file or create a database. Such models can be created much easier. In the easiest case it is realized by recording a game log into a csv file and then the CSV file is treated as a model for the game.
The disadvantage of data driven models is, that they can't be executed directly. For example a gamelog in a CSV database can't predict future states by it's own. An additional algorithm is needed which utilizes the data to a concrete simulation. But this is not a real disadvantage, because data driven models can be seen as a first step in a modeling workflow.
Animation sequences
Let us try to explain how to abstraction works for creating a longer animation sequence. Without any abstraction, the user has to create the animation manual. That means, he has to figure out the keyframes and paint them to the screen. This is a very complicated task which can take hours.
In contrast, an existing data driven model makes it much easier to animate a character. What the user has to do here is to provide a sequence of sprite IDs, for example [4,5,6,7] and then press the run button. The software will search in the database for the images and creates the animation on the screen. This simplifies the creating of longer animation sequences drastically.
With such understanding in mind, animation has no longer to do with painting graphics or figure out the correct pose but it has to do with entering a sequence in the format [a,b,c,d] and then the software model is doing everything else. This powerful principle is called an abstraction because it transforms a problem into an easy interaction.
see also: 6c1 Model based tracking 6c2a Game log recording

6b1 Data vs programming

The natural way in interacting with a computer is to program the machine. Many programming languages like Python, Java or C/C++ are available. And on the first look a programming language is the right tool for creating a model.
System identification means usually to convert a problem into a simulation. And a simulation is always a computer software which calculates something. The problem is, that programming a simulation is very complicated, even object oriented software engineering has increased productivity that much.
The alternative over object oriented programming is a data driven approach.The idea is to model a domain in a database which can be realized as a nosql database, an xml file or a json taxonomy.6d Bibliography [Kopp2018] [Baak2013] What all these dataformats have in common is, that they can't executed because they are not represented, the advantage is, that such files can be created much faster than normal computer code.
According to different studies, a highly skilled programmer can write down only 10 lines of code new codelines per day. This makes it unlikely that a single person is able to create a complex model in such a way. But, a single person can create a database of textual information much faster. Especially if the raw data are provided by a motion capture system which is basically a data logger. So the simple idea is to see modeling only under the perspective of a database and ignore that a model can be realized as a computer program as well.
Let us try to describe how data driven models are created. The main idea is, that there is empty json / xml / nosql database which is populated with textual information. Such information consists of tables, plain text and numerical data. The idea is, that all the information combined is equal to the problem's model. For example, there is a 50 kb large json file on the harddrive and this json file holds the model for a walking robot in a maze. The json file contains of the level map, the body of the robot and a list of possible events. Everything is stored in the ASCII format which means it can be shown with a text editor on the screen. The model doesn't holds executable programs written in lisp, python or java.

6b2 examples for data models

From a technical perspective, a data driven models is stored in a database  Possible file formats are json, xml or plain text.6b1 Data vs programming The problem with this understanding is, that such file formats are trivial. Creating ansi, unicode or even JSON files on a hard drive is nothing which can be improved that much.
The more interesting question is which content is provided. There are some examples available for a data driven model:
• level map, created with a level editor
• body pose taxonomy created with an XML editor
• sprite sheet, created with a graphics program 6a Sprite sheet
• game log stored in a CSV file which captures the keyframes 6c2a Game log recording
• motion graph 6d Bibliography [Kovar2008]

6b3 data driven task models

A task model is by definition a high level abstraction mechanism. The idea is to hide the detail and focus on long term planning horizon. Creating a task planner is from a technical perspective easy because similar to most planners it has to do with searching for a goal in the state space. There is a model and a number of actions and the planner has to find the shortest path in the game.
The bottleneck is, that for most robotics problem no task model is available, therefor it doesn't make sense to plan something. A method for creating models from scratch was provided in the section 6b2 examples for data models So it is likely, that task models can be created in the same way.
A good method for creating such models is a task taxonomy. This is a hierarchical dictionary for all the important words from a problem. For example a task taxonomy for a household robot would contain places like “kitchen, bathroom, floor” and provides actions like “goto, pickup, place”. From a programming perspective such a data model is stored in a json file as a plain text file. Such a model can't be executed and it can't be utilized by a task palnner directly, but it is a good starting point in creating a task simulation.
Let us try to investigate how to convert a task taxonomy into a task simulator. In contrast to a data driven model, a simulator can be executed. Such a system works like any other computer program. And the open question is how t convert data into a program?
The easiest way in creating a task simulator is by manual programming. That means, a programmer takes the task model as prototype model and creates around the data structure executable source code. For example the programmer defines, that after executing the action “goto” the robot position has changed to the new position. Such mapping can be realized in a language like python easily. What is needed is of course a variable for storing the current location of the robot and a method which is changing the position.

6c more modeling tools

Apart from sprite sheets 6a Sprite sheet there are many other possible candidates for creating a model for a problem. A lot of robotics literature were published around the topic of learning from demonstration (LfD) while other papers were written about animation languages.
In both cases the idea is to reduce the state space and solve the grounding problem. Animation languages and LfD as well are seen as abstraction mechanism.
The problem is, that it is hard to realize such principles in reality. For example the LfD idea looks great for the untrained ear. The idea is to record a motion trajectory and then use the recording to determine the parameters for dynamic movement primitives.6d Bibliography Kirk2016 Zhu2018 But it remains unclear who to do so exactly.
The same problem is available for dedicated animation languages.Webber1990 An animation language is a great tool if it was created already but generating a new language from scratch is a complicated task. Existing tools for modeling domain specific language are available, but they are not working well enough for practical applications.
From all the existing tools, a vanilla sprite sheet or a data driven model in general works best. The idea is, that in the first step the model is equal to a database which holds information about the problem. More advenced elements of a model like parametric movement primitives, an animation language or a prediction model are created on top of a database.
With this strict definition, a data driven model can become a .PNG file for storing a sprite sheet, a json file which stands for a database or maybe a CSV file for storing game log information. What a data driven model is not is a neural network, nor python source code and it is not a mathematical equations.
These advanced abstraction tools are created in a later step. That means there are simple data only models and more advanced source code oriented models available.6b data driven models

6c1 Model based tracking

Most existing robot projects are trying to create so called robot control systems. The idea is, that the software generates the signal for a robot arm and then the arm is doing something useful. The opposite idea over producing action is to perceive actions. The following section analyzes the tracking of activities in detail.
Activity tracking assumes, that meaningful actions are available already. They are created mostly by humans. And what the computer has to do is to recognize these movements in space. One example is hand gesture recognition.[De Smedt2017]
The most advanced form of action recognition works with a model in the loop. A model is used to interpret features. Let me give an example. Suppose there is a robot arm which contains of 4 elements which are connected together with joints. So what an intelligent vision system has to do is to take this pre-information as a template to interpret the movements much better. The structure of the robot (4 elements) is the model and the movements are parsed with this knowledge.[Filippi2007]
Another example would be a pose recognition system. The idea is, that a human body can hold different poses like walking, standing, sitting and what a computer has to do is to determine the correct pose ID. For example the human is doing something, and the computer prints on the screen that the human is in pose #3.
All these perception technique have in common that an underlying model is used to interpret the video signals. This model is able to annotate raw data with a meaning. In most cases the meaning is stored in natural language. That means, pose #3 is labeled with a textual string.
In most cases the tracking is realized with motion capture. And the models are based on data.6b data driven models An existing model is used to track movements. For example a body pose model is able to track the pose while a task model can track only high level tasks. From the perspective of a model the world looks a certain way. That means the model defines which parts of the reality are important and then the model tries to match this bias with the raw data from the video signal.

6c1a Interactive animation

Apart from model based tracking there is another strategy available how to create model based robotics. The idea is not to program the robot itself but to design a human computer interface. Basically spoken the human operator clicks somewhere on the screen and this will animate the robot on the screen.
The concept has much in teleoperation. A model is used to increase the automation level. But let us start the subject from the beginnings. Suppose the idea is to teleoperate a pick&place robot. For doing so the jaystick is mapped to the servo motors of the robot. Perhaps an additional GUI will allow to select the concrete servo motor on the screen. Such an interface works reasonable well but it will take many seconds until the robot arm can grasp objects.
The more advanced interaction technique is to define a handful of keyframe and use the mouse to browse through the keyframes. For example, keyframe #1 stands for ungrasp, while keyframe #2 for grasp. So the human operator doesn't controls the servo motors directly but he decides which system state he prefers in a certain moment. 6d Bibliography [Geijtenbeek2012]
The requirement to realize such advanced control system is a model. In the model the keyframes aka body poses are stored so the human operator can select one of the predefined IDs. The interaction with such a system is trivial, because the human operator can decide on a higher task level what action comes next.

6c1b Motion retrieval

The concept of a sprite sheet was discussed already.6a Sprite sheet The principle is used by game designer not from the beginning but frequently to simply ingame sprite animation. The idea is to a PNG file with the walk cycle and use this memory map in the game to animate the main character.
The interesting situation is, that spite sheets can be improved. The resulting motion retrieval system is used to query a mocap database. 6d Bibliography [Sakamoto2004] Similar to a sprite sheet there are some poses stored int he database, and then the software searches for the next pose in the database. This allows to create realistic motions with only a little amount cpu load.
The concept has much in common with model based tracking. The idea is that the underlying mocap database is the model and only body poses from this database can be drawn to the screen. The reason why this technique is highly efficient is because every pose has a unique number. Instead of adjusting all the 17 dof joints of a skeleton the  algorithm needs only to know a single reference number. For example, the sequence [4,1,19,2] is used to create a longer motion with the help of the underlying data model.

6c2 Activity recognition but why?

In the section 6c1 Model based tracking the idea of using a model to recognize existing actions was introduced briefly. The question which was left open is the reason why. Suppose it is possible to recognize that a robot in a maze has hit the wall, has this fact a value?
The reason why model based activity understanding is crucial in robotics is because it helps to create an abstract mechanism.6a2 Abstraction mechanism The idea is that there is a layer between a problem and the computer. The layer itself which is the model has no importance but it can be utilized for many purposes.
So the underlying problem can be summarized as mapping or grounding and has to do with reducing the state space of a problem. In the mentioned example with a robot which hits a wall the original state space has to do with objects which can do do something. There is a color, lots of pixels and endless amount of events. Such a state space is far to complex to understand by a computer and the only way for an Artificial Intelligence is to translate the problem space first into a model.
The interaction with a simplified model is much easier for a computer and can be realized in existing paradigm. For example with a programming language. It is possible to write down into a python program a statement like “if robot hits the wall, then stop motor”. But the precondition is, that at first an abstraction mechanism is available which allows to reduce the state space to a small list of possible events.

6c2a Game log recording

Before it is possible to program a certain sort of software a programmer has to defined first the objective. Suppose the idea is to program an activity recognition engine, where to start?
From a general perspective it can be realized by recording a game log. What all the game logs have in common is, that the keyframes are stored with a timestamp in a directory. for example:

The filename (0-5) represents the timestamp and the file itself holds the screenshot at this moment. So the game log is basically a frame accurate video of the game. Suppose a larger amount of such data were created, in the next step the goal is to parse the information.
A keyframe contains of possible actions, events and states. Each event has a unique number. For example, event #2 means “robot collides with the wall in the maze”. The compilation of all possible events, actions and states are stored in a taxonomy as a 6b data driven models. That means, there is somewhere a hierarchical table which holds all possible events.6d Bibliography [Karpov2013]
The game log parser has the obligation to math the taxonomy with the recorded gamelog. The result produces sense. That means, the keyframes are grounded and annotated. There are many ways for doing the matching. For example neural networks, but in the easiest case a manual created python script can do the task manually. That means, the classification module isn't trained by a learning algorithm, but it is hand-crafted.

6d Bibliography

Baak, Andreas, et al. "A data-driven approach for real-time full body pose reconstruction from a depth camera." Consumer depth cameras for computer vision. Springer, London, 2013. 71-98.
De Smedt, Quentin, et al. "Shrec'17 track: 3d hand gesture recognition using a depth and skeletal dataset." 3DOR-10th Eurographics Workshop on 3D Object Retrieval. 2017.
Engstrom, Henrik, Jenny Brusk, and Patrik Erlandsson. "Prototyping tools for game writers." The Computer Games Journal 7.3 (2018): 153-172.
Filippi, Hannes. "Wireless teleoperation of robotic arms." (2007).
Geijtenbeek, Thomas, and Nicolas Pronost. "Interactive character animation using simulated physics: A state‐of‐the‐art review." Computer graphics forum. Vol. 31. No. 8. Oxford, UK: Blackwell Publishing Ltd, 2012.
Karpov, Igor V., Jacob Schrum, and Risto Miikkulainen. "Believable bot navigation via playback of human traces." Believable bots. Springer, Berlin, Heidelberg, 2013. 151-170.
Kirk, James, Aaron Mininger, and John Laird. "Learning task goals interactively with visual demonstrations." Biologically Inspired Cognitive Architectures 18 (2016): 1-8.
Kopp, Oliver, Anita Armbruster, and Olaf Zimmermann. "Markdown Architectural Decision Records: Format and Tool Support." ZEUS. 2018.
Kovar, Lucas, Michael Gleicher, and Frédéric Pighin. "Motion graphs." ACM SIGGRAPH 2008 classes. 2008. 1-10.
Sakamoto, Yasuhiko, Shigeru Kuriyama, and Toyohisa Kaneko. "Motion map: image-based retrieval and segmentation of motion data." Proceedings of the 2004 ACM SIGGRAPH/Eurographics symposium on Computer animation. 2004.
Webber, Bonnie, and Barbara Di Eugenio. "Free adjuncts in natural language instructions." COLING 1990 Volume 2: Papers presented to the 13th International Conference on Computational Linguistics. 1990.
Zhu, Zuyuan, and Huosheng Hu. "Robot learning from demonstration in robotic assembly: A survey." Robotics 7.2 (2018): 17.

3d1 Pipeline for robot programming

< 3d The bottleneck in robot programming

The following simple pipeline allows to create fully autonomous robots for all domains:

1. Create a simulation with object oriented programming

2. control the robot in the simulation with teleoperation

3. Extract features like distance to goal and angle of the robot

4. convert the features into a cost function

A cost function in a simulation allows to solve the concrete domain. If such a cost function is available the domain was grounded.

 

see also: 3a1 Reward function

3d The bottleneck in robot programming

 < 3c The misconception about bottom up robotics

In contrast to a famous myth, the bottleneck is not to program the robot itself. But it has to do with defining the cost function. The initial situation is, that a teleoperated robot is available. For example a robot arm or a remote controlled car. What is missing is a numerical judgement if the actions of the robot are pleasant or not.

In most cases this decision is made by humans but not by algorithm. And this missing scoring system prevents that the task can be automated. So the challenge is to create a software which is judging about a teleoperated robot. For example if the car collides with an obstacle the costs are high and if the car stays on the lane the costs are low.

Such an algorithm or software is mostly a mathematical equation. It takes features as input and determines the cost value as output. This processing step is the core element of a virtual referee and the bottleneck in today's robotics. There is a large difference is such a scoring system is available or if it is missing.

Transition from teleoperation towards Object Action Complexes

The most powerful robot can be realized with teleoperation. Teleoperation means, that the robot is equipped with human level skills and can adapt to any situation. A pick&place with a teleoperated robot arm works perfect. The most interesting feature of remote control is, that no program is needed. The only piece of software transmits the joystick signals to the robot, but the robot movements itself are not determined by a program.

Suppose the idea is to program a robot which means, that the steering signal is not generated in realtime by a human but from a macro, script or any other robot program. The resulting question is which kind of software is needed for controlling the robot? In the easiest case a robot program is a list of points which are forming a trajectory. In the python language a typical robot program looks like the following example:


moveto(250,200)
time.sleep(1)
moveto(60,210)
time.sleep(1)
moveto(50,325)
time.sleep(1)


The robot program looks different from normal Python sourcecode, it has more in common with a list of absolute values which are executed by the robot. Will this program work? Oh yes it works great, the movements are executed precisely. The more complicated question is, if the robot movements are useful for the environment. That means, in a real life application an industrial robot is asked to do a task, for example to pick&place an object. The robot can fulfill the task or not.

The same robot program can become a failed robot project or a successful robot project. It depends on the task. If the task is easy the given robot trajectory will solve the problem. But if the environment changes to much, the trajectory of the robot doesn't make sense and it won't be able to pick&place any objects.

The overall successrate of the robot project depends on two factory. The robot program and the task description. The combination of a simple repetitive task and a simple robot program is a great choice. The problem is, if the task description is complicated but the robot program is an easy one. The result is a failed robot program.

Let us take a look into real applications. A welding robot is a typical example of an easy task description plus an easy robot program. The task for the robot is, to move the endeffector precisely along a list of points. The trajectory is always the same, and not kind of planning is needed. Such a task can be realized with the mentioned robot program which contains of two simple actions: moveto and time.sleep. The problems will upraise if the task description is more complicated. For example, if the robot should pick&place objects, but the objects can have a different location. In such a case, the easy fixed trajectory of the robot won't be succesful anymore. There are two options available to overcome the issue: first reduce the task description into something more easier or secondly, increase the complexity of the robot program.

A slightly more advanced form of creating a robot program is working with Object action complexes. This technique is derived from the STRIPS notation. The idea is not only provide a list of points, but provide a list of actions which can have preconditions and postconditions. Such motion primitives can be reordered so that the robot isn't executing a fixed trajectory, but is able to create different plans. The good news is, that the STRIPS notation can be used to generate a fixed trajectory as well.

From teleoperation to fully autonomous robot

Controlling a robot with a remote control is not very complicated. The human operaror has to press a button and the robot will move forward.The more advanced question is how to remove the human operator from the loop, so that the character runs autonomously.

On the first look the task can be solved with creating a script, very similar to automating a task on the computer with a Visual basic script. A potential program consists of building blocks like if-then, for loops and action statements. This will allow the human operator to take away the hands from the remote control and the robot will work by it's own. Really? No it was a rhetorical question because one important thing was ignored in that tutorial. The interesting fact with scripts for robots is, that they doesn't work in reality but only in a synthetic challenge.

Let us describe the pattern what is used by self-claimed robotics experts who like to proof that autonomous robotics is available. At first they are creating a macro for the robot. The script is doing a concrete task. For example, the robot is searching for line on the ground, then the robot follows the line, and if an obstacle is there it will activate a submodule to move around the box. In the next step, a game is constructed which contains of a line on the ground, a robot and an obstacle. Then the start button is pressed and the robot is working autonomously.

The problem with this demonstration is, that the task was created in response to the script. That means in the first step the macro was written and in the second step the problem for the macro was imagined. Unfortunately, the macro can't solve real tasks. This kind of bottleneck is often ignored. The assumption is, that the script can be extended to more demanding applications. It's funny to realize that not a single autonomous robot is available which is solving a practical application. So called autonomous robots are only available for trivial examples.

A possible explanation for this mismatch has to do with sorting tasks by it's complexity. The hypothesis is, that two sorts of problems are available: easy to automate problems and hard to automate problems. The problem “Follow a line” is an easy to automate problem. Replacing a human worker with a robot is a hard to automate task. What robotics engineers are able to automate are only trivial tasks. These tasks are constructed so that a robot is able to fulfill it autonomously. What robotics engineers aren't able to automate a real tasks which are important in the real life.

The funny thing is, that on the first look both task categories are looking the same. Suppose, in a factory there is a line on the ground and the transport vehicle has to move on that line from start to finish. It's exactly the same task which was automated by a script, so is the hypothesis wrong and it's possible to automate real life tasks? No it's not. A simple look into the reality will show, that not a single automated robot is available which is used for a line following task. If a company is using a robot for this task, a human operator controls the robot all the time.

Or let me explain the situation from a different perspective. Suppose, in a factory there is a transport vehicle which is moving on a line. The robot is remotely controlled by a human operator. The prediction is, that it's not possible to replace the human operator with a software program. Because what the human is doing is a little different from executing a simple line following algorithm.

To understand the paradox better we have to take a look at a task which is remote controlled already. The best example is a crane. A human operator sits behind a joystick and has to press the buttons. The operator doesn't invest physical energy into the system but only his ability to control the crane is requested. Such a crane is available on most construction sites in the reality. Now it make sense to think about increasing the productivity. The idea is, that the human operator costs too much and he can be replaced with software. Technically, a simple USB cable can be plugged into the joystick of the crane operator and then a computer is in charge of all the operation. The only what is missing is a piece of software. And at this point the problem will start. There is no such thing available like a crane control software. What the engineers have to do is to create it's own sort of software. A first step would be to create submodules for a program for the basic features of the crane like open the gripper, unload the box and so on. And then an overall highlevel planner has to decide which operation is next.

The problem with such an automation attempt is, that it will fail in reality. The computer controlled crane will behave different from a human controlled crane. That means, the system can't be used in the reality. What the human operator will do is to deactivate the program and control the crane with the normal joystick. And he is right, because it's the only option available.

The problem is not located in the crane itself. Because in the lab the software auto mode will work great. The crane is able to execute a longer program for pick&place objects. What is wrong is the reality, which provides a different kind of problem. That means, the task which is solved by the software and the task on a real construction site are different. A human operator is needed, because the real crane has to do with unstructured situations. Each day the problems on the construction site are a bit different.

A robot program

The idea of a teleoperated robot is, that all the actions are initiated by the human. Apart from the human-machine-interface no additional software is there. The opposite of teleoperation is a system which has to be programmed. A robot program is a script which runs without human intervention. For most real life applications no script is available. The answer to the problem is, to modify the application into the direction of an easy to automate task.

The reason why a program controlled robot is prefered over a teleoperation system is because it has a higher productivity. Instead of training a human operator, the idea is, that the robot can execute the task by it's own. Apart from the written script no additional input data are needed. The only problem is how to write a robust script which is able to solve important tasks?

Industrial robots are usually programmed with a trajectory table. That is a list of points in the space which are reached by the robotarm in a sequence. A typical robot program looks like:


p1, p2, p3, p4, p5, stop


The disadvantage of such script is, that it's not very robust. But in some cases it works. The question is, why exactly is a list of points powerful enough to control an industrial robot? The answer is, that the task was modified in a way, that the robot is doing something useful if he repeats over and over the same sequence. In general there are two strategies available for robot programming. The first one is to make the original task more easier for a robot, and secondly improve the software so that it can handle more complicated tasks. Let us try to improve the robustness of a robot program. A more elaborated example if formulated in the STRIPS notation. The robot executes steps, and each step has a precondition and a postcondition. For example:


step1, pre=gripper(100,100), box(50,50), post=gripper(200,100), box(50,50), action

step2, pre=gripper(200,100), box(50,50), post=gripper(250,100), box(150,50), action

step3, pre=gripper(100,100), box(50,50), post=gripper(200,100), box(50,50), action


Similar to the first example, the scripts executes a predefined robot trajectory. In this case, the some constraints checks are made to investigate if the robot and the box are inside the expected range. If not, the script stops with an error message. The advantage is, that smaller problems during the execution are recognized autonomously.

Both examples have the disadvantage that the script is static. That means, the script is executed from top to bottom and no planning is available.

Instead of explaining how to create more advanced robotics script the better idea is to focus on the most simple robot program which was given in the first example. In the simple case, the robot program contains of a point list which are traversed by the robot. It's not like a classical computer program but it looks like a pattern. The program is equal to a trajectory which gets executed with the start button. The only open problem is to find a task for such a program. That means, the robot by itself is working great, but what is missing is a situation in which the program makes sense. A very easy pick&place robot can be programmed with that principle in mind. The robot isn't able to detect objects, nor the system is able to avoid obstacles. Instead the environment has to be static so that the same trajectory results into a success. A possible usecase is, if all the products on the assembly line are at the same position and there is only size of the product available. The robot picks the object and transports it into the box, where the objects gets released.

Limits of remote controlled robots

The good news is, that a remote controlled robot can provide amazing skills. The robot can drive a car, pick&place objects, work in real life applications and so on. It's even possible to combine a biped robot with the ability of remote controlled. The resulting machine looks similar to what is known from the I, robot movie which means, that it's a humanoid biped robot which can walk on the street. Providing such functionality is technically not very complicated, because the joystick controls the servo motors and that is all the secret.

The sad news is, that all of these remote controlled robots provide the same productivity like a normal human. That means, if 10 humanoid robots should walk on the street, 10 human operators are needed. The same is true, if the robot should do a pick&place task. What is not possible is, that a single operator controls a robot fleet. This would be equal to provide a better productivity.

On the first look, the problem seems not very hard to solve. If it's technically possible to build a teleoperated robot, it's also possible that man machine interface will become more efficient. Unfortunately, this is not possible. The only option which is available is to reduce the complexity of the task. That means, if the robot should follow a line but is not asked to do something useful, that the remote control system can provide a higher productivity.

The problem is, that reduced complexity tasks are different from what a robot should do. In most cases, the idea is, that the robot is doing something useful, for example deliver a box to the destination. This kind of task has a certain complexity, which is fixed. If the complexity is reduced, the task will become something else.

I know the explanation isn't a bit complicated. Perhaps it make sense to go a step backward. Increasing the productivity of a robot has to do with programming a macro or an algorithm. The algorithm calculates the next movement and the human operator can relax. So the question is which kind of algorithm is needed to control a certain robot. And exactly this is the bottleneck. An algorithm which means an autonomous robot can only be created, if the task is very easy to handle. This is the case for computer games. If the rules are known in advance, it's possible to create some kind of solver, which transforms the remote controlled robot into an autonomous one.

Unfortunately, real robotics applications do not providing fixed rules. It's not possible to formalize the actions of a human worker in an algorithm. Surprisingly this is also the case for simple tasks like an pick&place operation. Even if the robot arm has nothing to do than pick and place an object, the task can only be handled with remote controlled but not with an algorithm.

Perhaps it make sense to research the topic from the opposite perspective. Suppose it's possible to program an algorithm for a pick&place task. A working algorithm can be executed autonomously without a human in the loop. In theory this is equal to the maximum productivity. Are such robots available? No they don't. Because this would be equal that an autonomous robot is able to fulfill a task which is important.

Let us summarize the situation a bit: teleoperated robots are working great for practical applications. The disadvantage is, that the productivity is low. It's not possible to increase the productivity, because this is equal to provide human level AI which is not available. The open question is, if such a telerobot makes sense for today's companies. In theory, it's possible to build some sort of cloud service in which human robot controller are providing the service to control all sorts of robots. These robots are able to replace normal human workers. At the same time, the workers in the cloud will produce labor costs as well. The advantage is, that the labor is located in a single place which can be requested by lots of robots.

The explanation why the productivity of a remote controlled robot is limited isn't available in the domain of Artificial Intelligence itself, but it has to do who the normal economy is organized. The normal workplace for a human is organized in a way to maximize the productivity. That means, the truck driver who transports a load isn't able to do a second task while he is driving and the worker at the assembly line is also fully occopied what he is doing right now. That means, the average workplace generates a certain amount of stress to the human worker.

If the worker is replaced by robot, the robot has to provide the same amount of work, that means, he must resist to the workload as well. The sensor signals are transmitted to a remote location and the human operator behind the joystick will get the same amount of stress like before. If the normal human worker is not able to reduce the workload, how should the remote operator can do so? And exactly of this reason it's possible to increase the productivity. No matter, if the crane operator sitts physical in the crane or is located 100 miles away, the workload for the human is the same.

The only way for reducing the stress would be to replace the human operator with an Artificial Intelligence, which is a software which doesn't need a human operator anymore. This kind of robot is the opposite of a remote controlled robot, it's an autonomous device. The problem with autonomous robots is, that they fail in reality. They are not working for practical applications. Instead of asking how to improve the robot, the more elaborated question is, why a certain workplace produces a certain workload?

The answer is located in the industrial revolution. The workplace of a crane operator is the result of the invention of the crane. What a crane operator is doing physically is to press some buttons. At the same time, this job is very hard, that means, if no crane operator is available the construction site gets in trouble. The same is true for other jobs, for example in the service industry. An existing job is a sign, that the economy has a certain workload which is important. This workload is different from playing a game.

In contrast, the task which are available in robotics challenges for example the line following task, provide a small or even a zero workload. That means, the robot how drives on the line in a circle isn't providing real work which is needed by the economy, but it's his own pleasure. Solving a zero workload task with an algorithm is easy going, but solving a high workload task with an algorithm is not possible.

That means, if somebody like to replace a real worker with a robot, he will need a teleoperated robot. And if somebody has build an autonomous robot which doesn't need a human in the loop, the task has only zero workload which means, it's a synthetic challenge which is not needed in reality.`



In the graphic, the desired goal is located in the bottom right, which is a combination of autonomous robot plus high workload which is available in reality. What most robotics engineers are trying to realize is to built a software controlled robot which can do real tasks. The reason why this combination is colored in red is because it's not possible in doing so. The reason is, that if a certain task is highly complex it's not possible to create an algorithm for it. And if a robot is controlled only by software it will only be able to solve low workload tasks. Let us make a small thought experiment. Suppose, there is a human worker available who is doing nothing else as walking back and forth on the street. He moves 100 meter from left to right, and then the same 100 meter in the opposite direction. In the thought experiment the human worker gets 20 US$ for each hour he is doing so. Automating such a task with a robot and replacing the human worker by an algorithm would be pretty easy. A simple python script in under 100 lines of code would do the job very well. The problem is, that such a task is not available in the reality. It's equal to a synthetic challenge given in a robotics competition, but it is nothing which is requested by the real economy. Real obs in which the human worker earns 20 US$ per hour are much more complicated. They can't be automated with a simple Python script in under 100 lines of code.

The perhaps most interesting feature of teleoperated robots is their ability to solve high workload tasks from the reality. A well designed humanoid robot is able to replace a human worker. The only disadvantage teleoperation has is, that they can't do much more. If a company likes to replace all the 1000 employees with robots they will need exactly 1000 humanoid robots plus 1000 human operators in the cloud. They are not able to do the same workload with only 500 human operators, because it's the same job with the same workload. It's up to the company to decide, if cloud based teleoperation make sense or not.

Workload reduction in teleoperation is a myth

Robotics is about industrial automation. The hope is to increase the productivity with modern technology. A first attempt in building a robot includes teleoperation. A teleoperated robot has the same workload, there is no advantage available for the human operator. What the engineeres are trying to achive is to reduce the workload. They want to design a human-robot-interface in which a single operator is able to control a swarm of robot. The interesting fact is, that such an interface can't be realized in reality. The reason why is a bit complicated. But for the moment it make sense to locate the increase of productivity outside of a control problem.

Let me give an example of a transportation problem. A load can be transported either by a truck or with the railroad. Logistics with the railroad is more efficient. A single operator is able to transport lots of container at once. In contrast, a fleet of trucks is needed to do the same task. The example is interesting because no artificial Intelligence at all is needed to increase the productivity. It seems, that the amount is connected to the mechanical vehicle but not to the control problem.

Somebody may argue, that from a technical perspective it's possible to invent a swarm based teleoperation device. Similar to what it's known from real time strategy games, the human operator selects 10 vehicles at once and command them to move to the new location. So he has reduced his own workload. The problem with this example is, that such a swarm control is a synthetic example. That means, in a newly created game the swarm is controlled in such a way. In reality, no such control problem is available.

This produces the question which kind of domains are available in reality? This question goes into the right direction. Jobs which are done by human workers are organized with a certain principle. In most cases, a task was optimized already. That means, that no potential for further improvement is available. The best example is a airplane pilot. What he is doing is to act inside an existing system. The combination of the airplane together with the pilot produces a certain productivity. That means, the overall system has costs and provide a service. The amount of costs is not determined by the pilot but by the system in general.

If the human pilot is replaced by a teleoperated robot, the same productivity is the result. Perhaps this is the most dominant reason why teleoperation is not discussed very often in the literature. In contrast to real Artificial Intelligence it doesn't provide extra productivity.

Let us analyze what the untold assumption of robotics engineers is. They are programming a software which can control an airplane by it's own. The idea is to install such software in all the airplanes in the world and then the human pilots are replaced by the software. Without the software, around 1000 human pilots are needed and with the working software 0 humans are needed because the system can fly by it's own. This is equal to a great productivity increase. The only problem with this outlook is, that the engineers have struggled in doing so. They are not able to write such a software. In the laboratory it works great, but in a real airplane the software is not able to replace a human pilot.

The interesting point is, that this is not a technical problem but it has to do with a bias of the engineers. They are focussed on technical problems for example how to calculate the trajectory or how to setup a neural network. What the engineers ignoring is the history of failed automation project. They are ignoring automatic airplane software from the past and they belief they can reinvent everything from scratch.

What modern computers and robots can do is to distribute work between humans more efficient. A teleoperated airplane can be controlled from the ground. What modern technology can't provide is to reduce the total amount of workload. That means, if the pilot in the airplance should be removed, a human operator on the ground is needed to do the same job.

The most efficient workflow

Suppose, the idea is combine telerobotics with a high productivity. The first step is to identify in the economy high efficient systems. The best example is an electric train plus cranes who can unload and load the containers. Such a system is highly efficient because it minimizes the demand for human work.

Now a robotic system can be used to remote control the system. The teleoperation won't increase the productivity further but it will allow the human workers to do the job from any location. The train gets remote controlled and the cranes for unloading too. The resulting system will need some humans in the loop, but not very much. And the most important feature is, that it can be realized. It's not a fictional scenario what robots can do in 30 years, but it can be realized with today's technology.

What is needed is cloud based teleoperation to control an electric train which is fulfilling a logistics task. The human operator behind the screen is replacing the physical operator in the train.

Teleoperation is a here to stay

Sometimes, Teleoperation is introduced as bridge technology for creating Artificial Intelligence. The idea is, that Teleoperation is much easier to realize than a fully autonomous system. Most engineers who have implemented Teleoperation are trying to increase the automation level further, because they want to remove the human operator from the loop. But is this vision realistic? After removing the human operator from the loop, the robot will loose it's human level AI capabilities. It's no longer possible to talk to the robot with natural language. For filling the gap of the human an advanced sort of artificial Intelligence has to be realized.

It's a well known fact, that a human level AI is not available. And it won't be available in the next 50 years. After removing the human operator the robot will have a much smaller cognitive capabilities. The naive assumption is, that a state-of-the-art AI can control the robot by it's own, and that the task doesn't require the full human level skills. But this kind of assumption is wrong.

The reason why software controlled robots are not used in reality is because all the tasks are asking for human level skills. It's not possible to control airplanes, cars, ships, drones or grasping robot with sub-human-level AI. Let us ask the engineers in which timespan human level AI in software becomes realistic. The answer is, that they don't know. What was demonstrated in robotics challenges is only a narrow AI. That is a software which drives a robot on a line, or which can do simple image recognition tasks. The engineers have no idea how to program a human level AI. That means, they are not able to provide such software. And because of this reason, it's not possible to remove a human operator from the loop. Teleoperation is the only working robot which is available.

From a fantasy point of view, it's possible that one day, human level AI is available. That is a piece of software which can do everything what a human has to offer. It includes understanding of natural language, detecting all sorts of objects, learn new things and provides empathy. In some science fiction movies, Human level AI systems were shown, for example Data from Star Trek TNG. But right now no such thing is available in reality. Therefore AI can't be realized.

What makes teleoperation so amazing is, that without complicated software, human level AI is provided as default. All what the human operator needs is a joystick and monitor and the robot is able to do everything what the human can do. It's not a real AI, because there is a human in the loop. But the robot is on the same level like a human. He can be used for practical applications.

Practical example

I'm not the first one who is arguing pro Teleoperated robots. The company “Phantom Auto” has developed a teleoperated self-driving car. The basic idea is, that a car needs at minimum human level AI, which means, that without a human driver a car is not allowed to drive in real traffic.

The interesting fact is, that self-driving car engineers in the past have argued a different way. The assumption was, that the robot car can be controlled with software and they have demonstrated it in synthetic driving challenges. Either for RC Cars which were controlled by Python script or with real cars. The problem is, that a software controlled car provides only sub-human level skills. It is able to do some tasks like path planning and automatic steering but it won't understand a simple sentence like “hello robot, what's up?”.

The main reason why robots are not available is because human level AI is the minimum for practical application. And what the company Phantom Auto is doing is to provide such feature. A teleoperated car is working on a human level. It will understand the mentioned sentence and will respond to it. This make it a good choice for practical applications.

The interesting point is, that a teleoperated car provides a lot of technology except Artificial Intelligence. The onboard computer provides only a connection to a remote location, but the computer is not able to control the car by it's own.

GOAP AI planning explained step by step

The Goal and action planning system has the same disadvantages like the original STRIPS planner. The algorithm itself works great, but it's complicated to explain. I won't invent the wheel twice but i will explain it more easier. At first, a GOAP tutorial needs two elements:

1. how the solver works

2. how to build the domain model

A GOAP planner contains of low level actions which can be executed on the robot directly, for example the action “left”. And it contains of tasks which doesn't produce an action but result into a goal. Let us take an example. The robot should go to the middle of a map. This goal is defined by sending the goal to the game engine: task(“gotomiddle”)

If the game engine receives the task it will put the robot to the middle by changing it's absolute coordinates. The planner has to figure out which goals are needed in which sequence and then the low level actions are determined. The inner working of such a planner is remarkable easy because it's a graph search algorithm which is traversing the symbolic game tree, and checks out potential paths.

The second important aspect of GOAP is how to create the domain model. Without a model the planner won't work. The domain model is equal to grammar which contains goals in natural language. It's a wordlist which contains detailed description for each entry. Building such a grammar is possible with learning from demonstration. That means, a human operator is observed how he is solving a task, and according to the annotated actions the tasks are identified.

Combining teleoperation with symbolic planning

Every solver is working with a goal in mind. The problem is, that goals are not available in the software itself, because they are referencing to request from the outside. The high level goal is an input which is feed into the planner. The source of the goal is not another module, but the human operator.

If the human operator defines the goal interactively, a semi-autonomous system is the result. An easy to grasp example is an inverse kinematic in which the human operators points to a position on the screen and the robot arms is following the trajectory. The interesting aspect is, that such a robot arm is moving faster than a human can execute the actions. The human needs only to point the mouse to the goal, but the lowlevel actions are determined by the system.

This interaction between human and robot can't be modeled in software. Every goal planner comes to a bottleneck because it's only in parts possible to determine the goals themself. That means, the robot itself doesn't has meaning, he becomes only a tool if a higher instance asks the system to do a certain action. Let us give an example:

A plan is provided to the robot control system, which is a list of waypoints. Why does the robot system should follow the waypoints? Nobody knows, because the plan is given by a human operator. He knows why the plan makes sense, but this knowledge isn't available for the robot. The communication between both separate system needs a shared language. That means, the robot control system should be able to parse the plan and the human operator as well. What the robot can do is to figure out the low level primitives for realizing the plan.

It's the same idea how modern compiled programming languages are working. The compiler needs as input a hello world program. For the compiler the program itself doesn't make sense, but it can convert the sourcecode into binary code.

Essence of GOAP

The problem with most AI paradigm is, that it's hard to grasp it's inner core. Is GOAP an algorithm, a piece of software or a programming language? Unfortunately none of them, and it's much harder to explain the overall concept. I would call it, an extension to a physics engine which provides abstract actions. The user can send a command like “movecenter” to the physics engine, even if the game itself doesn't know of such an action. The physics engine is put into a measurable state and a solver can try to reach this state.

Or to explain it from a different viewpoint. The goap concept means, that a plan notation is invented and grounded into the physics engine. This plan notation allows the human operator to provide a sequence of actions, and the goap planner can try to convert this plan into lowlevel actions. The term goal is equal to a macro action. For example a goal might be “reach waypoint”. It's an option to raise the abstraction level. A plan which contains of only 10 single steps can describe a complex longer sequence of movements in the game. A comparison with functions in a normal programming language make sense. For the programmer it's not important how the “isprim()” function works internally, but it delivers a return value back to the program. GOAP is using the paradigm for structuring the action space of a robot in a game.

Introduction into Learning from demonstration

I would like to introduce the topic with a small example, suppose an Artificial Intelligence should play Lemmings, because, quote “The game of Lemmings has been offered as a new Drosophila for AI research” [1] The naive approach would be to understand the Lemmings game as some kind of search problem in which the solver has to find the actions for winning the game. Because the state space is very large, this attempt will fail. To make things a bit shorter, the better alternative is to invent a plan language which will guide to search process. But let us go into the details.

A simple level of Lemmings in shown in the figure.

The entrance is on top in the middle and the lemmings must master the falling step, then a stopper has to be set on the right side, and the wall needs a digger before all the lemmings are allowed to go into the exit at the left bottom. The arrows in the map demonstrate what the solution is. The interesting point is, that these markers are equal to the walkthrough tutorial, that means, the level is already solved and the Lemmings has to follow only the guidance. Exactly this aspect is typical for learning from demonstration. Before the software gets started the walkthrough is available, which is formalized as a plan.

Learning from demonstration means in it's core to formalize a plan language. If the plan language exists a solver can calculate the subactions. Now let us imagine what will happen in a different Lemmings map. No plan is available and the Lemmings doesn't know what to do. The funny thing is, that the Artificial Intelligence is no longer forced to control the game in all details, but the only missing thing is a plan. The overall pipeline contains of two items: finding a plan for a map, using a plan for control the Lemmings.

The plan source can be human but it can be also a pathplanner which is working on a abstract level. Even if the human provides the plan, the system will work a bit autonomous. Because the human has to draw only the plan into the map and the system is doing the rest. From a certain standpoint this can be called cheating, because the AI only follows the walkthrough tutorial which is already there.

And exactly here comes “Learning from demonstration” into the game. The main principle is to invent a plan notation for a domain. This plan notation is used for recording and playback of demonstrations. The sad news is, that no standard is available for a plan language. It can be a graphical notation, a textbased langauge or a trajectory which is produced by Dynamic movement primitives. What is important to know, is that LfD is located between teleoperated robot and autonomous robot. The link between both consists of an abstract plan language.

Let us observe what a Lemmings AI is doing if the plan is known. The input for the system is the figure which contains the arrows. The plan is stored internally as a waypoint list which has some smaller annotation. For example the first step in the level is annotated with “parachute” while the second step is annotated with “stopper”. The AI Solver takes this plan and converts it into low level actions. It has to make sure that the plan is fulfilled. This is equal to a subgoal. For the solver, a subproblem is given which is only some seconds long and in which the general idea is provided by the plan. And the solver has to figure out only the detail adjustments to the plan. This can be done on a standard PC without much effort.

The funny thing is, that the same principle can be transfered to any game. No matter if it's called Sokoban, grasping robot, Lemmings or RC-Car control, in all these domains a plan notation is used as an intermediate between walkthrough-knowledge which is provided by humans and lowlevel solver which executes a plan.

Human guided teleoperation

The term “learning from demonstration” is a bit misleading because the association is, that some sort of machine learning algorithm takes place. The more exact terminology is to call such systems human guided teleoperation. Teleoperation means, that in the basic setup a human operator controls the robot with a joystick. Plan guided teleoperation is equal to replace the joystick with a plan notation which allows to control the system from an abstract level. In both cases the knowledge how to solve a robot problem comes from the outside. Either by direct human commands, or from an abstract plan provided also by a human.

The overall system doesn't have much in common with a classical AI planner, but it's more a human-to-robot interface. The plan notation is similar to a joystick an input device which transmits the human knowledge into machine readable information.

[1] Kendall, Graham, and Kristian Spoerer. "Scripting the game of Lemmings with a genetic algorithm." Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No. 04TH8753). Vol. 1. IEEE, 2004.