3a1d Programming an assembly line robot

< 3a1c Game design with petri nets

The picture shows an assembly line simulation game. The user can control a robot in the middle and the task is to sort incoming tokens. The robot has a battery level and all the actions are scored. If the robot puts the wrong token on the outgoing conveyor a certain amount of error costs are created. So the overall objective of the game is to reduce the costs.

Sounds not very complicated, right? The AI is located in the game engine. The game engine determines the score, and simulates pick&place actions in the game. The shown game can be played by a human player very well.

The interesting situation is that such a grounded domain can be automated easily. All what is needed is to solve the given optimization problem. The goal is, to minimize the costs for the robot and the costs are calculated by the game.

3a1 Reward function 

 

Let us try to elaborate the situation a bit. In bottom up robotics the idea is to program the robot in a certain way, that he is solving a task. Such a program is not needed here and it wasn't implemented. The idea of top down robotics is, that the AI is equal to the virtual referee. The virtual referee monitors a game and determines the score for a player.

The example simulation allows the robot to do certain actions. It can pick a token, it can place a token, it can walk around and it can charge the battery at the lower position. All these actions have consequences. For example if the battery level is below a certain threshold, the costs for the robot are growing fast. So it is a classical video game, except a strong emphasizes was put on the scoring function.

The interesting situation is that after starting the game in the command line the robot won't do anything. the reason is, that it wasn't the objective to program the robot. Instead the idea is that core element is the scoring function which is located inside the physics engine. What this scoring function is able to do is to judge about the actions. It converts possible behaviors in the game into a score. This score is shown on top left of the screen. It is a numerical feedback about the meaning of actions. Only actions which are generating low costs are sense making.

The principle has to do with social roles. There is an actor which is the robot. In theory, the robot can do anything which includes to put the token on the wrong conveyor. In the game, such actions are producing a higher costs. This virtual referee is sometimes called a critic because he judges about the robot.