Action model learning with deeplearning

The typical action model was created in the STRIPS AI planning language. It contains of items in the form: name, precondition, effect. An example would be:

pushbox, playeratbox=True, boxattarget

The interesting aspect on the STRIPS notation is, that it is working on many layers at the same time. There are lowlevel actions and high level actions. The bottleneck in robotics is to write down the STRIPS notation for a certain domain. In case of Sokoban the problem is not very complex, but for dexterous grasping it's unknown how the STRIPS file will look like.

A potential strategy to overcome the issue is to create the action model with the help of deeplearning. That means, the neural network isn't learning the policy how to control a game, but it learns only the game physics of the game. SImilar to the STRIPS planning language this is done hierarchically.

Right now it's unclear how well this techniques works in reality. A literature search for “action model learning deeplearning” has found a little (24) number of papers. The subject seems unexplored in the literature. Perhaps it make sense to explain what action model learning is. Basically spoken the idea is to create a physics engine for the game. This engine is called a forward prediction model. It is equal to the STRIPS notation which is divided into actions. The player has on the lowest layer the option to move the arrow keys, and on the middle and higher layer he can execute high level actions like “approach the box”. or “push box”. The advantage of a hierarchical forward model is, that it can predict the outcome of abstract actions. A normal physics engine, for example Box2d accepts only lowlevel inputs.

Let me explain it from a different perspective. In the normal Sokoban game, the only allowed actions are left, right, up, down. Each actions will move the robot by one field. No hierarchy at all is available. In contrast, the STRIPS description of Sokoban contains lots of high level actions, like “movetobox”. These actions can't executed directly in the game, but they are operating on a symbolic level. A Strips action is equal to a plan element. The plan is to move to the box, and the plan has to be realized by subactions from the arrow keys.