Programming with multiple demonstration

In the domain of robotics there is a technique available called Programming by demonstration. The idea is to provide the robotics trajectory in advance and the only thing what the robot software is doing is to replay the given trajectory. That means, the overall system can't be called an intelligent robot but it has more to do with a numerical controlled machine.

Let us describe the idea from a programmers perspective. The first thing to do is to provide a trajectory in a python script:

trajectory=[(0,0),(10,4),(20,6),(10,5)]

What the robot is doing is to follow the points in the list. The interesting question is how to use this idea for more advanced robotics control applications? Let us observe, what will happen if the environment of the robot is different than the provided trajectory. For example, if an obstacle is given which wasn't anticipated by the default trajectory. The result is, that the programming by demonstration has failed. That means, in the replay mode the robot will get blocked by the obstacle.

The answer to the issue is not to reject programming by demonstration in general, but to improve some details in the python script. A possible way in dealing with new situations is to use multiple-demonstration which are enhanced with preconditions and divided into skills. The resulting python datastructure is:

self.skill-library=[

skill1

- precondition: no obstacle

- trajectory: (0,0),(10,4),(20,6),(10,5)

skill2

- precondition: with obstacle

- trajectory (0,0),(60,4),(70,6),(10,5)

]

This datastructure will work in different situations. In the replay mode, the robot will check which trajectory can be applied. Such a skill based datastructure allows to execute more complex tasks. The underlying principle has to do with multiple demonstration. The vanilla “programming by demonstration” is working with a single demonstration. That means, only one trajectory is stored in the python program. This trajectory can be replayed or not. In the second case many trajectories are provided.


To understand the idea “programming by demonstration” better it's important to think about situation in which the technique doesn't work. Constructing a counter example for a single demonstrated trajectory is not very complicated. We have to imagine an environment in which the static trajectory will fail. Inventing a counter example for the second case is much harder. Because the datastructure is prepared to react differently.

A possible bug is, that the obstacle in the replay mode is larger than in the demonstration. That means, the replay mode will detect the obstacle, executes the second demonstration but it doesn't work. This will result into the need of inventing a more advanced method in storing the demonstrations. One option is to increase the number of demonstration to 10 which includes larger obstacles, the other option is to extrapolate the skills for new cases.