In the late 1980s, Rodney Brooks has argued, that reactive architectures are able to control simple robots. So called scripted AI and behavior trees are used for direct control of a robot. The idea is, that the programmer defines motion primitives like moveto, grasp, ungrasp and then more complex behaviors can be built. But there is a problem with this approach because in reality it's hard to scale up the idea and most of these reactive controlled robots are only able to follow a line.
The problem is, that it's unclear which motion primitives are needed in the concrete example to solve a task. The good news is, that there is more powerful technique available which is called model predictive control (MPC). The idea is to divide the programming task into two steps: system identification and controller design. In the first step it is defined what the concrete domain is. The step of model building is ignored in direct control but for MPC it's important.
So the question is: what is a model? A model is a short term for “forward model”, and this is equal to a game engine. For example, in the TicTacToe game, the model is the part of the software which formalizes the game rules. It's equal to an API, the programmer can send the next action to the game engine and this brings the engine into a follow up state. A forward model is able to predict future states. That means, it's possible to send a sequence of actions to the engine.
In case of model predictive control, the games which are played have to do with control games in physical environments. A typical example is a two wheeled robot which can move the left wheel or the right wheel. If the robot is doing so, he changes the direction. The amount of change is encoded in the forward model of the robot.
The reason why model predictive control is misinterpreted sometimes is because such a forward model is complicated to build and it's hard to imagine why it's useful. Suppose, an elaborated model was created for the two wheel robot, what's next? Right the model itself can't control the robot, this task has to be done by the controller which is located in the second step of the MPC-framework. The motivation is high to ignore the forward model and start with the AI controller. This principle is called direct control, or reactive control, because no model is available.
The disadvantage is, that without a model it's not possible to plan over longer horizons. And this results into low complexity systems. The better idea is to focus on model building first. Let me give an example. Suppose, there is a robotarm who should do a task. Instead of figure out who to program the robot the starting point is to analyze the domain. That means, the robot arm has to be converted into a game engine. The question is not, what is the next action but it's a “what-if” problem. What happens, if the robot moves the joint0 for 10 seconds to the left? What happens if the gripper is open?
Solving computer games is usually done with Artificial Intelligence. The idea is, that there is a game and then an advanced AI Solver is used to play the game autonomously. The bottleneck is not to program the solver for the game, but the challenge is to formalize the game. The rules of some easy games like pong are known. For example, if the ball collides with the border he will move it's direction. For robotics control problems, the game rules are not known. That means, each robot is playing a game, but the inner working of this game are not available.
What human operators can do is not only to control a robot, but they know also what will happen in a certain situation. In the literature this is sometimes called “qualitative reasoning”. It means, that for the human operator it's obvious that the after the gripper was opened, the object will fall down.
From remote control to fully autonomous robots
The first step in realizing a robotics system is a remote controlled system. That means, the human operator has to press the joystick and this will move the robotarm. Such a control technique is remarkable robust, because the human operator is using his own skills to produce the next control actions. Most cranes on real construction sites are working with this principle.
The more challenging problem is to remove the human operator from the loop and control the robot only with software. Such a system is needed for reducing the costs. Realizing such goal is possible with so called model predictive control (MPC). MPC means, that the human operator only provides higher goals and the motion controller is figuring out the detailed actions. A typical example for a robot crane would be, that the human operator places doesn't operates the crane directly but he drags and drops the container to the target location. It's up to the robot to figure out how to do so.
From a computational perspective a high level goal is equal to a constraint. What the human operator provides is a goal within the game tree. The solver has to explore the game tree for finding an action sequence which brings the system into the desired state. Browsing through the game tree is only possible if the game rules are known. This step is known as system identification. System identification describes the possible action of a robot and the result. It's the same like building a physics engine or a forward model.
The reason why MPC is a powerful technique for controlling a robot system is because it divides the overall task into two subproblems. First, system identification and second, fulfill the constraints by searching for an action sequence. The part with finding out the concrete actions can be ignored, because the problem can be solved easily with brute force search, similar how a chess engine is working. The gametree is explored, and if the computer hardware is fast enough, it will produce the next action in realtime. The more interesting problem is creating the forward model. The interesting fact is, that for nearly all robotics problem, no forward model is available. The only thin which goes into that direction are physics engines used in computer games, but there is a difference between a physics engine and an MPC controller.
What robotics programmer have to implement is a dedicated forward model which works much faster than a classical physics engine. It's equal to program a simulator which can answer “what-if” questions. For example, “What will happen if the crane is moving to the left?” On the first look, it's obvious what will happen, but the problem is, that the information isn't available for the computer. Creating a software module which can answer “what-if” question very fast is the fundamental step in realizing model predictive control.