Event processing in scripting AI

A common pattern in modern Game AI development is to store events in variables. An example is to create a variable “robotatdoor=True”, or “distance=100”. The first example is a boolean event which is called a trigger, and in the second case an integer variable was introduced to store detailed information about a situation.

In a behavior based architecture, the AI script takes the world state as input and calculates the actions in response to the event. A typical script would look like:

if robotatdoor: opendoor()
if distance<50: stop()

Unfortunately, there is a problem with event processing which can be called a categorization problem. The good news is, that in contrast to the robotics domain, all the event are certain. In a computer game it is sure, that the robot is really at the door, and that the distance is precisely 100 pixels. The categorization problem has to do with the program flow over a time period. Let us go into the details. A computer game consists of framesteps. In frame 0 the variable “robotatdoor” is False, in frame 10 the variable is False as well and at timecode, 20 the trigger gets activated and switches the state to True. Over a longer timespan the event can become true or false which is equal with a dual categorization. In case of the distance variable, there are also two categories available. In the first case the distance is smaller than 50 and in the second one it's greater. The robot behavior stop() is activated or not. The usage of categories are equal to formalize a situation. The problem is converted into a machine readable description which includes a decision making process. The algorithm for controlling the robot works deterministic, which means that the robot knows how the world looks like and what to do in each situation. Let us observe what will happen if the situation is unclear. Suppose the distance variable has no value:

distance=None
if distance<50: stop()

The if statement can't be executed because the value of the variable isn't available. The program will stop with an error. This is equal to a programming error. To overcome the issue, the programmer has to make sure, that each variable has a value. The “try except” statement in Python is a great help for doing so.

try:
  #distance=100
  distance=None
  if distance==None: raise ValueError
except ValueError:
  print("error")

The try except statement allows to stay within the program even if the variable has an unknown value. It prevents, that the Python interpreter exists to the command line. The unclear situation can be catched and handled with a subroutine for error management. It's important to build in an exception routine into an event processing system.

Measuring the worldstate Suppose a game consists of 3 sensor information which have all the boolean type:

input1=true/false
input2=true/false
input3=true/false

The total amount of possibilities for the worldstate is 2^3=8. It is possible to react for each world state separate, for example:

if worldstate=(0,0,1) then action1
if worldstate=(1,0,1) then action2

What will happen if the input variables have a different type? An 16 bit integer value can store values from 0 to 65535. The statespace is 2^16. If three input variables are given:

input1=0..65535
input2=0..65535
input3=0..65535

... the needed amount of storage space in the RAM memory is 3x16bit=48 Bit which can hold 2^48 worldstates = 2.81*10^14. It's not possible to decide for each worldstate which actions is needed:

if worldstate=(0,0,65535) then action1
if worldstate=(65535,0,65535) then action2

Especially in the domain of Q-learning the problem of the input space will become a problem. Because the number of rows and columns explodes shortly. The answer to the problem is to store the q-table in a neural network. The neural network is able to transform the complex input space of 2.81*10¹14 worldstates into a smaller one.

From an abstract point of view, it's important how many bits are needed to store the worldstate. In the first case, the entire worldstate can be stored in only 3 bits. In the second example with the integer values the amount of 48 bits are needed.