January 26, 2026

Improved chatbot for a kitchen robot

In addition to the previous post, the python script was improved a bit. There are more entries in the database, the amount of informaiton is higher, and very important a telemetry mapping function is available. This allows to monitor a teleoperation robot. The amount of codelines was increased to 80 but the software remains easy to understand.

The core element is a database with words. Every word is described with additional key-value informaiton for example a picture or a position. The AI takes the current sensory data and searches for a match in the database and the AI also searches for a text input from a user. If the AI has found an entry in the database its equal to understand a situation. In short, the AI is a database lookup algorithm. Here is an example interaction and of course the source code written in Python3.

----
gathering telemetry ...
attention near apple
robotpos near table
user: lookat table
search database ...
lookat action inspect object
table {'pos': (0, 0), 'desc': 'place for storing objects', 'word': 'noun'}

gathering telemetry ...
attention near apple
robotpos near table
user: grasp apple
search database ...
grasp action take an object
apple {'pos': (10, 3), 'word': 'noun', 'category': 'fruit', 'desc': 'is food to eat', 'filename': 'apple.jpg'}
----

"""
chatbot kitchen robot
a wordlist is stored as python dictionary, user enters command which is searched in the wordlist
application: Teleoperation monitoring
"""
class Chatbot:
  def __init__(self):
    self.data={
      # verb
      "open": "action open something",
      "grasp": "action take an object",
      "ungrasp": "action place object from hand to world",
      "eat": "action eat food",
      "lookat": "action inspect object",
      "walkto": { 
        "word": "verb",
        "category": "action",
        "desc": "move towards location",
        "motor": "legs",
      },
      # noun
      "apple": {
        "pos": (10,3),
        "word": "noun",
        "category": "fruit",
        "desc": "is food to eat",
        "filename": "apple.jpg",
      },
      "banana": {
        "desc": "noun food",
      },
      "table": { 
        "pos": (0,0),
        "desc": "place for storing objects",
        "word": "noun",
      },
      "fridge": {
        "pos": (1,0),
        "word": "noun",
        "status": "closed",
        "category": "furniture",
      },
      "plate": "noun food is served there",
      "door": "noun entrance to room",
    }
    self.telemetry()
    self.parser()
  def getdist(self,p1,p2): # return: manhattan_distance
    result=abs(p1[0]-p2[0])+abs(p1[1]-p2[1])
    return result
  def telemetry(self):
    self.sensor={
      "robotpos": (0,1),
      "camera": "cam02.jpg",
      "attention": (10,3),
    }
    # search robotpos and attention
    print("gathering telemetry ...")
    for i in self.data:
      if "pos" in self.data[i]:
        dist=self.getdist(self.sensor["robotpos"],self.data[i]["pos"])
        if dist<=1:
          print("robotpos near",i)
        dist=self.getdist(self.sensor["attention"],self.data[i]["pos"])
        if dist<=1:
          print("attention near",i)
  def parser(self):
    line=input("user: ") # manuel input
    line=line.split()
    print("search database ...")
    for i in line:
      if i in self.data:
        print(i,self.data[i])
      else:
        print(i,"not found")
    

if __name__ == '__main__':
  c=Chatbot()

No comments:

Post a Comment