Dataset for grounded language

 A DIKW pyramid consists of 4 layers for describing multiple abstraction level for a game state. In the context of a warehouse robot the following dataset is available.  The task for the neural network is to translate between the layers, this allows a human to machine communication. Human are expressing in full english sentences, while machines are measuring the environment in numerical sensor values.

[
  {
    "id": 1,
    "data": {"pos": [40, 10], "rgb_color": [255, 0, 0], "dist": 2.0, "trajectory": "traj3", "battery": 0.7},
    "information": ["box", "red", "kitchen", "entrance"],
    "knowledge": "Pick up the red box in kitchen room"
  },
  {
    "id": 2,
    "data": {"pos": [15, 5], "rgb_color": [0, 0, 255], "dist": 1.5, "trajectory": "traj1", "battery": 0.65},
    "information": ["box", "blue", "corridor", "obstacle"],
    "knowledge": "Move the blue box out of the corridor to clear the obstacle"
  },
  {
    "id": 3,
    "data": {"pos": [60, 30], "rgb_color": [255, 255, 0], "dist": 5.0, "trajectory": "traj_return", "battery": 0.15},
    "information": ["charging_station", "yellow", "dining_room", "low_battery"],
    "knowledge": "Aborting task: Return to dining room for immediate charging"
  },
  {
    "id": 4,
    "data": {"pos": [45, 12], "rgb_color": [100, 70, 20], "dist": 0.5, "trajectory": "none", "battery": 0.58},
    "information": ["obstacle", "brown", "kitchen", "entrance", "blocked"],
    "knowledge": "Wait for 10 seconds: Entrance is blocked by a moving obstacle"
  },
  {
    "id": 5,
    "data": {"pos": [10, 80], "rgb_color": [200, 200, 200], "dist": 10.0, "trajectory": "traj_search", "battery": 0.9},
    "information": ["room", "corridor", "empty", "pickup_zone"],
    "knowledge": "Scanning corridor: No items found in designated pickup zone"
  },
  {
    "id": 6,
    "data": {"pos": [33, 44], "rgb_color": [0, 255, 0], "dist": 0.1, "trajectory": "traj_dock", "battery": 0.5},
    "information": ["box", "green", "dining_room", "drop"],
    "knowledge": "Drop the green box at the delivery point in the dining room"
  },
  {
    "id": 7,
    "data": {"pos": [5, 5], "rgb_color": [255, 0, 0], "dist": 2.2, "trajectory": "traj_safety", "battery": 0.45},
    "information": ["hazard", "red", "corridor", "liquid_spill"],
    "knowledge": "External command: Avoid red marked area due to a spill in the corridor"
  },
  {
    "id": 8,
    "data": {"pos": [50, 50], "rgb_color": [255, 255, 255], "dist": 0.0, "trajectory": "none", "battery": 0.4},
    "information": ["inventory_list", "mismatch", "kitchen", "room"],
    "knowledge": "Import external data: Re-scan kitchen room to update missing inventory"
  }
]