Solving the first scene in Maniac Mansion with a DIKW pyramid

 Symbol grounding means basically to convert abstract description into detailed description. A concrete example with 3 layers for the first scene of the point&click adventure Maniac Mansion is shown next. The textual description can be understood by a human easily but can't be submitted directly to a computer. In contrast, the low level "pyautogui_commands" are hard to read for a human but can be processed by a computer program with ease.

Symbol grounding means basically that an algorithm converts high level descripiton into low level commands. With such a grounding algorithm its possible to script the game by providing textual description and the Artificial intelligence converts these description into mouse movements which are submitted to the SCUMMVM engine.

{
  "notecard_id": 1,
  "scene_title": "The Front Yard",
  "content": {
    "textual_description": {
      "objective": "Gain entry to the Edison Mansion.",
      "key_points": [
        "Start with Dave outside the main gate.",
        "Walk toward the front door of the mansion.",
        "The door is locked; the key is hidden nearby.",
        "Look under the doormat to find the silver key.",
        "Use the key to unlock the door and enter."
      ]
    },
    "low_level_representation": {
      "resolution_reference": "800x600",
      "actions": [
        {
          "step": 1,
          "action": "Select Verb: WALK TO",
          "pixel_coords": [120, 480],
          "note": "Clicking the 'Walk to' verb in the UI tray."
        },
        {
          "step": 2,
          "action": "Target: Front Door",
          "pixel_coords": [400, 300],
          "note": "Moving the character to the mansion entrance."
        },
        {
          "step": 3,
          "action": "Select Verb: PULL",
          "pixel_coords": [250, 480],
          "note": "Preparing to move the mat."
        },
        {
          "step": 4,
          "action": "Target: Doormat",
          "pixel_coords": [400, 420],
          "note": "Revealing the hidden key."
        },
        {
          "step": 5,
          "action": "Select Verb: PICK UP",
          "pixel_coords": [50, 520],
          "note": "Collecting the key."
        }
      ]
    },
    "pyautogui_commands": [
      "import pyautogui",
      "pyautogui.PAUSE = 0.5",
      "# Walk to door",
      "pyautogui.click(120, 480)",
      "pyautogui.click(400, 300)",
      "# Pull mat",
      "pyautogui.click(250, 480)",
      "pyautogui.click(400, 420)",
      "# Pick up key",
      "pyautogui.click(50, 520)",
      "pyautogui.click(405, 425)"
    ]
  }
}