History of the symbol grounding problem

 Symbol grounding is a multidisciplinary approach which has evolved over years. The follwoing timeline shows the milestones during the development. First important innovation was the invention of written language thousands of years ago, in the year 1844 the morse code was invented to transmit signs over distance. In 1968, the SHDRLU project was initiated which was a text to graphics interface on a computer. In 1991 Rodney brooks recognized that robots need to interact with the environment, in 2003 a voice controlled robot was built by Deb Roy and in the year 2023 the Lingo-1 Vision language model was published for controlling self driving cars.

The shared similarity between these innovations is the focus on natural language for connecting two systems. A speaker sends a command to a hearer. Implementing such an interface on a computer allows to teleoperate a robot with language. This is the precondition for automation. 

The crucial building block for symbol grounding is natural language which is used as a reference system to the external world. Thanks to predefined words for objects and activities its possible to label sensory data and receive high level commands. A robot can recognizhe that he stands in front of an obstacle, and a human operator can submit a commoand to the robot like "move around the obstacle to the left". It took decades and even centuries until human researchers have discovered the power of natural language and were able to implement textual interface for robots. Its likely that future robots invented in 20 years from now will work with the same principle.

3300 BC,Cuneiform writing system in Mesopotamia
1500 BC,sundial showing the time of the day
600 BC,Latin alphabet available in Italy
322 BC,correspondence theory of truth by Aristotle
1386,Salisbury Cathedral tower clock with a bell
1440,printing press by Johannes Gutenberg
1505,Pomander Watch by Peter Henlein
1792,optical telegraph by Claude Chappe
1844,morse code by Samuel Morse
1870,Engine Order Telegraph by William Chadburn
1876,commercial typewriter by Remington
1878,chronophotography "The Horse in Motion" by Eadweard Muybridge
1903,Telekino remote controlled boat by Leonardo Quevedo
1915,Therblig notation by Frank Gilbreth
1915,rotoscoping animation technique by Max Fleischer 
1920,AAC Communication board by F. Hall Roe
1928,Labanotation dance notation by Rudolf von Laban
1929,Televox robot by Westinghouse
1930,motion tracking by Nikolai Bernstein
1949,Turing test by Alan Turing
1954,Georgetown-IBM Experiment with russian translation
1959,Pandemonium architecture by Oliver Selfridge
1962,ANIMAC motion capture by Lee Harrison III
1963,ASCII code
1966,ELIZA chatbot by Joseph Weizenbaum
1968,SHRDLU natural language understanding by Terry Winograd
1971,Lexigram for communicating with apes by Ernst von Glasersfeld
1977,Zork I text adventure by Tim Anderson
1977,Tour model instruction following by Benjamin Kuipers MIT AI lab
1980,Chinese room argument by John Searle
1980,Commentator scene description by Bengt Sigurd
1980,Finite State machine in Pacman videogame by Tōru Iwatani
1981,Karel the robot programming language by Richard Pattis
1983,MIDI music protocol
1983,M.I.T. Graphical Marionette by Delle Maxwell
1984,Castle Adventure by Kevin Bales
1987,Maniac Mansion point&click adventure by Ron Gilbert
1987,Vitra visual translator by Wolfgang Wahlster
1989,Speech activated manipulator SAM by Michael Brown
1990,Physical Grounding Hypothesis by Rodney Brooks
1990,paper "The symbol grounding problem" by Stevan Harnad
1991,paper "Intelligence without Representation" by Rodney Brooks
1993,AnimNL computeranimation by Norman Badler
1993,conceptual spaces by Peter Gardenfors
1994,Abigail scene recognition by Jeffrey Siskind
1996,Interaction machines by Peter Wegner
1998,Rocco Robocup commentator by Dirk Voelz
1999,trec-8 Text Retrieval Conference
2000 Kismet social robot by Cynthia Breazeal M.I.T.
2003,M.I.T. Ripley robot by Deb Roy
2006,Marco route instruction following by Matt MacMahon
2007,Simbicon computer animation by Michiel Panne
2010,Motion grammar by Mike Stilman
2010,M.I.T. forklift by Stefanie Tellex
2011,IBM Watson Question answering by David Ferrucci
2013,Word2vec algorithm by Tomas Mikolov
2015,Poeticon++ trajectory recognition by Yiannis Aloimonos
2015,DAQUAR VQA dataset by Mateusz Malinowski
2017,Transformer Architecture by Ashish Vaswani
2020,Vision language model by different authors
2023,Wayve Lingo-1 self driving car