March 25, 2023

The difference between a pocket calculator and GPT-3 enabled Artificial Intelligence

In contrast to AI based software a pocket calculator was understood by the public quite well. Such technology is available at least since the 1970s and lots of books are explaining the inner working. The core element is a CPU which gets programmed in Assembly. And then the machine allows the user to enter a task like “2+4”. From a technical perspective a pocket calculator contains of the CPU which is in easiest case an 8bit model, there is some sort of onboard RAM and very important a program which takes the input of the user and sends instructions to the CPU.
Creating yet another pocket calculator in hardware is easy. Also it is possible to write a python software which emulates a pocket calculator. There are endless amount of software and hardware components available for this purpose and they can be explained easily to newbies. It is more complicated than normal mathematical but the technology is not very advanced.
In contrast, a modern gpt-3 driven Artificial Intelligence works very different to a pocket calculator. Even the system contains of hardware and software components it can't be grasped with the traditional terms used in computer science. Also it is more complicated to newbies what a neural network is about. The inner working of AI can be summarized the following way.
The AI software is able to convert back and forth from natural language into numerical arrays. The input of the user gets converted into numbers, then the system is doing something with the numbers, and the output is converted back into natural language. This number-word engine is the core element in Artificial Intelligence. it allows to solve any problem. Over decades it was unknown how to do so, and it was even unclear if such a transition is needed. Creating word embeddings is sometimes called the symbol grounding problem. For example the word cat is not only a sequence of single characters (C + A + T) but cat is represented in a conceptual space as a number next to other words like dog, mice and so on.
The surprising situation is, that after solving the word embedding problem, it is quite easy to construct a human level Artificial Intelligence. If a problem was reformulated as a numerical mathematical problem, existing computer technology can be applied to it which means the information are stored in the main memory and there are routines which are doing something. The only bottleneck is the transformation back and forth from words to numbers. The core element of any advanced Artificial Intelligence which allows to rebuild the gpt-3 software from scratch is a word embeddings algorithm. Such a software component takes an English sentence and converts it into a mathematical vector. The details of this sophisticated technology are not understood very well and it is a very new approach.