Minimal chatbot with distributed representation

 Existing large language models (LLM) are highly complex systems which are impossible to explain to newbies. The following blog post consists of a minimal example.

The human to chatbot interaction works with a vocabulary of 100 words stored in a word embeddings matrix of 20 dimensions. The human enters a sentence which gets converted into a 20d vector and the chatbot responds also with a vector.

================================================================================
          LOG FILE: USER-CHATBOT INTERACTION SESSION #0842
          SYSTEM: MINI-ENGLISH 100-WORD NEURAL COMMUNICATION INTERFACE
          ENCODING: 20-DIMENSIONAL CONTINUOUS DENSE VECTOR [d1..d20] (0.00-1.00)
          DATE: 2026-09-19 10:04:12 UTC
================================================================================

[SYSTEM INIT]
- Vocabulary Size: 100 Words (Discrete Lexicon Index: W000 - W099)
- Input Representation: Sequence of 20D Continuous Floating-Point Vectors
- Output Representation: Sequence of 20D Continuous Floating-Point Vectors
- Vector Quantization: L2 Euclidean Distance Matching (Threshold: < 0.15)
- Session Status: CONNECTED

--------------------------------------------------------------------------------
[TURN 01 - HUMAN INPUT]
Raw Selection: ["What", "Is", "Sun", "Color"]
Sequence Length: 4 Tokens

Input Vectors (20D Matrix):
  Token 1 ("What"):
    [0.91, 0.05, 0.12, 0.88, 0.02, 0.10, 0.00, 0.45, 0.11, 0.02, 0.80, 0.15, 0.03, 0.05, 0.92, 0.01, 0.10, 0.22, 0.05, 0.14]
  Token 2 ("Is"):
    [0.10, 0.02, 0.85, 0.12, 0.01, 0.05, 0.00, 0.90, 0.02, 0.11, 0.20, 0.04, 0.01, 0.08, 0.15, 0.00, 0.05, 0.10, 0.02, 0.05]
  Token 3 ("Sun"):
    [0.05, 0.95, 0.08, 0.10, 0.92, 0.88, 0.70, 0.15, 0.85, 0.00, 0.05, 0.10, 0.90, 0.12, 0.02, 0.08, 0.60, 0.05, 0.88, 0.10]
  Token 4 ("Color"):
    [0.12, 0.88, 0.20, 0.05, 0.15, 0.30, 0.10, 0.25, 0.10, 0.95, 0.08, 0.20, 0.15, 0.02, 0.10, 0.05, 0.12, 0.80, 0.10, 0.25]

[ENCODER PROCESSING]
- Reading sequence [4x20]...
- Temporal hidden state updated. Intent classified: [QUERY: ASTRO_PROPERTY]

[DECODER GENERATION & QUANTIZATION]
- Step 1: Raw Output Vector -> L2 Nearest Match: W082 ("Sun")     | Dist: 0.012
- Step 2: Raw Output Vector -> L2 Nearest Match: W018 ("Is")      | Dist: 0.008
- Step 3: Raw Output Vector -> L2 Nearest Match: W091 ("Yellow")  | Dist: 0.034
- Step 4: Raw Output Vector -> L2 Nearest Match: W095 ("White")   | Dist: 0.041
- Step 5: Raw Output Vector -> L2 Nearest Match: W099 ("<EOS>")   | Dist: 0.002

[TURN 01 - CHATBOT RESPONSE]
Reconstructed Text: "Sun Is Yellow White"

Output Vectors (20D Matrix):
  Token 1 ("Sun"):
    [0.05, 0.95, 0.08, 0.10, 0.92, 0.88, 0.70, 0.15, 0.85, 0.00, 0.05, 0.10, 0.90, 0.12, 0.02, 0.08, 0.60, 0.05, 0.88, 0.10]
  Token 2 ("Is"):
    [0.10, 0.02, 0.85, 0.12, 0.01, 0.05, 0.00, 0.90, 0.02, 0.11, 0.20, 0.04, 0.01, 0.08, 0.15, 0.00, 0.05, 0.10, 0.02, 0.05]
  Token 3 ("Yellow"):
    [0.08, 0.82, 0.15, 0.02, 0.10, 0.25, 0.05, 0.20, 0.08, 0.98, 0.05, 0.18, 0.10, 0.01, 0.08, 0.02, 0.10, 0.85, 0.08, 0.20]
  Token 4 ("White"):
    [0.02, 0.85, 0.10, 0.01, 0.08, 0.20, 0.02, 0.15, 0.05, 0.99, 0.02, 0.12, 0.05, 0.00, 0.05, 0.01, 0.08, 0.90, 0.05, 0.15]

--------------------------------------------------------------------------------
[TURN 02 - HUMAN INPUT]
Raw Selection: ["Why", "Sky", "Blue"]
Sequence Length: 3 Tokens

Input Vectors (20D Matrix):
  Token 1 ("Why"):
    [0.95, 0.02, 0.10, 0.91, 0.01, 0.08, 0.00, 0.30, 0.08, 0.01, 0.85, 0.10, 0.02, 0.04, 0.95, 0.00, 0.08, 0.15, 0.02, 0.10]
  Token 2 ("Sky"):
    [0.04, 0.90, 0.06, 0.08, 0.85, 0.80, 0.65, 0.10, 0.80, 0.00, 0.04, 0.08, 0.85, 0.10, 0.01, 0.05, 0.55, 0.04, 0.80, 0.08]
  Token 3 ("Blue"):
    [0.09, 0.85, 0.18, 0.04, 0.12, 0.28, 0.08, 0.22, 0.09, 0.96, 0.06, 0.19, 0.12, 0.01, 0.09, 0.03, 0.11, 0.88, 0.09, 0.22]

[ENCODER PROCESSING]
- Reading sequence [3x20]...
- Context memory appended from Turn 01 (Atmospheric / Light query scope).

[DECODER GENERATION & QUANTIZATION]
- Step 1: Raw Output Vector -> L2 Nearest Match: W080 ("Light")    | Dist: 0.028
- Step 2: Raw Output Vector -> L2 Nearest Match: W045 ("Scatter")  | Dist: 0.052
- Step 3: Raw Output Vector -> L2 Nearest Match: W030 ("In")       | Dist: 0.011
- Step 4: Raw Output Vector -> L2 Nearest Match: W084 ("Air")      | Dist: 0.039
- Step 5: Raw Output Vector -> L2 Nearest Match: W099 ("<EOS>")    | Dist: 0.001

[TURN 02 - CHATBOT RESPONSE]
Reconstructed Text: "Light Scatter In Air"

Output Vectors (20D Matrix):
  Token 1 ("Light"):
    [0.10, 0.88, 0.25, 0.15, 0.70, 0.65, 0.40, 0.30, 0.60, 0.50, 0.10, 0.15, 0.75, 0.05, 0.10, 0.04, 0.45, 0.50, 0.65, 0.18]
  Token 2 ("Scatter"):
    [0.78, 0.20, 0.40, 0.30, 0.15, 0.10, 0.05, 0.55, 0.20, 0.10, 0.35, 0.60, 0.25, 0.80, 0.20, 0.12, 0.15, 0.10, 0.20, 0.45]
  Token 3 ("In"):
    [0.02, 0.05, 0.10, 0.05, 0.02, 0.01, 0.00, 0.12, 0.05, 0.02, 0.05, 0.02, 0.01, 0.02, 0.05, 0.98, 0.02, 0.05, 0.02, 0.01]
  Token 4 ("Air"):
    [0.03, 0.92, 0.05, 0.06, 0.80, 0.75, 0.10, 0.08, 0.70, 0.00, 0.03, 0.06, 0.80, 0.08, 0.01, 0.04, 0.50, 0.03, 0.75, 0.06]

--------------------------------------------------------------------------------
[TURN 03 - HUMAN INPUT]
Raw Selection: ["Where", "Is", "Water"]
Sequence Length: 3 Tokens

Input Vectors (20D Matrix):
  Token 1 ("Where"):
    [0.88, 0.04, 0.15, 0.85, 0.02, 0.12, 0.00, 0.40, 0.10, 0.03, 0.82, 0.14, 0.04, 0.06, 0.88, 0.02, 0.12, 0.20, 0.04, 0.15]
  Token 2 ("Is"):
    [0.10, 0.02, 0.85, 0.12, 0.01, 0.05, 0.00, 0.90, 0.02, 0.11, 0.20, 0.04, 0.01, 0.08, 0.15, 0.00, 0.05, 0.10, 0.02, 0.05]
  Token 3 ("Water"):
    [0.04, 0.96, 0.05, 0.08, 0.88, 0.82, 0.15, 0.10, 0.78, 0.00, 0.04, 0.08, 0.82, 0.10, 0.01, 0.05, 0.50, 0.04, 0.78, 0.08]

[ENCODER PROCESSING]
- Reading sequence [3x20]...
- Scope shift detected: [QUERY: GEOGRAPHY_LOCATION]

[DECODER GENERATION & QUANTIZATION]
- Step 1: Raw Output Vector -> L2 Nearest Match: W086 ("Ocean")   | Dist: 0.019
- Step 2: Raw Output Vector -> L2 Nearest Match: W022 ("And")     | Dist: 0.005
- Step 3: Raw Output Vector -> L2 Nearest Match: W087 ("River")   | Dist: 0.021
- Step 4: Raw Output Vector -> L2 Nearest Match: W099 ("<EOS>")   | Dist: 0.001

[TURN 03 - CHATBOT RESPONSE]
Reconstructed Text: "Ocean And River"

Output Vectors (20D Matrix):
  Token 1 ("Ocean"):
    [0.02, 0.98, 0.04, 0.07, 0.90, 0.85, 0.20, 0.12, 0.82, 0.00, 0.03, 0.07, 0.85, 0.12, 0.01, 0.04, 0.52, 0.03, 0.80, 0.07]
  Token 2 ("And"):
    [0.01, 0.01, 0.05, 0.02, 0.01, 0.00, 0.00, 0.05, 0.01, 0.01, 0.02, 0.01, 0.00, 0.01, 0.02, 0.05, 0.01, 0.02, 0.01, 0.99]
  Token 3 ("River"):
    [0.03, 0.95, 0.05, 0.08, 0.87, 0.81, 0.18, 0.11, 0.79, 0.00, 0.04, 0.08, 0.81, 0.11, 0.01, 0.05, 0.49, 0.04, 0.77, 0.08]

--------------------------------------------------------------------------------
[SESSION TERMINATED BY USER]
================================================================================