March 07, 2020

Building a minimalist database from scratch

From a historic point of view, many database management system were developed over the years: dbase was used in the 1980s, MS-Access in the early 1990s, since the 2000s MS-Access used in combination with SQL server and since 2010 the situation has become very complex, because many companies are experimenting with Java, Linux, C# and PHP.

The idea is to throw away everything and reinvent a database from scratch with Open Source software. A good starting point is the sqlite software which is available out of the box in all Linux distributions. The problem is, that sqlite is only a small part of an overall database management system. To make things more comfortable a middle layer and a frontend is needed:

backend sqlite -> middleware python -> frontend python

Programming a python frontend is not very complicated. It has to do with drawing windows on the screen the wxwidget library and add some animations plus sounds to make the game more pleasant. The more complicated part is the middleware. To introduce the business logic layer we have to analyze who to interact with a normal sqlite database.

Suppose the user has started the python3 interpreter and is connected to the sqlite database. What the user can do is to submit an SQL statement. He has to type in the SQL request into the command line and gets the feedback from the database. A naive assumption is that the interaction can be improved with the help of a GUI frontend. On the longrun not a GUI frontend is needed but a middlelayer. A business layer is working on the textual layer and allows the user to enter high level textual commands. Instead of typing in:

SELECT * FROM Customers WHERE First_Name='John'

The user enters the command:

middleware.showcustomer(“john”)

The mdidleware program code converts the high level statement into low level SQL commands which are submitted to the sqlite database. The interesting point is, that no GUI interface is needed and the user can interact with the database very comfortable.