Homecomputers until the 1990s

 Before the advent of today's PC technology which is dominated by Windows and Linux operating system there was a much smaller community available of computer enthusiasts. From 1980-1990 most of todays computer hardware and software was invented in the first place and the magazines which were introducing the subject to a readership were sometimes very well informed. In that period two dominant computer systems were available which was the Commodore 64 and the MS DOS PC.

At first it should be mentioned that on IBM PC hardware until the year 1990 the well known Windows operating won't be running fast enough. The only available operating system for early DOS Computers was DOS itself which is a single user, single tasking operating system. In comparison to the Commodore 64 advantage was that it was much easier to write software for MS DOS than for the C64. 8bit homecomputers with 64kb and less main memory and without any harddrive are not capable in running compiled C programs. The only sense making programming technique is the Assembly language. In contrast, early MS DOS PC until 1990s are working fine with c compilers. These large scale programs including the build in libraries can be installed on the small hard drive (less than 100 MB) and it is possible to write and debug software direct on a MS DOS PC.

The reason why this workflow is described in detail is because its working the same like modern programming workflow in the 2020s. That means, in the last decades the programming itself hasn't changed that much. Typing in C code on a 286 PC and compile it into machine code or typing in java code into a mto a more recent 4 core PC is based on the same abstraction mechanism. The human programmer has a set of libraries and combines existing functions into a new software.

Somebody may argue, that the difference between Assembly language and Turbo C is small because both programming language were invented decades ago. This assumption is wrong. Learning assembly from scratch and write larger software is very complicated while the same task in C can be handled easily. The difference is that the C language is a problem oriented language while assembly is hardware oriented. The typical assembly program is written for a certain CPU and a certain adress space in the main memory, while c program are written around a certain domain like a game or a word processing applications.

The only negative point of the C language are the hardware requirement. C assumes that an entry level 286 MS DOS PC is available which has 600 kb of RAM and a harddisc of 10 MB and more. Its not possible to run  a compiler with less RAM and without a harddrive. Even if some C compilers are available for the C64, it can't be used in reality for writing programs. The cause is that a compiled C program is much slower and will need more RAM than a hand coded assembly program.

The main difference between the C64 and the MS DOS PC Is, that C64 programmers claim that Assembly language is a here to stay. This assumption is the result of the lower hardware of the C64 which prevents to use any other programming language than Assembly. Even if its possible to write Assembly program on the MS DOS PC most programmers prefer a c compiler because the language allows to increase the productivity. Especially if a graphics library is available and the programmer is familiar with the computer its possible to write simple games in a short amount of time, very similar to what today's programmer can archive. in other words the existence of a c compiler is the single cause why MS DOS PC have replaced 8bit homecomputers.

Benchmarking operating systems

 Before different operating systems can be compared against each other there is a need to define a scale for an objective judgment. Possible measurement in the past are: number of users, size in Megabytes, or easy of usage.

One important measurement isn't mentioned in the list which is hardware support. Device drivers are a seldom investigated subject in operating systems but they have a great impact on the success or failure of an os. The main difference between Linux and KolibriOS isn't the programming language (C vs. Assembly) but its the hardware support. Linux supports out of the all graphics cards, while KolibriOS is restricted to VESA modes. Linux supports wifi cards while KolibriOS has only support of ethernet cards.

Even if someone likes the idea of using the KolibriOS for daily usage he will notice that most of the hardware isn't working. So he will decide against the system. The main reason why Linux is rejected in favor of Windows can also be explained with device drivers. The support in Windows is better, and additional features like power savings are supported in WIndows, while Linux has only basic support which is provided with a delay of 3 years after new graphics cards are available.

Let us assume that are no existing operating systems available but the goal is to write a perfect operating system from scratch. The core feature for a desktop OS is to support all the hardware out of the box indlucing the advanced features like certain resolutions and Energy saving modes. Each operating system is judged by this ability.

The reason why device drivers are usually ignored as a benchmark criteria is because this important subject is difficult to realize. Programming the device drivers for all the hardware is a large project. Even in the Linux project (which is the largest open source project today) there is no enough manpower available for this task. The assumption is that around 250k different hardware devices are available and programming the drivers for all of them need will produce a binary blob file of 1 gigabyte an more.

A collection of device drivers is the core element of any operating system. Any other part like a c++ compiler, a GUI environment or certain application software can be added later. in case of doubt existing source code can be recompiled for a new operating system, but the device drivers can't. They have to be written from scratch.

Different operating systems are working with different philosophy how to create the device drivers. In the Windows ecosystem the assumption is, that hardware companies are producing the code in a closed source fashion. Linux assumes that a group of voluntiers are writing the hardware drivers for the Kernel while KolibriOS says that the hardware drivers are written in Assembly language for only basic devices like an ethernet card and USB mouse.

The main reason why desktop opeating system are hard to program is because the endless amount of hardware available for desktop PC. There are hundreds of different graphics cards, network cards and soundcards availalable. And each card has multiple parameters which can be changed. Apart from the VESA standard which wasn't updated since decades there is no hardware standard available and every new  hardware has a need for a new driver. Without a driver the device won't interacting with the computer or it will consume too much energy. So we can say that a device driver is the single point of failure in an operating system.

An additional problem is that device drivers are usually written in Assembly or in hardware level C which is both complicated to master and the amount of experts in this subject are low. This makes it unlikely that entire desktop operating system can be created from scratch. In contrast to writing application software, a device driver is needed by any user group. No matter if the PC is used for writing in a letter or programming something in both cases there is a need for a graphics driver otherwise the PC won't work at all.

In summary a device driver collection is difficult to program, has a huge code size, is needed by every user and has to support 250k different devices which results into a very complicated project. Only large organizations have the manpower to create operating systems but not amateurs.

Introduction to the Linux operating system

Books about the Windows operating system are assuming that the user has never seen a computer before and needs guidance for most tasks. The typical windows book explains, that a computer consists of a mouse, a USB interface, a printer and a monitor and the user is asked to start a program or modify settings in the menu.

It doesn't make sense to transfer this language style to Linux operating systems because Linux wasn't created for computer newbies but for experts. The simple difference between both user groups is, that computer newbies can't program in Python while computer experts are able to do so. As a consequence the typical book about Linux should assume that the user is familiar with the Python language.

The positive effect is, that the explanation what Linux is about can be shorten drastically. Linux allows to execute self written python programs, consists of a powerful package manager and is distributed for free. If the user have no need for such functionality he probably doesn't need open source software and is recommended to use existing Windows software.

The reason why the market share of Linux is much smaller is because the amount of programmers is small. What we can say for sure is that non programmers won't feel comfortable with Linux. Its not about a certain windows manager like gnome vs xfce and it is not about the position of the start menu, but without programming skills in at least one language the Linux OS doesn't make sense for the user.

From a programmers perspective it is pretty easy to understand the Linux operating system. In comparison to write a medium size software project in Python, the interaction with Linux is much easier. There is no need to call a method in a python program, nor implement a recursive function, but the installation of linux is gui driven and apart from clicking on some buttons no further skills are needed. 99% of existing software engineers are able to install and use Linux with ease.

Let us take a closer look into the numbers. There are 1000 million PC worldwide available and around 27 million programmers. This ratio of 2.7% is the maximum market share of Linux on the desktop. It is not possible to growth the market share of Linux above this level because this would imply that non programmers have installed Linux on the desktop PC in favor of Windows.

The Linux kernel as a device driver repository

 Most of the lines of code within the Linux project are about device drivers. Its not about about 1000 lines of code, nor 1 million lines of code but device drivers need around 20 million lines of code within the Kernel. Instead of analyzing what the drivers are doing from a technical perspective there is a need to describe the philosophy.

In classical closed source operaring systems, the device driver is provided by the hardware manufactorer. A certain company is producing a flatbad scanner and has to deliver the hardware itself plus a 3.5" floppy disc which contains of drivers to run the hardware. The same issue is there for a mouse, an usb stick, a camera and so on. In the 1990s it was common that device drivers where delivered on physical discs inside the box of the hardware. The end user was asked to insert the disc into the pc and run a program which was mostly setup.exe to install the drivers. Then and only then the hardware was working.

More recent version of Windows are installing the needed drivers in the background without human interention. The Windows operating system detects with plug and play which hardware is in use and downloads the drivers from the internet. These drivers are mostly writtein in C language and compiled into executable binary programs.

In contrast the linux kernel is working with open source hardware drivers. The Kernel is basically a collection of drivers for getting access to all the devices like cd-rom, ssd, ethernet card and so on. The shared similarity between Windows and Linux is that somebody has write all the drivers. Within the Windows ecodsystem this task is handled decentralized. Each company has to write its own drivers and doesn't explain to the public what the code is about. In contrast, Linux is working with a centralized model. There is only a single kernel and all the drivers are in the kernel.

The focus on the device drivers might explain why apart from the major three operating systems (win, linux and macos) there are no alternative projects available for desktop PC. Everybody who likes to establish a new operating system has to make sure that all the hardware is working with this operating system. The only way for doing so is to write all the needed drivers from scratch. This will take a lot o man years. Because of this single reasson there is no Forth operating system, and smaller projects like Haiku are not working well enough for production machines. The cause is, that most devices won't work with these devices. That means, the proud user of the Haiku OS is plugging in an USB stick into the PC but nothing happens. The OS isn't detecting the hardware and has no executable driver for this hardware.

The major cause why device drivers are released as closed source for Windows is because it is a time consuming task to write the software. A single expert programmer is able to create around 10 lines of code per day. Even if the programmer has access to all the hardware specification and has a lots of experience he will need months up to years until the driver for a certain device was written. It doesn't make sense for a hardware company to release the software as open source because the source code including the ability to write code for new devices is a asset for a company which can't be shared with other comapnies.

The Linux ecosystem is working the opposite way. Its a mandatory rule that all the code has to be released in the open source version. If a certain driver is not available then the device won't work with the kernel. The result is, that the quality of the hardware drivers in Linux is lower and that the amount of drivers is lower. There are lots of hardware available which is supported in Windows but not in Linux. Its not of technical requirements but because of the ecosystem and especially the time consuming effort to write a driver in C.

Suppose it would be possible to create a universal device driver in 10k lines of code which can interact with any possible hardware. Then it would be pretty easy to create new operating systems from scratch. All what is needed is this single device driver and some additional programs can be added. Unfurtunately there are technical limitations which prevents that such a universal driver can be realized. Existing computing hardware is so complex and so different that any single device will need a dedicated driver.

Let us estimate how many different hardware devices are available. suppose a single device like an ethernet card is controlled by driver with 10k lines of code. There are 20 million of codelines in the Linux project for hardware drivers., so the total amount of different devices is 2k. It is equal to a large museum fully equipped with computer hardware from floor to ceilling. In addition, modern computer hardware is more complex than previuos models For example the avarage mouse is equipped with infrared sensors while the typical mouse in the 1990s was using a simple rubber ball to detect the movements. So we can estimate that in the future the complexity will grow further which results into more different devices which have a need for more codelines.

Writing device drivers in C

 The core element of any operating system is a collection of device drivers. These drivers ensures that hardware components like a mouse, the graphics card, ethernet cards and keyboard is available for the end user. The only programming language for implementing a device driver is C. C gets compiled into assembly instructions and this ensures the maximum efficiency.

Suppose the idea is to realize an operating system in Forth with the aim to run it on a stackmachine. In such a case, the c language isn't available but the user is forced to rewrite the device drivers in Forth. This will produce a situation in which all the code isn't written yet but has to be rewritten. It will take lots of man years to rewrite c device driver in a Forth dialect. Even if the programmers are highly motivated, they won't be able to fullfill the task within the next 30 years.

One possible attempt to overcome the Forth bottleneck is a virtual machine and high level languages like BASIC. The basic programming language gets converted into byte code which is executed on a virtual machine. The virtual machine is running on top of a Forth chip so the programmer doesn't need to program in Forth anymore. The only problem is, that BASIC is a high level programming language while device drivers are written in a low level language.

Unfurtunately, it is not possible to execute C in a virtual machine because C code needs direct hardware access. The only way to execute c code is by compiling it into assembly language. But compiling c code into assembly is only possible for register machine, not for stackmachines. There is no such thing available like a C to forth converter, and even it is possible to implement such a thing it can't be applied to device drivers.

Device drivers are an important element of any operating system. They make sure that all the hardware like printers, usb port, webcam and so on are working. Programming an operating system by ignoring the device won't make sense. It seems that only the C  low level language is only option for writing device drivers. This situation makes it unlikely, that operating systems will work on stackmachines.

The problem is not located in technical terms. Forth is a great language for getting direct access to hardware. There are some microcontrollers available which are running with Forth at bare metal. The more serious problem is, that a desktop operating system consists of thousnands of different hardware devices. Writing the code for all the devices will take endless amount of man years.[1] This effort is very costly. Rewriting existing C device drivers into Forth is to expensive. This prevents that such a project gets started. It seems, that the x86 architecture is the only valid computer system which is able to run desktop operating systems.

Perhaps it makes sense to go a step backward and understand why exactly C was choosen in mainstream computing. The goal was to write device drivers which contains of millions of codelines. Instead of writing this code in assembly language which is different for each processor, the idea was to write the device drivers in C. C is more portable than Assembly and is easier to learn. What is needed in addition is a compiler for generating assembly instructions automatically. This paradigm is valid in computing since decades.

The only bottleneck for a c compiler is, that it needs a certain target architecture which is a compiler friendly x86 architecture. Possbile alternatives like a RISC Cpu and especially a Forth cpu are preventing that C code gets converted into assembly instructions.  

The linux kernel contains of at least 5 million lines of code reserved for device drivers.[1] A potential Linux alternative written in Forth has to provide the same functionality. From a technical point of view it is possible to rewrite the device drivers in Forth, but from an economical perspective it doesn't make sense. Its a well known fact that a single programmer can write down only 10 lines of code per day no matter which programming language he prefers. And the open question is who exactly should write all the code in Forth?

There is a certain reason available why all the desktop operating systems were written in C. Because existing code was written mostly in C and it is much easier to add something to an existing codebase than rewriting it from scratch. The untold assumption is, that all the 5 million lines of code are needed, otherwise the computer isn't able to detect or manage a certain hardware for example a graphics card or a network card. The second assumption is, that even the Forth language will need device drivers. It is not possible to write a Forth OS in 10k lines of code which provides the same functionality like the existing device drivers which are written in 5 million lines of code.  This might explain why Forth is not very popular in mainstream computing. Even if the concept is interesting from a theoretical point of view it can't answer the question how to write all the source code which is needed in an operating system. Existing Forth tutorials are explaining to the newbie what a stack machine is about and how to combine Forth words into programs. But this ability is not enough to realize full blown desktop opeating systems in the style of Linux, MacOS or Windows.

In contrast the C language explains very well how to handle complexity. According to the C language paradigm the programmer has to write C code for newly hardware, commit this C into the existing codebase and this will improve the functionality of the Linux kernel. That means, there are 5 million LoC already there, and a new device driver will add around 200 lines of code and its only a detail question how to program the code exactly.

So called Forth systems and stack oriented programming languages like Factorcode are ignoring the problem of device drivers. Especially the aspect how to create millions of codelines to get access to endless amount of existing hardware.

[1] Kadav, Asim, and Michael M. Swift. "Understanding modern device drivers." ACM SIGPLAN Notices 47.4 (2012): 87-98.

Programming an operating system in Forth

 Existing operating systems like Linux and Windows are very huge and have a lots of redundancy. For example in Linux there are many different GUI frameworks avaialble. Also the user has the choice between hundreds of programming languages likes Fortran, C, C++, pascal and so on.

A possible minimalist alternative to Linux would be realized in the Forth language which can be executed on a GA144 Forth cpu. the main advantage of bare metal stack machines is, that they have low register count which results into an energy efficient design. The major cause why Forth is not very popular in computer mainstrea is because it is much harder to program than C. Even with a good tutorial, it is complicated to write down instructions into forth. Even a register based assembly language is easier to explain than Forth.

But this single bottleneck can be solved with a high level language interpreter, for example a BASIC interpreter which is encoded in Forth. There are some examples available from the past. The idea is to write a program in Forth and this program is able to execute a BASIC program. The main advantage is, that newbe programmers are not forced to type in the routine in Forth, but they can do in a normal BASIC dialect.

Let us summarize the idea of a Forth operating system. The kernel is written of course in Forth because this code gets executed with the maximum speed on a stack machine. The kernel has access to the hardware including the graphics display. High level programs like a text editor, a hello world program or a prime number generator are written in BASIC which is executed by the basic interpreter. This ability makes it likely that a larger audience is interested to write new software or port existing software to the Forth operating system.

Estimate the costs for Forth programming

 
The current computing mainstream works with a combination of C programs which are running on x86 CPU. From the perspective of Forth advocates the main critics towards this principle is, that x86 hardware has a huge transistor count and software written in C is not very efficient. Let us investigate how a potential alternative will look like.

The goal is to create Workstation desktop with stackmachines which are programmed in the Forth language. Current desktop ready operating systems like Linux and Windows have a space requirement on the hard drive of around 50 GB in total for all the libraries and programs which is equal to 1250 million lines of code. Because the existing C code base can't be executed on GA144 and similar Forth procssor there is a need to rewrite the entire operating system in Forth. The assumption is, that similar to C a single Forth programmer is able to write 10 lines of code per day. So it will take 342k man years until this project is done The only bottleneck is that such a project will cost a huge amount of money.

Let us go a step backward and describe what the underlying problem is. Existing operating systems are mostly written in C code. In addition the amount of code lines is very huge. There are many attempts available in the past to m odify the development process. Some kernels for operating systems were written in pascal and even assembly lanugage, while other projects were created as a lightweight single floppy disc operating systems. None of these alternative were successful. It seems that only the combination of the c programming language plus a huge amount of code line results into a successful operating system.

This makes it hard to decide for the Forth ecosystem which is operating with a different understanding of excellence. The typical forth project is of course realized in the Forth language and it has a low amount of codelines which is described as high efficiency coding. In contrast, existing operating systems are critized as bloatware.

Nevertheless there is a need to discuss possible future computer architectures because of the end of the moores law. Current x86 CPU have an overheating problem as a result of too many transistors on small space. So there is a need to build hardware with a lower footprint. In the past such cpu design was available for eample the Pentium I was equipped with only 3 million transistors. Unfurtunately, the Pentium I is too slow for modern requirements and putting more voltage to the device won't work.

What is a needed is a low transistor count CPU architecture which can run C programs. A c program is never executed on the baremetal but it is translated first into assembly language. The bottleneck is located in compiler design. Only if a compiler is available for a new CPU it is possible to port existing c code towards this platform. So we can say it is not about ARM cpu, Forth cpu or RISC cpu but what matters are compilers for these hardware.

A compiler is an intermediate between high level C code which is already there and a certain computer hardware which has to be invented from scratch. The existing x86 hardware itself is not very powerful, what makes this hardware relevant is the existence of powerful compiler toolchains which are converting existing software into assembly instructions.

Programming a compiler is perhaps the most advanced topic within software engineering. in contrast to a normal program which is a video game or a spreadhseet application, a compiler can transform one program into another program. it has much in common with an interpreter which is easier to realize but is much slower. A good example for an interpreter written in Assembly is the BASIC interpreter in the Commodore 64 written by Microsoft. This Basic interpreter is able to fetch the next statement in a program and executes it on the 6502 CPU. For doing so the high level BASIC command is converted into low level assembly instructions.

The main task of a compiler / interpreter is to communicate between human programmers who are not familiar with Asembly language and the CPU which accepts only assembly instructions. Without such an interpreter the computer can't execute BASIC programs.

It doesn't make sense to explain to the BASIC newbie that he should earn Assemlby if he likes to paint a graphics on the screen. The reason why the programmer prefers BASIC over assembly is, that it is easier to use and allows him to code the same program in a shorter amount of time. What is needed instead is a fast interpreter or even better a fast compiler which provides an additional layer between man and machine.

Writing a compiler for Forth

 There is a reason available why non x86 cpu architecture are ignored by mainstream computing. Because these hardware isn't compiler friendly. For explaining this term we have to sort possible CPU architecture by their complexity.

The most easy to build hardware is a stackbased Forth CPU. Such a machine can be realized with a low transistor count. A Forth cpu supports a limited amount of assembly instructions. It has no registers but only a single datastructure which is the stack. The next logical step in processor design is the RISC architecture. RISC stands in the middle between CISC and stack machines. Typical examples for RISC machines are the MIPS cpu which has some registers but very few. On the other end of the scale there are full blown AMD64 compatible x86 processor like the famous Intel Core i series which is used in mainstream computing and it is powering more than 1 billion desktop PC worldwide.

The acceptance of RISC CPU is low, while the market share of Forth CPU is nearly zero. Both processors are difficult to program or to explain it more technically, it is difficult to write a C compiler for these systems. There are some C compilers available for MIPS processors but they are complicated because a lot of optimization is needed. in contrast writing a c compiler for AMD64 is much easier, because the underlying hardware provides more high level assembly instructions.

The best way to program Forth CPU and also MIPS processors is by typing in the assembly instruction manually. This is equal to avoid any compiler inbetween and the user has to think like the mentioned cpu. Its obvious that most programmers are not interested in such an interaction because it takes endless amount of time to program complex software direct in assembly. This situation prevents an upraising of RISC and stackbased forth CPUs.

What mainstream programmers are doing all the time is to formulate software in C. C is the only important language in modern software development. Nearly all the operating systems like Linux, windows, MacOS and even Haiku are written in C/C++. The main advantage over Assembly instructions is, that C code can be written much faster, this allows to create full blown GUI systems including libraries. The result is, that low efficiency CPU design like the x64 processor is prefered over advanced chip design like RISC and stack machines.

A possible attempt to make non x86 processors more popular would be the existence of advanced compilers. From a technical perspective every CPU is turing capable, that means the same algorithm written for CISC cpu can also be executed on a stackmachine. The only bottleneck is, that somebody has to create the code first. In modern computing the automatical compilation process will generate the code. So there is a need to create / program modern C compilers which are able to create code for targets like Forth CPUs and for MIPS cpu.

From a technical perspective, a compiler is a translator. it takes a C program as input and generates Assembly instruction as output. In case of a stackmachine the needed assembly instructions have a certain format which is known as Forth, or as Forth code. A forth like stackmachine is minimalist computer which is of course controlled by a program. This program needs to be written before useful behavior can be generated.

To understand the pros and cons of stack based machines better it makes sense to take a closer look into MIPS assembly. Risc based MIPS cpus have the role of an inbetween. They are not as minimalist as Forth but they are less complex than x86. MIPS cpu have an integrated stack which allows to push and pop values. So there is a similarity to Forth. in contrast to Forth, mips provides further storage capacity and more complex commands. Mips can be programmed in Assembly language and in high level C as well.  of course the assembly language is more efficient and especially for embedded systems it is the prefered choice of programmers. On the other hand, C has a faster development cycle so there is need to use this high level language as well.

What we can say for sure is, that mips assembly and Forth assembly are both examples for low level language. Even Forth advocates are claiming that Forth is also a high level language, the claim can be rejected. Because Forth is different from C. C is a high level language because it allows to formulate algorithms in a non CPU perspective. A  C programmer doesn't need to know how many registers the underlying hardware has or what a push to the stack is about. A c programmers writes the code only in C and then the compiler generates the machine instructions.