Benchmarking operating systems

 Before different operating systems can be compared against each other there is a need to define a scale for an objective judgment. Possible measurement in the past are: number of users, size in Megabytes, or easy of usage.

One important measurement isn't mentioned in the list which is hardware support. Device drivers are a seldom investigated subject in operating systems but they have a great impact on the success or failure of an os. The main difference between Linux and KolibriOS isn't the programming language (C vs. Assembly) but its the hardware support. Linux supports out of the all graphics cards, while KolibriOS is restricted to VESA modes. Linux supports wifi cards while KolibriOS has only support of ethernet cards.

Even if someone likes the idea of using the KolibriOS for daily usage he will notice that most of the hardware isn't working. So he will decide against the system. The main reason why Linux is rejected in favor of Windows can also be explained with device drivers. The support in Windows is better, and additional features like power savings are supported in WIndows, while Linux has only basic support which is provided with a delay of 3 years after new graphics cards are available.

Let us assume that are no existing operating systems available but the goal is to write a perfect operating system from scratch. The core feature for a desktop OS is to support all the hardware out of the box indlucing the advanced features like certain resolutions and Energy saving modes. Each operating system is judged by this ability.

The reason why device drivers are usually ignored as a benchmark criteria is because this important subject is difficult to realize. Programming the device drivers for all the hardware is a large project. Even in the Linux project (which is the largest open source project today) there is no enough manpower available for this task. The assumption is that around 250k different hardware devices are available and programming the drivers for all of them need will produce a binary blob file of 1 gigabyte an more.

A collection of device drivers is the core element of any operating system. Any other part like a c++ compiler, a GUI environment or certain application software can be added later. in case of doubt existing source code can be recompiled for a new operating system, but the device drivers can't. They have to be written from scratch.

Different operating systems are working with different philosophy how to create the device drivers. In the Windows ecosystem the assumption is, that hardware companies are producing the code in a closed source fashion. Linux assumes that a group of voluntiers are writing the hardware drivers for the Kernel while KolibriOS says that the hardware drivers are written in Assembly language for only basic devices like an ethernet card and USB mouse.

The main reason why desktop opeating system are hard to program is because the endless amount of hardware available for desktop PC. There are hundreds of different graphics cards, network cards and soundcards availalable. And each card has multiple parameters which can be changed. Apart from the VESA standard which wasn't updated since decades there is no hardware standard available and every new  hardware has a need for a new driver. Without a driver the device won't interacting with the computer or it will consume too much energy. So we can say that a device driver is the single point of failure in an operating system.

An additional problem is that device drivers are usually written in Assembly or in hardware level C which is both complicated to master and the amount of experts in this subject are low. This makes it unlikely that entire desktop operating system can be created from scratch. In contrast to writing application software, a device driver is needed by any user group. No matter if the PC is used for writing in a letter or programming something in both cases there is a need for a graphics driver otherwise the PC won't work at all.

In summary a device driver collection is difficult to program, has a huge code size, is needed by every user and has to support 250k different devices which results into a very complicated project. Only large organizations have the manpower to create operating systems but not amateurs.