The paradox success of the Linux operating system

 There are rumors out there that Linux on the desktop is dead. Even Linux advocates have to admit, that there are too many Linux distributions available and that the market share of the Linux desktop is low and will become smaller in the future. It is hard or even impossible to convince a Windows user to give the Open Source project a chance. This sad situation is new, some decades ago in the late 1990s, Linux was discussed differently. It was seen as a valid alternative to Microsoft Windows and there were serious attempts started to use it on production ready PCs.

These projects were stopped a long time ago and nearly all of them were a failure. The end users were not convinced, that Linux is better. Even in an university setup  it is nearly impossible to install Linux on one of the workstations computers. Beginners and expert users as well have decided that Windows is much better.

The paradox situation is, that at the same time, Linux has become a great success. The gnome desktop environment looks better than ever, and the Linux kernel is able to communicate with more hardware devices than in the 1990s. How can it be that at the same time, Linux is dismissed by the world, and the same world supports Linux that much? It is indeed hard to explain this paradox, but let us try.

What we can say for sure is, that the idea of embedding Linux into existing Windows installation has become a great success. There are lot of books available about Windows subsystem on Linux, running Linux in a virtual machine and using winget for installing Open Source software in Windows. These attempts can be called a Linux light. Instead of using Linux as an alternative to windows, a modern Linux distribution is started as an app inside Windows or inside the browser. It seems that such kind of interaction fulfills the needs of the user better. From a technical perspective the difference is the absence of the Linux kernel.

A classical Linux distribution like Ubuntu is started on bare metal hardware. The hardware is controlled by the kernel and on top the Xwindow system gets started. Such kind of Linux-only installation has to be called a dead end. What is used in reality is to use Windows as the bare metal operating system for a computer and then run Linux programs and Linux tools inside a windows computer. There are many options available for doing so.

To understand the situation we have to describe the components of a linux system. The low level layer is the kernel, the visible layer on top is the GUI. Both elements are not wanted by the mainstream PC user. As a kernel he is using the Windows operating system, and the GUI frontend is also rendered by Windows. What is interesting and new for the enduser are the Linux middleware programs

These programs are seldom described. For example it is the bash scripting language, the awk tool, the ghostscript program, the grep command, the ffmpeg tool, C++ compilers, webservers and the pandoc text converter. These programs are running in between the Linux kernel and the Gnome gui. They do not belong to the kernel and they do not belong to the Gnome GUI but they are hidden inside a Linux distribution.

The windows subsystem for Linux is doing the same. It is  providing access to Linux middleware tools. It seems, that the average Windows user has a need for these tools. They are powerful, highly mature and come free of charge. Also there are no Windows programs available which are providing the same functionality.

Let us describe the situation from the outside perspective. The classical Linux distribution is rejected by Windows users. The users have no need to install an alternative operating system and they don't need buggy hardware driver written by Linux Torvalds. On the other hand, the average Windows user has huge demand for the Linux middleware tools which are preinstalled in any Linux system. The user likes to run these programs within Windows.

This kind of description assumes that Linux consists of two parts. Some elements of a Linux distribution are dismissed by Windows users which are the kernel and the gui, while other parts (the middleware) are highly wanted.

In the past, there was a clear border between Linux and windows available. Programs like Latex, apache webserver, the pandoc tool or awk were only available in Llnux. For getting access the user was forced to boot its computer with a Linux distribution which was highly complicated. The idea was that the user has to decide between two operating system. Either he is using Windows and can use only Windows software, or he installs Linux and then he gets access to bash including powerful programs.

In more recent years, there has become a new paradigm available which can be called Linux without Linux. This paradox description is referencing to a situation in which linux has become a great success and a failure at the same time. What we can say for sure is, that the Linux kernel and the Gnome desktop both are a failure. Booting a PC with the Linux kernel and run the gnome desktop environment is a rare situation which is rejected by 99.9% of the PC users. The only thing what they are booting on their PC is the normal Windows installation including the windows kernel and the Windows GUI. The attempt to replace these operating system elements with open source alternatives has failed. But this doesn't means, that the users are not interested in Linux. They want to run a different sort of software which is the mentioned middleware. This middleware is missing in a normal Windows system because of different reasons. Running textual commands was a sign of a working Unix system while Windows was able to run GUI programs. These GUI programs can't compete with command line tools. Unix commands like grep, find, latex and other programs are very powerful commands which can't be replaced by windows GUI software.

Over decades windows user were not allowed to run these textual commands. They were forced to decide. Either they are using Windows or they are using Linux. But from a technical perspective it is possible to combine both systems into a single one. An existing Windows installation can be enhanced with Unix tools. This allows to run bash scripts, start a webserver, format latex documents and search in textfiles similar to a unix. Such kind of enhanced Windows can't be called a classical Windows OS, and it is not a Linux distribution, but its something in between. It combines the Windows kernel, the Windows GUI with Unix tools like AWK and bash scripting.

The chance is high that such kind of inbetween operating system will become the future. Many Windows only users are interested in these tools. It allows them to use Linux without installing bare metal. In contrast there is an alternative approach available which should be explained briefly. From a technical perspective it is possible to boot a linux distribution on a physical machine and then run Windows inside a virtual machine. Some Linux advocates are recommending this approach. The idea is that Linux will become a host operating system for Windows machines which are running in an emulator. This approach sees Linux as the only operating system on a computer and would like to put Windows into a sandbox. The interesting situation is, that such kind of Linux only approach isn't used in reality very much. Because it means basically to uninstall Windows, boot Linux on bare metal PC which isn't wanted by the average user. for different reasons.

In contrast, the idea behind WSL is the other way around. Linux or parts of Linux are put into a sandbox and then the user can run Linux as an app inside a working Windows computer. This idea is prefered and it is well documented in the literature. The concept has not a single name but it can be called Linux without Linux. The microsoft term is "Windows-Subsystem für Linux" which means to run a Linux distribution like Ubuntu in a virtual machine.

Its difficult how frequently WSL is used in the reality. The only valid information are available for the opposite. According to the market share for operating systems we can say that 0% of the PC are running Linux. while at the same time 90% of the computers are running Windows. The result is, that if someone likes to run Unix commands on a PC he will do so with WSL or he can install cygwin which also runs in Windows. This development has pros and cons. The negative perspective is, that Linux is indeed dead. Linux was a project to replace the windows operating system with open source hardware drivers. This project has failed. Many windows users have tried out a Linux distribution and they have decided against the software. The positive argument is, that Unix tools which are delivered inside a Linux distribution can become a success in the future. Tools are like grep and awk are not started to replace Windows as an operating system, but they will run on any operating system. Similar to a four in a row game or a text editor.

The Unix middleware is described in detail in [1][2]. It contains of basic commands like grep, ls and awk but there are also high level commands like ssh, zip, g++, git, and latex. The working thesis is, that Windows users have a great interest into these middleware tools. At the same time, these Windows user have no need for other component of a Linux distribution which is the kernel and the gnome system. The reason is, that Windows has a built in kernel and a built in GUI layer which are more powerful than the Linux kernel and gnome. It doesn't make sense to replace the windows kernel with the Linux kernel because this would be equal that most of the hardware won't running anymore. The only Linux component which is superior over the Windows counterpart is the middle layer. Windows has no or only poor tools for compressing zip files, for searching fulltext in files or for creating a git repository. These objectives are fulfilled by Unix tools with ease. So it makes sense to combine the best of both worlds.

[1] https://en.wikipedia.org/wiki/List_of_Unix_commands
[2] https://en.wikipedia.org/wiki/Cygwin