Is Programming in general obsolete?

Many blogposts were written about certain programming language who are outdated. There are rants available about Java, the C++ language, Python and of course many rants were written against Forth which is an esoteric language used in microcontroller programming. It's some kind of sport to critize all these languages and especially it make sense to explain why the C language is no longer recommended.

The question remains open, if Python, C, C++ and Java is all obsolete. And if new developments like Go, Ruby or Javascript doesn't fulfill the needs of the programmer, what is the programming language of the future? Instead of an answer i'd like to formulate it as a hypothesis. Programming in general is outdated. That means, in a world in which operating systems, games and applications are already written and programming is some kind of mainstream, there is no reason why somebody should learn programming.

To explain this paradox situation we have to go some years back. In the late 1980s, programming was a must have and was equal to using a computer at all. In the early 1990s, programming has become more important because this was the time of commercial software development. It was the area of object oriented programming. Languages like C++ and Java were widespread used to create large amount of software. Today there is the situation, that all the software is written already, and many programming languages were invented. The problems which have to solved in the future doesn't have to do with programming.

This sounds a bit uncommon, because programming was the answer to everything in the past. Let me give an example. In the past, the users had a certain need for a game. This specification in the 1980s was, that the game should provide smooth scrolling and a fancy background sound. To solve the issue, the game was programmed in the C language and this solved the issue. In the 1990s the needs were slightly different. A typical need in the 1990s was, that no webserver was available to deliver html documents. The answer to the problem was again to write a webserver. This time the language changed to C++ and it solved the problem too.

Today we have the situation, that with modern languages like C++, C# or PHP it's very easy to create all sorts of software. No matter if the project is about a game, a webserver, the script of a website or a desktop application. There is always a best practice programming language which can solve the issue. From a technical point of view, the existing languages like C++ and other solve the programming problem better than ever. And at the same time, the need for programming software is no longer there.

Software programming was the core discipline of classical computer history. It was located within commercial companies like Microsoft and was discovered by amateurs quickly. With the Python language it has become within the reach of a programmer newbie to create his own Desktop application within 1 month. And if doesn't like to write the sourcecode by himself, he can buy all sorts of the software in the Windows app store which was created by experts. But how exactly are software and software programming tools able to help current needs of computer users? Or to formulate the question the question quite different. The computer world in the past was dominated by programmers who have written software for non programmers. In the current age, everybody is a programmer. But the programmers believe, that there is a knowledge gap.

Perhaps a small example make sense. Suppose somebody is fluent in 2 programming languages because he has realized a practical project. The languages are Java and Python which are both excellent languages. Now the person is attending a university and tries to utilize this knowledge in a teaching experience. He has to search for newbies who can't programming at all and who likes to learn at least one language. Does the programmer finds non programmers at the university? No, that is not possible because programming is part of the mainstream culture and become familiar with Java is not the exception but the normal case. That means, there are 100 computerstudents who are all Java programmers but nobody in the class likes to learn Java from somebody else. As a result, the knowledge gap is no longer there and the situation is different from the 1990s.

The answer to the problem is to search for a new kind of knowledge gap. Is there a discipline available in which the wisdom is located at a few wizards, while the mainstream is not familiar with new technology at all? Sure, such a discipline is available but it's not called programming. Programming has become very democratic, that means, everybody is able to answer questions at Stackoverflow. Using a compiler or an interpreted language is no longer an esoteric discipline which is teached seldom, but it has become the precondition for computer science.

The reason why is simple: At first, since the 1980s the education in the school and university has changed a lot. Today, it's normal that all students, even if they don't like computers very much, are forced to learn a programming language. That means, as default 100% of the students have programming experience. Mostly in Java, but sometimes with other languages too. Secondly, the amount of documentation and programming languages have grown. Youtube offers thousands of introductionary courses, so that even if somebody never attended a computer course in the school he will learn to program by the way. And last but not least, the OpenSource revolution has reduced the entry cost for a programmer downto zero. The lastest LLVM compiler and together with the Python environment is available for free download in the internet. In the sum, there are no longer obstacles available which prevent somebody to learn programming. The result is that the discipline has become democratic. That means, everybody is able to write a moderate complex program.

It is a bit funny to read introduction chapters in programming books. They are trying to motivate the people to learn a computer language under the assumption that the knowledge about how to use the C++ compiler is a secret. This was true in the early 1990s because in that time, the C++ costs thousands of US$ and only a few computer users had access to that technology. The situation today has changed. Or to explain it from a different point of view. A technology can only be invented once. Learning programming in the early 1990s was an advanture and was recommended strongly, today it has become the normal case. We have to search very intensive to find a student who is not familiar with programming at all. I would guess on a normal university all students are able to program because it's part of the normal curriculum similar to speak English.

To answer the question from the beginning. The question is not if C, Python or Java is the better programming language, but the idea is to ignore programming at all and search for something which is new and exciting. The best way in doing so is to search for an asymmetric knowledge distribution. In the early 1990s the typical situation was, that less then 5% of all computer users were able to program. In that time, programmers were able to do something which wasn't available for normal computer users. It make sense to search for this knowledge gap again.

The reason why programming was perceived as a powerful technique was because it utilized a computer in a certain way. A normal computer user can start only an existing program. Under Windows 95 he doubleclicks on an icon and the computer is doing something. In contrast, the programmer has more power. He defines, what the software is doing. He creates the user interface and writes all the messages on the screen. That means, the programmer has more rights in relationship to a computer. He knows a technique to use the computer in an elaborated way.

It's not very complicated to identify the follow up discipline after programming. We have to ask for some kind of technique which allows to use a computer in a modern way. Are there are wizards or hackers available who have more rights than normal programmers? Or to ask the other way around? Which kind of computer usage is outside the scope of a normal Java programmer? What behavior of a computer is not available for the expert C++ programmer? On the first look a programmer is equal to the master of the universe. He is able to force the computer to do anything. The programmer imagined a new kind of software, and after some keystrokes the program runs on the screen. This is equal to a great power. If a non programmer is trying to do the same he will fail. Even he imagine a new kind of game or a different application he has not enough wisdom to convert his imagination into reality. He needs somebody else who can program the project for him.

Let me give an example. A non programmer has a demand for a database searching algorithm. He needs a gui interface which takes a userinput and searches in the SQL database. The non programmer has no plan who to realize such a project. He didn't even know how java can help to realize such goal. In contrast, the programmer knows exactly how to create the needed application. He has written a Database search engine in the past, or he knows how to create such application with less effort. This is the reason why programming is a useful discipline because it extends the ability to solve problems.

The situation in the now is, that the described case of a non programmer who isn't aware of the Java language is no longer available in reality. It is a look back into the history of computing in which the distinction between programmers and normal users were bigger. In the now, the gap has reduced to a minimum. That means, in reality we wouldn't find a non programmer who doesn't know how to search in the database.

The reason why has to do with a widespread usage of computer technology in all areas. Desktop computers, smartphones, programming languages and courses which are teaching computer programming are widespread available. This is different from the 1990s in which computers were not available for a wider audience. This has changed the importance of programming too. The ability to create new software has become the normal case.

Performance of Boost Python vs RESTful

In the mean time, I've done some experiments with RESTful interface and also with boost Python. The idea was to figure out, how to combine different programming languages. The RESTful interface is similar to the early SOAP protocol independent from a certain programming language. It combines C++, Java, Python, C# and any other language framework over the network interface of the operating system. IF both applications are installed on the same computer, it's the loop back interface which provides the maximum amount of performance.

The simplest way in connecting a python program to a RESTful interface is with the help of the FLASK library. Recently, there are some newer developments available which are trying to maximize the performance. In my own experiments, the maximum speed of RESTful on a local machine was 379 requests per seconds, https://trollheaven.blogspot.com/2019/09/creating-c-library-with-boost-python.html Which is enough for a game engine which is working only with 30 frames per seconds.

A potential alternative to RESTful are language wrappers, which can include c libraries into a scripting language. Boost Python is one of the famous example which can connect Python with existing C++ code. The major drawback over RESTful is, that every programming language has a different type of interface. For example, the procedure for including a c library into a Java program works completely different from Boost Python. The advantage is that the performance is much better. A hello world example with the Boost Python frameworks has produced a performance of 76923 requests per seconds, https://trollheaven.blogspot.com/2019/09/creating-c-library-with-boost-python.html which is 200x times faster than the RESTful approach.

What we can say in general, that's possible to connect different programming languages into a single project. It make sense to write some code fragments in C++ while using Python as a scripting language on top. If three and more languages should be combined in a project the usage of normal language wrappers like Boost Python can become difficult, The elegant alternative is RESTful.

A second conclusion from the experiment was, that the C/C++ language is a quasi standard in modern computing. That means, in case of doubt, it make sense to write the critical component of a project in C/C++ because all the other languages can handle this format. All the modern languages like Java, C#, Python or not more than an extension to the C/C++ standard.

C/C++

To find the reason why we have to go back into the history of computing. In the year 1969 the B programming language was invented, https://en.wikipedia.org/wiki/B_(programming_language) It has the same syntax like a modern C program. The B language contains of functions like “showscreen()”, it can handle variables, pointers and it has for loops. Since the year 1969 the C language was only extended by new functions but the concept itself remains the same. The sourcecode is compiled into fast assembly instructions and this allows to write larger software projects.

There are many paradigm available which are different from C/C++. The first one is called just-in-time compiler which was heavily used in the Java / C# environment. Another idea is to use functional programming over procedural programming. But none of these concepts have replaced the classical C/C++ paradigm. That means, if somebody tries to program an efficient library he will choose for 99% of the cases C/C++.

The reason why C is so popular has to do with pointers. On the newbie pointers seems to be obsolete. And modern languages like Python or C# don't need them. The problem is, that it's not possible to remove them from the C standard because pointers are used internally by the computer.The exact reason is located on the assembly language level. In assembly language, pointers are everywhere. The register of a cpu can only hold the address of an array but not the datastructure itself. That means, if the CPU likes to traverse through an array he will load the pointer to memory address into it's register. And if the assembly language cares about pointers, C/C++ has to do the same.

That means, C/C++ can be replaced by a different language in the future, if it's possible to remove pointers from the assembly language specification. Because this is hard or not possible, C/C++ is the quasi standard in modern programming and all the other languages including C# are less efficient.

More comparisons

RESTful 379 requests per seconds

Boost.Python 76923 requests per seconds

cython 123457 requests per seconds

It's important to know, that the cython compiler (a python to c translator) works also with the C/C++ programming language in the middle. It's performance is slightly better than the Boost-Python interface. The major problem what cython, swig, Boost python and ctypes have all in common is, that the concept can not be used in other programming languages. It's focused only on Python language bindings.

Other programming languages like Java have a different wrapper to include existing C libraries for reasons of speed performance. From a general perspective, it seems that a c library is the fastest option to run a program. Existing C compilers are used to create this libraries. This directs the investigation to an important question: Is there a need for libraries not written in C?

Let us go back and discuss why the Python language was invented. Python code can be created much easier than C code. It doesn't need filetypes like integer, it supports object oriented programming out of the box and it doesn't need a compiler but can run newly created code in the interpreted mode. This make Python a great choice for creating prototypes. The major disadvantage of Python is, that the language is slow. Another problem is, that it's not possible to program on a machine level with direct hardware access.

The combination of C and Python is an ideal programming environment. C allows to write fast libraries and operating system routines, while Python can utilize the libraries for testing out new projects. Let us ask first what the alternative over C is. There is no alternative. All the other languages like C#, Objective c or Java doesn't support pointers or they are slower than C. That means, for low level programming, a c compiler is a here to stay.

Now we have to ask which kind of programming can replace Python. Sometimes, the Java language is mentioned, because Java runs in a virtual environment. The disadvantage of Java is, that it's harder to learn than Python. There is a need to declare variables first.

Is Programming an academic discipline anymore?

Modern programming languages are working much better than their counterparts from the past. A state of the art python3 interpreter runs well under all major operating systems (Linux, Windows, MacOS), supports object oriented programming, and is described in the documentation very well so that newbies can start programming after a short time. Sometimes it was mentioned, that Python is a slow programming language, but this is only a minor problem. Because with the help of cython the speed can increased upto the performance of a c program and who doesn't like the concept of Python at all he can switch to the C# language which is also well documented and allows to program complex applications.

The problem with Python3 is located somewhere else and is the result of the success of the framework. A modern python interpreter works mostly great, is well documented and allows to create software very efficient. Python3 is doing the job much better than previous attempts in programming like the Turbo pascal environment which was popular in the 1980s or the Fortran compiler which was used in the time before. The most interesting fact is, that the average user is not able to explore the limits of Python. He won't find a serious issue which has no answer yet. The Python interpreter, and most modern programming languages in general are working like a well designed pencil in the hand of an artist. The limits are not located within the programming language itself, but in the libraries which are available and the imagination which kind of programs are making sense and which not.

The python language can be recommended as a here to stay technology, and at the same time it's a disappointed experience to take a look into the language. All the books and tutorials which are explaining Python doing a great job, but they are not located within academia. Learning Python is nothing which is teached at the university, but it's some kind of leisure activity similar to learn how to listen to music. This was not always the case. In the past, programming was a serious subject for computer scientists. They have discussed the problems of programming a computer on an academic level. In Google Scholar there are many papers available in the 1980s, in which compiler design and programming was explained. Later, the academic papers have adapted their focus a bit. Instead of explaining how to write hello world programs, the new topic in academia was, which kind of programming idea can be teached at the university. That means, the published papers in the 1990s were not about how to write a hello world program in C++ or Perl, but a typical had the title “how to teach object oriented programming in a computer course”.

THe shift in the topic is equal to lowering the quality. Because teaching computer programming has nothing to do with computer science in it's core sense. It is located within academia, but in the domain of Pedagogy. Some years later, in the year 2000 and later, the next step was visible. Because the topic of programming in general was no longer discussed in the literature. Or at least, not in academic literature. It was delegated to schools and companies who have made a business of it. The current textbooks about programming are published in school publishing companies, next to math education books. This kind of books is interesting to read, but is located outside of academic disciplines.

Programming is over

The development makes sense, because programming has become a mainstream issue. With the advent of modern frameworks like Python and C#, the art of programming has become easier than ever. And there is no need to cover the subject in a university class. Creating a hello world program in Python which prints out a line to the screen and draws a rectangle to the screen is to simple to teach the topic in a computer science class. The problem is, that by defintion in a computer science class the average student is 20 years old, has learned in his education career lots of interesting things already and likes to learn something new and which is more complicated. That means, it's not possible to teach the student how to program in Python.

This kind of shift in the curriculum is an interesting development, because in the past, it was common to do so. A computer course in the 1980s has teached how to program a hello world program with C or Fortran. Because in that time, programming was an esoteric discipline which was covered only at the university but nowhere else. It seems, that in the last 40 years something has changed. The topics which are teached at a computer science course at the university are different than in the past. The students are different and their demand for knowledge is higher.

Perhaps it make sense to ask the student what they want to learn in a university. Does the average student has a demand to learn how to write a hello world program with Python or C? Or are the needs different and are going beyond this kind of knowledge?`

Is Microsoft dotnet the future?

The C# language and the Java language are both potential alternatives to the well known C++ standard. What makes them unique is, that the code is executed in a virtual machine. The advantage is, that for the programmer it's easier to create the sourcecode, because the VM interprets the program in realtime and for the compiler writers, it's easier to create the VM over a dedicated compiler like the G++ compiler.

In case of the Java ecosystem the problem is, that after a while the language has become unrealiable. The recent updates are given under a different license, and it's unclear who exactly develops Java further. The situation in C# and dotnot is the oppposite. Dotnet is for the Windows ecosystem a quasi standard and every programmer is forced to use the platform.

The interesting point is, that for the Linux platform, no such development like dotnet was invented. Instead, the gnome desktop is trying to standardize the programming API a bit, but in general not very successful. The GTK+ library was written in C, but it's also possible to use the C++ language to get access to gnome. A unique programming interface comparable to C# is missing for Linux.

On the other hand, Microsoft is motivated to port the dortnet software to the Linux operating system as well. Under the term “Mono project” it was made possible to run C# code under Linux, https://www.mono-project.com/docs/advanced/runtime/ Does this make sense? It's unclear right now. On the first look, it's a bad idea to combine C# with an opensource operating system. On the other hand, Linux has a demand for a dotnet like architecture. Is dotnet something which will finds it way from Microsoft to Linux in the future?

Let us first imagine the other way around. What can Windows programmers learn from Linux users in developing object oriented software? The surprising answer is, that they can't learn anything, because programming under Windows is more comfortable than under Linux. In contrast to the Gnome ecosystem, under Windows the programmer gets a standardized ecosystem which has only one way in painting windows. And he is not forced to learn outdated c libraries or handle with C++ pointers. The main reason why C# was invented during the year 2000 was to avoid the chaotic situation in which the programmer is forced to program with C++. That means, Windows has made the next logical step to an interpreted environment, while under Linux the situation is chaotic.

Programming under a Linux operating system with the C compiler is only comfortable if the idea is to create console applications. In this use case, Windows has no advantage over Linux. In both cases, the programmer types in C like code syntax into the editor and after pressing the compile button, he gets the working application. Only in the case of creating desktop applications, Linux provides no help for the programmer. That's the reason why the amount of GUI programs for Linux is low and the market share of the Linux desktop too.

On the other hand, Microsoft has solved the Desktop programming issue very well. Their dotnet strategy is a here to stay, and gets developed further in the future. From a technical point of view it make sense to ask if the Linux community can learn from Microsoft how to program software in C#.

Before it's possible to judge about C# and the mono project we have to ask why the C# language was introduced under the Windows operating system. The goal is twofold. First, there was a needed for object-oriented programming in a virtual machine. The advantage over C++ is, that programming is much easier because the programmer can focus on the application but not on the internals of a computer. C# is comparable to the Python language, but can be used for creating productive code as well. The second reason why dotnet was introduced was, to provide a GUI API for creating desktop applications. The programmer gets predefined routines for drawing windows and programming games.

Now we can investigate if Linux programmer have the same needs. The first question is, if a Linux user needs to program an application with an object oriented language which is not C++, and the second question is if he needs a standardized GUI API. In both cases the answer is yes. Because the Linux desktop is similar to the Windows desktop a graphical environment and object oriented programming is the natural way in creating software. The alternative over the Mono project would be to use Python (which is technically a scripting language, but not a virtual machine) or to use the Java language (which had become some problems in the recent years).

A possible alternative over using mono and dotnet is, if the Linux community would develop it's own managed GUI API. It would be combination of Gnome, C++ and Wayland. The prediction is, that this kind of technology will not be realized. Because the market is too small. Red Hat would be the only vendor who is able to invent a new API on top of the Gnome desktop and combine this with a programming language. It's unclear, if they are plan to do so.

The reason for the missing Linux desktop environment is, that the user base who is using the Linux desktop is smaller than 1 million worldwide, while in contrast around 3 billion people worldwide are familiar with the Windows desktop. That means, GUI programming under a desktop operating system is equal to write C# code. And Linux is not able to convince the world, that they doesn't need desktop applications. The main problem with the Linux os is, that the focus of interest is located on the command line. The perfect Linux computer is a headless one, which runs the apache Webserver plus kernel drivers for accessing the network card. The desktop environment was put on top of the X11 display server and the amount of effort into this domain is small.

Even the largest Linux GUI framework, called Gnome, has a very small programming community. The amount of documentation is little and the existing API doesn't make much sense. Creating a GUI Application under Linux is some kind of adventure, which means that it's an unexplored journey. The problem is, that there are around 300 different GUI frameworks which can be used by lots of programming languages like TCL, C and python but there is no best practice method in writing a mainstream Linux application.

The raise and fall of computer programming

On the first look, the domain of computer programming is booming. Lots of new language were developed like Golang and C#, online forums like Stackoverflow are providing online based ressources, and thanks to the development of opensource, everybody gets the latest compilers at the lowest possible costs. Compared to the situation in the 1980s, it's easier than ever to learn programming and write working code. At the same time, there is something wrong with most programming languages and in the following blog post I'd like to explain the reason why.

Instead of describing computer programming by it's self-description it make sense to go a step backward and analyze the domain from an academic perspective on a timescale. The need for programming the newly invented computer machinery was obvious since the early 1950s. The late 1970s the object oriented paradigm was invented to improve the code efficiency. In the 1990s programming was transformed into a business and the famous Microsoft company has made a business model around it. From an academic perspective, programming is a solved problem. That means, it was researched in the past and doesn't provide open questions.

Before the advent of the Small talk programming language it was an interesting problem how to create larger programs which are working interactively. The problem is, that OOP can only be invented once. The smalltalk specification is available since the 1980s and it was the answer to the mentioned problem. But what comes after smalltalk, Just-in-time-compilers and opensource operating systems? Right, there is nothing on the horizon. Programming was the answer to the problem of how to use the computer in a certain way. It's not enough to put electric current into the CPU, but before the computer can do useful things he needs a program. If the computer is equipped with software, and if the software was designed well, then the computer can do anything.

The problem with the current mainstream programming languages like C++, Java, Python and Golang is, that they have no problem, but they are the answer to the issue of computer program. If a computer programming problem is there, C++ and all the other languages are the answer. The open question is, if programming in general is something which has an academic impact. In the 1970's and the 1980s the answer was yes. In that time, it was not clear what a compiler is, or what object oriented software development is. There was a need to research the topic in depth and put a lot of manpower into programming research. Unfortunately, the situation has changed. That means, the wheel was invented, the programmers are happy and now it's time to cancel these kind of projects to make room for something new.

What does it mean? It means, that computer programming hast lost his academic impact. It is not the leading discpiine in computer science but something which is located in a museum. Programming in general is not a future technology, or a here-to-stay, but what can be described by historians. For example it make sense to publish a book about the advent of the Turbo Pascal language in the 1980s. Such books doesn't have an impact to the computer science, but they are located within the social sciences. It's a look back into the past, similar to describe the emerging technology of the first steam engine in the 19th century.

It's important to frame existing attempt in computer programming with the correct academic background. Online forums like Stackoverflow, modern compilers like the LLVM suite or software repositories at github are not examples for computer science, but they are part of the retrocommunity who are imitating the past. Stackoverflow gives an outlook what computer science was 20 years ago. That means, the forum doesn't provide current problems of computer science, but pseudo problems which are part of computer history.

To analyze the situation right we have to imagine a situation in which a museum with old radios from the 1950s has the self-understanding to promote future technology. It looks very funny if an outdated technology is not presented in their past setting, but as a current technology. The same situation is obviously in computer programming. The self-understanding of most programmers is, that they are leading the computer revolution. Does this description make sense? Has the future something to do with converting C sourcecode into a binary file? Is computer science on an academic level the same like programming something in the C# dialect?

If we are going back the past the answer is yes. Using a C# like programming in the 1980s can be called an advanced example of computer science. In that period it was state of the state and a bit ahead of the time. The problem is, that the time has changed. The 1980s are gone since a while and they will never come back. It's important to declare all programming paradigm as obsolete and try to build a museum around them.

Somebody would argue, that programming is needed in the future more than ever and that it's not possible to program a computer without write source code. This is true and false at the same time. It's correct, that in the future the people will eat something, and they will use pen and paper. And for sure, they will use the C++ language as well. But his is not equal that these things are invented in the future, but they are tools from the past. They have transformed from an innovation into a household tool. They are normal, everywhere and standardized.

Programming languages are game engines too

Compilers are often described by it's inner structure. For example, they are containing of a parser and a lexer. What is missing in this description is what the purpose of a compiler is. The more elaborated way in thinking about programming languages is to imagine, how they utilized as a man machine interface. The programmer sends the sourcecode to the compiler and the compiler converts it into machine code.

An interpreter which can execute a programming language like Perl has much in common with a game engine. Similar to a game engine, the machine provides to the programmer an API interface. That's a list of actions who are allowed to enter. A sense making dialogue with a game engine would work in a way, that the programmers executes “move pacman to the left”. In response to this request, the game engine modifies the position of the character on the screen and adapts the graphics subroutines as well. The same is true for an interpreter who runs computer code. The users enters an action like “for (int i=0,i<5,i++)” and the interpreter is answering this request by executing a loop which contains of 5 single steps.

The interesting fact is, that for the normal user the inner working of a game engine or a compiler are not relevant. They only know which actions are allowed and use the machine in a pipeline. On the other hand, for the programmer of a game engine, it's not relevant who a game is played by the human. The only thing what a pacman game engine has to ensure is, that the program can handle certain situations and detect all the collisions with the map.

Fro reason of simplification it make sense to describe an engine as a rulebook in which certain actions are possible. The engine defines which commands can be entered by the outside and it defines how fast these actions are executed. The game engine is an intermediate which improves the communication. The engine itself is doing nothing, but it has to adapt to the needs of the environment.

Creating a C++ library with Boost Python

A short tutorial in the internet https://www.mantidproject.org/Boost_Python_Introduction have shown, how to combine C++ with Python. The C++ compiler is used to create a C++ library, and the Boost Python add on is used, to include this library in external Python programs.

Unfortunately, the recommended command line for starting the g++ compiler doesn't work in Fedora Linux. The correct parameter is:

g++ -Wall -Wextra -fPIC -shared -I/usr/include/python3.7m/ -lboost_python37 mantid.cpp -o mantid.so

The reason is, that the name of boost_python library is called slightly different in the current Fedora OS. All the library can be found in the directory /usr/lib64 The other options for the g++ compiler like the “-shared” switch are needed to create a “.so” library but not a standalone program.

Now we can go over to the funny part and test the newly created C++ library from within the python interpreter. We are creating a simple hello world app and measure the time.

import mantid
for i in range(1000000):
  print(i)
  mantid.sayHello()

"""
time python3 hello.py
real 0m13.070s
user 0m8.649s
sys 0m4.289s
"""

The program was able to execute 76923 requests per seconds. Only to get a comparison, a Python program which gets access to a C++ program over a RESTful interface will with a speed of 379 requests per seconds, https://trollheaven.blogspot.com/2019/08/testing-performance-of-restful-api-with.html That means, the boost Python framework is much faster then the RESTful interface.

Connecting programming languages with sockets

How exactly gets Python programmer access to existing C++ libraries? On the first look the question is easy to answer because Python can import C libraries easily. A more detailed look into the problem will show, that the existing wrapper and interface generators like SWIG and ctypes are not enough. And even in the case of success, they only allow to Python programmers to get access to a c library, but they do not provide a universal language independent format.

Every new programming language which was invented has the same problem. Inside the ecosystem all the features are working great, the only problem is to convince other programmers to use the same language. What programming languages are not designed for is to communicate with the outside world. Even Python is not prepared to include existing c libraries or Java programs.

A more recent technique for a communication standard is not working on the sourcecode level and is not part of a language compiler, but has to do with TCP/IP connection. An early attempt in connecting different programming languages was SOAP, a more recent development is RESTful and sockets.

Let us take a look into the gaming library SFML what they know about sockets, https://www.sfml-dev.org/tutorials/2.5/network-socket.php According to the documentation a socket is network connection which allows different applications to send and receive data. The interesting point is, that first the socket works with different languages and secondly, the code is running while he communicates. The SFML library was written in C++ but if a SFML application has opened up a socket, other languages like Java, Python and Javascript can get access to the connection.

In contrast to a SWIG like language interface which is working on a lower level, TCP/IP sockets have a poor performance. According to some tests, it's possible to get around 300 requests per seconds on the localhost interface. With some newer techniques like non-blocking RESTful interfaces the traffic will increase a bit.

The advantage of sockets over import a c library is, that the amount of needed explanation are shorter. Suppose, programmer A has written a game engine which allows other programmer to write plugins. In the past, programmer A has to document his sourcecode, because the plugin interface is working on the language level. For example, the game engine was written in C++, so the other programmer need the API and the documented C++ sourcecode. This allows them to write a plugin which can utilize the existing game engine.

The disadvantage is, that the number of programmers how are interesting in analyzing existing sourcecode, especially if it was written in C++ is low. The problem is not located in the game engine itself, because C++ is the perfect choice for creating such a project. It is ultrafast and most game engines are written in C++. The problem is, that it's very complicated to analyze an existing project. The more easier way is, if programmer B can use his language of choice. For example he is trying out the C# language, because he likes the syntax very much. And this opens the question how to connect a C# frontend with a C++ backend?

The sad news is, that C# doesn't provide access to existing C++ code. Sure, Stackoverflow is answering the problem with the reference to the COM interface https://stackoverflow.com/questions/3029031/connecting-c-to-c-sharp and the DllImport library, https://stackoverflow.com/questions/2958416/call-c-library-from-c-sharp But none of these techniques are making sense. And what's more important, they fail if the languages are slightly different.

On the first look this sounds paradox, because C++ and C# are both mainstream languages which a huge userbase and lots of energy which was invested in the improvement of the compilers. So the naive impression is, that there is a technique to connect a normal C# frontend with a C++ backend. But it seems, that the problem is too complicated even for the C# community. A closer look into modern programming languages, for example Java, Python, C and Javascript will show, that no one is mastering the issue. All of the languages are struggling if the user tries to connect different languages.