September 06, 2019

Performance of Boost Python vs RESTful


In the mean time, I've done some experiments with RESTful interface and also with boost Python. The idea was to figure out, how to combine different programming languages. The RESTful interface is similar to the early SOAP protocol independent from a certain programming language. It combines C++, Java, Python, C# and any other language framework over the network interface of the operating system. IF both applications are installed on the same computer, it's the loop back interface which provides the maximum amount of performance.
The simplest way in connecting a python program to a RESTful interface is with the help of the FLASK library. Recently, there are some newer developments available which are trying to maximize the performance. In my own experiments, the maximum speed of RESTful on a local machine was 379 requests per seconds, https://trollheaven.blogspot.com/2019/09/creating-c-library-with-boost-python.html Which is enough for a game engine which is working only with 30 frames per seconds.
A potential alternative to RESTful are language wrappers, which can include c libraries into a scripting language. Boost Python is one of the famous example which can connect Python with existing C++ code. The major drawback over RESTful is, that every programming language has a different type of interface. For example, the procedure for including a c library into a Java program works completely different from Boost Python. The advantage is that the performance is much better. A hello world example with the Boost Python frameworks has produced a performance of 76923 requests per seconds, https://trollheaven.blogspot.com/2019/09/creating-c-library-with-boost-python.html which is 200x times faster than the RESTful approach.
What we can say in general, that's possible to connect different programming languages into a single project. It make sense to write some code fragments in C++ while using Python as a scripting language on top. If three and more languages should be combined in a project the usage of normal language wrappers like Boost Python can become difficult, The elegant alternative is RESTful.
A second conclusion from the experiment was, that the C/C++ language is a quasi standard in modern computing. That means, in case of doubt, it make sense to write the critical component of a project in C/C++ because all the other languages can handle this format. All the modern languages like Java, C#, Python or not more than an extension to the C/C++ standard.
C/C++
To find the reason why we have to go back into the history of computing. In the year 1969 the B programming language was invented, https://en.wikipedia.org/wiki/B_(programming_language) It has the same syntax like a modern C program. The B language contains of functions like “showscreen()”, it can handle variables, pointers and it has for loops. Since the year 1969 the C language was only extended by new functions but the concept itself remains the same. The sourcecode is compiled into fast assembly instructions and this allows to write larger software projects.
There are many paradigm available which are different from C/C++. The first one is called just-in-time compiler which was heavily used in the Java / C# environment. Another idea is to use functional programming over procedural programming. But none of these concepts have replaced the classical C/C++ paradigm. That means, if somebody tries to program an efficient library he will choose for 99% of the cases C/C++.
The reason why C is so popular has to do with pointers. On the newbie pointers seems to be obsolete. And modern languages like Python or C# don't need them. The problem is, that it's not possible to remove them from the C standard because pointers are used internally by the computer.The exact reason is located on the assembly language level. In assembly language, pointers are everywhere. The register of a cpu can only hold the address of an array but not the datastructure itself. That means, if the CPU likes to traverse through an array he will load the pointer to memory address into it's register. And if the assembly language cares about pointers, C/C++ has to do the same.
That means, C/C++ can be replaced by a different language in the future, if it's possible to remove pointers from the assembly language specification. Because this is hard or not possible, C/C++ is the quasi standard in modern programming and all the other languages including C# are less efficient.
More comparisons
RESTful 379 requests per seconds
Boost.Python 76923 requests per seconds
cython 123457 requests per seconds
It's important to know, that the cython compiler (a python to c translator) works also with the C/C++ programming language in the middle. It's performance is slightly better than the Boost-Python interface. The major problem what cython, swig, Boost python and ctypes have all in common is, that the concept can not be used in other programming languages. It's focused only on Python language bindings.
Other programming languages like Java have a different wrapper to include existing C libraries for reasons of speed performance. From a general perspective, it seems that a c library is the fastest option to run a program. Existing C compilers are used to create this libraries. This directs the investigation to an important question: Is there a need for libraries not written in C?
Let us go back and discuss why the Python language was invented. Python code can be created much easier than C code. It doesn't need filetypes like integer, it supports object oriented programming out of the box and it doesn't need a compiler but can run newly created code in the interpreted mode. This make Python a great choice for creating prototypes. The major disadvantage of Python is, that the language is slow. Another problem is, that it's not possible to program on a machine level with direct hardware access.
The combination of C and Python is an ideal programming environment. C allows to write fast libraries and operating system routines, while Python can utilize the libraries for testing out new projects. Let us ask first what the alternative over C is. There is no alternative. All the other languages like C#, Objective c or Java doesn't support pointers or they are slower than C. That means, for low level programming, a c compiler is a here to stay.
Now we have to ask which kind of programming can replace Python. Sometimes, the Java language is mentioned, because Java runs in a virtual environment. The disadvantage of Java is, that it's harder to learn than Python. There is a need to declare variables first.