Compilers are often described by it's inner structure. For example, they are containing of a parser and a lexer. What is missing in this description is what the purpose of a compiler is. The more elaborated way in thinking about programming languages is to imagine, how they utilized as a man machine interface. The programmer sends the sourcecode to the compiler and the compiler converts it into machine code.
An interpreter which can execute a programming language like Perl has much in common with a game engine. Similar to a game engine, the machine provides to the programmer an API interface. That's a list of actions who are allowed to enter. A sense making dialogue with a game engine would work in a way, that the programmers executes “move pacman to the left”. In response to this request, the game engine modifies the position of the character on the screen and adapts the graphics subroutines as well. The same is true for an interpreter who runs computer code. The users enters an action like “for (int i=0,i<5,i++)” and the interpreter is answering this request by executing a loop which contains of 5 single steps.
The interesting fact is, that for the normal user the inner working of a game engine or a compiler are not relevant. They only know which actions are allowed and use the machine in a pipeline. On the other hand, for the programmer of a game engine, it's not relevant who a game is played by the human. The only thing what a pacman game engine has to ensure is, that the program can handle certain situations and detect all the collisions with the map.
Fro reason of simplification it make sense to describe an engine as a rulebook in which certain actions are possible. The engine defines which commands can be entered by the outside and it defines how fast these actions are executed. The game engine is an intermediate which improves the communication. The engine itself is doing nothing, but it has to adapt to the needs of the environment.
September 03, 2019
Creating a C++ library with Boost Python
A short tutorial in the internet https://www.mantidproject.org/Boost_Python_Introduction have shown, how to combine C++ with Python. The C++ compiler is used to create a C++ library, and the Boost Python add on is used, to include this library in external Python programs.
Unfortunately, the recommended command line for starting the g++ compiler doesn't work in Fedora Linux. The correct parameter is:
The reason is, that the name of boost_python library is called slightly different in the current Fedora OS. All the library can be found in the directory /usr/lib64 The other options for the g++ compiler like the “-shared” switch are needed to create a “.so” library but not a standalone program.
Now we can go over to the funny part and test the newly created C++ library from within the python interpreter. We are creating a simple hello world app and measure the time.
The program was able to execute 76923 requests per seconds. Only to get a comparison, a Python program which gets access to a C++ program over a RESTful interface will with a speed of 379 requests per seconds, https://trollheaven.blogspot.com/2019/08/testing-performance-of-restful-api-with.html That means, the boost Python framework is much faster then the RESTful interface.
Unfortunately, the recommended command line for starting the g++ compiler doesn't work in Fedora Linux. The correct parameter is:
g++ -Wall -Wextra -fPIC -shared -I/usr/include/python3.7m/ -lboost_python37 mantid.cpp -o mantid.so
The reason is, that the name of boost_python library is called slightly different in the current Fedora OS. All the library can be found in the directory /usr/lib64 The other options for the g++ compiler like the “-shared” switch are needed to create a “.so” library but not a standalone program.
Now we can go over to the funny part and test the newly created C++ library from within the python interpreter. We are creating a simple hello world app and measure the time.
import mantid for i in range(1000000): print(i) mantid.sayHello() """ time python3 hello.py real 0m13.070s user 0m8.649s sys 0m4.289s """
The program was able to execute 76923 requests per seconds. Only to get a comparison, a Python program which gets access to a C++ program over a RESTful interface will with a speed of 379 requests per seconds, https://trollheaven.blogspot.com/2019/08/testing-performance-of-restful-api-with.html That means, the boost Python framework is much faster then the RESTful interface.
September 02, 2019
Connecting programming languages with sockets
How exactly gets Python programmer access to existing C++ libraries? On the first look the question is easy to answer because Python can import C libraries easily. A more detailed look into the problem will show, that the existing wrapper and interface generators like SWIG and ctypes are not enough. And even in the case of success, they only allow to Python programmers to get access to a c library, but they do not provide a universal language independent format.
Every new programming language which was invented has the same problem. Inside the ecosystem all the features are working great, the only problem is to convince other programmers to use the same language. What programming languages are not designed for is to communicate with the outside world. Even Python is not prepared to include existing c libraries or Java programs.
A more recent technique for a communication standard is not working on the sourcecode level and is not part of a language compiler, but has to do with TCP/IP connection. An early attempt in connecting different programming languages was SOAP, a more recent development is RESTful and sockets.
Let us take a look into the gaming library SFML what they know about sockets, https://www.sfml-dev.org/tutorials/2.5/network-socket.php According to the documentation a socket is network connection which allows different applications to send and receive data. The interesting point is, that first the socket works with different languages and secondly, the code is running while he communicates. The SFML library was written in C++ but if a SFML application has opened up a socket, other languages like Java, Python and Javascript can get access to the connection.
In contrast to a SWIG like language interface which is working on a lower level, TCP/IP sockets have a poor performance. According to some tests, it's possible to get around 300 requests per seconds on the localhost interface. With some newer techniques like non-blocking RESTful interfaces the traffic will increase a bit.
The advantage of sockets over import a c library is, that the amount of needed explanation are shorter. Suppose, programmer A has written a game engine which allows other programmer to write plugins. In the past, programmer A has to document his sourcecode, because the plugin interface is working on the language level. For example, the game engine was written in C++, so the other programmer need the API and the documented C++ sourcecode. This allows them to write a plugin which can utilize the existing game engine.
The disadvantage is, that the number of programmers how are interesting in analyzing existing sourcecode, especially if it was written in C++ is low. The problem is not located in the game engine itself, because C++ is the perfect choice for creating such a project. It is ultrafast and most game engines are written in C++. The problem is, that it's very complicated to analyze an existing project. The more easier way is, if programmer B can use his language of choice. For example he is trying out the C# language, because he likes the syntax very much. And this opens the question how to connect a C# frontend with a C++ backend?
The sad news is, that C# doesn't provide access to existing C++ code. Sure, Stackoverflow is answering the problem with the reference to the COM interface https://stackoverflow.com/questions/3029031/connecting-c-to-c-sharp and the DllImport library, https://stackoverflow.com/questions/2958416/call-c-library-from-c-sharp But none of these techniques are making sense. And what's more important, they fail if the languages are slightly different.
On the first look this sounds paradox, because C++ and C# are both mainstream languages which a huge userbase and lots of energy which was invested in the improvement of the compilers. So the naive impression is, that there is a technique to connect a normal C# frontend with a C++ backend. But it seems, that the problem is too complicated even for the C# community. A closer look into modern programming languages, for example Java, Python, C and Javascript will show, that no one is mastering the issue. All of the languages are struggling if the user tries to connect different languages.
Every new programming language which was invented has the same problem. Inside the ecosystem all the features are working great, the only problem is to convince other programmers to use the same language. What programming languages are not designed for is to communicate with the outside world. Even Python is not prepared to include existing c libraries or Java programs.
A more recent technique for a communication standard is not working on the sourcecode level and is not part of a language compiler, but has to do with TCP/IP connection. An early attempt in connecting different programming languages was SOAP, a more recent development is RESTful and sockets.
Let us take a look into the gaming library SFML what they know about sockets, https://www.sfml-dev.org/tutorials/2.5/network-socket.php According to the documentation a socket is network connection which allows different applications to send and receive data. The interesting point is, that first the socket works with different languages and secondly, the code is running while he communicates. The SFML library was written in C++ but if a SFML application has opened up a socket, other languages like Java, Python and Javascript can get access to the connection.
In contrast to a SWIG like language interface which is working on a lower level, TCP/IP sockets have a poor performance. According to some tests, it's possible to get around 300 requests per seconds on the localhost interface. With some newer techniques like non-blocking RESTful interfaces the traffic will increase a bit.
The advantage of sockets over import a c library is, that the amount of needed explanation are shorter. Suppose, programmer A has written a game engine which allows other programmer to write plugins. In the past, programmer A has to document his sourcecode, because the plugin interface is working on the language level. For example, the game engine was written in C++, so the other programmer need the API and the documented C++ sourcecode. This allows them to write a plugin which can utilize the existing game engine.
The disadvantage is, that the number of programmers how are interesting in analyzing existing sourcecode, especially if it was written in C++ is low. The problem is not located in the game engine itself, because C++ is the perfect choice for creating such a project. It is ultrafast and most game engines are written in C++. The problem is, that it's very complicated to analyze an existing project. The more easier way is, if programmer B can use his language of choice. For example he is trying out the C# language, because he likes the syntax very much. And this opens the question how to connect a C# frontend with a C++ backend?
The sad news is, that C# doesn't provide access to existing C++ code. Sure, Stackoverflow is answering the problem with the reference to the COM interface https://stackoverflow.com/questions/3029031/connecting-c-to-c-sharp and the DllImport library, https://stackoverflow.com/questions/2958416/call-c-library-from-c-sharp But none of these techniques are making sense. And what's more important, they fail if the languages are slightly different.
On the first look this sounds paradox, because C++ and C# are both mainstream languages which a huge userbase and lots of energy which was invested in the improvement of the compilers. So the naive impression is, that there is a technique to connect a normal C# frontend with a C++ backend. But it seems, that the problem is too complicated even for the C# community. A closer look into modern programming languages, for example Java, Python, C and Javascript will show, that no one is mastering the issue. All of the languages are struggling if the user tries to connect different languages.
C++ is performing poor in a starfield simulation
The naive assumption was, that the C++ programming language together with the SFML library performs very well of ressource intensive tasks. So I've created a simple multi-agent simulation which contains of some circles on the screen who are moving towards a goal. The program follows the normal pattern of SFML programming and contains of a GUI for painting the graphics and physics backend which is adjusting the position.
// g++ -std=c++14 -lsfml-graphics -lsfml-window -lsfml-system
#include SFML/Graphics.hpp
#include iostream
#include random
// global function
int random_integer(int min, int max) {
return min + rand() % (( max + 1 ) - min); // fast random generator
}
class Robot {
public:
sf::Vector2f pos;
sf::Vector2f goal;
void move() {
int diffx=goal.x-pos.x;
if (diffx>0) diffx=1;
if (diffx<0) diffx=-1;
if (diffx==0) diffx=0;
int diffy=goal.y-pos.y;
if (diffy>0) diffy=1;
if (diffy<0) diffy=-1;
if (diffy==0) diffy=0;
pos=sf::Vector2f(pos.x+diffx,pos.y+diffy);
}
void setgoal() {
goal=sf::Vector2f(random_integer(0,600),random_integer(0,400));
}
void update() {
move();
if (pos==goal)
setgoal();
}
};
class Physics {
public:
std::vector myrobot;
unsigned numberrobot=5;
void update() {
for (unsigned i=0;i
With only five robots on the screen the performence is well. According to the Fedora taskmanager the program occupies 3% of the CPU which is better than a Python program would need. I was confident, that C++ is the perfect language for a multi-agent simulation.
After increasing the amount of robots on the screen to a higher value the performance was reduced massively. In the second trial, 500 robots are shown at the screen and the cpu consumption is 25%. While in the third example 3000 robots are painted to the screen and are producing a load of 43% to the CPU.
Only to get an impression for the relation. The SFML window has a size of 600x400 pixel which is equal to 240000 pixel. That means, if a single robot needs 1 pixel around 240000 robots can be displayed at the same time on the window. But the small amount of 3000 robots are too much for the C++ engine and producing a high workload.
I'm not sure, if the program can be made more efficient. The algorithm itself is supersimple. Every frame the physics engine is calculating the new position of a robot which is not very ressource intensive. It seems, that the SFML painting method is the bottleneck. Perhaps it's possible to improve the speed by painting the robots first on hidden surface and the blit the surface to the screen. The other option is, that with the sourcecode everything is fine, but the C++ language is not fast enough.
Update
According to the SFML handbook it's possible to draw on a hidden texture surface first and then blit the texture to the window.
// texture drawing
renderTexture.display();
const sf::Texture& texture = renderTexture.getTexture();
sf::Sprite sprite(texture);
window.draw(sprite);
Technically it works, because all the circles are visible with the procedure to the screen too. Unfortunately the performance is the same like drawing the circles directly to the SFML window. Here again the comparison:
window size: 600x400 pixel, 240000 total
framerate 40 frame per second
5 robots on the screen, 3% cpu consumption
500 robots on the screen, 25% cpu consumption
2000 robots on the screen, 43% cpu consumption
Literature
I'm not the first guy who has problems with drawing a particle simulation with SFML. Here https://en.sfml-dev.org/forums/index.php?topic=4295.0 is a thread about drawing thousands of sprites at the same time. The problem is well known and is called a low framerate. It seems, that in the SFML library there is a bottleneck which prevents the user to draw many sprites at the same time to the GPU.
It seems, that from a technical point of view, the bottleneck can be avoided but not with the normal SFML toolset. If the combination of C++ plus SFML doesn't result into a high efficiency, it make sense to switch back to Python. The advantage is, that nobody is surprised if the pygame rendering engine is too slow to draw more than one hundred particles at the same time to the screen.
Forth is faster than C++
On the first look, C++ is perceived as the queen of all programming languages. A modern C++ compiler is able to generate the fastest code, and the current C++ standard is equal to a powerful tool for creating all sorts of software. The only disadvantage of C++ over other programming langauges like Java or Python is, that C++ is perceived as complicated to learn. But with a bit of training it's easy to become fluent in C++.
There is an alternative available to C++, which is located outside the C/C++ universe. This alternative is Forth and it's the most interesting programming language ever invented. The reason why Forth looks interesting is because it allows the programmer to make a time travel back to the 1980s in which the amount of RAM in a computer was below 500 kb and in which no operating systems at all were available.
The reason why Forth is more powerful over C++ is because it's a minimalistic programming language. The only problem with Forth is, that the amount of tutorials is low, and the existing one can't explain what Forth is. Another problem for the newbies is, that no Forth interpreter at all is available and the best practice method in overcome the situation is write it's own interpreter from scratch. Does it make sense to learn a programing which has no interpreter? Sure, because Forth allows the programmer to really understand what computing is about. The interesting feature of Forth is, that it's oldschool and looks modern at the same time.
Let us take a look how a modern programming language is working which was invented after the year 2000. In modern times, the compiler techniques are going into the direction of Just in time compilers and Virtual machines. This allows to separate between the runtime environment and the compiler. Forth is working with the same principle. At foremost, Forth is a virtual machine which runs a scripting language called Forth. Before the Forth code can be executed, the VM has to be created first. There are many ways in doing so.
In the jonesforth project, the Forth VM was created in assembly language. But there are many projects available in which the VM was programmed in C, Java and even in Python. Instead of asking what Forth is, the better idea is to ask what a Forth VM looks like. This explains very well why Forth is the future. Because a Forth VM is easier to realize than a VM for Python, Java or C#. The average Forth VM can be created in less than 50 kb, some Forth VM are available which are much smaller.
The average filesize for a small Forth VM is 2000 lines of code. The Perlforth implementation has exactly this size, but a Forth VM written in Python needs the same amount of space, https://github.com/whaleygeek/pyforth/blob/master/src/forth.py Somebody may ask what is the idea behind writting all the Forth Interpreters? The main idea is, that Forth allows a single user to master the computer. In contrast to a C++ compiler project which takes many million of lines, a Forth VM can be realized in a low amount of code.
Forth vs. C++
It make sense to compare Forth with the C++ language in direct comparison. From C++ it's known, that it can reach nearly the speed of Assembly language. A Forth VM can't compete with this speed. A Forth VM written in Python runs very slow. Because the VM itself is executed by the Python interpreter. and the program in the Forth VM has to be interpreted as well.
There are some Forth projects available which are trying to provide the maximum speed. They are able to compile Forth sourcecode into assembly statements, similar to what a C++ compiler would do. Such systems are running much faster, but there speed is lower than a C++ compiler. A highly optimized Forth compiler which produces a binary file can reach the same speed like a modern C++ compiler. That means, the binary file produced by the GCC compiler would need around 100 seconds to run, while the binary file from the Forth compiler would take around 130 seconds.
The problem is, that from a technical perspective, it's not possible to beat a C++ compiler. Because at the end, code is always translated into assembly code, and if the process was done perfectly, there is no room left for improvement. That means, in reality, Forth comes close to C++ but it's not faster than C++.
There is a small exeception. If not only the raw speed, but also weak properties like codesize and programming environment is important, Forth is the number one. That means, a program written in Forth needs less space in RAM, that the same program written in C++. Secondly, it's possible to shirink a Forth interpreter into 2000 lines of code, while the same is not possible for a C++ compiler. And last but not least, Forth allows the programmer to invent the wheel from scratch. That means, even if the Forth VM runs 20x slower than the C++ compiler, Forth can teach the programmer very much.
From today's perspective Forth is an early stage. Even it was invented in the 1970's the amount of books and projects is low. Until now, there is no operating system written in Forth, but a few attempts in doing so are available. From the technical perspective it's possible to write in Forth an operating system similar to the early CP/M operating system or the modern menuetOS. The surprising fact is, that the amount of manpower for doing so is small. An operating system is mostly a system library for getting access to the hardware devices and it provides some routines for GUI drawing.
In the future, Forth has the potential to overcome the limitation of C. The C ecosystem has the bottleneck like most programming languages to become bloatware. That means, it is very easy to create new C sourcecode for all sorts of tasks, and at the end, the operating system occupies 2 GB on the harddrive and nobody knows what to do with all the programs, libraries and compilers.
Writing a Forth interpreter
The best starting point in doing so is the Python language. The first version of a Forth VM is created as a learning project. In the second iteration the code gets optimized. In the third iteration it make sense to convert the Python code into a C++ project. And then the Forth interpreter is modified into a Forth compiler which can produce assembly language. If all the code is working great, the next step is to replace some parts of the C++ project with Forth code, and minimize the amount of non-Forth code. At the end, the remaining parts of the C++ projects are converted into assembly language. This would be equal to a near perfect Forth compiler.
Perhaps we should go a step backward and analyze what the overall picture is. The main idea is that the programmer needs a virtual machine to executing a programing language. The virtual machine takes the sourcecode and converts it into assembly code. This opens up many questions:
1. which kind of language should be interpreted by the virtual machine?
2. how to create the virtual machine internally?
For didicatical reasons it make sense to simplify things, this leads the project into the direction of a stack based language similar to Forth. The second question, how to program the virtual machine, depends on the knowledge of the programmer. He has to choose an architecture and a programming language of choice for realizing such a project. So we can say, it's not about Forth but about creating a virtual machine from scratch.
Some Forth implementations are available at github. Most of them are working well, but all of these yet another forth projects have the same problem. They are not able to build a larger community. That means, somebody has created the code ten years ago, and nobody cares. The reason why is because it's hard to combine with existing programming languages. Suppose, we have written a super-efficient Forth VM. It was written in C and can execute a Forth program quite well. But how exactly can this functionality be used in a real workflow? The problem is, that most of the programming world is working with mainstream languages like C, Java and Python. And the newly created Forth VM is not able to call a routine from a C library. This prevents, that the Forth subroutine can be used to extend an existing program written in C.
To overcome the issue a simple but effective technique is recommended, called RESTful. Restful is a networking interface to connect different programming languages. If the Forth VM contains of a Restful API, it's able to communicate with other programming languages. That means, the Forth program can execute code in a different program, and programs written in C or Java can execute code within the Forth VM.
Or let us switch the point of view. Suppose, there is a Forth VM which doesn't contains a restful API. How can the Forth code interact with existing projects? The answer is that there is no way. The Forth program and the program in Java are different worlds which can not speak to each other. A java program can't execute Forth code, while the Forth code can't call a Java subroutine.
In the Gforth systeme there is a feature available to call an existing c library, similar to what ctypes provides for Python programming. But this interface doesn't work very well and even in the case of success, the gforth program is only able to execute C code, but not a library written in C++ nor C#. The minimum standard for integrating Forth into an existing envirionment is if the Forth VM Comes with a RESTful interface.
In the easiest case this is equal to a Forth VM written in Python which has an interface written in FLASK. This allows other programs to send data to the Forth VM. This is the precondition for using the newly created Forth VM in a productive environment. That means, it's possible to write some routines in Forth and combine this with existing code written in a different programming language. From the technical side the goal is to create a Forth VM which includes a RESTful API.
September 01, 2019
Faster vertex paint with SFML
In a previous blogpost it was explained, that SFML is not fast enough for drawing 2000 objects at the same time to the screen. After consulting some online forums the identified bottleneck is the drawing routine of SFML. It can be improved by using a so called vertexarray. The idea is first to store the particles into an array, and then blit the complete array to the screen. In the example screenshot, around 2000 robots are painted at the same time and the amount of cpu consumption is at 5.6 % which is much better than the 45% in the previous example.
According to the documentation a vertex is equal to a single point on the screen. For drawing a larger triangle, three dots are needed. The creating of new triangle is located in the add() function within the sourcecode. First, all the triangles are created, and secondly the entire array is painted to the screen. This seems to be a very fast alternative over drawing each primitive by it's own.
Let us try out, if the technique scales up to larger amount of robots. Suppose the swarm contains of 10000 robots which are displayed at the same time with 40 fps. On a standard notebook the simulation needs only 15% of the CPU ressources. It seems, that the idea of using a vertex array is suitable for large amount of objects on the screen.
Perhaps one word about the update method for the robots. A naive approach would be to put every robot in a separate thread. Since the advent of node.js and non-blocking event-driven programming it's known, that creating thousands of threads in a single program is not the best idea. So I've decided to imitate the idea of node.js and there is only one thread which is the game loop. And each robot is updated with event driven programming. The event is that the robot has reached a goal. Then a new goal is defined. This allows to move all the robots separate but only one thread is needed.
At the end a screenshot of a normal situation which is taken place in most Real time strategy games. On the screen 200 robots are shown which are moving with smooth 40 fps. The overall cpu consumption is 3%, which means, that the application can run 24/7 in the background. What can't be visualized in a screenshot is, that all the robots in the screens are moving. That means there is a lot of action going on.
According to the documentation a vertex is equal to a single point on the screen. For drawing a larger triangle, three dots are needed. The creating of new triangle is located in the add() function within the sourcecode. First, all the triangles are created, and secondly the entire array is painted to the screen. This seems to be a very fast alternative over drawing each primitive by it's own.
Let us try out, if the technique scales up to larger amount of robots. Suppose the swarm contains of 10000 robots which are displayed at the same time with 40 fps. On a standard notebook the simulation needs only 15% of the CPU ressources. It seems, that the idea of using a vertex array is suitable for large amount of objects on the screen.
Perhaps one word about the update method for the robots. A naive approach would be to put every robot in a separate thread. Since the advent of node.js and non-blocking event-driven programming it's known, that creating thousands of threads in a single program is not the best idea. So I've decided to imitate the idea of node.js and there is only one thread which is the game loop. And each robot is updated with event driven programming. The event is that the robot has reached a goal. Then a new goal is defined. This allows to move all the robots separate but only one thread is needed.
At the end a screenshot of a normal situation which is taken place in most Real time strategy games. On the screen 200 robots are shown which are moving with smooth 40 fps. The overall cpu consumption is 3%, which means, that the application can run 24/7 in the background. What can't be visualized in a screenshot is, that all the robots in the screens are moving. That means there is a lot of action going on.
August 31, 2019
Defining the core application of neural networks
A common problem in neural networks is, that the newbies are not sure, for what purpose they can utilize the technology in a meaningful way. The reason is, that neural networks are usually described from it's own perspective which has to do with a certain neuron layout, the weights, a training algorithm or a certain software framework which can be programmed in Python or Java. But this description doesn't answer the question what the meaning of neural networks is.
With a bit research in the existing papers it's possible to simplify neural networks to it's core feature which is called “regression analysis”. A regression analysis is usually done with the SPSS statistics package. The idea, that a dataset contains of input and output variables, and the mathematical model is able to predict the outcome for interpolated input values. The most simple form of a linear regression model can be realized in MS-Excel and here, the model contains of a simple line which goes through existing datapoints. Let me give an example:
In the MS-Excel sheet the dataset: 2,4,6,8,10 is given. If we are visualizing the information in a plot and draw a line through the dots, it's possible to interpolate the missing point. The model can answer the question what the output for new input signal will be. It can predict, that between 4 and 6 the number 5 is there and that on the right side, the next datapoint will become 12.
This kind of regression anaylsis can be solved with neural networks very well. Much better than with MS-Excel or SPSS. The advantage is, that neural networks fit to nonlinear data with more than a single input variable. The training process means, that the existing dataset is converted into the model. Then the neural network is able to predict new output values for unseen input values. It's some kind of advanced interpolation mechanism.
This understanding of neural networks is important because in the task “using neural networks for regression analysis” no or only a few problems will become obvious. That means, no matter which kind of dataset is given, the neural network is able to do the regression task for all of them. This is some kind of core functionality, which fits well to what neural networks can provide to the user.
Sometimes, neural networks are used to do other task outside of regression analysis. And in this domain the most problems become obvious. It will become unclear, how to do the task exactly, and if the model was trained it doesn't work to fulfill the needs of the user.
With a bit research in the existing papers it's possible to simplify neural networks to it's core feature which is called “regression analysis”. A regression analysis is usually done with the SPSS statistics package. The idea, that a dataset contains of input and output variables, and the mathematical model is able to predict the outcome for interpolated input values. The most simple form of a linear regression model can be realized in MS-Excel and here, the model contains of a simple line which goes through existing datapoints. Let me give an example:
In the MS-Excel sheet the dataset: 2,4,6,8,10 is given. If we are visualizing the information in a plot and draw a line through the dots, it's possible to interpolate the missing point. The model can answer the question what the output for new input signal will be. It can predict, that between 4 and 6 the number 5 is there and that on the right side, the next datapoint will become 12.
This kind of regression anaylsis can be solved with neural networks very well. Much better than with MS-Excel or SPSS. The advantage is, that neural networks fit to nonlinear data with more than a single input variable. The training process means, that the existing dataset is converted into the model. Then the neural network is able to predict new output values for unseen input values. It's some kind of advanced interpolation mechanism.
This understanding of neural networks is important because in the task “using neural networks for regression analysis” no or only a few problems will become obvious. That means, no matter which kind of dataset is given, the neural network is able to do the regression task for all of them. This is some kind of core functionality, which fits well to what neural networks can provide to the user.
Sometimes, neural networks are used to do other task outside of regression analysis. And in this domain the most problems become obvious. It will become unclear, how to do the task exactly, and if the model was trained it doesn't work to fulfill the needs of the user.
What can we learn from node.js and golang?
Every programming language which is developed from scratch is telling a story to the audience. In case of mainstream languages like C++ and Python the plot is well known. C++ is the defacto standard for creating super efficient software, while Python can be utilized for prototyping new sourcecode. The story of some recent languages like node.js and golang is not told very often but they have contributed a lot to the overall domain of computing.
Node.js and golang have become famous, because they support the idea of creating a RESTful API. RESTful is a network interface to connect existing programming infrastructure. It solves basically the problem how the different coding standards like C++, Java, Python, PHP and Lua can communicate to each other without convince the other side to switch the programming language.
To understand why this is important it make sense to describe the typical programming dialect controversy. The standard problem for newbies is, that there are around 50 different languages available and it's not possible to become familiar with all of them. So he have to pick one and hope that his language of choice will solve it's problem. Because most programmers have chosen a different language, there is a need to compare the language which results into a question like “is C++ or C# the better language?”. The problem is, that even somebody is able to answer the question he will fail in predicting the future development. It's unlikely, that C++ is able to repress C#, because both languages have a certain niche.
Instead the number of languages has grown rapidly, In contrast to the homecomputer scene in the 1980s which evolves into a single standard, called the IBM PC, there is no sign, that a single programming language will dominate all other. From the aspect of performance, C++ is often called the queen in programming, but the C++ is not strong enough to convince the other communities to switch to the language.
And exactly this is the reason why a network protocol like RESTful is an option. It helps to connect all the different languages into a single standard. The most interesting feature of RESTful is, that no one is forced into a single programming language, but it's ok to combine different languages. That means, user1 is prefering Windows 10 together with C#, user2 has chosen Ubuntu with Python, user3 has installed objective-c on a Mac OS X system and all the written code is able to talk to each other over the RESTful interface.
Sure, it's possible to create a REST server without node.js and without golang as well. Most languages like Java or Python are providing a library for this reason. But node.js and go have introduced the concept first. And that is the story they are selling to the public. And it make sense to follow the idea.
Literature
In the manual about the “8th programming language”, there is short notice about using RESTful words, https://8th-dev.com/manual.pdf (page 58)
Motivation
Why it make sense to send RESTful json formatted data over a network instead of using a single programming language for all purposes? To answer this problem we have to understand what make C# unique from Python, and what is the difference between C++ and Javascript. All of these languages were developed with a certain need. In case of C++ the motivation is well documented, because C++ is one of oldest languages available. The idea behind C++ was, that the user gets the maximum performance of a compiled language. In theory, everything can be written in C++ and the language is used today very often.
But if C++ is so great, why is the C# alternative available? The needs of C# are different from C++. C# was developed by the Microsoft company as a centralized platform for running applications. From the perspective of Microsoft it make sense to promote C# over C++. And this results into a paradox situation. It make sense to program in C# and C++ at the same time. Which results into complex infrastructure in which community A has written wonderful libraries and compilers, but community B has written different libraries and compilers.
But this is not all. Additionally to C# and C++ there is also space for another programming language called Python. The Python VM works different from the C# one, and brings it own libraries and documentation. It is easier to build different programming communities over convincing the existing programmers to stay in the same ecosystem. That means, Python programmers are not interested in writing C# libraries and C# compiler developers have no motivation to contribute to the upcoming C++20 standard. The prediction is, that his kind of heterogeneous infrastructure will grow in the future. That means, in the next 10 years, new programming languages will upraise but existing one will become used more often as well.
An inofficial clearing house which mirrors the existing programming ecosystem is https://repl.it/ It's a website which allows to run sourcecode from different programming languages. Today, around 60 languages are supported. It starts with some oldies like BASIC, Forth and APL, goes over mainstream languages like C++, C#, Java, Python and Javascript and supports also specialized languages like F#, NodeJS, Go and Haskel.
Many important languages like Matlab and Assembly language are missing on the list, but it shows very well what programming is about. Not a single language but around 50 different languages are used in reality. The problem is, that it's not possible to delete some entries from the list, because each of them fulfills a certain need. The only working mode which make sense is to add more languages to the list. This will increase the confusion and makes it harder to unify existing source code infrastructure.
REST in Assembly language
As a proof of concept it might be interesting to write a RESTful server in Assembly language. Not because it's superior to the alternative written in C, but to demonstrate, how to interconnect assembly language programs with high level programs.
How can an assembly program and a Python interact without RESTful? There is no way in doing so, because both languages are quite different. They have no common standard. Python can't execute assembly, while assembly can't parse python code. None of them is wrong. Assembly language is a here to stay but Python too. The problem is, that in general it's not possible for two languages to interact with each other. The same problem is there between Turbo Pascal and C, between Java and Lua, and between AutoIt and C++. The problem get more complicated, if the sourcecode runs under different operating systems. For example, the Java programs runs on a smartphone while the server database runs on a Linux headless server.
Node.js and golang have become famous, because they support the idea of creating a RESTful API. RESTful is a network interface to connect existing programming infrastructure. It solves basically the problem how the different coding standards like C++, Java, Python, PHP and Lua can communicate to each other without convince the other side to switch the programming language.
To understand why this is important it make sense to describe the typical programming dialect controversy. The standard problem for newbies is, that there are around 50 different languages available and it's not possible to become familiar with all of them. So he have to pick one and hope that his language of choice will solve it's problem. Because most programmers have chosen a different language, there is a need to compare the language which results into a question like “is C++ or C# the better language?”. The problem is, that even somebody is able to answer the question he will fail in predicting the future development. It's unlikely, that C++ is able to repress C#, because both languages have a certain niche.
Instead the number of languages has grown rapidly, In contrast to the homecomputer scene in the 1980s which evolves into a single standard, called the IBM PC, there is no sign, that a single programming language will dominate all other. From the aspect of performance, C++ is often called the queen in programming, but the C++ is not strong enough to convince the other communities to switch to the language.
And exactly this is the reason why a network protocol like RESTful is an option. It helps to connect all the different languages into a single standard. The most interesting feature of RESTful is, that no one is forced into a single programming language, but it's ok to combine different languages. That means, user1 is prefering Windows 10 together with C#, user2 has chosen Ubuntu with Python, user3 has installed objective-c on a Mac OS X system and all the written code is able to talk to each other over the RESTful interface.
Sure, it's possible to create a REST server without node.js and without golang as well. Most languages like Java or Python are providing a library for this reason. But node.js and go have introduced the concept first. And that is the story they are selling to the public. And it make sense to follow the idea.
Literature
In the manual about the “8th programming language”, there is short notice about using RESTful words, https://8th-dev.com/manual.pdf (page 58)
Motivation
Why it make sense to send RESTful json formatted data over a network instead of using a single programming language for all purposes? To answer this problem we have to understand what make C# unique from Python, and what is the difference between C++ and Javascript. All of these languages were developed with a certain need. In case of C++ the motivation is well documented, because C++ is one of oldest languages available. The idea behind C++ was, that the user gets the maximum performance of a compiled language. In theory, everything can be written in C++ and the language is used today very often.
But if C++ is so great, why is the C# alternative available? The needs of C# are different from C++. C# was developed by the Microsoft company as a centralized platform for running applications. From the perspective of Microsoft it make sense to promote C# over C++. And this results into a paradox situation. It make sense to program in C# and C++ at the same time. Which results into complex infrastructure in which community A has written wonderful libraries and compilers, but community B has written different libraries and compilers.
But this is not all. Additionally to C# and C++ there is also space for another programming language called Python. The Python VM works different from the C# one, and brings it own libraries and documentation. It is easier to build different programming communities over convincing the existing programmers to stay in the same ecosystem. That means, Python programmers are not interested in writing C# libraries and C# compiler developers have no motivation to contribute to the upcoming C++20 standard. The prediction is, that his kind of heterogeneous infrastructure will grow in the future. That means, in the next 10 years, new programming languages will upraise but existing one will become used more often as well.
An inofficial clearing house which mirrors the existing programming ecosystem is https://repl.it/ It's a website which allows to run sourcecode from different programming languages. Today, around 60 languages are supported. It starts with some oldies like BASIC, Forth and APL, goes over mainstream languages like C++, C#, Java, Python and Javascript and supports also specialized languages like F#, NodeJS, Go and Haskel.
Many important languages like Matlab and Assembly language are missing on the list, but it shows very well what programming is about. Not a single language but around 50 different languages are used in reality. The problem is, that it's not possible to delete some entries from the list, because each of them fulfills a certain need. The only working mode which make sense is to add more languages to the list. This will increase the confusion and makes it harder to unify existing source code infrastructure.
REST in Assembly language
As a proof of concept it might be interesting to write a RESTful server in Assembly language. Not because it's superior to the alternative written in C, but to demonstrate, how to interconnect assembly language programs with high level programs.
How can an assembly program and a Python interact without RESTful? There is no way in doing so, because both languages are quite different. They have no common standard. Python can't execute assembly, while assembly can't parse python code. None of them is wrong. Assembly language is a here to stay but Python too. The problem is, that in general it's not possible for two languages to interact with each other. The same problem is there between Turbo Pascal and C, between Java and Lua, and between AutoIt and C++. The problem get more complicated, if the sourcecode runs under different operating systems. For example, the Java programs runs on a smartphone while the server database runs on a Linux headless server.
Subscribe to:
Posts (Atom)