How to fix the traffic jam in India

Fixing something is usually done by putting the resources towards the right place. One attempt might be to build more roads in the attempt that this will reduce the amount of traffic jams. But the most efficient resource allocation is done at a different location, namely the railroad system. Similar to the car roads, the train system in india has a high demand but low amount of ressources. And building new railroad networks is expensive. A cost effective alternative is to measure the existing traffic with more sensors.

So a cheap version to improve the efficiency is to buy a simple arduino based sensor and count how many trains are arriving at the trains station and how many are leaving. Also the sensor counts the time and date.

Such a single sensor won't be able to fix all the problems of India, but it is a first step towards a possible solution to the traffic jam problem.

Linux is a large hard drive

 Computers in the past were easy to explain. The commodore 64 consists of ROM which holds the operating system and MS DOS needs a single boot floppy drive which consists of only three files: command.com, io.sys and msdos.sys

In contrast, a modern operating system like Linux looks similar to a mess. There are endless amount of small files which contains of config files, man files, documentation, binary files and log files. The exact amount of all files can be determined either the in graphical "disc usage analyzer" or on the command line with:
df -i /

The amount of active inodes is equal to the files which includes directories. For most fresh installed Linux distribution the amount is around 500k. Only for comparison reason a fresh Windows installation needs around 300k files.

Most Linux beginners and even experts will struggle to explain what is stored in the these files. In most cases it contains of system programs, applications and helper files for the fonts, icons and so on. The only thing what is for sure is, that Unix will need all these files and many real life webservers will need much more additional files for the home directory. The huge file count is perhaps the most obvious difference between a commodore 64 and a unix system.

It is not possible to install the Unix operating system on a single flopp drive or in the ROM of a the hardware. This is an advantage and disadvantage at the same time.

It should be mentioned, that the amount of 0.5 million files on a standard linux system is a small amount of files. The reason is, that only the trunk version of each file which is the latest was installed. In addition the git repository contains many more versions which is available in the cloud. So we can say that the Linux operating system contains of a much more different files which were created over the years. It seems, that this is not perceived as a problem, but it is strength of Unix. Because it allows to growth the system endless.

Unix is working with some underlying assumptions. The idea is not about running a program on a CPU. Such a paradigm was valid for homecomputers. In contrast Unix is working with the assumption that there are 500k files on the harddrive and with each update, additional files are created. Also the existing files are modified by the upstream in a version control system. So the overall Unix system has much in common with a repository of programs and config files.

On the Commodore 64 and early CP/M computers the principle was to reduce the amount of files. Fewer files can be stored on less floppy disks and are running faster on the computer. In addition, fewer files can be compiled faster and contains of fewer bugs. With the advent of Unix, such a minimalist approach has been abandoned.

Tribute to Ansi C

 

The C90 Ansi C standard is a great choice for programming. The language was invented 40 years ago and outperforms every modern language easily. C compilers are available for all computers and all operating systems and allow to write any sort of software. No matter if C is used to create videogames, webapplications, desktop apps or even operating systems libraries — c is always a great tool.

Every criticism against the c language is a sign for a misunderstand what programming is about. Programming is not about drag and drop GUI widgets into a frame and press a run button. Programming is not about importing a python library without understanding what the inner working is. But software engineering is about understanding from a low level perspective what the computer is doing internally. This includes to use raw pointers with direct memory access to compare two strings, and write sorting algorithms from scratch because they are missing in the standard C library. This is not problem, but is ensures that programming makes fun.

So called high level languages like Java or PHP are created by inexperienced programmers for non programmers and are a sign of failure. The best recommendation is to rewrite all these existing nonsons programs with Ansi C and create new software always in C as well. Only a compiled language can be executed fast enough and it is simply not true that the productivity in scripting languages is higher than in C. Most libraries are not needed in the reality. For example the matplot library which allows python programmers to draw charts can be simply emulated with a c program which draws the graphics into the video ram of the graphics ram. The underlying algorithm can be written from scratch in a short amount of time and is more cpu efficient than any precompiled library.

Object orientation which has become famous since the 1990s is a dead end because Ansi C has a more powerful feature called modules which allows to interconnect different programs into larger systems. There is no need to create complex C++ compilers or even invent jit compilers to execute PHP, but the same task can be realized with a vanilla C compiler much easier. Basically spoken, all these java programmers are wasting their time, if they are writing the software in so called high level languages. They have simply not understood what a linked list is or what the C preprocessor is doing internally.

With the c language everything is great, but the ecosystem around C has made major mistakes. In the last 20 years lots of new programming languages were invented with the promise to make programming easier to learn. Non of these attempts was able to replace the c language. Buzzwords like agile programming and object orientation were mostly invented by c programmers and can't be replicated in other programming languages. a well trained C programmer is able to write any sort of applications within a week. All what he needs is a VIM editor plus the gcc compiler and the c internal features are producing error free code by itself. The resulting binary file will run on a server and a desktop very well and it is unlikely that the code needs to be improved. And if another programmer will find an error, he can easily locate the error because the program is divided into smaller chunks. Ansi C has a natural tendency to evolve into stable software projects which are written once and then used as production ready code for decades. The C language stands for high software quality and maximum coding productivity.

One of the strength of C is string handling. There are more than 12 different ways available how to concat two strings together.[1] it is recommended to decide for one of them and the program will be protected against all sort of security issues.

[1] https://stackoverflow.com/questions/8465006/how-do-i-concatenate-two-strings-in-c

Which programming language is the best?

 

There are many attempts available to compare different programming languages each other. In contrast to a famous myth it is possible to determine which language is the best if some prerequisites are fulfilled. First thing to do is to define what “best” means. A possible definition is, that best= high level programming language. The assumption is, that programmers are preferring high level over low level languages. Then it should be defined what a high level language is about. High level means that no pointers are needed, that object orientation is available, that the language is interpreted and very important that a huge libraries with predefined functions is available.

The reason why high level language are preferred over low level language is because it allows to create the same app in a shorter amount of time Writing a hello world gui app will take in pure assembly language around 1 month, in the c language it will take 1 week, in the c# it will take 1 day and in python it can be realized in 10 minutes.

The interesting situation is, that the sorting order from low level to high level language can be measured on an objective basis. Around 99% of the programmer will agree that the following sorting order is correct:

1. assembly language

2. Ansi C

3. C++

4. Java, C#

6. Python, PHP

Low level language are compiled and are using pointers. Midlevel langages like C# doesn't use pointers and there is need to define the datatypes, while high level languages like Python doesn't need anything but the hello world Qt app is created with 5 lines of code and executed on any device and any operating system.

The only question left open is, if high level language are a here to stay or if python has some sort of disadvantages. In most cases the concern is, that high level languages are not great for production systems. Python is known to run slowly, while C# and Java is sometimes called as not snappy enough similar to C++. According to the raw numbers a jit compiler and especially Python like interpreters are much slower than low level languages. And the question is, if such constraints is important or not.

According to the amount of books about high level language and the number of newly created apps, high level languages have replaced low level languages. Most newly created software is not created in Ansi C anymore but in C# and other high level programming languages.Somebody may argue for or against this development but at least it is a fact. The motivation for the individual programmer is mostly to reduce the development time. Instead of programming an app with Ansi C within in 2 years, the idea is to program the same app with a high level language in only half of a year.

In the context of python this sort of programming style is sometimes criticized as creating throw away code. This implies that python code is equal to low quality software. In contrast, a dedicated low level language like Ansi C is labeled as high quality language because it takes much longer until the app is ready.

It is possible to discuss which of the opinion is correct. What we can say for sure is, that all the programing languages can be sorted in a linear chain from low level to high level. High level means, that the same app can be written with less amount of codelines. The result is, that it can be defined precisely if python is more high level than C# or Ansi C.

Python is not the top of the pyramid because there are some languages available which are more high level. GUI RAD builders and especially the MS Access database surpass easily the python language in terms of high level vs low level language. A MS Access database can be created without writing a single line of code. Somebody may ask if such a database has a high quality compared to a program written in C# or even Visual Basic, but at least the fact is, that MS Access is more high level than any programming language.

Another example for ultra high level languages are construction kits and level builder in Games. Here the user doesn't program the game or the app, but is modifying only a level. For example in a level editor for the lemmings game it is possible to draw some obstacles into the screen and after pressing the run button the game gets started. The possible alternative would be to create the game with C++ from scratch which will take for sure months upto years and a deep understanding of the C++ language. With a level builder the same objective can be realized much easier.

A while ago there was a debate at stackoverflow about rewriting an existing PHP webappliation in plain Ansi C.[1] The main argument against such an attempt was, that it will take too much manhours for doing so. Even if the php language runs a bit slower than Ansi C it is the better choice because it is more high level. High level are seen as equal to a good choice because it allows to save time and costs. Even some comments have argued pro C/C++ and the arguments were really good, it is easy to grasp that the stackoverflow post came to the conclusion that PHP is the better langauge for web development over C/C++. Simply because of the fact that only very few or even none web applications are written in C/C++.

[1] https://stackoverflow.com/questions/3079153/why-do-you-not-use-c-for-your-web-apps

Is C# obsolete?

 

In the history of programming language there is a tendency available from low level- towards high level languages. From the perspective of the C language all the programs written in Assembly are outdated. The reason is that the source code can be written in a shorter amount of time. Because of this reason the C# language has replaced the former C++ language entirely.

On the long hand the transition towards high level language will make even c# obsolete. The reason is, that in comparison to scripting languages like Python, C# can be called a low level language. There is need to use datatypes like int and string and many class initialization will need a lot of boiler plate code. Without any doubt the Python interpreter is more high level than C# so the C# language is outdated.

Writing a simple hello world gui app with C# will take around 30% more lines of code than the same app written in Python. Such an overhead is difficult to maintain and it will increase the time to market time.

According to the Rosetttacode website, the string append problem can be solved with many programming languages. The low level C language will need the most amount of written lines, then comes C# and the shortest code can be written in Python.

// c
#include<stdio.h>
#include<string.h>
#include<stdlib.h>

char *sconcat(const char *s1, const char *s2)
{
  char *s0 = malloc(strlen(s1)+strlen(s2)+1);
  strcpy(s0, s1);
  strcat(s0, s2);
  return s0;
}
int main()
{
  char str1[]="Hello ";
  char str2[]="World";
  char *str3=sconcat(str1,str2);
  printf("%s\n",str3);
  return 0;
}


// c#
class Program
{
    static void Main(string[] args)
    {
        string x = "foo";
        x += "bar";
        System.Console.WriteLine(x);
    }
}

// python
str = "12345678";
str += "9!";
print(str)

 

The difference between C# and Python is not huge, but C# will need around 30% more space for writing the semicolon, the datatypes and lots of extra code for anything. This additional effort has to be written by the programmer which reduces the productivity. Basically spoken, C# is outdated because it not high level enough.

Let us take a closer look into the C program. First thing to mention is, that the string concat method was implemented safe and is one of the easiest to understand. It is using in addition external routines to make things easier. It is possible to create more advanced string concat routines. Nevertheless the lines of code are difficult to understand. For C programmers it makes sense to use three different commands like malloc, strcpy and so on only to add str1 with str2 but from an outside perspective, the source code is a bit hard to read.

Such situation is available for many problems which should be implemented in the C language. In most cases, there are at least 40 different ways available how to solve it, and all of them have to do with pointer manipulation and manually typing in lots of code lines. In contrast, the written code in the python language is much easier to type and to understand. All what is needed is a statement like “str1+str2” without further subroutines.

The advantage is not located in Python directly, because string manipulation is also easy going in languages like PHP and Ruby. But it has to do with low level vs high level languages.

 

 

Writing larger programs in Ansi C and Forth

 

The dominant reason why object oriented programming languages have become successful is because of its ability to split larger problems into classes. Without this feature, the program code is stored in a single large file in which hundred of functions have access to hundred of variables. On the other hand, many larger programs were written in Forth and Ansi C without any sort of OOP feature so it seems, that some sort of hidden technique is available which works similar to object oriented programming.

What Ansi C programmers are aware of it is the module concept. A module is a file in which the scope of the functions and variables is limited to this file. it is possible to set a value in a module like “module1.msg=”hello world” and that call a function in that module with “module1.show()”. This allows to simulate object oriented programming.

Modular programming allows to reduce complexity without introducing dedicated OOP techniques. This allows to use much simpler compilers like the ansi C compiler or even minimalist languages like Forth. The question is not if a certain programming supports objects but if the concept of modules is available. Similar to OOP programming modular programmings allows to split larger problems into smaller one. Each function is limited to a single module and this allows to solve the problem in general. The pascal language which lacks also of OOP features has of course the feature to utilize modules. In pascal a module is called a unit and similar to C it is stored in a single file. From the perspective of Java and Python a single pascal unit is equal to a class which has variables and methods.

Modular programming with python

 

Sometimes it was asked how to implement object oriented programming in low level languages like C and assembly. A possible idea would be to improve the existing struct datatype with functions which are pointers. Then the user has to allocate memory and can use objects in C. But the overall pipeline is very complicated and there is an option available to simplify the idea drastically.

The idea is to dismiss object oriented programming at all and prefer modular programming. Modular programming is a complexity reduction technique which is available in most existing programming languages. It can be used in python and in C and pascal as well. The idea is that a single file is equal to a module. [1] The file contains of variables and functions and comes close to the concept of a class. in the written sourcecode a program written with modules and one with classes are looking nearly the same. A module allows to group items in the sourcecode into cluster and maintain the code separately.

Most object oriented programs are in reality normal modules distributed over many files. The only reason why OOP is the dominant programming paradigm and modular programming has become a niche is because of the existing programming tutorials. The self understanding of most books is, that a programming language consists of statements and functions and this enough to solve a problem. The problem is, that without classes aka modules it is impossible to solve larger problems. If the code is longer than 500 lines of code there is a clear need to split it into sub entities. Unfortunately this strategy is not implemented directly in a programming language but it has to do with project management and compiler preprocessors. It can be s

From the perspective of modular programming it is surprising to see, how similar programming works in different languages. In contrast to a common myth the programming languages Pascal, Python, C++, C and , Assembly and Forth have one thing in common. They are supporting all the concept of units. A unit is – again – a single file which can be included in other files and allows to split the code into logical groups. Similar to OOP it allows to group variables and methods into the same instance.

To understand why modular programming is not very common it is important to take a look back into the didactic of programming. In the 1980s the concept of structured programming was mentioned frequently. Structured programming means to divide the sourcecode into functions which can be realized with C and pascal. Advantage is, that each function can be maintained separately which allows to writer longer programs. What was ignored in the 1980s is, that structured programming alone is useless because if a file contains of around 10 functions it will become hard to maintain anymore. So the limit is, around 500 lines of code which can be managed with structured programming alone.

In the 1990s the concept of object orientation was promoted as the logical next step after structured programming. The idea was to encapsulate the code in object libraries which allows an unlimited code size. Nobody has questioned the idea at all or compared it with modular programming which provides a similar feature.

In theory, OOP is superior over normal modular programming. But in the reality, most programs won't profit from these additional advantages. Nearly all projects which are smaller than 100k lines of code can be realized with modular programming very well.

[1] https://python-course.eu/python-tutorial/modules-and-modular-programming.php

The surprising success of the C programming language

 

There is some sort of mystery in the history of computing. In spite of many criticism, the C language was never abandoned but it was used over decades for creating all sorts of programs. To understand the success we have to focus first on some of possible alternatives.

Since the early 1980s the C language was known and many attempts were made to program in a different language. Notable examples are Basic, Pascal and recently Python. All these programming languages are promising to the newbie that it is possible to write software without much effort. Basic was the first widely used interpreted languages. First versions were oriented on line numbers, while later Basic interpreters like Omikron Basic and Visual basic provided the ability to create large projects with structured and even object oriented features.

The problem with basic language was, that it was never standardized. A program written in Qbasic can't be executed in visual basic. And both systems have the problem that the generated executable code is very slow. Many programming languages were invented over the years. Some very successful examples in the recent years are python and C#. In contrast to BASIC, Python is standardized, cross platform ready and has a huge amount of libraries. Similar to Basic it is used by non programmers for creating a prototype. Then the code is deleted but never reused again.

Parallel to the development of these alternative language, the C language has remained stable over the years. What is known as modern C is the same what was used in the late 1980s as C90 standard. It is a standardized compiled language which is used to create production software for the public.

Roughly spoken, Basic and python are toy languages even they are used by 99% of the programmers. In contrast, C is practical language because the code is written once and then improved over decades. The chance that newly written Basic code can't be executed after 5 years is high, while the future of C is more secure. Created C code in now will be uptodate after 10 years from now for sure.

The obvious difference between python and C is, that python is easier to read. Similar to Pascal It is a great language for academia, but it fails for writing operating systems or even normal software. Basically spoken,the programming world is divided into two groups: c programmers and non c programmers. The non c community is much larger. Most books, tutorials and written lines of code are created in non c languages like:

Python, Basic, PHP, Ruby, C# and Java

But, non of these projects are real projects in a sense that the code is executed on a computer. But the code is written once for testing purposes and then it gets archived somewhere

A look into one of the existing C tutorials is disappointing. There is no explanation how to program games, build a database program or create simple hello world apps. But the C books are explaining how the c compiler generates Assembly code without any practical purpose. C has replaced the former assembly language in the late 1980s. During that time it was normal to write code in assembly language. This was done for unix systems and home computers as well. C has replaced this programming technique.

The malloc command within the c language explains very well what the difference is between C and non c language. Malloc allows to ask for memory block from the operating system. The programmer can ask for 1 GB of memory and then the newly written code will nned such amount of RAM for runtime. Such kind of low level memory occupation is not possible with alternative language or at least it doesn't make much sense. Because most modern programming languages have a garbage collector. In Python for example, the user doesn't specify the datatype at all but he appends something in a list and the rest is delegated to the python interpreter.

[1] Bradley L. Jones: Sams Teach Yourself C in 21 Days, 2002

[2] O'Reilly Practical C Programming