How to make hierarchical notes

Creating a text file in the Linux operating system is a well known task for all computer programmers. In addition it is possible to create text files which are structured hierarchically. This is done with the markdown syntax. A good texteditor is able to render the structure in an outline pane. This allows to jumps easily to any position in the text file.

Somebody may ask what is the reason for creating hierarchical notes in contrast to a normal text file? The reason is located in theh left pane. Similar to a list of methods in a python file,, the left pane shows the overview. All the details are hidden and only the sections and subsections are shown. This is some sort of mind mapping feature in which a problem is split into sub-topics. In contrast to a drawing on a sheet of paper, a markdown formatted text file is much easier to create and modify.

Another interesting feature is, that English text can be mixed with short notes. There is no need to enter full sentences but a section can be filled with 3-4 words long lines.

Creating professional documents with LaTeX

LaTeX is a great choice for professional formatting. The following tutorial is a bit unusual but it will result into a high quality typographic layout. First step is to write the text itself. The best way in doing so is the markdown syntax. The main advantage is, that hierarchical sections can be defined. These sections are displayed in a text editor as an outline so there is no need to use a dedicated outline editor but a normal text editor works pretty well for this job.

If the text was created and proofread it has to be converted into the TeX format.
pandoc -o paperlatex.tex -s paper.md

The next step is to modify the header of the latex file so that it will look like MS-Word and libreoffice:

\usepackage{wordlike} % word like fonts and TOC
\usepackage[none]{hyphenat} % deactivate hyphenation
\setlength{\parskip}{8pt} % deactivate vertical glue
\setlength{\topskip}{0pt}  % deactivate vertical glue

Hello world with Scribus

 

How to avoid LaTeX at all

LaTeX is solving a simple image conversation problem. The initial situation is that somebody has a folder with some files which are text files, vector graphics and photos and he likes to combine the content into a single file in the pdf format. According to the LaTeX community the only way in handling this task is by installing an entire ecosystem which contains of 15 GB in total which contains of lots of fonts, outdated macro packages, handreds of documentation and obsolete DVI preview programs.

Nevertheless LaTeX is able to handle the task very well but it is not the only program on earth which is able to do so. A good alterantive is the libreoffice program. The most basic feature of libreoffice is that the user can insert multi images, arange them on a US-letter page and export the result into a pdf file. Even libreoffice wasn't designed as an imagemagic alternative it is pretty good in this task. The user can even decide to reduce the resolution from 300 dpi to 100 dpi to save a lot of disc space.

Let us go to the initial problem. There is a folder which contains of different files and the task is to convert these files into the pdf format. And yes, libreoffice can handle this problem very well. Suppose the software has some bugs, one of them is that the programs a bit slow, especially with some images. But, these are only minor problems and can be fixed in future versions.

The interesting situation is, that Libreoffice can combine existing content much faster than what Latex is able to do. The reason is, that combining existing content into a single pdf file is not very complicated. The most time consuming step is to write down a text file which has for example 500 kb. Creating such a plain text file with an editor can take weeks. Also the creation of high quality jpeg, eps and SVG images can become very time consuming. If someone likes to paint images with the gimp program he will need also weeks until the image is rendered into the jpeg format.

But, if the content is already there and the only task is to combine the content into a single document which is a newspaper, a book or a presentation – such a step can be realized in under an hour. Let us take a closer look how libreoffice is working. After starting the program the user can select “insert ->text from file”. This inserts the plaintext file. In the next step the style sheets for the sections are assigned, and of course this step is made manual. But, the average document has only a low amount of such sections so it is done after a short time. In the next step, some pictures are added. This is also done with the insert menu. Either the picture can be added direct as a picture, or it can be added in a larger frame which allows to modify the position more precise.

Then a short click on “export ->pdf” allows to render the document into a single file. The overall workflow results always into a high quality pdf file, and in contrast to latex the user doesn't need to know the complicated markup language or has to install larger amount of software he doesn't need. The interesting situation is that libreoffice can handle a huge amount of images in a single document. This is realized with the “insert image as link”. The original libreoffice file has a small size and the images are stored externally in the directory folder.

So let us go a step backward. The most time consuming time for creating a technical document is to typing of the text and the painting of the images. The task of combining these information into a print ready pdf file is something which can be handled on the fly. There is no need to automate this task, because then the user will loose the control. The most disappointing moment for latex users is that they will recognize that they can't figure an image where they want. This is marketed as an advantage but in the reality, the user likes to take this decision by himself. Using latex is not recommended anymore, because there is more powerful alternative software available.

The screenshot shows a concrete example. First thing to mention is, that the the display of the images was deactivated in the option menu. This improves the performance of the program. Second thing to know is that both images were anchored to the page. This setting allows to freely position the images similar to a layout software. The user can show the result in the preview winder or export it into the pdf format. Somebody may argue that this ability of Libreoffice is not very exciting. Oh it is. LaTeX isn't able to do so and usually only advanced layout software can handle images that way. What most open source programs like gimp and imagemagic can do is to handle a single image file, but not 2 and more of them.

Text vs layout software

 

There are roughly spoken two different approaches available how to create pdf files: Scribus vs. LaTeX. Scribus is graphical oriented while latex is text oriented. To answer which one is better whe have to take a closer look into a working directory. Suppose the idea is to create a book. Then the working directory contains of some files for the graphics, a single text file and more graphics file in a vector format.

It is very likely that the file size is very different. The text file has maybe a size of 500 kb while the graphics file have combined around 20 MB. According to the size the graphics files are more important because they are bigger. And here is the problem, graphics oriented layout software puts a high priorioty towards the graphics file. The idea is these files are handled first. While the text file which has only 500 kb is treated with low priority. The idea is that nobody cares what is written in the text but the audience is interested only in the graphics.

At least this is the assumption behind the scribus software. The problem is that in the reality it is the other way around. Creating a 500 kb large text file takes a long time. A trained author will need weeks or months until he has written such a document. In contrast, creating vector graphics and especially photos with a camera will take only hours or minutes.

Let us take a look how LaTeX is seeing the world. LaTeX assumes that the most interesting content is located in the text file. The text file is rendered into a pdf document and in the first run the draft mode is activated. Taht means the images are not visible but only the text. This allows to proof read the text and new sections. LaTeX is not the only software which has a focus on text. For example the markdown format or the MS-Word software have a similar approach. In all these cases the idea is that graphics are an add on to the text but not the other way around. That means, text without images makes sense, but images without text not.

This understanding is not only a subjective measurement but it has to do how long does it take to create a certain sort of content. Suppose somebody is using a photo camera and is making some photos on a single day. How many content can this person produce? Very large amount of information. He is able to create 500 photos each one has 3 MB so it is 1500 MB in total. In contrast it is not possible to create on a single day a plain text file which has this size. The reason is that the typing speed is much smaller. And proofreading a text takes longer than applying a filter to a photo.

The initial question was which sort of software is great for creating books. The answer is that it should be a text oriented software in which graphics can't be displayed or are handled with a low priority.

Pros and cons of the cherrytree program

 

Cherrytree is an outliner software and the only example in the Debian repository. Outliner tools are a relative recent development in computing and were developed in parallel to word processing software which was recognized in mainstream.

Especially the idea of a two pane outliner software was available since the 2000s. The cherrytree software is one example for such tools. Unfortunately the software quality is low. The menus are not making much sense and the chance is high that the amount of users is low. But let us go through the program itself. The main feature is that the user can create sections in the pane on the left side and then enter a longer text on the right. So it can be compared with the navigator pane in libreoffice.

The other functions of cherry tree are little. The user can import and export some files and he can also insert images and tables. Nevertheless the idea of an outliner is so interesting that it makes sense to analyze the program in detail. An outliner is some sort of mindmap in a fulltext mode. The user is not required to enter a linear text from top to bottom but he jumps between the nodes similar to what programmers are doing if they are creating multiple classes. In addition the user can enter only keywords.

So the program has it strength in creating to do lists, short notes and very important book scripts. Outliner tools have became the defacto standard for writing screenplays and movies because of its ability to create complex hierarchical texts.

Let us talk about the cons of the cherrytree software in detail. Suppose the user has created a longer document and likes to export it into the pdf format. Technically a pdf file is created. But the margin can't be adjusted and the image resolution can't be changed. So it is more a preview function than a pdf generator. If the user likes to create a readable pdf paper he will need additional software like the previously mentioned libreoffice software.

So the question is why the user is creating first the notes in cherry tree if libreoffice writer has a similar feature under the term navigator? This would explain why the amount of users is low becauase if someone likes to create hierarchical notes and format them as a pdf paper he can do so with libreoffice writer much better. Another program which works with a similar principle is lyx. The disadvantage of lyx is that it needs for formatting purposes a fullblown latex installation which needs including the fonts around 15 GB on the hard drive.

But let us go back to the cherry tree program. The best way to imagine the idea is to take an existing text editor like gedit and enhance it with the ability to insert images and add a pane on the left side which shows the hierarchical structure. This results into cherry tree.

Creating a newspaper with libreoffice

 

The libreoffice suite has an interesting feature which wasn't recognized by the public yet. The feature is called siimply text box or frame and allows to transform Libreoffice into a software similar to Scribus. But let us slow down the workflow and bit. Suppose the idea is not to create a text but a newspaper with multiple columns. The existing Libreoffice or LaTeX software is not suited well enough for the task because a newspaper is working visual.

Instead of using a different software, the idea is to built around the frame concept the document structure. The interesting fact is that the frames in Libreoffice can be linked together, similar what Scribus provides. So the user can define different frames and let the text flow in this structure.

The screenshot shows the rendered pdf file. The interesting situation is that the filesize is very small (34 kb which includes the png image) and very important the user can create as many columns he like. He has the freedom of a layout software but can handle text in the frames.

Academic typesetting with layout oriented programs

 

In the LaTeX community there is some sort of missing knowledge obvious how to create a pdf paper without the tex engine. Sure it is known that apart from TeX some other programs are available like Quarkxpress, Scribus and layouting software in general but it remains unclear how exactly such tools are used to process text. Instead of explaining how a certain software is working the following blog post will introduce how to use layout oriented software in general.

A good general software which is working with the frame paradigm and is preinstalled in most Linux systems is the Libreoffice draw software. The surprising fact is that the inner working has much in common how scribus and other layout DTP programs are working. After starting the program, the user can create a new text box on the page and fill it with example text.

Right on the screen a pane is shown which allows to adjust the properties of the box which includes the font size and the paragraph justification. With a bit intervention the user can make the text box look different. Also an image can be added.

The idea is that the user creates each page as a separate file and saves it into the vector based pdf format. At the end an entire newspaper or book is created in this way. The main advantage is that the user has the maximum control over the layout, he can arrange the items, add headlines and so on.

So let us explain what the difference is between libreoffice draw and the scribus software. There are only minor improvements available. In scribus it is possible link two text boxes together also it is possible to hyphenate the content of a text box. But the general interaction with the software is the same. So we can say that the Libreoffice draw software is a great choice for creating a multi column newspaper which contains lots of graphics.

Scribus for LaTeX users

LaTeX is a widely known technical authoring tool. It has a community in the computer science department and in mathematics. It is used since the 1980s and allows to generate large complex pdf files mostly automatically. Even if the software is so great there is a need to investigate what potential alternative programs are available. Most users are familiar with the Libreoffice program already but a less described program is Scribus. Scribus is a Quarkxpress clone and is working similar to other layout software with the frame concept. The user can positioning text frames on a page.

The following blog post explains how exactly this workflow is done to create a longer academic paper. First thing to do is to make sure that the latex file is available and all the images are available either in the PNG or in the EPS format. And now it is up to the Scribus newbie to use this content to create a professional looking pdf file.

The first problem is that scribus can't import the LaTeX markup format. What is possible instead is to use plain text as source. So the user has to tag again all the sections in the file. This is realized of course manual but after a while all the sections are formatted this way.

A unique problem of the Linux version of scribus is that some dialog boxes can't be resized so in the very important style menu it is not able to update the style property. This makes it harder to assign a certain font to all the section. What the user can do instead is to format the words manuel which is technically possible but takes extra time.

The text is spread over many frames. Each page has a new frame and they are linked together. For reason of simplicity i have selected the single column mode. If the text was entered the next step is to add the figures. In contrast to LaTeX, there are no float images but the user has to position the pictures manual on the page. The advantage is that he can make the page layout more convenient. For example it is possible to place a wrap figure page on top of the page, which is not possible in LaTeX. Also a table was added.

Then a major problem has occured. Scribus can't create a Table of content. According to the documentation this is very complicated, so i have left out this step and made only a manual Table of contents. And now the long waiting moment is there. The document gets rendered into the pdf format. The output of scribus is much bigger compared to latex. The same document needs 2.5 MB vs. 400 kb.

A sbujective judgment comes to the conclusion that the pdf quality of scribus is worse than LaTeX. Technically it is possible to create pdf files with Scribus but it can't replace LaTeX. Nevertheless it was an interesting experience to see how Frame oriented layout software is working.

Content vs layout

After this report let us discuss what the most obvious difference is between both programs. LaTeX has a focus on text structure. It is some sort of advanced outline editor. The user creates sections and subsections and this allows to add new content to an existing text. In contrast, Scribus is based on the idea of a page. The user has to create new boxes and arrange them to fulfill certain layout requirements. It is not possible to add new subsections, but this will cause a complete reformatting of the project.

Some applause for the lyx software

 

It is some sort of mystery why the LaTeX ecosystem has so many followers. The only time in which LaTeX was new an shiny was in the 1980s but then many other powerful software were created but it seems that the TeX community doesn't recognizes the change.

Suppose the idea is that LaTeX and especially the Lyx frontend has become old and outdated and the user likes to use a better alternative. Which sort of software comes close to the TeX experience? There are three possible replacement available:

1. text based formatting like lout and markdown
2. WYSIWYG like Libreoffice writer
3. newspaper layout programs like indesign and scribus

Let us start with the first category which has to do with markdown and HTML. In theory, these programs are working similar to tex. The problem is that it remains hard to convert a markdown file into the pdf format. The well known pandoc renderer isn't able to do so natively but he is using LaTeX as a backend. And rendering a HTML file into pdf will result into lower quality.

What most non TeX users are prefering to write longer documents is perhaps libreoffice writer. The current version is 7.0. But libreoffice has some problems. First one is, that after inserting more than 3 images the GUI will become very slow. The recommneded workaround is to deactivate the display of the images. Indeed this will provide the maximum performance, but then the user can't see what the figure is about.

A second problem in libreoffice is, that the images can't be scaled simple to 50%. After resizing the frame with a mouse the new size will affect the ratio and the dialog box has no ability to maintain a certain ratio. Also libreoffice doesn't supports floating images and the ability to create fully justified text is low.

On the other hand of the line there are sophisticated layout programs available like scribus. Sometimes Scribus and indesign are recommended to fulfill professional needs. But a closer look at this programs will show, that they are doing the opposite from what a technical authors needs. The main idea is that the user is creating frames on the page, but this is not what a text author likes to do. A book or an academic paper isn't defined by it's visual structure but by hierarchical sections. That means a book is working with 1. section, 1.2 as a subsection and so on. This system isn't supported in Scribus. Even the creation of a table of contents is practical impossible.

We can reduce the comparison to a small question: Can libreoffice writer replace Lyx? The answer is no. Even Libreoffice has reached a high version number 7, it is in the current version not able to process lots of images in a technical document. That means there is a reason why latex, lyx or texniccenter is available. All the programs have a measurable advantage over libreoffice. The main idea behind TeX is very easy to explain: There is somewhere a plain text file which is less than 500 kb. This text file is edited in the software of choice and after running the LaTeX software the pdf file is rendered which can contain endless amount of figures, tables and high resolution pictures. The user won't get any problems in creating such large documents nor he gets a slowdown from the software while entering the text.

Comparison between Lyx and Libreoffice for technical writing

 

The good news is that the amount of possible software for creating larger technical documents is low. Software for creating entire newspapers like Scribus and Adobe Indesign are not suited for the task. These programs have a strength in arranging photos but they aren't supporting textual writing nor formatting of bibliographies. The remaining programs which are in theory suitable to support technical authors are LaTeX, Lyx, Libreoffice and MS-Word.

So it is some sort of contrast between LaTeX on the one hand and MS-Word on the other which represents WYSIWYG software. The open question is which one is better?

The screenshot on top of this postings shows both philosophies in direct comparison. What is shown in detail is the document structure which is later rendered into a pdf file. In both cases the user enters a hierarchical text similar to an outline tool and this is converted into linear text document.

Even if the Lyx frontend looks a bit more clean, the winner of the comparison is Libreoffice Writer. The reason is that the software can be used for many additional things than only technical writing and has the larger community. In contrast, the combination of lyx and LaTeX needs very high amount of dies space and the amount of people how are using the software has become smaller than in the past.

From a historical standpoint the LaTeX software was the dominant program for academic writing Because the program was available in the 1980s in which even MS-Word wasn't available. And even today, the ability of LaTeX to create mathematical formulas is working very well.

On the other hand, with a bit o guidance, the Libreoffice Writer software or of course MS-Word can be used very well for technical authoring. All what is needed is to activate the outline / navigator mode so that the user sees in the left pane the document structure. This allows to add new sections in addition to the existing ones.

Writing high quality documents with Libreoffice

There is some sort of shared knowledge available that existing WYSIWYG text processors are not able to create academic text especially if they are longer and if the quality requirement are high. And of course the more powerful alternative is LaTeX, Lyx and other authoring tools like Framemaker.

The reason why especially the Lyx software is a great choise for creating technical documents is because it combines an outline editor for creating the content on a textual level with the powerful latex backend to render the document in a high quality PDF version.

But sometimes there is a need to create documents without latex and the question is if this is possible at all? The following blog post tries to explain how to create technical documents only with the libreoffice program which is preinstalled in the Debian operating system.

After starting the Libreoffice software the first time it will look like in the screenshot. The user sees a blank page and a blinking cursor. Also the user sees lots of icons and formatting symbols to change the font which is pershaps the main reason why experienced latex users doesn't like such a software very much. In Libreoffice there is no clear seperation between content and layout and the user can modify both in the text which makes it hard to focus only on the text.

The first step is modify the GUI a bit so that it will look like the lyx GUI. The Libreoffice program has a feature called navigator which can be compared with the outline view of lyx. the navigator shows the document structure in a side tab.

Suppose, the user has entered some text and want to change the font. In LaTeX and lyx as well the only way in doing so is to adjust the global font parameter which will affect the entire document. The Libreoffice program has a similar feature which is hidden in styles->edit style->font

Getting a preview of the pdf file is a bit complicated. First thing to do is to swtich to the normal view and then the print preview button is visible.

The main problem with the software is, that there are an endless amount of buttons and menus available. The user has to learn to ignore most of them. In contrast, latex and lyx has a more minimalistic approach. But from a tehcnical perspective, libreoffice can replace lyx very well. It has also the ability to act as an outline editor and it is also possible to render the document into a pdf file.

Let us focus on the core element of an outline editor which is the outline pane. Unfurtunately the Libreoffice tool window looks a bit messy. It contains a lot of icons and the size is fixed. This makes the interaction compared with the Lyx software difficult. But, it is not impossible to use the software. In theory the user is able to create and edit outline oriented text content in Libreoffice similar to other authoring software like emacs org mode or the scrivener app.

How to write longer technical documents

 

There is some sort of mystery available about how to create longer documents and especially academic books. On the first look the problem is easy to manage because there are word processing tools like MS-Word available plus dedicated publishing tools like Framemaker. The problem is that most authors are not interested in using a certain software but they want to know in general how to write a document.

The first thing which is important to know that writing a document has nothing to do with formatting it. Formatting a document can be realized by pressing the CTRL+A key to select the document and then select a font in the menu. So the formatting task is the easier one. The more serious question is how to create the text. This is done with a software tool called outline editor. Without an outline editor it is not possible to write longer technical documents.

Let us take a closer look how the existing programs are working. Software used for writing books are MS-Word, framemaker, libreoffice, latex, lyx and many other. What all these programs have built in is an outline editor. The interesting situation is, that this feature isn't mentioned in the handbook and most authors doesn't know what an outline editor is. But they are using this mode all the time. Let me give an example.

The famous program texniccenter is an IDE for the LaTeX program and according to the amount of documention, texniccenter is used to write papers in the reality. There is a single feature available in the program which is the ability to manage a pane on the left side. This pane can be used to use texniccenter in the outline mode. That means, the user can create notes or longer text and order them in a hierarchical fashion.

The interesting situation that other state of the art programs like MS-Word, lyx or emacs have s similar built in feature. Writing a text means basiically to type in the text into an outline editor. How this editor is called doesn't take matter. And it is also not very important if it is using LaTeX or not. The only thing which is simportant that some sort of outline editor is available. Let us take a look into a seldom mentioned software available as opensource: Cherrytree.

Cherrytree is a normal outline editor without additional formatting options. it can be seen as a tripped down version of Lyx. Lyx needs around 10 GB on the harddrive which includes the TeX back end, but cherrytree works fine with 10 MB space. According to the self description in the debian repository, “CherryTree is a hierarchical note taking application.” And yes the program is more than suited to created longer technical documents.

The interesting fact is, that Cherrytree doesn't belongs to the categories of word processing software. It is not a WYSIWYG software like Word, and it is not LaTeX clone for text based markup. At the same time, the program is the core component of a technical authoring software.

Writing documents for the Academia.edu website

 

The Academia.edu website provides lots of academic papers for downloading. But sometimes there is need to upload something. The reason is because it makes fun, because somebody can learn how to write a paper or because somebody likes to share it's knowledge. The only problem is that creating an academic paper from scratch is a bit hard. The following style guides explains the basics for newbie authors and somebody who likes to behave in such a way.

First thing which is important is, that a complicated two column layout is not the best option for the Academia.edu website. The reason is, that documents are displayed as default in the HTML mode and something with the rendered images is wrong, so the chance is high that precise formatted documents won't like great on the screen. The better idea is to use a minimalist layout which contains of a single column and a larger fontsize. Also it is recommended to deactivate any sort of kerning and fully justified text and use only normal wrapping mode from the document software. Such a layout can be realize with many ways for example, Word, markdown and LaTeX My personal favorite is the Lyx software but it depends on the user's experience.

The more serious problem is what to write in the document. Academia.edu is a scientiic website so it makes sense to choose on of the disciplines like mathematics, physics, literature or psychology as subject. Also it is recommended to reference to external sources which is a sign that the document was created for academic purposes. Writing the text itself is much harder to explain. A good workflow is to use existing content written in the past which was stored on the local harddrive and then convert it into an academic paper. If the user has not written any draft document in the past and the local harddrive doesn't contain self written notes or documents than it is not possible to write an academic paper.

 

After uploading a document to Academia.edu something interesting will happen next. In contrast to the expectation the amount of readers for academic documents is low and even zero. Some blog authors are asking themself why their traffic is so low. In Academia.edu there is no problem with traffic at all because the average traffic for a document is 0 views after 6 months. That means, the author will put a lot of effort into writing a document and then it is ignored by the world.

The problem is not located in the website itself but it has to do with academic content in general. That means academic papers in general are producing no traffic at all. Even Acadedmia.edu has an advanced tool for measuring the daily traffic the chance is high that the charts are showing nothing but zero rreadership in the world.

Understanding the line breaking algorithm in TeX

 

In the default setting, TeX is formatting a text with the fully justified mode. In contrast, the screenshot shows the opposite which is called left-justified text. The open question is how big is the difference? On the first look there are some white spaces at the right edge, but not very much. It seems that a normal line wrapping algorithm makes sure that in average the right side looks smooth. But let us take a closer look into the example.

The paragraph on the left are mostly correct formatted. That means, no visible space or only a smaller one is available at the right edge. In contrast the left page has some visible spaces. The cause was that the word was wrapped to the next line and no hyphenation or character kerning was applied. But this affects only 2-3 paragraphs and not all. So we have to ask what exactly is the advantage if the entire text gets formatted fully justified?

The interesting situation is that the situation will become more complicated if the text gets blured. If the reader will look fro the distance to the layout it is harder to see the advantage of fully justified text.

Is fully justified text really needed?

The LaTeX community claims indirectly that only fully justified text looks clean. The TeX engine tries to create such appearance really hard. Advanced hyphenation, spacing and even intra word spacing is used to make sure that each line has a straight edge on left and the right side as well.

The interesting situation is that the average reader doesn't care about this feature and a causual look at the document can't tell if it was formatted in fully justified mode or not. The example screenshot shows a document which was formatted left justified without any hyphenation. In spite of this absence of high quality formatting, the right edge looks reasonable well adjusted. Sure. It is not forming a perfect line but the document doesn't look very uncommon or even chaotic. The thesis is that the goal of creating fully justified text is maybe no longer important?

In the current example an unsharp fiilter was applied to the image so that it is no longer possible to recognize each words. Instead the reader will perceive the document from a distant view. The interesting situation is the right edge looks reasonable well formatted, even no formal fully-justified mode was applied. It seems that on the average a simple left-justified line wrapping algorithm results into a near right edge even without adjusting each line with inter word spacings. Is the TeX internal algorithm for word spacing an example for over engineering in a sense that in the reality nobody needs such formatted documents?

From a technical perspective, a left-justified document has the same appearance like a text rendered in a browser. That means, the spacing between the words is always the same and if the line is full, the word gets wrapped into the next line. This sounds a bit un-typographic but it results into an easy to read text.