Symmetric typesetting with LaTeX

On the first look the LaTeX ecosystem looks highly complex. There are an endless amount of packages, tutorials and guidelines available. The surprising fact is that it is possible to reduce the idea behind latex to a single screenshot. What the LaTeX community tries to achive is shown in the top of the image. The typset paragraph looks similar to a poem. The headline is formatted with centering and the paragraph is fully justified. The justification was created by adjusting the glue between the words, plus minor adjustments of the microtype package.

In contrast, the picture at the bottom shows what the TeX community tries to prevent. In such a situation the symmetry is missing because everything was formatted with flush left.

The interesting situation is that this preference has nothing to do with the LaTeX software but with typography in general. So we have to understand why formatting something by centering it is perceived as beautiful. The idea is that the written word follows the principle of art. It is not only a book but it is a graphical creation. Or to be more specific, the reader of the text should get the impression. From an authors perspective it is trivial to format something with the centering mode.

Let us take a closer look into the shown paragraph. It is the same text, the same font and the software for rendering it was in both cases the latest version of the lualatex engine. The only difference is that in the second case the formatting style “flush left” was applied. Now the question to answer is which of the cases has a higher score in terms of typographic quality? The simple answer is that the top example gets the quality judgement “beautiful example for typesetting”, while the bottom case gets the judgment “low quality or even no typographic quality at all”.

This judgment is not based on any objective criteria but it is simple a preference for or against flush left. How can it be that the established latex software including the high quality Latin Modern Roman Font can create low quality typesetting? Because the untold assumption is that only centering text which includes the headline and the paragraph is beautiful and everything else is wrong.

The underlying reason why this rule is valid is because it takes more effort to create full justified text. If the text isn't created with an algorithm but with a hot metal printing press it will take endless amount of time to fully justify the text. What the typesetter has to do is determine the size of the words and calculate how many glue is needed. In contrast, putting the letters for the bottom example together can be realized much faster.

Comparison between flush left and justified text

 

The interesting situation is that there is no real debate which of the formatting styles is better. Other problems in typography can be discussed but everybody knows that it is offtopic to question the idea of justified text. The reason is that the problem of arranging words on a line is seen as equal to typography in general. That means typography is about justified text.

Nearly all the professional books and journals in the last 400 years were printed with justified text. This sounds incredible because in the past no software was used but mechanical typesetting machines. But it seems that the machine experts in the past have solved the problem of word justification over a line. The standard mode of the monotype typesetting machine was that it has procuded justified text and nothing else. The only question which was asked is how to realize this formatting style.

From a technical perspective it is possible to create left justified text with the monotype typesetting machine. And technically, the LaTeX software can do the same for an academic paper. But in the reality nobody is doing so.

To understand the situation we have to go back 100 years and take a look what a human operator at the monotype machine was doing. He gets a manuscript which was either written by a typewriting machine or by hand. And the objective is to convert this manuscript into a justified text stored on a punch tape. The idea behind the monotype machine is to justify the text. This is realized by counting the words, and measuring the spaces, and then the amount of free space is determined. This procedure is repeated for every line.

The interesting fact is the formatting style “Justification” is only one possible formatting style. The text can be read also very well if the step is missing. So why is this step done? Mostly because of tradition. THe idea is that the text is so important and will reach so many readers that the typesetting has to be made with the highest possible quality. And this is equal to fully justified formatting.

This self understanding sounds similar to how books are created before the advent of printing machines. The idea was, that the content of a book is so important that it has to be copied by hand. Creating a copy by hand takes longer and this shows the reverence for the content. Bascially spoken, creating a copy of book by hand is more costly than using a printing press. And creating fully justified text is more expensive than creating only left-justified paragraphs.

Left justified text means basically to lower the quality of typography. Let us make simple calculation. What will happen if the monotype machine is used to create left justified text? Right the duration for typesetting the document is shorter. The human operator needs to calculate less and he can typeset more manuscripts in the same time.

Belief systems in typography

 

The community of computer users is famous for its debate about the pros and cons of software, programming languages and operating systems. A less researched area for creating opposite oppinions is digital typography which is equal to the LaTeX software. The problem with typography is that the domain is very complex and it is difficult to find a single discussion topic which will polarize the opinion of the users.

A highly visible conflict is between the LaTeX community and non latex users. The LaTeX community understands themself as experts for typography and critizes heavily the alternative software which is MS-Word and indesign. MS Word can't be used for creating professional documents and indesign is only a WYSIWYG software which means that lots of mouse clicks are needed before the document was created. If a user has decided for the LaTeX software he will create as default high quality documents, this is at least the assumption.

There is a need to analyze the art of typography in detail to understand that the same software can be used right and wrong. Typography consists of many sub problems, like micro typography, macro typography, fonts, bibliography, creation of tables and so on. The question is which sort of setting is recommended? Is a document which was typeset in a 10pt font easier to read than a document which was typeset in a 11pt font? Not always, it depends on individual preferences. Therefor there is no conflict about this issue. The same non conflict is available about the margin space. In reality, documents are created which have a small margin and a big margin as well.

But there is one problem not discussed frequently which has the potential to divide the community. It is the problem of flush left paragraph vs justified paragraph. From a technical perspective this problem is easy to master. Every word processor has the ability to format a text in both style. In MS-Word there is button available to change the formatting and LaTeX knows the regged2d package for doing so. After applying the command, the paragraph will either have a straight right edge or it will be formatted in flush left.

The reason why this formatting style has a high conflict potential is because in the world of professional publishing there is a clear dogmatic rule available that only justified text is allowed. This rule is formulated in the guidelines for submitting documents to a conference proceeding and it is visible in existing documents. That means, everybody, which means really everybody is following this rule in theory and practice.

Let us make a short survey to analyze the current situation. We are taking a look into randomly selected books, newspapers, conference proceedings and academic journals. And we are enlarging the time period from the year 2010 to 1960s, and even to 1900. The surprising discovery is that all the documents were typeset with fully-justified formatting. Calling this perception a typographic rule is an understatement it has more in common with a dogma.

The reason why is hard to investigate. An often formulated argument is that a justified text looks more elegant. Another less mentioned reason is that otherwise the mechanical printing press will explode because of the imbalance of the right edge. That means professional typographers can be highly irrational in this point. The printing press is rotating very fast, and if the right edge is creating zig zag pattern, the rotating process is disturbed.

Nevertheless the dominant reason why fully justified text is prefered for professional publications like newspapers and journals is because of historical reasons. Since 200 years and longer, all the journals were typeset in this style. The font has changed, the font size can be adjusted and the image quality differs but every text was typeset in the fully justified mode.. This formatting style is so common that it is often identified with typography in general. That means, a fully justified text is equal to good typography and a flush left is equal to its absence.

Perhaps it should be mentioned that in the digital world the situation is the other way around. Most HTML pages are using the flush left rendering. From a technical perspective, a HTML page has no page width but the window's size can be adapted. And nobody has complained about it.

The sile typesetting system

 

A less known possible replacement for LaTeX is Sile. For a one to one comparison the sourcecode can be fetched and then the lines of code are counted:

git clone --depth 1 https://github.com/sile-typesetter/sile.git

cloc sile

No matter what the original documentation of the project likes to explain the cloc tool is an objective measurement what the project is about. According to the statistics, Sile was programmed was mainly programmed in Lua with some parts in C++. The overall lines of code is 450k. Now we can compare this number with the well known luatex project. CLOC will show for luatex that it contains of 1 million llines of code. So we can say that Silo has the half size of luatex.

Unfurtunately, the amount of 450k lines of code is similar to the original luatex project not a small github code repository but a larger one. Maintaining all the code will need a lot of work. Only to get a better understanding what the size is. One line of code needs around 40 Byte. Suppose the code is converted 1:1 into a binary file than 1 million line of code is equal to 40 MB for the binary file. So we are speaking about a 20 MB executable file for the sile project and about a 40 MB file for the luatex project.

Now it is possible to analyze the reason why these layout engines are so big. What Sile, luatex and possible future latex replacements have in common is that they are implementing commands for typessetting. Even if the vocabulary of all the latex commands was never standardized, there is some sort of list available which latex commands are used frequently. In this list around 250 different commands are available which includes commands for typesetting tables, mathematics and showing images in the document.

An well programmed layout engine has to implement these 250 commands at minimum. In case of sile the syntax is a bit different but this is only a detail problem. The open question is how to program a layout engine which takes the 250 possible commands as input and produdes a pdf file as output?

This is realized with lots of lines of code:

Sile, 450k lines of code 250 latex commands -> 1800 lines of code for a single command

Luatex, 1000k lines of code for 250 latex commands -> 4000 lines of code for a single command

So the only difference is how much LoC are needed for a single command per average. From a technical side it is possible to implement only 10 commands, but such a layout engine is less powerful than the original tex software. That means, even if someone reinvents latex he has to implement all the 250 commands.

Word processing on the computer

Around the year 1980 a technology revolution took place, namely the invention of the first word processing machines. In the beginning, they were standalone apparatus. It was a normal typewriter equipped with computer memory, but after a short time, the normal PC was used as a wordprocessing machine and the capability to edit text was realized by a software program.

The transition from mechanical typewriters to word processing on the computer was a large step in productivity. The remarkable thing is, that a computer can be reduced in it's functionality to as word processing only machine. That means, it's possible to use the IBM PC for nothing else than only typing in letters. The advent of computers for business and private users was motivated by the word processing capabilities. The advantage of a computerized word processor over a mechanical typewriter is huge. The most important feature is, that the text can be stored in the memory and reproduced many times. For today's computer users this kind of feature is normal, but in the early 1980s it was a revolution. It forces all the companies to throw away their old mechanical typewriters and buy millions of PCs from computer manufactors. The interesting point is, that even untrained users are using the PC as a word processor. It's the first application somebody discovers on the PC and understanding how a modern text software is working is very easy.

To understand the impact of the revolution we have to take a look how texts were written before the 1980s. All the decades before it was done with mechanical typewriters and before the year 1900 it was done with manual pen. That means, all the books and publications created before the 1980s were not typed in at the PC but there were produced mostly on mechanical typewriters. This effects the content of the text. If somebody is using a computer to type in a book the sentences are different if somebody is using a mechanical typewriting machine.

From a social perspective wordprocessing is not only a computer driven technology, but it's an educational device. If somebody learns to use the computer for writing texts, he learns to write texts in general. Or let me explain it from a different point of view. It's not possible that somebody learns a language without using tools like pen, typewriters and computers. As a result, the invention of the first word processing computers were equal to a global education program to teach the art of writing.