The case of raggedright in the TeX community

 

It might be surprising but about the most important issue in typesetting the amount of information is little. Most books about LaTeX are explaining what typography is, and why TeX is great but they do not question the difference between left-justification and fully justified text. One of the few exceptions were made in the Tex journal Tugboat [1].

Such a debate is more general than the common question how to realize a certain layout with Latex because fully justified text and typography are strongly connected:

quote “For centuries, book printing has applied justification to nearly all paragraphs” [2]

And yes the statement is correct, a short look into the history of typography will show exactly this preference. Not only documents generated with LaTeX are looking mostly the same, but all the academic books and journals since 300 years have the same standard layout. It contains of two columns which are typeset with justified text. This produces a perfect straight edge and is described by typographers as good typography. That means, the assumption is everything which is not fully justified is equal to low quality typography.

It is impossible to find pro or cons arguments here but what is possible instead is to describe the situation as it is. It is very hard to find books and journals which are typeset with flush left layout. For example, recent issues of the PLOS One journal is doing so. Plos one is a non-traditional digital only academic journal. And perhaps this gives a hint what the idea is. Fully justified text is a sign for printed traditional documents while flush left layout is typical for online only publication.

References

[1] Are justification and hyphenation good or bad for the reader? TUGboat, Volume 37 (2016), No. 2 First results, https://tug.org/TUGboat/tb37-2/tb116akhmadeeva.pdf

[2] Udo Wermuth: An attempt at ragged-right typesetting, TUGboat, Volume 41 (2020), No. 1, https://tug.org/TUGboat/tb41-1/tb127wermuth-ragged.pdf

What Python can learn from C

 

The Python language has become famous because of its object oriented features. This was the main improvement over previous scripting languages like Perl. On the other hand Python has a seldom described features which can be used as a replacement to classes. The concept of a module is to put all the functions in a single file and include this file somewhere else.

Somebody may argue, that a class should be stored in a file and including a module is equal to include a class. The situation is more complicated. Because modules can be used without classes. This programming style is what c programmers are doing: they are creating modules in which the scope of variables and functions is limited and then the module gets included into the main program.

Apart from modules Python support also packages. A package is a directory which contains of many modules. This allows to create larger programs and very important they can be converted easily into c code. That means the original Python program doesn't use classes there for the similarity with plain C is bigger.

The real problem in programming is not to learn a certain language but the problem has how to write programs which are larger than 100 lines of code. The amount of 100 LoC fits well into a single file. It is equal to define a small number of variables and apply 4-5 functions to them. The problem is that real programs will need much more code lines and hundreds of functions. Organizing them into a single file is technically possible but it will become unlikely that the programmer is able to maintain such a file. Object oriented programming provides one possible answer to this problem, but modular programming has also an answer.

Let us describe how to write a module in Python without using classes. The idea is to create a new file and import this file in the main program. The new file contains of variables similar to class variables and methods which can modify these variables. Some of the functions can be accessed from the outside so that the module communicates with the main program. In contrast to the C language the workflow was simplified drastically, there is no need to create dedicated header files and the program will run without a compilation step.

Comparison between flush left and justified text

 

The interesting situation is that there is no real debate which of the formatting styles is better. Other problems in typography can be discussed but everybody knows that it is offtopic to question the idea of justified text. The reason is that the problem of arranging words on a line is seen as equal to typography in general. That means typography is about justified text.

Nearly all the professional books and journals in the last 400 years were printed with justified text. This sounds incredible because in the past no software was used but mechanical typesetting machines. But it seems that the machine experts in the past have solved the problem of word justification over a line. The standard mode of the monotype typesetting machine was that it has procuded justified text and nothing else. The only question which was asked is how to realize this formatting style.

From a technical perspective it is possible to create left justified text with the monotype typesetting machine. And technically, the LaTeX software can do the same for an academic paper. But in the reality nobody is doing so.

To understand the situation we have to go back 100 years and take a look what a human operator at the monotype machine was doing. He gets a manuscript which was either written by a typewriting machine or by hand. And the objective is to convert this manuscript into a justified text stored on a punch tape. The idea behind the monotype machine is to justify the text. This is realized by counting the words, and measuring the spaces, and then the amount of free space is determined. This procedure is repeated for every line.

The interesting fact is the formatting style “Justification” is only one possible formatting style. The text can be read also very well if the step is missing. So why is this step done? Mostly because of tradition. THe idea is that the text is so important and will reach so many readers that the typesetting has to be made with the highest possible quality. And this is equal to fully justified formatting.

This self understanding sounds similar to how books are created before the advent of printing machines. The idea was, that the content of a book is so important that it has to be copied by hand. Creating a copy by hand takes longer and this shows the reverence for the content. Bascially spoken, creating a copy of book by hand is more costly than using a printing press. And creating fully justified text is more expensive than creating only left-justified paragraphs.

Left justified text means basically to lower the quality of typography. Let us make simple calculation. What will happen if the monotype machine is used to create left justified text? Right the duration for typesetting the document is shorter. The human operator needs to calculate less and he can typeset more manuscripts in the same time.

Justified text in comparison

To investigate the mystery around justified text, the LaTeX software to render three possible formatting styles: flush left, flush left with hyphenation and fully justified text. The third picture was created with the microtype package to minimize the white spaces in a line. The open question is which of the examples looks better?

What we can say for sure is that fully justified paragraph stands in tradition with typography in general. Most books and academic papers from the past were typeset in this formatting style. While using a flush left formatting is a bit unusual. The interesting situation is that all three examples need 11 lines to show the content. That means even activated hyphenation won't safe space that much.

A second fact is that thanks to the LaTeX internal algorithm the example with the justified text looks very good. That means, the white spaces between the words are not visible and the text looks well formatted. Nevertheless the difference between flush left text and justified text is small. It is mostly a question of personal preference. And most texts in the internet and on smartphone displays are formatted in the flush left mode except pdf documents which are mostly formatted in justified style.

The perhaps most intersesting question is if flush left paragraph formatting is something which has to be fixed. Or let me ask it different. Is there a need for firefox and other browsers to display text justified or is the existing flush left rendering future ready? The situation for screen layout is the opposite as for printed layout. Screen formatting was from the beginning working with the flush left mode only. It is ver

In theory the CSS Specification has the ability to render a text in justified mode, but nobody is using this preference.

Belief systems in typography

 

The community of computer users is famous for its debate about the pros and cons of software, programming languages and operating systems. A less researched area for creating opposite oppinions is digital typography which is equal to the LaTeX software. The problem with typography is that the domain is very complex and it is difficult to find a single discussion topic which will polarize the opinion of the users.

A highly visible conflict is between the LaTeX community and non latex users. The LaTeX community understands themself as experts for typography and critizes heavily the alternative software which is MS-Word and indesign. MS Word can't be used for creating professional documents and indesign is only a WYSIWYG software which means that lots of mouse clicks are needed before the document was created. If a user has decided for the LaTeX software he will create as default high quality documents, this is at least the assumption.

There is a need to analyze the art of typography in detail to understand that the same software can be used right and wrong. Typography consists of many sub problems, like micro typography, macro typography, fonts, bibliography, creation of tables and so on. The question is which sort of setting is recommended? Is a document which was typeset in a 10pt font easier to read than a document which was typeset in a 11pt font? Not always, it depends on individual preferences. Therefor there is no conflict about this issue. The same non conflict is available about the margin space. In reality, documents are created which have a small margin and a big margin as well.

But there is one problem not discussed frequently which has the potential to divide the community. It is the problem of flush left paragraph vs justified paragraph. From a technical perspective this problem is easy to master. Every word processor has the ability to format a text in both style. In MS-Word there is button available to change the formatting and LaTeX knows the regged2d package for doing so. After applying the command, the paragraph will either have a straight right edge or it will be formatted in flush left.

The reason why this formatting style has a high conflict potential is because in the world of professional publishing there is a clear dogmatic rule available that only justified text is allowed. This rule is formulated in the guidelines for submitting documents to a conference proceeding and it is visible in existing documents. That means, everybody, which means really everybody is following this rule in theory and practice.

Let us make a short survey to analyze the current situation. We are taking a look into randomly selected books, newspapers, conference proceedings and academic journals. And we are enlarging the time period from the year 2010 to 1960s, and even to 1900. The surprising discovery is that all the documents were typeset with fully-justified formatting. Calling this perception a typographic rule is an understatement it has more in common with a dogma.

The reason why is hard to investigate. An often formulated argument is that a justified text looks more elegant. Another less mentioned reason is that otherwise the mechanical printing press will explode because of the imbalance of the right edge. That means professional typographers can be highly irrational in this point. The printing press is rotating very fast, and if the right edge is creating zig zag pattern, the rotating process is disturbed.

Nevertheless the dominant reason why fully justified text is prefered for professional publications like newspapers and journals is because of historical reasons. Since 200 years and longer, all the journals were typeset in this style. The font has changed, the font size can be adjusted and the image quality differs but every text was typeset in the fully justified mode.. This formatting style is so common that it is often identified with typography in general. That means, a fully justified text is equal to good typography and a flush left is equal to its absence.

Perhaps it should be mentioned that in the digital world the situation is the other way around. Most HTML pages are using the flush left rendering. From a technical perspective, a HTML page has no page width but the window's size can be adapted. And nobody has complained about it.

Understanding the beauty of TeX

 

The TeX typesetting system is one of the largest and important programs in the history of the Unix operating system. The first electronic generated documents ever were created in the last 1970s with the TeX software and until now all the scientific journals are created with this software only. From an abstract perspective it is difficult to understand why LaTeX is so popular. The simple reason is that it has the build in feature of creating justified text.

To understand why this feature is important we have to go back in the history of typesetting. The monotype mechanical typesetting machine was invented around 1900. It contains of two parts. The desk for creating the punched tape, and the casting machine which takes the punched tape as input and produces the hot metal matrices. The interesting situation is, that even the monotype machine was able to create fully justified text. Workers who have used the machines 100 years ago can confirm this statement and very important, newspapers and journals from that area were typeset in this format.

How can it be that a magazine 100 years ago was using fully justified text? Isn't it really complicated to stretch and compress a line of word this way? Yes it is complicated, but the monotype machine was using this feature all the time. It was the main difference between ordinary typewriter produced manuscripts and professional created journals.

The ability to typeset fully justified text was re implemented by the TeX software later in the 1970s. Similar to the monotype machine, the tex software is using this formatting style as default. The assumption is that the needed output of the typesetting process is always the fully justified text. This workflow is described as beautiful typesetting. In contrast text which is only left justified is seen as the opposite.

Why is justified text better than flush text? The main reason is perhaps because the first category is harder to realize. Creating a text in this way requires more effort than producing ordinary text. It seems that this extra effort has become a tradition.

Let us analyze the typesetting process around 1900. The input for the human typesetter was a manuscript created with a mechanical typewriter. Like all manuscripts such a document was using a flush left alignment in a single column mode. Because such an output is easier to create for a typewriter. The main task of a monotype operator is to convert this input document into a fully justified description on the punched tape. That means, the operator beautifies the manuscript by realizing the justified mode.

In summary the monotype machine from 1900 was able to create fully justified text in multicolumn layout. This combination was needed for academic journals, books and newspapers as well. That means justified text is not an invention from the computer age, but it was common in the time of mechanical typesetting.

In contrast, the widespread usage of left justified text is a symbol of the digital age. The internet was the first medium in the history which was using this formatting style everywhere. The reason was, that before the internet was invented no other public medium like printed journals or newspaper were using left justified text. That means, the justified text which is more complicated to realize was invented much earlier and the flush left text which is easier to realize is a new thing available since the 1992.

Typography without justified alignment

 

The LaTeX typesetting system has become famous because of the high quality rendering of text. What most newbies doesn't know is that the latex community is using the term typography with a certain bias. The assumption is that good typography is equal to justified text. Justification means the opposite of flush left. It means, that the right edge is straight from top to bottom.

A short look into the history of mechanical book printing shows that the latex community is arguing with the same assumption like previous technicians. What the first gutenberg printing press, the linotype and the intertype photosetter have in common is the ability to produce fully justified paragraph. The goal of former typesetters wasn't only to print something on a paper but the goal was to print it with the fully justified layout.

It is a seldom described fact, but all the academic journals and newspapers in the last 200 were printed in the fully justified mode. This might be surprising because before the advent of desktop publishing only hot metal typesetting was available. That means there was a need for manual intervention to create a straight right edge.

It remains unclear why this unwritten rule in typography is valid for such along period. Perhaps it has to do with western tradition. The assumption is good typography is equal to justified text and this is the paradigm behind the latex software as well.

From a technical perspective all the printing device like the linotype machine and modern word processing software is able to create left flushed paragraph easily. In most cases the algorithm for doing so is much easier to realize. Because at the end of the line the last word gets wrapped to the next line and no in word adjustments are needed. But the interesting situation is that for final printings this ability is never used. Especially for academic publication there is an unwritten rule available that the text needs to be fully justified. The question is why?

Sometimes the argument is provided that a fully justified text can be read faster. Another argument is that the layout looks better because the columns can be recognized. But in most cases the real argument is that fully justified text has a long tradition and therefor new typesetting has to follow this rule too. Basically spoken it is not a need but it is personal preference why professional publishing is equal to fully justified formatting.

Since the internet age the situation has changed drastically. 99% of the web-rendered websites are using the flush left layout. All the browsers which includes lyx, firefox and the internet exploring are rendering text in this way. Similar to the situation with justfied text in a printed world, the main reason why all the browser are using flush-left rendering is because everybody else is doing so. That means in the internet world there is an unwriten rule that a paragraph needs to be displayed that way. The interesting fact is that the argument is the same like for the book printing world. The reason why firefox is using flush left rendering is because such a text can be read faster and it looks better.

The unsolved question is what is the best formatting style in an electronic pdf document? Nobody knows, a pdf document is using the us-letter format which is working different from a website. Technically it is possible to use justified and flush left formatting as well. So it is an individual decision how the typesetting is made. But according to the produced documents the amount of justified pdf papers is higher. A short look into the arxiv repository will show that 99% of the papers are fully justified. Especially the academic community beliefs that this unwritten rule is valid. A document formatted only with flush left is common for draft documents in which improvements are needed before the final publication.

Mass printing

What newspapers, books and academic journals have in common is that the amount of printed copies is high. A single issue of a newspaper gets printed many thousands times. This high circulation makes it economical useful to make the typography very accurate. It makes sense to use the Linotype printing machine to produce fully justified text. That means the human typesetter will need many hours until the layout is ready and nobody cares because many thousands people will read the result.

In contrast, manuscripts which are circulating with low amount of copies are created with simpler machines like a mechanical typewriter. A typewriter has no advanced featues like a fully justified text. Therefor normal manuscripts are flush left.

With the upraising of digital printing the situation has become softer. IN most programs it is only a simple click with the mouse to change the setting from justified to lush left. What we can say for sure is that in the past a justified text was equal to high amount of copies. Because a justified text was created by a professional typesetter and this can only be realized if a larger audience is interested in a book or journal.

Reducing the LaTeX software to the core

 

The current tex-llve package is not only a piece of software but it is a software distribution which contains of millions of lines of code which are distributed over endless amount of packages and commands. The newbie has make sure that around 5GB are available on the harddrive so it seems, that desktop pubishing with LaTeX is a demanding task. To understand the latex system better we have to reduce all the programs to the minimum. The question is what is the core idea behind LaTeX?

The surprising answer is that LaTeX has such a core idea. It is about typesetting of justified text. That means the main ability of tex is the combination of the word wrap algorithm, plus hyphenation and very important the microtype package. All these things make sure that the right border of a column is straight from top to down.

All the other typographic elements of LaTeX for example the ability to manage different fonts, referencing of literature, typesetting mathematical formulas and so on is only a gimmick on top of the paragraph justification algorithm.

Realizing such algorithm in software is a complicated task. Especially the microtype text which adjusts even the spaces in a word itself is an example for an elegant software. The generated output of justified text has a better quality than what Libreoffice has to offer and it is the main reason why LaTeX is used everywhere.

The interestion situation is that without a justified text the latex generated pdf document will look not very impressive. It has the same or even a lower quality than the libreoffice counterpart. So the question is why is the abilty to typeset justified text so important? Because it is assumed that this technical trick will improve the impression for the reader drasticaly. He will see on the first look that the document was typeset professional. In the time before the advent of digital typography it was very hard to produce justified text and with modern word processors the task remains a challenge. Only LaTeX and professional software like Indesign is able to do so while normal word processing software including pandoc, asciitops and emacs can't do so.

The situation will become more obvious if someone tries to program an algorithm from scratch which creates justified text. Creating a simple word wrapping algorithm can be realized in under 100 lines of code, but this won't produce a correct right edge. To stretch each character similar to the microtype package, an advanced software is needed which includes the ability to handle different fonts. This is an advanced typesetting problem and latex is an expert for this task.

There is less common package available in the texlive distributio which is ragged2d. This package produces a left justified text similar to what Libreoffice is doing. Activating this package is from a technical perspective trivial but it will generate a pdf document which looks very different from a conference proceeding and an academic journal.