From Treepad to Zimwiki – a small history of outliner software

The history of outliner software is a seldom explored subject within computer science. Early examples were created in the late 1980s as hypertext editors. A notable example was the Symbolics Concordia software which has much in common with the later Windows 3.1 help files. The idea is that the user creates a hierarchical tree and can insert hyperlinks for easier navigation. [1]

Sure from a technical perspective, there was the hypercard program which was published for the Apple computer earlier. The problem with this attempt was, that in the early 1980s the homecomputers were not equipped with harddrives. For creating a larger hypertext document is harddrive is needed because the filesize will become 500 kb and more. So it makes sense to assume that the first examples of working hypertext authoring tools were created in the late 1980s on dedicated workstation computers.[4]

In the year 1995 Henk Hagedoorn created for the Windows operating system the Treepad software.[2] It is the most important software in the history of outliner programs. It was the first mainstream program which was using a two pane editor. Left the user sees the structure of the document and in the right pane the full text is shown.

From a more critical perspective, the Treepad software looks like MS-Word in which the document outline was activated. So it is basically a word processing software with a focus on the structure of the document.

After Treepad was created many clones were programmed like Keynote, cherrytree and zimwiki. Technical the difference is low or even not there. The GUI has the same two pane structure and the only improvement of the keynote program was, that it was released as open source software.

An interesting but niche software are zkn3 and synapsen. These tools were created with the attempt to emulate a Niklas Luhmann Zettelkasten. They are used by scientists for maintaining annotated bibliographies. [3]

Another more recent approach is the Obsidian software which combines a chaotic graph structure with a powerful search feature.

There is a seldom analyzed common feature between the early outliner programs and more recent note taking apps. All these programs are using a key value structure for storing information. The key is shown in the left pane which is the name of a file or the name of a section. In the right pane the value is shown which is equal to a free form text which includes images and hyperlinks.

Sources

[1] Nielsen, Jakob. Multimedia and hypertext: The Internet and beyond. Morgan Kaufmann, 1995.

[2] Interview: Henk Hagedoorn, creator of TreePad http://mediaman.com.au/interviews/hagedoorn.html

[3] Markus Krajewski synapsen – A hypertextual card index User’s Manual http://www.verzetteln.de/synapsen/_docs/SynManual.pdf

[4] Computer chronicles Personal Information Management, 1989

Creating a minimal outliner in Linux

 

For creating a 1dArrayoutliner a first mockup is available.It was realized in 200 lines of code in the python language and shows the main window for an outliner. It is using a two pane layout. The user can select in the left menu the page and will see in the right pane the content of a page.

The page content is rendered with a self written parser. A pushdown automaton analyzes a markdown file and translates it into a rich text window. Right now, the GUI framework is wxpython but it seems that it is not powerful enough. So perhaps it makes sense to switch to tkinter?

The bottleneck of the current project is to save the information from the text editor into a markdown file. To realize it the text editor needs the abilty to save something as markdown, or the program needs to parse the information in the window manual. The task is needed because images and text are shown at the same time in the window.

That means, writing a text only outlier program is pretty easy, but if the images are shown inside th text it is much harder to program it.

 

Perhaps it makes sense to explain why there is need to program yet another outline editor. Because most of the programs from the past are very big projects It seems, that the average outliner / PKM software has at least a size of 1 MB and there is no upper limit so that current software has 200 MB and more.

The idea is to reduce the requirement to a minimum and write a much smaller outliner program which has 100 kb and less. For doing so there is need to specifiy which features should be implemented and which not. From a database persective an outliner has to implement the CRUD operations which stands for create, retrieve, update and delete. The interesting situation is, that more actions are not needed, if the user can add new pages and modify the existing he is happy. Editing a page means usually to enter text in the richt text window which includes adding images. I think it is not possible to ignore images and assume that a text only outliner will fulfill the needs of a user.

 

https://github.com/ManuelRodriguez331/1dArrayOutliner 

Review of the new Version of the Zim wiki

 

The latest version of the Zim desktop wiki is 0.74.3 which is not available in Debian. But the user can download and install the package manual with the dpkg tool. The first impression of the software is very good. The programmers have removed the toolbar, so that it is no longer possible to format something in bold Also the search box was removed The user needs an additional click if he likes to find something.

Some things remain unchanged. The rendering of the link map takes the same huge time like in the stable version. The user is pressing the button, has to wait for 20 seconds, then a popup menu from gnome asks to quit the application at all and then after a while, a messy chart of the nodes is drawn.

One cause of the slow speed is for sure the underlying programming language. Python is known as a beginner friendly programming language which is not recommended for productive software. So it is some sort of luck, if the user doesn't loose all the important information, if the program is crashing.

A closer look into the source code will show, that the Richtext WYSIWYG editor was realized in 600 kb of Python code. Most of the code was written a while ago in the Python language and never changed. This speaks for the zim project. There is no need for using the C++ language or creating a more efficient GUI because this would need too much manpower which would be missing somewhere else, outside the zimwiki project.

Overall the zim wiki is a great improvement over the previous version 0.73.5. If the end user is not satisfied with existing Outliner tools, he will dislike the zimwiki program as well. It is a pleasure to see, that even an ArchLinux package file is provided. This Linux distribution is a great replacement for Debian and other outdated projects because ArchLinux has always the newest version of a program ...

One bug should be mentioned at the end. In contrast to the previous version, the program is asking the user if he really likes to delete a certain page in the list.

A short history of outliner programs

 

Outliner programs are a seldom analyzed software category. On the same time, lots of these programs are available. So it seems that outliner are interesting for a larger amount of users. What makes the situation difficult to explain is the huge amount of different software programs. A rough estimation is that at least 300 different software programs were programmed since the 1980s and the amount is growing.

Let us explain the situation from a chronological perspective. The good news is, that over the decades only a few larger applications were popular. The 1980s was the decaded of the MS-DOS operating system. Outliner programs like Asksam, Grandview and MORE for the Apple system were created. The interesting situation is, that for the 1980s these programs had a large size. The more software was around 200 kb on a floppy disc. An outliner program was similar to a database and a word processing a complex application. There was even law case started around the PC-Outline program which looks similar to the grandview software from Symantec.

In the 1990s, nearly all the existing MS-DOS based outliner programs lost it's marketshare. Instead new applications like Treepad and Myinfo were programmed. This pattern is visible in the 2000s as well. New software was programmed from scratch like Notecase, Onenote and keynote. It seems, that the avarage lifespan of an outliner program is lower than 10 years.

Even larger software companies like Microsoft are not able to dominate the market. The Onenote software for example is not the market leader but stands in competition to evernote obsidian and of course the older outliners from the 2000s.

If so many different outliner programs are available it should be estimated that it is clear what an outliner is. This is not the case. Most programs are defining the software category slightly different. Some programs are referencing itself as a two pane outliner program, while other are prefering the term PIM or knowledge base. Other programs are calling themself note-taking app.

From a programming perspective an outliner is a combination between a text editor and a file browser. In the manual of the first popular Windows based outliner “treepad” there was a good description given. Treepad is according to the self understanding a combination of the Windows file explorer with the notepad application. That means, an outliner can be replaced by these single tools. In a minimal case it is equal to use the Linux command line for showing a directory and use a text editor to change the content of one of the files. The outliner program simplifies this task and in most cases an additional full text search engine is provided.

Let us take this explanation as a starting point to get a better understanding what an outliner is about. All the operating systems like Windows, unix and Apple have a directory structure on the harddrive. The user can create folders and store text files in these folders. Such a structure is some sort of knowledge base and most user will store information this way. An outliner program makes this use case more explicit and provides a well looking GUI interface which allows to create more complex folder structures.

Even if around 300 different outliner programs are available, most of these programs are working completely different from each other. So it makes sense that so many software was created. What the programmers have done in the last 40 years is to experiment with different GUI layout and program features. For example, in the years of the grandview and asksam, a left pane was missing. Most of the early outliner programs are working with a single pane window. Since the mentioned treepad program from 1995 the situation has changed. Nearly all the programs since the year 2000 are working with a two pane layout. Some programs have more features while other are very minimalist.

Under the LInux operating system there are text only outliner plugins available for the VIM and the emacs editor. These tools are not able to render images because the assumption is / was that images are not needed.

One possible reason why outliner are a seldom explained software category is, that most users think they have no need for such tools. They are using either the folder structure of the operating system or they are storing the outline into a MS-Word document. And indeed, MS-Word can show on the left side an outline window which allows to navigate in larger documents easily.

On the other hand, lots of users are not satisfied with a simple Word document. They have a need for a dedicated note taking app.

A minimalist outliner software

 

The amount of different outliner programs is huge. Around 400 software projects were written in the last 30 years. The interesting situation is that most of these programs are very huge in terms of codesize. Even the minmalist cherrytree program which is available as Debian package has a footprint of 650 kb. More complex project like the Zettlr app needs around 75 MB on the harddrive.

According to the famous outliner forum, the smallest possible outliner today are Mempad (210 kb), Theguide (1 MB) and Treepad light (450 kb). Especially the treepad light software is the unofficial champion in outliners because it was created very early and was available over many years. So it seems, that even lightweight outliners have a size of around 200 kb upto 600 kb.

This codesize can't be called lightweight. Creating a 500 kb softwareproject can't be realized in a short amount of time. So the dominant question is how to reduce the code size. Is it possible to write an outliner in 10% of the treepad light size which is 45 kb and less? This is perhaps the core question for modern outliner software. What most outliners available have in common is, that they are the opposite of a small software.

It is unclear how big a project like OneNote or scrivener is, but a rough estimation is, that such projects will occupy 10 MB and more on the harddrive. It is a typical example for a typical heavyweight PC software which is created in thousands and even million of codelines.

[1] lightweight 2-pane outliner for breaking up text quickly https://www.outlinersoftware.com/topics/viewt/4707/

Short history of outliner software

A while ago, somebody has created a list of outliner software. it was posted into the outliner forum in the year 2008 and the list is really long. Large scale software projects like asksam, myinfo and Onenote were mentioned. But some more recent projects like Obsidian and Citavi are missing. Also some smaller Open source projects like Cherrytree are not in the list yet.

But let us slow down the situation a bit and start from the beginning. The perhaps most popular outliner like software from the 1980s was Asksam which was an MS DOS application. The idea behind asksam was to create a full text database on the local floppy drive and then do a full text search in the database. This concept was new in the 1980 because it was the first time that larger amount of data were searchable.

In the 1990s more outliner programs were created notable IBM Lotus notes. The asksam project was stopped and the possible alternative is myinfo. In addition, lots of shareware and opensource outliner programs were created. The sstate of the software is working with a two pane window which allows similar to asksam to retrieve the database in fulltext.

The reason why so many different programs are available which are of course completely incompatible to each other is because the concept of an outliner software makes sense for many PC users. It seems, that each of the 3 billion computer users in the world has a need for an individual outliner tool.

Some popular projects in the recent years are Microsoft onenote and Obsidian. Similar to the first outliner software in the 1980s the idea is to structure knowledge in a database. Another popular software which is used mostly by programmers is the Emacs org-mode which is some sort of outliner for the emacs editor.

Let us try to investigate the subject in detail. There is a difference between a programmer IDE like eclipse and an outliner software like myinfo. The average outliner is able to edit many files at the same time. In a text editor, the user opens a single file and makes some changes. While in an outliner program, hundreds of files are opened in parallel in the left pane. In addition, the average outliner supports images and attachments. In contrast, a text editor used for programming software isn't doing so.

Perhaps it makes sense to take a closer look at the myinfo project, because it is a mature outliner software which includes all the important features. The main feature of the software is, that it can be used to create a structured text document. In the left pane, the user sees the outline which is the table of content of the text while the right pane shows the content. The text contains of prose text, bullet lists and very important images and tables. In addition, the user can do a full text search in the content so that he can find older entries easily. This is – in short – the basic principle of an outliner software.

The Obsidian software has modified the principle a bit. Obsidian is a graphical mind map editor. The standard layout is different from a word processor, but Obsidian shows a graph of connected nodes. The ability to link from one entry to another is very important.

What we can say for sure is, that the amount of outliner software will grow in the future. It seems, that there is a need to program different sort of software. A single tool which is used as the default outliner is not available. Even the large Microsoft Onenote software wasn't able yet to reach a larger audience.

A very interesting but seldom mentioned software is AZZ cardfile. The dominant difference to other outliner programs is, that AZZ cardfile doesn't support nested cards. There is no ability to create folders and populate them with subfolders, but all the cards are on the same level. This principle emulates a classical paper based notebox and it is different from a directory structure used in computing. The AZZ cardfile software was created as an improved cardfile.exe software which was preinstalled in the Win3.1 software. cardfile.exe was a very basic index card software.

Timeline

Freebyte Treepad, 1995-2019

Notecase Pro 2011-today

Notecase 2005-2008

AZZ cardfile 2001-today

asksam 1985-2008

Inforapid 1997-today

Onenote 2003-today

The majority of the programs were created from year 2000 until today. Software projects like Inforapid, Treepad were programmed mostly for the Windows operating system. Older examples for outliners like Grandview and Asksam were available in the 1980s for MS-DOS PC but they were never very popular or powerful. The interesting situation is, that most projects are promising to reinvent the idea. The user is explained what a personal knowledge system is about. But from a technical side all the programs are working with the same two pane outliner principle. In the left pane the outline is shown while in the right window the full text gets entered. Even more recent projects which are programmed from scratch are working with this principle.

The perhaps most dominant concern for the end user is, that the file format is not compatible to other programs and if the software is no longer maintained the user can't switch to a newer program.

 

References

[1] List of outliner software https://www.outlinersoftware.com/topics/viewt/807

Programming an outliner from scratch

 

There are endless amount of outliner programs available. The subject has emerged since the 1980s. One of the most interesting example is the zim-desktop wiki. From the graphical layout it works with the established two pane model. In the left window there are sections available while the right larger window shows the content of a section.

In contrast to other outliner tools like cherrytree, the difference is, that zimwiki is using the underlying file system. Instead of creating a single XML file which holds the database content, zimwiki stores the information into an entire directory. The advantage is, that external files like images and very important attachments can be stored in this way. For example, it is possible to create a new zimwiki entry and then append an Excel sheet to this entry.

One disadvantage of the project is, that it was realized in the python language. Python is a great choice for prototyping an app and for scientific purposes but it fails for writing productive code. The result are many unfixed bugs in the zimwiki software and the only way to overcome it is to rewrite the entire program in the C/C++ software which is the industry standard for production ready software.

Despite the disadvantages, zimwiki can be recommended because it shows how future outliner tools have to look like. The amount of features is not overwhelming and the preference of the markdown format as internal text formatting makes sense.

Pros and cons of the cherrytree program

 

Cherrytree is an outliner software and the only example in the Debian repository. Outliner tools are a relative recent development in computing and were developed in parallel to word processing software which was recognized in mainstream.

Especially the idea of a two pane outliner software was available since the 2000s. The cherrytree software is one example for such tools. Unfortunately the software quality is low. The menus are not making much sense and the chance is high that the amount of users is low. But let us go through the program itself. The main feature is that the user can create sections in the pane on the left side and then enter a longer text on the right. So it can be compared with the navigator pane in libreoffice.

The other functions of cherry tree are little. The user can import and export some files and he can also insert images and tables. Nevertheless the idea of an outliner is so interesting that it makes sense to analyze the program in detail. An outliner is some sort of mindmap in a fulltext mode. The user is not required to enter a linear text from top to bottom but he jumps between the nodes similar to what programmers are doing if they are creating multiple classes. In addition the user can enter only keywords.

So the program has it strength in creating to do lists, short notes and very important book scripts. Outliner tools have became the defacto standard for writing screenplays and movies because of its ability to create complex hierarchical texts.

Let us talk about the cons of the cherrytree software in detail. Suppose the user has created a longer document and likes to export it into the pdf format. Technically a pdf file is created. But the margin can't be adjusted and the image resolution can't be changed. So it is more a preview function than a pdf generator. If the user likes to create a readable pdf paper he will need additional software like the previously mentioned libreoffice software.

So the question is why the user is creating first the notes in cherry tree if libreoffice writer has a similar feature under the term navigator? This would explain why the amount of users is low becauase if someone likes to create hierarchical notes and format them as a pdf paper he can do so with libreoffice writer much better. Another program which works with a similar principle is lyx. The disadvantage of lyx is that it needs for formatting purposes a fullblown latex installation which needs including the fonts around 15 GB on the hard drive.

But let us go back to the cherry tree program. The best way to imagine the idea is to take an existing text editor like gedit and enhance it with the ability to insert images and add a pane on the left side which shows the hierarchical structure. This results into cherry tree.

The relationship of LaTeX to outliner software

 The LaTeX software is known as a powerful typesetting program used since decades for creating academic writing. From an outside perspective the LaTeX community prefer it's own program over potential alternatives like Framemaker or MS-Word. But the community struggles in explaining the reason why. How can it be, that a complex software like LaTeX which contains of thousands of macros has found so many advocates?

The reason is not located in the software itself but with the workflow in which LaTeX makes sense. What the average student, researcher or book author is doing can be summarized in the following workflow:

1. makes notes in an outliner

2. aggregate the notes into prose text

3. convert the prose text into a readable pdf paper

This workflow divides the task into two subtasks: creating the text and rendering the text. And both tasks can be realized with different software packages. There are on the one hand programs available for creating an outline which are Omnioutliner, a text editor or the notecase software. And on the other hand there is software available for converting the text into a pdf document.

Let us imagine how the workflow can be mastered without the famous LaTeX software in the loop. First step is to create the outline with a program like notecase which is an open source outliner for Linux. And then the text gets exported into the markdown format and converted in to a pdf paper.

This workflow comes close to what LaTeX users are doing with their software. They are using two different programs for the workflow. A typical combination is to combine the emacs editor for creating the outline, and the pdftex backend for creating the pdf file. Another frequently used combination is to use the texniccenter editor as the editor and the luatex engine for creating the pdf file.

The interesting situation is, that an outliner software doesn't need a preview how the text will be shown in the pdf format. During the outline subtask, the user doesn't care about the size of the font, or if the output is rendered in one column or twocolumn. This missing capabilitites of an outliner software is described by the latex community as an advantage. And they are right. The outline of a text which contains of sections, subsections, intext references and bibliograühic references has enough complexity. There is no need to render the information like it will be printed later.

Even if somebody doesn't prefer the latex software, the chance is high that he will use a divided workflow which contains of creating the text and formatting the text. Or let me explain it the situation from a different perspective. Classical WYSIWYG software like MS-Word, indesign or scribus are a poor choice if the idea is to use them as outliner tools. In theory it is possible, but this is not the purpose of these programs.

Perhaps we have to a go a step backward and answer the question what desktop publishing is in general? Has it something to do with printing out a text or rendering the text in the pdf format? No, the core task is to create the text in an outliner software. The interesting situation is, that the LaTeX program wasn't designed as an outliner software, but latex assumes that the outline is provided as input. That means, the user needs the structured text, and then he can run the latex program.

Some applause for the LaTeX software

 Even the chance is that, that most users are aware of the LaTeX software already it makes sense to emphasize the advantages again and again. The first thing to mention is, that most desktop publishing software from the past is gone. Software programs like Ventura publisher, Word for MS-DOS, Framemaker and even quarkxpress are forgotten. They were replaced by other tools. Only the LaTex has prooven it's stability over decades.

The perhaps most common question is, why should somebody learn LaTeX if he can typeset the document with MS-Word much easier. The greatest advantage is, that similar to the txt2ps command line tool under Linux, the LaTex program can handle easily larger file. It is not difficult to create a pdf file which has 3000 pages, contains lots of graphics and archives all the issues of journal. This feature is missing for possible alternatives over TeX.

Instead of describing how LaTeX was used 20 years ago it makes to give a short outlook how a modern approach is to creating print ready document. First step is to create a document in an outliner software like Emacs, Lyx or Emacs. This step needs time and has nothing to do with desktop publishing or typesetting. If the text is ready the funny part starts. The text is converted into the LaTeX format and converted in the pdf format. With the help of a template this step needs only a little amount of time. Even the amount of LaTex documentations is high and the possible options are endless, the average user will spend only a little amount of time with the typesetting software. It is basically a rendering engine which takes a raw text and produces a pdf file. This pdf file can be printed, uploaded into the internet or copied on a cd-rom.

Somebody may argue, that the ability to create a pdf file with 3000 pages is not a big thing and other programs like indesign or Word can do the same. No they can't. In contrast to a common myth, text processing is a computational demanding task. A single jpeg image with 300 dpi can have lot of megabyte in size. It makes sense to avoid WYSIWYG software and take advantage of desktop publishing software for handle the document creation process.

___Whats after LaTeX?___

The main advantage of LaTeX over alternative desktop publishing software like Word and Indesign is, that the author can focus on the content and ignores the layout. The problem of aranging the content in columns and place the images at the correct place is postponed to a later step and is handled by the latex software during a batch processing run. This reduces the obligation of the author to creating the text.

The interesting situation is, that LaTeX is not the advanced software for focusing on the text. What authors are using in the reality instead of LaTeX are outlining software tools. These gui applications have usually on the left side a pane in which the sections of the text can be arranged. Famous helping tools for the LaTeX software like Texniccenter and Lyx are providing an outline capabilities but many other tools outside of the LaTex ecosystem are available too.

From a technical perspective these programs are emphasizing the difference between text and layout much further. An outline only tool like cherrytree doesn't provide any ormatting capabilities but it is focussed on the text itself. That means, the dimension of the age are missing and most images are shown in a draft mode. In most cases outline tools are used for making notes. But they can be utilized for writing longer texts as well.

It is important to known that an outline tool is different from desktop publishing. Some outline tools are supporting the export of the file into the pdf format, but the quality is low. Instead, outline tools are trying to structure the text on a semantic level with the help of sections.

The simple reason why so many authors are fascinated by the LaTeX software is because it handle the layout process automatically. After the some decision were made about the amount of columns, the font and the size of a section, the latex software will layout the text by it's own. It fulfills the promise of database publishing.

The perhaps most impressive example for the strength of latex is the lyx software. Lyx is some sort of outliner with a pdf export option. Lyx asks the author to provide the text which consists of a hierarchical paragraph and then lyx can export this document into the pdf format. In contrast to alternative outliner software like cherrytree, the pdf file was typeset with the latex backend which is equal to get a high quality result back.

Let us construct an example. Suppose the user provides the lyx file for a 2000 page long book. The file contains of the raw text, which are the paragraphs, the sections, bibliographic references and tables. What lyx can do very well is to confert this raw text into a fancy pdf booki.

Sometimes the Lyx software is described as a dtp software similar to idesign or Word. But it's not. Lyx has more in common with outliner software like cherrytree, texniccenter or Omnioutliner on the mac. It's purpose is, that the author can enter notes and paragraphs.

___Outline editor___

Let us take a closer look at the cherrytree software. There is a similarity between cherrytree and Lyx. Both programs have on the left side an outline pane and the user is able to enter text, items and tables in the main window. The advantage of Lyx is, that it can generate a well looking pdf document on demand, while the export option of cherry tree is reduced to create a html file.

The similarity between lyx and cherrytree is that in both programs a document contains of a hierarchical structure which contains of sections, subsections and fulltext. It is possible to jump between the section with hyperlinks and a table of contents is available as default.

Suppose somebody invents a cherrytree plugin which allows the run the external latex engine with a command line. Then the cherrytree software will behave the same like lyx. The interesting situation is that cherrytree is widely recognized as an outliner tool. This is the correct description for these software programs.