Making notes on the computer

 

The reason why so many software programs are available is because the subject isn't understood very well. Well known programs for making notes are a plain text file, Onenote, asksam, jabref, endnote and many other programs. To summarize the information we have to introduce the software category of a two pane outliner.

A two pane outliner is some sort of defacto standard for creating notes. If a certain program tries to reinvent the wheel the tool has to explain why it is not an outliner but something else. Most existing programs like Obsidian or onenote are indeed normal outliner programs which are extended by additional features. So let us first investigate a typical outliner program in detail. The Treepad outliner is a good example program. The program is no longer available but for historical reason it can be seen as a good example.

Treepad was available for the windows and the Linux ecosystem as well. It is showing a gui to the screen which contains of the two pane layout. In the left window the structure of the document is shown, while in the right pane the text itself is presented.

It should be mentioned, that an outliner is different from a text editor, from a database and from a word processing application. It has little or no layout features, and has a strong focus on full text search. In contrast to a text editor, the user is investigating a very long document which contains of hundreds of subsections.

The outliner concept is so powerful, that it is hard to imagine a possible alternative. The same outliner program can be used to store different sort of information like source code, notes, and even bibliographic information. At the same time, the term outliner is seldom used in the PIM and note taking community. Why?

One possible explanation is, that most note taking apps doesn't see themself as an outliner but as something else. The paradox situation is, that this self awareness doesn't match to the reality. A closer look into note taking tools in the now and from the past have shown that most of them are outliner programs. Let me give an example. The asksam software which was very popular in the 1980s and early 1990s has described themself as a full text database. The reason was, that asksam was able to search in the content at very fast speed. This was the unique feature of the software. BUt a closer look into asksam will show, that before the content can be searched it has to be created. Ant his is done in a hierarchical fashion.

The inofficial successor of Asksam is the myinfo software. Myinfo looks similar to the mentioned treepad software. It is a normal outline editor which includes the ability to search in the text. It is the same sfotware category like Cherrytree or notecase.

Another interesting situation is, that larger programs like Obsidian and Onenote are avoiding the term outliner. THese programs are looking different from an outliner, because there is no left pane with the outline available. It needs some extra clicks in the GUI and in case of Obsidian the user has to install a plugin to make the outline visible. But, all these programs are asking the user to create sections. Otherwise the content can't be stored. So it is an outliner for sure, only the left pane is not visible.

The Treepad software was available from 1995-2019 and then the project was terminated. The major advantage over other note taking software was, that treepad defines themself as a two pane outliner. It was described in the manual what the inner working is. The user has to create sections which are visible and then he can enter text. This is the core principle of a note taking software. And the concept is so powerful that other programs apart from Treepad are working with the same idea in mind.

It is important to understand that note taking is equal to use an outliner program. No matter how the software title is and what the self understanding of a certain program is, all these tools are nothing else but ordinary two pane outline editors.

A short history of outliner programs

 

Outliner programs are a seldom analyzed software category. On the same time, lots of these programs are available. So it seems that outliner are interesting for a larger amount of users. What makes the situation difficult to explain is the huge amount of different software programs. A rough estimation is that at least 300 different software programs were programmed since the 1980s and the amount is growing.

Let us explain the situation from a chronological perspective. The good news is, that over the decades only a few larger applications were popular. The 1980s was the decaded of the MS-DOS operating system. Outliner programs like Asksam, Grandview and MORE for the Apple system were created. The interesting situation is, that for the 1980s these programs had a large size. The more software was around 200 kb on a floppy disc. An outliner program was similar to a database and a word processing a complex application. There was even law case started around the PC-Outline program which looks similar to the grandview software from Symantec.

In the 1990s, nearly all the existing MS-DOS based outliner programs lost it's marketshare. Instead new applications like Treepad and Myinfo were programmed. This pattern is visible in the 2000s as well. New software was programmed from scratch like Notecase, Onenote and keynote. It seems, that the avarage lifespan of an outliner program is lower than 10 years.

Even larger software companies like Microsoft are not able to dominate the market. The Onenote software for example is not the market leader but stands in competition to evernote obsidian and of course the older outliners from the 2000s.

If so many different outliner programs are available it should be estimated that it is clear what an outliner is. This is not the case. Most programs are defining the software category slightly different. Some programs are referencing itself as a two pane outliner program, while other are prefering the term PIM or knowledge base. Other programs are calling themself note-taking app.

From a programming perspective an outliner is a combination between a text editor and a file browser. In the manual of the first popular Windows based outliner “treepad” there was a good description given. Treepad is according to the self understanding a combination of the Windows file explorer with the notepad application. That means, an outliner can be replaced by these single tools. In a minimal case it is equal to use the Linux command line for showing a directory and use a text editor to change the content of one of the files. The outliner program simplifies this task and in most cases an additional full text search engine is provided.

Let us take this explanation as a starting point to get a better understanding what an outliner is about. All the operating systems like Windows, unix and Apple have a directory structure on the harddrive. The user can create folders and store text files in these folders. Such a structure is some sort of knowledge base and most user will store information this way. An outliner program makes this use case more explicit and provides a well looking GUI interface which allows to create more complex folder structures.

Even if around 300 different outliner programs are available, most of these programs are working completely different from each other. So it makes sense that so many software was created. What the programmers have done in the last 40 years is to experiment with different GUI layout and program features. For example, in the years of the grandview and asksam, a left pane was missing. Most of the early outliner programs are working with a single pane window. Since the mentioned treepad program from 1995 the situation has changed. Nearly all the programs since the year 2000 are working with a two pane layout. Some programs have more features while other are very minimalist.

Under the LInux operating system there are text only outliner plugins available for the VIM and the emacs editor. These tools are not able to render images because the assumption is / was that images are not needed.

One possible reason why outliner are a seldom explained software category is, that most users think they have no need for such tools. They are using either the folder structure of the operating system or they are storing the outline into a MS-Word document. And indeed, MS-Word can show on the left side an outline window which allows to navigate in larger documents easily.

On the other hand, lots of users are not satisfied with a simple Word document. They have a need for a dedicated note taking app.

A minimalist outliner software

 

The amount of different outliner programs is huge. Around 400 software projects were written in the last 30 years. The interesting situation is that most of these programs are very huge in terms of codesize. Even the minmalist cherrytree program which is available as Debian package has a footprint of 650 kb. More complex project like the Zettlr app needs around 75 MB on the harddrive.

According to the famous outliner forum, the smallest possible outliner today are Mempad (210 kb), Theguide (1 MB) and Treepad light (450 kb). Especially the treepad light software is the unofficial champion in outliners because it was created very early and was available over many years. So it seems, that even lightweight outliners have a size of around 200 kb upto 600 kb.

This codesize can't be called lightweight. Creating a 500 kb softwareproject can't be realized in a short amount of time. So the dominant question is how to reduce the code size. Is it possible to write an outliner in 10% of the treepad light size which is 45 kb and less? This is perhaps the core question for modern outliner software. What most outliners available have in common is, that they are the opposite of a small software.

It is unclear how big a project like OneNote or scrivener is, but a rough estimation is, that such projects will occupy 10 MB and more on the harddrive. It is a typical example for a typical heavyweight PC software which is created in thousands and even million of codelines.

[1] lightweight 2-pane outliner for breaking up text quickly https://www.outlinersoftware.com/topics/viewt/4707/

Creating GUI applications with wxpython

GUI Builder

The good news is, that all the tools are available in the Linux operating system to create powerful GUI programs. The disadvantage is, that the documentation is spreaded over many places and it is difficult to get an overview. The assumption is, that the reader is familiar with the Python programming language already and likes to create a full blow GUI desktop application. The tools which are needed are: wxpython library and the wxglade GUI builder.

The main problem with wxpython is, that the API contains endless amount of widgets and config parameters. The official API documentation is very large and reading all the material is not possible. The easier approach to create a window is a GUI builder. A GUI builder like wxglade allows the user to drag and drop the widgets into the frame. Such a gui builder is a must have tool, because otherwise the user has to enter the commands by hand into a programming editor. And here is the problem located which is visible for GUI programming in general. How does the user know which widget is created is created with which command? Right, he can't know because it is written in the documentation. But, searching for the correct statement in the docs takes time.

Let me give an example. Suppose the user likes to draw a text entry field. In the wxpython framework it is created with wx.TextCtrl(parameters). The reason why is because the API defines how to create such an element. Such command has nothing to do with programming in general but it is about using a certain GUI framework. The problem is that each GUI framework like tkinter wxwidget or Qt is using a different sort of commands. In addition, the parameterlist for all the commands is endless.

So the only option for the newbie is to the mentioned wxglade GUI builder. Here the user doesn't know the detail, but draws a window similar to create a PNG image in gimp. The GUI builder creates the code automatically which can be copied and pasted into the own application. A typical criticism against gui builders is that if the layout has changed the code has to updated manually. This concern is correct. It doesn't make sense to use a GUI builder for creating the code direct, but the GUI builder is a learning tool which allows to understand which commands are needed.

The user reads the automatically created code, and recognizes, that a certain widget was created with certain commands. This allows to get a faster overview over a widget. So the gui builder is the starting point to search for the details in the wxpython API documentation.

MVC principle

Another powerful concept to simplify the creation of a GUI app is the model view controller principle. MVC means to split a window into three different classes for the database backend, the layout and the events in the window. The layout contains of commands for creating the widgets. These commands are created with a gui builder or typed in manually. If the user is prressing on the buttons nothing will happen.

The event handling is done in another class which is the controller class. It is listen to the events and changes values into the form. If the action is more complicated the controller is using the backend class for additional processing.

Suppose the GUI app consists of 3 windows. Then the overall amount of classes is 9:

window1view

window1controller

window1model

window2view

window2controller

window2model

window3view

window3controller

window3model

 

References

[1] wxPython tutorial https://zetcode.com/wxpython/

[2] wxPython API Documentation https://docs.wxpython.org

[3] Youtube wxglade

Short history of outliner software

A while ago, somebody has created a list of outliner software. it was posted into the outliner forum in the year 2008 and the list is really long. Large scale software projects like asksam, myinfo and Onenote were mentioned. But some more recent projects like Obsidian and Citavi are missing. Also some smaller Open source projects like Cherrytree are not in the list yet.

But let us slow down the situation a bit and start from the beginning. The perhaps most popular outliner like software from the 1980s was Asksam which was an MS DOS application. The idea behind asksam was to create a full text database on the local floppy drive and then do a full text search in the database. This concept was new in the 1980 because it was the first time that larger amount of data were searchable.

In the 1990s more outliner programs were created notable IBM Lotus notes. The asksam project was stopped and the possible alternative is myinfo. In addition, lots of shareware and opensource outliner programs were created. The sstate of the software is working with a two pane window which allows similar to asksam to retrieve the database in fulltext.

The reason why so many different programs are available which are of course completely incompatible to each other is because the concept of an outliner software makes sense for many PC users. It seems, that each of the 3 billion computer users in the world has a need for an individual outliner tool.

Some popular projects in the recent years are Microsoft onenote and Obsidian. Similar to the first outliner software in the 1980s the idea is to structure knowledge in a database. Another popular software which is used mostly by programmers is the Emacs org-mode which is some sort of outliner for the emacs editor.

Let us try to investigate the subject in detail. There is a difference between a programmer IDE like eclipse and an outliner software like myinfo. The average outliner is able to edit many files at the same time. In a text editor, the user opens a single file and makes some changes. While in an outliner program, hundreds of files are opened in parallel in the left pane. In addition, the average outliner supports images and attachments. In contrast, a text editor used for programming software isn't doing so.

Perhaps it makes sense to take a closer look at the myinfo project, because it is a mature outliner software which includes all the important features. The main feature of the software is, that it can be used to create a structured text document. In the left pane, the user sees the outline which is the table of content of the text while the right pane shows the content. The text contains of prose text, bullet lists and very important images and tables. In addition, the user can do a full text search in the content so that he can find older entries easily. This is – in short – the basic principle of an outliner software.

The Obsidian software has modified the principle a bit. Obsidian is a graphical mind map editor. The standard layout is different from a word processor, but Obsidian shows a graph of connected nodes. The ability to link from one entry to another is very important.

What we can say for sure is, that the amount of outliner software will grow in the future. It seems, that there is a need to program different sort of software. A single tool which is used as the default outliner is not available. Even the large Microsoft Onenote software wasn't able yet to reach a larger audience.

A very interesting but seldom mentioned software is AZZ cardfile. The dominant difference to other outliner programs is, that AZZ cardfile doesn't support nested cards. There is no ability to create folders and populate them with subfolders, but all the cards are on the same level. This principle emulates a classical paper based notebox and it is different from a directory structure used in computing. The AZZ cardfile software was created as an improved cardfile.exe software which was preinstalled in the Win3.1 software. cardfile.exe was a very basic index card software.

Timeline

Freebyte Treepad, 1995-2019

Notecase Pro 2011-today

Notecase 2005-2008

AZZ cardfile 2001-today

asksam 1985-2008

Inforapid 1997-today

Onenote 2003-today

The majority of the programs were created from year 2000 until today. Software projects like Inforapid, Treepad were programmed mostly for the Windows operating system. Older examples for outliners like Grandview and Asksam were available in the 1980s for MS-DOS PC but they were never very popular or powerful. The interesting situation is, that most projects are promising to reinvent the idea. The user is explained what a personal knowledge system is about. But from a technical side all the programs are working with the same two pane outliner principle. In the left pane the outline is shown while in the right window the full text gets entered. Even more recent projects which are programmed from scratch are working with this principle.

The perhaps most dominant concern for the end user is, that the file format is not compatible to other programs and if the software is no longer maintained the user can't switch to a newer program.

 

References

[1] List of outliner software https://www.outlinersoftware.com/topics/viewt/807

Comparing Gnome with Qt in terms of commits per week

In the domain of Open Source software it is important to get an understanding which of the projects is more stable. The reason is, that for a single problem many software packages are available so the user has to select one of them. The problem is, that for the newbie it is very complicated to judge if Ubuntu or Fedora is the better distribution, if firefox or Chrome is the better browser or if gnome or qt is the better GUI environment.

Sure, these problems are discussed in public forums but at the end it depends on the individual user which software package he prefers. What is missing is a measuring unit which determines from an objective base which open source project is recommended.

A possible criteria is the amount of commits per week. This number can be determined from the version control system's logfile. And very important it be drawn into a fancy chart. This allows to compare two projects.

Programming an outliner from scratch

 

There are endless amount of outliner programs available. The subject has emerged since the 1980s. One of the most interesting example is the zim-desktop wiki. From the graphical layout it works with the established two pane model. In the left window there are sections available while the right larger window shows the content of a section.

In contrast to other outliner tools like cherrytree, the difference is, that zimwiki is using the underlying file system. Instead of creating a single XML file which holds the database content, zimwiki stores the information into an entire directory. The advantage is, that external files like images and very important attachments can be stored in this way. For example, it is possible to create a new zimwiki entry and then append an Excel sheet to this entry.

One disadvantage of the project is, that it was realized in the python language. Python is a great choice for prototyping an app and for scientific purposes but it fails for writing productive code. The result are many unfixed bugs in the zimwiki software and the only way to overcome it is to rewrite the entire program in the C/C++ software which is the industry standard for production ready software.

Despite the disadvantages, zimwiki can be recommended because it shows how future outliner tools have to look like. The amount of features is not overwhelming and the preference of the markdown format as internal text formatting makes sense.

Number of commits for Linux and Freebsd

 

 

 

A closer look into the Unix history has shown, that the Unix wars are not over but many incompatible kernels and Unix distributions are available. A measurement to determine the quality of a software project is to count the amount of commits. The Linux kernel is leading the statistics because it has over 200 daily commits.

In contrast, the BSD projects can't compete in terms of manpower. The amount of active developer is lower. The chance is high that on the long term projects like netbsd won't survive.

A Luhmann Zettelkasten and a mindmap

 

Even for experts in personal knowledge management it is hard to explain in a single sentence how to create a Luhmann style note card system. One of the best papers about the subject was written by Schmidt [1] but it was written for an academic audience. In contrast the picture shown on top of this blogpost summarizes the idea in a very compact format.

A luhmann note taking system is basically a mindmap. In the mindmap a hierarchical structure is available what is common for all mindmaps. A node can have subnodes which are linked with arrows. Mind maps are a well known creativity technique but the system can't scale up to larger needs. Writing down 20 nodes on a single sheet of paper is maybe possible but writing down 100 and more contradicts the idea of a mindmap.

The Luhmann notetaking system takes a mind map as starting point and projects the nodes on a 1d array which is the notebox. the 1d array is a storage container for a hierarchical mindmap. This mapping allows to create larger mind maps which have unlimited amount of entries. in theory it is possible to onvert a Luhmann note box back into a mindmap and print it out but the graph would become too large. What a user is prefering instead is to work only with the 1d array. Instead of creating new entries in the original mindmap the topics are created in the card box.

 

Larger mindmaps

... can be mapped also to a 1d array. The figure looks a bit messy but from a technical perspective all the arraws to mapped to a single target location. The idea is that the same semantic network is rendered either as a mindmap or as a 1d array. The 1d array allows to store the information more compact and it is easy to use paper based index cards for physical storage.

The unfolded card box is a normal mindmap. And if the amount of cards is higher it will become a very huge mindmap It would be equal to draw a mindmap on an entire wall. And if the wall #1 is running out of space the next wall is added. The interesting situation is that the entire system remains always a mindmap. So the shoebox of Luhmann which was in the reality a huge cabinet is holding a mindmap. And the mindmap is the second brain.

The example mindmap in the figure consists of only 15 nodes. After adding additional 15 nodes, a normal A4 sheet of paper will run out of space. This might explain, why only smaller mindmaps are drawn in the reality which makes it not practical for complex topics. It is not possible to write down 100 kb of notes into a mind.

At least this is the hypothesis if the luhmann numbering system is unknown. After mapping the mindmap to a 1d array it is pretty easy to add additional nodes and there is no upper limit. So the Luhmann method is a super-mindmap.

 References

[1] Schmidt, Johannes FK. "Niklas Luhmann’s Card Index: Thinking Tool, Communication Partner, Publication Machine." Forgetting Machines: Knowledge Management Evolution in Early Modern Europe. Brill, 2016. 287-311.

High level programming language

 

The pygame library is for sure a high level programming language. It would be interesting to know the reason why this is the case. The working thesis is, that it has to do with amount of commands which are provided.

The exact number can be determined with the official API description.[1] The amount of all commands in the pygame 2.0 library is 754 which is a lot. Every command has parameters and is described in the API. Creating such a library and writing the documentation takes a lot of time. In addition it is important to know that most python applications will need more than a single library. They include other libraries which have the same and more complexity.

From a user's perspective a complex large library is easier to use than a simple one. A game library which has fewer than 20 commands is much harder to use, because the user is asked to create many commands from scratch. In contrast, the pygame library provides many useful tools out of the box. There is a single command to initialize the graphics screen, another commands draws a line, and a third command create a timed loop with 60fps.

Let us compare pygame with low level programming like Assembly and very important with Forth. The assembly language has a low amount of commands. Forth has reduced the instruction set further. Most forth implementation have no additional libraries but the amount of commands is reduced to fewer than 50. The user is asked to write with only 50 commands useful programs. Technically this is possible, but it is a demanding task.

We can only guess how many commands are provided by a python environment. Perhaps the amount of all commands is 10k or even more. What we can say for sure is, that all the libraries combined have a huge documentation with thousands of high level commands. This ecosystem makes it easy to create high level applications like games and GUI applications in a few lines of code.

Other high level languages like Java and C# are working with the same paradigm. The amount of commands in the typical Java library is endless. Some programmers have written these APIs in the past. They have done so because it simplifies the programming. Today's scripting languages are the most high level languages available The disadvantage is, that such an ecosystem can't be created easily. It is possible for a single programmer to write a Forth environment but writing a python ecosystem which includes some important libraries is impossible for a single programmer.

In some older tutorials from 1980s it was assumed that Forth is a low level and a high level language at the same time. This assumption is wrong. Forth is a low level language and nothing else. In Forth there is no large library available and also a standard amount of commands is missing. The typical forth interpreter provides very few commands. This is an obvious sign, that it is located on the low level layer. The factor language which is forth dialect, is working with a different objector. Factor has similar to Python a large library with predefined commands which makes it easy to write applications. But, Factor is not a typical Forth implementation because it is not minimalist. The question is not about sequence of commands which are stack based but the more serous question is about the size of the library with predefined commands.

[1] https://devdocs.io/pygame/

Pros and cons of debian linux

 

Debian is a well known but seldom used Linux distributions. Especially newbies in the Open Source community are preferring alternative Linux distributions like Fedora, Arch Linux, Ubuntu and Suse. To understand the mismatch better why Debian on the one side is great but is seldom used it makes sense to analyze the distribution from a beginner's perspective.

Suppose a computer user has logged into a preinstalled Debian in a computer room at the university or has installed the system in a virtual environment on his home PC. After entering the command “uname -a” the user will notice that not the recent kernel version from kernel.org was installed but only a longterm kernel. The same picture is there after testing out the python3 interpreter. It will shown in the interactive mode that the installed python version is around 1.5 years old and was compiled with the previous GCC Compiler. Even if the user won't need the current software version the impression is, that the system is outdated. In the next step the user will try to playback perhaps a video in the H.265 format which is running fine in Windows and MacOS but in Linux there are some codecs missing.

In summary, the user will come to the conclusion that the distribution is working with outdated software and in addition is not very powerful. So the chance is high that the newbie will deinstall the system and prefer a different operating system. Which is maybe a Linux system but in a different distribution.

It is hard or even impossible to explain to the user why most of software is outdated and at the same time it is the most recent version available. Even if the explanation contains a detail description about the advantage of stable release management and the hint that that the software is working bugfree, the newbie won't agree to the Debian advocacy.

But let us try to take the challenge more serious and explain in a single version why Debian is using outdated software. Suppose, the software installed on the harddrive is never older than 1 week. The logicial consequence is, that the system maintainer has to run a complete reinstall of the software every week. That means, every sunday evening the local adminstrator has to run the update script which will download around 3 GB of new and fresh sourcecode from the internet, is processing all the updates for around 3k packages and then a short function test is made. The overall procedure will take even with a quadcore cpu and a fst internet connection many hours and requires manual intervention. Nobody likes to do so. And that is the reason why Debian is updated only every 2 years in such a way.

It is not possible to deliver an to date Linux distribution which doesn't need to be updated. It is a contradiction and only one objective can be realized. Either somebody can simply work with the system or the user has to manual update the system every week which includes the chance of a reboot failure.

The advantage of Debian over other Linux distribution is especially visible in larger computer rooms which have 30 and more workstations and it obvious for a server system. In both cases the admin of the computer is not motivated to run every week an entire update which includes to download gigabyte of .iso files from the internet and install the packages over the older versions. Such a complicated update should be run only once a year if nobody is using the machines.