How to use Jabref as a Zettelkasten

The quick answer is, that there is no way in doing so, but let us imagine the workflow from a theoretical standpoint. The jabref software allows to manage a bibtex file which includes to add new entries and push the bibtex key to the lyx software. Also it provides the ability to search for dubplicates and can search in the database for any field. This makes jabref a powerful bibliography manager which can be recommneded to latex and lyx users as well.

Unfortunately, it has little or no support for handling notes created by the user while he is reading documents. Suppose somebody has found an interesting pdf paper and would like to write down 3-5 important short sentences what was written in the article. The jabref software has similar to the bibtex standard a field “Note” which allows to formulate free text. In the screenshot the sentence “The quick brown fox jumps over the lazdy dog” was entered.

The problem is, that the field is only one under many. It is a detail feature within the bibtex ecosystem. It can't store bold formatting, it can store images, tables and it is not possible to link from one note to another. The chance is high that most users doesn't even know about this field, most bibtex database were created with additional notes.

On the other hand, for a single user who writes a paper it is very important to annotate a reference. It is the core element of active reading and nearly all common place books are created around personal notes of the users. It is unfair to blame the jabref software for the low quality notes field, because the underlying bibtex standard doesn't support it as well. A more realistic attempt to describe the situation is by recognizing that note taking was mostly ignored in the past. There are wonderful software programs available to format a document like latex and powerful frontends like lyx and jabref but what is missing is the ability to create a personal knowledge management database. The only program which goes in this direction is the emacs org-mode which was introduced in 2003. [1]

[1] Johnson, Timothy. "Emacs as a Tool for Modern Science." Johnson Matthey Technology Review (2022).

A rough description of the perplexity.ai chatbot

 

In contrast to the famous chatgpt software, the perplexity.ai chat is less known. It's main advantage is, that everybody can use it. Similar to chatgpt, it is difficult to get concrete information how the algorithm is working in the background. Somewhere it was mentioned, that the same gpt3 language model was used which says anything and nothing. The more interesting question is what are the limits of the chatbot? Is AI really new technology or is the hype around chatgpt exaggerated?

In contrast to a search engine like Google, the perplexity.ai software is a Q&A software and can answer queries formulated in natural English. To make the difference clear, we have to describe first what the limits of a normal search engine are. Suppose the user enters two and more keywords into the input field:

word1 AND word2 AND word3

In many cases the result list will become empty because a website contains only of two of the words, but not three of them. The google search engine can't guess what the user is asking for but the result page is generated with normal non AI technology.

In contrast, a Q&A software is converting the internet and the user request into a numerical vector and then searches for the best matching. So it is basically a neural network which understands English. This allows to parse and generate the information more robust.

Some years ago there was the “Wolfram Alpha” search engine which has promised to revolutionize online search but a detailed investigation has shown the limits of the project. So let us try what the perpelexity.ai is able to do.

The basic question for any Q&A software is a simple YES/NO question like, is the ocean blue, is the sun yellow and is pizza a food. The chatbot answers all of them correctly and gives smart additional information.

The next iteration is a more complicated request like “What is 10/3?” “Which books were written by Hemingway?” “When was the computer invented?” All these questions were answered by the chatbot with ease. The assumption is that the better known chatgpt is also capable of this sort of tasks.

So we can say, that the perplexity.ai chatbot is the first working q&a webseite which produces high quality output.

Let us take a look at the more serious part which is about possible failure of the software. What the perpelxity.ai software can't do is to write longer stories. All the output is limited to 3-5 sentences. Also the software can't tell a joke but it will provide only the URL to a website which hosts some jokes. In addition the chatbot can't display images and in most cases the answers explaining only the surface of a topic but not the detail information.

What perplexity.ai and chatgpt have demonstrated is, that from a technically side it is possible to program a Q&A website which is more powerful than a traditional search engine.

But let us go back to more basic tasks. The chatbot can list the major books written by hemmingway. In contrast to the google search engine it is not showing a website somewhere in the internet and it is not redirecting the user to Wikipedia which has the same list. But the chatbot prints out the information in his own words. For the untrained user this sounds like magic

To understand the difference to a chatbot from the past let us ask the same question to the Cleverbot chatbot:

Human: Which books were written by Hemingway?

Cleverbot: I don't remember.

Human: Again, Which books were written by Hemingway?

Cleverbot: What do you mean I was written?

Human: Hemingway list of books

Cleverbot: What is your favourite book?

Such kind of interaction was normal for pre chatgpt software. The software is using a predefined set of answers and can't answer concrete questions for facts.

In contrast, the perplexity.ai chat bot can parse the question and gives the correct answer. And it can so for a slightly different question as well.

This might explain why the chatgpt software has gained strong attention in the internet. Because the beta testers are surprised how powerful the software is and they want to know what the limits are. One possible answer is, that the engineers themself doesn't know the answer. A likely comparison is to describe chatgpt as some sort of chess playing engine which is able to beat the strongest human player in the world. It was difficult to accept for humans that computers can play better chess and some journalists have argued that the AI isn't understanding on a deep level what chess is about. Perhaps it is possible to give the same argument for modern Q&A website. On a technical level the machine can only process numbers but has no understanding what the meaning is.

The ability to process natural language is usually seen as a sign of intelligence. In most exams the students are asked to answer easier and more difficult questions formulated in English. In a typical exam about computer science a possible list of question would look like:

Who has invented the TeX software?

What is a c compiler?

Explain the advantages of object oriented programming.

How does the A* algorithm works.

Most students can't answer all these questions correct but only a fraction of them. The teacher will read the answer and determines the score and then the human students gets a certain review. Suppose a computer program is able all these question correctly, does this mean that the AI is on the same level like a human student? What we can say is, that AI is able to imitate humans in a surprisingly high level.

From Treepad to Zimwiki – a small history of outliner software

The history of outliner software is a seldom explored subject within computer science. Early examples were created in the late 1980s as hypertext editors. A notable example was the Symbolics Concordia software which has much in common with the later Windows 3.1 help files. The idea is that the user creates a hierarchical tree and can insert hyperlinks for easier navigation. [1]

Sure from a technical perspective, there was the hypercard program which was published for the Apple computer earlier. The problem with this attempt was, that in the early 1980s the homecomputers were not equipped with harddrives. For creating a larger hypertext document is harddrive is needed because the filesize will become 500 kb and more. So it makes sense to assume that the first examples of working hypertext authoring tools were created in the late 1980s on dedicated workstation computers.[4]

In the year 1995 Henk Hagedoorn created for the Windows operating system the Treepad software.[2] It is the most important software in the history of outliner programs. It was the first mainstream program which was using a two pane editor. Left the user sees the structure of the document and in the right pane the full text is shown.

From a more critical perspective, the Treepad software looks like MS-Word in which the document outline was activated. So it is basically a word processing software with a focus on the structure of the document.

After Treepad was created many clones were programmed like Keynote, cherrytree and zimwiki. Technical the difference is low or even not there. The GUI has the same two pane structure and the only improvement of the keynote program was, that it was released as open source software.

An interesting but niche software are zkn3 and synapsen. These tools were created with the attempt to emulate a Niklas Luhmann Zettelkasten. They are used by scientists for maintaining annotated bibliographies. [3]

Another more recent approach is the Obsidian software which combines a chaotic graph structure with a powerful search feature.

There is a seldom analyzed common feature between the early outliner programs and more recent note taking apps. All these programs are using a key value structure for storing information. The key is shown in the left pane which is the name of a file or the name of a section. In the right pane the value is shown which is equal to a free form text which includes images and hyperlinks.

Sources

[1] Nielsen, Jakob. Multimedia and hypertext: The Internet and beyond. Morgan Kaufmann, 1995.

[2] Interview: Henk Hagedoorn, creator of TreePad http://mediaman.com.au/interviews/hagedoorn.html

[3] Markus Krajewski synapsen – A hypertextual card index User’s Manual http://www.verzetteln.de/synapsen/_docs/SynManual.pdf

[4] Computer chronicles Personal Information Management, 1989

Creating a Luhmann style Zettelkasten with the Lyx software

One of the key problems with the Luhmann style note taking method is that it is very complicated to explain and it differs in many points from note taking in the past. The problem with manual note taking on index cards is, that it has much in common with an art project in which somebody has invented his own handwriting style and his own structure which is difficult to explain to others. In contrast, the problem with digital only note taking software is, that the programs are never working the same and endless amount of features are available. So the question is, what is Luhmann style note taking from a birds eye perspective?

A good starting point is to use an existing word processing software like Lyx and creating notes in this software. Advantage is, that not a certain note taking software is needed and the user is not asked to learn a certain software. What is working in an existing word like program can be transfered to other programs easily. In contrast to dedicated outline editors, the Lyx program has some disadvantages for example the full text search is poor, the hyperlinks are difficult to navigate and inserting images makes trouble. Nevertheless i have decided to take the software as an example to explain how to create notes.

Let us start the workflow from scratch with an empty document. The side pane in the left shows the sections of the document.

The main idea is to store the notes without folders but store the notes in a sorted list. This idea can be realized in Lyx with the paragraph style. Each paragraph name is shown in the left outline pane.

To make the note taking process more realistic, some information are added for each note which includes the creation date, keypoints and of course the sources. Until now the situation looks not very exciting, because the document contains of only some notes.

For implementing a typical luhmann style zettelkasten two additional elements are needed: a luhmann ID and a maximum length of a note.

The luhmann id defines the position of each node in the sorted list. It is defined once durcing the creation of the node and never changed. For adding a new node at a certain position the node-id is selected in a certain way. The screenshot shows an example for adding a new node between two older nodes.

The typical question of a newbie is why should somebody add new information between other nodes wouldn't it be more common to add new notes at the end? The idea of a Luhmann Zettelkasten is to group notes by it's similarity to existing nodes. It's a clustering mechanism which allows to recall the information later. This is maybe the most obvious difference to note taking in the past. In a luhmann zettelkasten it is a bit complicated to add something. Because the correct position has to identified first and then the appropriate luhmann id has to be created.

Adding more random notes to the existing structure will result into a longer outline list in the lyx editor. The idea is that each note has a maximum size of 700 Byte which is equal to write an A6 index card. The list will become longer and longer. There are no tags, groups or subsections but the node's hierarchy is defined only by the Luhmann id.

Sometimes the Luhmann Zettelkasten gets improved with further elements for example a dedicated bibliography, the ability to insert images and hyperlinks. Hyperlinks means that a link from one node is created to another node. But these improvement are only minor details which can be ignored in the first step. The basic principle is to make sure that each node has a maximum size of 700 bytes and that a Luhmann id is available.

The main strength of the method is, that it scales up to larger amount of content. The screenshot is showing only 9 nodes but technically it is possible to create 100, 200 and even 2000 nodes with the strategy easily. At the end the user has in the left outline pane a long list of nodes and will see in the main window the content for each node. Thanks to the Luhmann id, similar nodes are located nearby. The document is a linear list which can be scrolled from top to bottom. It is some sort of sorted chaos. At the one hand it looks very messy because of the huge amount of small nodes, on the other hand it is very well structured because every knowledge fits into the same sorted list.

How to write an academic paper without taking notes?

 

Taking notes is a fundamental technique in academic writing and since the advent of the Unix operating system lots of software was created for this purpose, right? It is a rhetorical question because the history of note taking software is not well researched. Sure in terms of quantity many attempts were made for creating notes. The most famous one are the emacs program, the texinfo hypertext format and the bibtex format. The surprising situation is, that none of these attempts are popular and most users are not aware of these tools.

Let us compare the situation in Unix with other operating systems like Windows. The situation in Windows isn't very different. On the one hand there are some dedicated bibliographic databases available like Endnote and Citavi and on the other hand there are text editors and the famous ms-word program for making notes. All these programs are not very powerful. For example in fixed bibliographic database it is not allowed to enter full text comments, while in MS-word like text editor it is hard to search full text in the information.

If all the existing operating systems and note taking software have a low quality and are not used in reality, what was used instead for creating notes in the past? The surprising answer is that note taking is not a big thing, even in academia the value is overestimated. To investigate the working thesis in detail let us describe how to write a paper without any notes or references.

What the author needs is some sort of typing tool which are mostly a classical typewriter or a pen or a software like MS-Word or LaTeX. Then the sentences are formulated. If the text was written some (not more than 3 bibliographic references are added) The exact description which includes the title, DOI number and the journal is retrieved from a library catalog. Then the text gets formatted into the PDF format and is printed on a laser printer. The interesting situation is, that during the entire writing process there was no need to create notes or recall existing notes. But the academic author sits in front of an empty sheet of paper and formulates what he knows about a subject.

The chance is high, that such kind of writing process was common in the past and was applied by the majority of authors. The content of the written text depends mostly on the individual knowledge about a subject. If the author is informed very well he can write a better text and vice versa. Basically spoken all the hype around note taking, outliner software and bibliographic file formats doesn't fit to the reality. For writing a normal academic text there is no need to get familiar with the bibtex software or with a high end free form database like MyInfo.

Even the famous writer and sociologist Niklas Luhmann wasn't using a dedicated note taking system. His so called Zettelkasten was mostly a joke. He has explained to his student that note taking is very important but he wasn't using the technique for himself. Most of the 90000 index cards from his scientific legacy were created only for the public to mimicry reading skills.

Comparing C#, Java and Python

 

All the three lanuages were creaed with attempt to replace the C++ language. It is much easier to write an application with C# and similar languages because no pointers are needed, and there is a large amount of libraries. All of these languages have cons and pros. Let us start with the C# language which has become famous in the Windows community.

C# is the default language in a windows operating system and can be compiled without additional software. The language standard is similar to Java but the syntax is easier to learn and much better documented. The big disadvantage of C# is, that it is not available for Linux. The existing mono ecosystem has to be called a joke because it can't run major components like WPF (GUI widgets) and ADO.net (database access). That means most of the C# programs won't run in Linux. It is pretty hard to create a C# app which runs in windows and Linux as well.

In contrast the java language was created for cross compatibility in mind. The runtime environment in Linux and Windows is the same Similar to C# lots of documentation is available and Java is a mature language. The only problem with java is, that it has never replaced C/C++ because Java programs are known as slow and poorly programmed. For internal projects java is used sometimes.

The third language is Python which has become a surprisingly successful language. Python was started as a scripting language which is around 20x slower than C/C++ code. On the other hand it is very easy to write a software in python and it will run on all major operating systems.

It is hard to say what of these languages is the best one. Perhaps none of them. C# won't run in Linux, Python is too slow, and java can't replace C++. The good news is, that with all these languages it is easy to write software. Even complex programs like games can be created in a few hundred lines of code. It should be mentioned that for unknown reasons none of the presented languages has replaced c/c++. If C and C++ is seen as the same language, more than 80% of the production ready code for windows and Linux is written in the language. In contrast, a language like Java is seen as a toy language without practical value.

From an abstract point of view, it is not possible to invent a pointer free language and use this language for practical application. It seems, that pointer oriented programming is the only way for writing production ready applications. Pointer-oriented means that the abstraction level is lower and the programmer has to decide in detail how to store data in the main memory.

Faster citation with the Lyx software

 

The LaTeX typesetting system allows to cite external literature. instead of providing one or two footnotes, LaTeX can create a reference list with hundreds of items. A huge amount of references is a typical requirement for an academic paper and it is not possible to redefine the rule. The only detail question is, how many citations are needed for each us-letter page. Are 8 references enough or does the paper needs 12 and more citations for every page?

 

Unfortunately it is a complicated to format the citations from the authors perspective. Sure, Tex has a built in bibtex format and the Lyx frontend allows to include a bibtex file at the end. The problem is, that most lliterature databases are not supporting bibtex, and adding new items on the fly might be a bit complicated. 

 

Let us make a practical example to show the bottleneck. Suppose somebody likes to insert a reference in lyx. First, he has to search in the internet for the correct bibtex string. Then, he has to insert it at the end of the bibtex file in an external texteditor. Then, he needs to know the bibtex key and the key is inserted in Lyx. Such a workflow can be mastered for one or two references but if someone likes to cite hundreds of papers a different approach is needed.

 

A possible, easier to manage alternative, is to avoid the bibtex format and cite everything in footnotes. Advantage is that the plain text citation can be inserted directly in lyx. To aggregate all the footnotes at the end of the document a bit of LateX programming is needed:

\usepackage{enotez}
\let\footnote=\endnote
\newcommand{\bracketme}[1]{\,[#1]}
\setenotez{
  totoc,
  backref=true,
  mark-cs={\bracketme},
  list-style=plain,
}
\DeclareInstance{enotez-list}{plain}{paragraph}{
  format=\normalsize\leftskip1.25em ,
}
\printendnotes

The enotez package is included in the preamble and some adjustment are made (increased font size, better indentation, square brackets for the footnotes and so on).

Train localization as a software problem

In the past, low quality sensors were blamed for imprecision in railroad monitoring.Most hall sensors can detect the train only at a certain position but can't provide GPS accurate information. A more robust technique are software defined feedback sensors which are particle filters.[1]

[1] Heirich, Oliver. "Bayesian train localization with particle filter, loosely coupled GNSS, IMU, and a track map." Journal of Sensors 2016 (2016).