Technical experiments wiki-pov fork

The precondition for a successful fork project is, that the fork gets all the patches from the upstream projects. Otherwise the branches are of sync. At the same time, the goal is to not sync the branches so that it make sense to create a fork which looks different. This sounds a bit complicated so let us go into the details of using the git tool for merging different branches.

The first attempt in using only 2 branches was not succesful. The idea was, that the upstream is copied in to the master branch, while the fork is edited in the issue1 branch. The first merge was working fine, but after the second merge some conflicts are the result.

The next next was to use three branches: upstream, master and issue1. This is working much better and it like to explain the idea. The first thing to do is to initialize in a working directory the git repository:

git init

git branch issue1

git branch upstream

git branch

On the screen there are three branches available in which the user can edit. In the upstream branch the snapshot from the wikipedia website are stored. The file article.txt hold the current version which is updated once a month. The upstream is merged into the master branch, and then it's merged into the issue branch. In the issue branch the fork can be edited. Now, the next version of the upstream version is stored in the upstream branch.

And now the magic happens, the user switches to the master branch and executed the following statement:

git merge upstream

git merge issue1

What the git software is doing is to combine the latest upstream version with the fork into a new file. The resulting article.txt contains all the improvements from Wikipedia but it contains also the updates from the fork.

I know, the overall procedure is very complicated because the user has to type in many commands into the terminal. So the prediction is, that some errors will upraise. But in general the idea is to use three branches and merge them into the master branch. In contrast, the upstream branch holds only the upstream version history, which is equal to the timeline of WIkipedia provided in the website.

The chart from the beginning will increase the confusion. What the user needs to know is that he has to copy the latest version of the Wikipedia article into the upstream branch, and commit the changes with “git commit”. It's also important to not delete the branches after merging, because the issue1 branch is the forked version which looks different from the upstream. If the user want's to edit the encyclopedia he is doing so only in the issue1 branch. The master branch is some kind of clearning branch in which the two other branches are combined.

POV forking of Wikiipedia

On the first look the git tool and the Wikipedia project are working the same, because they are supporting a version history. The difference is, that the Wikipedia project never was forked in his history, only local copies are created. A fork is technique used heavily at github to bypass the original community and start developing a new branch. The main feature of a fork is it's ability to integrate the updates of the upstream. That means, the fork contains the latest information plus extra content.

The subject overall is very complicated. So i have decided to make a simple experiment to test what will happen in the reality. For the first step, the fork is created only on the local harddrive but not in the Internet, and it's not the entire Wikipedia but only a few files. But it is well documented so that other users can reproduce the steps. It starts by creating a new git project in a working directory:

mkdir wiki-fork

git init

touch readme.txt

git add --all && git commit -m "initial commit"

Then the folder is populated with three files from the original wikipedia, With a copy&paste the latest markup-file is created in the directory. What we need also is a branch:

git add --all && git commit -m "create three files"

git branch issue1

The idea is, that the fork is maintained in the issue1 branch while the original project (upstream) stays in the master branch. The merge is done with the following command:

git checkout master

git merge issue1

git branch -d issue1 // delete branch

The idea is, that in the issue1 branch my own individualized Wikipedia version is available in which only i can edit, similar to a sandbox. The created edits are never send back into the Wikipedia but they are merged on the local harddrive into the master branch. The best visual understanding is a github project in which a fork is created. In theory, this allows to the developer to become independent from the original project.

The open question is, how does it work in reality. I have searched at Google for some information but didn't found something. So i have to test it out. The critical point is, that sometimes the upstream will update their content. That means, if the wikipedia community change one of the three files online, i have to update the content in the master too. The problem is, that the information in the issue1 branch are different from that so there is need to merge. It's unclear how often this is necessary. The hope is that a merge is needed only once a week, and that it can be done automatically. But in case of doubt it will result into a merge conflict and it's unclear how to solve it.

What we can say is, that the git tool is a here to stay. It's the most advanced forking / version control system available and was designed with the desired purpose in mind.

Alternatives to mediawiki

The mediawiki engine was programmed in the PHP language because of historical reasons. PHP is more advanced than outdated perl scripts but it has major performance problems. The question is which programming language fits more to modern needs? One idea is to use king of programming languages which is C++. C++ is the fastest language available which is supported by independent compilers. It can be used for creating web-applications and outperforms PHP easily. Another alternative is Python which is an easy to use beginner language.

Some wiki systems were written in the C++ language already but they are not used in reality. And python as a language is much slower than PHP. A possible third candidate is https://en.wikipedia.org/wiki/Wiki.js which is working with node.js in the background. In contrast to C++ the Javascript language is accepted widely for web-development. IT's only bottleneck is, that it doesn't provide object oriented features so it can't be used for building larger applications. But is this assumption correct from todays standpoint?

https://www.geeksforgeeks.org/prototype-in-javascript/ describes how to create prototypes in Javascript which are lighter object oriented templates. Recent versions of Javascript are equipped with full blown oop features including inheritance. So it make sense to take a closer look into the WIki-js rendering engine. The advantage is, that the GUI which is shown in the webbrowser and the backend application is written in the same programming language. This provides – in theory – more better user experience than the outdated mediawiki engine written in PHP.

An online demo for “wiki.js” isn't available. What goes into the direction is the visual editor of Wikipedia which is based on node.js. The normal mediawiki backend system was realized in PHP. It seems, that the developers are happy with this idea. One reason is, that PHP was designed as a backend language, while javascript is famous of it's ability to embedded textboxes and forms into the code.

How to fix the rm -f problem

Linux user are confronted with a serious problem. The operating system doesn't ask many questions but it will execute the rm -rf command. The problem is, that all the data are deleted and it's not possible to recover them. To prevent such a mishap it's a good idea to reconfigure the rm command. Perhaps it would make sense if all Linux distributions would do so by default. What the average user want's is to delete files only with the file-manager but not with the rm command.

gedit ~/.bashrc

alias rm='rm -i'

alias cp='cp -i'

alias mv='mv -i'

source ~/.bashrc

Hindi as a world language

A map of the Wikimedia foundation shows the readership of the famous encyclopedia for each country in the world, https://stats.wikimedia.org/wikimedia/animations/wivivi/wivivi.html The Hindi language has a wide distribution all over the world. It is used by India to retrieve information, but the language is also spoken in the US, China. The overall population which speaks hindi is 560 million according to the info box, but in reality it's much more. It's only a conservative estimation and the number is growing. Hindi has a good chance to become the worlds famous language. In india alone, over 50 million pageviews each month are generated by Hindi speaking users. That means, they are using the WIkipedia in their mother toque to retrieve and write information about their own country and the world in general. The overall population in India is 1300 million which means, that in the future the amount of pageviews in the Hindi language will grow.

The german wikipedia debates how to find new authors ...

Under the URL https://de.wikipedia.org/wiki/Wikipedia:Kurier the German version of the signpost informs the Wikipedia community and the public as well about the project status. The main topic in January 2020 is, that the amount of active contributors is to low, and the community argues about the reasons. The idea is to increase the amount of authors, but nobody knows how to do this exactly.

Even if my German language skills are great, it's hard to follow the debate. Not because of the vocabulary but because i have the opposite fear. The problem with Wikipedia is, that the amount of authors is too high and could explode in the future. What does that mean? On the first impression, the Wikipedia project is protected against chatbots, because a chat bot is not able to create a valuable edit. This is true for a complex article which contains lots of natural language and is equipped with extensive references at the end. The danger in Wikipedia is, that chatbots are utilized to generate a certain sort of Wikipedia content which is highly structure. This is called a stub. A stub is small article which contains of two sentences and can be generated from a RDF-ontology by a knowledge-to-text system.

The resulting stub will read like a normal article, except that it was generated by a computer program. In contrast to humans, it's very easy to make a copy of the computer program. A simple unix command like “cp chatbot.py chatbot2.py” is enough to create as much wikipedia authors are needed. The funny thing is, that according to the published papers at Google scholar, simple biography stubs are generated with bots already. That means, the amount of Wikipedia authors is higher than the german chapter is aware of it.

The pros and cons of Wikiprojects

An interesting meta-section in the Wikipedia encyclopedia is a Wikiproject. In contrast to the portals, most wikiprojects are available today. That means, it's a barrier to put a WIkiproject into the deletion discussion. On the other hand, the propability is high that this will happen in the future, because a WIkiproject has the same problem like a portal: low amount of users, and low benefit for the overall Wikipedia.

Instead of arguing pro WIkiproject deletion, it make sense to use the time to hear what the experienced users from Wikiproject have to explain why the project make sense. The interesting point is, that Wikiprojects are the key for reaching an audience in the university. The Wikiproject medicine for example was used sometimes by medical students in courses. It's a low entry option to become familiar with Wikipedia.

Let us investigate what medical students are interested in: they are not motivated to learn something about physics nor computer science, but they stay within their own subject. If somebody studies medicine he likes to read books about it, and that means, only books about medicine. So it's a natural choice to create a subpage in Wikipedia to coordinate a team of medical students who are interested in improving existing articles. That is basically spoken the idea behind Wikiproject and the reason why they were founded in the past.

Suppose the idea is to delete all wikiprojects, what is the future participations of medical students in Wikipedia? On the first one a deletion, would block all efforts to contribute to Wikipedia especially if the own domain is focussed on a single subject. Perhaps it make sense to go a step backward and describe what science in general is about. The idea behind science is to become a specialist on a single subject. The idea is to reduce the scope. The question is, where is the right place in the Wikipedia project for doing so?

The answer is very simple: Reducing the scope is done with keywords. A keyword like “Immune System” is more specialized than the general term “Medicine”. Contributing to the Wikipedia project is possible with article request, maintenance request, deletion request, peer review request, and photo request. That means, the user has to open the page and put in the term into that page. For example, the article request page contains of all the domains, like physics, literature and medicine. What the user is allowed to do is to put his specialized keyword into the section “medicine” in the article request page.

This sounds a bit complicated but it's equal to tag a request. That means, all the request are handled as request but they are tagged with domains like physics, medicine and literature. This kind of interaction provides the same feature like a wikiproject but is adressed to a broader audience. The advantage is, that all the users have to observe the article request page. That means, even non medical experts are allowed to enter new article requests.

The wikiproject concept is equal to a decentralized project coordination. The idea is to split the Wikipedi a into sub-sections which are domain specific. This produces a lot of inefficiency. The better is to centralize the requests and tag a request with it's domain. That means, if a user likes to improve the project, there is only one page available which is the request directory. And all the issues are put into this single page.

The effect is, that the page view is higher, more people will monitor this page and the productivity is better. Wikipedia is on a path towards this goal. Since mid 2019 many portals are deleted already and in the future, the wikiprojects will follow. The switch from decentralized requests for writing new articles into a centralized request page will make Wikipedia more professional.

Let us imagine how a medical student can contribute to Wikipedia. What the user has to do is to find a very complicated medical term which has no WIkipedia article right now. This complicated term is put into the article request page, because Wikipedia should be explored into that direction. The same user can write the article for the term, and if he is done, he deletes the term from the list. The trick is to put only terms on the list, which are specialized. That means, it is not directed towards a mainstream audience but the amount of papers about the term is very low. The resulting article will attract very few readers. At the same time the reputation for creating such an article is high. What researchers are doing is to aggregate knowledge about a complicated seldom used term from a specialized domain.

Let us take a look into the reality to determine if the proposed workflow make sense. https://en.wikipedia.org/wiki/Wikipedia:Requested_articles/Medicine is the request page for all medical terms. It has a pageview of 2 per day. Which is very low. It provides a links to other languages like English. And it has some keywords like “Marshall protocol” which is a specialized subject within immune system diagnosis. According to the changelog the page isn't edited very often.

In contrast, the Wikiproject Medicine has a dailypage view of 86. And in the talk section, lots of domain specific discussion is available. It seems, that today, the decentralized wikiproject medicine is more attractive to the users, than the centralized version. The problem with the Wikiproject medicine is, that even this specialized portal doesn't fulfill the needs of the users. So they have created many subprojects: Wikiproject Anatomy, Wikiproject Physiology and so on. THe result is some kind of WIkiproject spam, in which the amount of projects is growing, but the team behind it, is doing nothing. On the other hand, i think the Wikiproject idea is a good possibility to learn because it shows, what the users are interested in. In most cases the idea is to specialize on a single subject which is a good idea because this will bring science forward. Let us click on the item Wikiproject anatomy and observe what comes next. In the anatomy section a new subfolder waits for the user, it's called Category:Anatomy articles by topic, that means, the user can decide if likes to read articles about subsection of anatomy.

Wikipedia is restructuring it's portals

https://en.wikipedia.org/wiki/Wikipedia:Miscellany_for_deletion/Archived_debates/September_2019

Wikipedia contains of the mainpage, which has a large amount of traffic and from the main page the users are directed to subpages, called portals. There are portals about art, mathematics, physics and sports. Since around August 2019 there was a discussion started to delete all portals. Or to be more specific, all the 500 portals are discussed individual to keep them or delete them.

The reason for deletion is mostly the same. The page view of the portals is low, the last edit was made 5 years ago, and the interaction on the portal page is low. A similar concept to portals are wikiproject which are also subpages to coordinate the efforts about the same topic, for example about computer science. Some of the wikiprojects were deleted too, but it seems that the deletion energy is focussed first on portals.

The discussion can be described from a more abstract point of view. If the portals and wikiprojects are gone, where is the place to discuss about new articles, maintainance and peer review request? There is a place, called “request directory”. In that domain, all the domains (art, sport, science, film) are combined under a single page. That means, in the article request page, the different subjects are combined: mathematics has a subsection, literature has one and so forth. According to the pageviews, such maintenance pages are used very often by the users.

The deletion of the former portal pages is a major step in the development of Wikipedia. On the first look a portal make sense. It helps to combine different articles under a single page. The concept is comparable to a specialized library. That means, under the term “computing” only computer experts are discussing how to write content from that domain. So it's surprising that this concept has failed.

In the deletion debate, only the majority of users is pro deletion. In contrast to normal deletion debate the opposite opinion is very low. So the prediction is, that in 2020 all the other portals and perhaps the wikiprojects too gets deleted.

To understand how the meta section is working we have to take a look at the remaining Wikiproject computerscience, https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Computer_science

Right now, the Wikiproject wasn't deleted, so it's a good time to observe the idea behind it. SImilar to a portal, a Wikiproject aggregates the efforts for a single domain. It described how many high quality and low quality articles are available in the domain of computer science, and it has a to do list.

The to do list contains of sections for article request, cleanup, expand existing articles, infoboxes, photo request and stubs. There is a also a list of participants who have put their username into a list as an indicator that they are motivated to contribute to the wikiproject.

The perhaps most important part of the Wikiproject computerscience is the to do list which includes article request, cleanup and so on. The interesting point is, that every wikiproject has such a list. But the list is filtered by the domain. The more general idea is, to take a look at the Wikipedia wide general to do list, which contains the same categories and combines all the domains in a single page.

So we can say, that portals and wikiprojects are obsolete and will be replaced by the general to do list to maintain all the domains like computer science, sports, films and so on.

specialized library

How can it be, that in the year 2005 many hundred of portals were established in the WIkipedia and now the same community is motivated to delete all of them? It's about how to organize knowledge outside the Wikipedia project. Suppose, there is no Internet available and a classical library is used to create an article about a subject. In the university domain, a specialized library was the normal way in doing so. The advantage of a specialized library is, that it provides only a small number of books in a single building. This reduces the costs. It's possible to create a specialized library with printed books about a single topic for example computer science. The amount of journals, books and dissertations about this subject is small. It's possible to collect all of them and put them into the library.

This was the workmode before the Internet was invented. If somebody was interested in getting an overview about the topic or likes to create a new paper, he was going to a specialized library. This was without any doubt the motivation in 2005 to establish portal pages in the Wikipedia. It copies the well working principle from the offline world.

In 2019 the situation is different. Most information are stored online, and specialized printed libraries are under pressure. What the experts users in the university are doing to today is to visit a general library and use Internet for getting access to specialized papers. The same is true for Wikipedia users. Most of them are working with fulltext search engine to get the information they need. The most used entry page is not a specialized search engine for a certain domain, but a search engine works by entering the needed keyword. The same search engine allows to browse in different subjects. This makes a domain specific portal obsolete.

Expanding WIkipedia

In the self description, a portal provides an entry page for a domain, which is adressed to the readers, and a wikiproject is part of a portal to coordinate the effort of editors to improve the articles. The idea is, that it's not possible to coordinate the maintainance of all the 5 millions articles in the Wikipedia, so the task is split into domains likes art, history, science and music.

A large scale project is Wikiproject history, https://en.wikipedia.org/wiki/Wikipedia:WikiProject_History SImilar to other other wikiproject it looks a bit inactive. But we are ignoring the low traffic and take a look what the self-understanding of the project is. The project goals are to improve the history articles in the Wikipedia by creating new ones, expanding old ones and improve the quality if needed. Also the goal is to serve as a central discussion point and to answer queries from the reference desk.

The interesting fact is, which kind of topic is offtopic at the history wikiproject. Everything outside the domain of history. That means, if somebody likes to expend an article about robotics he won't get help in the history project. This is logical but it explains what a possible bottleneck is. But let us go back to the goals. Create new articles and improve the quality of existing one is an important task in Wikipedia. It's not possible to ignore this goal but this would be equal to a failure of the Wikipedia in general. So the question is how to do this task more efficient?

The best way in doing so by formulating requests from the environment. That means, a user tries to find an article about a subject, is disappointed because the article is missing and then he formulates a request like “i need an article about topic abc. Please create one or explain to me, why the topic isn't available in WIkipedia”. There are two options how to handle such requests. One option is to focus on the subject or to focus on the request in general. A wikiproject is focus on the subject. That means, if a user formulates a request about a missing article from the subject history he has to ask the history section, and if he likes to read something about computer science, he has to go to a different wikiproject.

The more efficient way for interaction is a centralized request page, in which all the domains are combined. For doing so, the existing portals and wikiprojects have to be deleted, while the request directory should be improved and become more user friendly. A centralized request desk allows to improve all the articles in the Wikipedia.