Monday, September 25, 2023
Unix philosophy contains the notion that each program should perform an single function (and perform that function exceptionally well), and then be used together with other single-function programs to form a powerful “toolbox”, with tools connected via the geek-famous pipe (“|”). The Wikimedia ecosystem has lots of tools that perform a specific function, but there […]
So the Wikidata query service (WDQS), currently powered by Blazegraph, does not perform well. Even simple queries time out, it lags behind the live site, and individual instances can (and do) silently go out of sync. The WMF is searching for a replacement. One of the proposed alternatives to blazegraph is Virtuoso. It models the […]
Lists. The plague of managing things. But also surprisingly useful for many tasks, including Wikimedia-related issues. Mix’n’match is a list of third-party entries. PetScan generates lists from Wikipedia and Wikidata. And Listeria generates lists on-wiki. But there is a need for generic, Wikimedia-related, user-curated lists. In the past, I have tried to quell that demand […]
Wednesday, January 25, 2023
A large part of Wikidata is the collection of external identifiers for items. For some item types, such as items about people (Q5), some of this is what is known as Authority Control (AC) data, for example, VIAF (P214). One thing that distinguishes AC data from other external IDs is that AC data sources are […]
Wednesday, November 2, 2022
So Toolforge is switching from grid engine to Kubernetes. This also means that tool owners such as myself need to change their tool background jobs to the new system. Mix’n’match was my tool with the most diverse job setup. But resource constraints and the requirement to “name” jobs meant that I couldn’t just port things […]
Thursday, November 12, 2020
My Listeria tool has been around for years now, and is used on over 72K pages across 80 wikis in the Wikimediaverse. And while it still works in principle, it has some issues, an, being a single PHP script, it is not exactly flexible to adapt to new requirements. Long story short, I rewrote the […]
One of my most-used WikiVerse tools is PetScan. It is a complete re-write of several other PHP-based tools, in C++ for performance reasons. PetScan has turned into the Swiss Army Knife of doing things with Wikipedia, Wikidata, and other projects. But PetScan has also developed a few issues over time. It is suffering from the […]
QuickStatments is a workhorse for Wikidata, but it had a few problems of late. One of those is bad performance with batches. Users can submit a batch of commands to the tool, and these commands are then run on the Labs server. This mechanism has been bogged down for several reasons: Batch processing written in […]
I have written about my attempt with Rust and MediaWiki before. This post is an update on my progress. I started out writing a MediaWiki API crate to be able to talk to MediaWiki installations from Rust. I was then pointed to a wikibase crate by Tobias Schönberg and others, to which I subsequently contributed […]
Rust is a up-and-coming programming language, developed by the Mozilla Foundation, and is used in the Firefox rendering engine, as well as the Node Package Manager, amongst others. There is a lot to say about Rust; suffice it to say that it’s designed to be memory-safe, fast (think: C or better), it can compile to […]