Skip to content

Category Archives: Rust

Merge and diff

Originally, I wanted to blog about adding new properties (taxon data speficially, NCBI, GBIF, and iINaturalist) to my AC2WD tool (originally described here). If you have the user script installed on Wikidata, AC2WD will automatically show up on relevant taxon items. But then I realized that the underlying tech might be useful to others, if […]

Get into the Flow

Unix philosophy contains the notion that each program should perform an single function (and perform that function exceptionally well), and then be used together with other single-function programs to form a powerful “toolbox”, with tools connected via the geek-famous pipe (“|”). The Wikimedia ecosystem has lots of tools that perform a specific function, but there […]

Musings on the backend

So the Wikidata query service (WDQS), currently powered by Blazegraph, does not perform well. Even simple queries time out, it lags behind the live site, and individual instances can (and do) silently go out of sync. The WMF is searching for a replacement. One of the proposed alternatives to blazegraph is Virtuoso. It models the […]

Lists. The plague of managing things. But also surprisingly useful for many tasks, including Wikimedia-related issues. Mix’n’match is a list of third-party entries. PetScan generates lists from Wikipedia and Wikidata. And Listeria generates lists on-wiki. But there is a need for generic, Wikimedia-related, user-curated lists. In the past, I have tried to quell that demand […]

Turn the AC on

A large part of Wikidata is the collection of external identifiers for items. For some item types, such as items about people (Q5), some of this is what is known as Authority Control (AC) data, for example, VIAF (P214). One thing that distinguishes AC data from other external IDs is that AC data sources are […]

Trust in Rust

So Toolforge is switching from grid engine to Kubernetes. This also means that tool owners such as myself need to change their tool background jobs to the new system. Mix’n’match was my tool with the most diverse job setup. But resource constraints and the requirement to “name” jobs meant that I couldn’t just port things […]

The Listeria Evolution

My Listeria tool has been around for years now, and is used on over 72K pages across 80 wikis in the Wikimediaverse. And while it still works in principle, it has some issues, an, being a single PHP script, it is not exactly flexible to adapt to new requirements. Long story short, I rewrote the […]

A Scanner Rusty

One of my most-used WikiVerse tools is PetScan. It is a complete re-write of several other PHP-based tools, in C++ for performance reasons. PetScan has turned into the Swiss Army Knife of doing things with Wikipedia, Wikidata, and other projects. But PetScan has also developed a few issues over time. It is suffering from the […]

Batches of Rust

QuickStatments is a workhorse for Wikidata, but it had a few problems of late. One of those is bad performance with batches. Users can submit a batch of commands to the tool, and these commands are then run on the Labs server. This mechanism has been bogged down for several reasons: Batch processing written in […]

Papers on Rust

I have written about my attempt with Rust and MediaWiki before. This post is an update on my progress. I started out writing a MediaWiki API crate to be able to talk to MediaWiki installations from Rust. I was then pointed to a wikibase crate by Tobias Schönberg and others, to which I subsequently contributed […]