{"id":617,"date":"2019-05-16T10:06:34","date_gmt":"2019-05-16T09:06:34","guid":{"rendered":"http:\/\/blog.magnusmanske.de\/?p=617"},"modified":"2019-05-17T16:20:57","modified_gmt":"2019-05-17T15:20:57","slug":"papers-on-rust","status":"publish","type":"post","link":"http:\/\/magnusmanske.de\/wordpress\/archives\/617","title":{"rendered":"Papers on Rust"},"content":{"rendered":"

I have written about my attempt with Rust and MediaWiki<\/a> before. This post is an update on my progress.<\/p>\n

I started out writing a MediaWiki API crate<\/a> to be able to talk to MediaWiki installations from Rust. I was then pointed to a wikibase crate<\/a> by Tobias Sch\u00f6nberg and others, to which I subsequently contributed some code, including improvements to the existing codebase, but also an “entity cache” struct (to retrieve and manage larger amounts of entities from Wikibase\/Wikidata), as well as an “entity diff” struct.<\/p>\n

The latter is something I had started in PHP before, but never really finished. The idea is that, when creating or updating an entity, instead of painstakingly testing if each statement\/label\/etc. exists, one simply creates a new, blank item, fills it with all the data that should be in there, and then generates a “diff” to a blank (for creating) or existing (for updating) entity. That diff can then be passed to the wbeditentity<\/a> API action. The diff generation can be fine-tuned, e.g. only add English labels, or add\/update (but not remove) P31 statements.<\/p>\n

Armed with these two crates, I went to re-create a functionality that I had written in PHP before: creation and updating of items for scientific publications, mainly used in my SouceMD tool<\/a>. The code underlying that tool has grown over the years, meaning it’s a mess, and has also developed some idiosyncrasies that lead to unfortunate edits.<\/p>\n

For a rewrite in Rust, I also wanted to make the code more modular, especially regarding the data sources. I did find a crate to query CrossRef<\/a>, but not much else. So I wrote new crates to query pubmed<\/a>, ORCID<\/a>, and Semantic Scholar<\/a>. All these crates are completely independent of MediaWiki\/Wikibase; they can be re-used in any kind of Rust code related to scientific publications. I consider them a sound investment into the Rust crate ecosystem.<\/p>\n

With these crates in a basic but usable state, I went to write papers<\/a>, Rust code (not a crate just yet) to gather data from the above sources, and inject them into Wikidata. I wrote a Rust trait to represent a generic source, and then wrote adapter structs for each of the sources. Finally, I added some wrapper code to take a list of adapters, query them about a paper, and update Wikidata accordingly. It can already<\/p>\n