Skip to content

Reaping the harvest

Harvest Templates by User:Pasleim  has been a key tool for synchronizing data between Wikipedia templates and Wikidata. However, Pasleim stopped updating the tool in 2021, and issues have been piling up since.

So I (and my not-so-trusted slop generator) set out to Do The Thing: Rewrite it in Rust. And while at it, fix the issues that had accumulated. And implement the feature requests.

The rewrite it now live at harvesttemplates.toolforge.org, the source is on github (since Codeberg frowns at code that wasn’t written in the One Proper Way). Some highlights:

  • The “harvests” now run on the server, not in the browser, so they keep running in the background, can be paused, restarted, etc.
  • Much of what used to be API queries that would timeout is now done via the database replicas, so no more hanging “Loading…”, and huge queries work fine (though they can take a minute)
  • Editing will not add statements twice, which was a much-bemoaned issue. Also, HT now checks by default if that statement was ever removed before, and won’t add it again. Each row gets its own edit status, and won’t fail the batch. You can also preview edits as a dry run.
  • Now supporting qualifiers, coordinates, numbers with units, combining several parameters, values from nested templates, multiple properties per run, as well as PetScan, SPARQL, and instance-of filters
  • Your old permalinks, and the shares queries, have been ported to the new DB, so they will still work (when adjusted for the new URL path)

So, if you liked the original Harvest Templates, or got curious and want to try it out (there is a help page, and some examples), I hope you’ll give the new version a whirl.