Skip to content

Musing on lists

As some of you may know, I write the occasional tool to help support Wikipedia, Wikidata, Commons, and other projects in the WikiVerse. Most of my tools work on the same basic principle: Get some data to start with, think about it, and present a result. The input data is often a list of pages (or Wikidata items, which is similar), defined by some sort of query.

Now, the number of potential sources for such lists have been multiplying over the years. Off the top of my head, I can think of:

  • Manual lists (paste in a box)
  • Lists on Wikis (numbered, unnumbered, with or without links, with comments, in tables etc.)
  • Category trees
  • Category tree intersections (e.g. QuickIntersection)
  • More complex intersections of category trees, templates, etc. (e.g. CatScan2)
  • Wikidata Queries
  • Complex intersections of categories, WDQ, lists etc. (e.g. AutoList)
  • SQL queries (e.g. Quarry)
  • SPARQL queries (e.g. WDQS)
  • All the tools that use any combination of the above, and generate page lists in return, could be sources again

The problem, however, is more complicated than this:

  • Most tools that process lists could potentially use most of the above sources, or combinations thereof, even if this is not apparent at first glance; a Wikidata tool can still use lists of French Wikipedia articles, as they can be “converted” into corresponding Wikidata items, and vice versa
  • Any of these sources can be combined in several ways; e.g. only pages that are in list A and (list B or list C)
  • These can be combined with non-list properties (last edited less than a month ago, excluding bots; created over 5 years ago; edited by one of these users; use the matching talk/content page; no redirects)
  • This can be done recursively; the same source “types” can be used several times, in a complex query

I have previously tried to allow users to construct a query pipeline, combining the outputs of different tools, and processing (e.g. filtering) them through more tools in new and interesting ways. However, that attempt was not taken up, neither by users nor tool developers.

I tried again to solve the issue, this time by putting the “pipeline” into JavaScript, running right in the users’ web browser. However, usage numbers (except for a single, quite active user) show that again, there was no uptake by users in general.

Maybe I am the only one in the WikiVerse thinking about this? Maybe my attempts are still too clunky for the average user? Maybe there is just no demand, and all the tools run perfectly fine as they are?

There seems to be some general interest in lists; my list generating bot appears to be reasonably popular with users on Wikipedia, albeit not in the article namespace. And an experimental, manual list-generating feature called Gather on mobile Wikipedia seems to be popular. Maybe I am just missing the “killer application” for lists, though the point is that all tools and applications could benefit from list management.