{"id":498,"date":"2017-12-08T12:55:39","date_gmt":"2017-12-08T11:55:39","guid":{"rendered":"http:\/\/blog.magnusmanske.de\/?p=498"},"modified":"2017-12-08T12:55:39","modified_gmt":"2017-12-08T11:55:39","slug":"recommendations","status":"publish","type":"post","link":"http:\/\/magnusmanske.de\/wordpress\/archives\/498","title":{"rendered":"Recommendations"},"content":{"rendered":"<p>Reading <a href=\"https:\/\/meta.wikimedia.org\/wiki\/Research:Recommending_Images_to_Wikidata_Items\">Recommending Images to Wikidata Items<\/a> by <a href=\"https:\/\/meta.wikimedia.org\/wiki\/User:Miriam_(WMF)\">Miriam<\/a>, which highlights missing areas of image coverage in Wikidata (despite being the most <a href=\"https:\/\/docs.google.com\/spreadsheets\/d\/1G2P89WW_GAkqShvaWdhkiCN66WZB0synLCV9X0h6-Ig\/edit?usp=sharing\">complete site in the WikimediaVerse, image-wise<\/a>), and strategies to address the issue, I was reminded of an annoying problem I have run into a few times.<\/p>\n<p>My <a href=\"https:\/\/tools.wmflabs.org\/fist\/wdfist\">WD-FIST tool<\/a> uses (primarily) <a href=\"https:\/\/query.wikidata.org\/\">SPARQL<\/a> to find items that might require images, and that usually works well. However, some larger queries do time out, either on SPARQL, or the subsequent image discovery\/filtering steps. Getting a list of all items about <a href=\"https:\/\/tools.wmflabs.org\/fist\/wdfist\/index.html?depth=3&amp;language=en&amp;project=wikipedia&amp;sparql=SELECT%20?item%20WHERE%20{%20?item%20wdt:P31%20wd:Q5%20.%20?item%20wdt:P21%20wd:Q6581072%20}&amp;no_images_only=1&amp;remove_used=1&amp;remove_multiple=1&amp;prefilled=1&amp;page_image_only=1\">women with image candidates<\/a> occasionally works, but not reliably so; all humans is out of the question.<\/p>\n<p>So I started an extension to WD-FIST: A caching mechanism that would run some large queries in a slightly different way, on a regular basis, and offer the results in the well-known WD-FIST interface. My first attempt is &#8220;humans&#8221;, and you can see some results <strong><a href=\"https:\/\/tools.wmflabs.org\/fist\/wdfist\/index.html?depth=3&amp;language=en&amp;project=wikipedia&amp;remove_used=1&amp;prefilled=1&amp;wiki_candidates=Q5\">here<\/a><\/strong>. As of now, there are 275,500 candidate images for 160,508 items; the link shows you all images that are used on three or more Wikipedias associated with the same item (to improve signal-to-noise ratio).<\/p>\n<p>One drawback of this system is that it has some &#8220;false positive&#8221; items; because it bypasses SPARQL, it gets some items that link to &#8220;human&#8221; (Q5), but not via &#8220;instance of&#8221; (P31). Also, matching an image to an items, or using &#8220;ignore&#8221; on the images, might not immediately reflect on reload, but the daily update should take care of that.<\/p>\n<p>Update code is <a href=\"https:\/\/bitbucket.org\/magnusmanske\/fist\/src\/b2a5a89763f5903f3c619e6041ded319fff589e8\/update_wiki_candidates.php?at=master&amp;fileviewer=file-view-default\">here<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Reading Recommending Images to Wikidata Items by Miriam, which highlights missing areas of image coverage in Wikidata (despite being the most complete site in the WikimediaVerse, image-wise), and strategies to address the issue, I was reminded of an annoying problem I have run into a few times. My WD-FIST tool uses (primarily) SPARQL to find [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-498","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"http:\/\/magnusmanske.de\/wordpress\/wp-json\/wp\/v2\/posts\/498","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/magnusmanske.de\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/magnusmanske.de\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/magnusmanske.de\/wordpress\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/magnusmanske.de\/wordpress\/wp-json\/wp\/v2\/comments?post=498"}],"version-history":[{"count":1,"href":"http:\/\/magnusmanske.de\/wordpress\/wp-json\/wp\/v2\/posts\/498\/revisions"}],"predecessor-version":[{"id":499,"href":"http:\/\/magnusmanske.de\/wordpress\/wp-json\/wp\/v2\/posts\/498\/revisions\/499"}],"wp:attachment":[{"href":"http:\/\/magnusmanske.de\/wordpress\/wp-json\/wp\/v2\/media?parent=498"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/magnusmanske.de\/wordpress\/wp-json\/wp\/v2\/categories?post=498"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/magnusmanske.de\/wordpress\/wp-json\/wp\/v2\/tags?post=498"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}