{"id":547,"date":"2018-07-06T16:48:34","date_gmt":"2018-07-06T15:48:34","guid":{"rendered":"http:\/\/blog.magnusmanske.de\/?p=547"},"modified":"2018-07-06T16:48:34","modified_gmt":"2018-07-06T15:48:34","slug":"more-topics","status":"publish","type":"post","link":"http:\/\/magnusmanske.de\/wordpress\/archives\/547","title":{"rendered":"More topics"},"content":{"rendered":"

After my recent blog post<\/a> about the TopicMatcher<\/a> tool, I had quite a few conversations about the general area of “main topic”, especially relating to the plethora of scientific publications represented on Wikidata. Here’s a round-up of related things I did since:<\/p>\n

As a first attempt, I queried all subspecies items from Wikidata, searched for scientific publications, and added them to TopicMatcher<\/a>.<\/p>\n

That worked reasonably well, but didn’t yield a lot of results, and they need to be human-confirmed. So I came at the problem the other way: Start with a scientific publication, try to find a taxon (species etc.) name, and them add the “main subject” match. Luckily, many such publications put taxon names in () in the title. Once I have the text in between, I can query P225<\/a> for an exact match (excluding cases where there are more than one!), and then add “main subject” directly to the paper item, without having to confirm it by a user. I am aware that this will cause a few wrong matches, but I imagine those are few and far between, can be easily corrected when found, and are dwarfed by the usefulness of having publications annotated this way.<\/p>\n

There are millions of publications to check, so this is running on a cronjob, slowly going through all the scientific publications on Wikidata. I find quite a few topic in () that are not taxa, or have some issue with the taxon name; I am recording those, to run some analysis (and maybe other, advanced auto-matching) at a later date. So far, I see mostly disease names, which seem to be precise enough to match, in many cases.<\/p>\n

Someone suggested to use Mix’n’match sets to find e.g. chemical substances in titles that way, but this requires both “common name” and ID to be present in the title for a sufficient degree of reliability, which is rarely the case. Some edits<\/a> have been made for E numbers, though. I have since started a similar mechanism running directly off Wikidata (initial results<\/a>).<\/p>\n

Then, I discovered some special cases of publications that lend themselves to automated matching, especially obituaries<\/a>, as they often contain name, birth, and death date of a person, which is precise enough to automatically set the “main subject” property. For cases where there is no match found, I add them to TopicMatcher<\/a>, for manual resolution.<\/p>\n

I have also added “instance of:erratum” to ~8,000 papers indicating this from the title. This might be better places in “genre”, but at least we have a better handle on those now.<\/p>\n

Both errata and obituaries will run regularly, to update any new publications accordingly.<\/p>\n

As always, I am happy to get more ideas to deal with this vast realm of publications versus<\/em> topic.<\/p>\n","protected":false},"excerpt":{"rendered":"

After my recent blog post about the TopicMatcher tool, I had quite a few conversations about the general area of “main topic”, especially relating to the plethora of scientific publications represented on Wikidata. Here’s a round-up of related things I did since: As a first attempt, I queried all subspecies items from Wikidata, searched for […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9],"tags":[],"class_list":["post-547","post","type-post","status-publish","format-standard","hentry","category-wikidata"],"_links":{"self":[{"href":"http:\/\/magnusmanske.de\/wordpress\/wp-json\/wp\/v2\/posts\/547","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/magnusmanske.de\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/magnusmanske.de\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/magnusmanske.de\/wordpress\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/magnusmanske.de\/wordpress\/wp-json\/wp\/v2\/comments?post=547"}],"version-history":[{"count":1,"href":"http:\/\/magnusmanske.de\/wordpress\/wp-json\/wp\/v2\/posts\/547\/revisions"}],"predecessor-version":[{"id":548,"href":"http:\/\/magnusmanske.de\/wordpress\/wp-json\/wp\/v2\/posts\/547\/revisions\/548"}],"wp:attachment":[{"href":"http:\/\/magnusmanske.de\/wordpress\/wp-json\/wp\/v2\/media?parent=547"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/magnusmanske.de\/wordpress\/wp-json\/wp\/v2\/categories?post=547"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/magnusmanske.de\/wordpress\/wp-json\/wp\/v2\/tags?post=547"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}