first snapquery results for qlever ticket queries

[WolfgangFahl]

We have imported all queries mentioned in qlever github issues (tickets) to snapquery. There is a namespace for these queries now.

A preliminary result is

I am working on a paper for EKAW 2024 to report on this - feel free to contact me if you'd like get access to a preprint.

[hannahbast]

@WolfgangFahl Can you explain what "qlever ticket" is? And yes, please do send a preprint.

[WolfgangFahl]

@hannahbast thx for the swift response https://github.com/WolfgangFahl/snapquery/blob/main/tests/test_qlever.py and https://github.com/WolfgangFahl/snapquery/blob/main/snapquery/qlever.py

loop over all qlever tickets/issues and retrieve any short_urls pointing to SPARQL queries. The SPARQL Queries are then collected and prepared for import as Named Queries see https://github.com/WolfgangFahl/snapquery/blob/main/snapquery/samples/qlever.json

an example is

 {
      "query_id": "QLever.Ticket#1339-query0",
      "namespace": "qlever.cs.uni-freiburg.de/wikidata/tickets",
      "name": "Ticket#1339-query0",
      "sparql": "PREFIX wd: <http://www.wikidata.org/entity/>\nPREFIX wdt: <http://www.wikidata.org/prop/direct/>\nSELECT * WHERE {\n  ?person wdt:P31 wd:Q5 .\n  ?pesron wdt:P106 ?occupation .\n}",
      "url": "https://qlever.cs.uni-freiburg.de/wikidata/h6p82D",
      "title": "QLever github issue #1339-query0",
      "description": "Better error message for accidental Cartesian-product join",
      "comment": "See ticket https://github.com/ad-freiburg/qlever/issues/1339 and query https://qlever.cs.uni-freiburg.de/wikidata/h6p82D"
    },

Currently there is a mistake in the handling of the queries the namespaces and names contain characters with special url measing such as / and #. I intend to change that ASAP to make the examples better accessible in our UI.

Snapquery will test the namedqueries against different endpoints and show the results. The idea is to make queries endpoint independent and hide the technical details to make refactoring simpler and avoid "Query Rot".

[WolfgangFahl]

The screenshot above is a bit misleading

is better since it shows that most of the queries run on the qlever endpoint as they should. The assumption is that the other queries are just counter-examples. We have asked @joka921 to rate some of the queries manually whether they should work or not - we intend to let an LLM guess from the ticket text whether the queries should run or not and then be able to categorize the queries properly as positive or counter examples and let the tool display the overall "health state" of the query set.

Also the wikidata examples are somewhat interesting - quite a few of those use blazegraph specific features and we intend to create platform independent versions of those. The same holds true for the scholia query set which has multiple potentialy query rot future problems - e.g. motivated by the wikidata graph split or by simply wanting to use a different endpoint e.g. migrating from blazegraph to QLever. - More details are in the upcoming paper draft. The abstract is due July 1st and the final version July 12th. It would be great to get feedback from you.

[WolfgangFahl]

There is now a namespace performance-dblp for https://github.com/ad-freiburg/qlever/wiki/QLever-performance-evaluation-and-comparison-to-other-SPARQL-engines