Alternative strategy for Named Graphs

[sennierer]

We are starting a new project (in the Digital Humanities) and we are currently evaluating backend solutions to use. The project will be mainly about pulling data from various resources and harmonizing/linking these datasets. We have been using triplestores for these tasks before (blazegraph, jena fuseki, graphdb), but have always made extensive use of named graphs (to keep provenance data of the source datasets).
Given that QLever does not support Named Graphs my questions are:

• Is there work done on implementing Named Graphs?
• Is there any alternative that we can make use of that is supported by QLever (like RDF star)?
• Or is it feasible to create separate QLever instances for all the datasets and use federated queries to integrate them? What about performance using federated queries to lets say 20 different QLever instances?
  Thanks very much!

[hannahbast]

@sennierer Yes, named graphs are on our agenda. And yes, there are alternatives to realizing named graphs with the current version of QLever. How large is your dataset and can you provide a link to it? Then we could have a concrete look and maybe provide a concrete solution right away.

[sennierer]

Thanks very much for the fast reply. Actually we are currently working on the datasets that will be ingested. However, the datasets will look quite similar to the ones you can find here: https://github.com/InTaVia/source-data/tree/main/datasets . In addition to these datasets we generated datasets from Wikidata and Europeana, and did some enrichment/linking directly on the graphs with pipelines running in our cluster. Currently the production blazegraph of that project contains 33.828.265 triples (but we expect it to be more in the new project).

[sennierer]

@hannahbast do you have any estimation when named graphs will become available? Are we speaking of months, or years? We are about to start data preparation and if NGs are not going to come before 2025, we need another solution. Using eg crm:E13 as suggested by @aindlq or something else. Another possibility came to my mind: as the idea of our project is to combine data from different datasets in one triplestore, the instances from one dataset will share a common basedomain. Is it feasible in QLever to just filter on the URI of instances? Something along the lines of

SELECT * WHERE {

?s ?p ?o .
FILTER(STRSTARTS(STR(?s), "http://www.wikidata.org"))
}

Given that the datasets do not share URIs (at least not for the entities they describe) and are only linked via owl:sameAs this would actually do if it is performant.

[joka921]

@sennierer We currently have the NAMED Graph support under active development.

There already is a working PR for adding the Graph IRI of each triple to QLever's index structures (#1337), And I am currently working on an NQUAD parser. What conceptually is still missing is the usage of the Graph IRI with the appropriate queries, but that is on my list. So something like weeks (to a working prototype) to a few months (until it can be merged and is usable) sounds reasonable to expect.

As for your second suggestion concerning the FILTER: In principle this can be done and this is also somewhere on our list. We already have efficient binary search filters for Prefixes, but currently your would basically have to manually rewrite your query in a very cumbersome way. But thanks for reminding me of this also needed feature.

[sennierer]

Thanks for this clarification @joka921 . Thats very good to hear! We will wait for the implementation of the Named Graphs then!

[aindlq]

we have the same challenge in PHAROS. So any suggestions are very appreciated.

In my view there are two main use cases for named graphs, one to track provenance and second to query datasets separately or in different combination.

Unfortunately, as I understand it, the later use case can't be currently address with QLever, as I outlined it here #493 (comment) . Where with named graphs one can simply use FROM to limit SPARQL query to certain graphs:

SELECT ?artwork 
WHERE 
FROM <dataset/A>
FROM <dataset/B> {
    ?artwork crm:P108i_was_produced_by/crm:P32_used_general_technique/owl:sameAs/skos:broader* <http://vocab.getty.edu/aat/300053271> .
}

@hannahbast Is there any alternative way to do it with QLever?

@sennierer For provenance use case, we decided to maintain provenance for every RDF triple in the database using crm:E13_Attribute_Assignment from CIDOC-CRM (which is essentially, in a way, equivalent to normal RDF Reification with subject, predicate, object). Luckily QLever scales really well so it is feasible to do that even for pretty large datasets.

[sennierer]

Thanks very much. Actually the main use case for us is the latter. In a previous project we built a RestAPI on top of the triplestore and we used the named graphs to include/exclude datasets easily from the queries.
We used E13_Attribute_Assignment in other projects to have provenance on instance level, but it would also be nice to have an easy solution (eg via the NGs) to add provenance on dataset level.

[tuukka]

Since you're also working with Wikidata, it might be interesting to use Wikidata-style reification (two-part predicates and qualifiers).

[sennierer]

Thanks for the suggestion. Actually we converted data from wikidata (using federated queries, construct and update on our SPARQL endpoint) to our target ontology (which was based on cidoc crm/biocrm). So we could have used crm:E13, but actually never came to implement provenance on instance level.

[Stiksels]

@joka921 with PR #1444 merged, there is now a NQuad parser and it seems to work. However, the sparql server doesn't support the GRAPH and FROM NAMED clauses yet, so the named graphs are still queried as if they are all part from one large graph. Do you have a timeline for when these clauses are supported?

[dreerr]

ist there already an update on querying named graphs?

[Stiksels]

@dreerr the GRAPH clause is supported now, both with a graph uri or a variable as value

SELECT ?s ?p ?o ?g
WHERE {
  GRAPH ?g {
    ?s ?p ?o .
  }
}