Skip to content

Commit

Permalink
feat: Import dumps to SPARQL endpoint
Browse files Browse the repository at this point in the history
  • Loading branch information
ddeboer committed Jul 4, 2024
1 parent 0cd890d commit 6d05549
Show file tree
Hide file tree
Showing 6 changed files with 13,080 additions and 12,671 deletions.
22 changes: 18 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ A pipeline consists of one or more **stages**. Each stage has:
- one or more **generators**, which generate triples about each URI using SPARQL CONSTRUCT queries.

Stages can be chained together, with the output of one stage becoming the input of the next.
The output of each stage combined becomes the final output of the pipeline.

### Design principles

Expand Down Expand Up @@ -133,18 +134,31 @@ To query large local files, you may need to load them into a SPARQL store first.
for example Oxigraph:
```shell
docker run --rm -v $PWD/data:/data -p 7878:7878 oxigraph/oxigraph --location /data serve
docker run --rm -v $PWD/data:/data -p 7878:7878 oxigraph/oxigraph --location /data serve --bind 0.0.0.0:7878
```
Then configure the store in your pipeline:
Then configure the store in your pipeline, configuring at least one store under `stores`
and using the `importTo` parameter to import the `endpoint`’s data to the store,
referencing the store’s `queryUrl`:
```yaml
# config.yml
stores:
- queryUrl: "http://localhost:7878/query"
storeUrl: "http://localhost:7878/store"
- queryUrl: "http://localhost:7878/query" # SPARQL endpoint for read queries.
storeUrl: "http://localhost:7878/store" # SPARQL Graph Store HTTP Protocol endpoint.
stages:
- name: ...
iterator:
query: ...
endpoint: file://../../../static/tests/iris.nt
importTo: http://localhost:7878/query
generator:
- query: ...
```
The data is loaded into a named graph `<import:filename>`, so in this case `<import:iris.nt>`.
#### Example configuration
```yaml
Expand Down
4 changes: 2 additions & 2 deletions jest.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ export default {
coverageReporters: ['json-summary', 'text'],
coverageThreshold: {
global: {
lines: 71.15,
statements: 71.25,
lines: 70.9,
statements: 71.01,
branches: 66.66,
functions: 76.59,
},
Expand Down
Loading

0 comments on commit 6d05549

Please sign in to comment.