Adding NIF 2.1 support

[jnehring]

NIF 2.1 support of the APIs should be moved forward. Needs to be implemented end of september together with provenance. But provenance relies on NIF 2.1 so we should start this topic soon.

I think it consists of the following subtasks:

1. Adjusting the outputs of e-Terminology, e-Translation, FREME NER to the new format
2. Maybe we create converters from NIF 2.0 -> NIF 2.1 and maybe back using a SPARQL filter. Maybe these converters can also be used in step 1
3. Necessary adjustments in e-Internationalization
4. e-Translation uses the output of e-Terminology so e-Translation needs adjustments to support NIF 2.1 input.
5. Brainstorming on other parts of FREME that are influenced by this, e.g. e-Link templates, existing workflows

@m1ci do you see another task?
@andish do you plan to add NIF 2.1 support to the Tilde APIs also? Do you have time to implement this in the next weeks?

[andish]

Yes, I can update our services to NIF 2.1 in next weeks.
@m1ci is somewhere changelog what must change to update from 2.0 to 2.1 version? As I rememmber there were 2 major things - new URI pattern and AnnotationUnit.

For e-Terminology all is clear. It is already using annotationUnits and must change only pattern of URIs.
For e-Translation annotationUnits not actual because it do not annotate anything. But must take in account Phil's proposal for MT segmentation info here: freme-project/Broker#169.

Provenance information will be added by FREME server?

[m1ci]

@m1ci is somewhere changelog what must change to update from 2.0 to 2.1 version? As I rememmber there were 2 major things - new URI pattern and AnnotationUnit.

Yes, I have compiled migration instructions in this document
https://docs.google.com/document/d/1uxT7GZN5YVw8QdmS2I95tI2cMaAITYnkiIsux71p1uQ/edit#

For e-Translation annotationUnits not actual because it do not annotate anything. But must take in account Phil's proposal for MT segmentation info here: freme-project/Broker#169.

The NIF 2.1 migration will also affect the e-Translation regarding the URI scheme. Note that, annotationUnits might be also relevant for e-Translation.

Provenance information will be added by FREME server?

I would prefer to handle the provenance information by the services.

[m1ci]

@jnehring

1. Maybe we create converters from NIF 2.0 -> NIF 2.1 and maybe back using a SPARQL filter. Maybe these converters can also be used in step 1

No need for sparql filter for the basic changes such as the URI scheme and datatypes.

@m1ci do you see another task?

no

Brainstorming on other parts of FREME that are influenced by this, e.g. e-Link templates, existing workflows

IMO this will not influence e-Link. Not sure about other workflows.

[andish]

Thanks @m1ci, this is usefull.

1. What is correct namespace declaration for nif prefix, for 2.1 version?
2. As we plan to use also annotationUnits for MT, what would be right class for translation? like nif:EntityOccurrence, nif:TermOccurrence.

[m1ci]

1. What is correct namespace declaration for nif prefix, for 2.1 version?

The same as now, http://persistence.uni-leipzig.org/nlp2rdf/ontologies/nif-core#

1. As we plan to use also annotationUnits for MT, what would be right class for translation? like nif:EntityOccurrence, nif:TermOccurrence.

There is no such class for translations.
You can express the language with the language tag @en, @de, ... or using the nif:predLang property (http://persistence.uni-leipzig.org/nlp2rdf/ontologies/nif-core/nif-core.html#d4e964).

I recommend using the nif:predLang. Some examples you can find in https://docs.google.com/document/d/1AzR2P7SBR-L7IJ_vsGNbDx21bXnxky9bSwqC8rSGuAw/edit

[m1ci]

Hey @andish,

sorry for the noise, but actually there is class for translations, its nif:Translation.
See its formal definition and the definitions of the related translation properties in https://github.com/NLP2RDF/ontologies/blob/nif2.1/nif-core/nif-core.ttl#L769-L810

Here is also an example for translations:

@prefix nif: <http://persistence.uni-leipzig.org/nlp2rdf/ontologies/nif-core#> .
@prefix itsrdf: <http://www.w3.org/2005/11/its/rdf#> .
@prefix dbp: <http://dbpedia.org/resource/> .
@prefix isolang: <http://www.lexvo.org/id/iso639-3/> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix ex: <http://example.org/nif-21/translation-berlin#> .

ex:doc_offset_0_18
  a nif:OffsetBasedString, nif:Context ;
  nif:beginIndex "0"^^xsd:nonNegativeInteger ;
  nif:endIndex "17"^^xsd:nonNegativeInteger ;
  nif:isString "Welcome to Berlin!"^^xsd:string ;
  nif:predLang isolang:eng ;
  nif:translation ex:trans_de_offset_0_21, ex:trans_es_offset_0_21 .

ex:doc_offset_11_17
  a nif:OffsetBasedString ;
  nif:beginIndex "11"^^xsd:nonNegativeInteger ;
  nif:endIndex "17"^^xsd:nonNegativeInteger ;
  nif:anchorOf "Berlin"^^xsd:string ;
  nif:referenceContext ex:doc_offset_0_18 ;
  itsrdf:taIdentRef dbp:Berlin .

ex:trans_de_offset_0_21
  a nif:Translation, nif:Context ;
  nif:beginIndex "0"^^xsd:nonNegativeInteger ;
  nif:endIndex "21"^^xsd:nonNegativeInteger ;
  nif:isString "Willkommen in Berlin!"^^xsd:string ;
  nif:predLang isolang:deu ;
  nif:wasTranslatedFrom ex:doc_offset_0_18 ;
  nif:translationProv <http://aksw.org/MarkusAckermann.rdf> ;
  nif:translationConf "1"^^xsd:decimal .

ex:trans_de_14_2
  a nif:OffsetBasedString ;
  nif:beginIndex "14"^^xsd:nonNegativeInteger ;
  nif:endIndex "21"^^xsd:nonNegativeInteger ;
  nif:anchorOf "Berlin"^^xsd:string ;
  nif:referenceContext ex:trans_de_offset_0_21 ;
  itsrdf:taIdentRef dbp:Berlin .

ex:trans_es_offset_0_21
  a nif:Translation, nif:Context ;
  nif:beginIndex "0"^^xsd:nonNegativeInteger ;
  nif:endIndex "21"^^xsd:nonNegativeInteger ;
  nif:isString "¡Bienvenido a Berlín!"^^xsd:string ;
  nif:predLang isolang:spa ;
  nif:translationProv <https://translate.google.com> ;
  nif:translationConf "0.8"^^xsd:decimal .

ex:trans_es_14_20
  a nif:OffsetBasedString ;
  nif:beginIndex "14"^^xsd:nonNegativeInteger ;
  nif:endIndex "21"^^xsd:nonNegativeInteger ;
  nif:anchorOf "Berlín"^^xsd:string ;
  nif:referenceContext ex:trans_es_offset_0_21 ;
  itsrdf:taIdentRef dbp:Berlin .

[jnehring]

This issue is solved