How to handle abstracts with JATS tags

[brunoamaral]

Journal Article Tag Suite (JATS) is a specification for structuring data in science papers.

Right now, Gregory MS has 3,316 articles with JATS tags from a total of 14,573. This wouldn't be an issue if feeds like PubMed gave out the full abstract.

Querying crossref.org like we usually do with the DOI number means we will get the full abstract but with JATS tags structuring that string.

An example

<jats:sec>
	<jats:title>Purpose</jats:title>
	<p:import href="http://transpect.io/jats2html/xpl/jats2html.xpl"/>
	<jats:p>This exploratory study sought to identify acoustic variables explaining rate-related variation in intelligibility for speakers with dysarthria secondary to multiple sclerosis.</jats:p>
</jats:sec>
<jats:sec>
	<jats:title>Method</jats:title>
	<jats:p>
		Seven speakers with dysarthria due to multiple sclerosis produced the same set of Harvard sentences at habitual and slow rates. Speakers were selected from a larger corpus on the basis of rate-related intelligibility characteristics. Four speakers demonstrated improved intelligibility and three speakers demonstrated reduced intelligibility when rate was slowed. A speech analysis resynthesis paradigm termed
		<jats:italic>hybridization</jats:italic>
		was used to create stimuli in which segmental (i.e., short-term spectral) and suprasegmental variables (i.e., sentence-level fundamental frequency, energy characteristics, and duration) of sentences produced at the slow rate were donated individually or in combination to habitually produced sentences. Online crowdsourced orthographic transcription was used to quantify intelligibility for six hybridized sentence types and the original habitual and slow productions.
	</jats:p>
</jats:sec>
<jats:sec>
	<jats:title>Results</jats:title>
	<jats:p>Sentence duration alone was not a contributing factor to improved intelligibility associated with slowed rate. Speakers whose intelligibility improved with slowed rate showed higher intelligibility scores for duration spectrum hybrids and energy hybrids compared to the original habitual rate sentences, suggesting these acoustic cues contributed to improved intelligibility for sentences produced with a slowed rate. Energy contour characteristics were also found to play a role in intelligibility losses for speakers with decreased intelligibility at slowed rate. The relative contribution of speech acoustic variables to intelligibility gains and losses varied considerably between speakers.</jats:p>
</jats:sec>
<jats:sec>
	<jats:title>Conclusions</jats:title>
	<jats:p>Hybridization can be used to identify acoustic correlates of intelligibility variation associated with slowed rate. This approach has further elucidated speaker-specific and individualized speech production adjustments when slowing rate.</jats:p>
</jats:sec>

Options available:

1. Keep the tags and let the end user decide how to proceed
2. Translate the tags to standard html

I like option 1 because it gives more information to the user. The downside is that the browser doesn't parse these tags. I feel that the correct way to move forward would be to keep the tags and provide information on how to style them on html, which at the moment is outside of my ability.

[antoniolopes]

I have nothing against leaving as it is (option 1), but you have to consider that the ML-model will have difficulty processing those tags. The model is only able to process clean text.

[brunoamaral]

Then we will have to pass the clean text version. I'm planning to refactor the db_maintenance tasks, and can do that while I'm at it.