You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When loading the SpaCy models, all models are loaded even if they are not used. See file spacy_models.py line 19 and following.
When adding more languages or using lg models, this might become a bottleneck and slow down the extraction process significantly.
Suggestion:
Check which language is requested and only load the required model, e.g. by changing from line 58 to (removing spacy_model = spacy_models[lang]):
if lang == 'de':
spacy_model = de_core_news_md.load()
elif lang == 'en':
spacy_model = en_core_web_md.load()
...
Then you would also be able to remove line 19-26.
What do you think?
The text was updated successfully, but these errors were encountered:
When loading the SpaCy models, all models are loaded even if they are not used. See file spacy_models.py line 19 and following.
When adding more languages or using lg models, this might become a bottleneck and slow down the extraction process significantly.
Suggestion:
Check which language is requested and only load the required model, e.g. by changing from line 58 to (removing
spacy_model = spacy_models[lang]
):Then you would also be able to remove line 19-26.
What do you think?
The text was updated successfully, but these errors were encountered: