Extract functions not extracting properly

[SirBac0n]

bulk_extract_corpora and extract_corpora do not remove all lemmas and strong numbers from translations such as hbo_uhb and others from Door43

[ddaspit]

Can you give a specific example (verse) from the project? Or does it occur for all wordlist (\w) markers?

[SirBac0n]

This is the original GEN 1:11 in hbo_uhb before extract:
\v 11

  |   | \w וַ⁠יֹּ֣אמֶר|lemma="אָמַר" strong="c:H0559" x-morph="He,C:Vqw3ms"\w*
  |   | \w אֱלֹהִ֗ים|lemma="אֱלֹהִים" strong="H0430" x-morph="He,Ncmpa" x-tw="rc:///tw/dict/bible/kt/god"\w
  |   | \w תַּֽדְשֵׁ֤א|lemma="דָּשָׁא" strong="H1876" x-morph="He,Vhj3fs"\w*
  |   | \w הָ⁠אָ֨רֶץ֙|lemma="אֶרֶץ" strong="d:H0776" x-morph="He,Td:Ncbsa" x-tw="rc:///tw/dict/bible/other/earth"\w
  |   | \w דֶּ֔שֶׁא|lemma="דֶּשֶׁא" strong="H1877" x-morph="He,Ncmsa"\w*
  |   | \w עֵ֚שֶׂב|lemma="עֶשֶׂב" strong="H6212" x-morph="He,Ncmsa"\w*
  |   | \w מַזְרִ֣יעַ|lemma="זָרַע" strong="H2232" x-morph="He,Vhrmsa" x-tw="rc:///tw/dict/bible/other/seed"\w
  |   | \w זֶ֔רַע|lemma="זֶרַע" strong="H2233" x-morph="He,Ncmsa" x-tw="rc:///tw/dict/bible/other/seed"\w
  |   | \w עֵ֣ץ|lemma="עֵץ" strong="H6086" x-morph="He,Ncmsc"\w*
  |   | \w פְּרִ֞י|lemma="פְּרִי" strong="H6529" x-morph="He,Ncmsa" x-tw="rc:///tw/dict/bible/other/fruit"\w
  |   | \w עֹ֤שֶׂה|lemma="עָשָׂה" strong="H6213a" x-morph="He,Vqrmsa"\w*
  |   | \w פְּרִי֙|lemma="פְּרִי" strong="H6529" x-morph="He,Ncmsa" x-tw="rc:///tw/dict/bible/other/fruit"\w
  |   | \w לְ⁠מִינ֔⁠וֹ|lemma="מִין" strong="l:H4327" x-morph="He,R:Ncmsc:Sp3ms" x-tw="rc:///tw/dict/bible/other/kind"\w
  |   | \w אֲשֶׁ֥ר|lemma="אֲשֶׁר" strong="H0834a" x-morph="He,Tr"\w*
  |   | \w זַרְע⁠וֹ|lemma="זֶרַע" strong="H2233" x-morph="He,Ncmsc:Sp3ms" x-tw="rc:///tw/dict/bible/other/seed"\w־\w ב֖⁠וֹ|lemma="" strong="b" x-morph="He,R:Sp3ms"\w*
  |   | \w עַל|lemma="עַל" strong="H5921a" x-morph="He,R"\w*־\w הָ⁠אָ֑רֶץ|lemma="אֶרֶץ" strong="d:H0776" x-morph="He,Td:Ncbsa" x-tw="rc:///tw/dict/bible/other/earth"\w
  |   | \w וַֽ⁠יְהִי|lemma="הָיָה" strong="c:H1961" x-morph="He,C:Vqw3ms"\w*־\w כֵֽן|lemma="כֵּן" strong="H3651c" x-morph="He,D"\w*׃

And this is GEN 1:11 after extract:
וַ⁠יֹּ֣אמֶר אֱלֹהִ֗ים תַּֽדְשֵׁ֤א הָ⁠אָ֨רֶץ֙ דֶּ֔שֶׁא עֵ֚שֶׂב מַזְרִ֣יעַ זֶ֔רַע עֵ֣ץ פְּרִ֞י עֹ֤שֶׂה פְּרִי֙ לְ⁠מִינ֔⁠וֹ אֲשֶׁ֥ר זַרְע⁠וֹ־ב֖⁠וֹ|lemma="" strong="b" x-morph="He,R:Sp3ms" עַל־הָ⁠אָ֑רֶץ וַֽ⁠יְהִי־כֵֽן׃

[ddaspit]

This is issue is really about adding support for Door43 projects.

[ddaspit]

See #456.