Wk source zips #6
Conversation
Deploying languageapi with
|
| Latest commit: |
1bb22ab
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://61fe2e14.languageapi.pages.dev |
| Branch Preview URL: | https://wk-source-zips.languageapi.pages.dev |
There was a problem hiding this comment.
@PurpleGuitar @danparisd - I might should have put this in a different PR than just attaching it source zips, but being that both will need reworking if we adjust architecture, then I figured its fine to just tack it on here.
controller/src/db/schema/schema.ts
Outdated
There was a problem hiding this comment.
@PurpleGuitar @danparisd - this is an initial proposition for tracking localization of bible book names and of words like "tw" -> translation words. As seen, I figured the most logical unique index (and in this case I just made it the composite primary key) is the ietf already in the database plus a "key" for the string.
The one question I wonder about is do we want to add something like a "domain" to this table as well. I.e. the domain of bible books woudl be "bible_book" and then of "tn" would be something like resource types. There aren't currently conflicts between things like Gen and tn, and I'm not aware of any potential conflicts since the bible book slugs are set in englihs. But thoughts?
Forgot to add the lines: Those are here: https://github.com/WycliffeAssociates/languageapi/pull/6/files/afea3a4b1556f3db8b0f4ccadd3d3547a845c8eb#diff-ea2dd11638ccb90cd55880ec76d010447b0b2737763ba9563fac7455956d9096R321-R338
| // const query = sql.raw(`SELECT book_name, book_slug, ietf_code, id | ||
| // FROM (SELECT book_name, book_slug, l.ietf_code, c.id, | ||
| // ROW_NUMBER() OVER (PARTITION BY l.ietf_code, book_slug ORDER BY book_slug) AS rn | ||
| // FROM scriptural_rendering_metadata AS srm | ||
| // JOIN rendering AS r ON r.id = srm.rendering_id | ||
| // JOIN content AS c ON r.content_id = c.id | ||
| // JOIN language AS l ON l.ietf_code = c.language_id | ||
| // JOIN git_repo AS gr ON c.git_id = gr.id | ||
| // WHERE gr.username ILIKE '%wa-catalog%' | ||
| // AND c.domain = 'scripture' | ||
| // AND book_slug IS NOT NULL | ||
| // ) AS subquery | ||
| // WHERE rn = 1 | ||
| // ORDER BY ietf_code, book_slug;`); |
There was a problem hiding this comment.
@PurpleGuitar @danparisd - I've done it with the ORM instead of raw sql, but I imaigne the commented out raw sql might be more readable. In short, this query has those hardcoded dependencies on the git repo username being wa-catalog there. We only need one localized version for each slug (e.g. Gen, Exo), hence the select from rn =1 for each partition.
The result looks like this:
| app.timer("manageLocalizationTable", { | ||
| schedule: "0 0 0 * * *", | ||
| handler: populateLocalization, | ||
| useMonitor: false, | ||
| }); |
There was a problem hiding this comment.
We can ruin this not on a cron, but for now outside of setting up something more complicated such as doing event driven inserts based on the metadata table, this is a really straightforward way, and this day surely isn't gonna be that time critical I'd image.
controller/src/localizations/en.ts
Outdated
| const en = { | ||
| tw: "Translations Words", | ||
| tn: "Translation Notes", | ||
| }; | ||
| export type keysType = keyof typeof en; | ||
| export default {dict: en, ietf: "en"}; |
There was a problem hiding this comment.
For now, I know that you mentioned doing this in crowdin, and we could certainly do it via api most likely. For now though, I've put these into TS files. We probably need to decide what's the scope of these to translate, and moreover, there's some junky data currently in content resource types that needs cleaning up where the resource type is clearly not somethign we would consider a resource type. Probably worth having a discussion on .
|
Re: the source.zip list, I think this is a great first pass. If I read it right, it should return a JSON document containing all the source zips with some metadata. Looks good to me. 👍 |
…ting renderings table
Revert to only en
wkelly17
force-pushed
the
wk-source_zips
branch
from
e72296e
to
e9f6f08
Compare
@PurpleGuitar , @danparisd - This is branched off dev before merge, so it looks like 5 commits, but this is really just one:
9d38750.
This pr adds a view for Mark's requested source_zips if they are actually what they wants. Replicated here with some notes:
Not sure if this what you want or not exactly Craig, and I can adjust it to add on full language name or something if Orature or other needs that pretty easy.
Some notes.
Here is an example result:
If this isn't what you had in mind, we can certainly pivot, but this is at least what I understand that Mark/Orature was asking for.