Skip to content

Commit

Permalink
docs: add the Python Repo section into OEP-58 docs
Browse files Browse the repository at this point in the history 
document the new translation workflow

This contribution is part of the [FC-0012 project](https://openedx.atlassian.net/l/cp/XGS0iCcQ) which is sparked by the [Translation Infrastructure update OEP-58](https://open-edx-proposals.readthedocs.io/en/latest/architectural-decisions/oep-0058-arch-translations-management.html#specification).
  • Loading branch information
@OmarIthawi
OmarIthawi committed Sep 4, 2023
1 parent 047c7ed commit 619d20c
Showing 1 changed file with 186 additions and 11 deletions.
197 changes: 186 additions & 11 deletions source/developers/how-tos/enable-translations-new-repo.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,21 +5,196 @@ Enabling Translations on a New Repo
.. contents::
:local:
:depth: 2

===========
Python Repo
===========

TODO: How to enable translations infrastructure on a Python repo
============
Introduction
============

Enabling translations on a new repo is mostly starting from an up-to-date cookiecutter
such as the `frontend-template-application`_ for Microfrontends, `edx-cookiecutters`_ for Python repos.
The rest is to ensure that the repo is properly configured as shown in this guide.

The `OEP-58 - Translations Management`_ proposal introduced new workflows and tooling to the Open edX translation
management:

- Created the new `openedx-translations project`_ in Transifex.
- Created the new `openedx-translations repo`_ in GitHub.
- Setup two-way links between the Transifex project and the GitHub repo via the
`Transifex GitHub app <https://github.com/apps/transifex-integration>`_.
- Added new the `atlas <https://github.com/openedx/openedx-atlas>`_ command line tool to pull translations which will
be used in the repo Makefiles, build and deploy processes to fetch translations from the
`openedx-translations repo`_.

After configuring the repo as shown in the sections below, the translations workflow will automatically:

- Add the extracted source (English) translation files from the repository you added to the
`openedx-translations repo`_ in GitHub.
- Create a new Transifex resource for your repo in the `openedx-translations project`_ in Transifex.
- Add the translations (Arabic, French, etc) to the GitHub `openedx-translations repo`_ when translation entries are
updated by translators.

The full background and rationale for the new workflows and tooling can be found in the
`OEP-58 - Translations Management`_ docs.

The next sections cover the steps to enable translations on a new repo depending its type:

=================================
Python Django Plugins and XBlocks
=================================

This section covers repositories such as XBlocks, Open edX plugins, Python libraries and other Python repositories
except if it's a Django microservice which is covered in the next section.

To include a repo in the translations workflow follow the steps below:

1. If you are not working within an existing repository, create a new repo using the: `edx-cookiecutters`_
python-library for either ``django-app`` or ``xblock``.
If the repo already exists continue to the next steps.

2. For existing repos which don't have the ``make extract_translations`` command, it can be copied from the
`edx-cookiecutters`_ Makefile in the corresponding template e.g. ``cookiecutter-xblock`` Makefile for XBlocks.

**Note:** Some repositories uses ``django-manage makemessages``. The recommendation is to use ``i18n_tool extract``
because it provides more options and cleans ``.po`` files.
However, it should be fine to use either as long as the translation files are extracted and working .

3. Run the ``make extract_translations``. Verify that it extracts the files within the ``my_xblock_module/conf/locale``
directory. The PO file names will vary depending on the use case:

- The XBlocks will have a ``text.po`` extracted.
- Django plugins will have ``django.po``.
- Other plugins may include ``djangojs.po`` if the repo has a ``static`` directory with JavaScript use of
``gettext``.

4. Add the repository to `extract-translation-source-files.yml`_ in the `openedx-translations repo`_.

Add a new entry under the ``python-translations`` section. For example for the XBlock located in
``https://github.com/openedx/MyXBlock`` and has the XBlock Python module name ``my_ai_bot_xblock`` it should have
the following entry::

python-translations:
strategy:
matrix:
repo:
...
- repo_name: MyAIBotXBlock
python_module_name: my_ai_bot_xblock
...

When adding a theme or any other module that's not an Open edX plugin or an XBlock the entry should
added without ``python_module_name`` as demonstrated below::

python-translations:
strategy:
matrix:
repo:
...
- repo_name: credentials-themes
...

5. Create a pull request in the `openedx-translations repo`_ and wait for review and approval.
Once the pull request is merged the GitHub/Transifex workflow will be ongoing as described in the
Introduction section. No additional configuration is needed.

6. Verify the workflow is syncing the translations properly as described in the Debugging section below.

7. Install the XBlock or plugin in a devstack and run the ``OPENEDX_ATLAS_PULL=true make pull_translations`` in
the edx-platform repo to preview the translations in devstack.

This step is somewhat oversimplified and assumes you're familiar with the Open edX development workflow whether
you're using `Tutor <https://docs.tutor.overhang.io/>`_ or the
`Open edX devstack <https://github.com/openedx/devstack/>`_.

================================
Django Microservice Repos (IDAs)
================================

In the context of Open edX both of the Microservice and Independently Deployable Application (IDA) terms are used
interchangeably. There are many Django microservices in the Open edX ecosystem.
The most common ones are: ``edx-platform`` itself, ``credentials``, ``ecommerce`` and ``course-discovery``.

1. If you are not working within an existing repository, create a new repo using the `cookiecutter-django-ida`_
template. If the repo already exists continue to the next steps.

TODO: How to enable translations infrastructure on a Django microservice

5. Create a pull request in the `openedx-translations repo`_ and wait for review and approval.
Once the pull request is merged the GitHub/Transifex workflow will be ongoing as described in the
Introduction section. No additional configuration is needed.

6. Verify the workflow is syncing the translations properly as described in the Debugging section below.

7. Run the ``OPENEDX_ATLAS_PULL=true make pull_translations`` to verify that it pulls the translations from the
`openedx-translations repo`_ into ``conf/locale`` directory. Depending on the repo
``make static``/``make static.dev`` is usually needed to generate the JavaScript translation files.

===========
Django Repo
React Repos
===========

TODO: How to enable translations infrastructure on a Django repo

==========
React Repo
==========
1. If you are not working within an existing repository, create a new repo using the `frontend-template-application`_
template. If the repo already exists continue to the next steps.

TODO: How to enable translations infrastructure on a React repo

4. Run the ``make extract_translations``. Verify that it extracts the ``src/i18n/transifex_input.json`` file
in the micro-frontend repo. This file should be excluded from the repo via the ``.gitignore`` file.

5. Create a pull request in the `openedx-translations repo`_ and wait for review and approval.
Once the pull request is merged the GitHub/Transifex workflow will be ongoing as described in the
Introduction section.

6. Verify the workflow is syncing the translations properly as described in the Debugging section below.

7. Depending on how you deploy the micro-frontend, include the ``pull_translations`` make rule with the
``OPENEDX_ATLAS_PULL`` environment variable set to ``true`` e.g
``$ OPENEDX_ATLAS_PULL=true make pull_translations``.

This command needs to run before ``npm build`` in order to include updated translations in final micro-frontend build.

===================================================
Debugging the ``openedx-translations`` two-way sync
===================================================

After adding the to-be-translated repository `openedx-translations repo`_ verify the following in the next day:

1. The `extract-translation-source-files.yml`_ GitHub workflow worked successfully and the build passes in the
`openedx-translations GitHub Actions tab`_. If something fails, ask for help in the `#wg-translations`_ Open edX
Slack channel. An example of a successfully generated and merged pull request by the workflow's ``edx-transifex-bot`` is the
`chore - add updated translation source files #615`_
pull request.

2. Verify that the `openedx-translations project`_ has a new resource for the repo.

3. Ensure the new Transifex resource is 100% translated. Alternatively, Open edX Transifex admins can force sync via
the "Manual Sync" button in the `Transifex GitHub App sync logs`_ page.

4. Wait for the next sync to kick in. The sync is managed by Transifex and usually it kicks in less than an hour
which we'll verify in the next step. The `Transifex GitHub App sync logs`_ shows the most recent sync results.

5. Verify that the Transifex GitHub App created sync pull requests and auto-merged it to the repo.
An example of a successfully merged pull request is the
`Updates for file translations/frontend-app-learning/src/i18n/transifex_input.json in de on branch main #598`_ pull
request.

6. Verify that the translations can be pulled in the repo as described in the sections above depending on the repo
type.





.. _openedx-translations repo: https://github.com/openedx/openedx-translations
.. _edx-cookiecutters: https://github.com/openedx/edx-cookiecutters
.. _frontend-template-application: https://github.com/openedx/frontend-template-application
.. _OEP-58 - Translations Management: https://docs.openedx.org/projects/openedx-proposals/en/latest/architectural-decisions/oep-0058-arch-translations-management.html
.. _extract-translation-source-files.yml: https://github.com/openedx/openedx-translations/blob/2566e0c9a30d033e5dd8d05d4c12601c8e37b4ef/.github/workflows/extract-translation-source-files.yml#L36-L43
.. _Transifex GitHub App sync logs: https://github.apps.transifex.com/projects/o:open-edx:p:openedx-translations/openedx/openedx-translations
.. _cookiecutter-django-ida: https://github.com/openedx/edx-cookiecutters/tree/master/cookiecutter-django-ida
.. _openedx-translations project: https://app.transifex.com/open-edx/openedx-translations/dashboard/
.. _openedx-translations GitHub Actions tab: https://github.com/openedx/openedx-translations/actions
.. _#wg-translations: https://openedx.slack.com/archives/C037XDB9KN1


.. _chore - add updated translation source files #615: https://github.com/openedx/openedx-translations/pull/615
.. _Updates for file translations/frontend-app-learning/src/i18n/transifex_input.json in de on branch main #598: https://github.com/openedx/openedx-translations/pull/598

0 comments on commit 619d20c

Please sign in to comment.