prepare for v0.1.0

LICENSE

@@ -186,7 +186,7 @@
       same "printed page" as the copyright notice for easier
       identification within third-party archives.
 
-   Copyright [yyyy] [name of copyright owner]
+   Copyright [2024] [Lokalise group, Ilya Krukowski]
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.

README.md

@@ -1 +1,87 @@
-# lokalise-pull-action
+# GitHub action to pull translation files from Lokalise
+
+GitHub action to download translation files from [Lokalise TMS](https://lokalise.com/) to your GitHub repository in the form of a pull request.
+
+**Step-by-step tutorial covering the usage of this action is available on [Lokalise Developer Hub](https://developers.lokalise.com/docs/github-actions).** To upload translation files from GitHub to Lokalise, use the [lokalise-push-action](https://github.com/lokalise/lokalise-push-action).
+
+## Usage
+
+Use this action in the following way:
+
+```yaml
+name: Demo pull with tags
+
+on:
+  workflow_dispatch:
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout Repo
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - name: Pull from Lokalise
+        uses: lokalise/[email protected]
+        with:
+          api_token: ${{ secrets.LOKALISE_API_TOKEN }}
+          project_id: LOKALISE_PROJECT_ID
+          translations_path: TRANSLATIONS_PATH
+          file_format: FILE_FORMAT
+          additional_params: ADDITIONAL_CLI_PARAMS
+```
+
+## Configuration
+
+### Parameters
+
+You'll need to provide some parameters for the action. These can be set as environment variables, secrets, or passed directly. Refer to the [General setup](https://developers.lokalise.com/docs/github-actions#general-setup-overview) section for detailed instructions.
+
+The following parameters are **mandatory**:
+
+- `api_token` — Lokalise API token.
+- `project_id` — Your Lokalise project ID.
+- `translations_path` — Path to your translation files.
+- `file_format` — The format of your translation files.
+- `base_lang` — Your project base language.
+
+**Optional** parameters include:
+
+- `additional_params` — Extra parameters to pass to the [Lokalise CLI when pulling files](https://github.com/lokalise/lokalise-cli-2-go/blob/main/docs/lokalise2_file_download.md). For example, you can use `--indentation 2sp` to manage indentation. Multiple CLI arguments can be added, like: `--indentation 2sp --placeholder-format icu`.
+- `temp_branch_prefix` — A prefix for the temporary branch used to create the pull request. This value will be part of the branch name. For example, using `lok` will result in a branch name starting with `lok`. The default value is `lok`.
+- `always_pull_base` — By default, changes in the base language translation files (defined by the `base_lang` option) are ignored when checking for updates. Set this option to `true` to include changes in the base language translations in the pull request. The default value is `false`.
+* `max_retries` — Maximum number of retries on rate limit errors (HTTP 429). The default value is `3`.
+* `sleep_on_retry` — Number of seconds to sleep before retrying on rate limit errors. The default value is `1`.
+
+### Permissions
+
+1. Go to your repository's **Settings**.
+2. Navigate to **Actions > General**.
+3. Under **Workflow permissions**, set the permissions to **Read and write permissions**.
+4. Enable **Allow GitHub Actions to create and approve pull requests** on the same page (under "Choose whether GitHub Actions can create pull requests or submit approving pull request reviews").
+
+## Technical details
+
+### How this action works
+
+When triggered, this action performs the following steps:
+
+1. Installs Lokalise CLIv2.
+2. Downloads translation files for all languages from the specified Lokalise project. The keys included in the download bundle are filtered by the tag named after the triggering branch. For example, if the branch is called `lokalise-hub`, only the keys with this tag will be downloaded.
+3. If any changes in the translation files are detected, a pull request will be created for the triggering branch. This pull request will be sent from a temporary branch.
+
+For more information on assumptions, refer to the [Assumptions and defaults](https://developers.lokalise.com/docs/github-actions#assumptions-and-defaults) section.
+
+### Default parameters for the pull action
+
+By default, the following command-line parameters are set when downloading files from Lokalise:
+
+- `--token` — Derived from the `api_token` parameter.
+- `--project-id` — Derived from the `project_id` parameter.
+- `--format` — Derived from the `file_format` parameter.
+- `--original-filenames` — Set to `true`.
+- `--directory-prefix` — Set to `/`.
+- `--include-tags` — Set to the branch name that triggered the workflow.

action.yml

@@ -0,0 +1,150 @@
+name: 'Pull from Lokalise'
+description: 'Pull translation files from Lokalise'
+author: 'Lokalise Team'
+inputs:
+  api_token:
+    description: 'API token for Lokalise with read/write permissions'
+    required: true
+    secret: true
+  project_id:
+    description: 'Project ID for Lokalise'
+    required: true
+  base_lang:
+    description: 'Base language (e.g., en, fr_FR)'
+    required: true
+    default: 'en'
+  translations_path:
+    description: 'Path to translation files'
+    required: true
+    default: 'locales'
+  file_format:
+    description: 'Format of the translation files (e.g., json). Find all supported file formats at https://developers.lokalise.com/reference/api-file-formats'
+    required: true
+    default: 'json'
+  additional_params:
+    description: 'Additional parameters for Lokalise CLI on pull. Find all supported options at https://github.com/lokalise/lokalise-cli-2-go/blob/main/docs/lokalise2_file_download.md'
+    required: false
+    default: ''
+  temp_branch_prefix:
+    description: 'Prefix for the temp branch to create pull request'
+    required: false
+    default: 'lok'
+  always_pull_base:
+    description: 'By default, changes in the base language translation files are ignored. Set this to true to include base language translations in the PR.'
+    required: false
+    default: false
+  max_retries:
+    description: 'Maximum number of retries on rate limit errors'
+    required: false
+    default: 3
+  sleep_on_retry:
+    description: 'Number of seconds to sleep before retrying'
+    required: false
+    default: 1
+
+branding:
+  icon: 'download-cloud'
+  color: 'orange'
+
+runs:
+  using: "composite"
+  steps:
+    - name: Install Lokalise CLI
+      shell: bash
+      run: |
+        chmod +x "${{ github.action_path }}/src/scripts/install_lokalise_cli.sh"
+        "${{ github.action_path }}/src/scripts/install_lokalise_cli.sh"
+
+    - name: Pull translation files from Lokalise
+      id: pull-files
+      shell: bash
+      env:
+        CLI_ADD_PARAMS: ${{ inputs.additional_params }}
+        MAX_RETRIES: ${{ inputs.max_retries }}
+        SLEEP_TIME: ${{ inputs.sleep_on_retry }}
+        FILE_FORMAT: ${{ inputs.file_format }}
+      run: |
+        chmod +x "${{ github.action_path }}/src/scripts/lokalise_download.sh"
+
+        . "${{ github.action_path }}/src/scripts/lokalise_download.sh"
+        
+        download_files "${{ inputs.project_id }}" "${{ inputs.api_token }}"
+              
+        if [ $? -ne 0 ]; then
+          echo "Error during file download"
+          echo "has_changes=false" >> $GITHUB_OUTPUT
+          exit 1
+        fi
+
+        if [[ "${{ inputs.always_pull_base }}" == "true" ]]; then
+          STATUS_CMD=$(git status "${{ inputs.translations_path }}/**/*.${{ inputs.file_format }}" --untracked-files=no --porcelain)
+          UNTRACKED_FILES=$(git ls-files --others --exclude-standard "${{ inputs.translations_path }}/**/*.${{ inputs.file_format }}")
+        else
+          STATUS_CMD=$(git status "${{ inputs.translations_path }}/**/*.${{ inputs.file_format }}" --untracked-files=no --porcelain | grep -v "${{ inputs.translations_path }}/${{ inputs.base_lang }}" || true)
+          UNTRACKED_FILES=$(git ls-files --others --exclude-standard "${{ inputs.translations_path }}/**/*.${{ inputs.file_format }}" | grep -v "${{ inputs.translations_path }}/${{ inputs.base_lang }}" || true)
+        fi
+
+        if [[ -z "$STATUS_CMD" && -z "$UNTRACKED_FILES" ]]; then
+          echo "No translation file changes detected after pulling from Lokalise"
+          echo "has_changes=false" >> $GITHUB_OUTPUT
+        else
+          echo "Translation file changes detected after pulling from Lokalise"
+          echo "has_changes=true" >> $GITHUB_OUTPUT
+        fi
+
+    - name: Commit and push changes
+      id: commit-and-push
+      if: steps.pull-files.outputs.has_changes == 'true'
+      shell: bash
+      run: |
+        git config --global user.name "${GITHUB_ACTOR}"
+        git config --global user.email "${GITHUB_ACTOR}@users.noreply.github.com"
+
+        TIMESTAMP=$(date +%s)
+        SHORT_SHA=${GITHUB_SHA::6}
+        BRANCH_NAME="${{ inputs.temp_branch_prefix }}_${GITHUB_REF_NAME}_${SHORT_SHA}_${TIMESTAMP}"
+        BRANCH_NAME=$(echo "$BRANCH_NAME" | tr -cd '[:alnum:]_-' | cut -c1-255)
+
+        echo "branch_name=$BRANCH_NAME" >> $GITHUB_ENV
+
+        git checkout -b "$BRANCH_NAME" || git checkout "$BRANCH_NAME"
+      
+        if [[ "${{ inputs.always_pull_base }}" == "true" ]]; then
+          git add "${{ inputs.translations_path }}/**/*.${{ inputs.file_format }}" --force
+        else
+          git add "${{ inputs.translations_path }}/**/*.${{ inputs.file_format }}" --force ":!${{ inputs.translations_path }}/${{ inputs.base_lang }}"
+        fi
+        git commit -m 'Translations update'
+        git push origin "$BRANCH_NAME"
+
+    - name: Create or Update Pull Request
+      if: steps.pull-files.outputs.has_changes == 'true'
+      uses: actions/github-script@v7
+      with:
+        github-token: ${{ github.token }}
+        script: |
+          try {
+            const { data: pullRequests } = await github.rest.pulls.list({
+              owner: "${{ github.repository_owner }}",
+              repo: "${{ github.event.repository.name }}",
+              head: "${{ github.repository_owner }}:${{ env.branch_name }}",
+              base: "${{ github.ref_name }}",
+              state: 'open'
+            });
+
+            if (pullRequests.length > 0) {
+              console.log(`PR already exists: ${pullRequests[0].html_url}`);
+            } else {
+              const { data: newPr } = await github.rest.pulls.create({
+                owner: "${{ github.repository_owner }}",
+                repo: "${{ github.event.repository.name }}",
+                title: "Lokalise translations update",
+                head: "${{ env.branch_name }}",
+                base: "${{ github.ref_name }}",
+                body: "This PR updates translations from Lokalise.",
+              });
+              console.log(`Created new PR: ${newPr.html_url}`);
+            }
+          } catch (error) {
+            core.setFailed(`Failed to create or update pull request: ${error.message}`);
+          }

src/scripts/install_lokalise_cli.sh

@@ -0,0 +1,11 @@
+#!/bin/bash
+
+if ! command -v lokalise2 >/dev/null 2>&1; then
+    echo "Installing Lokalise CLI..."
+    curl -sfL https://raw.githubusercontent.com/lokalise/lokalise-cli-2-go/master/install.sh | sh || {
+        echo "Failed to install Lokalise CLI"
+        exit 1
+    }
+else
+    echo "Lokalise CLI is already installed, skipping installation."
+fi

src/scripts/lokalise_download.sh

@@ -0,0 +1,82 @@
+#!/bin/bash
+
+return_with_error() {
+    echo "Error: $1" >&2
+    return 1
+}
+
+download_files() {
+    local project_id=$1
+    local token=$2
+    local additional_params="${CLI_ADD_PARAMS:-}"
+    local attempt=0
+    local max_retries="${MAX_RETRIES:-5}"
+    local sleep_time="${SLEEP_TIME:-1}"
+    local max_sleep_time=60
+    local max_total_time=300
+    local start_time=$(date +%s)
+    local file_format="${FILE_FORMAT}"
+    local github_ref_name="${GITHUB_REF_NAME}"
+
+    [[ -z "$project_id" ]] && return_with_error "project_id is required and cannot be empty."
+    [[ -z "$token" ]] && return_with_error "token is required and cannot be empty."
+
+    if [[ "$sleep_time" -lt 1 ]]; then
+        sleep_time=1
+    elif [[ "$sleep_time" -gt "$max_sleep_time" ]]; then
+        sleep_time=$max_sleep_time
+    fi
+
+    if ! [[ "$max_retries" =~ ^[0-9]+$ ]] || [[ "$max_retries" -lt 1 ]]; then
+        max_retries=5
+    fi
+
+    if ! [[ "$max_total_time" =~ ^[0-9]+$ ]] || [[ "$max_total_time" -lt $max_sleep_time ]]; then
+        max_total_time=300
+    fi
+
+    echo "Starting download for project: $project_id"
+    while [ $attempt -lt $max_retries ]; do
+        echo "Attempt $((attempt + 1)) of $max_retries"
+
+        set +e
+
+        output=$(./bin/lokalise2 --token="$token" \
+            --project-id="$project_id" \
+            file download \
+            --format="$file_format" \
+            --original-filenames=true \
+            --directory-prefix="/" \
+            --include-tags="$github_ref_name" \
+            $additional_params 2>&1)
+
+        exit_code=$?
+
+        set -e
+
+        if [ $exit_code -eq 0 ]; then
+            echo "Successfully downloaded files"
+            return 0
+        elif echo "$output" | grep -q 'API request error 429'; then
+            attempt=$((attempt + 1))
+            current_time=$(date +%s)
+            elapsed_time=$((current_time - start_time))
+            if [ $elapsed_time -ge $max_total_time ]; then
+                return_with_error "Max retry time exceeded before sleeping. Exiting."
+            fi
+            echo "Attempt $attempt failed with API request error 429. Retrying in $sleep_time seconds..."
+            sleep $sleep_time
+            sleep_time=$((sleep_time * 2))
+            if [ $sleep_time -gt $max_sleep_time ]; then
+                sleep_time=$max_sleep_time
+            fi
+        elif echo "$output" | grep -q 'API request error 406'; then
+            echo "API request error 406: No keys for export with current export settings. Exiting..."
+            return 0
+        else
+            return_with_error "Error encountered during download: $output"
+        fi
+    done
+
+    return_with_error "Failed to download files after $max_retries attempts"
+}