Build scripts using a template engine

[b-fein]

Motivation

Currently, the Bash scripts that are created from the Windfiles are ‘templated’ by manually adding them line-by-line to a result string. This has multiple disadvantages:

• It is repetitive: add_line() needs to be called for every line. Each time the indentation has to be specified.
• It is potentially error-prone and/or hard to understand:
  • The indentation has to be kept in mind for all commands.
  • Escaping in strings with special characters like \ and quotation marks.
  • It is hard to see the overall concept of a block of the script, since each line is always made up of some Python code + the actual line that will appear in the Bash script.

Proposed solution

A template engine like for example Jinja2 would allow to create the scripts as templates with placeholders that can be filled in. It also allows for repeated blocks and blocks that are only ‘rendered’ (i.e., added to the script) based on some condition.
Especially when combined with a TypedDict that holds the variables passed into the template, this allows for script templates that are easier to maintain.

Multiple smaller template files could be used and then be combined into the whole script.

More complex individual shell commands that are built from multiple conditional parameters could still be built in Python code and then passed in as a whole to the template to not have to deal with too many optional parameters in the template.

Example

Approximate translation of

Aeolus/cli/generators/cli.py

Line 58 in 789bcb0

def add_postfix(self) -> None:

into a template.

main() {
{% if needs_lifecycle_parameter %}
  local current_lifecycle="${1}"
{% endif %}

{% if needs_subshells %}
  if [[ "${1}" == "aeolus_sourcing" ]]; then
    # just source to use the methods in the subshell, no execution
    return 0
  fi
  local _script_name
  _script_name=$(realpath "${0}")
{% endif %}

{% if has_always_actions %}
  trap final_aeolus_post_action EXIT
{% endif %}

{% for function in functions %}
  {% if needs_subshells %}
  bash -c "source ${_script_name} aeolus_sourcing; {{ function.name }} {{ function.parameter }}"
  {% else %}
  {{ function.name }} {{ function.parameter }}
  {% endif %}

  {% if has_multiple_steps %}
  cd "{{ initial_directory_variable }}"
  {% endif %}
{% end %}
}

main "${@}"

Additional considerations

Jinja2 is just one example of a template engine I am somewhat familiar with. Other template engines might exist that are better suited for this concrete task.

The proposed refactoring is not essential to provide the core feature set of Aeolus. I would consider this as a possible nice-to-have to ensure long-term maintainability of the tool.

[reschandreas]

Hi,

I appreciate your feedback and the suggestion to use Jinja2 for script generation. At the start of the project I looked into jinja2 as I was told it could be useful for Aeolus, but back then Bamboo was of priority so I started with the current approach and kept adapting it instead of doing it the right way. I'll definitely look into it in the coming weeks.

[reschandreas]

I added jinja2 templates for cli and jenkins in. #53 and #54. It definitely increased readability and maintainability. Thanks for nudging me in the right direction.