Release/0.0.110 (#48)

* update version * update pyproject * update docs * update docs * add names * make execution id more random * improve name * add config map * add --makepath to rscync * update docs * expose env * use _get_all_envs * remove direct property call * add insert_task * rename methods * make more methods protected * introduce inject methods * adjust docs
state-alchemists · Nov 9, 2023 · b54e4e3 · b54e4e3
1 parent 5f39977
commit b54e4e3
Show file tree

Hide file tree

Showing 21 changed files with 413 additions and 233 deletions.
diff --git a/docs/concepts/tasks/README.md b/docs/concepts/tasks/README.md
@@ -12,7 +12,7 @@ There are many task types in Zrb. Every task has its own specific use cases:
 - [ResourceMaker](resource-maker.md): Generate artifacts/resources based on templates
 - [FlowTask](flow-task.md): Put `CmdTask` and `python task` into single flow.
 - [RemoteCmdTask](remote-cmd-task.md)
-- [RsyncTask](remote-cmd-task.md)
+- [RsyncTask](rsync-task.md)
 - [Checkers (HttpChecker, PortChecker, and PathChecker)](checkers.md): Check parent task's readiness.
 
 As every task are extended from `BaseTask`, you will see that most of them share some common parameters.
@@ -400,13 +400,45 @@ Every task share some common methods like `run`, `check`, and `to_function`.
 
 Deep copy current task
 
-## `add_envs`
+## `inject_env`
 
-## `add_env_files`
+To be overridden
 
-## `add_inputs`
+## `insert_env`
 
-## `add_upstreams`
+## `add_env`
+
+## `inject_env_file`
+
+To be overridden
+
+## `insert_env_file`
+
+## `add_env_file`
+
+## `inject_input`
+
+To be overridden
+
+## `insert_input`
+
+## `add_input`
+
+## `inject_upstream`
+
+To be overridden
+
+## `insert_upstream`
+
+## `add_upstream`
+
+## `inject_checker`
+
+To be overridden
+
+## `insert_checker`
+
+## `add_checker`
 
 ## `set_name`
 

diff --git a/docs/concepts/tasks/remote-cmd-task.md b/docs/concepts/tasks/remote-cmd-task.md
@@ -27,4 +27,13 @@ install_curl = RemoteCmdTask(
 runner.register(install_curl)
 ```
 
+RemoteCmdTask exposes several environments that you can use on your `cmd` and `cmd_path`
+
+- `_CONFIG_HOST`
+- `_CONFIG_PORT`
+- `_CONFIG_SSH_KEY`
+- `_CONFIG_USER`
+- `_CONFIG_PASSWORD`
+- `_CONFIG_MAP_<UPPER_SNAKE_CASE_NAME>`
+
 🔖 [Table of Contents](../../README.md) / [Concepts](../README.md) / [Tasks](README.md)
diff --git a/docs/concepts/tasks/rsync-task.md b/docs/concepts/tasks/rsync-task.md
@@ -1,13 +1,13 @@
 🔖 [Table of Contents](../../README.md) / [Concepts](../README.md) / [Tasks](README.md)
 
-# RSyncTask
+# RsyncTask
 
 ```python
 from zrb import (
-    runner, CmdTask, RSyncTask, RemoteConfig, PasswordInput, StrInput
+    runner, CmdTask, RsyncTask, RemoteConfig, PasswordInput, StrInput
 )
 
-upload = RSyncTask(
+upload = RsyncTask(
     name='upload',
     inputs=[
         PasswordInput(name='passsword'),
@@ -18,17 +18,20 @@ upload = RSyncTask(
         RemoteConfig(
             host='192.168.1.10,
             user='ubuntu,
-            password='{{input.password}}'
+            password='{{input.password}}',
+            config_map={
+                'dir': '192-168-1-10'
+            }
         )
     ],
     is_remote_src=False,
-    is_remote_dst=True
-    src='{{input.src}}',
+    src='$_CONFIG_MAP_DIR/{{input.src}}',
+    is_remote_dst=True,
     dst='{{input.dst}}',
 )
 runner.register(upload)
 
-download = RSyncTask(
+download = RsyncTask(
     name='download',
     inputs=[
         PasswordInput(name='passsword'),
@@ -43,11 +46,20 @@ download = RSyncTask(
         )
     ],
     is_remote_src=True,
-    is_remote_dst=False
     src='{{input.src}}',
-    dst='{{input.dst}}',
+    is_remote_dst=False,
+    dst='$_CONFIG_MAP_DIR/{{input.dst}}',
 )
 runner.register(download)
 ```
 
+RsyncTask exposes several environments that you can use on your `src` and `dst`
+
+- `_CONFIG_HOST`
+- `_CONFIG_PORT`
+- `_CONFIG_SSH_KEY`
+- `_CONFIG_USER`
+- `_CONFIG_PASSWORD`
+- `_CONFIG_MAP_<UPPER_SNAKE_CASE_NAME>`
+
 🔖 [Table of Contents](../../README.md) / [Concepts](../README.md) / [Tasks](README.md)
diff --git a/docs/oops-i-did-it-again/using-the-same-variable-to-define-different-task.md b/docs/oops-i-did-it-again/using-the-same-variable-to-define-different-task.md
@@ -5,7 +5,35 @@
 ```python
 from zrb import runner, CmdTask
 
+prepare = CmdTask(
+    name='prepare-python-project',
+    cmd='pip install -R requirements.txt'
+)
+runner.register(prepare)
 
+prepare = CmdTask(
+    name='prepare-node-project',
+    cmd='npm install'
+)
+runner.register(prepare)
+
+run_fastapi = CmdTask(
+    name='run-fastapi',
+    cmd='uvicon main:app'
+    upstreams=[
+        prepare, # <-- Here is the problem, `npm install`` or `pip install`? 
+    ]
+)
+runner.register(run_fastapi)
 ```
 
+You can see that `prepare-python-project` and `prepare-node-project` are assigned to the same variable.
+
+Using that variable as upstream or checker will lead to a tricky situation. In our case, we want to perform `pip install` before starting Fast API. But since we re-assign the variable to `prepare-node-project`, we will got `npm install` instead.
+
+# Avoiding the Problem
+
+Beware of your variable name. Give your variable the same name as your task name.
+
+
 🔖 [Table of Contents](../README.md) / [Oops, I Did It Again](README.md)
diff --git a/docs/quirks.md b/docs/quirks.md
@@ -18,6 +18,5 @@
     - `env` will override each other, the last one takes greater priority
     - If you define a `DockerComposeTask`, it will automatically fill your environment with the ones you use in your docker-compose file. The environment defined that way will have a very low priority. They will be overridden by both `env_files` and `env`.
 - You cannot have an input named: `_task`, `_args` or `_execution_id`
-- You cannot have an environment named `_execution_id`
 
 🔖 [Table of Contents](README.md)
diff --git a/docs/tutorials/copy-and-reuse-task.md b/docs/tutorials/copy-and-reuse-task.md
@@ -20,10 +20,10 @@ local_hello = hello.copy()
 
 # Update name, input, and env
 local_hello.set_name('hello-local')
-local_hello.add_inputs(
+local_hello.add_input(
     StrInput(name='name', description='Name', default='dunia')
 )
-local_hello.add_envs(
+local_hello.add_env(
     Env(name='GREETINGS', os_name='', default='Halo')
 )
 

diff --git a/docs/tutorials/extending-cmd-task.md b/docs/tutorials/extending-cmd-task.md
@@ -56,18 +56,23 @@ class SlackPrintTask(CmdTask):
         self._slack_app_token = slack_app_token
         self._message = message
 
-    def run(self, *args: Any, **kwargs: Any):
-        # Inject environment variables
-        self.inject_env_map(
-            env_map={
-                'CHANNEL_ID': self.render_str(self._slack_channel_id),
-                'TOKEN': self.render_str(self._slack_app_token),
-                'MESSAGE': self.render_str(self._message)
-            }
+    def inject_envs(self):
+        self.add_envs(
+            Env(
+                name='CHANNEL_ID', os_name='',
+                default=self.render_str(self._slack_channel_id)
+            ),
+            Env(
+                name='TOKEN', os_name='',
+                default=self.render_str(self._slack_app_token)
+            ),
+            Env(
+                name='MESSAGE', os_name='',
+                default=self.render_str(self._message)
+            )
         )
-        return super().run(*args, **kwargs)
 
-    def _get_cmd_str(self, *args: Any, **kwargs: Any):
+    def get_cmd_script(self, *args: Any, **kwargs: Any):
         # contruct json payload and replace all `"` with `\\"`
         json_payload = jsons.dumps({
             'channel': '$CHANNEL_ID',

diff --git a/pyproject.toml b/pyproject.toml
@@ -4,7 +4,7 @@ build-backend = "flit_core.buildapi"
 
 [project]
 name = "zrb"
-version = "0.0.109"
+version = "0.0.110"
 authors = [
   { name="Go Frendi Gunawan", email="[email protected]" },
 ]

diff --git a/src/zrb/action/runner.py b/src/zrb/action/runner.py
@@ -28,7 +28,7 @@ def __init__(self, env_prefix: str = ''):
 
     def register(self, task: AnyTask):
         task.set_has_cli_interface()
-        cmd_name = task.get_complete_cmd_name()
+        cmd_name = task.get_full_cmd_name()
         logger.debug(colored(f'Register task: {cmd_name}', attrs=['dark']))
         self._tasks.append(task)
         logger.debug(colored(f'Task registered: {cmd_name}', attrs=['dark']))
@@ -77,7 +77,7 @@ def _get_cli_group(self, task_group: TaskGroup) -> click.Group:
         return group
 
     def _create_cli_command(self, task: AnyTask) -> click.Command:
-        task_inputs = task.get_all_inputs()
+        task_inputs = task._get_combined_inputs()
         task_cmd_name = task.get_cmd_name()
         task_description = task.get_description()
         task_function = task.to_function(

diff --git a/src/zrb/builtin/git.py b/src/zrb/builtin/git.py
@@ -1,7 +1,6 @@
 from zrb.helper.typing import Any
 from zrb.builtin.group import git_group
 from zrb.task.decorator import python_task
-from zrb.task.task import Task
 from zrb.task_input.str_input import StrInput
 from zrb.task_input.bool_input import BoolInput
 from zrb.runner import runner
@@ -26,7 +25,7 @@
             prompt='Commit hash/Tag',
             default='HEAD'
         ),
-       BoolInput(
+        BoolInput(
             name='include-new',
             description='include new files',
             prompt='Include new files',
@@ -52,7 +51,6 @@ async def get_file_changes(*args: Any, **kwargs: Any):
     include_new = kwargs.get('include_new', True)
     include_removed = kwargs.get('include_removed', True)
     include_updated = kwargs.get('include_updated', True)
-    task: Task = kwargs['_task']
     modified_file_states = get_modified_file_states(commit)
     modified_file_keys = []
     output = []

diff --git a/src/zrb/helper/accessories/name.py b/src/zrb/helper/accessories/name.py
@@ -10,19 +10,29 @@ def get_random_name(
     digit_count: int = 4
 ) -> str:
     prefixes = [
-        "aurum", "argentum", "platinum", "mercurius", "sulfur", "sal",
-        "luna", "sol", "ferrum", "cuprum", "argent", "aurora", "citrin",
-        "coral", "diamond", "dragon", "emerald", "garnet", "jade", "onyx",
-        "opal", "pearl", "ruby", "sapphire", "topaz", "turquoise", "verde",
-        "zircon"
+        'albedo', 'argent', 'argentum', 'aurora', 'aurum', 'azure',
+        'basilisk', 'cerulean', 'chimeric', 'citrin', 'coral', 'crimson',
+        'diamond', 'draco', 'dragon', 'emerald', 'ethereal', 'ferrum',
+        'flammeus', 'garnet', 'glacial', 'glimmering', 'glistening', 'golden',
+        'helios', 'igneous', 'imperial', 'jade', 'luminous', 'luna', 'lunar',
+        'mystic', 'nephrite', 'nocturnal', 'obsidian', 'opal', 'pearl',
+        'platinum', 'prismatic', 'ruby', 'sapphire', 'serpentine', 'silver',
+        'sol', 'solar', 'spiritual', 'stellar',  'tempest', 'topaz',
+        'turquoise', 'verde', 'vermillion', 'vitreous', 'zephyr', 'zircon'
     ]
     suffixes = [
-        "philosophorum", "spiritus", "tinctura", "essentia", "elixir",
-        "praeparatum", "aether", "vitae", "lapis", "metallum", "aureum",
-        "caelestis", "chrysopoeia", "cosmicum", "deum", "draconis",
-        "elementorum", "hermetica", "illuminationis", "magnum", "mysticum",
-        "occultum", "omnipotentis", "philosophia", "praestantissimum",
-        "quintessentia", "regeneratio", "universalis"
+        'aether', 'albedo', 'alchemy', 'arcana', 'aureum', 'aetheris',
+        'anima', 'astralis', 'caelestis', 'chrysopoeia', 'cosmicum',
+        'crystallum', 'deum', 'divinitas', 'draconis', 'elementorum', 'elixir',
+        'essentia', 'eternis', 'ethereus', 'fatum', 'flamma', 'fulgur',
+        'hermetica', 'ignis', 'illuminationis', 'imperium', 'incantatum',
+        'infinitum', 'lapis', 'lux', 'magicae', 'magnum', 'materia',
+        'metallum', 'mysticum', 'natura', 'occultum', 'omnipotentis',
+        'opulentia', 'philosophia', 'philosophorum', 'praeparatum',
+        'praestantissimum', 'prima', 'primordium', 'quintessentia',
+        'regeneratio', 'ritualis', 'sanctum', 'spiritus', 'tenebris',
+        'terra', 'tinctura', 'transmutationis', 'universalis', 'vapores',
+        'venenum', 'veritas', 'vitae', 'volatus'
     ]
     prefix = random.choice(prefixes)
     suffix = random.choice(suffixes)

diff --git a/src/zrb/helper/env_map/fetch.py b/src/zrb/helper/env_map/fetch.py
@@ -16,7 +16,7 @@ def fetch_env_map_from_group(
         sub_env_map: Mapping[str, str] = fetch_env_map_from_group(
             env_map, sub_group
         )
-        env_map = cascade_env_map(env_map, sub_env_map)
+        env_map = _cascade_env_map(env_map, sub_env_map)
     return env_map
 
 
@@ -25,33 +25,33 @@ def fetch_env_map_from_task(
     env_map: Mapping[str, str], task: AnyTask
 ):
     task_env_map: Mapping[str, str] = {}
-    for env_file in task.get_env_files():
+    for env_file in task._get_env_files():
         envs = env_file.get_envs()
-        task_env_map = add_envs_to_env_map(task_env_map, envs)
-    task_env_map = add_envs_to_env_map(task_env_map, task._envs)
-    env_map = cascade_env_map(env_map, task_env_map)
-    for upstream in task.get_upstreams():
+        task_env_map = _add_envs_to_env_map(task_env_map, envs)
+    task_env_map = _add_envs_to_env_map(task_env_map, task._envs)
+    env_map = _cascade_env_map(env_map, task_env_map)
+    for upstream in task._get_upstreams():
         task_env_map = fetch_env_map_from_task(env_map, upstream)
-    for checker in task.get_checkers():
+    for checker in task._get_checkers():
         task_env_map = fetch_env_map_from_task(env_map, checker)
     return env_map
 
 
 @typechecked
-def add_envs_to_env_map(
+def _add_envs_to_env_map(
     env_map: Mapping[str, str], envs: List[Env]
 ) -> Mapping[str, str]:
     for env in envs:
         if env.os_name == '':
             continue
-        env_name = get_env_name(env)
-        env_default = get_env_default(env)
+        env_name = _get_env_name(env)
+        env_default = _get_env_default(env)
         env_map[env_name] = env_default
     return env_map
 
 
 @typechecked
-def cascade_env_map(
+def _cascade_env_map(
     env_map: Mapping[str, str],
     other_env_map: Mapping[str, str]
 ) -> Mapping[str, str]:
@@ -63,14 +63,14 @@ def cascade_env_map(
 
 
 @typechecked
-def get_env_name(env: Env) -> str:
+def _get_env_name(env: Env) -> str:
     if env.os_name is None:
         return env.name
     return env.os_name
 
 
 @typechecked
-def get_env_default(env: Env) -> str:
+def _get_env_default(env: Env) -> str:
     if is_probably_jinja(env.default):
         return ''
     return env.default