Skip to content

Commit

Permalink
Merge branch 'main' of github.com:state-alchemists/zrb
Browse files Browse the repository at this point in the history
  • Loading branch information
goFrendiAsgard committed Nov 9, 2023
2 parents 1559240 + b54e4e3 commit f6ce3ca
Show file tree
Hide file tree
Showing 21 changed files with 413 additions and 233 deletions.
42 changes: 37 additions & 5 deletions docs/concepts/tasks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ There are many task types in Zrb. Every task has its own specific use cases:
- [ResourceMaker](resource-maker.md): Generate artifacts/resources based on templates
- [FlowTask](flow-task.md): Put `CmdTask` and `python task` into single flow.
- [RemoteCmdTask](remote-cmd-task.md)
- [RsyncTask](remote-cmd-task.md)
- [RsyncTask](rsync-task.md)
- [Checkers (HttpChecker, PortChecker, and PathChecker)](checkers.md): Check parent task's readiness.

As every task are extended from `BaseTask`, you will see that most of them share some common parameters.
Expand Down Expand Up @@ -400,13 +400,45 @@ Every task share some common methods like `run`, `check`, and `to_function`.

Deep copy current task

## `add_envs`
## `inject_env`

## `add_env_files`
To be overridden

## `add_inputs`
## `insert_env`

## `add_upstreams`
## `add_env`

## `inject_env_file`

To be overridden

## `insert_env_file`

## `add_env_file`

## `inject_input`

To be overridden

## `insert_input`

## `add_input`

## `inject_upstream`

To be overridden

## `insert_upstream`

## `add_upstream`

## `inject_checker`

To be overridden

## `insert_checker`

## `add_checker`

## `set_name`

Expand Down
9 changes: 9 additions & 0 deletions docs/concepts/tasks/remote-cmd-task.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,13 @@ install_curl = RemoteCmdTask(
runner.register(install_curl)
```

RemoteCmdTask exposes several environments that you can use on your `cmd` and `cmd_path`

- `_CONFIG_HOST`
- `_CONFIG_PORT`
- `_CONFIG_SSH_KEY`
- `_CONFIG_USER`
- `_CONFIG_PASSWORD`
- `_CONFIG_MAP_<UPPER_SNAKE_CASE_NAME>`

🔖 [Table of Contents](../../README.md) / [Concepts](../README.md) / [Tasks](README.md)
30 changes: 21 additions & 9 deletions docs/concepts/tasks/rsync-task.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
🔖 [Table of Contents](../../README.md) / [Concepts](../README.md) / [Tasks](README.md)

# RSyncTask
# RsyncTask

```python
from zrb import (
runner, CmdTask, RSyncTask, RemoteConfig, PasswordInput, StrInput
runner, CmdTask, RsyncTask, RemoteConfig, PasswordInput, StrInput
)

upload = RSyncTask(
upload = RsyncTask(
name='upload',
inputs=[
PasswordInput(name='passsword'),
Expand All @@ -18,17 +18,20 @@ upload = RSyncTask(
RemoteConfig(
host='192.168.1.10,
user='ubuntu,
password='{{input.password}}'
password='{{input.password}}',
config_map={
'dir': '192-168-1-10'
}
)
],
is_remote_src=False,
is_remote_dst=True
src='{{input.src}}',
src='$_CONFIG_MAP_DIR/{{input.src}}',
is_remote_dst=True,
dst='{{input.dst}}',
)
runner.register(upload)

download = RSyncTask(
download = RsyncTask(
name='download',
inputs=[
PasswordInput(name='passsword'),
Expand All @@ -43,11 +46,20 @@ download = RSyncTask(
)
],
is_remote_src=True,
is_remote_dst=False
src='{{input.src}}',
dst='{{input.dst}}',
is_remote_dst=False,
dst='$_CONFIG_MAP_DIR/{{input.dst}}',
)
runner.register(download)
```

RsyncTask exposes several environments that you can use on your `src` and `dst`

- `_CONFIG_HOST`
- `_CONFIG_PORT`
- `_CONFIG_SSH_KEY`
- `_CONFIG_USER`
- `_CONFIG_PASSWORD`
- `_CONFIG_MAP_<UPPER_SNAKE_CASE_NAME>`

🔖 [Table of Contents](../../README.md) / [Concepts](../README.md) / [Tasks](README.md)
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,35 @@
```python
from zrb import runner, CmdTask

prepare = CmdTask(
name='prepare-python-project',
cmd='pip install -R requirements.txt'
)
runner.register(prepare)

prepare = CmdTask(
name='prepare-node-project',
cmd='npm install'
)
runner.register(prepare)

run_fastapi = CmdTask(
name='run-fastapi',
cmd='uvicon main:app'
upstreams=[
prepare, # <-- Here is the problem, `npm install`` or `pip install`?
]
)
runner.register(run_fastapi)
```

You can see that `prepare-python-project` and `prepare-node-project` are assigned to the same variable.

Using that variable as upstream or checker will lead to a tricky situation. In our case, we want to perform `pip install` before starting Fast API. But since we re-assign the variable to `prepare-node-project`, we will got `npm install` instead.

# Avoiding the Problem

Beware of your variable name. Give your variable the same name as your task name.


🔖 [Table of Contents](../README.md) / [Oops, I Did It Again](README.md)
1 change: 0 additions & 1 deletion docs/quirks.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,5 @@
- `env` will override each other, the last one takes greater priority
- If you define a `DockerComposeTask`, it will automatically fill your environment with the ones you use in your docker-compose file. The environment defined that way will have a very low priority. They will be overridden by both `env_files` and `env`.
- You cannot have an input named: `_task`, `_args` or `_execution_id`
- You cannot have an environment named `_execution_id`

🔖 [Table of Contents](README.md)
4 changes: 2 additions & 2 deletions docs/tutorials/copy-and-reuse-task.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,10 @@ local_hello = hello.copy()

# Update name, input, and env
local_hello.set_name('hello-local')
local_hello.add_inputs(
local_hello.add_input(
StrInput(name='name', description='Name', default='dunia')
)
local_hello.add_envs(
local_hello.add_env(
Env(name='GREETINGS', os_name='', default='Halo')
)

Expand Down
25 changes: 15 additions & 10 deletions docs/tutorials/extending-cmd-task.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,18 +56,23 @@ class SlackPrintTask(CmdTask):
self._slack_app_token = slack_app_token
self._message = message

def run(self, *args: Any, **kwargs: Any):
# Inject environment variables
self.inject_env_map(
env_map={
'CHANNEL_ID': self.render_str(self._slack_channel_id),
'TOKEN': self.render_str(self._slack_app_token),
'MESSAGE': self.render_str(self._message)
}
def inject_envs(self):
self.add_envs(
Env(
name='CHANNEL_ID', os_name='',
default=self.render_str(self._slack_channel_id)
),
Env(
name='TOKEN', os_name='',
default=self.render_str(self._slack_app_token)
),
Env(
name='MESSAGE', os_name='',
default=self.render_str(self._message)
)
)
return super().run(*args, **kwargs)

def _get_cmd_str(self, *args: Any, **kwargs: Any):
def get_cmd_script(self, *args: Any, **kwargs: Any):
# contruct json payload and replace all `"` with `\\"`
json_payload = jsons.dumps({
'channel': '$CHANNEL_ID',
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "flit_core.buildapi"

[project]
name = "zrb"
version = "0.0.109"
version = "0.0.110"
authors = [
{ name="Go Frendi Gunawan", email="[email protected]" },
]
Expand Down
4 changes: 2 additions & 2 deletions src/zrb/action/runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ def __init__(self, env_prefix: str = ''):

def register(self, task: AnyTask):
task.set_has_cli_interface()
cmd_name = task.get_complete_cmd_name()
cmd_name = task.get_full_cmd_name()
logger.debug(colored(f'Register task: {cmd_name}', attrs=['dark']))
self._tasks.append(task)
logger.debug(colored(f'Task registered: {cmd_name}', attrs=['dark']))
Expand Down Expand Up @@ -77,7 +77,7 @@ def _get_cli_group(self, task_group: TaskGroup) -> click.Group:
return group

def _create_cli_command(self, task: AnyTask) -> click.Command:
task_inputs = task.get_all_inputs()
task_inputs = task._get_combined_inputs()
task_cmd_name = task.get_cmd_name()
task_description = task.get_description()
task_function = task.to_function(
Expand Down
4 changes: 1 addition & 3 deletions src/zrb/builtin/git.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
from zrb.helper.typing import Any
from zrb.builtin.group import git_group
from zrb.task.decorator import python_task
from zrb.task.task import Task
from zrb.task_input.str_input import StrInput
from zrb.task_input.bool_input import BoolInput
from zrb.runner import runner
Expand All @@ -26,7 +25,7 @@
prompt='Commit hash/Tag',
default='HEAD'
),
BoolInput(
BoolInput(
name='include-new',
description='include new files',
prompt='Include new files',
Expand All @@ -52,7 +51,6 @@ async def get_file_changes(*args: Any, **kwargs: Any):
include_new = kwargs.get('include_new', True)
include_removed = kwargs.get('include_removed', True)
include_updated = kwargs.get('include_updated', True)
task: Task = kwargs['_task']
modified_file_states = get_modified_file_states(commit)
modified_file_keys = []
output = []
Expand Down
32 changes: 21 additions & 11 deletions src/zrb/helper/accessories/name.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,19 +10,29 @@ def get_random_name(
digit_count: int = 4
) -> str:
prefixes = [
"aurum", "argentum", "platinum", "mercurius", "sulfur", "sal",
"luna", "sol", "ferrum", "cuprum", "argent", "aurora", "citrin",
"coral", "diamond", "dragon", "emerald", "garnet", "jade", "onyx",
"opal", "pearl", "ruby", "sapphire", "topaz", "turquoise", "verde",
"zircon"
'albedo', 'argent', 'argentum', 'aurora', 'aurum', 'azure',
'basilisk', 'cerulean', 'chimeric', 'citrin', 'coral', 'crimson',
'diamond', 'draco', 'dragon', 'emerald', 'ethereal', 'ferrum',
'flammeus', 'garnet', 'glacial', 'glimmering', 'glistening', 'golden',
'helios', 'igneous', 'imperial', 'jade', 'luminous', 'luna', 'lunar',
'mystic', 'nephrite', 'nocturnal', 'obsidian', 'opal', 'pearl',
'platinum', 'prismatic', 'ruby', 'sapphire', 'serpentine', 'silver',
'sol', 'solar', 'spiritual', 'stellar', 'tempest', 'topaz',
'turquoise', 'verde', 'vermillion', 'vitreous', 'zephyr', 'zircon'
]
suffixes = [
"philosophorum", "spiritus", "tinctura", "essentia", "elixir",
"praeparatum", "aether", "vitae", "lapis", "metallum", "aureum",
"caelestis", "chrysopoeia", "cosmicum", "deum", "draconis",
"elementorum", "hermetica", "illuminationis", "magnum", "mysticum",
"occultum", "omnipotentis", "philosophia", "praestantissimum",
"quintessentia", "regeneratio", "universalis"
'aether', 'albedo', 'alchemy', 'arcana', 'aureum', 'aetheris',
'anima', 'astralis', 'caelestis', 'chrysopoeia', 'cosmicum',
'crystallum', 'deum', 'divinitas', 'draconis', 'elementorum', 'elixir',
'essentia', 'eternis', 'ethereus', 'fatum', 'flamma', 'fulgur',
'hermetica', 'ignis', 'illuminationis', 'imperium', 'incantatum',
'infinitum', 'lapis', 'lux', 'magicae', 'magnum', 'materia',
'metallum', 'mysticum', 'natura', 'occultum', 'omnipotentis',
'opulentia', 'philosophia', 'philosophorum', 'praeparatum',
'praestantissimum', 'prima', 'primordium', 'quintessentia',
'regeneratio', 'ritualis', 'sanctum', 'spiritus', 'tenebris',
'terra', 'tinctura', 'transmutationis', 'universalis', 'vapores',
'venenum', 'veritas', 'vitae', 'volatus'
]
prefix = random.choice(prefixes)
suffix = random.choice(suffixes)
Expand Down
26 changes: 13 additions & 13 deletions src/zrb/helper/env_map/fetch.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ def fetch_env_map_from_group(
sub_env_map: Mapping[str, str] = fetch_env_map_from_group(
env_map, sub_group
)
env_map = cascade_env_map(env_map, sub_env_map)
env_map = _cascade_env_map(env_map, sub_env_map)
return env_map


Expand All @@ -25,33 +25,33 @@ def fetch_env_map_from_task(
env_map: Mapping[str, str], task: AnyTask
):
task_env_map: Mapping[str, str] = {}
for env_file in task.get_env_files():
for env_file in task._get_env_files():
envs = env_file.get_envs()
task_env_map = add_envs_to_env_map(task_env_map, envs)
task_env_map = add_envs_to_env_map(task_env_map, task._envs)
env_map = cascade_env_map(env_map, task_env_map)
for upstream in task.get_upstreams():
task_env_map = _add_envs_to_env_map(task_env_map, envs)
task_env_map = _add_envs_to_env_map(task_env_map, task._envs)
env_map = _cascade_env_map(env_map, task_env_map)
for upstream in task._get_upstreams():
task_env_map = fetch_env_map_from_task(env_map, upstream)
for checker in task.get_checkers():
for checker in task._get_checkers():
task_env_map = fetch_env_map_from_task(env_map, checker)
return env_map


@typechecked
def add_envs_to_env_map(
def _add_envs_to_env_map(
env_map: Mapping[str, str], envs: List[Env]
) -> Mapping[str, str]:
for env in envs:
if env.os_name == '':
continue
env_name = get_env_name(env)
env_default = get_env_default(env)
env_name = _get_env_name(env)
env_default = _get_env_default(env)
env_map[env_name] = env_default
return env_map


@typechecked
def cascade_env_map(
def _cascade_env_map(
env_map: Mapping[str, str],
other_env_map: Mapping[str, str]
) -> Mapping[str, str]:
Expand All @@ -63,14 +63,14 @@ def cascade_env_map(


@typechecked
def get_env_name(env: Env) -> str:
def _get_env_name(env: Env) -> str:
if env.os_name is None:
return env.name
return env.os_name


@typechecked
def get_env_default(env: Env) -> str:
def _get_env_default(env: Env) -> str:
if is_probably_jinja(env.default):
return ''
return env.default
Loading

0 comments on commit f6ce3ca

Please sign in to comment.