Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release/0.0.110 #48

Merged
merged 22 commits into from
Nov 9, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 37 additions & 5 deletions docs/concepts/tasks/README.md
Original file line number Diff line number Diff line change
@@ -12,7 +12,7 @@ There are many task types in Zrb. Every task has its own specific use cases:
- [ResourceMaker](resource-maker.md): Generate artifacts/resources based on templates
- [FlowTask](flow-task.md): Put `CmdTask` and `python task` into single flow.
- [RemoteCmdTask](remote-cmd-task.md)
- [RsyncTask](remote-cmd-task.md)
- [RsyncTask](rsync-task.md)
- [Checkers (HttpChecker, PortChecker, and PathChecker)](checkers.md): Check parent task's readiness.

As every task are extended from `BaseTask`, you will see that most of them share some common parameters.
@@ -400,13 +400,45 @@ Every task share some common methods like `run`, `check`, and `to_function`.

Deep copy current task

## `add_envs`
## `inject_env`

## `add_env_files`
To be overridden

## `add_inputs`
## `insert_env`

## `add_upstreams`
## `add_env`

## `inject_env_file`

To be overridden

## `insert_env_file`

## `add_env_file`

## `inject_input`

To be overridden

## `insert_input`

## `add_input`

## `inject_upstream`

To be overridden

## `insert_upstream`

## `add_upstream`

## `inject_checker`

To be overridden

## `insert_checker`

## `add_checker`

## `set_name`

9 changes: 9 additions & 0 deletions docs/concepts/tasks/remote-cmd-task.md
Original file line number Diff line number Diff line change
@@ -27,4 +27,13 @@ install_curl = RemoteCmdTask(
runner.register(install_curl)
```

RemoteCmdTask exposes several environments that you can use on your `cmd` and `cmd_path`

- `_CONFIG_HOST`
- `_CONFIG_PORT`
- `_CONFIG_SSH_KEY`
- `_CONFIG_USER`
- `_CONFIG_PASSWORD`
- `_CONFIG_MAP_<UPPER_SNAKE_CASE_NAME>`

🔖 [Table of Contents](../../README.md) / [Concepts](../README.md) / [Tasks](README.md)
30 changes: 21 additions & 9 deletions docs/concepts/tasks/rsync-task.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
🔖 [Table of Contents](../../README.md) / [Concepts](../README.md) / [Tasks](README.md)

# RSyncTask
# RsyncTask

```python
from zrb import (
runner, CmdTask, RSyncTask, RemoteConfig, PasswordInput, StrInput
runner, CmdTask, RsyncTask, RemoteConfig, PasswordInput, StrInput
)

upload = RSyncTask(
upload = RsyncTask(
name='upload',
inputs=[
PasswordInput(name='passsword'),
@@ -18,17 +18,20 @@ upload = RSyncTask(
RemoteConfig(
host='192.168.1.10,
user='ubuntu,
password='{{input.password}}'
password='{{input.password}}',
config_map={
'dir': '192-168-1-10'
}
)
],
is_remote_src=False,
is_remote_dst=True
src='{{input.src}}',
src='$_CONFIG_MAP_DIR/{{input.src}}',
is_remote_dst=True,
dst='{{input.dst}}',
)
runner.register(upload)

download = RSyncTask(
download = RsyncTask(
name='download',
inputs=[
PasswordInput(name='passsword'),
@@ -43,11 +46,20 @@ download = RSyncTask(
)
],
is_remote_src=True,
is_remote_dst=False
src='{{input.src}}',
dst='{{input.dst}}',
is_remote_dst=False,
dst='$_CONFIG_MAP_DIR/{{input.dst}}',
)
runner.register(download)
```

RsyncTask exposes several environments that you can use on your `src` and `dst`

- `_CONFIG_HOST`
- `_CONFIG_PORT`
- `_CONFIG_SSH_KEY`
- `_CONFIG_USER`
- `_CONFIG_PASSWORD`
- `_CONFIG_MAP_<UPPER_SNAKE_CASE_NAME>`

🔖 [Table of Contents](../../README.md) / [Concepts](../README.md) / [Tasks](README.md)
Original file line number Diff line number Diff line change
@@ -5,7 +5,35 @@
```python
from zrb import runner, CmdTask

prepare = CmdTask(
name='prepare-python-project',
cmd='pip install -R requirements.txt'
)
runner.register(prepare)

prepare = CmdTask(
name='prepare-node-project',
cmd='npm install'
)
runner.register(prepare)

run_fastapi = CmdTask(
name='run-fastapi',
cmd='uvicon main:app'
upstreams=[
prepare, # <-- Here is the problem, `npm install`` or `pip install`?
]
)
runner.register(run_fastapi)
```

You can see that `prepare-python-project` and `prepare-node-project` are assigned to the same variable.

Using that variable as upstream or checker will lead to a tricky situation. In our case, we want to perform `pip install` before starting Fast API. But since we re-assign the variable to `prepare-node-project`, we will got `npm install` instead.

# Avoiding the Problem

Beware of your variable name. Give your variable the same name as your task name.


🔖 [Table of Contents](../README.md) / [Oops, I Did It Again](README.md)
1 change: 0 additions & 1 deletion docs/quirks.md
Original file line number Diff line number Diff line change
@@ -18,6 +18,5 @@
- `env` will override each other, the last one takes greater priority
- If you define a `DockerComposeTask`, it will automatically fill your environment with the ones you use in your docker-compose file. The environment defined that way will have a very low priority. They will be overridden by both `env_files` and `env`.
- You cannot have an input named: `_task`, `_args` or `_execution_id`
- You cannot have an environment named `_execution_id`

🔖 [Table of Contents](README.md)
4 changes: 2 additions & 2 deletions docs/tutorials/copy-and-reuse-task.md
Original file line number Diff line number Diff line change
@@ -20,10 +20,10 @@ local_hello = hello.copy()

# Update name, input, and env
local_hello.set_name('hello-local')
local_hello.add_inputs(
local_hello.add_input(
StrInput(name='name', description='Name', default='dunia')
)
local_hello.add_envs(
local_hello.add_env(
Env(name='GREETINGS', os_name='', default='Halo')
)

25 changes: 15 additions & 10 deletions docs/tutorials/extending-cmd-task.md
Original file line number Diff line number Diff line change
@@ -56,18 +56,23 @@ class SlackPrintTask(CmdTask):
self._slack_app_token = slack_app_token
self._message = message

def run(self, *args: Any, **kwargs: Any):
# Inject environment variables
self.inject_env_map(
env_map={
'CHANNEL_ID': self.render_str(self._slack_channel_id),
'TOKEN': self.render_str(self._slack_app_token),
'MESSAGE': self.render_str(self._message)
}
def inject_envs(self):
self.add_envs(
Env(
name='CHANNEL_ID', os_name='',
default=self.render_str(self._slack_channel_id)
),
Env(
name='TOKEN', os_name='',
default=self.render_str(self._slack_app_token)
),
Env(
name='MESSAGE', os_name='',
default=self.render_str(self._message)
)
)
return super().run(*args, **kwargs)

def _get_cmd_str(self, *args: Any, **kwargs: Any):
def get_cmd_script(self, *args: Any, **kwargs: Any):
# contruct json payload and replace all `"` with `\\"`
json_payload = jsons.dumps({
'channel': '$CHANNEL_ID',
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -4,7 +4,7 @@ build-backend = "flit_core.buildapi"

[project]
name = "zrb"
version = "0.0.109"
version = "0.0.110"
authors = [
{ name="Go Frendi Gunawan", email="gofrendiasgard@gmail.com" },
]
4 changes: 2 additions & 2 deletions src/zrb/action/runner.py
Original file line number Diff line number Diff line change
@@ -28,7 +28,7 @@ def __init__(self, env_prefix: str = ''):

def register(self, task: AnyTask):
task.set_has_cli_interface()
cmd_name = task.get_complete_cmd_name()
cmd_name = task.get_full_cmd_name()
logger.debug(colored(f'Register task: {cmd_name}', attrs=['dark']))
self._tasks.append(task)
logger.debug(colored(f'Task registered: {cmd_name}', attrs=['dark']))
@@ -77,7 +77,7 @@ def _get_cli_group(self, task_group: TaskGroup) -> click.Group:
return group

def _create_cli_command(self, task: AnyTask) -> click.Command:
task_inputs = task.get_all_inputs()
task_inputs = task._get_combined_inputs()
task_cmd_name = task.get_cmd_name()
task_description = task.get_description()
task_function = task.to_function(
4 changes: 1 addition & 3 deletions src/zrb/builtin/git.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
from zrb.helper.typing import Any
from zrb.builtin.group import git_group
from zrb.task.decorator import python_task
from zrb.task.task import Task
from zrb.task_input.str_input import StrInput
from zrb.task_input.bool_input import BoolInput
from zrb.runner import runner
@@ -26,7 +25,7 @@
prompt='Commit hash/Tag',
default='HEAD'
),
BoolInput(
BoolInput(
name='include-new',
description='include new files',
prompt='Include new files',
@@ -52,7 +51,6 @@ async def get_file_changes(*args: Any, **kwargs: Any):
include_new = kwargs.get('include_new', True)
include_removed = kwargs.get('include_removed', True)
include_updated = kwargs.get('include_updated', True)
task: Task = kwargs['_task']
modified_file_states = get_modified_file_states(commit)
modified_file_keys = []
output = []
32 changes: 21 additions & 11 deletions src/zrb/helper/accessories/name.py
Original file line number Diff line number Diff line change
@@ -10,19 +10,29 @@ def get_random_name(
digit_count: int = 4
) -> str:
prefixes = [
"aurum", "argentum", "platinum", "mercurius", "sulfur", "sal",
"luna", "sol", "ferrum", "cuprum", "argent", "aurora", "citrin",
"coral", "diamond", "dragon", "emerald", "garnet", "jade", "onyx",
"opal", "pearl", "ruby", "sapphire", "topaz", "turquoise", "verde",
"zircon"
'albedo', 'argent', 'argentum', 'aurora', 'aurum', 'azure',
'basilisk', 'cerulean', 'chimeric', 'citrin', 'coral', 'crimson',
'diamond', 'draco', 'dragon', 'emerald', 'ethereal', 'ferrum',
'flammeus', 'garnet', 'glacial', 'glimmering', 'glistening', 'golden',
'helios', 'igneous', 'imperial', 'jade', 'luminous', 'luna', 'lunar',
'mystic', 'nephrite', 'nocturnal', 'obsidian', 'opal', 'pearl',
'platinum', 'prismatic', 'ruby', 'sapphire', 'serpentine', 'silver',
'sol', 'solar', 'spiritual', 'stellar', 'tempest', 'topaz',
'turquoise', 'verde', 'vermillion', 'vitreous', 'zephyr', 'zircon'
]
suffixes = [
"philosophorum", "spiritus", "tinctura", "essentia", "elixir",
"praeparatum", "aether", "vitae", "lapis", "metallum", "aureum",
"caelestis", "chrysopoeia", "cosmicum", "deum", "draconis",
"elementorum", "hermetica", "illuminationis", "magnum", "mysticum",
"occultum", "omnipotentis", "philosophia", "praestantissimum",
"quintessentia", "regeneratio", "universalis"
'aether', 'albedo', 'alchemy', 'arcana', 'aureum', 'aetheris',
'anima', 'astralis', 'caelestis', 'chrysopoeia', 'cosmicum',
'crystallum', 'deum', 'divinitas', 'draconis', 'elementorum', 'elixir',
'essentia', 'eternis', 'ethereus', 'fatum', 'flamma', 'fulgur',
'hermetica', 'ignis', 'illuminationis', 'imperium', 'incantatum',
'infinitum', 'lapis', 'lux', 'magicae', 'magnum', 'materia',
'metallum', 'mysticum', 'natura', 'occultum', 'omnipotentis',
'opulentia', 'philosophia', 'philosophorum', 'praeparatum',
'praestantissimum', 'prima', 'primordium', 'quintessentia',
'regeneratio', 'ritualis', 'sanctum', 'spiritus', 'tenebris',
'terra', 'tinctura', 'transmutationis', 'universalis', 'vapores',
'venenum', 'veritas', 'vitae', 'volatus'
]
prefix = random.choice(prefixes)
suffix = random.choice(suffixes)
26 changes: 13 additions & 13 deletions src/zrb/helper/env_map/fetch.py
Original file line number Diff line number Diff line change
@@ -16,7 +16,7 @@ def fetch_env_map_from_group(
sub_env_map: Mapping[str, str] = fetch_env_map_from_group(
env_map, sub_group
)
env_map = cascade_env_map(env_map, sub_env_map)
env_map = _cascade_env_map(env_map, sub_env_map)
return env_map


@@ -25,33 +25,33 @@ def fetch_env_map_from_task(
env_map: Mapping[str, str], task: AnyTask
):
task_env_map: Mapping[str, str] = {}
for env_file in task.get_env_files():
for env_file in task._get_env_files():
envs = env_file.get_envs()
task_env_map = add_envs_to_env_map(task_env_map, envs)
task_env_map = add_envs_to_env_map(task_env_map, task._envs)
env_map = cascade_env_map(env_map, task_env_map)
for upstream in task.get_upstreams():
task_env_map = _add_envs_to_env_map(task_env_map, envs)
task_env_map = _add_envs_to_env_map(task_env_map, task._envs)
env_map = _cascade_env_map(env_map, task_env_map)
for upstream in task._get_upstreams():
task_env_map = fetch_env_map_from_task(env_map, upstream)
for checker in task.get_checkers():
for checker in task._get_checkers():
task_env_map = fetch_env_map_from_task(env_map, checker)
return env_map


@typechecked
def add_envs_to_env_map(
def _add_envs_to_env_map(
env_map: Mapping[str, str], envs: List[Env]
) -> Mapping[str, str]:
for env in envs:
if env.os_name == '':
continue
env_name = get_env_name(env)
env_default = get_env_default(env)
env_name = _get_env_name(env)
env_default = _get_env_default(env)
env_map[env_name] = env_default
return env_map


@typechecked
def cascade_env_map(
def _cascade_env_map(
env_map: Mapping[str, str],
other_env_map: Mapping[str, str]
) -> Mapping[str, str]:
@@ -63,14 +63,14 @@ def cascade_env_map(


@typechecked
def get_env_name(env: Env) -> str:
def _get_env_name(env: Env) -> str:
if env.os_name is None:
return env.name
return env.os_name


@typechecked
def get_env_default(env: Env) -> str:
def _get_env_default(env: Env) -> str:
if is_probably_jinja(env.default):
return ''
return env.default
Loading