From 13b31804b59a6b52c24aae158c815e9538c24ebe Mon Sep 17 00:00:00 2001 From: goFrendiAsgard Date: Thu, 19 Oct 2023 20:27:12 +0700 Subject: [PATCH 1/9] update version --- pyproject.toml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pyproject.toml b/pyproject.toml index 1fabd7b4..9ba5374e 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "flit_core.buildapi" [project] name = "zrb" -version = "0.0.105" +version = "0.0.106" authors = [ { name="Go Frendi Gunawan", email="gofrendiasgard@gmail.com" }, ] From fd873187436e76a85499071297f0a469e37725d1 Mon Sep 17 00:00:00 2001 From: goFrendiAsgard Date: Thu, 9 Nov 2023 13:19:14 +0700 Subject: [PATCH 2/9] WIP: release 0.0.111 --- pyproject.toml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pyproject.toml b/pyproject.toml index 24451fe5..e264e157 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "flit_core.buildapi" [project] name = "zrb" -version = "0.0.110" +version = "0.0.111" authors = [ { name="Go Frendi Gunawan", email="gofrendiasgard@gmail.com" }, ] From 162779c5d8a867829b27ece12646c7049353293a Mon Sep 17 00:00:00 2001 From: goFrendiAsgard Date: Fri, 10 Nov 2023 09:31:32 +0700 Subject: [PATCH 3/9] update getting started --- docs/getting-started.md | 114 ++++++++++++++++++++++++++++------------ 1 file changed, 80 insertions(+), 34 deletions(-) diff --git a/docs/getting-started.md b/docs/getting-started.md index 5557b12d..ed4e0d2f 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -2,18 +2,24 @@ # Getting started -Zrb is an automation tool. With Zrb you can run tasks using command-line-interface. +Welcome to Zrb's getting started guide. -You can also define Zrb tasks using Python. +We will cover all the basic you need to know before working with Zrb. You will learn about: -This getting-started guide covers all the basic you need to know before working with Zrb. +- How to run a task +- How to create a Zrb project +- How to create a simple task +- How to define upstreams +- How to create a long running task + +This guide assume you have some familiarity with CLI and Python. # Running a task -You can run any Zrb task by invoking the following pattern: +Once you installed Zrb, you can run some built-in tasks immediately. To run any Zrb task, you need to follow the following pattern: ```bash -zrb [task-groups] [task-parameters] +zrb [task-groups...] [task-parameters...] ``` For example, you want to run `encode` that is located under `base64` group, you can do so by execute the following command: @@ -25,22 +31,32 @@ zrb base64 encode --text "non-credential-string" ``` Support zrb growth and development! โ˜• Donate at: https://stalchmst.com/donation -๐Ÿ™ Submit issues/pull requests at: https://github.com/state-alchemists/zrb +๐Ÿ™ Submit issues/PR at: https://github.com/state-alchemists/zrb ๐Ÿค Follow us at: https://twitter.com/zarubastalchmst -๐Ÿค– โžœ 2023-06-11T05:09:06.283002 โš™ 3549 โžค 1 of 1 โ€ข ๐Ÿ‹ zrb base64 encode โ€ข zrb base64 encode completed in 0.11719107627868652 seconds -To run again: zrb base64 encode --text "non-credential-string" +๐Ÿค– โ—‹ โ—ท 2023-11-10 09:08:33.183 โ 35276 โ†’ 1/1 ๐ŸŽ zrb base64 encode โ€ข Completed in 0.051436424255371094 seconds bm9uLWNyZWRlbnRpYWwtc3RyaW5n +To run again: zrb base64 encode --text "non-credential-string" ``` -Related tasks are usually located under the same group. For example, you have `decode` task under `base64` group as well. +> __WARNING:__ Anyone can easily decode a base64-encoded string. Don't use it to encrypt your password or any important credentials. + +Related tasks are usually located under the same `task-group`. + +For example, you have all base64 related tasks (e.g., `decode` and `encode`) under `base64` task-group. + +Let's try to decode our base64-encoded text: ```bash zrb base64 decode --text "bm9uLWNyZWRlbnRpYWwtc3RyaW5n" ``` -# Getting available tasks/task groups +You should get your original text back. + +> __NOTE:__ You don't have to memorize any `task-group` or `task` name. The next two subsections will show you how to locate and execute any `task`. -To see all available task/task groups, you can type `zrb` and press enter. +## Getting available tasks/task groups + +To find out what tasks/task-groups are available, you can type `zrb` and press enter. ```bash zrb @@ -49,26 +65,52 @@ zrb ``` Usage: zrb [OPTIONS] COMMAND [ARGS]... - Super framework for your super app. + bb + zzzzz rr rr bb + zz rrr r bbbbbb + zz rr bb bb + zzzzz rr bbbbbb 0.0.109 + _ _ . . . _ . _ . . . + +Super framework for your super app. + +โ˜• Donate at: https://stalchmst.com/donation +๐Ÿ™ Submit issues/PR at: https://github.com/state-alchemists/zrb +๐Ÿค Follow us at: https://twitter.com/zarubastalchmst Options: --help Show this message and exit. Commands: - base64 Base64 operations - concat concat - devtool Developer tools management - env Environment variable management - eval Evaluate Python expression - md5 MD5 operations - explain Explain things - project Project management - ubuntu Ubuntu related commands - update Update zrb - version Get Zrb version -``` - -You can keep doing this until you find the task you want to execute + base64 Base64 operations + build Build Zrb + build-image Build docker image + build-latest-image Build docker image + devtool Developer tools management + env Environment variable management + eval Evaluate Python expression + explain Explain things + git Git related commands + install-symlink Install Zrb as symlink + md5 MD5 operations + playground Playground related tasks + process Process related commands + project Project management + publish Publish new version + publish-pip Publish zrb to pypi + publish-pip-test Publish zrb to testpypi + push-image Push docker image + push-latest-image Push docker image + serve-test Serve zrb test result + start-container Run docker container + stop-container remove docker container + test Run zrb test + ubuntu Ubuntu related commands + update Update zrb + version Get Zrb version +``` + +You can then type `zrb [task-group...]` until you find the task you want to execute. For example, you can invoke the following command to see what tasks are available under `base64` group: ```bash zrb base64 @@ -87,9 +129,13 @@ Commands: encode Encode base64 task ``` -# Using prompt +> __NOTE:__ A `task-group` might contains some `tasks` or other `task-groups` -Once you find your task, you can just type the task without bothering about the parameters. Zrb will automatically prompt you to fill the parameter interactively. +## Using input prompt + +Once you find the task you want to execute, you can just type `zrb [task-groups...] ` without bothering about `task-parameters`. + +Zrb will automatically prompt you to provide the parameter interactively. ```bash zrb base64 encode @@ -99,20 +145,20 @@ zrb base64 encode Text []: non-credential-string Support zrb growth and development! โ˜• Donate at: https://stalchmst.com/donation -๐Ÿ™ Submit issues/pull requests at: https://github.com/state-alchemists/zrb +๐Ÿ™ Submit issues/PR at: https://github.com/state-alchemists/zrb ๐Ÿค Follow us at: https://twitter.com/zarubastalchmst -๐Ÿค– โžœ 2023-06-11T05:27:07.824839 โš™ 3740 โžค 1 of 1 โ€ข ๐ŸŒ zrb base64 encode โ€ข zrb base64 encode completed in 0.11709976196289062 seconds -To run again: zrb base64 encode --text "non-credential-string" +๐Ÿค– โ—‹ โ—ท 2023-11-10 09:10:58.805 โ 35867 โ†’ 1/1 ๐Ÿˆ zrb base64 encode โ€ข Completed in 0.053427934646606445 seconds bm9uLWNyZWRlbnRpYWwtc3RyaW5n +To run again: zrb base64 encode --text "non-credential-string" ``` > __NOTE:__ To disable prompt, you can set `ZRB_SHOW_PROMPT` to `0` or `false`. Please refer to [configuration section](./configurations.md) for more information. # Creating a project -To make things more manageable, you can put related task definitions and resources under the same project. +To make things more manageable, you can create a project to contain all your resources and task definitions. -You can create a project under `my-project` directory by invoking the following command: +Suppose you want to create a project under `my-project`, then you can invoke the following command: ```bash zrb project create --project-dir my-project @@ -140,7 +186,7 @@ drwxr-xr-x 2 gofrendi gofrendi 4096 Jun 11 05:29 src -rw-r--r-- 1 gofrendi gofrendi 54 Jun 11 05:29 zrb_init.py ``` -Every Zrb project contains a file named `zrb_init.py`. This file is your entry point to define all tasks/configurations. +Every Zrb project has a file named `zrb_init.py` on it's top level directory. This file is your entry point to define your task definitions. It is recommended that you define your tasks under `_automate` directory and import them into your `zrb_init.py`. This will help you manage the [separation of concerns](https://en.wikipedia.org/wiki/Separation_of_concerns). From 70218e6a1a00540c35c8809f9bef733e9fea7e11 Mon Sep 17 00:00:00 2001 From: goFrendiAsgard Date: Sat, 11 Nov 2023 08:35:38 +0700 Subject: [PATCH 4/9] update docs --- docs/concepts/tasks/README.md | 4 ++-- docs/getting-started.md | 14 +++++++------- .../not-registering-a-task-to-a-runner.md | 2 +- .../development-to-deployment-low-code.md | 4 ++-- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/docs/concepts/tasks/README.md b/docs/concepts/tasks/README.md index 8ca7580c..4794196e 100644 --- a/docs/concepts/tasks/README.md +++ b/docs/concepts/tasks/README.md @@ -38,7 +38,7 @@ Task CmdTask ResourceMaker FlowTask BaseRemoteCmdTask HttpChecker PortCheck Aside from the documentation, you can always dive down into [the source code](https://github.com/state-alchemists/zrb/tree/main/src/zrb/task) to see the detail implementation. -> __Note:__ Never initiate `BaseTask` directly, use `Task` instead. +> __๐Ÿ“ NOTE:__ Never initiate `BaseTask` directly, use `Task` instead. # Task Overview @@ -116,7 +116,7 @@ As `start_jupyterlab` has `show_banner` as it's upstream, you can expect the `sh A task might also have multiple upstreams. In that case, the upstreams will be executed concurrently. -> __Note:__ Only tasks registered to `runner` are directly accessible from the CLI. +> __๐Ÿ“ NOTE:__ Only tasks registered to `runner` are directly accessible from the CLI. # Task Lifecycle diff --git a/docs/getting-started.md b/docs/getting-started.md index ed4e0d2f..858e6d15 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -6,8 +6,8 @@ Welcome to Zrb's getting started guide. We will cover all the basic you need to know before working with Zrb. You will learn about: -- How to run a task -- How to create a Zrb project +- [How to run a task](#running-a-task) +- [How to create a project](#creating-a-project) - How to create a simple task - How to define upstreams - How to create a long running task @@ -38,7 +38,7 @@ bm9uLWNyZWRlbnRpYWwtc3RyaW5n To run again: zrb base64 encode --text "non-credential-string" ``` -> __WARNING:__ Anyone can easily decode a base64-encoded string. Don't use it to encrypt your password or any important credentials. +> __โš ๏ธ WARNING:__ Anyone can easily decode a base64-encoded string. Don't use it to encrypt your password or any important credentials. Related tasks are usually located under the same `task-group`. @@ -52,7 +52,7 @@ zrb base64 decode --text "bm9uLWNyZWRlbnRpYWwtc3RyaW5n" You should get your original text back. -> __NOTE:__ You don't have to memorize any `task-group` or `task` name. The next two subsections will show you how to locate and execute any `task`. +> __๐Ÿ’ก HINT:__ You don't have to memorize any `task-group` or `task` name. The next two subsections will show you how to locate and execute any `task` without memorize anything. ## Getting available tasks/task groups @@ -129,7 +129,7 @@ Commands: encode Encode base64 task ``` -> __NOTE:__ A `task-group` might contains some `tasks` or other `task-groups` +> __๐Ÿ“ NOTE:__ A `task-group` might contains some `tasks` or other `task-groups` ## Using input prompt @@ -152,11 +152,11 @@ bm9uLWNyZWRlbnRpYWwtc3RyaW5n To run again: zrb base64 encode --text "non-credential-string" ``` -> __NOTE:__ To disable prompt, you can set `ZRB_SHOW_PROMPT` to `0` or `false`. Please refer to [configuration section](./configurations.md) for more information. +> __๐Ÿ“ NOTE:__ To disable prompt, you can set `ZRB_SHOW_PROMPT` to `0` or `false`. When prompts are disabled, Zrb will automatically use default values. Please refer to [configuration section](./configurations.md) for more information. # Creating a project -To make things more manageable, you can create a project to contain all your resources and task definitions. +To make things more manageable, you need to put all your resources and task definitions in a `project`. Suppose you want to create a project under `my-project`, then you can invoke the following command: diff --git a/docs/oops-i-did-it-again/not-registering-a-task-to-a-runner.md b/docs/oops-i-did-it-again/not-registering-a-task-to-a-runner.md index f6fdf1b6..002d7bb4 100644 --- a/docs/oops-i-did-it-again/not-registering-a-task-to-a-runner.md +++ b/docs/oops-i-did-it-again/not-registering-a-task-to-a-runner.md @@ -18,7 +18,7 @@ zrb hello If you want your task to be available through the CLI, you need to register it to Zrb's runner. -> __NOTE:__ A task that is not registered to Zrb's runner will not be available through the CLI, but can still be used as upstreams/checkers. +> __๐Ÿ“ NOTE:__ A task that is not registered to Zrb's runner will not be available through the CLI, but can still be used as upstreams/checkers. # Avoiding the Problem diff --git a/docs/tutorials/development-to-deployment-low-code.md b/docs/tutorials/development-to-deployment-low-code.md index c15bec81..57af499b 100644 --- a/docs/tutorials/development-to-deployment-low-code.md +++ b/docs/tutorials/development-to-deployment-low-code.md @@ -26,7 +26,7 @@ Before starting the demo, you need to make sure you have the following software - Zrb - Access to a Kubernetes cluster -> __Note:__ Installing docker desktop will give you docker + docker compose extension, as well as local kubernetes cluster (if enabled). +> __๐Ÿ“ NOTE:__ Installing docker desktop will give you docker + docker compose extension, as well as local kubernetes cluster (if enabled). Once everything is set, you can run the following commands on your terminal: @@ -158,7 +158,7 @@ To run `myapp` as containers you can invoke: zrb project start-myapp-container --myapp-run-mode microservices ``` -> __Note:__ You can also run the container as a monolith if you want to. Just invoke `zrb project start-myapp-container --myapp-run-mode monolith` +> __๐Ÿ“ NOTE:__ You can also run the container as a monolith if you want to. Just invoke `zrb project start-myapp-container --myapp-run-mode monolith` Now let's see how things look like by invoking: From c038f37b0c36a8537209c20bf1a855422387e8cb Mon Sep 17 00:00:00 2001 From: goFrendiAsgard Date: Sat, 11 Nov 2023 15:33:30 +0700 Subject: [PATCH 5/9] update getting started example --- README.md | 162 ++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 120 insertions(+), 42 deletions(-) diff --git a/README.md b/README.md index 1a9bf275..93bc8704 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,126 @@ Zrb is a [CLI-based](https://en.wikipedia.org/wiki/Command-line_interface) autom To use Zrb, you need to be familiar with CLI. -Zrb task definitions are written in [Python](https://www.python.org/), and we have a [very good reason](https://github.com/state-alchemists/zrb/blob/main/docs/faq/why-python.md) behind the decision. +Zrb task definitions are written in [Python](https://www.python.org/), and we have a [good reason](https://github.com/state-alchemists/zrb/blob/main/docs/faq/why-python.md) behind the decision. + +## Zrb is A Task-Automation Tool + +At the very core, Zrb is a task automation tool. It helps you to automate some tedious jobs so that you can focus on what matters. + +Let's say you want to describe the statistics property of any public CSV. To do this, you need to perform three tasks like the following: + +- Install pandas. +- Download the CSV dataset (at the same time). +- Show statistics properties of the CSV dataset using pandas (right after the two first tasks are completed). + +``` + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ”‚ โ”‚ + โ”‚ Install Pandas โ”œโ”€โ” + โ”‚ โ”‚ โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ–บโ”‚ โ”‚ + โ”‚ Show Statistics โ”‚ + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ–บโ”‚ โ”‚ + โ”‚ โ”‚ โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ Download Dataset โ”œโ”€โ”˜ + โ”‚ โ”‚ + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ +``` + +To do this, you can create a file named `zrb_init.py` and define the tasks as follows: + +```python +# File name: zrb_init.py +from zrb import runner, CmdTask, python_task, StrInput + +# Define a task to install pandas +install_pandas = CmdTask( + name='install-pandas', + cmd='pip install pandas' +) + +# Define a task to download dataset +download_dataset = CmdTask( + name='download-dataset', + inputs=[ + # Allow user to put the CSV dataset URL. + StrInput( + name='url', + default='https://raw.githubusercontent.com/state-alchemists/datasets/main/iris.csv' + ) + ], + cmd='wget -O dataset.csv {{input.url}}' +) + +# Define a task to show the statistics properties of the dataset +show_stat = CmdTask( + name='show-stat', + upstreams=[ + # Let the following tasks to be show_stat's upstream + download_dataset, + install_pandas + ], + cmd='python -c "import pandas as pd; df=pd.read_csv(\'dataset.csv\'); print(df.describe())"' # noqa +) + +# Register show_stat, so that the task is accessible from the CLI (i.e., zrb show-stat) +runner.register(show_stat) +``` + +Once you do so, you can invoke the task and get the output. + +```bash +zrb show-stat +``` + +``` +Url [https://raw.githubusercontent.com/state-alchemists/datasets/main/iris.csv]: +๐Ÿค– โ—‹ โ 27694 โ†’ 1/3 ๐Ÿ‰ download-dataset โ€ข Run script: wget -O dataset.csv https://raw.githubusercontent.com/state-alchemists/datasets/main/iris.csv +๐Ÿค– โ—‹ โ 27694 โ†’ 1/3 ๐Ÿ‰ download-dataset โ€ข Working directory: /home/gofrendi/playground/myproject +๐Ÿค– โ—‹ โ 27694 โ†’ 1/3 ๐ŸฆŠ install-pandas โ€ข Run script: pip install pandas +๐Ÿค– โ—‹ โ 27694 โ†’ 1/3 ๐ŸฆŠ install-pandas โ€ข Working directory: /home/gofrendi/playground/myproject +๐Ÿค– โ–ณ โ 27700 โ†’ 1/3 ๐Ÿ‰ download-dataset โ€ข --2023-11-11 15:29:01-- https://raw.githubusercontent.com/state-alchemists/datasets/main/iris.csv +๐Ÿค– โ–ณ โ 27700 โ†’ 1/3 ๐Ÿ‰ download-dataset โ€ข Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.111.133, 185.199.109.133, ... +๐Ÿค– โ–ณ โ 27700 โ†’ 1/3 ๐Ÿ‰ download-dataset โ€ข Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected. +๐Ÿค– โ—‹ โ 27702 โ†’ 1/3 ๐ŸฆŠ install-pandas โ€ข Requirement already satisfied: pandas in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (2.1.3) +๐Ÿค– โ—‹ โ 27702 โ†’ 1/3 ๐ŸฆŠ install-pandas โ€ข Requirement already satisfied: numpy<2,>=1.22.4 in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (from pandas) (1.26.1) +๐Ÿค– โ–ณ โ 27700 โ†’ 1/3 ๐Ÿ‰ download-dataset โ€ข HTTP request sent, awaiting response... 200 OK +๐Ÿค– โ–ณ โ 27700 โ†’ 1/3 ๐Ÿ‰ download-dataset โ€ข Length: 4606 (4.5K) [text/plain] +๐Ÿค– โ–ณ โ 27700 โ†’ 1/3 ๐Ÿ‰ download-dataset โ€ข Saving to: โ€˜dataset.csvโ€™ +๐Ÿค– โ–ณ โ 27700 โ†’ 1/3 ๐Ÿ‰ download-dataset โ€ข 0K .... 100% 4.19M=0.001s +๐Ÿค– โ–ณ โ 27700 โ†’ 1/3 ๐Ÿ‰ download-dataset โ€ข 2023-11-11 15:29:01 (4.19 MB/s) - โ€˜dataset.csvโ€™ saved [4606/4606] +๐Ÿค– โ–ณ โ 27700 โ†’ 1/3 ๐Ÿ‰ download-dataset โ€ข +๐Ÿค– โ—‹ โ 27694 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข Run script: python -c "import pandas as pd; df=pd.read_csv('dataset.csv'); print(df.describe())" +๐Ÿค– โ—‹ โ 27694 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข Working directory: /home/gofrendi/playground/myproject +๐Ÿค– โ—‹ โ 27746 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข sepal_length sepal_width petal_length petal_width +๐Ÿค– โ—‹ โ 27746 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข count 150.000000 150.000000 150.000000 150.000000 +๐Ÿค– โ—‹ โ 27746 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข mean 5.843333 3.054000 3.758667 1.198667 +๐Ÿค– โ—‹ โ 27746 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข std 0.828066 0.433594 1.764420 0.763161 +๐Ÿค– โ—‹ โ 27746 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข min 4.300000 2.000000 1.000000 0.100000 +๐Ÿค– โ—‹ โ 27746 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข 25% 5.100000 2.800000 1.600000 0.300000 +๐Ÿค– โ—‹ โ 27746 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข 50% 5.800000 3.000000 4.350000 1.300000 +๐Ÿค– โ—‹ โ 27746 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข 75% 6.400000 3.300000 5.100000 1.800000 +๐Ÿค– โ—‹ โ 27746 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข max 7.900000 4.400000 6.900000 2.500000 +Support zrb growth and development! +โ˜• Donate at: https://stalchmst.com/donation +๐Ÿ™ Submit issues/PR at: https://github.com/state-alchemists/zrb +๐Ÿค Follow us at: https://twitter.com/zarubastalchmst +๐Ÿค– โ—‹ โ 27746 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข Completed in 1.3806817531585693 seconds + sepal_length sepal_width petal_length petal_width +count 150.000000 150.000000 150.000000 150.000000 +mean 5.843333 3.054000 3.758667 1.198667 +std 0.828066 0.433594 1.764420 0.763161 +min 4.300000 2.000000 1.000000 0.100000 +25% 5.100000 2.800000 1.600000 0.300000 +50% 5.800000 3.000000 4.350000 1.300000 +75% 6.400000 3.300000 5.100000 1.800000 +max 7.900000 4.400000 6.900000 2.500000 +To run again: zrb show-stat --url "https://raw.githubusercontent.com/state-alchemists/datasets/main/iris.csv" +``` + + +To learn more about this, you can visit [our getting started guide](https://github.com/state-alchemists/zrb/blob/main/docs/getting-started.md). + ## Zrb is A Low-Code Framework @@ -60,47 +179,6 @@ zrb project deploy-fastapp --fastapp-deploy-mode "microservices" You can visit [our tutorials](https://github.com/state-alchemists/zrb/blob/main/docs/tutorials/README.md) to see more cool tricks. -## Zrb is A Task-Automation Tool - -Aside from the built-in capabilities, Zrb also allows you to define your automation commands in Python. To do so, you must create/modify a file named `zrb_init.py`. - -```python -# filename: zrb_init.py -from zrb import runner, CmdTask, StrInput - -hello = CmdTask( - name='hello', - inputs=[StrInput(name='name', description='Name', default='world')], - cmd='echo Hello {{input.name}}' -) -runner.register(hello) -``` - -Once defined, your command will be instantly available from the CLI: - -```bash -zrb hello -# You can also provide the parameter directly: -# zrb hello --name "Go Frendi" -``` - -``` -Name [world]: Go Frendi -๐Ÿค– โ—‹ โ—ท 2023-09-18 07:37:40.849 โ 47932 โ†’ 1/3 ๐ŸŒ zrb hello โ€ข Run script: echo Hello Go Frendi -๐Ÿค– โ—‹ โ—ท 2023-09-18 07:37:40.849 โ 47932 โ†’ 1/3 ๐ŸŒ zrb hello โ€ข Working directory: /home/gofrendi/zrb/playground -๐Ÿค– โ—‹ โ—ท 2023-09-18 07:37:40.854 โ 47933 โ†’ 1/3 ๐ŸŒ zrb hello โ€ข Hello Go Frendi -Support zrb growth and development! -โ˜• Donate at: https://stalchmst.com/donation -๐Ÿ™ Submit issues/PR at: https://github.com/state-alchemists/zrb -๐Ÿค Follow us at: https://twitter.com/zarubastalchmst -๐Ÿค– โ—‹ โ—ท 2023-09-18 07:37:40.899 โ 47933 โ†’ 1/3 ๐ŸŒ zrb hello โ€ข Completed in 0.052213191986083984 seconds -To run again: zrb hello --name "Go Frendi" -Hello Go Frendi -``` - -To learn more about this, you can visit [our getting started guide](https://github.com/state-alchemists/zrb/blob/main/docs/getting-started.md). - - # ๐Ÿซฐ Installation ## โš™๏ธ In Local Machine From 6f17c36518c216a0ef1351698e44f0f6c3bd2be9 Mon Sep 17 00:00:00 2001 From: goFrendiAsgard Date: Sat, 11 Nov 2023 17:04:36 +0700 Subject: [PATCH 6/9] update README --- README.md | 124 ++++++++++++++++++++++++++++++++++++----------------- install.sh | 44 +++++++++++++++++++ 2 files changed, 129 insertions(+), 39 deletions(-) create mode 100755 install.sh diff --git a/README.md b/README.md index 93bc8704..575806ef 100644 --- a/README.md +++ b/README.md @@ -21,9 +21,10 @@ Let's say you want to describe the statistics property of any public CSV. To do - Show statistics properties of the CSV dataset using pandas (right after the two first tasks are completed). ``` + ๐Ÿผ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ โ”‚ - โ”‚ Install Pandas โ”œโ”€โ” + โ”‚ Install Pandas โ”œโ”€โ” ๐Ÿ“Š โ”‚ โ”‚ โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ–บโ”‚ โ”‚ โ”‚ Show Statistics โ”‚ @@ -32,6 +33,7 @@ Let's say you want to describe the statistics property of any public CSV. To do โ”‚ Download Dataset โ”œโ”€โ”˜ โ”‚ โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โฌ‡๏ธ ``` To do this, you can create a file named `zrb_init.py` and define the tasks as follows: @@ -40,13 +42,13 @@ To do this, you can create a file named `zrb_init.py` and define the tasks as fo # File name: zrb_init.py from zrb import runner, CmdTask, python_task, StrInput -# Define a task to install pandas +# ๐Ÿผ Define a task to install pandas install_pandas = CmdTask( name='install-pandas', cmd='pip install pandas' ) -# Define a task to download dataset +# โฌ‡๏ธ Define a task to download dataset download_dataset = CmdTask( name='download-dataset', inputs=[ @@ -59,11 +61,11 @@ download_dataset = CmdTask( cmd='wget -O dataset.csv {{input.url}}' ) -# Define a task to show the statistics properties of the dataset +# ๐Ÿ“Š Define a task to show the statistics properties of the dataset show_stat = CmdTask( name='show-stat', upstreams=[ - # Let the following tasks to be show_stat's upstream + # Let the following tasks to be show_stat's upstream: download_dataset, install_pandas ], @@ -71,6 +73,7 @@ show_stat = CmdTask( ) # Register show_stat, so that the task is accessible from the CLI (i.e., zrb show-stat) +# WARNING: You should register the variable, not the name of the task runner.register(show_stat) ``` @@ -80,39 +83,51 @@ Once you do so, you can invoke the task and get the output. zrb show-stat ``` +> __๐Ÿ“ NOTE:__ When executing a Zrb task, you can also provide the parameter directly, for example: +> +> ```bash +> zrb show-stat --url https://raw.githubusercontent.com/state-alchemists/datasets/main/iris.csv +> ``` + ``` Url [https://raw.githubusercontent.com/state-alchemists/datasets/main/iris.csv]: -๐Ÿค– โ—‹ โ 27694 โ†’ 1/3 ๐Ÿ‰ download-dataset โ€ข Run script: wget -O dataset.csv https://raw.githubusercontent.com/state-alchemists/datasets/main/iris.csv -๐Ÿค– โ—‹ โ 27694 โ†’ 1/3 ๐Ÿ‰ download-dataset โ€ข Working directory: /home/gofrendi/playground/myproject -๐Ÿค– โ—‹ โ 27694 โ†’ 1/3 ๐ŸฆŠ install-pandas โ€ข Run script: pip install pandas -๐Ÿค– โ—‹ โ 27694 โ†’ 1/3 ๐ŸฆŠ install-pandas โ€ข Working directory: /home/gofrendi/playground/myproject -๐Ÿค– โ–ณ โ 27700 โ†’ 1/3 ๐Ÿ‰ download-dataset โ€ข --2023-11-11 15:29:01-- https://raw.githubusercontent.com/state-alchemists/datasets/main/iris.csv -๐Ÿค– โ–ณ โ 27700 โ†’ 1/3 ๐Ÿ‰ download-dataset โ€ข Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.111.133, 185.199.109.133, ... -๐Ÿค– โ–ณ โ 27700 โ†’ 1/3 ๐Ÿ‰ download-dataset โ€ข Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected. -๐Ÿค– โ—‹ โ 27702 โ†’ 1/3 ๐ŸฆŠ install-pandas โ€ข Requirement already satisfied: pandas in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (2.1.3) -๐Ÿค– โ—‹ โ 27702 โ†’ 1/3 ๐ŸฆŠ install-pandas โ€ข Requirement already satisfied: numpy<2,>=1.22.4 in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (from pandas) (1.26.1) -๐Ÿค– โ–ณ โ 27700 โ†’ 1/3 ๐Ÿ‰ download-dataset โ€ข HTTP request sent, awaiting response... 200 OK -๐Ÿค– โ–ณ โ 27700 โ†’ 1/3 ๐Ÿ‰ download-dataset โ€ข Length: 4606 (4.5K) [text/plain] -๐Ÿค– โ–ณ โ 27700 โ†’ 1/3 ๐Ÿ‰ download-dataset โ€ข Saving to: โ€˜dataset.csvโ€™ -๐Ÿค– โ–ณ โ 27700 โ†’ 1/3 ๐Ÿ‰ download-dataset โ€ข 0K .... 100% 4.19M=0.001s -๐Ÿค– โ–ณ โ 27700 โ†’ 1/3 ๐Ÿ‰ download-dataset โ€ข 2023-11-11 15:29:01 (4.19 MB/s) - โ€˜dataset.csvโ€™ saved [4606/4606] -๐Ÿค– โ–ณ โ 27700 โ†’ 1/3 ๐Ÿ‰ download-dataset โ€ข -๐Ÿค– โ—‹ โ 27694 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข Run script: python -c "import pandas as pd; df=pd.read_csv('dataset.csv'); print(df.describe())" -๐Ÿค– โ—‹ โ 27694 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข Working directory: /home/gofrendi/playground/myproject -๐Ÿค– โ—‹ โ 27746 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข sepal_length sepal_width petal_length petal_width -๐Ÿค– โ—‹ โ 27746 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข count 150.000000 150.000000 150.000000 150.000000 -๐Ÿค– โ—‹ โ 27746 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข mean 5.843333 3.054000 3.758667 1.198667 -๐Ÿค– โ—‹ โ 27746 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข std 0.828066 0.433594 1.764420 0.763161 -๐Ÿค– โ—‹ โ 27746 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข min 4.300000 2.000000 1.000000 0.100000 -๐Ÿค– โ—‹ โ 27746 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข 25% 5.100000 2.800000 1.600000 0.300000 -๐Ÿค– โ—‹ โ 27746 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข 50% 5.800000 3.000000 4.350000 1.300000 -๐Ÿค– โ—‹ โ 27746 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข 75% 6.400000 3.300000 5.100000 1.800000 -๐Ÿค– โ—‹ โ 27746 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข max 7.900000 4.400000 6.900000 2.500000 +๐Ÿค– โ—‹ โ 36725 โ†’ 1/3 ๐Ÿจ download-dataset โ€ข Run script: wget -O dataset.csv https://raw.githubusercontent.com/state-alchemists/datasets/main/iris.csv +๐Ÿค– โ—‹ โ 36725 โ†’ 1/3 ๐Ÿจ download-dataset โ€ข Working directory: /home/gofrendi/playground/myproject +๐Ÿค– โ—‹ โ 36725 โ†’ 1/3 ๐Ÿป install-pandas โ€ข Run script: pip install pandas +๐Ÿค– โ—‹ โ 36725 โ†’ 1/3 ๐Ÿป install-pandas โ€ข Working directory: /home/gofrendi/playground/myproject +๐Ÿค– โ–ณ โ 36746 โ†’ 1/3 ๐Ÿจ download-dataset โ€ข --2023-11-11 16:15:54-- https://raw.githubusercontent.com/state-alchemists/datasets/main/iris.csv +๐Ÿค– โ–ณ โ 36746 โ†’ 1/3 ๐Ÿจ download-dataset โ€ข Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.109.133, 185.199.108.133, ... +๐Ÿค– โ–ณ โ 36746 โ†’ 1/3 ๐Ÿจ download-dataset โ€ข Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected. +๐Ÿค– โ—‹ โ 36748 โ†’ 1/3 ๐Ÿป install-pandas โ€ข Requirement already satisfied: pandas in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (2.1.3) +๐Ÿค– โ–ณ โ 36746 โ†’ 1/3 ๐Ÿจ download-dataset โ€ข HTTP request sent, awaiting response... 200 OK +๐Ÿค– โ–ณ โ 36746 โ†’ 1/3 ๐Ÿจ download-dataset โ€ข Length: 4606 (4.5K) [text/plain] +๐Ÿค– โ–ณ โ 36746 โ†’ 1/3 ๐Ÿจ download-dataset โ€ข Saving to: โ€˜dataset.csvโ€™ +๐Ÿค– โ–ณ โ 36746 โ†’ 1/3 ๐Ÿจ download-dataset โ€ข +๐Ÿค– โ–ณ โ 36746 โ†’ 1/3 ๐Ÿจ download-dataset โ€ข 0K .... 100% 1.40M=0.003s +๐Ÿค– โ–ณ โ 36746 โ†’ 1/3 ๐Ÿจ download-dataset โ€ข +๐Ÿค– โ–ณ โ 36746 โ†’ 1/3 ๐Ÿจ download-dataset โ€ข 2023-11-11 16:15:54 (1.40 MB/s) - โ€˜dataset.csvโ€™ saved [4606/4606] +๐Ÿค– โ–ณ โ 36746 โ†’ 1/3 ๐Ÿจ download-dataset โ€ข +๐Ÿค– โ—‹ โ 36748 โ†’ 1/3 ๐Ÿป install-pandas โ€ข Requirement already satisfied: numpy<2,>=1.22.4 in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (from pandas) (1.26.1) +๐Ÿค– โ—‹ โ 36748 โ†’ 1/3 ๐Ÿป install-pandas โ€ข Requirement already satisfied: python-dateutil>=2.8.2 in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (from pandas) (2.8.2) +๐Ÿค– โ—‹ โ 36748 โ†’ 1/3 ๐Ÿป install-pandas โ€ข Requirement already satisfied: pytz>=2020.1 in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (from pandas) (2023.3.post1) +๐Ÿค– โ—‹ โ 36748 โ†’ 1/3 ๐Ÿป install-pandas โ€ข Requirement already satisfied: tzdata>=2022.1 in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (from pandas) (2023.3) +๐Ÿค– โ—‹ โ 36748 โ†’ 1/3 ๐Ÿป install-pandas โ€ข Requirement already satisfied: six>=1.5 in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (from python-dateutil>=2.8.2->pandas) (1.16.0) +๐Ÿค– โ—‹ โ 36725 โ†’ 1/3 ๐Ÿ“ zrb show-stat โ€ข Run script: python -c "import pandas as pd; df=pd.read_csv('dataset.csv'); print(df.describe())" +๐Ÿค– โ—‹ โ 36725 โ†’ 1/3 ๐Ÿ“ zrb show-stat โ€ข Working directory: /home/gofrendi/playground/myproject +๐Ÿค– โ—‹ โ 36795 โ†’ 1/3 ๐Ÿ“ zrb show-stat โ€ข sepal_length sepal_width petal_length petal_width +๐Ÿค– โ—‹ โ 36795 โ†’ 1/3 ๐Ÿ“ zrb show-stat โ€ข count 150.000000 150.000000 150.000000 150.000000 +๐Ÿค– โ—‹ โ 36795 โ†’ 1/3 ๐Ÿ“ zrb show-stat โ€ข mean 5.843333 3.054000 3.758667 1.198667 +๐Ÿค– โ—‹ โ 36795 โ†’ 1/3 ๐Ÿ“ zrb show-stat โ€ข std 0.828066 0.433594 1.764420 0.763161 +๐Ÿค– โ—‹ โ 36795 โ†’ 1/3 ๐Ÿ“ zrb show-stat โ€ข min 4.300000 2.000000 1.000000 0.100000 +๐Ÿค– โ—‹ โ 36795 โ†’ 1/3 ๐Ÿ“ zrb show-stat โ€ข 25% 5.100000 2.800000 1.600000 0.300000 +๐Ÿค– โ—‹ โ 36795 โ†’ 1/3 ๐Ÿ“ zrb show-stat โ€ข 50% 5.800000 3.000000 4.350000 1.300000 +๐Ÿค– โ—‹ โ 36795 โ†’ 1/3 ๐Ÿ“ zrb show-stat โ€ข 75% 6.400000 3.300000 5.100000 1.800000 +๐Ÿค– โ—‹ โ 36795 โ†’ 1/3 ๐Ÿ“ zrb show-stat โ€ข max 7.900000 4.400000 6.900000 2.500000 Support zrb growth and development! โ˜• Donate at: https://stalchmst.com/donation ๐Ÿ™ Submit issues/PR at: https://github.com/state-alchemists/zrb ๐Ÿค Follow us at: https://twitter.com/zarubastalchmst -๐Ÿค– โ—‹ โ 27746 โ†’ 1/3 ๐Ÿฏ zrb show-stat โ€ข Completed in 1.3806817531585693 seconds +๐Ÿค– โ—‹ โ 36795 โ†’ 1/3 ๐Ÿ“ zrb show-stat โ€ข Completed in 2.24128794670105 seconds sepal_length sepal_width petal_length petal_width count 150.000000 150.000000 150.000000 150.000000 mean 5.843333 3.054000 3.758667 1.198667 @@ -125,13 +140,37 @@ max 7.900000 4.400000 6.900000 2.500000 To run again: zrb show-stat --url "https://raw.githubusercontent.com/state-alchemists/datasets/main/iris.csv" ``` +Now, you can get the statistics properties of any public CSV dataset. + +In this example, we purposely only use `CmdTask` to execute CLI scripts. However, Zrb has many other task types. For example, you can rewrite `show_stat` task using `@python_task` decorator. + +```python +# ๐Ÿ“Š Define a task to show the statistics properties of the dataset +@python_task( + name='show-stat', + upstreams=[ + # Let the following tasks to be show_stat's upstream: + download_dataset, + install_pandas + ] +) +def show_stat(*args, **kwargs): + import pandas as pd + df = pd.read_csv('dataset.csv') + return df.describe() +``` + +This will make more sense since `show_stat` is better written in Python. -To learn more about this, you can visit [our getting started guide](https://github.com/state-alchemists/zrb/blob/main/docs/getting-started.md). +You can also run a Docker compose file, start a Web server, generate a CRUD application, or even set up multiple servers at once. +See [our getting started guide](https://github.com/state-alchemists/zrb/blob/main/docs/getting-started.md) to learn more about the detail. ## Zrb is A Low-Code Framework -Let's see how you can build and run a [CRUD](https://en.wikipedia.org/wiki/Create,_read,_update_and_delete) application. +Aside from defining your own tasks, Zrb also comes with some builtin tasks. Those builtin tasks allows you to build and run a [CRUD](https://en.wikipedia.org/wiki/Create,_read,_update_and_delete) application. + +Let's see the following example. ```bash # Create a project @@ -156,12 +195,11 @@ zrb project add fastapp-field --project-dir . --app-name "fastapp" --module-name zrb project start-fastapp --fastapp-run-mode "monolith" ``` -You will be able to access the application by pointing your browser to [http://localhost:3000](http://localhost:3000) +Once you invoke the commands, you will be able to access the CRUD application by pointing your browser to [http://localhost:3000](http://localhost:3000) ![](https://raw.githubusercontent.com/state-alchemists/zrb/main/_images/fastapp.png) -Furthermore, you can run the same application as `microservices`, run the application as `docker containers`, and even do some deployments into your `kubernetes cluster`. - +Furthermore, you also split your application into `microservices`, run them as `docker containers`, and even deploy them to your `kubernetes cluster`. ```bash # Run Fastapp as microservices @@ -181,7 +219,15 @@ You can visit [our tutorials](https://github.com/state-alchemists/zrb/blob/main/ # ๐Ÿซฐ Installation -## โš™๏ธ In Local Machine +## ๐Ÿš€ Using Installation Script + +We provide an installation script to help you install `pyenv` and `Zrb`. You can run the installation script as follow: + +```bash +curl https://raw.githubusercontent.com/state-alchemists/zrb/main/install.sh | bash +``` + +## โš™๏ธ As Python Package Installing Zrb in your system is as easy as typing the following command in your terminal: @@ -193,7 +239,7 @@ Like any other Python package, you can install Zrb in your [virtual environment] > โš ๏ธ If the command doesn't work, you probably don't have Pip/Python on your computer. See `Main prerequisites` subsection to install them. -## ๐Ÿ‹ With Docker +## ๐Ÿ‹ As Docker Container If you prefer to work with Docker, you can create a file named `docker-compose.yml` diff --git a/install.sh b/install.sh new file mode 100755 index 00000000..6f809786 --- /dev/null +++ b/install.sh @@ -0,0 +1,44 @@ +set -e + +# Make sure pyenv is installed +if [ ! -d "$HOME/.pyenv" ] +then + echo "Installing Pyenv" + curl https://pyenv.run | bash + + # Register .pyenv to .zshrc + if [ -f "$HOME/.zshrc" ] + then + echo "Registering Pyenv to .zshrc" + echo '' >> $HOME/.zshrc + echo 'export PYENV_ROOT="$HOME/.pyenv"' >> $HOME/.zshrc + echo 'command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"' >> $HOME/.zshrc + echo 'eval "$(pyenv init -)"' >> $HOME/.zshrc + fi + + # Register .pyenv to .bashrc + if [ -f "$HOME/.bashrc" ] + then + echo "Registering Pyenv to .bashrc" + echo '' >> $HOME/.bashrc + echo 'export PYENV_ROOT="$HOME/.pyenv"' >> $HOME/.bashrc + echo 'command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"' >> $HOME/.bashrc + echo 'eval "$(pyenv init -)"' >> $HOME/.bashrc + fi + + # Activate pyenv + echo "Activating pyenv" + export PYENV_ROOT="$HOME/.pyenv" + command -v pyenv >/dev/null || (export PATH="$PYENV_ROOT/bin:$PATH" && eval "$(pyenv init -)") + + # Install python 3.10.0 + echo "Installing python 3.10.0" + pyenv install 3.10.0 + + # Set global python to 3.10.0 + echo "Set python 3.10.0 as global" + pyenv global 3.10.0 +fi + +echo "Installing Zrb" +pip install zrb \ No newline at end of file From 7741b89042ffefe75972172a044ab38d2e6d8591 Mon Sep 17 00:00:00 2001 From: goFrendiAsgard Date: Sun, 12 Nov 2023 07:31:37 +0700 Subject: [PATCH 7/9] update getting started --- .vscode/settings.json | 8 +++- README.md | 103 +++++++++++++++++++++++----------------- docs/getting-started.md | 60 +++++++++++++++++++---- error.txt | 7 +++ output.txt | 1 + 5 files changed, 124 insertions(+), 55 deletions(-) create mode 100644 error.txt create mode 100644 output.txt diff --git a/.vscode/settings.json b/.vscode/settings.json index 33fe63f7..09cb8000 100644 --- a/.vscode/settings.json +++ b/.vscode/settings.json @@ -1,4 +1,10 @@ { "python.linting.flake8Enabled": true, - "python.linting.enabled": true + "python.linting.enabled": true, + "grammarly.selectors": [ + { + "language": "markdown", + "scheme": "file" + } + ] } \ No newline at end of file diff --git a/README.md b/README.md index 575806ef..a7b7f6b5 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ ![](https://raw.githubusercontent.com/state-alchemists/zrb/main/_images/zrb/android-chrome-192x192.png) -[๐Ÿ“– Documentation](https://github.com/state-alchemists/zrb/blob/main/docs/README.md) | [๐Ÿ Getting Started](https://github.com/state-alchemists/zrb/blob/main/docs/getting-started.md) | [๐Ÿ’ƒ Oops, I did it Again](https://github.com/state-alchemists/zrb/blob/main/docs/oops-i-did-it-again/README.md) | [โ“ FAQ](https://github.com/state-alchemists/zrb/blob/main/docs/faq/README.md) +[๐Ÿซฐ Installation](#๐Ÿซฐ-installation) | [๐Ÿ“– Documentation](https://github.com/state-alchemists/zrb/blob/main/docs/README.md) | [๐Ÿ Getting Started](https://github.com/state-alchemists/zrb/blob/main/docs/getting-started.md) | [๐Ÿ’ƒ Oops, I did it Again](https://github.com/state-alchemists/zrb/blob/main/docs/oops-i-did-it-again/README.md) | [โ“ FAQ](https://github.com/state-alchemists/zrb/blob/main/docs/faq/README.md) Zrb is a [CLI-based](https://en.wikipedia.org/wiki/Command-line_interface) automation [tool](https://en.wikipedia.org/wiki/Programming_tool) and [low-code](https://en.wikipedia.org/wiki/Low-code_development_platform) platform. Once installed, Zrb will help you automate day-to-day tasks, generate projects and applications, and even deploy your applications to Kubernetes with a few commands. @@ -10,7 +10,7 @@ To use Zrb, you need to be familiar with CLI. Zrb task definitions are written in [Python](https://www.python.org/), and we have a [good reason](https://github.com/state-alchemists/zrb/blob/main/docs/faq/why-python.md) behind the decision. -## Zrb is A Task-Automation Tool +## Zrb as A Task-Automation Tool At the very core, Zrb is a task automation tool. It helps you to automate some tedious jobs so that you can focus on what matters. @@ -77,18 +77,32 @@ show_stat = CmdTask( runner.register(show_stat) ``` +> __๐Ÿ“ NOTE:__ In this example, we exclusively use `CmdTask` to execute CLI scripts. However, Zrb has many other task types, like `python_task`. You can rewrite `show_stat` task using `@python_task` decorator as follow: +> +> ```python +> # ๐Ÿ“Š Define a task to show the statistics properties of the dataset +> @python_task( +> name='show-stat', +> upstreams=[ +> # Let the following tasks to be show_stat's upstream: +> download_dataset, +> install_pandas +> ] +> ) +> def show_stat(*args, **kwargs): +> import pandas as pd +> df = pd.read_csv('dataset.csv') +> return df.describe() +> ``` +> +> This will make more sense since `show_stat` is better written in Python. + Once you do so, you can invoke the task and get the output. ```bash zrb show-stat ``` -> __๐Ÿ“ NOTE:__ When executing a Zrb task, you can also provide the parameter directly, for example: -> -> ```bash -> zrb show-stat --url https://raw.githubusercontent.com/state-alchemists/datasets/main/iris.csv -> ``` - ``` Url [https://raw.githubusercontent.com/state-alchemists/datasets/main/iris.csv]: ๐Ÿค– โ—‹ โ 36725 โ†’ 1/3 ๐Ÿจ download-dataset โ€ข Run script: wget -O dataset.csv https://raw.githubusercontent.com/state-alchemists/datasets/main/iris.csv @@ -139,36 +153,24 @@ min 4.300000 2.000000 1.000000 0.100000 max 7.900000 4.400000 6.900000 2.500000 To run again: zrb show-stat --url "https://raw.githubusercontent.com/state-alchemists/datasets/main/iris.csv" ``` - Now, you can get the statistics properties of any public CSV dataset. -In this example, we purposely only use `CmdTask` to execute CLI scripts. However, Zrb has many other task types. For example, you can rewrite `show_stat` task using `@python_task` decorator. - -```python -# ๐Ÿ“Š Define a task to show the statistics properties of the dataset -@python_task( - name='show-stat', - upstreams=[ - # Let the following tasks to be show_stat's upstream: - download_dataset, - install_pandas - ] -) -def show_stat(*args, **kwargs): - import pandas as pd - df = pd.read_csv('dataset.csv') - return df.describe() -``` - -This will make more sense since `show_stat` is better written in Python. +> __๐Ÿ“ NOTE:__ When executing a task, you can also provide the parameter directly, for example you want to provide the `url` +> +> ```bash +> zrb show-stat --url https://raw.githubusercontent.com/state-alchemists/datasets/main/iris.csv +> ``` +> +> When you provide the parameter directly, Zrb will not prompt you to supply the URL. +> Another way to disable the prompt is by set `ZRB_SHOW_PROMPT` to `0` or `false`. -You can also run a Docker compose file, start a Web server, generate a CRUD application, or even set up multiple servers at once. +You can also run a Docker compose file, start a Web server, generate a CRUD application, or set up multiple servers simultaneously. -See [our getting started guide](https://github.com/state-alchemists/zrb/blob/main/docs/getting-started.md) to learn more about the detail. +See [our getting started guide](https://github.com/state-alchemists/zrb/blob/main/docs/getting-started.md) to learn more about the details. -## Zrb is A Low-Code Framework +## Zrb as A Low-Code Framework -Aside from defining your own tasks, Zrb also comes with some builtin tasks. Those builtin tasks allows you to build and run a [CRUD](https://en.wikipedia.org/wiki/Create,_read,_update_and_delete) application. +Aside from defining your own tasks, Zrb also has some built-in tasks. Those built-in tasks allow you to build and run a [CRUD](https://en.wikipedia.org/wiki/Create,_read,_update_and_delete) application. Let's see the following example. @@ -221,7 +223,7 @@ You can visit [our tutorials](https://github.com/state-alchemists/zrb/blob/main/ ## ๐Ÿš€ Using Installation Script -We provide an installation script to help you install `pyenv` and `Zrb`. You can run the installation script as follow: +We provide an [installation script](https://github.com/state-alchemists/zrb/blob/main/install.sh) to help you install `pyenv` and `Zrb`. You can run the installation script as follows: ```bash curl https://raw.githubusercontent.com/state-alchemists/zrb/main/install.sh | bash @@ -235,47 +237,60 @@ Installing Zrb in your system is as easy as typing the following command in your pip install zrb ``` -Like any other Python package, you can install Zrb in your [virtual environment](https://docs.python.org/3/library/venv.html). This will allow you to have many versions of Zrb on the same computer. +Like any Python package, you can install Zrb in your [virtual environment](https://docs.python.org/3/library/venv.html). Installing Zrb in a virtual environment allows you to have many versions of Zrb on the same computer. > โš ๏ธ If the command doesn't work, you probably don't have Pip/Python on your computer. See `Main prerequisites` subsection to install them. ## ๐Ÿ‹ As Docker Container -If you prefer to work with Docker, you can create a file named `docker-compose.yml` +If you prefer to work with Docker, you can create a `docker-compose.yml` file as follows: ```yaml version: '3' + +x-logging: &default-logging + options: + max-size: "100m" + max-file: "5" + driver: json-file + networks: zrb: name: zrb + external: true + services: zrb: build: dockerfile: Dockerfile context: . - image: docker.io/stalchmst/zrb:latest + args: + ZRB_VERSION: ${ZRB_VERSION:-latest} + image: ${ZRB_IMAGE:-docker.io/stalchmst/zrb} + logging: *default-logging container_name: zrb hostname: zrb + command: sleep infinity volumes: - - /var/run/docker.sock:/var/run/docker.sock - - ./project:/project - networks: - - zrb + - /var/run/docker.sock:/var/run/docker.sock + - ./project:/project ports: - - 3001:3001 # or/and any other ports you want to expose. + - 3001:3001 # or/and any other ports you want to expose. + networks: + - zrb ``` -Once your docker-compose file is created, you can invoke the following command: +Once you create a docker-compose file, you can invoke the following command to start the container: ```bash docker compose up -d ``` -You will be able to access Zrb by using docker exec: +You will be able to run any Zrb tasks by accessing the container's bash: ```bash -docker exec -it zrb zsh +docker exec -it zrb bash ``` # โœ… Main Prerequisites diff --git a/docs/getting-started.md b/docs/getting-started.md index 858e6d15..2c44efde 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -4,25 +4,32 @@ Welcome to Zrb's getting started guide. -We will cover all the basic you need to know before working with Zrb. You will learn about: +We will cover everything you need to know before working with Zrb. You will learn about: - [How to run a task](#running-a-task) - [How to create a project](#creating-a-project) -- How to create a simple task +- How to define a simple task - How to define upstreams -- How to create a long running task +- How to define a long-running task -This guide assume you have some familiarity with CLI and Python. +This guide assumes you have some familiarity with CLI and Python. # Running a task -Once you installed Zrb, you can run some built-in tasks immediately. To run any Zrb task, you need to follow the following pattern: +Once you have installed Zrb, you can run some built-in tasks immediately. To run any Zrb task, you need to follow the following pattern: ```bash zrb [task-groups...] [task-parameters...] ``` -For example, you want to run `encode` that is located under `base64` group, you can do so by execute the following command: +For example, you want to run the `base64 encode` task with the following information: + +- __Task group:__ base64 +- __Task name:__ encode +- __Task parameters:__ + - `text` = `non-credential-string` + +Based on the pattern, you will need to invoke the following command: ```bash zrb base64 encode --text "non-credential-string" @@ -38,13 +45,46 @@ bm9uLWNyZWRlbnRpYWwtc3RyaW5n To run again: zrb base64 encode --text "non-credential-string" ``` -> __โš ๏ธ WARNING:__ Anyone can easily decode a base64-encoded string. Don't use it to encrypt your password or any important credentials. +You can see that Zrb encoded `non-credential-string` into `bm9uLWNyZWRlbnRpYWwtc3RyaW5n`. + +> __โš ๏ธ WARNING:__ Base64 is a encoding algorithm that allows you to transform any characters into an alphabet which consists of Latin letters, digits, plus, and slash. +> +> Anyone can easily decode a base64-encoded string. __Never use it to encrypt your password or any important credentials!__ + +## Redirect Task's Output + +You can use any task's output for further processing. For example, redirect a task's output and error into files. + +```bash +zrb base64 encode --text "non-credential-string" > output.txt 2> stderr.txt +cat output.txt +cat stderr.txt +``` + +You can also use a task's output as other CLI command's parameter + +```bash +echo $(zrb base64 encode --text "non-credential-string" 2> error.txt) +``` + +Finally, you can also use the pipe operator to redirect a task's output as other CLI command's input + +```bash +zrb base64 encode --text "non-credential-string" 2> error.txt | lolcat +``` + +> __๐Ÿ“ NOTE:__ You can install lolcat by following [it's documentation](https://github.com/busyloop/lolcat). If you are using Linux, and you don't like `snap`, you can try to use your OS's package manager (e.g., `sudo apt install lolcat`) + +## How Task Groups Organize Tasks + +We usually put related `tasks` under the same `task-group`. -Related tasks are usually located under the same `task-group`. +For example, we have two tasks under `base64` group: -For example, you have all base64 related tasks (e.g., `decode` and `encode`) under `base64` task-group. +- encode +- decode -Let's try to decode our base64-encoded text: +Let's decode our base64-encoded text: ```bash zrb base64 decode --text "bm9uLWNyZWRlbnRpYWwtc3RyaW5n" diff --git a/error.txt b/error.txt new file mode 100644 index 00000000..2bee7a7c --- /dev/null +++ b/error.txt @@ -0,0 +1,7 @@ +Support zrb growth and development! +โ˜• Donate at: https://stalchmst.com/donation +๐Ÿ™ Submit issues/PR at: https://github.com/state-alchemists/zrb +๐Ÿค Follow us at: https://twitter.com/zarubastalchmst +๐Ÿค– โ—‹ โ—ท 2023-11-12 07:25:41.345 โ 17023 โ†’ 1/1 ๐Ÿป zrb base64 encode โ€ข Completed in 0.051529645919799805 seconds +To run again: zrb base64 encode --text "non-credential-string" + \ No newline at end of file diff --git a/output.txt b/output.txt new file mode 100644 index 00000000..48e05b24 --- /dev/null +++ b/output.txt @@ -0,0 +1 @@ +bm9uLWNyZWRlbnRpYWwtc3RyaW5n From 0552eba55de3e80071c414188930c6577f3d07d3 Mon Sep 17 00:00:00 2001 From: goFrendiAsgard Date: Sun, 12 Nov 2023 07:31:59 +0700 Subject: [PATCH 8/9] remove files --- error.txt | 7 ------- output.txt | 1 - 2 files changed, 8 deletions(-) delete mode 100644 error.txt delete mode 100644 output.txt diff --git a/error.txt b/error.txt deleted file mode 100644 index 2bee7a7c..00000000 --- a/error.txt +++ /dev/null @@ -1,7 +0,0 @@ -Support zrb growth and development! -โ˜• Donate at: https://stalchmst.com/donation -๐Ÿ™ Submit issues/PR at: https://github.com/state-alchemists/zrb -๐Ÿค Follow us at: https://twitter.com/zarubastalchmst -๐Ÿค– โ—‹ โ—ท 2023-11-12 07:25:41.345 โ 17023 โ†’ 1/1 ๐Ÿป zrb base64 encode โ€ข Completed in 0.051529645919799805 seconds -To run again: zrb base64 encode --text "non-credential-string" - \ No newline at end of file diff --git a/output.txt b/output.txt deleted file mode 100644 index 48e05b24..00000000 --- a/output.txt +++ /dev/null @@ -1 +0,0 @@ -bm9uLWNyZWRlbnRpYWwtc3RyaW5n From a61d3040729d6aeb002646feaabe5b0dcd8ef844 Mon Sep 17 00:00:00 2001 From: goFrendiAsgard Date: Sun, 12 Nov 2023 10:36:28 +0700 Subject: [PATCH 9/9] update README --- docs/getting-started.md | 488 +++++++++++++++++++++++++------------- src/zrb/builtin/base64.py | 4 +- 2 files changed, 330 insertions(+), 162 deletions(-) diff --git a/docs/getting-started.md b/docs/getting-started.md index 2c44efde..abb3554e 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -1,20 +1,27 @@ ๐Ÿ”– [Table of Contents](README.md) -# Getting started +# Getting Started -Welcome to Zrb's getting started guide. - -We will cover everything you need to know before working with Zrb. You will learn about: +Welcome to Zrb's getting started guide. We will cover everything you need to know to work with Zrb. In this article, you will learn about: - [How to run a task](#running-a-task) + - [Redirect task's output](#redirect-tasks-output) + - [How tasks are organized](#how-tasks-are-organized) + - [Getting available tasks/task groups](#getting-available-taskstask-groups) + - [Using input prompts](#using-input-prompt) - [How to create a project](#creating-a-project) -- How to define a simple task -- How to define upstreams -- How to define a long-running task + - [Using/creating virtual environment](#activating-virtual-environment) +- [How to define a task](#creating-a-task) + - [Scaffolding a task](#scaffolding-a-task) + - [Updating task definition](#updating-task-definition) + - [Common task parameters](#common-task-parameters) + - [Cmd task parameters](#cmdtask-parameters) + - [Python task parameters](#python_task-parameters) +- [How to define a long-running task]() This guide assumes you have some familiarity with CLI and Python. -# Running a task +# Running a Task Once you have installed Zrb, you can run some built-in tasks immediately. To run any Zrb task, you need to follow the following pattern: @@ -75,7 +82,7 @@ zrb base64 encode --text "non-credential-string" 2> error.txt | lolcat > __๐Ÿ“ NOTE:__ You can install lolcat by following [it's documentation](https://github.com/busyloop/lolcat). If you are using Linux, and you don't like `snap`, you can try to use your OS's package manager (e.g., `sudo apt install lolcat`) -## How Task Groups Organize Tasks +## How Tasks are Organized We usually put related `tasks` under the same `task-group`. @@ -94,9 +101,9 @@ You should get your original text back. > __๐Ÿ’ก HINT:__ You don't have to memorize any `task-group` or `task` name. The next two subsections will show you how to locate and execute any `task` without memorize anything. -## Getting available tasks/task groups +## Getting Available Tasks/Task Groups -To find out what tasks/task-groups are available, you can type `zrb` and press enter. +To see what `tasks`/`task-groups` are available, type `zrb` and press enter. ```bash zrb @@ -165,15 +172,15 @@ Options: --help Show this message and exit. Commands: - decode Decode base64 task - encode Encode base64 task + decode Decode a base64 encoded text + encode Encode a text using base64 algorithm ``` -> __๐Ÿ“ NOTE:__ A `task-group` might contains some `tasks` or other `task-groups` +> __๐Ÿ“ NOTE:__ A `task-group` might contains other `task-groups` ## Using input prompt -Once you find the task you want to execute, you can just type `zrb [task-groups...] ` without bothering about `task-parameters`. +Once you find the task you want to execute, you can type `zrb [task-groups...] ` without bothering about `task-parameters`. Zrb will automatically prompt you to provide the parameter interactively. @@ -192,19 +199,21 @@ bm9uLWNyZWRlbnRpYWwtc3RyaW5n To run again: zrb base64 encode --text "non-credential-string" ``` -> __๐Ÿ“ NOTE:__ To disable prompt, you can set `ZRB_SHOW_PROMPT` to `0` or `false`. When prompts are disabled, Zrb will automatically use default values. Please refer to [configuration section](./configurations.md) for more information. +> __๐Ÿ“ NOTE:__ If you need to disable prompt entirely, you can set `ZRB_SHOW_PROMPT` to `0` or `false`. Please refer to [configuration section](./configurations.md) for more information. +> +> When prompts are disabled, Zrb will automatically use default values. # Creating a project -To make things more manageable, you need to put all your resources and task definitions in a `project`. +To make things more manageable, you must put all related resources and task definitions under a `project`. A project is a directory containing `zrb_init.py`. -Suppose you want to create a project under `my-project`, then you can invoke the following command: +You can create a project manually or use Zrb's built-in task to generate the project. Suppose you want to create a project named `my-project`. ```bash -zrb project create --project-dir my-project +zrb project create --project-dir my-project --project-name my-project ``` -Once invoked, you will have a directory named `my-project`. You can move to the directory and start to see how a project looks like: +Once invoked, you will have a directory named `my-project`. Let's see how the project looks like: ```bash cd my-project @@ -212,35 +221,30 @@ ls -al ``` ``` -total 44 -drwxr-xr-x 5 gofrendi gofrendi 4096 Jun 11 05:29 . -drwxr-xr-x 4 gofrendi gofrendi 4096 Jun 11 05:29 .. -drwxr-xr-x 7 gofrendi gofrendi 4096 Jun 11 05:29 .git --rw-r--r-- 1 gofrendi gofrendi 21 Jun 11 05:29 .gitignore --rw-r--r-- 1 gofrendi gofrendi 1776 Jun 11 05:29 README.md -drwxr-xr-x 3 gofrendi gofrendi 4096 Jun 11 05:29 _automate --rwxr-xr-x 1 gofrendi gofrendi 1517 Jun 11 05:29 project.sh --rw-r--r-- 1 gofrendi gofrendi 12 Jun 11 05:29 requirements.txt -drwxr-xr-x 2 gofrendi gofrendi 4096 Jun 11 05:29 src --rw-r--r-- 1 gofrendi gofrendi 34 Jun 11 05:29 template.env --rw-r--r-- 1 gofrendi gofrendi 54 Jun 11 05:29 zrb_init.py +total 52 +drwxr-xr-x 6 gofrendi gofrendi 4096 Nov 12 07:52 . +drwxr-xr-x 14 gofrendi gofrendi 4096 Nov 12 07:52 .. +drwxr-xr-x 7 gofrendi gofrendi 4096 Nov 12 07:52 .git +drwxr-xr-x 3 gofrendi gofrendi 4096 Nov 12 07:52 .github +-rw-r--r-- 1 gofrendi gofrendi 27 Nov 12 07:52 .gitignore +-rw-r--r-- 1 gofrendi gofrendi 7 Nov 12 07:52 .python-version +-rw-r--r-- 1 gofrendi gofrendi 1937 Nov 12 07:52 README.md +drwxr-xr-x 3 gofrendi gofrendi 4096 Nov 12 07:52 _automate +-rwxr-xr-x 1 gofrendi gofrendi 1507 Nov 12 07:52 project.sh +-rw-r--r-- 1 gofrendi gofrendi 13 Nov 12 07:52 requirements.txt +drwxr-xr-x 2 gofrendi gofrendi 4096 Nov 12 07:52 src +-rw-r--r-- 1 gofrendi gofrendi 118 Nov 12 07:52 template.env +-rw-r--r-- 1 gofrendi gofrendi 54 Nov 12 07:52 zrb_init.py ``` -Every Zrb project has a file named `zrb_init.py` on it's top level directory. This file is your entry point to define your task definitions. - -It is recommended that you define your tasks under `_automate` directory and import them into your `zrb_init.py`. This will help you manage the [separation of concerns](https://en.wikipedia.org/wiki/Separation_of_concerns). +Every Zrb project has a file named `zrb_init.py` under the top-level directory. This file is your entry point to define your task definitions. -Aside from `zrb_init.py`, you will also find some other files/directory: +By convention, a project usually contains two sub-directories: -- `.git` and `.gitignore`, indicating that your project is also a git repository. -- `README.md`, your README file. -- `project.sh`, a shell script to initiate your project. -- `requirements.txt`, list of necessary python packages to run start a project. Make sure to update this if you declare a task that depends on other Python library. -- `template.env`, your default environment variables. -- `_automate`, a directory contains task definitions that should be imported in `zrb_init.py`. -- `src`, your project resources (e.g., source code, docker compose file, helm charts, etc) +- `_automate`: This folder contains all your automation scripts and task definitions +- `src`: This folder contains all your resources like Docker compose file, helm charts, and source code. -By default, Zrb will create a default `task-group` named `project`. Try to type: +When you make a project using `zrb project create` command, Zrb will generate a default `task-group` named `project`. This `task-group` contains some tasks to run/deploy everything. Try to type `zrb project` to see what tasks are available by default: ```bash zrb project @@ -268,70 +272,254 @@ Commands: stop-containers Stop project containers ``` -# Creating a simple task +## Activating Virtual Environment -Once your project has been created, you can add some new tasks to your project. - -Let's say you work for a company named `Arasaka`, and you want to show a cool CLI banner for your company. +If you generate the project by invoking `zrb project create`, then you have to run the following command now: ```bash -zrb project add cmd-task --project-dir . --task-name show-banner +source project.sh ``` -Zrb will automatically do a few things for you: - -- Create `_automate/show_banner.py` -- Import `_automate.show_banner` into `zrb_init.py`. +The command will ensure that you work under the project's virtual environment. -Now you can try to run the task: +If you create the project manually, you need to make a virtual environment for your project: ```bash -zrb project show-banner +python -m venv .venv +source .venv/bin/activate ``` +> __โš ๏ธ WARNING:__ You have to make sure you are working under virtual environment everytime you work with Zrb project, either by invoking `source project.sh` or `source .venv/bin/activate`. + + +# Creating a Task + +Zrb has a powerful command to create tasks under a project. Let's re-create the tasks we make in our [README.md](../README.md). + +The goal of the tasks is to download any public CSV dataset and provide the statistical properties of the dataset. To do that, you need to: +- Ensure that you have Pandas installed on your machine +- Ensure that you have downloaded the dataset +- Run the Python script to get the statistical properties of the dataset + ``` -๐Ÿค– โžœ 2023-06-11T05:52:27.267892 โš™ 4388 โžค 1 of 3 โ€ข ๐Ÿ“ zrb project show-banner โ€ข Run script: echo show banner -๐Ÿค– โžœ 2023-06-11T05:52:27.268193 โš™ 4388 โžค 1 of 3 โ€ข ๐Ÿ“ zrb project show-banner โ€ข Current working directory: /home/gofrendi/zrb/playground/my-project -๐Ÿค– โžœ 2023-06-11T05:52:27.272726 โš™ 4389 โžค 1 of 3 โ€ข ๐Ÿ“ zrb project show-banner โ€ข show banner -Support zrb growth and development! -โ˜• Donate at: https://stalchmst.com/donation -๐Ÿ™ Submit issues/pull requests at: https://github.com/state-alchemists/zrb -๐Ÿค Follow us at: https://twitter.com/zarubastalchmst -๐Ÿค– โžœ 2023-06-11T05:52:27.318296 โš™ 4389 โžค 1 of 3 โ€ข ๐Ÿ“ zrb project show-banner โ€ข zrb project show-banner completed in 0.11460638046264648 seconds -To run again: zrb project show-banner -show banner + ๐Ÿผ + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ”‚ โ”‚ + โ”‚ Install Pandas โ”œโ”€โ” ๐Ÿ“Š + โ”‚ โ”‚ โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ–บโ”‚ โ”‚ + โ”‚ Show Statistics โ”‚ + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ–บโ”‚ โ”‚ + โ”‚ โ”‚ โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ Download Dataset โ”œโ”€โ”˜ + โ”‚ โ”‚ + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โฌ‡๏ธ ``` -Now let's make the banner cooler with `figlet`. You can do so by editing `_automate/show_banner.py`. If you are using VSCode, you can type `code .` in your terminal. +## Scaffolding a Task + +Zrb has a powerful command to scaffold your project. To do so, you need to invoke the following command: + +> __โš ๏ธ WARNING:__ Make sure you have activate your virtual environment, either by invoking `source project.sh` or `source .venv/bin/activate`. -> โš ๏ธ We will use `figlet`. Try to type `figlet hello` and see whether things work or not. If you are using Ubuntu, you might need to install figlet by invoking `sudo apt install figlet`. +```bash +zrb project add python-task --project-dir "." --task-name "show-stats" +``` -Make sure to modify the `cmd` property of your `show_banner` task, so that it looks like the following: +Once you invoke the command, Zrb will make a file named `_automate/show_stats.py` ```python -from zrb import CmdTask, runner -from zrb.builtin._group import project_group +from typing import Any, Mapping +from zrb import Task, python_task, runner +from zrb.builtin.group import project_group ############################################################################### # Task Definitions ############################################################################### -show_banner = CmdTask( - name='show-banner', - description='show banner', + +@python_task( + name='show-stats', + description='show stats', group=project_group, - cmd=[ - 'figlet Arasaka' - ] + runner=runner ) -runner.register(show_banner) +async def show_stats(*args: Any, **kwargs: Any) -> Any: + task: Task = kwargs.get('_task') + env_map: Mapping[str, str] = task.get_env_map() + input_map: Mapping[str, str] = task.get_input_map() + task.print_out(f'Env map: {env_map}') + task.print_out(f'Input map: {input_map}') + return 'ok' +``` +We will modify the task later to match our use case. + +You will also notice that Zrb automatically imports `_automate/show_stats.py` into `zrb_init.py`. + +```python +import _automate._project as _project +import _automate.show_stats as show_stats +assert _project +assert show_stats ``` -Cool. You make it. [Saburo Arasaka](https://cyberpunk.fandom.com/wiki/Saburo_Arasaka) will be proud of you ๐Ÿ˜‰. +This modification allows you to invoke `show-stats` from the CLI + +``` +zrb project show-stats +``` + +## Updating Task Definition + +To serve our use case, you need to add two more tasks: + +- install-pandas +- download-dataset + +You also need to ensure both of them are registered as `show-stats` upstreams. You also need to modify `show-stats` a little bit. + +```python +from typing import Any +from zrb import CmdTask, python_task, StrInput, runner +from zrb.builtin.group import project_group + +# ๐Ÿผ Define a task to install pandas +install_pandas = CmdTask( + name='install-pandas', + group=project_group, + cmd='pip install pandas' +) + +# Make install_pandas accessible from the CLI (i.e., zrb project install-pandas) +runner.register(install_pandas) + +# โฌ‡๏ธ Define a task to download dataset +download_dataset = CmdTask( + name='download-dataset', + group=project_group, + inputs=[ + # Allow user to put the CSV dataset URL. + StrInput( + name='url', + default='https://raw.githubusercontent.com/state-alchemists/datasets/main/iris.csv' + ) + ], + cmd='wget -O dataset.csv {{input.url}}' +) + +# Make download_dataset accessible from the CLI (i.e., zrb project download-dataset) +runner.register(download_dataset) + + +# ๐Ÿ“Š Define a task to show the statistics properties of the dataset +@python_task( + name='show-stats', + description='show stats', + group=project_group, + upstreams=[ + # Make sure install_pandas and download_dataset are successfully executed before running show_stats + install_pandas, + download_dataset + ], + runner=runner # Make show_stats accessible from the CLI (i.e., zrb project show-stats) +) +async def show_stats(*args: Any, **kwargs: Any) -> Any: + import pandas as pd + df = pd.read_csv('dataset.csv') + return df.describe() +``` + +We define `install_pandas` and `download_dataset` using `CmdTask` since writing them using shell script is easier. We also make `show_stats` depend on `install_pandas` and `download_dataset` by defining `show_stats`'s upstream. + +### Common Task Parameters + +`CmdTask` and `@python_task` decorator has some parameters in common: + +- __name__: The name of the task. When you invoke the task using the CLI, you need to use this name. +- __description__: The description of the task. +- __group__: The task group where the task belongs to +- __inputs__: The inputs and their default values. + - If you are using a `CmdTask`, you can access the input using a Jinja Template (e.g., `{{input.input_name}}`) + - If you are using a `@python_task` decorator, you can access the input by using `kwargs` argument (e.g., `kwargs.get('input_name')`) +- __upstreams__: Upstreams of the task. You can provide `AnyTask` as upstream. + +### CmdTask Parameters + +Aside from common task properties, `CmdTask` has other properties: + +- __cmd__: String, or List of String, containing the shell script +- __cmd_path__: String, or List of String, containing the path to any shell scripts you want to use + +### PythonTask Parameters + +Aside from common task properties, `@python_task` has a runner parameter. This parameter is unique to `@python_task`. Any task created with `@python_task` with `runner` on it will be accessible from the CLI. + +## Running show-stats + +Finally, you can show the statistics property of any public CSV dataset quickly. + +``` +zrb project show-stats +``` + +``` +Url [https://raw.githubusercontent.com/state-alchemists/datasets/main/iris.csv]: +๐Ÿค– โ—‹ โ—ท 2023-11-12 09:45:12.132 โ 43598 โ†’ 1/3 ๐Ÿฎ zrb project install-pandas โ€ข Run script: pip install pandas +๐Ÿค– โ—‹ โ—ท 2023-11-12 09:45:12.132 โ 43598 โ†’ 1/3 ๐Ÿฎ zrb project install-pandas โ€ข Working directory: /home/gofrendi/playground/my-project +๐Ÿค– โ—‹ โ—ท 2023-11-12 09:45:12.139 โ 43598 โ†’ 1/3 ๐Ÿ“ zrb project download-dataset โ€ข Run script: wget -O dataset.csv https://raw.githubusercontent.com/state-alchemists/datasets/main/iris.csv +๐Ÿค– โ—‹ โ—ท 2023-11-12 09:45:12.139 โ 43598 โ†’ 1/3 ๐Ÿ“ zrb project download-dataset โ€ข Working directory: /home/gofrendi/playground/my-project +๐Ÿค– โ–ณ โ—ท 2023-11-12 09:45:12.151 โ 43603 โ†’ 1/3 ๐Ÿ“ zrb project download-dataset โ€ข --2023-11-12 09:45:12-- https://raw.githubusercontent.com/state-alchemists/datasets/main/iris.csv +๐Ÿค– โ–ณ โ—ท 2023-11-12 09:45:12.218 โ 43603 โ†’ 1/3 ๐Ÿ“ zrb project download-dataset โ€ข Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.109.133, 185.199.110.133, ... +๐Ÿค– โ–ณ โ—ท 2023-11-12 09:45:12.246 โ 43603 โ†’ 1/3 ๐Ÿ“ zrb project download-dataset โ€ข Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected. +๐Ÿค– โ–ณ โ—ท 2023-11-12 09:45:12.803 โ 43603 โ†’ 1/3 ๐Ÿ“ zrb project download-dataset โ€ข HTTP request sent, awaiting response... 200 OK +๐Ÿค– โ–ณ โ—ท 2023-11-12 09:45:12.806 โ 43603 โ†’ 1/3 ๐Ÿ“ zrb project download-dataset โ€ข Length: 4606 (4.5K) [text/plain] +๐Ÿค– โ–ณ โ—ท 2023-11-12 09:45:12.808 โ 43603 โ†’ 1/3 ๐Ÿ“ zrb project download-dataset โ€ข Saving to: โ€˜dataset.csvโ€™ +๐Ÿค– โ–ณ โ—ท 2023-11-12 09:45:12.810 โ 43603 โ†’ 1/3 ๐Ÿ“ zrb project download-dataset โ€ข +๐Ÿค– โ–ณ โ—ท 2023-11-12 09:45:12.812 โ 43603 โ†’ 1/3 ๐Ÿ“ zrb project download-dataset โ€ข 0K .... 100% 1.39M=0.003s +๐Ÿค– โ–ณ โ—ท 2023-11-12 09:45:12.814 โ 43603 โ†’ 1/3 ๐Ÿ“ zrb project download-dataset โ€ข +๐Ÿค– โ–ณ โ—ท 2023-11-12 09:45:12.816 โ 43603 โ†’ 1/3 ๐Ÿ“ zrb project download-dataset โ€ข 2023-11-12 09:45:12 (1.39 MB/s) - โ€˜dataset.csvโ€™ saved [4606/4606] +๐Ÿค– โ–ณ โ—ท 2023-11-12 09:45:12.817 โ 43603 โ†’ 1/3 ๐Ÿ“ zrb project download-dataset โ€ข +๐Ÿค– โ—‹ โ—ท 2023-11-12 09:45:12.978 โ 43601 โ†’ 1/3 ๐Ÿฎ zrb project install-pandas โ€ข Requirement already satisfied: pandas in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (2.1.3) +๐Ÿค– โ—‹ โ—ท 2023-11-12 09:45:13.042 โ 43601 โ†’ 1/3 ๐Ÿฎ zrb project install-pandas โ€ข Requirement already satisfied: numpy<2,>=1.22.4 in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (from pandas) (1.26.1) +๐Ÿค– โ—‹ โ—ท 2023-11-12 09:45:13.044 โ 43601 โ†’ 1/3 ๐Ÿฎ zrb project install-pandas โ€ข Requirement already satisfied: python-dateutil>=2.8.2 in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (from pandas) (2.8.2) +๐Ÿค– โ—‹ โ—ท 2023-11-12 09:45:13.045 โ 43601 โ†’ 1/3 ๐Ÿฎ zrb project install-pandas โ€ข Requirement already satisfied: pytz>=2020.1 in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (from pandas) (2023.3.post1) +๐Ÿค– โ—‹ โ—ท 2023-11-12 09:45:13.047 โ 43601 โ†’ 1/3 ๐Ÿฎ zrb project install-pandas โ€ข Requirement already satisfied: tzdata>=2022.1 in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (from pandas) (2023.3) +๐Ÿค– โ—‹ โ—ท 2023-11-12 09:45:13.049 โ 43601 โ†’ 1/3 ๐Ÿฎ zrb project install-pandas โ€ข Requirement already satisfied: six>=1.5 in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (from python-dateutil>=2.8.2->pandas) (1.16.0) +Support zrb growth and development! +โ˜• Donate at: https://stalchmst.com/donation +๐Ÿ™ Submit issues/PR at: https://github.com/state-alchemists/zrb +๐Ÿค Follow us at: https://twitter.com/zarubastalchmst +๐Ÿค– โ—‹ โ—ท 2023-11-12 09:45:14.366 โ 43598 โ†’ 1/3 ๐ŸŽ zrb project show-stats โ€ข Completed in 2.2365798950195312 seconds + sepal_length sepal_width petal_length petal_width +count 150.000000 150.000000 150.000000 150.000000 +mean 5.843333 3.054000 3.758667 1.198667 +std 0.828066 0.433594 1.764420 0.763161 +min 4.300000 2.000000 1.000000 0.100000 +25% 5.100000 2.800000 1.600000 0.300000 +50% 5.800000 3.000000 4.350000 1.300000 +75% 6.400000 3.300000 5.100000 1.800000 +max 7.900000 4.400000 6.900000 2.500000 +To run again: zrb project show-stats --url "https://raw.githubusercontent.com/state-alchemists/datasets/main/iris.csv" +``` # Creating a long-running task +Commonly, you can determine whether a task is successful/failed after it is completed. However, some tasks might run forever, and you can only see whether the task is completed or failed by checking other behaviors. For example, a web server is successfully running if you can get the expected HTTP response from the server. + +Zrb has some checking mechanisms to handle this use case. + +Let's start by scaffolding a CmdTask named `run-jupyterlab`. + +```bash +zrb project add cmd-task --project-dir "." --task-name "run-jupyterlab" +``` + +Once you do so, you can start modifying `_automate/` + +In some cases, your task has to run forever (i.e., web server). + Arasaka is a data-driven (and family-driven) company. They need their data scientists to experiment a lot to present the most valuable information/knowledge. For this, they need to be able to create a lot of notebooks for experimentation. @@ -380,107 +568,87 @@ zrb project add cmd-task --project-dir . --task-name start-jupyterlab Now, let's modify `_automate/start_jupyterlab.py` into the following: ```python -from zrb import CmdTask, runner, IntInput, PortChecker -from zrb.builtin._group import project_group -from _automate.show_banner import show_banner +from zrb import CmdTask, PortChecker, IntInput, runner +from zrb.builtin.group import project_group import os -############################################################################### -# Task Definitions -############################################################################### -notebook_path = os.path.join( - os.path.dirname(os.path.dirname(__file__)), 'src', 'notebooks' +install_jupyterlab = CmdTask( + name='install-jupyterlab', + group=project_group, + cmd='pip install jupyterlab' ) +runner.register(install_jupyterlab) + -start_jupyterlab = CmdTask( - name='start-jupyterlab', - description='start jupyterlab', +notebook_path = os.path.join( + os.path.dirname(os.path.dirname(__file__)), 'src' +) +run_jupyterlab = CmdTask( + name='run-jupyterlab', + description='run jupyterlab', group=project_group, + upstreams=[install_jupyterlab], inputs=[ - IntInput(name='jupyterlab-port', default=8080) + IntInput(name='jupyterlab-port', default=8080), + ], + cmd=[ + 'jupyter lab \\', + ' --port {{input.jupyterlab_port}}', + f' --notebook-dir "{notebook_path}"' ], - upstreams=[show_banner], - cmd='jupyter lab --no-browser --port={{input.jupyterlab_port}} ' + - f'--notebook-dir="{notebook_path}"', checkers=[ - PortChecker(port='{{input.jupyterlab_port}}') + PortChecker(name='check-jupyterlab', port='{{input.jupyterlab_port}}') ] ) -runner.register(start_jupyterlab) - -``` - -First of all, we import `IntInput` and `PortChecker`, so that we can ask the user to choose the port number and check whether jupyterlab has been started on that port. - -We also need to import `os`, so that we can determine the location of your `notebook_path`. - -Finally we add some properties to `start_jupyterlab`: - -- `inputs`: List of user inputs. We add a new input named `jupyterlab-port`. By default, the value will be `8080`. -- `upstreams`: List of tasks that should be completed before the current task is started. We want `show_banner` to be executed here. -- `cmd`: Command to be executed. You can use Jinja templating here. `{{input.jupyterlab_port}}` refers to the value `jupyterlab-port` input. -- `checkers`: List of task to determine whether current task has been completed or not. In this case we want to make sure that the port has already available for requests. - -## Starting the jupyterlab - -Finally, let's see and make sure things are working: - -``` -Jupyterlab port [8080]: -๐Ÿค– โžœ 2023-06-11T06:45:42.731189 โš™ 6237 โžค 1 of 3 โ€ข ๐Ÿจ zrb project show-banner โ€ข Run script: figlet Arasaka -๐Ÿค– โžœ 2023-06-11T06:45:42.731499 โš™ 6237 โžค 1 of 3 โ€ข ๐Ÿจ zrb project show-banner โ€ข Current working directory: /home/gofrendi/zrb/playground/my-project -๐Ÿค– โžœ 2023-06-11T06:45:42.736205 โš™ 6238 โžค 1 of 3 โ€ข ๐Ÿจ zrb project show-banner โ€ข _ _ -๐Ÿค– โžœ 2023-06-11T06:45:42.736518 โš™ 6238 โžค 1 of 3 โ€ข ๐Ÿจ zrb project show-banner โ€ข / \ _ __ __ _ ___ __ _| | ____ _ -๐Ÿค– โžœ 2023-06-11T06:45:42.736782 โš™ 6238 โžค 1 of 3 โ€ข ๐Ÿจ zrb project show-banner โ€ข / _ \ | '__/ _` / __|/ _` | |/ / _` | -๐Ÿค– โžœ 2023-06-11T06:45:42.737349 โš™ 6238 โžค 1 of 3 โ€ข ๐Ÿจ zrb project show-banner โ€ข / ___ \| | | (_| \__ \ (_| | < (_| | -๐Ÿค– โžœ 2023-06-11T06:45:42.737637 โš™ 6238 โžค 1 of 3 โ€ข ๐Ÿจ zrb project show-banner โ€ข /_/ \_\_| \__,_|___/\__,_|_|\_\__,_| -๐Ÿค– โžœ 2023-06-11T06:45:42.737940 โš™ 6238 โžค 1 of 3 โ€ข ๐Ÿจ zrb project show-banner โ€ข -๐Ÿค– โžœ 2023-06-11T06:45:42.741398 โš™ 6237 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข Run script: jupyter lab --no-browser --port=8080 --notebook-dir="/home/gofrendi/zrb/playground/my-project/src/notebooks" -๐Ÿค– โžœ 2023-06-11T06:45:42.741681 โš™ 6237 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข Current working directory: /home/gofrendi/zrb/playground/my-project -๐Ÿค– โš  2023-06-11T06:45:43.347664 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข [I 2023-06-11 06:45:43.347 ServerApp] Package jupyterlab took 0.0000s to import -๐Ÿค– โš  2023-06-11T06:45:43.354037 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข [I 2023-06-11 06:45:43.353 ServerApp] Package jupyter_lsp took 0.0061s to import -๐Ÿค– โš  2023-06-11T06:45:43.354341 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข [W 2023-06-11 06:45:43.353 ServerApp] A `_jupyter_server_extension_points` function was not found in jupyter_lsp. Instead, a `_jupyter_server_extension_paths` function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server. -๐Ÿค– โš  2023-06-11T06:45:43.357141 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข [I 2023-06-11 06:45:43.356 ServerApp] Package jupyter_server_terminals took 0.0029s to import -๐Ÿค– โš  2023-06-11T06:45:43.357496 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข [I 2023-06-11 06:45:43.357 ServerApp] Package notebook_shim took 0.0000s to import -๐Ÿค– โš  2023-06-11T06:45:43.357800 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข [W 2023-06-11 06:45:43.357 ServerApp] A `_jupyter_server_extension_points` function was not found in notebook_shim. Instead, a `_jupyter_server_extension_paths` function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server. -๐Ÿค– โš  2023-06-11T06:45:43.358139 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข [I 2023-06-11 06:45:43.357 ServerApp] jupyter_lsp | extension was successfully linked. -๐Ÿค– โš  2023-06-11T06:45:43.360703 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข [I 2023-06-11 06:45:43.360 ServerApp] jupyter_server_terminals | extension was successfully linked. -๐Ÿค– โš  2023-06-11T06:45:43.364479 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข [I 2023-06-11 06:45:43.364 ServerApp] jupyterlab | extension was successfully linked. -๐Ÿค– โš  2023-06-11T06:45:43.489074 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข [I 2023-06-11 06:45:43.488 ServerApp] notebook_shim | extension was successfully linked. -๐Ÿค– โš  2023-06-11T06:45:43.538464 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข [I 2023-06-11 06:45:43.537 ServerApp] notebook_shim | extension was successfully loaded. -๐Ÿค– โš  2023-06-11T06:45:43.539844 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข [I 2023-06-11 06:45:43.539 ServerApp] jupyter_lsp | extension was successfully loaded. -๐Ÿค– โš  2023-06-11T06:45:43.540686 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข [I 2023-06-11 06:45:43.540 ServerApp] jupyter_server_terminals | extension was successfully loaded. -๐Ÿค– โš  2023-06-11T06:45:43.541056 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข [I 2023-06-11 06:45:43.540 LabApp] JupyterLab extension loaded from /home/gofrendi/zrb/venv/lib/python3.9/site-packages/jupyterlab -๐Ÿค– โš  2023-06-11T06:45:43.541399 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข [I 2023-06-11 06:45:43.540 LabApp] JupyterLab application directory is /home/gofrendi/zrb/venv/share/jupyter/lab -๐Ÿค– โš  2023-06-11T06:45:43.541722 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข [I 2023-06-11 06:45:43.541 LabApp] Extension Manager is 'pypi'. -๐Ÿค– โš  2023-06-11T06:45:43.543932 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข [I 2023-06-11 06:45:43.543 ServerApp] jupyterlab | extension was successfully loaded. -๐Ÿค– โš  2023-06-11T06:45:43.544397 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข [I 2023-06-11 06:45:43.543 ServerApp] Serving notebooks from local directory: /home/gofrendi/zrb/playground/my-project/src/notebooks -๐Ÿค– โš  2023-06-11T06:45:43.544742 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข [I 2023-06-11 06:45:43.544 ServerApp] Jupyter Server 2.6.0 is running at: -๐Ÿค– โš  2023-06-11T06:45:43.545059 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข [I 2023-06-11 06:45:43.544 ServerApp] http://localhost:8080/lab?token=74085eb7b8304271e028c5e0e01237ebaadbb13a54a64921 -๐Ÿค– โš  2023-06-11T06:45:43.545395 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข [I 2023-06-11 06:45:43.544 ServerApp] http://127.0.0.1:8080/lab?token=74085eb7b8304271e028c5e0e01237ebaadbb13a54a64921 -๐Ÿค– โš  2023-06-11T06:45:43.545720 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข [I 2023-06-11 06:45:43.544 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation). -๐Ÿค– โš  2023-06-11T06:45:43.547067 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข [C 2023-06-11 06:45:43.546 ServerApp] -๐Ÿค– โš  2023-06-11T06:45:43.547407 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข -๐Ÿค– โš  2023-06-11T06:45:43.547855 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข To access the server, open this file in a browser: -๐Ÿค– โš  2023-06-11T06:45:43.548628 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข file:///home/gofrendi/.local/share/jupyter/runtime/jpserver-6240-open.html -๐Ÿค– โš  2023-06-11T06:45:43.549002 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข Or copy and paste one of these URLs: -๐Ÿค– โš  2023-06-11T06:45:43.549389 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข http://localhost:8080/lab?token=74085eb7b8304271e028c5e0e01237ebaadbb13a54a64921 -๐Ÿค– โš  2023-06-11T06:45:43.549734 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข http://127.0.0.1:8080/lab?token=74085eb7b8304271e028c5e0e01237ebaadbb13a54a64921 -๐Ÿค– โžœ 2023-06-11T06:45:43.641677 โš™ 6237 โžค 1 of 1 โ€ข ๐Ÿ port-check โ€ข Checking localhost:8080 (OK) +runner.register(run_jupyterlab) +``` + +You may notice that `run_jupyterlab` has a `PortChecker` on it. If the `PortChecker` can get TCP response, then `run_jupyterlab` is considered successful. +Let's run the task: + +```bash +zrb project run-jupyterlab +``` + +``` +Jupyterlab port [8080]: +๐Ÿค– โ—‹ โ—ท 2023-11-12 10:26:32.759 โ 58728 โ†’ 1/3 ๐Ÿจ zrb project install-jupyterlab โ€ข Run script: pip install jupyterlab +๐Ÿค– โ—‹ โ—ท 2023-11-12 10:26:32.759 โ 58728 โ†’ 1/3 ๐Ÿจ zrb project install-jupyterlab โ€ข Working directory: /home/gofrendi/playground/my-project +๐Ÿค– โ—‹ โ—ท 2023-11-12 10:26:33.109 โ 58731 โ†’ 1/3 ๐Ÿจ zrb project install-jupyterlab โ€ข Requirement already satisfied: jupyterlab in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (4.0.8) +๐Ÿค– โ—‹ โ—ท 2023-11-12 10:26:33.149 โ 58731 โ†’ 1/3 ๐Ÿจ zrb project install-jupyterlab โ€ข Requirement already satisfied: async-lru>=1.0.0 in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (from jupyterlab) (2.0.4) +๐Ÿค– โ—‹ โ—ท 2023-11-12 10:26:33.151 โ 58731 โ†’ 1/3 ๐Ÿจ zrb project install-jupyterlab โ€ข Requirement already satisfied: ipykernel in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (from jupyterlab) (6.26.0) +๐Ÿค– โ—‹ โ—ท 2023-11-12 10:26:33.153 โ 58731 โ†’ 1/3 ๐Ÿจ zrb project install-jupyterlab โ€ข Requirement already satisfied: jinja2>=3.0.3 in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (from jupyterlab) (3.1.2) +๐Ÿค– โ—‹ โ—ท 2023-11-12 10:26:33.156 โ 58731 โ†’ 1/3 ๐Ÿจ zrb project install-jupyterlab โ€ข Requirement already satisfied: jupyter-core in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (from jupyterlab) (5.5.0) +๐Ÿค– โ—‹ โ—ท 2023-11-12 10:26:33.968 โ 58731 โ†’ 1/3 ๐Ÿจ zrb project install-jupyterlab โ€ข Requirement already satisfied: pycparser in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (from cffi>=1.0.1->argon2-cffi-bindings->argon2-cffi->jupyter-server<3,>=2.4.0->jupyterlab) (2.21) +๐Ÿค– โ—‹ โ—ท 2023-11-12 10:26:34.041 โ 58731 โ†’ 1/3 ๐Ÿจ zrb project install-jupyterlab โ€ข Requirement already satisfied: arrow>=0.15.0 in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (from isoduration->jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.6.0->jupyter-server<3,>=2.4.0->jupyterlab) (1.3.0) +๐Ÿค– โ—‹ โ—ท 2023-11-12 10:26:34.093 โ 58731 โ†’ 1/3 ๐Ÿจ zrb project install-jupyterlab โ€ข Requirement already satisfied: types-python-dateutil>=2.8.10 in /home/gofrendi/zrb/.venv/lib/python3.10/site-packages (from arrow>=0.15.0->isoduration->jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.6.0->jupyter-server<3,>=2.4.0->jupyterlab) (2.8.19.14) +๐Ÿค– โ—‹ โ—ท 2023-11-12 10:26:34.717 โ 58728 โ†’ 1/3 ๐Ÿน zrb project run-jupyterlab โ€ข Run script: + 0001 | jupyter lab \ + 0002 | --port 8080 + 0003 | --notebook-dir "/home/gofrendi/playground/my-project/src" +๐Ÿค– โ—‹ โ—ท 2023-11-12 10:26:34.717 โ 58728 โ†’ 1/3 ๐Ÿน zrb project run-jupyterlab โ€ข Working directory: /home/gofrendi/playground/my-project +๐Ÿค– โ–ณ โ—ท 2023-11-12 10:26:35.693 โ 58920 โ†’ 1/3 ๐Ÿน zrb project run-jupyterlab โ€ข [I 2023-11-12 10:26:35.675 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation). +๐Ÿค– โ–ณ โ—ท 2023-11-12 10:26:36.789 โ 58920 โ†’ 1/3 ๐Ÿน zrb project run-jupyterlab โ€ข [C 2023-11-12 10:26:36.788 ServerApp] +๐Ÿค– โ–ณ โ—ท 2023-11-12 10:26:36.791 โ 58920 โ†’ 1/3 ๐Ÿน zrb project run-jupyterlab โ€ข +๐Ÿค– โ–ณ โ—ท 2023-11-12 10:26:36.793 โ 58920 โ†’ 1/3 ๐Ÿน zrb project run-jupyterlab โ€ข To access the server, open this file in a browser: +๐Ÿค– โ–ณ โ—ท 2023-11-12 10:26:36.795 โ 58920 โ†’ 1/3 ๐Ÿน zrb project run-jupyterlab โ€ข file:///home/gofrendi/.local/share/jupyter/runtime/jpserver-58922-open.html +๐Ÿค– โ–ณ โ—ท 2023-11-12 10:26:36.798 โ 58920 โ†’ 1/3 ๐Ÿน zrb project run-jupyterlab โ€ข Or copy and paste one of these URLs: +๐Ÿค– โ–ณ โ—ท 2023-11-12 10:26:36.799 โ 58920 โ†’ 1/3 ๐Ÿน zrb project run-jupyterlab โ€ข http://localhost:8080/lab?token=58eecd6aa4a56445ecf8b8d8c2f2148d47a7ce8456ecd680 +๐Ÿค– โ–ณ โ—ท 2023-11-12 10:26:36.801 โ 58920 โ†’ 1/3 ๐Ÿน zrb project run-jupyterlab โ€ข http://127.0.0.1:8080/lab?token=58eecd6aa4a56445ecf8b8d8c2f2148d47a7ce8456ecd680 +๐Ÿค– โ—‹ โ—ท 2023-11-12 10:26:36.807 โ 58728 โ†’ 1/1 ๐Ÿน check-jupyterlab โ€ข Checking localhost:8080 (OK) Support zrb growth and development! โ˜• Donate at: https://stalchmst.com/donation -๐Ÿ™ Submit issues/pull requests at: https://github.com/state-alchemists/zrb +๐Ÿ™ Submit issues/PR at: https://github.com/state-alchemists/zrb ๐Ÿค Follow us at: https://twitter.com/zarubastalchmst -๐Ÿค– โžœ 2023-06-11T06:45:43.643523 โš™ 6240 โžค 1 of 3 โ€ข ๐Ÿถ zrb project start-jupyterlab โ€ข zrb project start-jupyterlab completed in 1.103727102279663 seconds -To run again: zrb project start-jupyterlab --jupyterlab-port "8080" +๐Ÿค– โ—‹ โ—ท 2023-11-12 10:26:36.807 โ 58920 โ†’ 1/3 ๐Ÿน zrb project run-jupyterlab โ€ข Completed in 4.050489664077759 seconds ``` -Open up your browser on `http://localhost:8080` and start working. +Open up your browser on [http://localhost:8080](http://localhost:8080) to start working with the notebook. # Now you are ready -We have cover the minimum basics to work ~~for Arasaka~~ with Zrb. +We have covered everything you need to know to work with Zrb. To learn more about tasks and other concepts, you can visit [Zrb concept section](concepts/README.md). diff --git a/src/zrb/builtin/base64.py b/src/zrb/builtin/base64.py index 8788d4d6..7b38ef42 100644 --- a/src/zrb/builtin/base64.py +++ b/src/zrb/builtin/base64.py @@ -26,7 +26,7 @@ name='encode', group=base64_group, inputs=[text_input], - description='Encode base64 task', + description='Encode a text using base64 algorithm', retry=0, runner=runner ) @@ -40,7 +40,7 @@ async def encode(*args: str, **kwargs: Any): name='decode', group=base64_group, inputs=[text_input], - description='Decode base64 task', + description='Decode a base64 encoded text', retry=0, runner=runner )