From dc0e85f398903ca74886a20eed5e3efd197ef796 Mon Sep 17 00:00:00 2001 From: Enrico Deusebio Date: Thu, 17 Oct 2024 14:10:29 +0200 Subject: [PATCH 01/19] [MISC] Adding documentation for AWS and Azure deployment --- docs/how-to/h-deploy-aws.md | 194 +++++++++++++++++++++++++++ docs/how-to/h-deploy-azure.md | 239 ++++++++++++++++++++++++++++++++++ 2 files changed, 433 insertions(+) create mode 100644 docs/how-to/h-deploy-aws.md create mode 100644 docs/how-to/h-deploy-azure.md diff --git a/docs/how-to/h-deploy-aws.md b/docs/how-to/h-deploy-aws.md new file mode 100644 index 00000000..e9fae717 --- /dev/null +++ b/docs/how-to/h-deploy-aws.md @@ -0,0 +1,194 @@ +# How to deploy on AWS + +[Amazon Web Services](https://aws.amazon.com/) is a popular subsidiary of Amazon that provides on-demand cloud computing platforms on a metered pay-as-you-go basis. Access the AWS web console at [console.aws.amazon.com](https://console.aws.amazon.com/). + +## Summary +* [Install AWS and Juju tooling](#install-aws-and-juju-tooling) + * [Authenticate](#authenticate) +* [Bootstrap Juju controller on AWS EC2](#bootstrap-juju-controller-on-aws-ec2) +* [Deploy charms](#deploy-charms) +* [Expose database (optional)](#expose-database-optional) +* [Clean up](#clean-up) + +--- + +## Install AWS and Juju tooling + +Install Juju via snap: +```shell +sudo snap install juju +``` + +Follow the installation guides for: +* [AWs CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) - the Amazon Web Services CLI + +To check they are all correctly installed, you can run the commands demonstrated below with sample outputs: + +```console +~$ juju version +3.5.4-genericlinux-amd64 + +~$ aws --version +aws-cli/2.13.25 Python/3.11.5 Linux/6.2.0-33-generic exe/x86_64.ubuntu.23 prompt/off +``` +### Authenticate +[Create an IAM account](https://docs.aws.amazon.com/eks/latest/userguide/getting-started-console.html) (or use legacy access keys) to operate AWS EC2: +```shell +mkdir -p ~/.aws && cat <<- EOF > ~/.aws/credentials.yaml +credentials: + aws: + NAME_OF_YOUR_CREDENTIAL: + auth-type: access-key + access-key: SECRET_ACCESS_KEY_ID + secret-key: SECRET_ACCESS_KEY_VALUE +EOF +``` + + + +## Bootstrap Juju controller on AWS EC2 + +Add AWS credentials to Juju: +```shell +juju add-credential aws -f ~/.aws/credentials.yaml +``` +Bootstrap Juju controller ([check all supported configuration options](https://juju.is/docs/juju/amazon-ec2)): +```shell +juju bootstrap aws +``` +[details="Output example"] +```shell +> juju bootstrap aws +Creating Juju controller "aws-us-east-1" on aws/us-east-1 +Looking for packaged Juju agent version 3.5.4 for amd64 +Located Juju agent version 3.5.4-ubuntu-amd64 at https://juju-dist-aws.s3.amazonaws.com/agents/agent/3.5.4/juju-3.5.4-linux-amd64.tgz +Launching controller instance(s) on aws/us-east-1... + - i-0f4615983d113166d (arch=amd64 mem=8G cores=2) +Installing Juju agent on bootstrap instance +Waiting for address +Attempting to connect to 54.226.221.6:22 +Attempting to connect to 172.31.20.34:22 +Connected to 54.226.221.6 +Running machine configuration script... +Bootstrap agent now started +Contacting Juju controller at 54.226.221.6 to verify accessibility... + +Bootstrap complete, controller "aws-us-east-1" is now available +Controller machines are in the "controller" model + +Now you can run + juju add-model +to create a new model to deploy workloads. +``` +[/details] + +You can check the [AWS EC2 instance availability](https://us-east-1.console.aws.amazon.com/ec2/home?region=us-east-1#Instances:instanceState=running) (ensure the right AWS region chosen!): +![image|690x118](upload://putAO5NyHdaeWE6jXI8X1hZHTYv.png) + +## Deploy charms + +Create a new Juju model, if needed: +```shell +juju add-model +``` +> (Optional) Increase the debug level if you are troubleshooting charms: +> ```shell +> juju model-config logging-config='=INFO;unit=DEBUG' +> ``` + +Then, Charmed Kafka can be deployed as usual. However, note that the smallest instance types on Azure may not have enough resources for hosting +a Kafka broker. We therefore recommend you to select some types that provides at the very least 8GB of RAM and 4 cores, although for production use-case +we recommend you to use the guidance provided in the [requirement page](/t/charmed-kafka-reference-requirements/10563). You can find more information about +the available instance types in the [Azure web page](https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/overview). + +```shell +juju deploy zookeeper -n3 --channel 3/stable +juju deploy kafka -n3 --constraints "instance-type=Standard_A4_v2" --channel 3/stable +juju integrate kafka zookeeper +``` + +We also recommend to deploy a [Data Integrator](https://charmhub.io/data-integrator) for creating an admin user to manage the content of the Kafka cluster: + +```shell +juju deploy data-integrator admin --channel edge \ + --config extra-user-roles=admin \ + --config topic-name=admin-topic +``` + +And integrate it with the Kafka application: + +```shell +juju integrate kafka admin +``` + +For more information on Data Integrator and how to use it, please refer to the [how-to manage applications](/t/charmed-kafka-how-to-manage-app/10285) user guide. + +## Clean up + +[note type="caution"] +Always clean AWS resources that are no longer necessary - they could be costly! +[/note] + +To destroy the Juju controller and remove AWS instance (warning: all your data will be permanently removed): +```shell +> juju controllers +Controller Model User Access Cloud/Region Models Nodes HA Version +aws-us-east-1* - admin superuser aws/us-east-1 1 1 none 3.5.4 + +> juju destroy-controller aws-us-east-1 --destroy-all-models --destroy-storage --force +``` + +Next, check and manually delete all unnecessary AWS EC2 instances, to show the list of all your EC2 instances run the following command (make sure the correct region used!): +```shell +aws ec2 describe-instances --region us-east-1 --query "Reservations[].Instances[*].{InstanceType: InstanceType, InstanceId: InstanceId, State: State.Name}" --output table +``` +[details="Output example"] +```shell +------------------------------------------------------- +| DescribeInstances | ++---------------------+----------------+--------------+ +| InstanceId | InstanceType | State | ++---------------------+----------------+--------------+ +| i-0f374435695ffc54c| m7i.large | terminated | +| i-0e1e8279f6b2a08e0| m7i.large | terminated | +| i-061e0d10d36c8cffe| m7i.large | terminated | +| i-0f4615983d113166d| m7i.large | terminated | ++---------------------+----------------+--------------+ +``` +[/details] + +List your Juju credentials: +```shell +> juju credentials +... +Client Credentials: +Cloud Credentials +aws NAME_OF_YOUR_CREDENTIAL +... +``` +Remove AWS EC2 CLI credentials from Juju: +```shell +> juju remove-credential aws NAME_OF_YOUR_CREDENTIAL +``` + +Finally, remove AWS CLI user credentials (to avoid forgetting and leaking): +```shell +rm -f ~/.aws/credentials.yaml +``` diff --git a/docs/how-to/h-deploy-azure.md b/docs/how-to/h-deploy-azure.md new file mode 100644 index 00000000..e54c5899 --- /dev/null +++ b/docs/how-to/h-deploy-azure.md @@ -0,0 +1,239 @@ +# How to deploy on Azure + +[Azure](https://azure.com/) is the cloud computing platform developed by Microsoft. It has management, access and development of applications and services to individuals, companies, and governments through its global infrastructure. Access the Azure web console at [portal.azure.com](https://portal.azure.com/). + +## Summary +* [Install Azure and Juju tooling](#install-azure-and-juju-tooling) + * [Authenticate](#authenticate) +* [Bootstrap Juju controller on Azure](#bootstrap-juju-controller-on-azure) +* [Deploy charms](#deploy-charms) +* [Clean up](#clean-up) + +--- + +## Install Client Environment + +> **WARNING**: The current limitations: +> * Only supported starting Juju 3.6 (currently edge) +> * Juju cli should be on Azure VM for it to be able to reach cloud metadata endpoint. +> * Managed Identity and the Juju resources should be on the same Azure subscription +> * The current setup has been tested on Ubuntu 22.04+ + +### Juju + +Install Juju via snap: + +```shell +sudo snap install juju --channel 3.6/edge +``` + +Check that the Juju version is correctly installed: + +```shell +juju version +``` + +which should show: + +```shell +3.6-rc1-genericlinux-amd64 +``` + +### Azure CLI + +Follow the [user guide](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-linux?pivots=apt) for installing the Azure CLI on Linux distributions. + +Verify that it is correctly installed running the command below: + +```shell +az --version +``` + +which should show the following output: + +```shell +azure-cli 2.65.0 + +core 2.65.0 +telemetry 1.1.0 + +Dependencies: +msal 1.31.0 +azure-mgmt-resource 23.1.1 + +Python location '/opt/az/bin/python3' +Extensions directory '/home/deusebio/.azure/cliextensions' + +Python (Linux) 3.11.8 (main, Sep 25 2024, 11:33:44) [GCC 11.4.0] + +Legal docs and information: aka.ms/AzureCliLegal + + +Your CLI is up-to-date. +``` + +### Authenticate + +> For more information on how to authenticate, refer to the [Juju documentation](/t/the-microsoft-azure-cloud-and-juju/1086) and [a dedicated user guide on how to register Azure on Juju](/t/how-to-use-juju-with-microsoft-azure/15219). + +First of all, retrieve your subscription-id + +```shell +az login +``` + +After providing the authentication via web browser, you will be shown a list of information and a table with the subscriptions connected to your user, e.g.: + +```shell + +... + +No Subscription name Subscription ID Tenant +----- ------------------------------------ ------------------------------------ ------------- +[1] * canonical.com +[2] canonical.com + +... + +``` + +In the prompt, select the subscription id you would like to connect the controller to, and store the id +as it will be needed in the next step when bootstrapping the controller. + +### Bootstrap Juju controller on Azure + +First, you need to add a set of credentials to your Juju client: + +```shell +juju add-credentials azure +``` + +This will start a script that will help you set up the credentials, where you will be asked: + +* `credential-name`: fill it with a sensible name that will help you identify the credential set, say `` +* `region`: select any default region is more convenient for you to deploy your controller and applications. Note that credentials are nevertheless not region specific +* `auth type`: select `interactive`, which is the recommended way to authenticate to Azure using Juju +* `subscription_id`: use the value `` taken in the previous step +* `application_name`: please generate a random string to avoid collision with other users or applications +* `role-definition-name`: please generate a random string to avoid collision with other users or applications, and store it as `` + +After prompting all these information, you will be asked to authenticate the requests via web browser, as shown in the +outputs that will display + +```shell +To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code to authenticate. +``` + +In the browser, open the [authentication page](https://microsoft.com/devicelogin) and enter the code `` provided in the output. + +You will be asked to authenticate twice, for allowing the creation of two different resources in Azure. + +If everything was fine, you will be confirmed that the credentials have been correctly added locally: + +```shell +Credential added locally for cloud "azure". +``` + +Once the credentials are correctly added, we can bootstrap a controller + +```shell +juju bootstrap azure +``` + +## Deploy charms + +Create a new Juju model, if needed: +```shell +juju add-model +``` +> (Optional) Increase the debug level if you are troubleshooting charms: +> ```shell +> juju model-config logging-config='=INFO;unit=DEBUG' +> ``` + +Then, Charmed Kafka can be deployed as usual. However, note that the smallest instance types on Azure may not have enough resources for hosting +a Kafka broker. We therefore recommend you to select some types that provides at the very least 8GB of RAM and 4 cores, although for production use-case +we recommend you to use the guidance provided in the [requirement page](/t/charmed-kafka-reference-requirements/10563). You can find more information about +the available instance types in the [Azure web page](https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/overview). + +```shell +juju deploy zookeeper -n3 --channel 3/stable +juju deploy kafka -n3 --constraints "instance-type=Standard_A4_v2" --channel 3/stable +juju integrate kafka zookeeper +``` + +We also recommend to deploy a [Data Integrator](https://charmhub.io/data-integrator) for creating an admin user to manage the content of the Kafka cluster: + +```shell +juju deploy data-integrator admin --channel edge \ + --config extra-user-roles=admin \ + --config topic-name=admin-topic +``` + +And integrate it with the Kafka application: + +```shell +juju integrate kafka admin +``` + +For more information on Data Integrator and how to use it, please refer to the [how-to manage applications](/t/charmed-kafka-how-to-manage-app/10285) user guide. + +## Clean up + +[note type="caution"] +Always clean Azure resources that are no longer necessary - they could be costly! +[/note] + +To destroy the Juju controller and remove Azure instance (warning: all your data will be permanently removed): +```shell +juju destroy-controller --destroy-all-models --destroy-storage --force +``` + +> Use `juju list-controllers` to retrieve the names of the controllers that have been registered to your local client. + +Next, check and manually delete all unnecessary Azure VM instances, to show the list of all your Azure VMs run the following command (make sure the correct region used!): +```shell +az resource list +``` + +List your Juju credentials: +```shell +> juju credentials +... +Client Credentials: +Cloud Credentials +azure NAME_OF_YOUR_CREDENTIAL +... +``` +Remove Azure CLI credentials from Juju: +```shell +> juju remove-credential azure NAME_OF_YOUR_CREDENTIAL +``` + +After deleting the credentials, the `interactive` process may still leave the role resource and its assignment hanging around. +We recommend you to check if these are still present by + +```shell +az role definition list --name +``` + +Or also use it without specifying the `--name` argument to get the full list. You can also check whether you still have a +role assignment bound to `` registered using + +```shell +az role assigment list --role +``` + +If this is the case, you can remove the role assignment first and then the role itself with the following commands: + +```shell +az role assignment delete --role +az role definition delete --name +``` + + +Finally, logout Azure CLI user credentials to prevent any credential leakage: +```shell +az logout +``` + From 5cde17a3b640f51e657cbcfaeadcad5827a68d48 Mon Sep 17 00:00:00 2001 From: Enrico Deusebio Date: Thu, 17 Oct 2024 14:18:06 +0200 Subject: [PATCH 02/19] fixing typos --- docs/how-to/h-deploy-aws.md | 1 - docs/how-to/h-deploy-azure.md | 2 +- 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/how-to/h-deploy-aws.md b/docs/how-to/h-deploy-aws.md index e9fae717..3fc7a810 100644 --- a/docs/how-to/h-deploy-aws.md +++ b/docs/how-to/h-deploy-aws.md @@ -7,7 +7,6 @@ * [Authenticate](#authenticate) * [Bootstrap Juju controller on AWS EC2](#bootstrap-juju-controller-on-aws-ec2) * [Deploy charms](#deploy-charms) -* [Expose database (optional)](#expose-database-optional) * [Clean up](#clean-up) --- diff --git a/docs/how-to/h-deploy-azure.md b/docs/how-to/h-deploy-azure.md index e54c5899..09597cda 100644 --- a/docs/how-to/h-deploy-azure.md +++ b/docs/how-to/h-deploy-azure.md @@ -221,7 +221,7 @@ Or also use it without specifying the `--name` argument to get the full list. Yo role assignment bound to `` registered using ```shell -az role assigment list --role +az role assignment list --role ``` If this is the case, you can remove the role assignment first and then the role itself with the following commands: From 2a6bdb87981b3a5a3c1f8f87800311abf2662f9a Mon Sep 17 00:00:00 2001 From: Enrico Deusebio Date: Fri, 18 Oct 2024 19:01:10 +0200 Subject: [PATCH 03/19] small fix --- docs/how-to/h-deploy-azure.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/how-to/h-deploy-azure.md b/docs/how-to/h-deploy-azure.md index 09597cda..1fffcc48 100644 --- a/docs/how-to/h-deploy-azure.md +++ b/docs/how-to/h-deploy-azure.md @@ -3,7 +3,7 @@ [Azure](https://azure.com/) is the cloud computing platform developed by Microsoft. It has management, access and development of applications and services to individuals, companies, and governments through its global infrastructure. Access the Azure web console at [portal.azure.com](https://portal.azure.com/). ## Summary -* [Install Azure and Juju tooling](#install-azure-and-juju-tooling) +* [Install Azure and Juju tooling](#install-client-environment) * [Authenticate](#authenticate) * [Bootstrap Juju controller on Azure](#bootstrap-juju-controller-on-azure) * [Deploy charms](#deploy-charms) From 66da627d21de1cac7b360e91a430df425ec4d2b4 Mon Sep 17 00:00:00 2001 From: deusebio Date: Tue, 22 Oct 2024 11:21:54 +0200 Subject: [PATCH 04/19] Update docs/how-to/h-deploy-aws.md Co-authored-by: Vladimir Izmalkov <48120135+izmalk@users.noreply.github.com> --- docs/how-to/h-deploy-aws.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/how-to/h-deploy-aws.md b/docs/how-to/h-deploy-aws.md index 3fc7a810..5685bd1a 100644 --- a/docs/how-to/h-deploy-aws.md +++ b/docs/how-to/h-deploy-aws.md @@ -3,6 +3,7 @@ [Amazon Web Services](https://aws.amazon.com/) is a popular subsidiary of Amazon that provides on-demand cloud computing platforms on a metered pay-as-you-go basis. Access the AWS web console at [console.aws.amazon.com](https://console.aws.amazon.com/). ## Summary + * [Install AWS and Juju tooling](#install-aws-and-juju-tooling) * [Authenticate](#authenticate) * [Bootstrap Juju controller on AWS EC2](#bootstrap-juju-controller-on-aws-ec2) From 9e900144fa9ef8aed837fbcc177007b278f8c0a6 Mon Sep 17 00:00:00 2001 From: deusebio Date: Tue, 22 Oct 2024 11:29:20 +0200 Subject: [PATCH 05/19] Update docs/how-to/h-deploy-aws.md Co-authored-by: Vladimir Izmalkov <48120135+izmalk@users.noreply.github.com> --- docs/how-to/h-deploy-aws.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/how-to/h-deploy-aws.md b/docs/how-to/h-deploy-aws.md index 5685bd1a..31452c05 100644 --- a/docs/how-to/h-deploy-aws.md +++ b/docs/how-to/h-deploy-aws.md @@ -15,6 +15,7 @@ ## Install AWS and Juju tooling Install Juju via snap: + ```shell sudo snap install juju ``` From bc209660a93a816ff2d58156823e5b8b4ec15e7d Mon Sep 17 00:00:00 2001 From: deusebio Date: Tue, 22 Oct 2024 11:29:42 +0200 Subject: [PATCH 06/19] Update docs/how-to/h-deploy-aws.md Co-authored-by: Vladimir Izmalkov <48120135+izmalk@users.noreply.github.com> --- docs/how-to/h-deploy-aws.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/how-to/h-deploy-aws.md b/docs/how-to/h-deploy-aws.md index 31452c05..8cb29290 100644 --- a/docs/how-to/h-deploy-aws.md +++ b/docs/how-to/h-deploy-aws.md @@ -23,7 +23,7 @@ sudo snap install juju Follow the installation guides for: * [AWs CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) - the Amazon Web Services CLI -To check they are all correctly installed, you can run the commands demonstrated below with sample outputs: +To check whether both Juju and AWS CLI are correctly installed, run commands to display their versions: ```console ~$ juju version From 6fc838bbfc2e53b526c8d2dc9d5f5a4193845cdc Mon Sep 17 00:00:00 2001 From: deusebio Date: Tue, 22 Oct 2024 11:46:15 +0200 Subject: [PATCH 07/19] Update docs/how-to/h-deploy-aws.md Co-authored-by: Vladimir Izmalkov <48120135+izmalk@users.noreply.github.com> --- docs/how-to/h-deploy-aws.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/how-to/h-deploy-aws.md b/docs/how-to/h-deploy-aws.md index 8cb29290..a9cf61f4 100644 --- a/docs/how-to/h-deploy-aws.md +++ b/docs/how-to/h-deploy-aws.md @@ -32,7 +32,9 @@ To check whether both Juju and AWS CLI are correctly installed, run commands to ~$ aws --version aws-cli/2.13.25 Python/3.11.5 Linux/6.2.0-33-generic exe/x86_64.ubuntu.23 prompt/off ``` + ### Authenticate + [Create an IAM account](https://docs.aws.amazon.com/eks/latest/userguide/getting-started-console.html) (or use legacy access keys) to operate AWS EC2: ```shell mkdir -p ~/.aws && cat <<- EOF > ~/.aws/credentials.yaml From 88c82d60e122aab6a09d43cfdb16a4f132efe1d5 Mon Sep 17 00:00:00 2001 From: deusebio Date: Tue, 22 Oct 2024 11:46:32 +0200 Subject: [PATCH 08/19] Update docs/how-to/h-deploy-azure.md Co-authored-by: Vladimir Izmalkov <48120135+izmalk@users.noreply.github.com> --- docs/how-to/h-deploy-azure.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/how-to/h-deploy-azure.md b/docs/how-to/h-deploy-azure.md index 1fffcc48..78932ea7 100644 --- a/docs/how-to/h-deploy-azure.md +++ b/docs/how-to/h-deploy-azure.md @@ -217,7 +217,9 @@ We recommend you to check if these are still present by az role definition list --name ``` -Or also use it without specifying the `--name` argument to get the full list. You can also check whether you still have a +To get the full list, use it without specifying the `--name` argument. + +You can check whether you still have a role assignment bound to `` registered using ```shell From 285e03b7f6c082feb48d491c4cd111df47179611 Mon Sep 17 00:00:00 2001 From: deusebio Date: Tue, 22 Oct 2024 11:46:45 +0200 Subject: [PATCH 09/19] Update docs/how-to/h-deploy-azure.md Co-authored-by: Vladimir Izmalkov <48120135+izmalk@users.noreply.github.com> --- docs/how-to/h-deploy-azure.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/how-to/h-deploy-azure.md b/docs/how-to/h-deploy-azure.md index 78932ea7..0b03286b 100644 --- a/docs/how-to/h-deploy-azure.md +++ b/docs/how-to/h-deploy-azure.md @@ -226,7 +226,7 @@ role assignment bound to `` registered using az role assignment list --role ``` -If this is the case, you can remove the role assignment first and then the role itself with the following commands: +If there is an unwanted role left, you can remove the role assignment first and then the role itself with the following commands: ```shell az role assignment delete --role From 4224c0e50f6c03609729dae9f0dc74a6813547c8 Mon Sep 17 00:00:00 2001 From: deusebio Date: Tue, 22 Oct 2024 11:46:55 +0200 Subject: [PATCH 10/19] Update docs/how-to/h-deploy-azure.md Co-authored-by: Vladimir Izmalkov <48120135+izmalk@users.noreply.github.com> --- docs/how-to/h-deploy-azure.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/how-to/h-deploy-azure.md b/docs/how-to/h-deploy-azure.md index 0b03286b..72387284 100644 --- a/docs/how-to/h-deploy-azure.md +++ b/docs/how-to/h-deploy-azure.md @@ -233,7 +233,6 @@ az role assignment delete --role az role definition delete --name ``` - Finally, logout Azure CLI user credentials to prevent any credential leakage: ```shell az logout From dc4ad995fb86e7cd2a7edc790fe4d3d7bf876e26 Mon Sep 17 00:00:00 2001 From: deusebio Date: Tue, 22 Oct 2024 11:47:13 +0200 Subject: [PATCH 11/19] Update docs/how-to/h-deploy-azure.md Co-authored-by: Vladimir Izmalkov <48120135+izmalk@users.noreply.github.com> --- docs/how-to/h-deploy-azure.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/how-to/h-deploy-azure.md b/docs/how-to/h-deploy-azure.md index 72387284..74092be5 100644 --- a/docs/how-to/h-deploy-azure.md +++ b/docs/how-to/h-deploy-azure.md @@ -234,6 +234,7 @@ az role definition delete --name ``` Finally, logout Azure CLI user credentials to prevent any credential leakage: + ```shell az logout ``` From cc257c6e479ddec64d83c471ca0621094ae50e74 Mon Sep 17 00:00:00 2001 From: deusebio Date: Tue, 22 Oct 2024 11:48:25 +0200 Subject: [PATCH 12/19] Update docs/how-to/h-deploy-aws.md Co-authored-by: Vladimir Izmalkov <48120135+izmalk@users.noreply.github.com> --- docs/how-to/h-deploy-aws.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/how-to/h-deploy-aws.md b/docs/how-to/h-deploy-aws.md index a9cf61f4..f952fb22 100644 --- a/docs/how-to/h-deploy-aws.md +++ b/docs/how-to/h-deploy-aws.md @@ -69,6 +69,7 @@ Check AWS credentials: ## Bootstrap Juju controller on AWS EC2 Add AWS credentials to Juju: + ```shell juju add-credential aws -f ~/.aws/credentials.yaml ``` From bfa5499147290c112030833acd5625ef61ca43bd Mon Sep 17 00:00:00 2001 From: deusebio Date: Tue, 22 Oct 2024 12:00:52 +0200 Subject: [PATCH 13/19] Apply suggestions from code review Co-authored-by: Vladimir Izmalkov <48120135+izmalk@users.noreply.github.com> --- docs/how-to/h-deploy-aws.md | 35 ++++++++++++++++-------- docs/how-to/h-deploy-azure.md | 51 +++++++++++++++++------------------ 2 files changed, 48 insertions(+), 38 deletions(-) diff --git a/docs/how-to/h-deploy-aws.md b/docs/how-to/h-deploy-aws.md index f952fb22..7a64e3ef 100644 --- a/docs/how-to/h-deploy-aws.md +++ b/docs/how-to/h-deploy-aws.md @@ -73,10 +73,13 @@ Add AWS credentials to Juju: ```shell juju add-credential aws -f ~/.aws/credentials.yaml ``` + Bootstrap Juju controller ([check all supported configuration options](https://juju.is/docs/juju/amazon-ec2)): + ```shell juju bootstrap aws ``` + [details="Output example"] ```shell > juju bootstrap aws @@ -103,12 +106,13 @@ to create a new model to deploy workloads. ``` [/details] -You can check the [AWS EC2 instance availability](https://us-east-1.console.aws.amazon.com/ec2/home?region=us-east-1#Instances:instanceState=running) (ensure the right AWS region chosen!): +Check the [AWS EC2 instance availability](https://us-east-1.console.aws.amazon.com/ec2/home?region=us-east-1#Instances:instanceState=running) to ensure the right AWS region is chosen: ![image|690x118](upload://putAO5NyHdaeWE6jXI8X1hZHTYv.png) ## Deploy charms Create a new Juju model, if needed: + ```shell juju add-model ``` @@ -118,9 +122,11 @@ juju add-model > ``` Then, Charmed Kafka can be deployed as usual. However, note that the smallest instance types on Azure may not have enough resources for hosting -a Kafka broker. We therefore recommend you to select some types that provides at the very least 8GB of RAM and 4 cores, although for production use-case -we recommend you to use the guidance provided in the [requirement page](/t/charmed-kafka-reference-requirements/10563). You can find more information about -the available instance types in the [Azure web page](https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/overview). +a Kafka broker. We recommend selecting an instance type that provides at the very least `8` GB of RAM and `4` cores. +For more guidance on production environment sizing, see the [Requirements page](/t/charmed-kafka-reference-requirements/10563). +You can find more information about the available instance types in the [Azure documentation](https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/overview). + +Deploy and integrate Kafka and ZooKeepe, for example: ```shell juju deploy zookeeper -n3 --channel 3/stable @@ -142,15 +148,18 @@ And integrate it with the Kafka application: juju integrate kafka admin ``` -For more information on Data Integrator and how to use it, please refer to the [how-to manage applications](/t/charmed-kafka-how-to-manage-app/10285) user guide. +For more information on Data Integrator and how to use it, please refer to the [how-to manage applications](/t/charmed-kafka-how-to-manage-app/10285) guide. ## Clean up [note type="caution"] -Always clean AWS resources that are no longer necessary - they could be costly! +Always clean AWS resources that are no longer necessary! Abandoned resources are tricky to detect and they can become expensive over time. [/note] -To destroy the Juju controller and remove AWS instance (warning: all your data will be permanently removed): +To list all controllers use the `juju controllers` command. + +To destroy the Juju controller and remove AWS instance (**Warning**: all your data will be permanently deleted): + ```shell > juju controllers Controller Model User Access Cloud/Region Models Nodes HA Version @@ -159,10 +168,12 @@ aws-us-east-1* - admin superuser aws/us-east-1 1 1 none 3.5 > juju destroy-controller aws-us-east-1 --destroy-all-models --destroy-storage --force ``` -Next, check and manually delete all unnecessary AWS EC2 instances, to show the list of all your EC2 instances run the following command (make sure the correct region used!): +Next, check and manually delete all unnecessary AWS EC2 instances, to show the list of all your EC2 instances run the following command (make sure to use the correct region): + ```shell aws ec2 describe-instances --region us-east-1 --query "Reservations[].Instances[*].{InstanceType: InstanceType, InstanceId: InstanceId, State: State.Name}" --output table ``` + [details="Output example"] ```shell ------------------------------------------------------- @@ -178,9 +189,9 @@ aws ec2 describe-instances --region us-east-1 --query "Reservations[].Instances[ ``` [/details] -List your Juju credentials: +List your Juju credentials with the `juju credentials` command: + ```shell -> juju credentials ... Client Credentials: Cloud Credentials @@ -188,11 +199,13 @@ aws NAME_OF_YOUR_CREDENTIAL ... ``` Remove AWS EC2 CLI credentials from Juju: + ```shell -> juju remove-credential aws NAME_OF_YOUR_CREDENTIAL +juju remove-credential aws NAME_OF_YOUR_CREDENTIAL ``` Finally, remove AWS CLI user credentials (to avoid forgetting and leaking): + ```shell rm -f ~/.aws/credentials.yaml ``` diff --git a/docs/how-to/h-deploy-azure.md b/docs/how-to/h-deploy-azure.md index 74092be5..cde4e7c8 100644 --- a/docs/how-to/h-deploy-azure.md +++ b/docs/how-to/h-deploy-azure.md @@ -3,6 +3,7 @@ [Azure](https://azure.com/) is the cloud computing platform developed by Microsoft. It has management, access and development of applications and services to individuals, companies, and governments through its global infrastructure. Access the Azure web console at [portal.azure.com](https://portal.azure.com/). ## Summary + * [Install Azure and Juju tooling](#install-client-environment) * [Authenticate](#authenticate) * [Bootstrap Juju controller on Azure](#bootstrap-juju-controller-on-azure) @@ -41,15 +42,15 @@ which should show: ### Azure CLI -Follow the [user guide](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-linux?pivots=apt) for installing the Azure CLI on Linux distributions. +Install the Azure CLI on Linux distributions by following the [Azure CLI's installation guide](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-linux?pivots=apt). -Verify that it is correctly installed running the command below: +Verify that it is correctly installed: ```shell az --version ``` -which should show the following output: +The above command should produce output similar to the following: ```shell azure-cli 2.65.0 @@ -85,16 +86,10 @@ az login After providing the authentication via web browser, you will be shown a list of information and a table with the subscriptions connected to your user, e.g.: ```shell - -... - No Subscription name Subscription ID Tenant ----- ------------------------------------ ------------------------------------ ------------- [1] * canonical.com [2] canonical.com - -... - ``` In the prompt, select the subscription id you would like to connect the controller to, and store the id @@ -117,8 +112,7 @@ This will start a script that will help you set up the credentials, where you wi * `application_name`: please generate a random string to avoid collision with other users or applications * `role-definition-name`: please generate a random string to avoid collision with other users or applications, and store it as `` -After prompting all these information, you will be asked to authenticate the requests via web browser, as shown in the -outputs that will display +Next, you will be asked to authenticate the requests via your web browser with the following message: ```shell To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code to authenticate. @@ -128,13 +122,13 @@ In the browser, open the [authentication page](https://microsoft.com/devicelogin You will be asked to authenticate twice, for allowing the creation of two different resources in Azure. -If everything was fine, you will be confirmed that the credentials have been correctly added locally: +You will see a message that the credentials have been added locally: ```shell Credential added locally for cloud "azure". ``` -Once the credentials are correctly added, we can bootstrap a controller +Once the credentials are correctly added, we can bootstrap a controller: ```shell juju bootstrap azure @@ -143,6 +137,7 @@ juju bootstrap azure ## Deploy charms Create a new Juju model, if needed: + ```shell juju add-model ``` @@ -152,9 +147,11 @@ juju add-model > ``` Then, Charmed Kafka can be deployed as usual. However, note that the smallest instance types on Azure may not have enough resources for hosting -a Kafka broker. We therefore recommend you to select some types that provides at the very least 8GB of RAM and 4 cores, although for production use-case -we recommend you to use the guidance provided in the [requirement page](/t/charmed-kafka-reference-requirements/10563). You can find more information about -the available instance types in the [Azure web page](https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/overview). +a Kafka broker. We recommend selecting an instance type that provides at the very least `8` GB of RAM and `4` cores. +For more guidance on production environment sizing, see the [Requirements page](/t/charmed-kafka-reference-requirements/10563). +You can find more information about the available instance types in the [Azure documentation](https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/overview). + +Deploy and integrate Kafka and ZooKeepe, for example: ```shell juju deploy zookeeper -n3 --channel 3/stable @@ -176,42 +173,42 @@ And integrate it with the Kafka application: juju integrate kafka admin ``` -For more information on Data Integrator and how to use it, please refer to the [how-to manage applications](/t/charmed-kafka-how-to-manage-app/10285) user guide. +For more information on Data Integrator and how to use it, please refer to the [how-to manage applications](/t/charmed-kafka-how-to-manage-app/10285) guide. ## Clean up [note type="caution"] -Always clean Azure resources that are no longer necessary - they could be costly! +Always clean Azure resources that are no longer necessary! Abandoned resources are tricky to detect and they can become expensive over time. [/note] -To destroy the Juju controller and remove Azure instance (warning: all your data will be permanently removed): +To destroy the Juju controller and remove the Azure instance (Warning: all your data will be permanently removed): + ```shell juju destroy-controller --destroy-all-models --destroy-storage --force ``` > Use `juju list-controllers` to retrieve the names of the controllers that have been registered to your local client. -Next, check and manually delete all unnecessary Azure VM instances, to show the list of all your Azure VMs run the following command (make sure the correct region used!): +Next, check and manually delete all unnecessary Azure VM instances, to show the list of all your Azure VMs run the following command (make sure to use the correct region): ```shell az resource list ``` -List your Juju credentials: +List your Juju credentials with the `juju credentials` command: + ```shell -> juju credentials -... Client Credentials: Cloud Credentials azure NAME_OF_YOUR_CREDENTIAL -... ``` Remove Azure CLI credentials from Juju: + ```shell -> juju remove-credential azure NAME_OF_YOUR_CREDENTIAL +juju remove-credential azure NAME_OF_YOUR_CREDENTIAL ``` After deleting the credentials, the `interactive` process may still leave the role resource and its assignment hanging around. -We recommend you to check if these are still present by +We recommend checking if these are still present by running: ```shell az role definition list --name @@ -220,7 +217,7 @@ az role definition list --name To get the full list, use it without specifying the `--name` argument. You can check whether you still have a -role assignment bound to `` registered using +role assignment bound to `` registered by using: ```shell az role assignment list --role From 87115f2e1031fc367c99e6b3228c14fe18996b23 Mon Sep 17 00:00:00 2001 From: Enrico Deusebio Date: Tue, 22 Oct 2024 22:04:10 +0200 Subject: [PATCH 14/19] review --- docs/how-to/h-deploy-aws.md | 40 ++++++++++++++++------------------- docs/how-to/h-deploy-azure.md | 4 +++- 2 files changed, 21 insertions(+), 23 deletions(-) diff --git a/docs/how-to/h-deploy-aws.md b/docs/how-to/h-deploy-aws.md index 7a64e3ef..55e34c1e 100644 --- a/docs/how-to/h-deploy-aws.md +++ b/docs/how-to/h-deploy-aws.md @@ -20,8 +20,7 @@ Install Juju via snap: sudo snap install juju ``` -Follow the installation guides for: -* [AWs CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) - the Amazon Web Services CLI +Follow the [AWS documentation](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) for guidance on how to install the Amazon Web Services CLI. To check whether both Juju and AWS CLI are correctly installed, run commands to display their versions: @@ -35,7 +34,8 @@ aws-cli/2.13.25 Python/3.11.5 Linux/6.2.0-33-generic exe/x86_64.ubuntu.23 prompt ### Authenticate -[Create an IAM account](https://docs.aws.amazon.com/eks/latest/userguide/getting-started-console.html) (or use legacy access keys) to operate AWS EC2: +[Create an IAM account](https://docs.aws.amazon.com/eks/latest/userguide/getting-started-console.html) or use legacy user access keys and secret key to operate AWS EC2: + ```shell mkdir -p ~/.aws && cat <<- EOF > ~/.aws/credentials.yaml credentials: @@ -106,9 +106,6 @@ to create a new model to deploy workloads. ``` [/details] -Check the [AWS EC2 instance availability](https://us-east-1.console.aws.amazon.com/ec2/home?region=us-east-1#Instances:instanceState=running) to ensure the right AWS region is chosen: -![image|690x118](upload://putAO5NyHdaeWE6jXI8X1hZHTYv.png) - ## Deploy charms Create a new Juju model, if needed: @@ -121,23 +118,24 @@ juju add-model > juju model-config logging-config='=INFO;unit=DEBUG' > ``` -Then, Charmed Kafka can be deployed as usual. However, note that the smallest instance types on Azure may not have enough resources for hosting -a Kafka broker. We recommend selecting an instance type that provides at the very least `8` GB of RAM and `4` cores. +Then, Charmed Kafka can be deployed as usual. However, note that the smallest instance types on AWS may not have enough resources for hosting +a Kafka broker. We recommend selecting an instance type that provides at the very least 8 GB of RAM and 4 cores. For more guidance on production environment sizing, see the [Requirements page](/t/charmed-kafka-reference-requirements/10563). -You can find more information about the available instance types in the [Azure documentation](https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/overview). +You can find more information about the available instance types in the [AWS documentation](https://us-east-1.console.aws.amazon.com/ec2/home?region=us-east-1#Instances:instanceState=running), +ensuring that you select the correct AWS. -Deploy and integrate Kafka and ZooKeepe, for example: +Deploy and integrate Kafka and ZooKeeper, for example: ```shell juju deploy zookeeper -n3 --channel 3/stable -juju deploy kafka -n3 --constraints "instance-type=Standard_A4_v2" --channel 3/stable +juju deploy kafka -n3 --constraints "instance-type=m7i.xlarge" --channel 3/stable juju integrate kafka zookeeper ``` We also recommend to deploy a [Data Integrator](https://charmhub.io/data-integrator) for creating an admin user to manage the content of the Kafka cluster: ```shell -juju deploy data-integrator admin --channel edge \ +juju deploy data-integrator admin \ --config extra-user-roles=admin \ --config topic-name=admin-topic ``` @@ -161,14 +159,12 @@ To list all controllers use the `juju controllers` command. To destroy the Juju controller and remove AWS instance (**Warning**: all your data will be permanently deleted): ```shell -> juju controllers -Controller Model User Access Cloud/Region Models Nodes HA Version -aws-us-east-1* - admin superuser aws/us-east-1 1 1 none 3.5.4 - -> juju destroy-controller aws-us-east-1 --destroy-all-models --destroy-storage --force +juju destroy-controller --destroy-all-models --destroy-storage --force ``` -Next, check and manually delete all unnecessary AWS EC2 instances, to show the list of all your EC2 instances run the following command (make sure to use the correct region): +Use `juju list-controllers` to list the available controllers the client has registered to. + +After destroying the controller, check and manually delete all unnecessary AWS EC2 instances, to show the list of all your EC2 instances run the following command (make sure to use the correct region): ```shell aws ec2 describe-instances --region us-east-1 --query "Reservations[].Instances[*].{InstanceType: InstanceType, InstanceId: InstanceId, State: State.Name}" --output table @@ -181,10 +177,10 @@ aws ec2 describe-instances --region us-east-1 --query "Reservations[].Instances[ +---------------------+----------------+--------------+ | InstanceId | InstanceType | State | +---------------------+----------------+--------------+ -| i-0f374435695ffc54c| m7i.large | terminated | -| i-0e1e8279f6b2a08e0| m7i.large | terminated | -| i-061e0d10d36c8cffe| m7i.large | terminated | -| i-0f4615983d113166d| m7i.large | terminated | +| i-0f374435695ffc54c| m7i.xlarge | terminated | +| i-0e1e8279f6b2a08e0| m7i.xlarge | terminated | +| i-061e0d10d36c8cffe| m7i.xlarge | terminated | +| i-0f4615983d113166d| m7i.xlarge | terminated | +---------------------+----------------+--------------+ ``` [/details] diff --git a/docs/how-to/h-deploy-azure.md b/docs/how-to/h-deploy-azure.md index cde4e7c8..98b593dd 100644 --- a/docs/how-to/h-deploy-azure.md +++ b/docs/how-to/h-deploy-azure.md @@ -141,6 +141,7 @@ Create a new Juju model, if needed: ```shell juju add-model ``` + > (Optional) Increase the debug level if you are troubleshooting charms: > ```shell > juju model-config logging-config='=INFO;unit=DEBUG' @@ -151,7 +152,7 @@ a Kafka broker. We recommend selecting an instance type that provides at the ver For more guidance on production environment sizing, see the [Requirements page](/t/charmed-kafka-reference-requirements/10563). You can find more information about the available instance types in the [Azure documentation](https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/overview). -Deploy and integrate Kafka and ZooKeepe, for example: +Deploy and integrate Kafka and ZooKeeper, for example: ```shell juju deploy zookeeper -n3 --channel 3/stable @@ -201,6 +202,7 @@ Client Credentials: Cloud Credentials azure NAME_OF_YOUR_CREDENTIAL ``` + Remove Azure CLI credentials from Juju: ```shell From a569e9d578dc725e3504e58574027eb2518fcb42 Mon Sep 17 00:00:00 2001 From: deusebio Date: Wed, 20 Nov 2024 22:28:59 +0100 Subject: [PATCH 15/19] Update docs/how-to/h-deploy-azure.md Co-authored-by: Vladimir Izmalkov <48120135+izmalk@users.noreply.github.com> --- docs/how-to/h-deploy-azure.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/how-to/h-deploy-azure.md b/docs/how-to/h-deploy-azure.md index 98b593dd..c37e7460 100644 --- a/docs/how-to/h-deploy-azure.md +++ b/docs/how-to/h-deploy-azure.md @@ -105,12 +105,12 @@ juju add-credentials azure This will start a script that will help you set up the credentials, where you will be asked: -* `credential-name`: fill it with a sensible name that will help you identify the credential set, say `` -* `region`: select any default region is more convenient for you to deploy your controller and applications. Note that credentials are nevertheless not region specific -* `auth type`: select `interactive`, which is the recommended way to authenticate to Azure using Juju -* `subscription_id`: use the value `` taken in the previous step -* `application_name`: please generate a random string to avoid collision with other users or applications -* `role-definition-name`: please generate a random string to avoid collision with other users or applications, and store it as `` +* `credential-name` — a sensible name that will help you identify the credential set, say `` +* `region` — a default region that is most convenient to deploy your controller and applications. Note that credentials are not region-specific +* `auth type` — authentication type. Select `interactive`, which is the recommended way to authenticate to Azure using Juju +* `subscription_id` — the value `` taken in the previous step +* `application_name` — any unique string to avoid collision with other users or applications +* `role-definition-name` — any unique string to avoid collision with other users or applications, and store it as `` Next, you will be asked to authenticate the requests via your web browser with the following message: From 2ff0782681da0c51b8449ca34dad30bd507f9d5f Mon Sep 17 00:00:00 2001 From: Enrico Deusebio Date: Wed, 20 Nov 2024 22:29:42 +0100 Subject: [PATCH 16/19] adding admonitions --- docs/how-to/h-deploy-aws.md | 11 +++++++---- docs/how-to/h-deploy-azure.md | 10 ++++++---- 2 files changed, 13 insertions(+), 8 deletions(-) diff --git a/docs/how-to/h-deploy-aws.md b/docs/how-to/h-deploy-aws.md index 55e34c1e..9e55bdc4 100644 --- a/docs/how-to/h-deploy-aws.md +++ b/docs/how-to/h-deploy-aws.md @@ -113,10 +113,13 @@ Create a new Juju model, if needed: ```shell juju add-model ``` -> (Optional) Increase the debug level if you are troubleshooting charms: -> ```shell -> juju model-config logging-config='=INFO;unit=DEBUG' -> ``` + +[note type="caution"] +(Optional) Increase the debug level if you are troubleshooting charms: +```shell +juju model-config logging-config='=INFO;unit=DEBUG' +``` +[/note] Then, Charmed Kafka can be deployed as usual. However, note that the smallest instance types on AWS may not have enough resources for hosting a Kafka broker. We recommend selecting an instance type that provides at the very least 8 GB of RAM and 4 cores. diff --git a/docs/how-to/h-deploy-azure.md b/docs/how-to/h-deploy-azure.md index c37e7460..118df58e 100644 --- a/docs/how-to/h-deploy-azure.md +++ b/docs/how-to/h-deploy-azure.md @@ -142,10 +142,12 @@ Create a new Juju model, if needed: juju add-model ``` -> (Optional) Increase the debug level if you are troubleshooting charms: -> ```shell -> juju model-config logging-config='=INFO;unit=DEBUG' -> ``` +[note type="caution"] +(Optional) Increase the debug level if you are troubleshooting charms: +```shell +juju model-config logging-config='=INFO;unit=DEBUG' +``` +[/note] Then, Charmed Kafka can be deployed as usual. However, note that the smallest instance types on Azure may not have enough resources for hosting a Kafka broker. We recommend selecting an instance type that provides at the very least `8` GB of RAM and `4` cores. From 49f3dbd5fe5b09dcca1000f62d6bc91bd5bfa1a2 Mon Sep 17 00:00:00 2001 From: Enrico Deusebio Date: Fri, 22 Nov 2024 15:43:21 +0100 Subject: [PATCH 17/19] code review --- docs/how-to/h-deploy-aws.md | 47 +++++++++++++---------------------- docs/how-to/h-deploy-azure.md | 24 +++++++++++------- 2 files changed, 32 insertions(+), 39 deletions(-) diff --git a/docs/how-to/h-deploy-aws.md b/docs/how-to/h-deploy-aws.md index 9e55bdc4..b1c1163f 100644 --- a/docs/how-to/h-deploy-aws.md +++ b/docs/how-to/h-deploy-aws.md @@ -17,7 +17,7 @@ Install Juju via snap: ```shell -sudo snap install juju +sudo snap install juju --channel 3.5/stable ``` Follow the [AWS documentation](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) for guidance on how to install the Amazon Web Services CLI. @@ -47,25 +47,6 @@ credentials: EOF ``` - - ## Bootstrap Juju controller on AWS EC2 Add AWS credentials to Juju: @@ -121,20 +102,22 @@ juju model-config logging-config='=INFO;unit=DEBUG' ``` [/note] -Then, Charmed Kafka can be deployed as usual. However, note that the smallest instance types on AWS may not have enough resources for hosting -a Kafka broker. We recommend selecting an instance type that provides at the very least 8 GB of RAM and 4 cores. -For more guidance on production environment sizing, see the [Requirements page](/t/charmed-kafka-reference-requirements/10563). -You can find more information about the available instance types in the [AWS documentation](https://us-east-1.console.aws.amazon.com/ec2/home?region=us-east-1#Instances:instanceState=running), -ensuring that you select the correct AWS. - -Deploy and integrate Kafka and ZooKeeper, for example: +Deploy and integrate Kafka and ZooKeeper: ```shell -juju deploy zookeeper -n3 --channel 3/stable -juju deploy kafka -n3 --constraints "instance-type=m7i.xlarge" --channel 3/stable +juju deploy zookeeper -n3 --channel 3/stable [--constraints "instance-type="] +juju deploy kafka -n3 --channel 3/stable [--constraints "instance-type="] juju integrate kafka zookeeper ``` +[note type="caution"] +Note that the smallest instance types on AWS may not have enough resources for hosting +a Kafka broker. We recommend selecting an instance type that provides at the very least 8 GB of RAM and 4 cores, e.g. `m7i.xlarge`. +For more guidance on production environment sizing, see the [Requirements page](/t/charmed-kafka-reference-requirements/10563). +You can find more information about the available instance types in the [AWS documentation](https://us-east-1.console.aws.amazon.com/ec2/home?region=us-east-1#Instances:instanceState=running), +ensuring that you select the correct AWS. +[/note] + We also recommend to deploy a [Data Integrator](https://charmhub.io/data-integrator) for creating an admin user to manage the content of the Kafka cluster: ```shell @@ -165,7 +148,11 @@ To destroy the Juju controller and remove AWS instance (**Warning**: all your da juju destroy-controller --destroy-all-models --destroy-storage --force ``` -Use `juju list-controllers` to list the available controllers the client has registered to. +> Use `juju list-controllers` to retrieve the names of the controllers that have been registered to your local client. + +Should the destroying process take long time or being seemingly stuck, proceed to delete EC2 resources also manually +via the AWS portal. See [Amazon AWS documentation](https://repost.aws/knowledge-center/terminate-resources-account-closure) for more information +on how to remove active resources no longer needed. After destroying the controller, check and manually delete all unnecessary AWS EC2 instances, to show the list of all your EC2 instances run the following command (make sure to use the correct region): diff --git a/docs/how-to/h-deploy-azure.md b/docs/how-to/h-deploy-azure.md index 118df58e..688730a6 100644 --- a/docs/how-to/h-deploy-azure.md +++ b/docs/how-to/h-deploy-azure.md @@ -83,7 +83,7 @@ First of all, retrieve your subscription-id az login ``` -After providing the authentication via web browser, you will be shown a list of information and a table with the subscriptions connected to your user, e.g.: +After authenticating via the web browser, you will be shown a list of information and a table with the subscriptions connected to your user, e.g.: ```shell No Subscription name Subscription ID Tenant @@ -149,19 +149,21 @@ juju model-config logging-config='=INFO;unit=DEBUG' ``` [/note] -Then, Charmed Kafka can be deployed as usual. However, note that the smallest instance types on Azure may not have enough resources for hosting -a Kafka broker. We recommend selecting an instance type that provides at the very least `8` GB of RAM and `4` cores. -For more guidance on production environment sizing, see the [Requirements page](/t/charmed-kafka-reference-requirements/10563). -You can find more information about the available instance types in the [Azure documentation](https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/overview). - -Deploy and integrate Kafka and ZooKeeper, for example: +Deploy and integrate Kafka and ZooKeeper: ```shell -juju deploy zookeeper -n3 --channel 3/stable -juju deploy kafka -n3 --constraints "instance-type=Standard_A4_v2" --channel 3/stable +juju deploy zookeeper -n3 --channel 3/stable [--constraints "instance-type="] +juju deploy kafka -n3 --channel 3/stable [--constraints "instance-type="] juju integrate kafka zookeeper ``` +[note type="caution"] +Note that the smallest instance types on Azure may not have enough resources for hosting +a Kafka broker. We recommend selecting an instance type that provides at the very least `8` GB of RAM and `4` cores, e.g. `Standard_A4_v2`. +For more guidance on production environment sizing, see the [Requirements page](/t/charmed-kafka-reference-requirements/10563). +You can find more information about the available instance types in the [Azure documentation](https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/overview). +[/note] + We also recommend to deploy a [Data Integrator](https://charmhub.io/data-integrator) for creating an admin user to manage the content of the Kafka cluster: ```shell @@ -192,6 +194,10 @@ juju destroy-controller --destroy-all-models --destroy-storage > Use `juju list-controllers` to retrieve the names of the controllers that have been registered to your local client. +Should the destroying process take long time or being seemingly stuck, proceed to delete VM resources also manually +via the Azure portal. See [Azure documentation](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/manage-resources-portal) for more information +on how to remove active resources no longer needed. + Next, check and manually delete all unnecessary Azure VM instances, to show the list of all your Azure VMs run the following command (make sure to use the correct region): ```shell az resource list From 37dcc2f706af39a86af8cf14b415f91bc483f5df Mon Sep 17 00:00:00 2001 From: deusebio Date: Wed, 27 Nov 2024 22:58:58 +0100 Subject: [PATCH 18/19] Apply suggestions from code review Co-authored-by: Vladimir Izmalkov <48120135+izmalk@users.noreply.github.com> --- docs/how-to/h-deploy-aws.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/docs/how-to/h-deploy-aws.md b/docs/how-to/h-deploy-aws.md index b1c1163f..e285763d 100644 --- a/docs/how-to/h-deploy-aws.md +++ b/docs/how-to/h-deploy-aws.md @@ -111,11 +111,9 @@ juju integrate kafka zookeeper ``` [note type="caution"] -Note that the smallest instance types on AWS may not have enough resources for hosting -a Kafka broker. We recommend selecting an instance type that provides at the very least 8 GB of RAM and 4 cores, e.g. `m7i.xlarge`. -For more guidance on production environment sizing, see the [Requirements page](/t/charmed-kafka-reference-requirements/10563). -You can find more information about the available instance types in the [AWS documentation](https://us-east-1.console.aws.amazon.com/ec2/home?region=us-east-1#Instances:instanceState=running), -ensuring that you select the correct AWS. +The smallest AWS instance types may not provide sufficient resources to host a Kafka broker. We recommend choosing an instance type with a minimum of `8` GB of RAM and `4` CPU cores, such as `m7i.xlarge`. + +For more guidance on sizing production environments, see the [Requirements page](/t/charmed-kafka-reference-requirements/10563). Additional information about AWS instance types is available in the [AWS documentation](https://us-east-1.console.aws.amazon.com/ec2/home?region=us-east-1#Instances:instanceState=running). [/note] We also recommend to deploy a [Data Integrator](https://charmhub.io/data-integrator) for creating an admin user to manage the content of the Kafka cluster: From ebe93fca97da7f956b12a65e964c6e2e0baec4e3 Mon Sep 17 00:00:00 2001 From: Enrico Deusebio Date: Wed, 27 Nov 2024 23:02:09 +0100 Subject: [PATCH 19/19] final fixes --- docs/how-to/h-deploy-azure.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/docs/how-to/h-deploy-azure.md b/docs/how-to/h-deploy-azure.md index 688730a6..93473538 100644 --- a/docs/how-to/h-deploy-azure.md +++ b/docs/how-to/h-deploy-azure.md @@ -14,11 +14,14 @@ ## Install Client Environment -> **WARNING**: The current limitations: -> * Only supported starting Juju 3.6 (currently edge) -> * Juju cli should be on Azure VM for it to be able to reach cloud metadata endpoint. -> * Managed Identity and the Juju resources should be on the same Azure subscription -> * The current setup has been tested on Ubuntu 22.04+ + +[note type="caution"] +**WARNING**: The current limitations: +* Only supported starting Juju 3.6 (currently edge) +* Juju cli should be on Azure VM for it to be able to reach cloud metadata endpoint. +* Managed Identity and the Juju resources should be on the same Azure subscription +* The current setup has been tested on Ubuntu 22.04+ +[/note] ### Juju