Commvault Cloud - Cloudburst Recovery for AWS Solution

Example Infrastructure as Code (IaC) solution using AWS CloudFormation, HashiCorp Terraform to deploy on-demand and Amazon EC2 instances Spot to perform a massively parallel recovery using Commvault Backup & Recovery.

📚 Background

Commvault Backup & Recovery is the industry leading data protection and cyber-resilience solution for protecting AWS services, SaaS, and other hybrid workloads. You can use Commvault Backup & Recovery to secure, defend, and recover your business applications and data (learn more at www.commvault.com.

This example solution show-cases the elasticity of Amazon EC2, the scalability of Amazon S3, and the power of the Commvault Backup & Recovery data platform to achieve a massivelly parallel restore with on-demand ephemeral resources deployed using Infrastructure as Code.

A part of any modern data management plan includes an understanding of your business recovery time objectives and in extreme circumstances (i.e., where multiple applications are affected), the elastic nature of AWS compute allows a rapid and cost effective rapid recovery solution for many systems in parallel.

🛠️ Setup

At a high-level the process for setup and execution this solution (depicted above) is as follows:

1. Deploy Commvault Backup & Recovery BYOL from the AWS Marketplace using AWS CloudFormation.
2. Review the CommvaultBackupAndRecovery AWS IAM role and policies that allow Commavult to backup and recovery your AWS workloads.
3. Complete the Core Setup Wizard and run a backup of your EC2 instances and/or other supported workloads to Amazon S3.
4. Deploy 100 x Commvault Cloud Access Node ARM BYOL with CloudFormation template and configure.
5. Run a massively parallel restore.

Note

There are HashiCorp Terraform examples in the terraform folder to quickly deploy on-demand or Amazon EC2 Spot Instances, complete with random data generation on first boot.

Note

100 instances are used only as an example to demonstrate the massively parallel nature of Commvault Backup & Recovery, Amazon S3, and Amazon EC2 compute. Commvault recommends performing regular testing and GameDays in your IT, Security, and Application teams to find the right mix of recovery speed (throughput) and overall recovery cost.

The key benefit of this solution is that the recovery resources are ephemeral and may be destroyed immediately after the restore is complete.

💲Note: The Commvault Backup & Recovery BYOL product comes with a free 30-day trial so you can try this out yourself. You will incur additional costs for the AWS services you utilize during your test. Consult the AWS pricing pages for more details.

🛠️ Setup - Step 1 - Deploy Commvault Backup & Recovery from the AWS Marketplace

This step deploys Commvault Backup & Recovery as a single Amazon EC2 instance running Microsoft Windows. If you already have a Commvault Backup & Recovery deployment, you may skip this step.

1. Login to AWS Console as a user that can deploy new Amazon EC2, Amazon S3. and AWS IAM resources using Amazon CloudFormation.
2. Open new browser tab to AWS Marketplace and search for Commvault.
3. Click Commvault Backup & Recovery BYOL.
4. Select CloudFormation Template Fulfilment Option and supply requested information.
5. Select the I acknowledge that AWS CloudFormation might create IAM resources with custom names. checkbox.
6. Click Submit to deploy.

🛠️ Setup - Step 2 - Review the created AWS IAM role and policies that allow Commavult to backup and recovery your AWS workloads.

Commvault will create a single AWS IAM Role called CommvaultBackupAndRecovery.

Detailed information of the AWS IAM Policies required by Commvault (per AWS workload) may be viewed here.

🛠️ Setup - Step 3 - Complete the Core Setup Wizard and run a backup.

Creating a Commvault admin account

1. Obtain your Administrator password for your newly created Commvault Backup & Recovery instance.
2. Login using Remote Desktop Protocol (RDP)
3. Wait for the ******* Starting Commserve image customization ******** powershell first-boot configuration script to complete.
4. A browser will open to complete the remaining setup.
5. Provide the Email address that will be associated with the Commvault admin user (break glass account).
6. Provide the Password that will be associated with the Commvault admin user.
7. Click Create account.
8. You will be greeted with Command Center login screen. Login with newly created admin user and password.
9. Click OK to accept the License and Registration warning.

[!NOTE] Commvault recommends using Amazon EC2 Instance Connect for secure access to your Commvault instance without the need to expose public IP addresses, or manage bastion hosts.

[!WARNING] If you are using a trial license the Cloud Storage license will be constrained to a maximum of ten (10) concurrent MediaAgents during the restore. If you have a paid Commvault license, you can submit a request to extend your Cloud Storage license to match your required parallism (i.e., the total number of Access Nodes you will have active in your Commvault environment).

Completing Commvault Core Setup

Next, you just need to tell Commvault how often you want to run backups, and where to store your backups (i.e., Amazon S3).

1. Click Let’s get started.
2. Click Cloud in the Add storage page.
3. Provide a Cloud library Name (i.e., Amazon S3-IA - Backups us-east-1).
4. Select Amazon S3 as the Cloud storage Type.
5. Set the Service host to s3.us-east-1.amazonaws.com (your Region may differ).
6. Select IAM role for the credentials.
7. Enter the **bucket name** created during your AWS CloudFormation deployment.

[!NOTE] You can find the bucket name in the AWS CloudFormation Console, in your stack, on the Outputs tab, as CvltCloudLibraryBucketName

9. Leave Storage class, as the default S3 Standard-Infrequent Access (S3 Standard-IA).
10. Enter the Deduplication DB location, use the volume pre-setup, pre-formatted with correct block-size (i.e., H:\Amqzon-S3-IA-DDB)
11. Click Save to accept defaults for your server plan.
12. You must be running the most current Platform Release 2023e Maintenance Release. Upgrading your Commvault software to the latest Maintenance Release before moving to the next step.

Deploying some test Amazon EC2 instances

If you are simply testing this solution, navigate to the terraform file where you can find an example to deploy one hundred (100) on-demand Amazon EC2 test hosts.

Run an initial backup

You will need an initial backup of your protected workloads before you can run a massively parallel restore. Assuming you used the terraform example above, perform the following to configure and run a backup.

1. Add your AWS account to Commvault Backup & Recovery.
2. Create an Amazon EC2 group to auto-discover EC2 instances and protect them.
3. Run a backup

🛠️ Setup - Step 4 - Deploy 100 x Cloud Access Nodes and configure

OK, it's time to setup your set of parallel Cloud Access Nodes, Commvault uses Cloud Access Nodes to perform backup, replication, restores. Commvault recommends AWS Graviton based Access Nodes for best price-performance and so you can meet your Shared Sustainabiltiy Responsibility in AWS.

You can acclerate recovery time by increasing the number of Access Nodes used, allowing more parallel recovery activities to run at the same time. Not only does this increase business agility, it also saves cost as you are only paying for what you use (during the restore).

1. Navigate to Manage > CommCell and enable Requires authcode for installation toggle.
2. Click the authcode and save it somewhere safe, you will need it next.
3. Download the 100 x Amazon EC2 Cloud Access Nodes template.yml and update to match your environment. You will find instructions at the top of the template for performing updates

#
# INSTRUCTIONS: 
# - Ensure AWS Account where instances are deployed has an AWS IAM role called 'CommvaultBackupAndRecovery' (this role is created during deployment of the 'Commvault Backup & Recovery BYOL' product in AWS Marketplace).
# - Update all occurrences of the KeyName parameter to a Key pair Name in your AWS account.
# - Update all occurrences of the SubnetId parameter with the Subnet ID that your 'Commvault Backup & Recovery' instance resides within - this allows use of Commvault HotAdd recovery
# - Update all occurrences of the GroupSet parameter with a Security group ID that allows incoming TCP (8400, 8403) and ICMP (PING) from the 'Commbvault Backup & Recovery' instance.
# - Optionally update the ImageId, InstanceType
# 
# INSTRUCTIONS to auto-register instances with your Commvault Backup & Recovery instance
# - Replace all occurrences of the -CSHost parameter (see UserData section) to include the IPv4 address of your 'Commvault Backup & Recovery' instance.
# - Replace all occurrences of the -CSName parameter (see UserData section) to match the fully-qualified hostname returned by `nslookup <YOUR-CS-HOST>` (or your CommServe friendly name as shown in Commvault Command Center home screen).
# - Obtain the authcode for your CommServe and replace all occurrences of -authcode parameter (see UserData section).
#

4. Create and launch a new AWS CloudFormation Stack with your customized template.yml using either the console or the AWS CLI.
5. Navigate to Manage > Server groups and click Add server group
6. Provide a freeform text Name for your group. This group will contain all of the Access Nodes you just deployed using CloudFormation.
7. Select automatic association and add a rule with the following settings, then click Save

Client Scope - Clients in this CommCell
Rule: Package Installed == Virtual Server
Rule: OS Version contains 'Amazon Linux'
Rule: Power State == ON

9.Select the Server group and select Actions > Upgrade software to upgrade the Access Nodes to the latest software release. 10. Navigate to your previously configured Amazon EC2 group and add your group for the access nodes. 11. Add a new entity setting called nStartAgentThreads with the values specified below, and click Save.

Setting	Value
Name:	nStartAgentThreads
Entity:	Select your previously created server group
Category:	VirtualServer
Type:	Integer
Value:	100

12. Add a new entity-based additional setting called MaxRestoreStreams and set to 100.

Setting	Value
Name:	MaxRestoreStreams
Entity:	Select your previously created server group
Category:	VirtualServer
Type:	Integer
Value:	100

13. Open Commvault CommCell Console and noativate to the Cloud library where your backups are stored, expand the libarry right-click Mount-Path.

• Select Share Mount Path
• Click the Share button and add each new Access Node with an Access Mode of Read
• Click Save

[!NOTE] You can use Amazon EC2 Spot Instances for your Cloud Access Nodes, but if the instance is reclaimed during the recovery, the restore will fail for any instance(s) being restored by the reclaimed instance. See the Cloudformation template.yml to deploy your Access Nodes from the Spot Market.

🏃Run - Step 5 - Run a restore

Simply [run a restore](https://documentation.commvault.com/2023e/essential/87257_restoring_full_amazon_ec2_instance_in_place.html) from **Commvault Commvault Center**

⏩ Recovery results

Commvault lab testing was performed using the following setup:

• 1 x Commvault Backup & Recovery / Amazon EC2 instance (M6a.2xlage) (8 vCPU, 32GiB RAM)
• 100 x Commvault Cloud Access Nodes / Amazon EC2 instances (C6g.large (2 vCPU, 4GiB) AWS Graviton ⭐⭐⭐
• 100 x Amazon EC2 test instances totalling 1TiB of randomly generated data.

Cleanup

Don't forget to cleanup (terminate) your 100 x Commvault Cloud Access Node ARM BYOL instances using Amazon CloudFormation.

Once terminated you can remove the MediaAgents from your Commvault Backup & Recovery system using the supplied powershell scripts.

⚖️ License

This code is offered and licensed under the Apache 2.0 license.

❓Getting Support

If you need assistance with your Commvault Backup & Recovery software you can Join in at the commuunity, Check the docs, or Log a support call.

🤝 Contributing

Although we're extremely excited to receive contributions from the community, we're still working on the best mechanism to take in examples from external sources. Please bear with us in the short-term if pull requests take longer than expected or are closed. Please read our contributing guidelines if you'd like to open an issue or submit a pull request.