Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add gcp provider support #141

Merged
merged 7 commits into from
Dec 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 90 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,14 +28,41 @@ To facilite the usage of `crc-cloud`, a [container image](https://quay.io/reposi

### Authetication

All operations require to set the authentication mechanism in place. As so any `aws` authentication mechanism is supported by `crc-cloud`:
All operations require to set the authentication mechanism in place.

As so any `aws` authentication mechanism is supported by `crc-cloud`:

- long term credentials `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` as environment variables
- short lived credentials (in addition to `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` would require `AWS_SESSION_TOKEN`)
- credentials on config file (default file ~/.aws/config), in case of multiple profiles it will also accepts `AWS_PROFILE`

As so any `gcp` authentication mechanism is supported by `crc-cloud`:

- Credentials `GOOGLE_APPLICATION_CREDENTIALS` as environment variable
- Project ID `GCLOUD_PROJECT` as environment variable
- Region `GCLOUD_REGION` as environment variable
- Zone `GCLOUD_ZONE` as environment variable

### Restrictions

**Note**: `import` operation is not supported on `gcp` provider. As of now please use following manual steps to import the image on `gcp`:
```bash
# Download the required bundle from https://mirror.openshift.com/pub/openshift-v4/x86_64/clients/crc/bundles/openshift/
curl -L -O https://mirror.openshift.com/pub/openshift-v4/clients/crc/bundles/openshift/4.14.3/crc_libvirt_4.14.3_amd64.crcbundle
# Extract the bundle
tar -xvf crc_libvirt_4.14.3_amd64.crcbundle && cd crc_libvirt_4.14.3_amd64
# Convert the qcow2 image to raw
qemu-img convert crc.qcow2 disk.raw
# Compress the raw image to tar.gz
tar --format=oldgnu -Sczf /tmp/crc.tar.gz disk.raw
# Upload the tar.gz to GCS
gsutil cp /tmp/crc.tar.gz gs://crc-bundle-github-ci
# Create the image on GCP
gcloud compute images create crc --source-uri gs://crc-bundle-github-ci/crc.tar.gz
# List the images and check for crc one
gcloud compute images list --no-standard-images | grep crc
```

The `import` operation downloads and transform the bundle offered by crc into an image supported by `AWS`, as so there are some disk demanding operation. So there is a requirement of at least 70G free on disk to run this operation.

The AWS instance type of choice is *c6a.2xlarge* with 8vcpu and 16 GB of RAM. This will be customizable in the future, for the moment this fixed type imposes some [restrictions](https://aws.amazon.com/about-aws/whats-new/2022/12/amazon-ec2-m6a-c6a-instances-additional-regions/) on available regions to run crc cloud, those regions are:
Expand Down Expand Up @@ -98,7 +125,7 @@ podman run -d --rm \

`create` operation is responsible for create all required resources on the cloud provider to spin the Openshift Single Node Cluster.

Usage:
Usage: In case of `aws` provider

```bash
create crc cloud instance on AWS
Expand All @@ -121,6 +148,29 @@ Global Flags:
--pullsecret-filepath string path for pullsecret file
```

Usage: In case of `gcp` provider

```bash
create crc cloud instance on GCP

Usage:
crc-cloud create gcp [flags]

Flags:
--gcp-disk-size string Disk size in GB for the machine running the cluster. Default is 100.
--gcp-image-id string GCP image identifier
--gcp-instance-type string Instance type for the machine running the cluster. Default is n1-standard-8.
-h, --help help for gcp

Global Flags:
--backed-url string backed for stack state. Can be a local path with format file:///path/subpath or s3 s3://existing-bucket
--key-filepath string path to init key obtained when importing the image
--output string path to export assets
--project-name string project name to identify the instance of the stack
--pullsecret-filepath string path for pullsecret file
--tags stringToString tags to add on each resource (--tags name1=value1,name2=value2) (default [])
```

Outputs:

- `kubeconfig` file with the kube config to connect withint the cluster
Expand All @@ -129,7 +179,7 @@ Outputs:
- `id_rsa` key to connect the remote host
- `password` password generated for `kubeadmin` and `developer` default cluster users

Sample
Sample for `aws` provider:

```bash
podman run -d --rm \
Expand All @@ -149,6 +199,27 @@ podman run -d --rm \
--key-filepath "/workspace/id_ecdsa"
```

Sample for `gcp` provider:

```bash
podman run -d --rm \
-v ${PWD}:/workspace:z \
-e GOOGLE_APPLICATION_CREDENTIALS=${gcp_credentials} \
-e GCLOUD_PROJECT=${gcp_project_id} \
-e GCLOUD_REGION=${gcp_region} \
-e GCLOUD_ZONE=${gcp_zone} \
quay.io/crcont/crc-cloud:v0.0.2 create gcp \
--project-name "crc-ocp412" \
--backed-url "file:///workspace" \
--output "/workspace" \
--tags account=qe-pt,profile=builder \
--gcp-image-id "gcp-xxxx" \
--gcp-instance-type "n1-standard-8" \
--gcp-disk-size "100" \
--pullsecret-filepath "/workspace/pullsecret" \
--key-filepath "/workspace/id_ecdsa"
```

#### Destroy

`destroy` operation will remove any resource created at the cloud provider, it uses the files holding the state of the infrastructure which has been store at location defined by parameter `backed-url` on `create` operation.
Expand All @@ -168,7 +239,7 @@ Flags:
--provider string target cloud provider
```

Sample
Sample for `aws` provider:

```bash
podman run -d --rm \
Expand All @@ -181,3 +252,18 @@ podman run -d --rm \
--backed-url "file:///workspace" \
--provider "aws"
```

Sample for `gcp` provider:

```bash
podman run -d --rm \
-v ${PWD}:/workspace:z \
-e GOOGLE_APPLICATION_CREDENTIALS=${gcp_credentials} \
-e GCLOUD_PROJECT=${gcp_project_id} \
-e GCLOUD_REGION=${gcp_region} \
-e GCLOUD_ZONE=${gcp_zone} \
quay.io/crcont/crc-cloud:v0.0.2 destroy \
--project-name "crc-ocp412" \
--backed-url "file:///workspace" \
--provider "gcp"
```
4 changes: 2 additions & 2 deletions cmd/cmd/create/aws.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ func getAWSProviderCmd() *cobra.Command {
}
// Provider dependent params
providerParams := make(map[string]string)
for name := range manager.CreateParams() {
for name := range manager.CreateParams(manager.AWS) {
if viper.IsSet(name) {
providerParams[name] = viper.GetString(name)
}
Expand All @@ -49,7 +49,7 @@ func getAWSProviderCmd() *cobra.Command {

flagSet := pflag.NewFlagSet(awsProviderName, pflag.ExitOnError)
// Provider dependent params
for name, description := range manager.CreateParams() {
for name, description := range manager.CreateParams(manager.AWS) {
flagSet.StringP(name, "", "", description)
}

Expand Down
1 change: 1 addition & 0 deletions cmd/cmd/create/create.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ func GetCreateCmd() *cobra.Command {
createCmd.PersistentFlags().AddFlagSet(flagSet)

createCmd.AddCommand(getAWSProviderCmd())
createCmd.AddCommand(getGCPProviderCmd())

return createCmd
}
59 changes: 59 additions & 0 deletions cmd/cmd/create/gcp.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
package create

import (
"fmt"
"os"

"github.com/crc/crc-cloud/cmd/cmd/constants"
"github.com/crc/crc-cloud/pkg/manager"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
"github.com/spf13/viper"
)

const (
gcpProviderName string = "gcp"
gcpProviderDescription string = "create crc cloud instance on GCP"
)

func getGCPProviderCmd() *cobra.Command {
gcpProviderCmd := &cobra.Command{
Use: gcpProviderName,
Short: gcpProviderDescription,
RunE: func(cmd *cobra.Command, args []string) error {
if err := viper.BindPFlags(cmd.Flags()); err != nil {
return err
}
// Provider dependent params
providerParams := make(map[string]string)
for name := range manager.CreateParams(manager.GCP) {
if viper.IsSet(name) {
providerParams[name] = viper.GetString(name)
}
}
fmt.Printf("providerParams: %v", providerParams)
if err := manager.Create(
viper.GetString(constants.ProjectName),
viper.GetString(constants.BackedURL),
viper.GetString(constants.OutputFolder),
manager.GCP,
providerParams,
viper.GetString(constants.OcpPullSecretFilePath),
viper.GetString(constants.KeyFilePath),
viper.GetStringMapString(constants.Tags)); err != nil {
fmt.Printf("error creating the cluster with %s provider: %s\n", manager.GCP, err)
os.Exit(1)
}
return nil
},
}

flagSet := pflag.NewFlagSet(gcpProviderName, pflag.ExitOnError)
// Provider dependent params
for name, description := range manager.CreateParams(manager.GCP) {
flagSet.StringP(name, "", "", description)
}

gcpProviderCmd.Flags().AddFlagSet(flagSet)
return gcpProviderCmd
}
3 changes: 2 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ require (
github.com/pulumi/pulumi-aws/sdk/v6 v6.4.0
github.com/pulumi/pulumi-azure-native-sdk/resources/v2 v2.10.0
github.com/pulumi/pulumi-azure-native-sdk/storage/v2 v2.10.0
github.com/pulumi/pulumi-gcp/sdk/v6 v6.65.0
github.com/pulumi/pulumi-tls/sdk/v4 v4.11.0
github.com/pulumi/pulumi/sdk/v3 v3.87.0
github.com/spf13/cobra v1.7.0
Expand Down Expand Up @@ -99,7 +100,7 @@ require (
github.com/xanzy/ssh-agent v0.3.3 // indirect
go.uber.org/atomic v1.11.0 // indirect
golang.org/x/crypto v0.16.0 // indirect
golang.org/x/exp v0.0.0-20231127185646-65229373498e
golang.org/x/exp v0.0.0-20231127185646-65229373498e // indirect
golang.org/x/net v0.19.0 // indirect
golang.org/x/sys v0.15.0 // indirect
golang.org/x/term v0.15.0 // indirect
Expand Down
Loading