Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Templating Update #358

Open
wants to merge 23 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
77c6544
Refactor, add CustomTemplateValues support
dahendel Jun 20, 2024
1c473cc
Refactor to use renderGoTemplating function
dahendel Jun 20, 2024
84ff1c2
Add new template-based detokenization logic
dahendel Jul 5, 2024
a478882
``` Refactor detokenize logic; remove redundant code ```
dahendel Jul 5, 2024
9f2e7ce
Remove unused toTemplateVariable function
dahendel Jul 5, 2024
3c5bd70
go fmt
dahendel Jul 9, 2024
96bef4f
Adding some code comments
dahendel Jul 9, 2024
98eb9b8
complete test suite and fixes
dahendel Jul 10, 2024
d0faba8
updated delims to not conflict with Terraform <<EOT, more tests
dahendel Jul 10, 2024
60e8ebd
refactor detokenize to handle gitops and metaphor
dahendel Jul 11, 2024
a6eb775
add optional cleanup step based on K1_TEST_CLEANUP
dahendel Jul 11, 2024
0a5e2c6
Merge branch 'main' into main
dahendel Jul 12, 2024
b1040e0
add toJson func for templating
dahendel Jul 16, 2024
c886ff8
chore: bump gitops template version (#366)
CristhianF7 Jul 15, 2024
20fd38a
[skip ci] [CI SKIP] setting kubefirst-api production to chart version…
Jul 15, 2024
a228198
fix : timeout issue for argocd and vault for public clouds (#363)
jokestax Jul 15, 2024
10455f9
go.mod dep update
dahendel Jul 16, 2024
d75f6fb
fmt
dahendel Jul 16, 2024
548959a
Merge remote-tracking branch 'upstream/main'
dahendel Jul 16, 2024
ef3bd8a
update doc comment on return statement ToTemplateVars
dahendel Jul 25, 2024
ffe250f
addressing pr comments
dahendel Jul 25, 2024
5a69273
Update repo URL and add directory existence check
dahendel Jul 25, 2024
6a02964
correct a DetokenizeGitGitops Call from services
dahendel Jul 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 14 additions & 4 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ require (
cloud.google.com/go/container v1.24.0
cloud.google.com/go/secretmanager v1.10.0
cloud.google.com/go/storage v1.29.0
github.com/Masterminds/sprig/v3 v3.2.2
github.com/argoproj/argo-cd/v2 v2.6.7
github.com/argoproj/gitops-engine v0.7.3
github.com/atotto/clipboard v0.1.4
Expand Down Expand Up @@ -38,6 +39,7 @@ require (
github.com/rs/zerolog v1.29.1
github.com/segmentio/analytics-go v3.1.0+incompatible
github.com/sirupsen/logrus v1.9.0
github.com/stretchr/testify v1.8.4
github.com/swaggo/files v1.0.0
github.com/swaggo/gin-swagger v1.5.3
github.com/swaggo/swag v1.16.1
Expand All @@ -54,12 +56,20 @@ require (
)

require (
github.com/Masterminds/goutils v1.1.1 // indirect
github.com/alicebob/gopher-json v0.0.0-20200520072559-a9ecdc9d1d3a // indirect
github.com/alicebob/miniredis/v2 v2.23.1 // indirect
github.com/docker/go-connections v0.4.0 // indirect
github.com/docker/go-units v0.4.0 // indirect
github.com/go-resty/resty/v2 v2.11.0 // indirect
github.com/go-test/deep v1.0.4 // indirect
github.com/hashicorp/go-hclog v1.3.0 // indirect
github.com/opencontainers/image-spec v1.0.3-0.20211202183452-c5a74bcca799 // indirect
github.com/go-test/deep v1.0.3 // indirect
github.com/huandu/xstrings v1.3.1 // indirect
github.com/mitchellh/copystructure v1.0.0 // indirect
github.com/mitchellh/reflectwalk v1.0.0 // indirect
github.com/opencontainers/image-spec v1.0.2 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/shopspring/decimal v1.2.0 // indirect
github.com/yuin/gopher-lua v0.0.0-20220504180219-658193537a64 // indirect
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect
gotest.tools/v3 v3.4.0 // indirect
)
Expand Down Expand Up @@ -265,7 +275,7 @@ require (
gopkg.in/ini.v1 v1.67.0 // indirect
gopkg.in/square/go-jose.v2 v2.6.0 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
gopkg.in/yaml.v3 v3.0.1
k8s.io/apiextensions-apiserver v0.26.0 // indirect
k8s.io/apiserver v0.24.2 // indirect
k8s.io/cli-runtime v0.24.2 // indirect
Expand Down
37 changes: 25 additions & 12 deletions go.sum

Large diffs are not rendered by default.

8 changes: 4 additions & 4 deletions internal/controller/argocd.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import (
"context"
"fmt"
"os/exec"
"time"
"time"

argocdapi "github.com/argoproj/argo-cd/v2/pkg/client/clientset/versioned"
awsext "github.com/kubefirst/kubefirst-api/extensions/aws"
Expand Down Expand Up @@ -178,7 +178,7 @@ func (clctrl *ClusterController) DeployRegistryApplication() error {
if err != nil {
return err
}

log.Info().Msg("applying the registry application to argocd")

registryURL, err := clctrl.GetRepoURL()
Expand Down Expand Up @@ -221,11 +221,11 @@ func (clctrl *ClusterController) DeployRegistryApplication() error {
}
log.Info().Msgf("Error creating Argo CD application on attempt number #%d: %v\n", attempt, err)
time.Sleep(5 * time.Second)
continue
continue
}

log.Info().Msgf("Argo CD application created successfully on attempt #%d: %s\n", attempt, app.Name)
break
break
}


Expand Down
6 changes: 3 additions & 3 deletions internal/vultr/kubernetes.go
Original file line number Diff line number Diff line change
Expand Up @@ -90,15 +90,15 @@ func (c *VultrConfiguration) DeleteBlockStorage(blockStorage []govultr.BlockStor
return nil
}

func (c *VultrConfiguration) GetKubeconfig(clusterName string)(string, error) {
func (c *VultrConfiguration) GetKubeconfig(clusterName string) (string, error) {
clusters, _, _, err := c.Client.Kubernetes.ListClusters(c.Context, &govultr.ListOptions{})

if err != nil {
return "", err
}

var clusterId string
for _, cluster := range clusters {
for _, cluster := range clusters {
if cluster.Label == clusterName {
clusterId = cluster.ID
continue
Expand All @@ -112,7 +112,7 @@ func (c *VultrConfiguration) GetKubeconfig(clusterName string)(string, error) {
kubeConfig, _, err := c.Client.Kubernetes.GetKubeConfig(c.Context, clusterId)

if err != nil {
return "", err
return "", err
}

return kubeConfig.KubeConfig, nil
Expand Down
4 changes: 2 additions & 2 deletions pkg/providerConfigs/adjustDriver.go
Original file line number Diff line number Diff line change
Expand Up @@ -985,7 +985,7 @@ func PrepareGitRepositories(

// DETOKENIZE
//* detokenize the gitops repo
DetokenizeGitGitops(gitopsDir, gitopsTokens, gitProtocol, useCloudflareOriginIssuer)
err = Detokenize(gitopsDir, gitopsTokens, gitProtocol, useCloudflareOriginIssuer)
if err != nil {
return err
}
Expand All @@ -999,7 +999,7 @@ func PrepareGitRepositories(

// DETOKENIZE
//* detokenize the metaphor repo
DetokenizeGitMetaphor(metaphorDir, metaphorTokens)
err = Detokenize(metaphorDir, metaphorTokens, gitProtocol, useCloudflareOriginIssuer)
if err != nil {
return err
}
Expand Down
62 changes: 27 additions & 35 deletions pkg/providerConfigs/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,8 @@ package providerConfigs

import (
"fmt"
"os"

"github.com/rs/zerolog/log"
"os"
)

type ProviderConfig struct {
Expand All @@ -27,35 +26,32 @@ type ProviderConfig struct {
VultrToken string
CloudflareAPIToken string
CloudflareOriginCaIssuerAPIToken string

GithubToken string
GitlabToken string

ArgoWorkflowsDir string
DestinationGitopsRepoHttpsURL string
DestinationGitopsRepoGitURL string
DestinationGitopsRepoURL string
DestinationMetaphorRepoHttpsURL string
DestinationMetaphorRepoGitURL string
DestinationMetaphorRepoURL string
GitopsDir string
GitProvider string
GitProtocol string
K1Dir string
Kubeconfig string
KubectlClient string
KubefirstBotSSHPrivateKey string
KubefirstConfig string
LogsDir string
MetaphorDir string
RegistryAppName string
RegistryYaml string
SSLBackupDir string
TerraformClient string
ToolsDir string

GitopsDirectoryValues *GitopsDirectoryValues
MetaphorDirectoryValues *MetaphorTokenValues
GithubToken string
GitlabToken string
ArgoWorkflowsDir string
DestinationGitopsRepoHttpsURL string
DestinationGitopsRepoGitURL string
DestinationGitopsRepoURL string
DestinationMetaphorRepoHttpsURL string
DestinationMetaphorRepoGitURL string
DestinationMetaphorRepoURL string
GitopsDir string
GitProvider string
GitProtocol string
K1Dir string
Kubeconfig string
KubectlClient string
KubefirstBotSSHPrivateKey string
KubefirstConfig string
LogsDir string
MetaphorDir string
RegistryAppName string
RegistryYaml string
SSLBackupDir string
TerraformClient string
ToolsDir string
GitopsDirectoryValues *GitopsDirectoryValues
MetaphorDirectoryValues *MetaphorTokenValues
}

// GetConfig - load default values from kubefirst installer
Expand All @@ -69,12 +65,10 @@ func GetConfig(
cloudflareOriginCaIssuerAPIToken string,
) *ProviderConfig {
config := ProviderConfig{}

homeDir, err := os.UserHomeDir()
if err != nil {
log.Fatal().Msgf("something went wrong getting home path: %s", err)
}

// cGitHost describes which git host to use depending on gitProvider
var cGitHost string
switch gitProvider {
Expand All @@ -83,7 +77,6 @@ func GetConfig(
case "gitlab":
cGitHost = GitlabHost
}

config.DestinationGitopsRepoURL = fmt.Sprintf("https://%s/%s/gitops.git", cGitHost, gitOwner)
config.DestinationGitopsRepoGitURL = fmt.Sprintf("git@%s:%s/gitops.git", cGitHost, gitOwner)
config.DestinationMetaphorRepoURL = fmt.Sprintf("https://%s/%s/metaphor.git", cGitHost, gitOwner)
Expand All @@ -105,6 +98,5 @@ func GetConfig(
config.SSLBackupDir = fmt.Sprintf("%s/.k1/%s/ssl/%s", homeDir, clusterName, domainName)
config.TerraformClient = fmt.Sprintf("%s/.k1/%s/tools/terraform", homeDir, clusterName)
config.ToolsDir = fmt.Sprintf("%s/.k1/%s/tools", homeDir, clusterName)

return &config
}
5 changes: 4 additions & 1 deletion pkg/providerConfigs/constants.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,11 @@ const (
LocalhostOS = runtime.GOOS
LocalhostArch = runtime.GOARCH
TerraformClientVersion = "1.3.8"
ArgocdHelmChartVersion = "4.10.5"

ArgocdPortForwardURL = runtimepkg.ArgocdPortForwardURL
VaultPortForwardURL = runtimepkg.VaultPortForwardURL

TokenRegexPattern = "<([A-Z_0-9-]+)>"
leftDelimiter = "${k1"
rightDelimiter = "}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After discussing with the rest of the team, we want to go with these for delimiters:

Suggested change
leftDelimiter = "${k1"
rightDelimiter = "}"
leftDelimiter = "<%"
rightDelimiter = "%>"

)
Loading