would it be possible to concatenate label value with global labels now?

[virtuman]

Now that #325 and #152 are marked as resolved, would this be possible to achieve for non-helm-based resources:

we are adding a series of ExternalSecrets on each new cluster that is spun up. Unfortunately we have not yet found a way to register new cluster in hashicorp vault, that remains a manual step, but we would like for the externalSecrets to be formed properly to reflect the cluster and the role names that are based on cluster name.

so essentualy, we need to concatenate string with value from cluster-display-name label, such as - this is PSEUDO-CODE-EXAMPLE not intended to work:
I cluster name is:

global: 
  fleet: 
    clusterLabels.management.cattle.io/cluster-display-name: gke_some-cluster-external

and I want to add something like "-role" to the cluster name as a value to a helm attribute:

apiVersion: 'kubernetes-client.io/v1'
kind: ExternalSecret
metadata:
  name: jaeger-operator-jaeger
spec:
  backendType: vault
  vaultMountPoint: $(global.fleet.clusterLabels.management.cattle.io/cluster-display-name)-external-secrets
  vaultRole: $(global.fleet.clusterLabels.management.cattle.io/cluster-display-name)-role
  kvVersion: 2
  data:
  - name: ES_USERNAME
    key: infra-secrets/data/elastic-cloud
    property: ES_USERNAME

to achieve something like this as a final result that gets applied to the cluster (see vaultMountPoint and vaultRole keys ):

apiVersion: 'kubernetes-client.io/v1'
kind: ExternalSecret
metadata:
  name: jaeger-operator-jaeger
spec:
  backendType: vault
  vaultMountPoint: gke_some-cluster-external-secrets
  vaultRole: gke_some-cluster-role
  kvVersion: 2
  data:
  - name: ES_USERNAME
    key: infra-secrets/data/elastic-cloud
    property: ES_USERNAME

[StrongMonkey]

That's more like a feature to add to allow templating on raw k8s yaml, but it might be easier to convert your externalSecret into a simple helm chart

[virtuman]

no go for this feature? i've actually found quite a few apps that follow this exact same pattern with replacing encapsulated values only

[virtuman]

for example using static IPs on GKE cluster by referenced names, we have several clusters in some GCP projects, and we'd love to have the ability to name it something like gke-prod-tr01-ingress, gke-prod-tr01-thanos-ip, gke-prod-dc02-ingress , etc. but right now this is not possible or I only through kustomize, which means I have to describe it for each cluster, overly complex for something simple. Can I not get anyone else interested in this type of feature?

[wzrdtales]

easier example. ingress domain prefixing. this should be a feature request

[philomory]

Agree with @wzrdtales , this would be hugely helpful for ingress domains. Even when you are using Helm,most public helm charts that generate Ingress object ask you to pass the full DNS name as a value, which means if you want to get just a part of the domain name from the cluster labels, you end up needing to fork all your helm charts.

[nickgerace]

Spoke with @StrongMonkey on this and have the following question: what would this feature enable you to do that converting to a Helm chart would not?

We are wary of having templating-like functionality on raw k8s YAML since Fleet tries to be relatively "un-opinionated". However, I'd like to discuss in case we are missing something: @philomory @wzrdtales @virtuman

[dennyc-au]

Hi @nickgerace

I have the following use case which is using a Helm Chart via Fleet. I would like to annotate a K8s Service Account in order to use AWS' IAM Roles For Service Accounts feature. The annotation would be different for each cluster.

Contents of fleet.yaml file:

defaultNamespace: my-namespace
helm:
  releaseName: my-release
  repo: https://repo-url/helm-charts
  chart: chart-name
  version: 1.0.0
  values:
    serviceAccount:
      annotations:
        eks.amazonaws.com/role-arn: arn:aws:iam::global.fleet.${clusterLabels.aws_account_id}:role/${global.fleet.clusterLabels.iam_role_name}

The above currently does not work because Cluster Labels cannot be interpolated within the String.

From what I can see, allowing variable interpolation in the Values parameter would allow it to work. Is there an alternative way?

[StrongMonkey]

@dennyc-au I believe you can use workaround like this

# serviceAccount.yaml
apiVersion: v1
kind: ServiceAcoount
meta:
  annotation:
   eks.amazonaws.com/role-arn: arn:aws:iam::{{ .Values.awsAccountId }}:role/{{ .Value.IamRoleName }}
---
# fleet.yaml
helm:
  releaseName: my-release
  repo: https://repo-url/helm-charts
  chart: chart-name
  version: 1.0.0
  values:
    awsAccountId: $(global.fleet.clusterLabels.aws_account_id}
    IamRoleName: $(global.fleet.clusterLabels.iam_role_name}

So first parameterize those two labels as values.yaml .Values.awsAccountId and .Value.IamRoleName and then expose those two values using cluster label during the runtime.

[dennyc-au]

@StrongMonkey

Thanks for your response.

Is your suggestion to put the serviceAccount.yaml file in the same folder as fleet.yaml?

I tested by putting the following serviceAccount.yaml in the same folder as fleet.yaml. It did not annotate the Service Account.

apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/role-name
  name: service-account-name

If your suggestion is actually to insert the serviceAccount.yaml in the Helm Chart I'm using in helm.chart in fleet.yaml, I can see how that can work if the Helm Chart is developed in house. However, I am using a Helm Chart maintained by a third party. I prefer not to fork and modify that Helm Chart.

[StrongMonkey]

@dennyc-au Yes, my suggestion is to modify the helm chart as a workaround for now.

[dennyc-au]

@StrongMonkey ok, understood.

As I mentioned, modifying the Helm Chart is not ideal. Is this sufficient reasoning to request the variable interpolation functionality to be implemented?

[StrongMonkey]

@dennyc-au Yes, we will discuss this along with team. cc @nickgerace

[dennyc-au]

Thank you @StrongMonkey

[philomory]

I'd concur with @dennyc-au, having to maintain a private fork of every single public helm chart we deploy to our fleet would be suboptimal, so having basic variable interpolation in fleet.yaml would be wonderful.

[nickgerace]

We see the use case! I've marked this as a Release Candidate accordingly. Thanks for sticking with us for exploring this user need for this feature.

[manno]

I believe this is now implemented with #671