kafka: SCRAM secret association

[chlunde]

What resource do you need?

ScramSecretAssociation.kafka

What is your use case?

Provision users with access to kafka

Would you be willing to contribute it using code generator?

Yes

I started looking at adding support for attaching secretsmanager secrets to kafka clusters in provider-aws / kafka. The MSK API for SCRAM secrets has List/BatchAssociate/BatchDisassociate, but no Get (or filter for List).

So it would be fairly expensive to create a model where each user is a distinct object (i.e. for 100 users, one would call List 100 times every 5 minutes).

So I wonder if we should have one object containing the list of all secrets that should be associated with the resource.

Using referencers, one can easily select any amount of secrets that should be attached to the cluster.

Design 1 - one ScramSecretAssociation per cluster

apiVersion: kafka.aws.crossplane.io/v1alpha1
kind: ScramSecretAssociation
metadata:
  name: example-assoc
spec:
  forProvider:
    region: eu-north-1
    clusterARN: arn:aws:kafka:eu-north-1:...:cluster/a50...aug/84bb3593-ab64-1234-1234-061e115a4eeb-4
    secretARNListSelector:
      matchLabels:
        for-cluster: foo
  providerConfigRef:
    name: example

Design 2 - one ScramSecretAssociation per user, multiple ScramSecretAssociation per cluster

apiVersion: kafka.aws.crossplane.io/v1alpha1
kind: ScramSecretAssociation
metadata:
  name: example-assoc
spec:
  forProvider:
    region: eu-north-1
    clusterARN: arn:aws:kafka:eu-north-1:...:cluster/a50...aug/84bb3593-ab64-1234-1234-061e115a4eeb-4
    secretARN: arn:..kms..foo
  providerConfigRef:
    name: example

In this case, we would call the List API endpoint (returning 1000 users) 1000 times per minute, if we have a reconcile interval of one minute and 1000 users in the cluster (the current limit)

[davidcurrie]

@chlunde - did you make any headway on this? I fear its absence is going to make Crossplane a non-starter for us...

[github-actions]

Crossplane does not currently have enough maintainers to address every issue and pull request. This issue has been automatically marked as stale because it has had no activity in the last 90 days. It will be closed in 14 days if no further activity occurs. Leaving a comment starting with /fresh will mark this issue as not stale.

[davidcurrie]

/fresh This is certainly still an issue - we currently have our Crossplane usage wrapped up with some bash scripting to do the association which is fairly unpleasant.

[github-actions]

Crossplane does not currently have enough maintainers to address every issue and pull request. This issue has been automatically marked as stale because it has had no activity in the last 90 days. It will be closed in 14 days if no further activity occurs. Leaving a comment starting with /fresh will mark this issue as not stale.