[Bug]: CannotObserveExternalResource when providing input to spec from another resource using getComposedResource from function-go-templating

[alanrichman]

Is there an existing issue for this?

• I have searched the existing issues

Affected Resource(s)

• cloudplatform.gcp.upbound.io/v1beta1 - Project
• cloudplatform.gcp.upbound.io/v1beta1 - ServiceAccount

Resource MRs required to reproduce the bug

Providers

apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-gcp-cloudplatform
spec:
  package: xpkg.upbound.io/upbound/provider-gcp-cloudplatform:v1.11.0

Functions

apiVersion: pkg.crossplane.io/v1beta1
kind: Function
metadata:
  name: function-go-templating
spec:
  package: xpkg.upbound.io/crossplane-contrib/function-go-templating:v0.8.0
  runtimeConfigRef:
    name: mount-templates
---
apiVersion: pkg.crossplane.io/v1beta1
kind: Function
metadata:
  name: function-auto-ready
spec:
  package: xpkg.upbound.io/crossplane-contrib/function-auto-ready:v0.3.0

XRD and Composition

apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: xsampleresources.crossplane.example.com
spec:
  group: crossplane.example.com
  names:
    kind: XSampleResource
    plural: xsampleresources
  claimNames:
    kind: SampleResource
    plural: sampleresources
  versions:
    - name: v1alpha1
      served: true
      referenceable: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                name:
                  type: string
                  description: name
              required:
                - name
---
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: sample-resource-composition
spec:
  compositeTypeRef:
    apiVersion: crossplane.example.com/v1alpha1
    kind: XSampleResource
  mode: Pipeline
  pipeline:
    - step: infra
      functionRef:
        name: function-go-templating
      input:
        apiVersion: gotemplating.fn.crossplane.io/v1beta1
        kind: GoTemplate
        source: Inline
        inline:
          template: |
            --- # Create Project
            apiVersion: cloudplatform.gcp.upbound.io/v1beta1
            kind: Project
            metadata:
              name: sample-resource-{{ .observed.composite.resource.spec.name }}
              annotations:
                gotemplating.fn.crossplane.io/composition-resource-name: project
            spec:
              forProvider:
                name: {{ (printf "sample-resource-%.8s" .observed.composite.resource.spec.name) }}
                projectId: {{ (printf "sample-resource-%.8s" .observed.composite.resource.spec.name) }}
                folderId: [REDACTED]
                billingAccount: [REDACTED]
              providerConfigRef:
                name: default
            --- # Enable APIs
            # IAM API
            apiVersion: cloudplatform.gcp.upbound.io/v1beta1
            kind: ProjectService
            metadata:
              name: sample-resource-{{ .observed.composite.resource.spec.name }}-iam
              annotations:
                gotemplating.fn.crossplane.io/composition-resource-name: iam-api
            spec:
              forProvider:
                projectSelector:
                  matchControllerRef: true
                service: iam.googleapis.com
              providerConfigRef:
                name: default
            --- # Create Service Account
            apiVersion: cloudplatform.gcp.upbound.io/v1beta1
            kind: ServiceAccount
            metadata:
              name: my-sample-sa
              annotations:
                gotemplating.fn.crossplane.io/composition-resource-name: serviceaccount
            spec:
              forProvider:
                project: {{ (getComposedResource . "project").status.atProvider.projectId }}
                displayName: {{ (getComposedResource . "project").status.atProvider.projectId }}
                description: Service account for sample resource `{{ .observed.composite.resource.spec.name }}`
              providerConfigRef:
                name: default
    - step: ready
      functionRef:
        name: function-auto-ready

Claim

apiVersion: crossplane.example.com/v1alpha1
kind: SampleResource
metadata:
  name: my-sample-resource
  namespace: default
spec:
  name: 1212-1103 # Just needs to be unique since it creates a GCP project, this is just `MMDD-HHmm` of when I ran this specific test

Steps to Reproduce

1. Apply providers
2. Apply functions
3. Apply XRD and Composition
4. Apply Claim
5. Observe creation of Project and ProjectService
6. Observe spec and status of ServiceAccount

What happened?

When the ServiceAccount MR is created the spec contains <no value> for both displayName and projectId. This is expected since the status of the Project referenced by getComposedResource is not present until the Project is Synced and Ready.

From kubectl describe serviceaccount my-sample-sa:

Spec:
  Deletion Policy:  Delete
  For Provider:
    Description:   Service account for sample resource `1212-1103`
    Display Name:  <no value>
    Project:       <no value>
  Init Provider:
  Management Policies:
    *
  Provider Config Ref:
    Name:  default

The ServiceAccount has an event logging the failure to create the external resource (expected):

Warning  CannotObserveExternalResource  1s (x5 over 15s)  managed/cloudplatform.gcp.upbound.io/v1beta1, kind=serviceaccount  failed to observe the resource: [{0 Error when reading or editing Service Account "projects/<no value>/serviceAccounts/my-sample-sa@<no value>.iam.gserviceaccount.com": googleapi: Error 400: Unknown error, badRequest  []}]

When the value is present from the Project, the spec of the ServiceAccount is updated correctly:

Spec:
  Deletion Policy:  Delete
  For Provider:
    Description:   Service account for sample resource `1212-1103`
    Display Name:  sample-resource-1212-110
    Project:       sample-resource-1212-110
  Init Provider:
  Management Policies:
    *
  Provider Config Ref:
    Name:  default

However it does not appear that the API call made to GCP is updated with the new value in the spec:

Warning  CannotObserveExternalResource  38s (x21 over 14m)  managed/cloudplatform.gcp.upbound.io/v1beta1, kind=serviceaccount  failed to observe the resource: [{0 Error when reading or editing Service Account "projects/<no value>/serviceAccounts/my-sample-sa@<no value>.iam.gserviceaccount.com": googleapi: Error 400: Unknown error, badRequest  []}]

After the projectId field of the spec is updated, deleting the ServiceAccount and allowing Crossplane to recreate it causes the GCP Service Account resource to be created correctly on the first attempt.

Relevant Error Output Snippet

Warning  CannotObserveExternalResource  38s (x21 over 14m)  managed/cloudplatform.gcp.upbound.io/v1beta1, kind=serviceaccount  failed to observe the resource: [{0 Error when reading or editing Service Account "projects/<no value>/serviceAccounts/my-sample-sa@<no value>.iam.gserviceaccount.com": googleapi: Error 400: Unknown error, badRequest  []}]

### Crossplane Version

1.18.0

### Provider Version

1.11.0

### Kubernetes Version

1.29.4

### Kubernetes Distribution

Kind, GKE

### Additional Info

In this particular example the piece of information being retrieved is knowable without using `getComposedResource` as the project ID needs to be specified at time of creation for the project. There are other circumstances where I am trying to perform an identical process where the information is not knowable at time of creating the resource.

For example, there are times when you need to use the project _number_ rather than the _ID_. Since this number is generated by GCP when the project is created there is no way to use it without referencing it from the `Project` resource.

Thank you!