Question about integration with CRDs & external data

[HurricanKai]

👋 This is looking really cool, I'm excited to try this out.
I'm wondering, is there a way to integrate this with existing CRDs? Seems like there's a bunch of machinery (finalizers?) that may or may not be required.
I'd be interested to integrate this with existing CRDs, like from a CNI, to automatically expose info to grafana and enable some UI inside grafana.

Simple example I can think of is a view that shows Services & Endpoint Slices, or load balancers / gateways w/ their public VIP. This wouldn't even involve any CRDs, just reading exiting well-known kinds.

I can see that I could create a mirroring CRD that only contains the relevant information and sync it myself, but this seems like it would create a fairly large amount of load on etcd for no reason. Two-way sync would also be fairly hard.

In a similar thought, can external resources be used? I see a few mentions of other non-k8s stores getting supported eventually, but that's not quite the same as just accessing it, similar to what I explained for the k8s scenario above.

I'll give it a spin asap to deepen my understanding here, but I'd love to understand where this is going too!

[IfSentient]

Hi @HurricanKai, so sorry that I missed your question until now! You can integrate with existing CRDs without needing any of the extra opinionated machinery, but you currently have to avoid using the simple package, which by default uses the opinionated logic (for writing non-simple operators, see Writing an Operator as a starting point). If you just want to expose them via grafana, without any reconciliation logic, you can create just a plugin that has the Create/Read/Update/Delete/List endpoints exposed via the CallResource API. There is also work being done on grafana (which can be enabled via feature flags as it is still experimental) to allow you to aggregate an API server underneath an exposed grafana Kubernetes-compatible API server, without the need for the app-sdk or a plugin.

If you wish to handle these existing CRDs in-code using the app-sdk, you'll need a type which implements resource.Object, which is a superset of kubernetes interfaces with some additional extra methods. There are a few different ways to do this:

• Define a go spec type, or import it from another go module, and use resource.TypedSpecObject[ExistingCRDGoSpec]
• Re-create the spec of the desired CRD in CUE, and generate go code for it using grafana-app-sdk generate
• Use resource.UntypedObject and have no concrete go types
• Implement the interface yourself--not recommended, as while the app-sdk is still experimental and pre-v1, this interface may change, and the above methods will all update automatically.

In a similar thought, can external resources be used? I see a few mentions of other non-k8s stores getting supported eventually, but that's not quite the same as just accessing it, similar to what I explained for the k8s scenario above.

Non-k8s stores will likely not be supported in the general sense at this point (some of the docs may still be a bit out of date), meaning that a resource.Client implementation for a non-kubernetes system probably won't exist in the near future in the app-sdk (though there is nothing stopping anyone from writing one!). However, we are exploring the ability for a user to create an app which is backed by something other than CRDs, and uses a full API Server pattern instead of just being CRD + Operator. Practically, that means you'd build your app to both have your reconciliation loop, and also serve the resources (instead of depending on the apiextensions server to serve up your CRs). This approach is still very much in development at this point, though, and is subject to change as we work on it.

Please feel free to ask further questions and I'll make sure I get back to you in a more timely manner in the future!