Monitoring IAM Controller

[piyush0609]

Any plans of adding monitoring and metrics related to functioning of iam controller? Any ideas around how it can be implemented?

[mikkeloscar]

What we monitor is that the credentials are getting refreshed which can be observed by looking at the status of the AWSIAMRole resources and checking that the expiry time is not closer than x minutes to now(). This we can do without adding anything to the controller.

Do you have something else in mind?

[piyush0609]

Yeah. Sounds reasonable enough to start monitoring its functioning. Apart from this I was thinking of having some custom metrics to monitor IAM controller-AWS IAM interaction. But this will require changes to the controller. Do you guys have plan for this in near future?

[piyush0609]

What we monitor is that the credentials are getting refreshed which can be observed by looking at the status of the AWSIAMRole resources and checking that the expiry time is not closer than x minutes to now(). This we can do without adding anything to the controller.

Do you have something else in mind?

Also I would like to know how you have currently implemented this. Is there some out of the box solution or you have written something custom?

[mikkeloscar]

Yeah. Sounds reasonable enough to start monitoring its functioning. Apart from this I was thinking of having some custom metrics to monitor IAM controller-AWS IAM interaction. But this will require changes to the controller. Do you guys have plan for this in near future?

We are not planning anything actively in this direction. We are evaluating the official solution from AWS so we may not be using this solution in the future. (Does not mean we will not try to support it if there is an interest).

Also I would like to know how you have currently implemented this. Is there some out of the box solution or you have written something custom?

We are using our custom monitoring solution for this which does what I described (Unfortunately not something we could share that would be directly useful to you). I basically goes to the APIserver and gets the AWSIAMRole resources and checks the expiration in the status.

[ITler]

We are evaluating the official solution from AWS

What solution do you refer to? IRSA?

[mikkeloscar]

What solution do you refer to? IRSA?

Yes, we migrated to IRSA some years ago (we don't run EKS, but implemented the same thing in our cluster) and successfully use that.