Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable concurrency for ListMetrics #2

Open
justin-watkinson-sp opened this issue May 28, 2019 · 2 comments
Open

Enable concurrency for ListMetrics #2

justin-watkinson-sp opened this issue May 28, 2019 · 2 comments

Comments

@justin-watkinson-sp
Copy link

While running this exporter in one of my environments, I noticed that the initial spin-up time to do all of the ListMetrics can take some time depending on the namespace. The main reason in my case is that we have a large number of resources under a particular namespace (e.g. AWS/SQS).

To improve performance, I believe we can parallelize these calls by namespace. This should reduce start-up time.

To address API limits, some handling of throttling exceptions and retrying them would be in order. I imagine something along the lines of N goroutines (let's say, 10) are started to call ListMetrics, and configurable via the config file if the implementation wants to raise or lower it.

Thanks!

@frioux
Copy link
Contributor

frioux commented May 29, 2019

Seems reasonable to me. We also run into that startup issue, which is why we built it to generally run in a different thread.

@justin-watkinson-sp
Copy link
Author

Indeed - been a busy month, but hopefully will get back to some activities such as this in the near future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants