Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom SQL for most heavy queries #3

Open
kdembler opened this issue Dec 1, 2023 · 0 comments
Open

Custom SQL for most heavy queries #3

kdembler opened this issue Dec 1, 2023 · 0 comments
Assignees

Comments

@kdembler
Copy link
Member

kdembler commented Dec 1, 2023

While the squid by itself brings a huge performance difference compared to the QN (10-15 minutes -> 30 sec) when fetching all the objects that a given distributor should serve, 30 sec at 100% CPU usage is still not a great result considering we fetch "only" 1M objects. My guess is that the auto-generated SQL query is quite unoptimized. Since getting all objects for a given worker will be done quite often, there's a lot of space for optimization. I think we could write a custom query with hand-written SQL that ensures good performance for those big queries.

@zeeshanakram3 zeeshanakram3 self-assigned this Dec 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants