Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modify the staging k6 load testing to allow for accurate tests with lower resources #5179

Open
obulat opened this issue Nov 23, 2024 · 0 comments
Assignees
Labels
💻 aspect: code Concerns the software code in the repository 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🟧 priority: high Stalls work on the project or its dependents 🧱 stack: frontend Related to the Nuxt frontend 🔒 staff only Restricted to staff members

Comments

@obulat
Copy link
Contributor

obulat commented Nov 23, 2024

Problem

The current frontend staging k6 load tests require the the staging tasks to have the CPU of 2 vCPU to run. When running with 0.25 vCPU, the tests fail before any memory leak can be detected due to CPU load being over 100%.

Description

Based on the suggestion from @sarayourfriend , we should scale the staging load tests down until they reliably pass, rather than staging tasks up.

To simplify finding the right balance of VU and number of iteration, we should try using the constant-arrival-rate executor with timeUnit: 1s, rate: 2, duration: 5m and then manually ramp rate up by +1 until staging starts to fall over. The goal would be to find the highest rate where it consistently passes so you don't end up with flaky tests, but still have an aggressive enough load test that something like a memory leak would show up.

To test if this testing mode catches a memory leak, we can intentionally merge in the changes reverted in #4864.
Trying the same with something CPU bound would also be good, but I don't know of such a change to use for a test.

Alternatives

Keep using the higher CPU values for staging. This would cost more.

Additional context

@obulat obulat added 💻 aspect: code Concerns the software code in the repository 🔒 staff only Restricted to staff members 🟧 priority: high Stalls work on the project or its dependents 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🧱 stack: frontend Related to the Nuxt frontend labels Nov 23, 2024
@obulat obulat self-assigned this Nov 23, 2024
@openverse-bot openverse-bot moved this to 📋 Backlog in Openverse Backlog Nov 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
💻 aspect: code Concerns the software code in the repository 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🟧 priority: high Stalls work on the project or its dependents 🧱 stack: frontend Related to the Nuxt frontend 🔒 staff only Restricted to staff members
Projects
Status: 📋 Backlog
Development

No branches or pull requests

1 participant