-
Notifications
You must be signed in to change notification settings - Fork 34
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(client): validate all known_hosts on ScyllaAPI client creation #3716
fix(client): validate all known_hosts on ScyllaAPI client creation #3716
Conversation
0ea281b
to
3ecbc9d
Compare
ad9d8f3
to
9041bb5
Compare
It seems that some tests are failing. |
Yes, and these are weird failures. All are passing when executed locally. |
OK, it fails on IPV6 cluster integration tests actually. I havent check it locally... |
9041bb5
to
494e620
Compare
Fixed. Test was using IP from IPv4 range directly. |
I just added one small comment, but apart from that it looks good! |
b105417
to
d98f159
Compare
d98f159
to
5d97d6a
Compare
The fix for #3707 includes extending the cluster kept in the DB with the 'host' property, which represents the initial host used when adding a cluster to Scylla Manager. Scylla-operator passes the DNS name here, making it immune to the ephemeral IPs in the Kubernetes environment. This commit adds the initial host as the first host to query in order to discover node IPs.
5d97d6a
to
37d7410
Compare
Fixes #3707
This PR changes how the Scylla API client is created.
Previously, as Scylla Manager stores all known hosts in the DB, it was passing all of them to the API client initially upon creation. The client then called one of the endpoints to discover available hosts. This approach could lead to non-existing IPs being passed during client creation, and failure when calling the specified endpoint would later indicate that the host doesn't exist.
The client created at this first step is expected to call the Scylla API to retrieve information about existing/available nodes and their IPs, and update this information in the DB. To avoid a situation where the client selects a non-existing IP from the pool, this PR iterates through all known hosts and creates the initial client with just a single IP. If calling the discover hosts endpoint fails, it proceeds to the next host.
The host used when adding the cluster to Scylla Manager is included in the hosts that will be checked against the discover hosts endpoint.
Scylla Manager caches Scylla API clients and invalidates them after 15 minutes. This PR makes the timeout configurable and includes it as part of the
scylla-manager.yaml
configuration. It defaults to 15 minutes.Please make sure that: