Add HA control-plane design doc [K8SSAND-1284] #433
Conversation
jdonenine
changed the title
There was a problem hiding this comment.
Very well written doc @Miles-Garnsey.
I think automated leader election in a multicluster environment is going to be tough to manage with simplicity (if we have 2 clusters and lose one, how can we ensure consistency in the leader election process?).
But here's my question: why would we want this to be automated? Can't we provide a manual procedure to switch control planes?
We will also need to copy over client configs to the data plane so that it can connect to the API server of remote k8s clusters. Not sure if that should be replicated from the start or done manually when needed (it could be unsafe to shuffle those around).
|
|
||
| This topic touches on a number of operational questions that users are likely to face in the real world. I'm not sure how many of these we have/want to consider, but I thought it'd be good to document them somewhere. | ||
|
|
||
| - A user has an existing CassandraDataCenter from Cass operator. How can they join it to a new K8ssandra cluster? |
There was a problem hiding this comment.
The procedure is documented in this blog post: https://k8ssandra.io/blog/tutorials/how-to/how-to-migrate-an-existing-cluster-to-k8ssandra-operator-without-any-downtime/
There was a problem hiding this comment.
That's good from a migration standpoint. There's a question still as to whether we support it long term.
There was a problem hiding this comment.
NB: we should also note that this ends up in a situation where the clusters are joined to the same Cassandra ring, but the cass-operator cluster isn't under the management of k8ssandra-operator. So there's two meanings to 'how can they join it to a new k8ssandra cluster' in that sentence.
| - A user has an existing CassandraDataCenter from Cass operator. How can they join it to a new K8ssandra cluster? | ||
| - What is the process for migrating control plane DCs and data plane DCs to new k8s clusters generally? | ||
| - What cases do we support around running some DCs under k8ssandra and others externally? (e.g. in a traditional bare metal DC.) | ||
| - Do we support mixed (k8s/bare-metal) clusters only at migration or permanently. |
There was a problem hiding this comment.
My opinion there is that we can support them permanently, but the automation we have in k8ssandra-operator won't spill over to the external clusters. Operations spanning over both clusters will have to be handled manually, which can come with some surprises.
There was a problem hiding this comment.
So I guess we'd recommend against it.
There was a problem hiding this comment.
I'm happy to recommend against. I suggest we leave the question in the doc though.
What this PR does:
What it says on the tin.
Which issue(s) this PR fixes:
Fixes #435
Checklist