Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[controller][server] Remove SIT ready-to-serve check for A/A and non-AGG store during L/F transition #1409

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

sixpluszero
Copy link
Contributor

@sixpluszero sixpluszero commented Dec 19, 2024

[server] Remove SIT ready-to-serve check for A/A and non-AGG store during L/F transition

Ready-to-serve check happens in two threads today:
(1) Drainer: which is reasonable
(2) SIT thread: there are two sub-cases:

  • During L/F transition: It is only for empty RT topic, and today we have HB for non-agg and A/A already, so it only applies to the lingering AGG system store. In this PR, it is checked against store data replication type, if it is AGG we will still have it. We will completely remove it once AGG mode is removed.
  • Before subscribe, it has a pre-check for short circuit to complete, and this is fine, as it is before the subscribe.

Also, I noticed that there are a bunch of integration tests which enables incremental push on NON-AA store. This is totally invalid setup and after removing the additional RTS check, these tests start to fail/very flaky.
To make sure we don't do this in test and prod, this PR added a new check in update store command in parent controller, which will fail loudly if the new update request is to set incremental push to be true on a non-A/A store.
I have another thinking about enabling all the configs that are enabled in production to be enabled in the test suite, but I'd like to do it in another separate PR, so it is easier for reviewers.

Beyonds that, fixed a flaky unit test that can throw NPE.

How was this PR tested?

Add a sinlge unit test.

Does this PR introduce any user-facing changes?

  • No. You can skip the rest of this section.
  • Yes. Make sure to explain your proposed changes and call out the behavior change.

@sixpluszero sixpluszero marked this pull request as draft December 20, 2024 01:04
@sixpluszero sixpluszero marked this pull request as ready for review December 24, 2024 00:43
@sixpluszero sixpluszero changed the title [server] Remove SIT ready-to-serve check for A/A and non-AGG store during L/F transition [controller][server] Remove SIT ready-to-serve check for A/A and non-AGG store during L/F transition Dec 24, 2024
&& hybridStoreConfig.get().getDataReplicationPolicy().equals(DataReplicationPolicy.AGGREGATE);
}

ReadyToServeCheck getReadyToServerChecker() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ReadyToServeCheck getReadyToServerChecker() {
ReadyToServeCheck getReadyToServeChecker() {

You must've been typing 'server' too much.

&& !setStore.activeActiveReplicationEnabled) {
throw new VeniceHttpException(
HttpStatus.SC_BAD_REQUEST,
"Hybrid store config invalid. Cannot have incremental push enabled while A/A not enabled",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some failing tests will need to be updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants