Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pick upstreams #483

Closed
wants to merge 7 commits into from
Closed

Pick upstreams #483

wants to merge 7 commits into from

Conversation

lmr3796
Copy link

@lmr3796 lmr3796 commented Oct 30, 2023

This PR is not for merge, just demonstration on the final build result

ijuma and others added 7 commits October 30, 2023 02:00
…e#11966)

LI_DESCRIPTION=This may improve p99 inter-broker latency under TLS1.3

-- Original message --

Key updates with TLS 1.3 trigger code paths similar to renegotiation with TLS 1.2.
Update the read/write paths not to throw an exception in this case (kept the exception
in the `handshake` method).

With the default configuration, key updates happen after 2^37 bytes are encrypted.
There is a security property to adjust this configuration, but the change has to be
done before it is used for the first time and it cannot be changed after that. As such,
it is best done via a system test (filed KAFKA-13779).

To validate the change, I wrote a unit test that forces key updates and manually ran
a producer workload that produced more than 2^37 bytes. Both cases failed without
these changes and pass with them.

Note that Shylaja Kokoori attached a patch with the SslTransportLayer fix and hence
included them as a co-author of this change.

Reviewers: Rajini Sivaram <[email protected]>

Co-authored-by: Shylaja Kokoori
… closed upon IOException (apache#11504)

This patch ensures that SocketChannel in Acceptor#accept is closed if an IOException is thrown while the socket is configured.

Reviewers:  Luke Chen <[email protected]>, David Jacot <[email protected]>
…pache#11618)

LI_DESCRIPTION=Potentially reduce flaky tests

-- Original message --

Due to resource leaks in the NioEchoServer, at times it won't start
properly to accept clients and will throw an exception in the
ServerSocketChannel.accept() call. Previous to this change, the error
was not being logged. The logged error was that there were too many open
files. Using the UnixOperatingSystemMXBean, I was able to detect that
use of the NioEchoServer creates several FDs but does not close them.
This then caused the client to never be able to connect to the server,
so the waitForCondition failed intermittently.

This change closes the internal Selector and the AcceptorThread's
selector so that the file descriptors are reclaimed.

Reviewers: Ismael Juma <[email protected]>
…re the offsets are deleted for unstable groups (apache#11742)

This patch ensures that the committed offsets are not expired while the group is rebalancing. The issue is that we can't rely on the subscribed topics if the group is not stable.

Reviewers: David Jacot <[email protected]>
…tChannel is removed twice (apache#12111)

Fix ResponseSendTimeMs metric in RequestChannel is removed twice

Reviewers: Luke Chen <[email protected]>
…ot shut down when ZK session expires

FinalizedFeatureChangeListener shuts the broker down when it encounters an issue trying to process feature change
events. However, it does not distinguish between issues related to feature changes actually failing and other
exceptions like ZooKeeper session expiration. This introduces the possibility that Zookeeper session expiration
could cause the broker to shutdown, which is not intended. This patch updates the code to distinguish between
these two types of exceptions. In the case of something like a ZK session expiration it logs a warning and continues.
We shutdown the broker only for FeatureCacheUpdateException.

Reviewers: Kamal Chandraprakash <[email protected]>, Christo Lolov <[email protected]>, Colin P. McCabe <[email protected]>
@lmr3796 lmr3796 closed this Nov 1, 2023
@lmr3796
Copy link
Author

lmr3796 commented Nov 1, 2023

All patches merged separately

@lmr3796 lmr3796 deleted the pick-upstreams branch November 2, 2023 21:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants