Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OSS Sync | Mon Sep 18 15:23:11 UTC 2023 #570

Open
wants to merge 56 commits into
base: master
Choose a base branch
from

Conversation

berman7
Copy link
Collaborator

@berman7 berman7 commented Sep 18, 2023

No description provided.

leonbur and others added 30 commits September 18, 2023 15:16
* format: data streams/

* format: iptf/

GitOrigin-RevId: a31eb0dc0eaff014511665e52497dea67a3f9ff1
* Remove unused test deps

* Remove unused imports

* Add runtime deps

* Initial cross-repo check

* Updating cross-repo check

* Updating cross-repo check

* Updating cross-repo check

---------

Co-authored-by: wixapiregistry <[email protected]>
GitOrigin-RevId: bd83b6b6f4ca9706f6b9e0f247a8cac8bd3f1292
* Blocking retries attempts tracking fix #pr

* Factored out RetryAttempt logic, TODO tests

* Added RetryAttemptTest

* Reverted RetryHeader back

* Fixed retry test

GitOrigin-RevId: 7e0e8cbccdd4c5fe6446775262b6695697785b2b
Revert "Blocking retries attempts tracking fix (#32757)"

This reverts commit 7e0e8cbccdd4c5fe6446775262b6695697785b2b.

GitOrigin-RevId: eb859fcf70860c06a0bd0492b746ed3ff00bc8ac
not create retry topics inside RecordConsumer builder, from wix adapter (it's already created). Removing last reference to an actual AdminClient in wix-adapter when asking to use proxy.

GitOrigin-RevId: fb1416141b6b05559b1b606576cc03556b6a78f4
GitOrigin-RevId: f178c94663c7cbcb22bd7266c3e15919d8997d8c
…33827)

* Revert "Revert "Blocking retries attempts tracking fix" (#33818)"

This reverts commit eb859fcf70860c06a0bd0492b746ed3ff00bc8ac.

* fix bug on retryAttempt resolution - ignore propagated headers

* CR change

* fix test. change retry name to correctly structured one

GitOrigin-RevId: 53fb7cf3efa315f496a00ade574397e30f20eeaf
GitOrigin-RevId: ab792e2227f389d5642802944a982f3a8d3587d8
* [greyhound] code cleanup #pr

* fix for paused partitions unrevoked in cooperative

GitOrigin-RevId: 7cf8fe80c2ac9108b3ebe72472355fe73158df9a
GitOrigin-RevId: c2a4d2c2de669eff3c98b2277be8502d834de3e4
GitOrigin-RevId: 5c9cc1ea4fcc5935c6f905a5269ac17f6b82c294
* [gh-consumers-proxy] s3 bridge #pr #skipreview

* .

* .

* .

* .

* .

* .

* .

* .

GitOrigin-RevId: fed83b505e4772a09ae189c6a01ac15368e60600
GitOrigin-RevId: 611f9b35285657e84dfaf69cf2af1744b54672e8
GitOrigin-RevId: 3cbafbea85e93a678fc7528f1b3f4bb75514b9f2
GitOrigin-RevId: fe53459fd02e8e627356cbe68d71e4d4bc6b3d24
GitOrigin-RevId: 9e6dc2bb61a9b56aa55640567702ecb66c67563a
…nd persisted) (#35207)

* make GreyhoundNG RetryConfig serializable (so can be sent over gRpc and persisted) #pr

* fix lp build

* full cross-repo

* Initial cross-repo check

* Updating cross-repo check

* Updating cross-repo check

* Updating cross-repo check

* Updating cross-repo check

* CR changes

---------

Co-authored-by: wixapiregistry <[email protected]>
GitOrigin-RevId: feaf45b3c77f9074a3445bffdd6f748d3df77056
GitOrigin-RevId: e7a2532372964be70c34350951ca99bfc52f7684
GitOrigin-RevId: af1fd4bede9f00146454b2f28657bc1c845ae682
* batch event loop fix

* fix worker timeout interruption. cleanup code

GitOrigin-RevId: 31023194ccc982dfe8720620bfbfe26dc274918c
…t exist (temporary until full config is done) (#35428)

* [greyhound-consumer-proxy] start from latest offset when group doesn't exist (temporary until full config is done) #automerge #skipreview

* .

GitOrigin-RevId: 34b953997eee3d167bb2beb23d50045daf03460f
…it (#35423)

GitOrigin-RevId: 10b761cb8fcd926178aebf586e99ed1a8829c734
* greyhound proxy retry: API + failing test + gh side

* extract and unit test current state

* remainder

* fix UT

* fix ITs

* try with increasing eventually

GitOrigin-RevId: e60cbb04d6d6b603e22954bbe4a8658ef594712a
* new kafka monitor server for proxy (currently) #automerge #skipreview

* .

* .

* .

GitOrigin-RevId: 156ff1cbde2b11cfb35259600de00b47e1012367
GitOrigin-RevId: d9758275c502278ed9fd845b3781cd92b7152f8a
…ffset from extra properties (#35845)

ConsumerIT: add a test: allow to override offsetReset with autoResetOffset from extra properties #automerge
GitOrigin-RevId: a9159b9c15445daa243e84a0b4bcaeae8db8b29d
…o.offset.reset (#35848)

GitOrigin-RevId: 7b878399acd1e25003ccae071e817ed22b2fba21
…t from extra properties taking into account a non-zero rewindUncommittedOffsetsBy (#35863)

* gh RecordConsumer - allow to override offsetReset with autoResetOffset from extra properties taking into account a non-zero rewindUncommittedOffsetsBy #automerge

* fix tests depending on com.wixpress.greyhound.GreyhoundTestingSupport

* try another tactic to fix tests

* fix build file #automerge

GitOrigin-RevId: dcf84b5740a3f99e1a637fca84c0ef6059b0e1a9
[greyhound] gaps limit calculation script #automerge

GitOrigin-RevId: 75a060f20589036570bad2077d3e906fa8db9bcd
[greyhound] parallel consumer - update gaps limit #automerge

GitOrigin-RevId: bebbfb8b314cb96c4f7ce09d79686f2163bc7a6d
berman7 and others added 26 commits September 18, 2023 15:20
… move forward to first offset (#35924)

GitOrigin-RevId: 881fe31e29499ec28bfbd5cd6f526feac6b4d363
fix build for greyhound publish to maven #automerge

GitOrigin-RevId: 65676f4cc39274c75267220d7b90e904c9d1d5d8
* [greyhound] parallel consumer - improve logging

* dummy commit

GitOrigin-RevId: 60e02e82d457076a06b0c0f2a575130051db8c69
[greyhound] parllel consumer - add logs

GitOrigin-RevId: 2d3421c1d110e20b68222e195cd82227b772dc93
GitOrigin-RevId: 5b75a27782dec7e534f72f57d3b3ba38c4194d1e
…ect (#36206)

GitOrigin-RevId: 43388b3fc6c16782373261553029c86e5111286d
GitOrigin-RevId: 3332f5e1405eba5397bfcc93174b0415f2220703
GitOrigin-RevId: 8fe84370c72e4bf864708f42ed0e6fe435cddc92
GitOrigin-RevId: 8f75ec7018fc2b8723e22f70b42c1bf7f8404b99
GitOrigin-RevId: 5b55ff311413a0ca90f0c66b7b5be0f12c8d3582
[greyhound] parallel cosnumer - add init log #automerge

GitOrigin-RevId: 8e6b0c2f3a877fe4b9805f60a98a814a144b2ae6
GitOrigin-RevId: eb0dc817fa50494422fba10994dd4382f1a58585
[greyhound] add shutdown visibility #automerge

GitOrigin-RevId: a65aaecff072d8a19e9e70f68d4da713971bd538
…#36642)

GitOrigin-RevId: aaccb23bdc18584d42b2240e5b5e63c7d9432838
GitOrigin-RevId: 84bff3d0d5156c9cd3315423becf3811fe374776
GitOrigin-RevId: cfb918350d137d551a9ca88d2e88a207e4b00fc6
* enrich commit metadata with pod and timestamp

GitOrigin-RevId: 2669ad3b6fdb31749768eda02cf4cf0872933e74
* enrich commit metadata with pod and timestamp

* fix for metrics layer

GitOrigin-RevId: 8f4f026e95f2008cfe9e958a518381f31630e6c1
GH commit metadata ts is in UTC #pr #skipreview

GitOrigin-RevId: 2efd9f9a307f625f4112ca172cd53e256bcd24f7
…#36783)

moved enrich metadata out of GH core. add it to commit on rebalance #pr #skipreview

GitOrigin-RevId: bfdcccbc4c5210870118a1d049063db157d9f771
GitOrigin-RevId: 67e32052558deb48f4373262a724d1ccd1751b82
fix retry bug - was taking all records in batch once the first record was finished backing off, resulting in too-soon retries for the rest of the batch (up to 2000 records)

GitOrigin-RevId: 0a0abfd8c905ddb7f89696655162a92021a6fcd6
* [greyhound] consumer: produceWithoutShutdown under flag #pr

Signed-off-by: Asaf Jaffi <[email protected]>

* [greyhound] consumer: produceWithoutShutdown under flag: with the actual flag condition

Signed-off-by: Asaf Jaffi <[email protected]>

---------

Signed-off-by: Asaf Jaffi <[email protected]>
GitOrigin-RevId: 1985d9423d595a449839d0ff340aa8eaba8bd053
…titions (#37048)

* failing test + refactor OffsetsInitializer

* gh-consumer-proxy - init offsets 6 minutes back for non-committed partitions #pr

GitOrigin-RevId: 30bc3e1eedecf3518233a4b509e85989a80998ca
…ell (#37095)

GitOrigin-RevId: f9b12e8030fb67924cdaaa27a57462ed256dd238
still WIP, no handling of commits. should be used in POC only.

GitOrigin-RevId: 1f9b561e9de9df62d2a4863bf0bdf7fccbf1ff26
@berman7
Copy link
Collaborator Author

berman7 commented Sep 18, 2023

#autorebase

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants