Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

logs are forwarded to a processor in slot and trx order #953

Merged
merged 8 commits into from
Dec 19, 2024

Conversation

EasterTheBunny
Copy link
Contributor

Description

Log ingestion heavily leverages async go-routines but LogPoller will rely on logs delivered in slot/block order. This commit adds a new component to the log ingestion to order logs before forwarding them to LogPoller.

@EasterTheBunny EasterTheBunny force-pushed the etb/ordered-log-processing branch from 094f9cb to 72089b2 Compare December 2, 2024 16:44
@EasterTheBunny EasterTheBunny force-pushed the etb/ordered-log-processing branch from f407d8f to f9824d9 Compare December 4, 2024 18:40
@EasterTheBunny EasterTheBunny marked this pull request as ready for review December 4, 2024 19:02
@EasterTheBunny EasterTheBunny requested a review from a team as a code owner December 4, 2024 19:02

func (p *orderedParser) sendReadySlots() error {
// start at the lowest block and find ready blocks
for element := p.blocks.Front(); element != nil; element = element.Next() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
for element := p.blocks.Front(); element != nil; element = element.Next() {
for element := p.blocks.Front(); element != nil; element = p.blocks.Front() {

To drop

                temp := element.Prev()

		if temp == nil {
			break
		}

		element = temp

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also due to break, there is a leak of expectations.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh ok. I was just using the iteration method in the docs. This works better. Thanks

Copy link
Contributor

@jadepark-dev jadepark-dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@aalu1418 aalu1418 merged commit 2433636 into develop Dec 19, 2024
36 checks passed
@aalu1418 aalu1418 deleted the etb/ordered-log-processing branch December 19, 2024 18:22
dhaidashenko pushed a commit that referenced this pull request Dec 20, 2024
* logs are forwarded to a processor in slot and trx order

* make tests pass again

* simplify block and expectation ordering
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants