Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(block-producer): improve mempool config #543

Merged
merged 7 commits into from
Nov 29, 2024

Conversation

Mirko-von-Leipzig
Copy link
Contributor

@Mirko-von-Leipzig Mirko-von-Leipzig commented Nov 5, 2024

This PR improves mempool configuration and simplifies the block-producer errors exposed during batch and block building.

Mempool configuration is now separated out into a builder, and constraint support is added as a sort of batch/block budget that gets exhausted during the batch/block selection. In other words, batches/blocks produced by the mempool are guaranteed to adhere to the limits placed on them, removing the need for asserting these errors in the building process.

I've removed the transaction data passing in the error variant, which was used by the previous FIFO implementation to return the failed transactions into the pool. This is no longer required as the mempool retains a copy of all inflight transactions.

We may still want to perform these checks in the builders regardless; though I'd prefer only doing this in a single place.

This completes some tasks in #519

Copy link
Contributor

@polydez polydez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thank you! I've left a couple of nit comments.

crates/block-producer/src/mempool/mod.rs Outdated Show resolved Hide resolved
crates/block-producer/src/mempool/batch_graph.rs Outdated Show resolved Hide resolved
while let Some(batch_id) = self.inner.roots().first().copied() {
// SAFETY: Since it was a root batch, it must definitely have a processed batch
// associated with it.
let batch = self.inner.get(&batch_id).unwrap().clone();
Copy link
Contributor

@polydez polydez Nov 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have no such limitations in our coding style yet, but I think, we should avoid using unwraps in favor of expect. We can put safety description in panic message here. Even if it panics, the user/developer will have info why it panicked.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've gone back and forth on unwrap vs expect a few times. expect should indicate what you expected to happen, and will mostly be shorter than the SAFETY comment I would write instead.

To me the two aren't quite the same i.e. will contain different strings. Or put differently, I prefer having a lengthy SAFETY paragraph instead of a one-line expect message.

Though maybe I should just be doing both.

What I will say is that with expect I often find myself writing/copying the same message in various places which sort of negates identifying the actual point-of-failure. In which case we should be running with BACKTRACE=1 always so we get actual line and panic info. There isn't really a good reason not to do this.


cc @bobbinth for his style guide input :D

crates/block-producer/src/mempool/mod.rs Outdated Show resolved Hide resolved
crates/block-producer/src/mempool/mod.rs Outdated Show resolved Hide resolved
crates/block-producer/src/mempool/mod.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@bobbinth bobbinth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Thank you! I left a couple of small comments inline.

In general, I think this mechanism should work well. We may need to make some adjustments based on where we land with 0xPolygonMiden/miden-base#973, but the overall approach will probably remain the same.

crates/block-producer/src/mempool/mod.rs Outdated Show resolved Hide resolved
crates/block-producer/src/mempool/mod.rs Outdated Show resolved Hide resolved
crates/block-producer/src/mempool/mod.rs Outdated Show resolved Hide resolved
@Mirko-von-Leipzig
Copy link
Contributor Author

@polydez I've added some expect messages, and rebased. Let me know if its good now.

Address more review comments

Minor cleanup
Copy link
Contributor

@polydez polydez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks good to me, thanks for addressing review comments!

@Mirko-von-Leipzig Mirko-von-Leipzig merged commit 312d06a into next-block-producer Nov 29, 2024
8 checks passed
@Mirko-von-Leipzig Mirko-von-Leipzig deleted the mirko-cleanup-mempool branch November 29, 2024 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants