Skip to content
This repository has been archived by the owner on Jan 20, 2022. It is now read-only.

Add output batching support to Intermediate flatMap/source #461

Open
johnynek opened this issue Feb 27, 2014 · 0 comments
Open

Add output batching support to Intermediate flatMap/source #461

johnynek opened this issue Feb 27, 2014 · 0 comments

Comments

@johnynek
Copy link
Collaborator

In storm, batching up a big block of items into a single tuple has been shown to improve performance by reducing contention on the queues.

The same trick could improve throughput between intermediate flatMaps by not flattening the results, but instead emitting lists.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant