You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In order to support higher throughput for EventHorizon, it can be rewritten to support batching.
This requires updating the contract to support it, and leveraging them in the consumer & producer infrastructure in the runtime.
Design requirements
Backwards compatibility
New versions of the runtime should be able to communicate as normal with older ones and vice versa. This can be solved by adding optional fields to the subscription request, which will only be recognized by runtimes running the updated version. Thus it can select the batch mode if the consumer asks for it, and publish the old message types if not.
Resource limits
Since we are making the producer a lot faster with sending batches, the consumer will need to be able to limit work in progress. This can be done by specifying batch sizes & using bounded channels to prevent a slow DB from causing OOM-issues.
The text was updated successfully, but these errors were encountered:
Description
In order to support higher throughput for EventHorizon, it can be rewritten to support batching.
This requires updating the contract to support it, and leveraging them in the consumer & producer infrastructure in the runtime.
Design requirements
Backwards compatibility
New versions of the runtime should be able to communicate as normal with older ones and vice versa. This can be solved by adding optional fields to the subscription request, which will only be recognized by runtimes running the updated version. Thus it can select the batch mode if the consumer asks for it, and publish the old message types if not.
Resource limits
Since we are making the producer a lot faster with sending batches, the consumer will need to be able to limit work in progress. This can be done by specifying batch sizes & using bounded channels to prevent a slow DB from causing OOM-issues.
The text was updated successfully, but these errors were encountered: