Align API for Mailboxes to how other middleware works

[rogeralsing]

In actors we have ReceiverMiddleware and SenderMiddleware.
In Mailboxes we have MailboxStatistics (which was the very first move towards observability in ProtoActor)

This API is named differently, and structurally different than the middleware.
Should and Could we align these two?

cc @AsynkronIT/developers

[some1one]

Hi, I would like to use protoactor in a lightweight event sourcing client. I could potentially take this on as a task. However, since I am not thoroughly familiar with the project, I would need at least some kind of direction on how it should be implemented. For example would it only have MailboxMiddleware or would there be multiple middle-wares? I know could potentially create a custom queue for this but I feel like filtering out duplicate events would be better as middle-ware.

Reasoning:
I want to avoid orleans because it is definitely not lightweight and over complicated for a front-end client that isn't worried about networking (because it's handled by the azure api). I also have heard many complaints about production implementations with it.

I will most likely be making my own providers for Azure EventHubs, EventGrid, IoT Hub, and Functions (for reading the persisted event hubs data and snapshots as the others are live events only) with multiple actors based on some kind of data partitioning and state hierarchy. Because of this, making sure every actor and provider is idempotent in an at-least-once delivery system would be tedious and repetitive. It would be much easier if I had a mailbox implementation that took care of that itself. It sounds like this could be implemented with middle-ware instead of creating a completely new mailbox implementation.

So um, thoughts? Suggestions?

[alexeyzimarev]

Idempotency is not something to handle in a middleware. The best way to do it is to keep ids of processed events (a limited number of them) in the actor itself. We do it all the time and it works really well.

I am not sure how do you plan to persist events in EventHub. It's just a transport that gives you a limited ability to replay. After max 7 days, things disappear.

We do have a production system built with Proto.Actor that consumes events from EventHub. However, we used Azure Tables to persist actors, both events and snapshot. That works fine unless you have billions of events per month like we do.

P.S. It might've been a good idea not to go off-topic for the original issue and open a new issue...

[some1one]

I apologize, while I disagree with some of what you said, I did not mean to start an architecture conversion. I still want to take this on so at least extending mailboxes would be clearer, cleaner, and more in line with the rest of the API.

Fyi, EventsHubs can store partition files into Data Lake Storage before clearing via a setting called EventHubs Capture. This can be read much faster than table storage via Stream Analytics but I prefer Functions because it's cheaper with a slight development cost.

[alexeyzimarev]

I know about EventHub Capture. It shovels all events as Avro files to DataLake, making folders per day.

Event sourcing in Proto.Actor requires the ability to restore a single entity state from events and/or latest snapshot. There's no way it can be done using DataLake.

Clemens Vasters clearly described the aim of each messaging solution in Azure. None of them is considered as persistent storage. EventHub, in particular, serves the purpose of ingestion of events on scale.