Fail and delay batch items #6
Conversation
AzureBatchQueue/TimerBatch.cs
Outdated
| FlushIfEmpty(remaining); | ||
| } | ||
|
|
||
| public void Delay(BatchItemId itemId, TimeSpan delayTime) |
There was a problem hiding this comment.
I am unsure about exposing such a specific API in a general-purpose queue. It smells a bit.
Maybe it's better to implement delays in place where they are needed. I.e. by putting data that need to be delayed as a separate message into the queue.
Imagine a case when we receive a batch of mentions that we want to delay (e.g. some news from the provider). And those mentions have different delay times. We will try to process same queue message several times then (as each time, we will delay the rest of the unprocessed mentions). Eventually, the queue message might go to quarantine because of exceeding the dequeue count.
I would leave delay functionality to library clients and try to keep the batch queue as simple as possible (only cover cases that are a must for all the scenarios).
There was a problem hiding this comment.
I don't think the concept of Delay is a specific case in a general-purpose queue. What we do here is basically use the visibility timeout feature of the AzureQueue. But the implementation of Delay in my case is really not thought well-enough and needs a lot more details, so I will move it to the clients.
This reverts commit 1959ab0.
Implement Fail functionality of batch items
Fail
When only failed items left -> update message with these items.
When flush is triggered by timeout -> update message with both failed and not processed items.
Delay
When one or more items are delayed -> update message with failed and not processed items with a minimum visibility timeout present.
We consider a situation when a batch is triggered by timeout together with a visibility timeout a very rare situation. So we don't care that other items in a batch are affected by a visibility timeout of another item.