what happens when you call Session.sendToTarget while resend is in progress?

[QwertGold]

We recently encountered a network outage, and after QFJ detected missing heartbeat it disconnected, and reconnected.
After logon both parties issue resend request, to get the missing messages.
While QFJ is resending, we get a new order*, and call Session.sendToTarget, as we would normally do.

incoming: Resend Request ResendRequest, BeginSeqNum=15703
outgoing: New OrderSingle seqNum=15703 possDup=true
.... resending message ins sequence
outgoing: New Order Single seqNum =15830 possDup=true
outgoing: New Order Signle seqNum = 15862*
outgoing: New Order Single seqNum =15831 possDup=true
incoming: Resend Request BeginSeqNum=15831

My incoming messages are logged from Application::toApp and the received resends from Application::fromAdmin

This has lead me to the conclusion that QFJ, does not prevent the sending of messages while a resend is in progress. In my case the remote party does not cache the out of sequence messages, instead they issue a new resend request. I have not read the fix specification, so my question is.

1. Should the Client delay sending the message until resend is complete, or
2. Should the server store out of sequence messages during resend, and then process them in-order later.

Any info would be much appreciated.

Sounds a bit like this question I found on Stack Overflow

[chrjohn]

This has lead me to the conclusion that QFJ, does not prevent the sending of messages while a resend is in progress.

Correct, there is no reason to not send messages in that case.
As I commented on that StackOverflow question there does not seem a strict requirement to cache out-of-sequence messages, however it is recommended to cache messages.

The relevant details are here: https://www.fixtrading.org/standards/fix-session-layer-online/#message-recovery e.g. in figure 13.

Especially see here: https://www.fixtrading.org/standards/fix-session-layer-online/#fix-session-state-matrix points 11 and 12 where it is stated to queue messages when in resend processing.

One thing to note, however, is that it is also recommended to wait a short time after the connection was established before transmitting new or queued messages.

The initiator and acceptor should wait a short period of time following receipt of the Logon(35=A) message from the counterparty before transmitting queued or new application messages to permit both sides to synchronize the FIX session.

Alternatively, the initiator and acceptor may send a TestRequest(35=1) message and wait for the HeartBeat(35=0) message sent in response to the TestRequest(35=1) message, before sending queued or new messages, as this practice forces both sides to perform message synchronization.

This is no QFJ built-in behaviour but could be easily implemented in your application by sending a TestRequest with a specific ID (e.g. timestamp) and only start your message transmission after the corresponding Heartbeat reply has been received.

[QwertGold]

Thanks a lot.
I did not realize that the state matrix covered this point 12 (This is probably the exact argument I need, for the outage discussion with the acceptor implementer).

In our case, the acceptor we are connecting to, does not queue out of sequence messages, so they will send a new ResendRequest. I don't know if TestRequest would work in our case, since it may also be considered out of sequence.
The solution we ended up with, is to not send any new (application) messages until we have sent the first HeartBeat, so we just look at the toApp callback to see what that is sent - @chrjohn, do you see any issues with that approach?

[chrjohn]

I think it is a good workaround.
You could fine-tune it by storing the seqnum of your last app message and start sending your new app messages when you send out a PossDup message with your last app message's seqnum. That way you won't have to wait for the first Heartbeat.
Edit: you would also need to store the seqnums of outgoing Reject messages since they could also be resent.

[QwertGold]

cool, I will keep that optimization in mind.
Is the rule about Reject, because it is the only admin level messages which is also resent? whereas Logon and HeartBeat are not?

Right now I think we can live with 30 second wait, if the alternative is an hour long resend loop. The resend loops cause the receiver of NewOrderSingle to reject orders because of the possDupe flag. I don't understand why they would do that, even if I send the same message 10 times. In my mental model the receiver of a fix message should behave something like this:

The reject at the bottom, is not Admin reject, but the appropriate application level reject, like ExecutionReport with ordStatus/execType Rejected, or OrderCancelReject. @chrjohn is than in line with you experience, or is there something I'm overlooking?

[chrjohn]

Is the rule about Reject, because it is the only admin level messages which is also resent? whereas Logon and HeartBeat are not?

Exactly.

W.r.t. PossDup: from my experience there are counterparties which act upon that field and some don't.
Especially on orders there is the decision to make if the counterparty still wants to send it to the market. The order might be considered stale in the meantime, prices have moved etc... So it is not always a simple yes/no decision. Some counterparties allow for a timeout after which an order is considered stale.

Hope that helps.