Receive notification of acknowledgments when using async publish

[MattBrittan]

Is your feature request related to a problem? Please describe.

When using async PUBLISH (i.e. PublishWithOptions with PublishMethod_AsyncSend) it's not currently possible to determine when the message has been acknowledged (with QOS 2 messages it would be useful to know when both steps are complete).

Describe the solution you'd like
This was discussed a bit in issue #207 options include:

One way to achieve this is to change AddToSession() in SessionManager to return a function that will block until the ack is received. The paho client would just return that function to the user. The function would be valid for use for as long as the session is.

My thought on this was to add callbacks to PublishOptions; we could then store the PublishOptions in state.clientPackets (making it easy to call when an associated event occurs). I prefer the callback approach because I think it's easier for the end user (and avoids the need to have a lot of go-routines hanging around waiting for acknowledgements). One note of caution is that the session may outlast the client instance (and I can't think of a good way of handling that!).

[MattBrittan]

Carrying on discussion from #207 comment by @vishnureddy17

One note of caution is that the session may outlast the client instance (and I can't think of a good way of handling that!).

I think that's expected and necessary. As a user, I'd want to know when the publish was handled in the session (regardless of how many reconnections happened in the interim). In the solution you suggest, that would mean that the callback can potentially be called long after a particular client is dead.

Just to clarify; by "client instance" I meant the users program (rather than an instance of paho/autopaho). As the session can be stored on disk it's quite possible for the following to happen:

1. User publishes message and it's added into session (or queue in autopaho)
2. Client application restarts (say a server restart or application update)
3. Message actually sent

This raises a few questions:

• Is the callback called when the application is shutting down (only possible on a clean shutdown obviously)?
• Does the library provide a way to 'reregister' the callbacks (or whatever mechanism is used) when starting up?

My thought was always that if autopaho is in use then whatever notification method is selected should survive reconnections (these should be mostly hidden from the end user). We do also need to handle the situation where the session is lost (either due to Clean Start or the session expiring) so probably want to be able to pass an error to the user when this happens.

[vishnureddy17]

Does the library provide a way to 'reregister' the callbacks (or whatever mechanism is used) when starting up?

This is an interesting issue. Currently, the SessionManager interface doesn't expose a way to get the un-acknowledged messages that are in the session. So on an application restart, the application has no way of knowing which messages to re-register callbacks for. In other words, the application doesn't have knowledge of the messages that are pending. A similar problem exists for Queue in autopaho.

Another issue is that even if the library provided a way to re-register callbacks, the user would have to know to register it before a connection is established since messages get resent in ConAckReceived.

One possible solution is to have a callback for "orphaned" messages. However, this callback may be of little use since the callback is for publishes that the application has lost track of. This could perhaps be mitigated by including the re-sent packets as part of arguments supplied to the callback.

I think the problem of application restarts does not need to be solved immedidately. The solution for non-restarting applications is clearer, so it may be a better idea to solve that problem first.

My thought was always that if autopaho is in use then whatever notification method is selected should survive reconnections (these should be mostly hidden from the end user). We do also need to handle the situation where the session is lost (either due to Clean Start or the session expiring) so probably want to be able to pass an error to the user when this happens.

Agree with this.

[MattBrittan]

Have been thinking about this following PR #264 (not yet accepted). The issue here is that packets are stored (to memory/disk) in two places (queue and session store), so in order to notify the client that a message has been sent we need to store some kind of ID so that the specific message can be identified and tracked through the queue/store. @shaileshahuja has had a go at this for the queue by adding a UUID that's maintained in a seperate map (or part of the filename fot the filestore). This would work but requires a range of changes and I had some concerns about requiring a UUID.

I wonder if an alternative option is to make it the users responsibility to provide the ID and:

• Have the user add a property (i.e. 0x26 User Property with a known identifier (i.e. PahoID)
• Add callbacks that notify the user that the ID has passed various milestones (i.e. In session, transmitted, complete)
• For QOS2 copy the property into the PUBREL packet before sehdning this to the session store (so it's available when the PUBCOMP is received)
• Remove the property just before the message is transmitted (potentially optional).

This would simplify the changes required (would be minimal) whilst providing the notification required. It would also enable the authors of stores (e.g. a databased queue) to extract the ID into a different database field if they want (parse the packet). It's definitly not perfect but I think it's a workable solution so thought I'd put it out there to see what others think.