Producer errors when using async facade

[MafcoCinco]

I'm not sure this is an issue. It could be something that is already possible but I'm not aware how to handle it. Basically, how does one receive errors (or success messages) when using the async facade for the producer? Looking at https://github.com/pyr/kinsky/blob/master/src/kinsky/async.clj#L309, it seems the result of the call to send! is lost in the future loop.

Though I have never used it before, it seems that the async facade accepts a response channel. Would it be appropriate to use this to send the result, if it is supplied by the user? The primary error that I have run into on the producer side is limitations on message size and it is pretty important in my domain to know when this happens.

I'm happy to put a PR together for this change, provided it is necessary. Just wanted to confirm that it is necessary and that there are currently no means to get the response from the call to send!.

Thanks!

[codingkarthik]

I face the same issue with the kinsky producer. Although, I have noticed that if the message size is too big I get a FutureFailure in the object returned. It would be great if an exception is returned.

[joefromct]

I see this as well.

I was hoping based on the comment for #23 errors such as org.apache.kafka.common.errors.RecordTooLargeException would bubble up as well...

[pyr]

I am inclined to retire the core.async facade. It has lagged between the standard consumer facade and it's not code that sees production use on our side

[fmjrey]

I've been working on a fork with the improvement of having operation responses sent to the out channel or to an optional response-ch: https://github.com/fmjrey/kinsky/tree/fmjrey2

Warning: work in progress, I may force push on that branch, so use at your own risk.

The async tests I wrote are passing and now I just need to update my using project to better validate what I've done. It's not battled tested though, however it makes this async facade more functional and usable, so feedback welcomed.

If this async part of kinsky gets dropped I'll make it a separate lib instead of a PR. I still think it's worth having some async facade to kafka, in my case it helps me switch from kafka to simple channels in order to have a faster workflow and test suite.

[pyr]

Hi @fmjrey ,

I think a library on top of kinsky which would revive the async facade would be nice, we're bound to continue supporting the base library but unlikely to use core.async interoperability and thus we would be bad maintainers of the facade. It is now removed on master and I'm happy to point to a separately maintained core.async extension.

[fmjrey]

Hi @pyr,
I've created a separate lib for the async facade: https://github.com/fmjrey/kinsky-async
I also created PR #57 in this repository to support it.