FailureCallback does not expose the Exception that caused the failure

[staff0rd]

Does this issue relate to a new feature or an existing bug?

• Bug
• New Feature

Please describe the current behavior?
FailureCallback doesn't expose the Exception causing the failure

Please describe the expected behavior?
FailureCallback could expose the Exception that caused the failure

I came across this myself, and then saw a similar question on Stackoverflow. Is there any appetite for merging something that would expose the exception, for example, something like this?

[mivano]

The purpose is to somehow store the lost events to some other location, not directly handle the underlying exception. That exception will be the same for all the event that suffered this problem while being stored to Elasticsearch. Like a connection exception.
What is your use case that you want to handle this exception as well?

[jdiosdado]

The purpose is to somehow store the lost events to some other location, not directly handle the underlying exception. That exception will be the same for all the event that suffered this problem while being stored to Elasticsearch. Like a connection exception.
What is your use case that you want to handle this exception as well?

There can be multiple causes on why the logs are not being stored, how would someone fix the issue if there is no error data?
You are right, the exception will be the same but in order to research how to fix one needs to know what the actual issue is.

[bferdinandus]

+1
I would also like to see the exeption in the failure callback. That would simplify the debugging process as to discover why an event could not be send to elastic. Now I had to make use of a web debugging proxy (Charles)

The logevent even already has an "exeption" property, but it does not get populated.

[iamitdubey]

+1
Need this to be able to know the reason for failed events

[Towmeykaw]

My reason for wanting this is to be able to filter out the failed events which needs to be handled. Like mapper_parsing_exception which I would like to escalate but then ignore stuff like connection errors

[Towmeykaw]

@mivano Is there a chance to add this feature?

[mivano]

Looks like something useful based on the feedback. Care for a PR?

[Rombersoft]

Hello everybody. I'm in shock! Why is not this problem resolved yet? Do not you have two years enough to fix it?

[mivano]

I have merged the mentioned PR, so that should give you a beta package at least.

Regarding the tone of your message; as far as I know, you are not paying for support, and this is part of the deal of using open-source components. See all the reasoning here: #486

So sorry that you are shocked that this took two years, but do remember this is not my full-time job to maintain an open source framework and do the unpaid tasks for everybody that wants something...