Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add processor (?and row?) information to exception object #126

Open
cschloer opened this issue Mar 31, 2020 · 1 comment · Fixed by #134
Open

Add processor (?and row?) information to exception object #126

cschloer opened this issue Mar 31, 2020 · 1 comment · Fixed by #134

Comments

@cschloer
Copy link
Contributor

Would it be possible to add the index (relative to the top level Flow) and name of the flow that failed when an exception is thrown in dataflows? I'm imagining something that intercepts every exception that comes out of running each flow, adds the relevant information either to the stack trace or the exception object, and then raises it again.

This is especially useful if you have multiple load steps and don't know which one is failing. An easy workaround would be to run them one at a time but this would make it a bit easier to work with :)

One step further would also be to find out which row # triggered an error. Obviously wouldn't be relevant for some processors, but for those that do manipulate at a row level it would be a significant help in debugging a failing dataflow.

@cschloer
Copy link
Contributor Author

To be clear, what I'm talking about already exists with ValidationErrors (https://github.com/datahq/dataflows/blob/master/dataflows/base/schema_validator.py#L6) but it would be great if it could be extended beyond just validation and also the actual running of the flows.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant