Logs are too verbose and generate high amount of data

[christian-roggia]

First of all, I would like to thank you for the amazing work you did here, outstanding!

This ticket is related to issues we encountered in a cloud environment, and more specifically in Kubernetes deployments.

A little background about our setup: we are migrating around 10 million records from ETCD to CETE for a total of ca. 30 GB of data. The data migrated contains JSON, HTML, and other formats.

I would like to address the issue of logs as currently too much traffic/data is being generated and logs are too verbose:

• Readiness / Liveness check should not log anything unless the log level is set to DEBUG or anything similar. (NOTE: In a kubernetes environment checks are executed regularly every 1-5 seconds).
• Set / Get / Delete / etc. operations should not generate a log for each change executed - that is a DEBUG log, to give some context when we launched the first migration we have seen over 50 MB of logs generated per second. We have been forced to set the log level to WARNING to avoid high costs for logs ingestion by our cloud provider, which is not ideal.
• NOT FOUND errors should be less verbose (there are 3 logs generated for each failed lookup, one of which contains the stacktrace). I am not even sure whether a NOT FOUND error should be logged at all - especially with level ERROR - as it is the expected behavior and the server did not encounter any error, the resource was simply missing.

I will update this ticket with more information about other logs as we encounter them.

[mosuka]

Sorry it took me so long to reply.
Thanks for the valuable feedback!
I'll fix it when I have time.

Thanks!

[christian-roggia]

Thank you for your answers and don't worry about the late reply - it was the weekend after all.

I would like to propose a PR for the logs, I think we found a good balance between too many logs and too few.
You might wanna take a look at these two specific commits animeshon@7041f8a animeshon@b1edde6

If it's fine for you, I am gonna go ahead and open a WIP pull request for the logs.