Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(scheduler): add error logging on run interruption #3766

Merged
merged 1 commit into from
Mar 27, 2024

Conversation

karol-kokoszka
Copy link
Collaborator

Fixes #3764

It contains simple implementation of ErrorLogListener that embeds nopListener but overrides OnRunError, OnRunWindowEnd and OnRunStop to log errors that were silenced before.


Please make sure that:

  • Code is split to commits that address a single change
  • Commit messages are informative
  • Commit titles have module prefix
  • Commit titles have issue nr. suffix

@karol-kokoszka karol-kokoszka force-pushed the log_errors_after_task_execution branch from 7698860 to 214ab08 Compare March 26, 2024 11:09
@Michal-Leszczynski
Copy link
Collaborator

@karol-kokoszka PR looks good, but do you have some sample logs to validate that it works as expected?

@karol-kokoszka
Copy link
Collaborator Author

@Michal-Leszczynski here it is:

17:06:25.001    ERROR   scheduler.40f7fa02      OnRunError      {"key": "69cff263-cc2a-4a9e-8233-bf9bed9495e1", "retry": 0, "error": "put run: gocql: no hosts available in the pool", "errorStack": "github.com/scylladb/scylla-manager/v3/pkg/service/scheduler.(*Service).run\n\t/home/karkok/dev/scylla-manager/pkg/service/scheduler/service.go:413\ngithub.com/scylladb/scylla-manager/v3/pkg/scheduler.(*Scheduler[...]).asyncRun.func1\n\t/home/karkok/dev/scylla-manager/pkg/scheduler/scheduler.go:404\nruntime.goexit\n\t/home/karkok/go/go1.21.1/src/runtime/asm_amd64.s:1650\n"}
github.com/scylladb/go-log.Logger.log
        /home/karkok/dev/scylla-manager/vendor/github.com/scylladb/go-log/logger.go:101
github.com/scylladb/go-log.Logger.Error
        /home/karkok/dev/scylla-manager/vendor/github.com/scylladb/go-log/logger.go:84
github.com/scylladb/scylla-manager/v3/pkg/scheduler.errorLogListener[...].OnRunError
        /home/karkok/dev/scylla-manager/pkg/scheduler/listener.go:86
github.com/scylladb/scylla-manager/v3/pkg/scheduler.(*Scheduler[...]).onRunEnd
        /home/karkok/dev/scylla-manager/pkg/scheduler/scheduler.go:420
github.com/scylladb/scylla-manager/v3/pkg/scheduler.(*Scheduler[...]).asyncRun.func1
        /home/karkok/dev/scylla-manager/pkg/scheduler/scheduler.go:405

@karol-kokoszka karol-kokoszka merged commit 6b38cea into master Mar 27, 2024
19 of 21 checks passed
@karol-kokoszka karol-kokoszka deleted the log_errors_after_task_execution branch March 27, 2024 16:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fix error handling in Scheduler
2 participants