busy runner pod is deleted by recreate feature

[masa213f]

What

If the timing is bad, the recreate feature will delete busy runner pods.

2022-01-31 10:27:48    {"level":"info","ts":1643592468.2755053,"logger":"controllers.RunnerManager","msg":"deleted runner pod that exceeded recreate deadline","runnerpool":"<Namespace>/<RunnerPool>","pod":"<Namespace>/<RunnerPod>"}
2022-01-31 10:29:48    {"level":"error","ts":1643592588.5469356,"logger":"controllers.RunnerManager","msg":"failed to remove runner","runnerpool":"<Namespace>/<RunnerPool>","runner":"<Runner>","runner_id":59,"error":"DELETE https://api.github.com/repos/***/***/actions/runners/NN: 422 Bad request - Runner \"<Runner>\" is still running a job\" []","stacktrace":"github.com/cybozu-go/meows/controllers.(*manageProcess).run\n\tgithub.com/cybozu-go/meows/controllers/runner_manager.go:256\ngithub.com/cybozu-go/meows/controllers.(*manageProcess).start.func1\n\tgithub.com/cybozu-go/meows/controllers/runner_manager.go:223\ngithub.com/cybozu-go/well.(*Environment).Go.func1\n\tgithub.com/cybozu-go/[email protected]/env.go:133"}
2022-01-31 10:29:48    {"level":"error","ts":1643592588.5470736,"logger":"controllers.RunnerManager","msg":"failed to run a runner manager process","runnerpool":"<Namespace>/<RunnerPool>","error":"DELETE https://api.github.com/repos/***/***/actions/runners/NN: 422 Bad request - Runner \"<Runner>\" is still running a job\" []","stacktrace":"github.com/cybozu-go/well.(*Environment).Go.func1\n\tgithub.com/cybozu-go/[email protected]/env.go:133"}

How

Describe how to address the issue.

Checklist

• Finish implementation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions