Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mysqlctl: Improve handling of the lock file #15404

Merged
merged 1 commit into from
Mar 7, 2024
+18 −2
Merged

mysqlctl: Improve handling of the lock file #15404

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion go/vt/mysqlctl/mysqld.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -460,6 +460,10 @@
log.Errorf("%v: error parsing pid from lock file: %v", ts, err)
return err
}
if os.Getpid() == p {
log.Infof("%v: lock file at %s is ours, removing it", ts, lockPath)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd take the opportunity to rename ts as ts == *topo.Server in the code base. :-)

No idea why that was chosen for:

ts := fmt.Sprintf("Mysqld.Start(%v)", time.Now().Unix())

time start, I guess? Up to you if you want to change it.

return os.Remove(lockPath)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really want to do this? I'm curious how this happens, where we have the same PID we had before? Or am I missing something. That being said, I suppose it's fine since eventually the PID will get reused and could be reused by another mysqld process. In that case though I'm also not sure what "ours" really means in that context.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattlord It can happen inside containers. Each container is unique and gets new PIDs so it's possible there to have PID reusage much more often.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, when using persistent storage for the lock file that lives across containers. Makes sense. Thanks!

}
proc, err := os.FindProcess(p)
if err != nil {
log.Errorf("%v: error finding process: %v", ts, err)
Expand All @@ -469,7 +473,13 @@
if err == nil {
// If the process still exists, it's not safe to
// remove the lock file, so we have to keep it around.
log.Errorf("%v: not removing socket lock file: %v with pid %v", ts, lockPath, p)
cmdline, err := os.ReadFile(fmt.Sprintf("/proc/%d/cmdline", p))
if err == nil {
name := string(bytes.ReplaceAll(cmdline, []byte{0}, []byte(" ")))
log.Errorf("%v: not removing socket lock file: %v with pid %v for %q", ts, lockPath, p, name)
} else {
log.Errorf("%v: not removing socket lock file: %v with pid %v (failed to read process name: %v)", ts, lockPath, p, err)

Check warning on line 481 in go/vt/mysqlctl/mysqld.go

View check run for this annotation

Codecov / codecov/patch

go/vt/mysqlctl/mysqld.go#L481 

Added line #L481 was not covered by tests
}
return fmt.Errorf("process %v is still running", p)
}
if !errors.Is(err, os.ErrProcessDone) {
Expand Down
8 changes: 7 additions & 1 deletion go/vt/mysqlctl/mysqld_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -193,8 +193,14 @@ func TestCleanupLockfile(t *testing.T) {
assert.NoError(t, cleanupLockfile("mysql.sock", ts))
assert.NoFileExists(t, "mysql.sock.lock")

// If the lockfile exists, and the process is found, we don't clean it up.
// If the lockfile exists, and the process is found, but it's for ourselves,
// we clean it up.
os.WriteFile("mysql.sock.lock", []byte(strconv.Itoa(os.Getpid())), 0o600)
assert.NoError(t, cleanupLockfile("mysql.sock", ts))
assert.NoFileExists(t, "mysql.sock.lock")

// If the lockfile exists, and the process is found, we don't clean it up.
os.WriteFile("mysql.sock.lock", []byte(strconv.Itoa(os.Getppid())), 0o600)
assert.Error(t, cleanupLockfile("mysql.sock", ts))
assert.FileExists(t, "mysql.sock.lock")
}
Loading