Skip to content

Commit

Permalink
Instrument cleanup tracer to log weird volume removal flake
Browse files Browse the repository at this point in the history
Debug for #23913, I though if we have no idea which process is nuking
the volume then we need to figure this out. As there is no reproducer
we can (ab)use the cleanup tracer. Simply trace all unlink syscalls to
see which process deletes our special named volume. Given the volume
name is used as path on the fs and is deleted on volume rm we should
know exactly which process deleted it the next time hopefully.

Signed-off-by: Paul Holzinger <[email protected]>
  • Loading branch information
Luap99 committed Oct 30, 2024
1 parent f139bc1 commit d633824
Show file tree
Hide file tree
Showing 2 changed files with 16 additions and 1 deletion.
14 changes: 14 additions & 0 deletions hack/podman_cleanup_tracer.bt
Original file line number Diff line number Diff line change
Expand Up @@ -149,3 +149,17 @@ tracepoint:syscalls:sys_enter_write
$offset += $len
}
}

// HACK: debug for https://github.com/containers/podman/issues/23913
// The test uses "ebpf-debug-23913" volume name and because and volume rm
// will delete the path we can trap the process here to find out who actually
// deletes it.
tracepoint:syscalls:sys_enter_unlink*
/ strcontains(str(args.pathname), "ebpf-debug-23913") /
{
printf("Special issue 23913 volume deleted by pid %d: ", pid);
// This can fail to open the file it is done in user space and
// thus racy if the process exits quickly.
cat("/proc/%d/cmdline", pid);
print("");
}
3 changes: 2 additions & 1 deletion test/system/600-completion.bats
Original file line number Diff line number Diff line change
Expand Up @@ -270,7 +270,8 @@ function _check_no_suggestions() {
random_image_name="i-$(safename)"
random_image_tag=$(random_string 5)
random_network_name="n-$(safename)"
random_volume_name="v-$(safename)"
# Do not change the suffix, it is special debug for #23913
random_volume_name="v-$(safename)-ebpf-debug-23913"
random_secret_name="s-$(safename)"
random_secret_content=$(random_string 30)
secret_file=$PODMAN_TMPDIR/$(random_string 10)
Expand Down

0 comments on commit d633824

Please sign in to comment.