Add an option to disable enforcing of force_mask='700' for network file systems #1839
Comments
|
@giuseppe PTAL The problem with network file systems is that the remote side does not understand user namespace, From its point of view it sees UID 1000 attempting to chown a file to UID 100001 for example when doing a When we run fuse-overlayfs on top we can change the underlying file system to not chown the file on the remote server but to only change an Xattr on the file, which would be allowed on the server side. The fuse-overlayfs file system then would report the changed UID on the client side to the container. Don't know if BeeGFS has similar support or your fuse file system. |
|
Hi, I did some tests with rootlesskit on a BeeGFS directory with a non-privileged user. I was able to change ownership of a file in that directory to one of my subuids: $ cd /mnt/beegfs
$ touch test
$ rootlesskit chown 1:1 test # Works
$ ls -l
-rw-rw-r-- 1 1410720 1410720 0 Feb 20 12:31 testSo to me it seems like BeeGFS understands subuids and thus does not require any special treatment. In fact, setting xattr does not seem to be supported. So I think having an option to switch this behavior off would be beneficial. Best, |
|
can you show the output for |
|
Hi Guiseppe,
Best, |
|
thanks, could you try that patch anyway? I've no access to a beegfs file system |
and sorry, I've messed up the command. I think we need |
|
Okay, I need to figure out how to build podman first. I will get back to you. |
|
|
great, then I think the patch above should work well |
|
So I just added the patch and built v4.9.3, which seems to run fine: |
it seems to honor user namespaces, so no need to treat it as a network file system. Feedback got as part of the discussion on the issue. Closes: containers#1839 Signed-off-by: Giuseppe Scrivano <[email protected]>
it seems to honor user namespaces, so no need to treat it as a network file system. Feedback got as part of the discussion on the issue. Closes: containers#1839 Signed-off-by: Giuseppe Scrivano <[email protected]>
|
thanks, opened a PR: #1840 |
|
Thanks a lot for your help! |
|
@TimSchneider42 are you using podman with this fix more extensively nowadays? How is it working for you? I tried to replicate a setup with rootless podman using a container storage directory on a BeeGFS mount, and while I can run various small images like the ubuntu you showed above, or fedora minimal, I see failures with running other images, such as or with image builds I installed BeeGFS by following the quickstart https://doc.beegfs.io/7.4.3/quick_start_guide/quick_start_guide.html#example-setup, using the RHEL 9 packages from https://www.beegfs.io/release/beegfs_7.4.3/dists/. I mistakenly used unsupported 9.4 and had to patch things up by installing older kernel (otherwise client kernel module would not build). I also set up the extended file attributes which podman seems to need by folowing https://doc.beegfs.io/7.3.4/advanced_topics/acl.html I installed a supported kernel using To tests that extended file attributes are enabled, I did And to install podman, I tried first a centos koji build I found https://kojihub.stream.centos.org/koji/buildinfo?buildID=61248 and then the latest build on copr at https://copr.fedorainfracloud.org/coprs/rhcontainerbot/podman-next/. Both exhibited the unsatisfactory results from above. I manged to make podman work when I switched from overlay to vfs, but that's glacially slow. I don't feel like I have enough info collected for a competent issue ticket, so I'm just leaving a comment. Furthermore, the experience made me realize I don't want to do a network filesystem setup with podman. Instead of exposing the disk through BeeGFS for podmans on different machines to use, I should instead expose podman socket and use remote podman. That would work much better and there will be fewer low level problems like this. |
|
Hi @jiridanek, We have indeed experienced the same problems you described. It seems like However, what works is pulling and importing images, which means that images can be built somewhere else and transferred into remote storage. It is a bit annoying but feasible in our case. What also works is using BeeGFS as an additional image store (via the I am unsure if Podman actually supports multiple devices accessing a shared container storage location concurrently. I vaguely remember doing some tests with it (back when we were still on NFS) that did not go well, but I no longer remember the details. So, to summarize, in our case, we have one machine that uses the BeeGFS storage as a main storage location for Podman. Unfortunately, it cannot build images, and running containers is also restricted due to the issues you mentioned. However, it can pull and import images into BeeGFS storage. All other machines in the network use the BeeGFS storage as an additional (read-only) image store. This works quite well except for the building problems on the main machine. If you have any insights into this filesystem problem or any idea how to debug it, let me know. Best, |
|
I decided to try plain NFS instead of BeeGFS, where I encountered which is something I saw with BeeGFS too, but since I managed to reproduce this without any networked filesystem whatsoever, I reported it that way, hoping it will get more attention and will be easier to reproduce and fix. |
Feature request description
Hi,
I run a small compute cluster for my research group in which we use rootless Podman. As file system for the user homes, we use BeeGFS (formerly FhGFS). BeeGFS seems to understand subuids, which is a crucial advantage over NFS for deploying rootless containers.
While running containers works fine, I am experiencing an issue with building images:
I believe this issue is caused by the force_mask setting, which is applied because podman detects that BeeGFS is a network filesystem. If I trick Podman into believing that /home is a regular filesystem with an additional fuse-bindfs mount, it does not apply force_mask, and everything works fine, albeit at a performance penalty due to fuse-bindfs:
I understand that network filesystems are currently not supported, but in this case, I feel it could be as easy as providing an option for disabling the force_mask setting for network filesystems. Unless the bindfs-trick does something else I am not aware of.
Thanks a lot in advance!
Suggest potential solution
Adding an option for disabling the forced setting of force_mask="700" for network filesystems. Or adding an explicit exemption for BeeGFS, since it seems to work with Podman.
Have you considered any alternatives?
A workaround is to trick Podman into believing the storage directory is not in a network filesystem with an additional fuse-bindfs mount as I described above. However, this has a serious impact on performance.
Additional context
No response
The text was updated successfully, but these errors were encountered: