lchown permission denied (non-root user)

[GRX]

Hi,

I'm trying to get the sshfs volume working for my backup stack, but everytime I try to mount the endpoint I get prompted with permission denied.

With the user that's running docker-compose, I can access the folder so I'm not sure why this is going wrong.

Error when running docker-compose:
Error response from daemon: failed to copy file info for /var/lib/docker/plugins/a1318307eab6ed3e2bcfffa7a5226d557e7a2a6f12b4d48113d2e79063da848d/propagated-mount/e6c0b9b0593381f295d4a5c925f8ad8b: failed to chown /var/lib/docker/plugins/a1318307eab6ed3e2bcfffa7a5226d557e7a2a6f12b4d48113d2e79063da848d/propagated-mount/e6c0b9b0593381f295d4a5c925f8ad8b: lchown /var/lib/docker/plugins/a1318307eab6ed3e2bcfffa7a5226d557e7a2a6f12b4d48113d2e79063da848d/propagated-mount/e6c0b9b0593381f295d4a5c925f8ad8b: permission denied.

[andanotherusername]

You can workaround this issue by leaving a dump file on the remote location.

[sbriels]

I didnt try it yet with docker compose, but I also couldn't make any connections. I finally solved it by:

1. installing the plugin as: docker plugin install --grant-all-permissions vieux/sshfs sshkey.source=/root/.ssh/
2. setting up public/private key for root (sudo -i , then ssh-keygen -b 4096 -t rsa )
3. setting up the volume: docker volume create -d vieux/sshfs -o [email protected]:/home/user/ sshvolume

Hope this helps either you or the development team.

[chrlutz]

I'm getting this error message as well:

# docker run -ti -v nextcloud-html:/var/www/html nextcloud /bin/bash
docker: Error response from daemon: failed to copy file info for /var/lib/docker/plugins/2bb483b4a3521a55bff3d7284d4f1646b74092430c307ba7fea29306ea92fa89/propagated-mount/21558aea1829ef2d4bd4c0b95f4e5f60: failed to chown /var/lib/docker/plugins/2bb483b4a3521a55bff3d7284d4f1646b74092430c307ba7fea29306ea92fa89/propagated-mount/21558aea1829ef2d4bd4c0b95f4e5f60: lchown /var/lib/docker/plugins/2bb483b4a3521a55bff3d7284d4f1646b74092430c307ba7fea29306ea92fa89/propagated-mount/21558aea1829ef2d4bd4c0b95f4e5f60: permission denied.
See 'docker run --help'.

In my case this seems to happen because I have use two sshfs volumes (nextcloud-db and nextcloud-html) on the same remote ssh server like in this example:

# docker volume inspect nextcloud-db 
[
    {
        "CreatedAt": "0001-01-01T00:00:00Z",
        "Driver": "vieux/sshfs:latest",
        "Labels": null,
        "Mountpoint": "/mnt/volumes/2e01dcda104ec5521369b7d308810632",
        "Name": "nextcloud-db",
        "Options": {
            "allow_other": "",
            "gid": "999",
            "password": "xxx",
            "sshcmd": "[email protected]:nextcloud_db",
            "uid": "999"
        },
        "Scope": "local"
    }
]

# docker volume inspect nextcloud-html
[
    {
        "CreatedAt": "0001-01-01T00:00:00Z",
        "Driver": "vieux/sshfs:latest",
        "Labels": null,
        "Mountpoint": "/mnt/volumes/21558aea1829ef2d4bd4c0b95f4e5f60",
        "Name": "nextcloud-html",
        "Options": {
            "allow_other": "",
            "password": "xxx",
            "sshcmd": "[email protected]:",
            "uid": "33"
        },
        "Scope": "local"
    }
]

I figured out that it seems I cannot use both volumes at the same time.

It is possible to mount volume nextcloud-db to a mariadb container, but if I startup a second container (see docker run in my 3rd Line) this happens. If I stop/rm the mariadb container first, my 3rd line executes successfully.

Please note that both volumes are hosted on the same host (but accessed via different ssh-users). Could it be related to the different uid-Values I use in the volumes?

best regards,
Christoph

[chrlutz]

You can workaround this issue by leaving a dump file on the remote location.

What exactly do you mean by that? Do you have got an example? How can I produce such a dump file?

[blackfeather9]

I am having the same issue but may have a solution. Trying to use docker-compose to attach sshfs volumes.

docker version: 20.10.7
host: ubuntu 20.04 (kernel 5.4.0-80-generic)

Tried with public key authentication as well as password. Volume is created, user has full r/w/x over ssh with either method, can manually edit. Run docker-compse and watch auth logs on storage server, no problems there. Removed ACLs, set remote dirs to 777 and the parent folders.

Every time after the volume driver authenticates, docker-compose was failing with the following error:

failed to copy file info for /var/lib/docker/plugins/33378cc08f72a307a93f8f14391ef77eb78de6ed6f97fc427a3c096d0f162956/propagated-mount/364cea55c25d60e3da22a307ac46c260: failed to chown /var/lib/docker/plugins/33378cc08f72a307a93f8f14391ef77eb78de6ed6f97fc427a3c096d0f162956/propagated-mount/364cea55c25d60e3da22a307ac46c260: lchown /var/lib/docker/plugins/33378cc08f72a307a93f8f14391ef77eb78de6ed6f97fc427a3c096d0f162956/propagated-mount/364cea55c25d60e3da22a307ac46c260: permission denied

Tried above suggestions, including creating the volume in advance, and the suggestions in issues 17, 58. 65.

Possible solution: the "dump" file mentioned by @andanotherusername

Steps to Recreate:

• create your docker vol by hand or using docker-compose
• lchown fails with permission denied
• connect to storage target over ssh and cd into the volume dir, i.e. /mnt/volumes/containers/mycontainer
• create a dump file, i.e. touch dump -- that's it, it doesn't even have to be called dump.
• run your container/compose file again and watch as the volume successfully mounts

This did not work when I set custom uid/gid in my compose file, but with the defaults it seems to be successful and the container can now mount + write to the sshfs volume!

But isn't this still an issue, because without manually creating files within an sshfs volume, they will not mount? This will break any docker automation unless you create scripts to stage your volume storage in advance