GCloud: Mounted directories not writable

[VasLem]

Bug report

Hi, I have a simple scenario, which does not seem to work. I am supplying a directory in gs to my process, and I am trying to write to it. Say, that directory is called gs://my-bucket/my-directory. A simple touch inside the process is enough to raise a Permission denied error.
If this is a feature, to prevent writing directly to the mount, I would expect to see it somewhere in the documentation. Similarly, I would also expect for an option to exist to permit this behavior. The publishDir way works alright, but for my use case it is far more intuitive just to write directly to the mount. (It is also something I need to have, as I am losing all progress on the files I have gotten up to the task failure point, as nothing is moved to the mount at the end, if an exception occurs.)

I recall that I had tried this in the past, like two months ago, and it worked like a charm.

Expected behavior and actual behavior

Expected: being able to write to a gs mounted directory
Actual: Permission denied

Steps to reproduce the problem

Example code:

process example {
input:
 path inputdir
shell:
'''
 touch !{inputdir}/test
'''
}

workflow {
 example(file("gs://my-bucket/my-directory"))
}

Just to make sure, I made a different process with the sleep command within and entered the instance:

gcloud compute ssh --zone $MY_ZONE $INSTANCE --project $MY_PROJECT --command="dockername=\$(docker ps --format \"{{.Names}}\") && docker container exec -it \$dockername bash"

and then cd-ed to /mnt/disks/my-bucket/any-directory

and ran touch test. I got touch: failed to close 'test': Permission denied

Environment

• Nextflow version: 24.04.2
• Java version: 21
• Operating system: Linux

[pditommaso]

This sounds like a problem with the bucket IAM permissions. Make sure you have read-write access to it

[VasLem]

Thanks for the response. If I had IAM bucket write permission issues, I would not be able to publish whatsoever, right? publishDir works just fine, only when I am writing inside the process script/shell it fails.