Cannot set up NXF variable for nextflow kuberun [kubernetes]

[xsvato01]

Issue with Setting NXF_ENABLE_FS_SYNC in Nextflow on a Shared Disk

Hello,

I am encountering a problem with Nextflow (v. 24.04.2) truncating large output files on a shared disk. I believe that setting the NXF_ENABLE_FS_SYNC environment variable should resolve this issue.

However, I am having trouble setting this environment variable correctly. Here’s what I’ve tried so far:

1. Adding to nextflow.config:

   env { 
       NXF_ENABLE_FS_SYNC = true 
   }

   This generates the warning message:

   The env scope provides environment variables to tasks, not Nextflow itself. Nextflow environment variables such as NXF_VER should be set in the environment in which Nextflow is launched.
   

   Result: No NXF_ENABLE_FS_SYNC in the k8s driver nor worker pod config.

2. Setting the environment variable before running Nextflow:

   export NXF_ENABLE_FS_SYNC=true
   nextflow kuberun xxx

   Result: No NXF_ENABLE_FS_SYNC in the k8s driver nor worker pod config.

3. Adding env config to pod in nextflow.config:

   pod = [
       [env: 'NXF_ENABLE_FS_SYNC', value: 'true']
   ]

   Result: This creates NXF_ENABLE_FS_SYNC in the worker pods.

I am not sure if the NXF_ENABLE_FS_SYNC variable will work if it is only in the worker pods and not in the driver itself.

Any guidance on this issue would be greatly appreciated!

Thank you,
Jan