BUG: split cannot open path error

[YinCY123]

Describe the bug
during filtering step the split: cannot open path error consist occur.

To Reproduce
###########################################
#Welcome to zUMIs
#below, please fill the mandatory inputs
#We expect full paths for all files.
###########################################

#define a project name that will be used to name output files
project: GSE129798

#Sequencing File Inputs:
#For each input file, make one list object & define path and barcode ranges
#base definition vocabulary: BC(n) UMI(n) cDNA(n).
#Barcode range definition needs to account for all ranges. You can give several comma-separated ranges for BC & UMI sequences, eg. BC(1-6,20-26)
#you can specify between 1 and 4 input files
sequence_files:
file1:
name: /home/yincy/git/disk14/kidney/GSE129798/bam/Mouse_adult_Female1_zone1.bam
base_definition:
- #example: BC(1-6)
- #example: UMI(7-16)
file2:
name: #path to second file
base_definition:
- #example: cDNA(1-50)

#reference genome setup
reference:
STAR_index: /home/yincy/git/data/biodata/index/mouse/star_mm10_index/
GTF_file: /home/yincy/git/data/biodata/annotation/mouse/gencode.vM23.primary_assembly.annotation.gtf
exon_extension: no #extend exons by a certain width?
extension_length: 0 #number of bp to extend exons by
scaffold_length_min: 0 #minimal scaffold/chromosome length to consider (0 = all)
additional_files: #Optional parameter. It is possible to give additional reference sequences here, eg ERCC.fa
additional_STAR_params: #Optional parameter. you may add custom mapping parameters to STAR here

#output directory
out_dir: /home/yincy/git/zUMIs

###########################################
#below, you may optionally change default parameters
###########################################

#number of processors to use
num_threads: 10
mem_limit: null #Memory limit in Gigabytes, null meaning unlimited RAM usage.

#barcode & UMI filtering options
#number of bases under the base quality cutoff that should be filtered out.
#Phred score base-cutoff for quality control.
filter_cutoffs:
BC_filter:
num_bases: 1
phred: 20
UMI_filter:
num_bases: 1
phred: 20

#Options for Barcode handling
#You can give either number of top barcodes to use or give an annotation of cell barcodes.
#If you leave both barcode_num and barcode_file empty, zUMIs will perform automatic cell barcode selection for you!
barcodes:
barcode_num: null
barcode_file: null
barcode_sharing: null #Optional for combining several barcode sequences per cell (see github wiki)
automatic: yes #Give yes/no to this option. If the cell barcodes should be detected automatically. If the barcode file is given in combination with automatic barcode detection, the list of given barcodes will be used as whitelist.
BarcodeBinning: 1 #Hamming distance binning of close cell barcode sequences.
nReadsperCell: 100 #Keep only the cell barcodes with atleast n number of reads.
demultiplex: no #produce per-cell demultiplexed bam files.

#Options related to counting of reads towards expression profiles
counting_opts:
introns: yes #can be set to no for exon-only counting.
intronProb: no #perform an estimation of how likely intronic reads are to be derived from mRNA by comparing to intergenic counts.
downsampling: 0 #Number of reads to downsample to. This value can be a fixed number of reads (e.g. 10000) or a desired range (e.g. 10000-20000) Barcodes with less than will not be reported. 0 means adaptive downsampling. Default: 0.
strand: 0 #Is the library stranded? 0 = unstranded, 1 = positively stranded, 2 = negatively stranded
Ham_Dist: 0 #Hamming distance collapsing of UMI sequences.
velocyto: yes #Would you like velocyto to do counting of intron-exon spanning reads
primaryHit: yes #Do you want to count the primary Hits of multimapping reads towards gene expression levels?
multi_overlap: no #Do you want to assign reads overlapping to multiple features?
fraction_overlap: 0 #minimum required fraction of the read overlapping with the gene for read assignment to genes
twoPass: yes #perform basic STAR twoPass mapping

#produce stats files and plots?
make_stats: yes

#Start zUMIs from stage. Possible TEXT(Filtering, Mapping, Counting, Summarising). Default: Filtering.
which_Stage: Filtering

#define dependencies program paths

#below, fqfilter will add a read_layout flag defining SE or PE
samtools_exec: samtools
pigz_exec: pigz
STAR_exec: STAR
Rscript_exec: Rscript
zUMIs_directory: /home/yincy/tools/zUMIs
read_layout: PE

Screenshots

Good news! A newer version of zUMIs is available at https://github.com/sdparekh/zUMIs

---

Using miniconda environment for zUMIs!
note: internal executables will be used instead of those specified in the YAML file!
Warning message:
package ‘yaml’ was built under R version 4.4.1

You provided these parameters:
YAML file: git/kidney/bash/zUMIs_GSE129798.run.yaml
zUMIs directory: /home/yincy/tools/zUMIs
STAR executable STAR
samtools executable samtools
pigz executable pigz
Rscript executable Rscript
RAM limit: null
zUMIs version 2.9.7e

Mon Oct 7 07:46:08 PM CST 2024
WARNING: The STAR version used for mapping is 2.7.3a and the STAR index was created using the version 20201. This may lead to an error while mapping. If you encounter any errors at the mapping stage, please make sure to create the STAR index using STAR 2.7.3a.
Filtering...
split: cannot open '#path' for reading: No such file or directory
ls: cannot access '/home/yincy/git/zUMIs/zUMIs_output/.tmpMerge/#pathGSE129798*': No such file or directory
Warning message:
package ‘yaml’ was built under R version 4.4.1
Warning message:
package ‘yaml’ was built under R version 4.4.1
Warning message:
package ‘yaml’ was built under R version 4.4.1
Warning message:
package ‘yaml’ was built under R version 4.4.1
Warning message:
package ‘yaml’ was built under R version 4.4.1
Warning message:
package ‘yaml’ was built under R version 4.4.1
Warning message:
package ‘yaml’ was built under R version 4.4.1
couldn't open WARNING: for reading: No such file or directory
couldn't open WARNING: for reading: No such file or directory
couldn't open ignoring for reading: No such file or directory
couldn't open ignoring for reading: No such file or directory
couldn't open environment for reading: No such file or directory
couldn't open environment for reading: No such file or directory
couldn't open value for reading: No such file or directory
couldn't open value for reading: No such file or directory
couldn't open of for reading: No such file or directory
couldn't open of for reading: No such file or directory
couldn't open R_HOME for reading: No such file or directory
couldn't open R_HOME for reading: No such file or directory
Warning message:
package ‘yaml’ was built under R version 4.4.1
sh: 1: NULL: not found
sh: 1: NULL: not found
couldn't open WARNING: for reading: No such file or directory
couldn't open ignoring for reading: No such file or directory
couldn't open environment for reading: No such file or directory
couldn't open value for reading: No such file or directory
couldn't open of for reading: No such file or directory
couldn't open R_HOME for reading: No such file or directory
Warning message:
package ‘yaml’ was built under R version 4.4.1
couldn't open WARNING: for reading: No such file or directory
couldn't open ignoring for reading: No such file or directory
couldn't open environment for reading: No such file or directory
couldn't open value for reading: No such file or directory
couldn't open of for reading: No such file or directory
couldn't open R_HOME for reading: No such file or directory
couldn't open WARNING: for reading: No such file or directory
couldn't open ignoring for reading: No such file or directory
couldn't open environment for reading: No such file or directory
couldn't open value for reading: No such file or directory
couldn't open of for reading: No such file or directory
couldn't open R_HOME for reading: No such file or directory
couldn't open WARNING: for reading: No such file or directory
couldn't open ignoring for reading: No such file or directory
couldn't open environment for reading: No such file or directory
couldn't open value for reading: No such file or directory
couldn't open of for reading: No such file or directory
couldn't open R_HOME for reading: No such file or directory
sh: 1: NULL: not found
sh: 1: NULL: not found
sh: 1: NULL: not found
sh: 1: NULL: not found
couldn't open WARNING: for reading: No such file or directory
couldn't open ignoring for reading: No such file or directory
couldn't open environment for reading: No such file or directory
couldn't open value for reading: No such file or directory
couldn't open of for reading: No such file or directory
couldn't open R_HOME for reading: No such file or directory
sh: 1: NULL: not found
couldn't open WARNING: for reading: No such file or directory
couldn't open ignoring for reading: No such file or directory
couldn't open environment for reading: No such file or directory
couldn't open value for reading: No such file or directory
couldn't open of for reading: No such file or directory
couldn't open R_HOME for reading: No such file or directory
sh: 1: NULL: not found
couldn't open WARNING: for reading: No such file or directory
couldn't open ignoring for reading: No such file or directory
couldn't open environment for reading: No such file or directory
couldn't open value for reading: No such file or directory
couldn't open of for reading: No such file or directory
couldn't open R_HOME for reading: No such file or directory
sh: 1: NULL: not found
ls: cannot access '/home/yincy/git/zUMIs/zUMIs_output/.tmpMerge//GSE129798..filtered.tagged.bam': No such file or directory
cat: '/home/yincy/git/zUMIs/zUMIs_output/.tmpMerge//GSE129798..BCstats.txt': No such file or directory
rm: cannot remove '/home/yincy/git/zUMIs/zUMIs_output/.tmpMerge/#path*gz': No such file or directory
Mon Oct 7 07:48:00 PM CST 2024
WARNING: ignoring environment value of R_HOME
Error: package or namespace load failed for ‘data.table’:
.onLoad failed in loadNamespace() for 'data.table', details:
call: fun(libname, pkgname)
error: This is R 3.6.3 but data.table has been installed using R 4.4.1. The major version must match. Please reinstall data.table.
In addition: Warning message:
package ‘data.table’ was built under R version 4.4.1
Execution halted
Mapping...
WARNING: ignoring environment value of R_HOME
Warning message:
package ‘yaml’ was built under R version 4.4.1
Warning message:
package ‘data.table’ was built under R version 4.4.1
[1] "2024-10-07 19:48:00 CST"
Error: .onLoad failed in loadNamespace() for 'data.table', details:
call: fun(libname, pkgname)
error: This is R 3.6.3 but data.table has been installed using R 4.4.1. The major version must match. Please reinstall data.table.
Execution halted
Mon Oct 7 07:48:07 PM CST 2024
Counting...
WARNING: ignoring environment value of R_HOME
Error: package or namespace load failed for ‘data.table’:
.onLoad failed in loadNamespace() for 'data.table', details:
call: fun(libname, pkgname)
error: This is R 3.6.3 but data.table has been installed using R 4.4.1. The major version must match. Please reinstall data.table.
In addition: Warning message:
package ‘data.table’ was built under R version 4.4.1
Execution halted
Mon Oct 7 07:48:07 PM CST 2024
WARNING: ignoring environment value of R_HOME
Loading required package: yaml
Warning message:
package ‘yaml’ was built under R version 4.4.1
Loading required package: Matrix
Failed with error: ‘This is R 3.6.3, package ‘Matrix’ needs >= 4.4.0’
[1] "loomR found"
Warning messages:
1: package ‘loomR’ was built under R version 4.4.1
2: package ‘R6’ was built under R version 4.4.1
3: package ‘hdf5r’ was built under R version 4.4.1
4: package ‘iterators’ was built under R version 4.4.1
5: package ‘itertools’ was built under R version 4.4.1
Error in gzfile(file, "rb") : cannot open the connection
Calls: rds_to_loom -> readRDS -> gzfile
In addition: Warning message:
In gzfile(file, "rb") :
cannot open compressed file '/home/yincy/git/zUMIs/zUMIs_output/expression/GSE129798.dgecounts.rds', probable reason 'No such file or directory'
Execution halted
Mon Oct 7 07:48:08 PM CST 2024
WARNING: ignoring environment value of R_HOME
Warning message:
package ‘yaml’ was built under R version 4.4.1
[1] "2024-10-07 19:48:08 CST"
[1] "Preparing bam file for velocyto..."
[E::hts_open_format] Failed to open file NA
samtools view: failed to open "NA" for reading: No such file or directory
[1] "2024-10-07 19:48:08 CST"
Error: .onLoad failed in loadNamespace() for 'data.table', details:
call: fun(libname, pkgname)
error: This is R 3.6.3 but data.table has been installed using R 4.4.1. The major version must match. Please reinstall data.table.
Execution halted
Descriptive statistics...
WARNING: ignoring environment value of R_HOME
[1] "I am loading useful packages for plotting..."
[1] "2024-10-07 19:48:08 CST"
Error: package or namespace load failed for ‘data.table’:
.onLoad failed in loadNamespace() for 'data.table', details:
call: fun(libname, pkgname)
error: This is R 3.6.3 but data.table has been installed using R 4.4.1. The major version must match. Please reinstall data.table.
In addition: Warning message:
package ‘data.table’ was built under R version 4.4.1
Execution halted
Mon Oct 7 07:48:08 PM CST 2024

Desktop (please complete the following information):

• OS: Ubuntu
• Version 20.04

Additional context
Add any other context about the problem here.

[YinCY123]

Hi @swatidparekh, can you help fix this bug?