Issue with distribute_reads_to_targets.py

[Kedraous]

Hello,

I am having an issue with distribute_reads_to_targets.py in Hybpiper.
Here is the command I used:

while read name;
do hybpiper assemble -t_dna mega353.fasta -r ./trimmed_reads/$name*.fastq --prefix $name --bwa;
done < ./trimmed_reads/namelist.txt

Below is the error message:

[INFO]: HybPiper version 2.1.6 was called with these arguments:
/services/tools/anaconda3/2023.03/bin/hybpiper assemble -t_dna mega353.fasta -r
./trimmed_reads/trimmed-AG1-1_S131_L001_R1_001.fastq
./trimmed_reads/trimmed-AG1-1_S131_L001_R2_001.fastq --prefix trimmed-AG1-1
--bwa

[INFO]: Number of cpus/threads not specified, using all available (40).
[INFO]: Checking for external dependencies:

blastx found at /services/tools/ncbi-blast/2.13.0+/bin/blastx
exonerate found at /services/tools/exonerate/2.4.0/bin/exonerate
parallel found at /services/tools/parallel/20220422/bin/parallel
makeblastdb found at /services/tools/ncbi-blast/2.13.0+/bin/makeblastdb
spades.py found at /services/tools/SPAdes/3.15.5/bin/spades.py
bwa found at /services/tools/bwa/0.7.17/bwa
samtools found at /services/tools/anaconda-2.1.0/bin/samtools
bbmap.sh found at /services/tools/bbmap/38.90/bbmap.sh
bbmerge.sh found at /services/tools/bbmap/38.90/bbmerge.sh
diamond found at /services/tools/anaconda-2.1.0/bin/diamond
mafft found at /services/tools/mafft/7.490/bin/mafft

[INFO]: Everything looks good!
[INFO]: Checking target file FASTA header formatting...
[INFO]: The target file FASTA header formatting looks good!
[INFO]: The target file contains at least one sequence for 353 unique genes.
[WARNING]: There are 926 sequences in your target file that contain unexpected stop codons
when translated in the first forwards frame. If your target file contains only
protein-coding sequences, please check these sequences, and/or run "hybpiper
fix_targetfile". Sequence names can be found in the sample log file (if running
"hybpiper assemble") or printed below (if running "hybpiper check_targetfile").

[WARNING]: There are 1616 sequences in your target file that are not multiples of three.
If your target file contains only protein-coding sequences, please check these
sequences, and/or run "hybpiper fix_targetfile". Sequence names can be found in
the sample log file (if running "hybpiper assemble") or printed below (if
running "hybpiper check_targetfile").

[INFO]: Output will be written to the directory: ./trimmed-AG1-1
[CMD]: bwa mem -t 40 mega353.fasta /home/projects/ku_00047/data/raw/angiosperm353/
raw_data_OleTarget/trimmed_reads/trimmed-AG1-1_S131_L001_R1_001.fastq /home/
projects/ku_00047/data/raw/angiosperm353/raw_data_OleTarget/trimmed_reads/
trimmed-AG1-1_S131_L001_R2_001.fastq | samtools view -h -b -S - >
trimmed-AG1-1.bam
[INFO]: Gathering IDs for mapped reads...
[INFO]: In total, 67776 reads from the paired-end read files will be distributed to gene directories
[NOTE]: Distributing paired reads to gene directories
Traceback (most recent call last):
File "/services/tools/anaconda3/2023.03/bin/hybpiper", line 8, in
sys.exit(main())
File "/services/tools/anaconda3/2023.03/lib/python3.10/site-packages/hybpiper/assemble.py", line 1873, in main
args.func(args)
File "/services/tools/anaconda3/2023.03/lib/python3.10/site-packages/hybpiper/assemble.py", line 1444, in assemble
distribute_bwa(bamfile, readfiles, targetfile, target, unpaired_readfile, args.exclude,
File "/services/tools/anaconda3/2023.03/lib/python3.10/site-packages/hybpiper/assemble.py", line 738, in distribute_bwa
distribute_reads_to_targets.distribute_reads(readfiles, read_hit_dict, merged=merged, single_end=single_end,
File "/services/tools/anaconda3/2023.03/lib/python3.10/site-packages/hybpiper/distribute_reads_to_targets.py", line 326, in distribute_reads
for ID1_long, Seq1, Qual1 in progressbar.progressbar(iterator1, max_value=num_reads_in_readfile,
TypeError: 'module' object is not callable

Kindly let me know what you think might be causing this.

Best regards,
Kedra

[chrisjackson-pellicle]

Hi Kedra,

It looks like you've run in to the same issue mentioned here.

Installing the correct version of progressbar, as described in the Dependencies here, should fix things.

Cheers,

Chris

[KaraLayton]

Hi Chris,
I'm just circling back to this post because I'm having a similar issue (i.e. the same error at the same spot in the pipeline). When I run pip or conda list it confirms that progressbar2 is installed. Some important context though- I'm now using the github version of Hybpiper on my new Mac M3 Pro. I had no problem with the conda installation of Hybpiper (and no issue running Hybpiper) on my previous Mac, but now it tells me the package isn't available on this new Mac (I checked other packages on bioconda and some are available while others aren't).
Any insight would be greatly appreciated!
Thanks in advance,
Kara

[KaraLayton]

Progress made: I managed to find hybpiper on conda by reconfiguring conda with the osx-arm64 flag (should've done this in the first place!), and then I installed python3.9 with brew. But even when installing hybpiper through conda and specifying python3.9 I still get a series of incompatibility errors. Nonetheless, I rectified the issue by installing Hybpiper with git clone, using pip3.9 install . It's up and running!

[chrisjackson-pellicle]

Hi @KaraLayton - great you got this working!

I don't have an ARM M1/M2 Mac to test this on, so I'm of limited help with the conda install, I'm afraid (although I hope to have an M2 laptop in the next couple of months...).

There is currently no ARM64 (i.e., the CPU architecture used by the Apple M1/M1/M3 Macs) conda package for HybPiper, so you'll need to use the x64 package for the moment. To do this, my understanding is that you need to configure your HybPiper conda environment as an x64 environment, see e.g. https://stackoverflow.com/questions/71515117/how-to-set-up-a-conda-osx-64-environment-on-arm-mac

As soon as I can test this properly on local Mac I'll update the HybPiper readme with proper instructions, and I'll look in to building an ARM64 conda package for HybPiper.

Cheers,

Chris