Custom DB and custom taxonomy (GTDB or similar)

[osvatic]

Is there a way to create a completely new database and non-NCBI taxonomy for that database? I am primarily interested in using something like GTDB, which has a genome download and taxonomy tsv file available.

I understand that I can add the contigs/genomes to a custom database using "kraken2-build --add-to-library" but the creation of a new taxonomy doesn't seem straight forward.

[ch4rr0]

Hello,

We are in the process of adding GTDB support to the k2 wrapper (python rewrite of the kraken2 scripts), stay tuned.

[lingrongjin]

I'm also interested in building a custom database with GTDB taxonomy! When can we expect to have this function available?

[ch4rr0]

I pushed a change to the k2 wrapper today.

Here's how you can build a GTDB database with the changes:

k2 build --special gtdb --gtdb-files gtdb_genomes_reps.tar.gz --threads 6 --db gtdb_reps

The --gtdb-files parameter takes a list of filenames or a pattern that matches the files needed
to build the GTDB database.

You can find the list of file names here https://data.gtdb.ecogenomic.org/releases/latest/genomic_files_reps/
or https://data.gtdb.ecogenomic.org/releases/latest/genomic_files_all/

The command line will also offer a list of files if you specify the wrong file name:

k2 build --special --gtdb-files foo.tar.gz --db gtdb_reps
[ERROR - 2024-11-05 17:57:48,567]: Could not find any files matching foo.tar.gz
[ERROR - 2024-11-05 17:57:48,567]: Here are a list of candidates:
bac120_msa_marker_genes_all.tar.gz
ar53_msa_marker_genes_all.tar.gz
ssu_all.fna.gz
bac120_marker_genes_all.tar.gz
ar53_marker_genes_all.tar.gz
bac120_ssu_reps.fna.gz
ar53_msa_marker_genes_reps.tar.gz
gtdb_proteins_aa_reps.tar.gz
bac120_msa_marker_genes_reps.tar.gz
bac120_msa_reps.faa.gz
bac120_marker_genes_reps.tar.gz
ar53_msa_reps.faa.gz
ar53_marker_genes_reps.tar.gz
gtdb_proteins_nt_reps.tar.gz
gtdb_genomes_reps.tar.gz
ar53_ssu_reps.fna.gz

We welcome your testing in making sure that this feature works as expected.

[YiAngBeao]

Traceback (most recent call last):
File "10software/mambaforge/envs/kraken2/bin/k2.py", line 3582, in
k2_main()
~~~~~~~^^
File "10software/mambaforge/envs/kraken2/bin/k2.py", line 3562, in k2_main
build_gtdb_database(args)
~~~~~~~~~~~~~~~~~~~^^^^^^
File "/10software/mambaforge/envs/kraken2/bin/k2.py", line 1768, in build_gtdb_database
for accession, filepath in result[0]:
^^^^^^^^^^^^^^^^^^^
ValueError: too many values to unpack (expected 2)

I pushed a change to the k2 wrapper today.

Here's how you can build a GTDB database with the changes:

k2 build --special gtdb --gtdb-files gtdb_genomes_reps.tar.gz --threads 6 --db gtdb_reps

The parameter takes a list of filenames or a pattern that matches the files needed to build the GTDB database.--gtdb-files

You can find the list of file names here https://data.gtdb.ecogenomic.org/releases/latest/genomic_files_reps/ or https://data.gtdb.ecogenomic.org/releases/latest/genomic_files_all/

The command line will also offer a list of files if you specify the wrong file name:

k2 build --special --gtdb-files foo.tar.gz --db gtdb_reps
[ERROR - 2024-11-05 17:57:48,567]: Could not find any files matching foo.tar.gz
[ERROR - 2024-11-05 17:57:48,567]: Here are a list of candidates:
bac120_msa_marker_genes_all.tar.gz
ar53_msa_marker_genes_all.tar.gz
ssu_all.fna.gz
bac120_marker_genes_all.tar.gz
ar53_marker_genes_all.tar.gz
bac120_ssu_reps.fna.gz
ar53_msa_marker_genes_reps.tar.gz
gtdb_proteins_aa_reps.tar.gz
bac120_msa_marker_genes_reps.tar.gz
bac120_msa_reps.faa.gz
bac120_marker_genes_reps.tar.gz
ar53_msa_reps.faa.gz
ar53_marker_genes_reps.tar.gz
gtdb_proteins_nt_reps.tar.gz
gtdb_genomes_reps.tar.gz
ar53_ssu_reps.fna.gz

We welcome your testing in making sure that this feature works as expected.

Hello, I tried to build a GTDB database with the k2 you provided, but it resulted in this error.

[ch4rr0]

I pushed a fix for this issue a few days ago and have also pushed a fix for potential crashes while masking the genomes. Can you try pulling these changes and trying again?

[phiweger]

hi, is there any update to this issue? thanks :)

[osvatic]

I was able to run the command with no errors after the push.

[ch4rr0]

We will soon be publishing an index for the GTDB representative genomes, stay tuned.

[replikation]

That would be really great, thank you.

[ch4rr0]

The database is now available, see: https://benlangmead.github.io/aws-indexes/k2

[replikation]

thank you