IMG 5.3 update - databases

[aclum]

Placeholder ticket for updating

• KEGG db
• KEGG product lookup tables
• Pfam database
• Pfam product lookup tables

Marcel to provide finalized info.

cc @kaijli

[ssarrafan]

Hmm I'll assign this one to you for now @aclum

[aclum]

Natalia is still reviewing the KEGG results so will bump this a sprint.

[aclum]

Pfam will be version v37.0

[aclum]

we should be getting finalized databases, product lookup files and updated last parameters from Marcel this sprint.

[aclum]

database updates-

• Pfam version 37.0
  /global/dna/projectdirs/microbial/omics/databases/Pfam/Pfam-A/37.0

• img-nr
  /global/dna/projectdirs/microbial/omics/databases/IMG-NR/20240916

• product lookup files
  /global/cfs/cdirs/m3408/refdata/img/Product_Name_Mappings/20250123/pfam.tsv
  /global/cfs/cdirs/m3408/refdata/img/Product_Name_Mappings/20250123/kegg.tsv

software updates-

• LAST

1. Update container to use LAST version 1584
2. Add -m 180 to the lastal argument in the ko_ec task (line 198). This is separate from the -m argument for lastal_img_nr_ko_ec_gene_phylo_hit_selector.py
3. memory should be 256 gb to get half a permutter node based on Marcel's testing

• genomad -being tested in annotation update to 5.3 -genomad #36

[aclum]

permission issues copying over md5Hash2Data.tsv and taxonOId2Taxonomy.tsv

[aclum]

Permissions have been resolved, the databases have been copied over. @kaijli this is ready for work.

[kaijli]

attaching to genomad branch because i just need to work out file structures before it's complete / all part of 5.3 update

[aclum]

Marcel tested new product lookup table and there was 1 id missing so Natalia needs to provide an updated table. Actively in progress, moving to the next sprint.

[aclum]

@kaijli final product lookup files are above.

[kaijli]

Status update: genomad is working, new image with LAST update has been create, running tests on full workflow, some snags with ko_ec task testing on interactive node

[kaijli]

successful run at jaws id 98507