ENH: Added action normalize that can normalize a FeatureTable[Frequency] by gene length, library size and composition.

[VinzentRisch]

This PR closes #23

• This PR adds a new action that can normalize a FeatureTable[Frequency] with common RNA-seq methods. The action uses the package RNAnorm.
• The action takes as input one FeatureTable[Frequency].
• The gene-lengths input is used to pass the gene lengths and can only be used with the methods TPM and FPKM. It accepts artifacts of types CARDAlleleAnnotation and CARDGeneAnnotation and also FeatureData[SequenceCharacteristics % Properties("length")].
• The parameters m-trim and a-trim can only be used with the methods TMM and CTF.

Set up an environment

CONDA_SUBDIR=osx-64 mamba create -yn q2-amr \
  -c conda-forge -c bioconda -c qiime2 -c defaults \
  -c https://packages.qiime2.org/qiime2/2024.5/metagenome/staged/ \
  qiime2 q2cli q2templates q2-types rgi rnanorm tqdm

conda activate q2-amr
conda config --env --set subdir osx-64

pip install --no-deps --force-reinstall \
  git+https://github.com/misialq/rgi.git@py38-fix \

Run it locally

1. First, clone the repo and checkout the PR branch:

git clone https://github.com/bokulich-lab/q2-amr.git
cd q2-amr
git fetch origin pull/44/head:pr-44
git checkout pr-44
pip install -e .

2. Download test files
   PR_23.zip

3. Test it out!
   go to PR_norm directory

cd PR_norm 

Run all commands from PR_norm directory

qiime amr normalize --i-table data/allele_feature_table.qza --o-normalized-table output/norm_tmm.qza --p-method "tmm"  --p-m-trim 0.4 --p-a-trim 0.04
qiime amr normalize --i-table data/allele_feature_table.qza --o-normalized-table output/norm_ctf.qza --p-method "ctf"  --p-m-trim 0.4 --p-a-trim 0.04
qiime amr normalize --i-table data/allele_feature_table.qza --i-gene-length data/amr_allele_annotation.qza --o-normalized-table output/norm_fpkm.qza --p-method "fpkm" 
qiime amr normalize --i-table data/allele_feature_table.qza --i-gene-length data/amr_allele_annotation.qza --o-normalized-table output/norm_tpm.qza --p-method "tpm"
qiime amr normalize --i-table data/allele_feature_table.qza --o-normalized-table output/norm_uq.qza --p-method "uq" 
qiime amr normalize --i-table data/allele_feature_table.qza --o-normalized-table output/norm_cuf.qza --p-method "cuf"
qiime amr normalize --i-table data/allele_feature_table.qza --o-normalized-table output/norm_cpm.qza --p-method "cpm"
qiime amr normalize --i-table data/gene_feature_table.qza --i-gene-length data/amr_gene_annotation.qza --o-normalized-table output/norm_fpkm_gene.qza --p-method "fpkm" 

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.15%. Comparing base (75d4efa) to head (011abf4).

❗ Current head 011abf4 differs from pull request most recent head 8bafcb2

Please upload reports for the commit 8bafcb2 to get more accurate results.

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #44      +/-   ##
==========================================
- Coverage   94.29%   94.15%   -0.15%     
==========================================
  Files          20       20              
  Lines        1491     1197     -294     
==========================================
- Hits         1406     1127     -279     
+ Misses         85       70      -15     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Sann5]

Hey @VinzentRisch the environment does not work for me. Why are you installing types-genomics? I uninstalled it and installed the latest version of q2-types. Then qiime tools refresh-cache works.

[Sann5]

Also, two of the commands in the PR message fail for me:

qiime amr normalize --i-table data/allele_feature_table.qza --i-gene-length data/amr_allele_annotation.qza --o-normalized-table output/norm_fpkm.qza --p-method "fpkm"
Plugin error from amr:

  Missing one or more files for CARDAlleleAnnotationDirectoryFormat: '.+sorted.length_100.bam$'

Debug info has been saved to /var/folders/6p/vfbtsnkn1x738nqkqzcprf5w0000gn/T/qiime2-q2cli-err-yhv9osp_.log

qiime amr normalize --i-table data/allele_feature_table.qza --i-gene-length data/amr_allele_annotation.qza --o-normalized-table output/norm_tpm.qza --p-method "tpm"
Plugin error from amr:

  Missing one or more files for CARDAlleleAnnotationDirectoryFormat: '.+sorted.length_100.bam$'

Debug info has been saved to /var/folders/6p/vfbtsnkn1x738nqkqzcprf5w0000gn/T/qiime2-q2cli-err-b4_m_l6z.log

Here are the logs:
logs.zip

[VinzentRisch]

Ou yes Im sorry I forgot to update the files. Now they are the correct ones.
And also forgot to adjust the installation instructions. Should be resolved now sorry

[Sann5]

Hey @VinzentRisch. Nice work! I made some comments with some suggestions but nothing major. All the tests pass and the actions complete without issues.

[VinzentRisch]

Hi @Sann5

• So now the defaults for the trim parameters are None and they get set to the default later>
• I created two helper functions, one that handles the validation of the parameters and the other that processes and validates the gene lengths.

[Sann5]

Hey @VinzentRisch. For all except the first two commands in the PR message i get the following error

Plugin error from amr:

  Parameters m-trim and a-trim can only be used with methods TMM and CTF.

Debug info has been saved to /var/folders/6p/vfbtsnkn1x738nqkqzcprf5w0000gn/T/qiime2-q2cli-err-9cgfrctm.log

Log file:

Traceback (most recent call last):
  File "/Users/santiago/mambaforge/envs/q2-amr/lib/python3.8/site-packages/q2cli/commands.py", line 520, in __call__
    results = self._execute_action(
  File "/Users/santiago/mambaforge/envs/q2-amr/lib/python3.8/site-packages/q2cli/commands.py", line 581, in _execute_action
    results = action(**arguments)
  File "<decorator-gen-67>", line 2, in normalize
  File "/Users/santiago/mambaforge/envs/q2-amr/lib/python3.8/site-packages/qiime2/sdk/action.py", line 342, in bound_callable
    outputs = self._callable_executor_(
  File "/Users/santiago/mambaforge/envs/q2-amr/lib/python3.8/site-packages/qiime2/sdk/action.py", line 566, in _callable_executor_
    output_views = self._callable(**view_args)
  File "/Users/santiago/eth/23-2/HIWI/q2-amr/q2_amr/card/normalization.py", line 16, in normalize
    m_trim, a_trim = _validate_parameters(method, m_trim, a_trim, gene_length)
  File "/Users/santiago/eth/23-2/HIWI/q2-amr/q2_amr/card/normalization.py", line 57, in _validate_parameters
    raise ValueError(
ValueError: Parameters m-trim and a-trim can only be used with methods TMM and CTF.

[VinzentRisch]

That is weird.
For me they all run without problem.
Can you fetch and pip install and try again?

[Sann5]

Looks good to me! I dont know what I did wrong before but now the action works 😅 🚀

[VinzentRisch]

Code was moves to q2-feature-table/_normalize