Skip to content

deep-spin/speech-continuous-attention

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

speech-continuous-attention

Speech Classification with Continuous Attention Mechanisms.

This is the code for speech classification experiments of the paper:

which builds upon:

The code is based on the following speechbrain recipes:

We provide scripts for UrbanSound8k only. The datasets should be placed in the data/ folder.

Installation

First, install the spcdist library to get support for sparse continuous distributions:

pip3 install git+https://github.com/deep-spin/sparse_continuous_distributions#egg=spcdist

Then install other requirements via:

pip3 install -r requirements.txt

Run 10-fold

Valdiating and testing on fold 1:

# discrete-attention models
for max_act in "softmax", "entmax1333", "entmax15", "sparsemax"
do
    mkdir "results/urbansound8k_discrete_${max_act}"
    python3 urbansound8k_train.py hparams/urbansound8k_acrnn.yaml \
        --train_fold_nums=[2, 3, 4, 5, 6, 7, 8, 9, 10] \
        --valid_fold_nums=[1] \
        --test_fold_nums=[1] \
        --attn_domain discrete \
        --attn_max_activation ${max_act} \
        --output_folder=results/urbansound8k_discrete_${max_act}/fold_1 \
        --device=cuda:0
done

# continuous-attention models
for max_act in "softmax", "triweight", "biweight", "sparsemax"
do
    mkdir "results/urbansound8k_continuous_${max_act}"
    python3 urbansound8k_train.py hparams/urbansound8k_acrnn.yaml \
        --train_fold_nums=[2, 3, 4, 5, 6, 7, 8, 9, 10] \
        --valid_fold_nums=[1] \
        --test_fold_nums=[1] \
        --attn_domain continuous \
        --attn_max_activation ${max_act} \
        --output_folder=results/urbansound8k_continuous_${max_act}/fold_1 \
        --device=cuda:0 
done

Do the same for the other folds and average in the end to get the final results as in the paper. See the config files in the hparams folder for more information.

Save and plot attention maps

First, save the spectrograms:

mkdir -p specs/urbansound8k_continuous_sparsemax
python3 urbansound8k_save_specs.py hparams/urbansound8k_acrnn.yaml \
    --train_fold_nums=[2, 3, 4, 5, 6, 7, 8, 9, 10] \
    --valid_fold_nums=[1] \
    --test_fold_nums=[1] \
    --attn_domain continuous \
    --attn_max_activation sparsemax \
    --output_folder=results/urbansound8k_continuous_sparsemax/fold_1 \
    --device=cpu \
    --attn_num_samples 50 \
    --spec_dname specs/urbansound8k_continuous_sparsemax/

And then save the attention maps:

mkdir saved_attentions
python3 urbansound8k_save_attentions.py hparams/urbansound8k_acrnn.yaml \
    --train_fold_nums=[2, 3, 4, 5, 6, 7, 8, 9, 10] \
    --valid_fold_nums=[1] \
    --test_fold_nums=[1] \
    --attn_domain continuous \
    --attn_max_activation sparsemax \
    --output_folder=results/urbansound8k_continuous_sparsemax/fold_1 \
    --device=cuda:2 \
    --attn_num_samples 200 \
    --attn_fname saved_attentions/urbansound8k_continuous_sparsemax.csv

For plotting attention densities you can follow the step-by-step instructions in this notebook: https://colab.research.google.com/drive/1Ce51VB_rgmNmxB5lggRSUDZ__-zu-Xov?usp=sharing

Citation

If you use this codebase, or otherwise found our work valuable, please cite:

todo

Releases

No releases published

Packages

No packages published

Languages