STGATE

This repo is dedicated to the STGATE: Spatial-Temporal Graph Attention network with a Transformer Encoder for EEG-based emotion recognition link to the original paper

Structure of the repo

• lib: The directory with current DL models, scripts for running some preprocessing and so on

• experiments: Several runs in jupyter notebooks to evaluate models from lib

• data: The folder to put the EEG data from DREAMER, SEED and SEED IV datasets (and write in code relative path to it)

• literature: The sources for our research

Goals of the research

• Receive the Accuracy score from the article:
  • 90.37% in SEED
  • 76.43% in SEED-IV
  • 77.44% and 75.26% in the valence and arousal dimensions of the DREAMER (Done for GCNN model)
• [ ]

TODO list

• Create preprocessing script (in lib/) for EEG data to pass data into the model (now it is in )
• Understand the input shape and format of data for the GNN model (Georgiy), until 18.10
• Almost copy-paste Implement the GNN model from https://github.com/xyk0058/STGAT (Georgiy), until 19.10
• (OPTIONAL) Maybe add some architecture from https://github.com/hulianyuyy/STGAT/tree/main

Experiments

All experiments with architectures and data preprocessing are described as .ipynb and other logs/reports in experiments/ folder.

• STGAT_METR_LA_colab_run.ipynb

  First attempt with STGAT architecture was on METR-LA dataset. The code (train.py, utils.py, lib/generate_training_data.py, lib/data_preparation.py, lib/metrics.py, lib/utils.py) was taken simply from STGAT repo.

  There was regression problem of traffic forecasting with GNN. The main disadvantage - we should somehow estimate the original adjacency matrix.

  But this model simply was used to check the architecture - the logs is located in Weights&Biases here and generated as the report in this repo STGAT_reproducibility report

• experiments/EEG_DatasetsCreation.ipynb

  This notebook was created after several attempts of data preprocessing. We've found the torcheeg library with great datasets preparation for Dataloders for CNN's architectures as well as for GNNs.

  The methods in this notebooks will create tmp directory with the specific observations of time series. As the example, we've used DREAMER dataset. The preprocessing for it was only adding the adjacency matrix (from the electrode placement, which is stored also in constants in torcheeg).

  For SEED and SEED IV we also generated the preprocessed data, but it takes 40 Gb of the space, so we reproduce (up to the 2023.10.28) only the DREAMER results of classification.

• STGATE_run_DREAMER

  The main notebook for the classification model on the DREAMER. Instead of using the attention layers on 2d (as for traffic and skeleton-based STGAT), we should take 1d Convs on our channels. But then, in the attention layer we had the shape mismatch problem (up to the 2023.10.28). So, this notebook is about the unsuccess

• DGCNN_RUN

  If simply take the CNN instead of attention layers, we will receive the DGCNN network. The run of this model is described in this notebook.

  The logs of two runs (for arousal and valence classes of DREAMER) are located here, WB and, as the pdf-report, in this repo DGCNN_DREAMER report

Models

All architectures, somehow related to the STGATE, are located in models/ directory.

The main architectures are models/stgat.py and models/stgate.py. But they are not working properly for the classification (up to the 2023.10.28)