Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Finalize afni_proc.py settings #29

Open
tsalo opened this issue Oct 21, 2023 · 14 comments
Open

Finalize afni_proc.py settings #29

tsalo opened this issue Oct 21, 2023 · 14 comments
Labels
question Further information is requested

Comments

@tsalo
Copy link
Member

tsalo commented Oct 21, 2023

@handwerkerd do you have any recommendations for running afni_proc.py on multi-echo data? Is there a way to ensure that the outputs are roughly similar to BIDS format?

Here's a copy from the help (feel free to edit as you wish):

#!/bin/tcsh

# --------------------------------------------------
# note fixed top-level directories
set data_root = /main/location/of/all/data

set input_root = $data_root/scanner_data
set output_root = $data_root/subject_analysis

# --------------------------------------------------
# get a list of subjects, or just use one (consider $argv)
cd $input root
set subjects = ( subj* )
cd -

# or perhaps just process one subject?
set subjects = ( subj_017 )

# --------------------------------------------------
# process all subjects
foreach subj_id ( $subjects )

  # --------------------------------------------------
  # note input and output directories
  set subj_indir = $input_root/$subj_id
  set subj_outdir = $output_root/$subj_id

  # --------------------------------------------------
  # if output dir exists, this subject has already been processed
  if ( -d $subj_outdir ) then
     echo "** results dir already exists, skipping subject $subj_id"
     continue
  endif

  # --------------------------------------------------
  # otherwise create the output directory, write an afni_proc.py
  # command to it, and fire it up
  mkdir -p $subj_outdir
  cd $subj_outdir

  # create a run.afni_proc script in this directory
  cat > run.afni_proc << EOF

  # notes:
  #   - consider different named inputs (rather than OutBrick)
  #   - verify how many time points to remove at start (using 5)
  #   - note which template space is preferable (using MNI)
  #   - consider non-linear alignment via -tlrc_NL_warp
  #   - choose blur size (using FWHM = 4 mm)
  #   - choose basis function (using BLOCK(2,1), for example)
  #   - assuming 4 CPUs for linear regression
  #   - afni_proc.py will actually run the proc script (-execute)
  afni_proc.py -subj_id $subj_id                          \
      -blocks tshift align tlrc volreg blur mask regress  \
      -copy_anat $subj_indir/anat+orig                    \
      -dsets                                              \
          $subj_indir/epi_r1+orig                         \
          $subj_indir/epi_r2+orig                         \
          $subj_indir/epi_r3+orig                         \
      -tcat_remove_first_trs 5                            \
      -align_opts_aea -cost lpc+ZZ                        \
      -tlrc_base MNI152_2009_template.nii.gz              \
      -tlrc_NL_warp                                       \
      -volreg_align_to MIN_OUTLIER                        \
      -volreg_align_e2a                                   \
      -volreg_tlrc_warp                                   \
      -blur_size 4.0                                      \
      -regress_motion_per_run                             \
      -regress_censor_motion 0.3                          \
      -regress_reml_exec -regress_3dD_stop                \
      -regress_stim_times                                 \
          $stim_dir/houses.txt                            \
          $stim_dir/faces.txt                             \
          $stim_dir/doughnuts.txt                         \
          $stim_dir/pizza.txt                             \
      -regress_stim_labels                                \
          house face nuts za                              \
      -regress_basis 'BLOCK(2,1)'                         \
      -regress_opts_3dD                                   \
          -jobs 4                                         \
          -gltsym 'SYM: house -face' -glt_label 1 H-F     \
          -gltsym 'SYM: nuts -za'    -glt_label 2 N-Z     \
      -regress_est_blur_errts                             \
      -blocks tshift align tlrc volreg mask combine blur scale regress \
      -dsets_me_echo   epi_run*_echo_01.nii                            \
      -dsets_me_echo   epi_run*_echo_02.nii                            \
      -dsets_me_echo   epi_run*_echo_03.nii                            \
      -echo_times      15 30.5 41                                      \
      -mask_epi_anat   yes                                             \
      -combine_method  OC
      -execute

EOF
# EOF terminates the 'cat > run.afni_proc' command, above
# (it must not be indented in the script)

  # now run the analysis (generate proc and execute)
  tcsh run.afni_proc

# end loop over subjects
end
@tsalo tsalo added the question Further information is requested label Oct 21, 2023
@handwerkerd
Copy link
Member

In addition to the section you copied from https://afni.nimh.nih.gov/pub/dist/doc/program_help/afni_proc.py.html examples 12-13 list multiple ways to use multi-echo with AFNI. The one you copied includes -combine_method OC which means it will proceed using just the optimally combined data. To work with tedana -combine_method should be one of the following options:

            m_tedana         : tedana from MEICA group (dn_ts_OC.nii.gz)
            m_tedana_OC      : tedana OC from MEICA group (ts_OC.nii.gz)
            m_tedana_m_tedort: tedana from MEICA group (dn_ts_OC.nii.gz)
                               "tedort" from MEICA group
                               (--tedort: "good" projected from "bad")

If any of those options are used, it will run whichever version of tedana is at the top of the user's path.

-combine_opts_tedana allows users to pass any additional options directly to tedana One issue, which I suspect is the core of your question is that I tried -combine_opts_tedana --convention bids and that didn't work because afni_proc was always passing --convention orig. This is necessary because AFNI has its own file naming conventions. It would be possible to have AFNI use the tedana file name registry to identify the correct files for the next steps, but AFNI would still continue to use its own file naming conventions for all later processing steps.

Last time I asked @mrneont and @afni-rickr they had no plans to have afni_proc natively use something similar to BIDS-derivatives filenaming, but afni_proc creates a file name registry. It should be very possible to take that registry and remap all files into a BIDS structure. I think they did some work on that, but I don't know the current status.

@tsalo
Copy link
Member Author

tsalo commented Oct 23, 2023

To work with tedana -combine_method should be one of the following options

My goal would actually be to run tedana separately, since these datasets are meant to act as inputs to tedana. Can afni_proc be used to produce minimally-preprocessed echo-wise outputs?

It should be very possible to take that registry and remap all files into a BIDS structure. I think they did some work on that, but I don't know the current status.

That's a great idea! I'll try that.

@handwerkerd
Copy link
Member

If you use you the example you copied with -combine_method OC then it would unnecessary calculate a T2* map and the optimally combined time series, but it would give you close to what you want.
For the -blocks option, you'd also want to remove everything after combine. That is replace -blocks tshift align tlrc volreg mask combine blur scale regress with -blocks tshift align tlrc volreg mask combine With the shorter version, some useful quality control outputs wouldn't be given, but that may or may not be an issue depending on how you plan to use these data. (Also there's something wrong with the command you copied above in that has two -blocks optiosn and both -dsets & -dsets_me_echo This is different from their docs so I'm not sure where you got it from)

@tsalo tsalo mentioned this issue Oct 23, 2023
@tsalo
Copy link
Member Author

tsalo commented Oct 23, 2023

@handwerkerd sorry for the noise. I don't know where I copied the incorrect parameters from.

As far as a good repo for a template script goes, what about the Multi-echo masking test dataset (https://openneuro.org/datasets/ds002156)? There's only one subject, so it's small.

The inputs can be:

sub-23638/ses-01/anat/sub-23638_ses-01_acq-mpragePURE_run-1_T1w.nii.gz
sub-23638/ses-01/func/sub-23638_ses-01_task-rest_run-1_echo-1_bold.nii.gz
sub-23638/ses-01/func/sub-23638_ses-01_task-rest_run-1_echo-2_bold.nii.gz
sub-23638/ses-01/func/sub-23638_ses-01_task-rest_run-1_echo-3_bold.nii.gz
sub-23638/ses-01/func/sub-23638_ses-01_task-rest_run-1_echo-4_bold.nii.gz

The echo times are 15.1, 28.7, 42.3, and 55.9.

I'm hoping that afni_proc can be used to produce (1) minimally-preprocessed echo-wise data to run tedana on, (2) a brain mask in the echo-wise images' space for tedana, and (3) optimally-combined data in standard space, which I could denoise using the tedana results. What do you think?

@mrneont
Copy link

mrneont commented Oct 24, 2023

Howdy-

We actually do have a mapping of afni_proc.py outputs to current BIDS-derivatives naming; we just haven't had time to implement that as an option yet. Hopefully soon after the Bootcamp...

For multiecho FMRI data, we do have a fair number of afni_proc.py examples in the APMULTI demo, which is downloadable for resting state FMRI with:

@Install_APMULTI_Demo1_rest

https://afni.nimh.nih.gov/pub/dist/OHBM2022/OHBM2022_tayloretal_apmulti.pdf

@tsalo : it sounds like you want just the datasets output by using OC or tedana, then? Or with further regression processing? Are the pbcombine datasets what you would like, at present?

--pt

@handwerkerd
Copy link
Member

@tsalo Do you want the data aligned to a template space or in native space? If you want a template, which one?

@tsalo
Copy link
Member Author

tsalo commented Oct 24, 2023

@mrneont that's awesome! I mainly want these AFNI derivatives to be useful for testing tedana, so the primary output I'm interested in is minimally-preprocessed individual echoes, along with some way to either transform outputs from running tedana separately on those echoes to standard space or optimally combined data in standard space that can be denoised using the results from running tedana separately. I'd love to have regressors that could be used for regression (e.g., motion parameters), but not denoised data.

@handwerkerd if possible, I'd love to have the echo-wise outputs in native space (per our recommendations in the tedana docs) and optimally combined outputs in standard space. Maybe MNI152NLin6Asym, since that's the one I'll use for fMRIPrep?

@handwerkerd
Copy link
Member

When afni_proc aligns to a template it does a single spatial transformation for both motion correction and alignment. (Probably time to revisit that recommendation ME-ICA/tedana#990 ).

Do get what you want, I can write one afni_proc with and one without spatial alignment, or I can just run with spatial alignment and use a stand-along AFNI command to create volumes for each echo that aren't aligned to the template.

AFNI includes a bunch of MNI templates, but I'm not sure which one matches MNI152NLin6Asym. Can you point me to the template volumes fMRIPrep uses so I can make sure to align to the exact same one?

@tsalo
Copy link
Member Author

tsalo commented Oct 24, 2023

Do get what you want, I can write one afni_proc with and one without spatial alignment, or I can just run with spatial alignment and use a stand-along AFNI command to create volumes for each echo that aren't aligned to the template.

That would be great, thank you!

AFNI includes a bunch of MNI templates, but I'm not sure which one matches MNI152NLin6Asym. Can you point me to the template volumes fMRIPrep uses so I can make sure to align to the exact same one?

fMRIPrep grabs its templates from templateflow (https://www.templateflow.org/browse/). The template is under the tpl-MNI152NLin6Asym folder. It's the file tpl-MNI152NLin6Asym_res-02_T1w.nii.gz.

@afni-rickr
Copy link

afni-rickr commented Oct 25, 2023 via email

@handwerkerd
Copy link
Member

Here is a sample script for the above dataset. I have a version that runs regress to get a bunch of quality control outputs. This really might be useful, but, for this run something was going wrong with the MNI alignment. My best guess is that either the template image or the original data has something that diverges from the NIFTI standard and makes AFNI unhappy. I can talk more to @afni-rickr to check/confirm the problem isn't on AFNI's side, but the following should work fine for your current plans.

sbjID=sub-23638
ses=ses-01
task=rest_run-1
echo_times=(15.1 28.7 42.3 55.9)
rootdir=~/code/meica/data_testing/Multi-echo_masking_test_dataset/${sbjID}/${ses}
anat=anat/${sbjID}_${ses}_acq-mpragePURE_run-1_T1w.nii.gz

# The data had several non-steady state volumes at the beginning.
# This will be used to remove the first four
remove_first_trs=4

# template from here: https://www.templateflow.org/browse/ in tpl-MNI152NLin6Asym
# Using the T1 that was already skull stripped
template=../../template/tpl-MNI152NLin6Asym_res-02_desc-brain_T1w.nii.gz

# You probably want to put files elsewhere, but fine for this demo
cd $rootdir

# Runs AFNI proc in native space
# The pre-processed runs to input into tedana are called pb03.${sbjID}.r01.e0?.volreg+orig.HEAD
afni_proc.py -subj_id ${sbjID} \
   -blocks despike tshift align volreg mask combine tlrc \
   -dsets_me_run  ./func/${sbjID}_${ses}_task-${task}_echo-1_bold.nii.gz \
       ./func/${sbjID}_${ses}_task-${task}_echo-2_bold.nii.gz \
       ./func/${sbjID}_${ses}_task-${task}_echo-3_bold.nii.gz \
       ./func/${sbjID}_${ses}_task-${task}_echo-4_bold.nii.gz \
   -tlrc_base $template \
   -echo_times $echo_times \
   -reg_echo 2 \
   -copy_anat $anat \
   -anat_has_skull yes \
   -mask_epi_anat yes \
   -align_unifize_epi local \
   -volreg_align_e2a \
   -tcat_remove_first_trs $remove_first_trs \
   -execute

# The above afni_proc calculates an alignment to template space, but doesn't use it
# The following will apply the transform to write out the echo data in template space
# The new files are called pb03a.${sbjID}.r01.e01.volreg+tlrc.HEAD (lousy naming, acknowledged) 
cd ${rootdir}/${sbjID}.results
echo_list=(01 02 03 04)
for eind in ${echo_list[@]}; do
    3dAllineate -base ../$template    \
                   -input pb03.${sbjID}.r01.e${eind}.volreg+orig.HEAD                 \
                   -1Dmatrix_apply warp.anat.Xat.1D                    \
                   -mast_dxyz 3.5                                             \
                   -prefix pb03a.${sbjID}.r01.e${eind}.volreg+tlrc.HEAD
done

@afni-rickr
Copy link

Hi @handwerkerd. To get the EPI into standard space (without the extra resampling at the end), use:
-blocks despike tshift align tlrc volreg mask combine regress
It needs the tlrc block before volreg in order to concatenate the transformations in the volreg block. And I just threw the 'regress' block in there for QC.
I will try to get this data and try the command locally.

@afni-rickr
Copy link

Hi @handwerkerd. The script worked, at least at a quick glance. I did use the blocks from above
-blocks despike tshift align tlrc volreg mask combine regress
and to actually warp the EPI into standard space requires the -volreg_tlrc_warp option.

@tsalo
Copy link
Member Author

tsalo commented Nov 10, 2023

Thanks so much @afni-rickr @handwerkerd @mrneont! Our server is down at the moment, but I will try the updated command on one of the example datasets soon.

@tsalo tsalo pinned this issue Feb 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

4 participants