Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FIX: Ensure dataobj datatype is altered #527

Merged
merged 4 commits into from
May 26, 2020
Merged

Conversation

mgxd
Copy link
Contributor

@mgxd mgxd commented May 26, 2020

brought to attention in nipreps/fmriprep#2142

@pull-assistant
Copy link

pull-assistant bot commented May 26, 2020

Score: 0.79

Best reviewed: commit by commit


Optimal code review plan (1 warning)

     FIX: coerce dataobj to desired data dtype

     Update niworkflows/interfaces/bids.py

Update niworkflows/interfaces/bids.py

niworkflows/interfaces/bids.py 50% changes removed in FIX: coerce to dtype...

     FIX: coerce to dtype after rounding

Powered by Pull Assistant. Last update 9b6c995 ... 2d6d2be. Read the comment docs.

@effigies
Copy link
Member

This will have no effect. img.set_data_dtype() is just a proxy:

https://github.com/nipy/nibabel/blob/966a01e63fd7ce7e58d5971184e31b3305d350f6/nibabel/analyze.py#L931-L932

@effigies
Copy link
Member

I don't fully understand the data_dtype parameter. Is that strictly for the on-disk format? Or is it supposed to coerce the data array, as well?

@codecov
Copy link

codecov bot commented May 26, 2020

Codecov Report

Merging #527 into master will increase coverage by 0.05%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #527      +/-   ##
==========================================
+ Coverage   64.08%   64.14%   +0.05%     
==========================================
  Files          43       43              
  Lines        5288     5285       -3     
  Branches      773      769       -4     
==========================================
+ Hits         3389     3390       +1     
+ Misses       1754     1750       -4     
  Partials      145      145              
Flag Coverage Δ
#documentation 33.88% <0.00%> (-0.01%) ⬇️
#masks ?
#reportlettests 100.00% <ø> (ø)
#travis 59.27% <100.00%> (+<0.01%) ⬆️
#unittests 47.53% <100.00%> (+<0.01%) ⬆️
Impacted Files Coverage Δ
niworkflows/interfaces/bids.py 96.45% <100.00%> (+0.01%) ⬆️
niworkflows/interfaces/images.py 57.37% <0.00%> (+0.15%) ⬆️
niworkflows/viz/utils.py 80.35% <0.00%> (+0.23%) ⬆️
niworkflows/interfaces/mni.py 42.52% <0.00%> (+0.24%) ⬆️
niworkflows/utils/bids.py 75.00% <0.00%> (+0.74%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9b0f646...1a5286e. Read the comment docs.

@mgxd
Copy link
Contributor Author

mgxd commented May 26, 2020

It was meant to only effect the on-disk format. I'm a bit hesitant to coerce the data within the datasink interface, as there is already a lot going on there. If we want to go that route, I'd feel more comfortable splitting that into its own Node

@mgxd mgxd force-pushed the fix/dtype-header branch from 1a5286e to 9b6c995 Compare May 26, 2020 17:30
@mgxd mgxd changed the title FIX: Ensure header datatype is also altered FIX: Ensure dataobj datatype is altered May 26, 2020
Copy link
Member

@oesteban oesteban left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we also want to revise the aparcaseg resampling. This change is necessary (it doesn't make much sense not to cast the data array, despite of the risk of corrupting the data), but we need to make sure the datatype is correctly set on the resampled output.

f"Changing {out_file} dtype from {orig_dtype} to {data_dtype}"
)
# coerce dataobj to new data dtype
new_data = np.asanyarray(nii.dataobj).astype(data_dtype)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
new_data = np.asanyarray(nii.dataobj).astype(data_dtype)
new_data = np.asanyarray(nii.dataobj, dtype=data_dtype)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this going to cast BOLD data to int16?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think nipreps/smriprep#195 is part of the problem, but there is some further step downstream that is upcasting to float. But this is also necessary for any conversion to match source_file (currently just boldref).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@effigies this should only affect data suffixes:

        ("mask", "uint8"),
        ("dseg", "int16"),
        ("probseg", "float32"),
        ("boldref", "source")

unless the data_dtype flag is explicitly checked, which I don't believe it ever is.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think poldracklab/smriprep#195 is part of the problem, but there is some further step downstream that is upcasting to float.

commented there, yes there is a resampling downstream in this case - moving to standard space.

@mgxd
Copy link
Contributor Author

mgxd commented May 26, 2020

I kept running into:

TypeError: No loop matching the specified signature and casting was found for ufunc rint

when trying to set dtype in the rint() call.

Logically, it would have to round first and then apply the dtype to the array anyways.

@mgxd
Copy link
Contributor Author

mgxd commented May 26, 2020

i'll merge this in now and cut a new release, thank you both for the reviews

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants