Gadgetron topic

[aTrotier]

Hi,

I am starting to play around with the julia interface to Gadgetron (last version here) that @dchansen already developed it and it seems to work.

I will first convert one of my paper which use matlab as an example (and also maybe compare the reconstruction time between the 2 interfaces).

I am expecting to have some issue or reflexion when using it with MRIReco.jl because Gadgetron.jl is using specific structure :

• Acquisition
• AcquisitionBucket
• ReconData
• Waveform
• Image

Maybe we can start a little task force to discuss how we should interface it in order to perform online reconstruction

Some of you are interested in this project ? @mrikasper @alexjaffray

[dchansen]

Let me know if there's something I can do to ease the process.

[aTrotier]

Thanks,
Is there some features that are missing from Gadgetron.jl compared to matlab / python ?

I think most of the work will be on the MRIReco.jl package in order to handle the conversion from the gadgetron structure to the MRIReco.RawAcquisitionData and MRIReco.AcquisitionData

[dchansen]

No, it should be feature complete.

However, both Matlab and Python have libraries for working with the (ISMR)MRD types and Julia does not, except for the reader in MRIReco.

[mrikasper]

Excellent initiative, Aurélien @aTrotier!

I wholeheartedly support it, but I don't think I have the bandwidth at the moment to help with the implementation right now. I am adding @nbwuzhe here, who maintains our Gadgetron server at @BRAIN-TO

Maybe we can get some funding to get a few more people involved (and paid) for such an effort?

[aTrotier]

Let's first build a working example and see how it goes.

After that I think it might be great to create the documentation with documenter.jl. We can try to adapt the matlab example that I have created for the GadgetronOnlineClass

[aTrotier]

Gadgetron can send different object to julia. One of them is an Acquisition structure.

In MRIBase.jl, the profile object is a reimplementation of the same object.

Ideally, we should get accumulate the gadgetron Acquisition object in a vector and then convert them into the RawAcquisitionData then AcquisitionData to perform the reconstruction with MRIReco.jl

@tknopp Do you have a suggestion about how we can merge the 2 implementations ?

[tknopp]

Haha, can we agree on that the Gadgetron.jl implementation is are reimplementation of MRIBase.jl?

Honestly, from my perspective Gadgetron.jl should simply use the implementation from MRIBase.jl and MRIFiles.jl. And if there is something missing than lets just extend these packages. That was actually the vision of MRIReco.jl and this Github organization MagneticResonanceImaging. To have a coherent and strong Julia software stack where we all work together. So, I would be very happy if the implementations could be consolidated. @dchansen

[dchansen]

Haha, can we agree on that the Gadgetron.jl implementation is are reimplementation of MRIBase.jl?
Nope. Only so many ways you can reimplement the ISMRMRD C struct in Julia.

So, Gadgetron.jl predates MRIBase.jl and MRIFiles.jl being separate packages, and all of MRIReco.jl was considered too large a dependence.

MRIFiles.jl unfortunately only supports reading a (very useful) subset of MRD files, and no writing at all.
I had begun work on something more fully featured, but writing complex HDF5 files from Julia is not the best experience. Nothing that couldn't be fixed, but I simply ran out of time.

Using the MRIBase.jl types does not seem ideal to me. The profile object contains redundant data (particularly the size of the data and trajectories are stored both in the AcquisitionHeader and in the actual fields themselves), which makes it a pain to serialize again if the values are not kept in sync manually.

In general, the MRIReco libraries are focused on going as fast as possible from profiles to AcquisitionData, which makes a ton of sense for offline reconstruction.
In an online setting, where data arrives continuously, processing profile by profile is often desirable, and the base types should make this easy and convenient.

All of this being said, I do applaud the goal of a unified MRI stack for Julia. Would updating the types in MRIBase.jl be palatable, or are they widely used outside of MRD support?

Essentially adding RawAcquisitionHeader from Gadgetron.jl to MRIBase and remove number_of_samples and active_channels from AcquisitionHeader. That way we could move Gadgetron.jl to the MRIBase types and @aTrotier 's life would be easy.

[aTrotier]

Let's first built a working example and maybe compare the reconstruction time between Julia and Python/Matlab.

My use-case might not be really impacted by the conversion from the Gadgetron.Acquisition to MRIBase.profile because I can convert them on the fly and stored them in a vector while the acquisition is in progress.
At the end of the acquisition, i'll convert that into a RawAcquisitionData object and then follow the standard MRIReco process.

What about your (futur :) ) implementation @mrikasper ?

[tknopp]

Thanks @dchansen!

So, Gadgetron.jl predates MRIBase.jl and MRIFiles.jl being separate packages, and all of MRIReco.jl was considered too large a dependence.
:-) fair enough. Very good point!

I have not looked deeper into the differences but just some quick replies:

MRIFiles.jl unfortunately only supports reading a (very useful) subset of MRD files, and no writing at all.
That is not correct. Read/Write are both supported. I am also not sure what subset you mean. We should support everything.

Using the MRIBase.jl types does not seem ideal to me. The profile object contains redundant data (particularly the size of the data and trajectories are stored both in the AcquisitionHeader and in the actual fields themselves), which makes it a pain to serialize again if the values are not kept in sync manually.
I don't get this. The RawAcquisitionData object is basically a 1to1 representation of the MRD. The only thing we do differently ist to store the XML in a dictionary. Please don't confuse it with AcquisitionsData, which indeed it tailored towards offline reconstruction.

Would updating the types in MRIBase.jl be palatable, or are they widely used outside of MRD support?

There are some projects using them but we can always make a major version bump. In my view, merging the implementations would be absolutely great. Also getting input from you would be absolutely fantastic.

[aTrotier]

I am also not sure what subset you mean. We should support everything.

I think it means the others type that can be sent from gadgetron to the external software or from the external to gadgetron :

• Acquisition Bucket (it is mostly the vector of acquisition)
• ReconData (I think it is a mostly a zero-filled kspace or something close to that)
• Image

[dchansen]

MRIFiles.jl unfortunately only supports reading a (very useful) subset of MRD files, and no writing at all.
That is not correct. Read/Write are both supported. I am also not sure what subset you mean. We should support everything.
I missed that it supports writing data. That's really nice. What it's missing for full MRD file support is Images and Waveforms.

Using the MRIBase.jl types does not seem ideal to me. The profile object contains redundant data (particularly the size of the data and trajectories are stored both in the AcquisitionHeader and in the actual fields themselves), which makes it a pain to serialize again if the values are not kept in sync manually.
I don't get this. The RawAcquisitionData object is basically a 1to1 representation of the MRD. The only thing we do differently ist to store the XML in a dictionary. Please don't confuse it with AcquisitionsData, which indeed it tailored towards offline reconstruction.

So, the issue here stems from the fact that the C AcquisitionHeader contains not just the header information, but also the size of the data and trajectory arrays. In pretty much any other language, that data is already stored with the array, so it becomes easy to write code like

function pad_acq(acq::Acquisition)
    acq.data = padarray(acq.data, Pad(1, 1)) # Now acq.head.number_of_samples is wrong
end

The ISMRMRD python library spends a lot of energy trying to keep these variables in sync. Gadgetron will silently break if you try to serialize an acquisition like this. I realize this seems like a small issue, but I've seen a lot of users shooting themselves in the foot with this.
In newer MRD projects I've worked on, like Gadgetron.jl, i've done away with the problem by simply not having the duplicate information around.

Would updating the types in MRIBase.jl be palatable, or are they widely used outside of MRD support?

There are some projects using them but we can always make a major version bump. In my view, merging the implementations would be absolutely great. Also getting input from you would be absolutely fantastic.
Sounds good to me.

A last thing is the use of a Dict of the MRD header. This makes it really easy to consume a header, but really hard to write a correct one. The latter is important for things like dicom generation for different scanner vendors.