GPU issues when creating supervoxels

[nclprz]

Hi,

I have installed SuRVoS from conda channel (using Anaconda on Windows), following all the steps of the Installation guide. The environment uses Python 3.6, and I have checked all the dependencies (I was initially experiencing crashes when clicking some buttons of the GUI, but that seems to have been fixed by updating some packages according to a more recent requirements.txt that I found among the closed issues).
Now all the different filters seem to work just fine, however when I try to compute supervoxels the GUI crashes and I receive this error message on the Anaconda prompt:

create_supervoxels

• Loading data into memory
• Computing supervoxels
  [GPU] Selected CUDA device: 0
  [GPU] Device 0: "Quadro K4200" with compute capability 3.0, 1024 threads pb.
  [CUDA ERROR] Memcpy back: unspecified launch failure
  QObject::~QObject: Timers cannot be stopped from another thread

Any ideas on what might be the issue? I have tried googling but what I found would require me to change the code, which I am not feeling comfortable enough in trying (it also seems odd that I would be the first to report such an issue, if that was the case).

All the best,
Niccolò

[OllyK]

Hi Niccolo,

Sorry to hear about the issues. Do you always get the same error even if you change the supervoxel parameters? Also it would be useful to know what code your googling suggested needs changing to fix it and was the requirements.txt file you found the one from this issue #105 ?

Best Wishes,

Olly

[nclprz]

Hi Olly,

thank you for the quick reply!
Following your question I actually ran some further tests and I managed to have it run successfully (with default supervoxel parameters) up to a ROI of 300 x 350 x 350. I was too ambitious, I was trying to run it on a 500 x 500 x 500 ROI from the get go.
Unfortunately it crashed again with the same error message as soon as I tried to increase the ROI to 350 x 350 x 350. Is it a hardware limit that I have already hit? As you can see in my error message, my GPU is a Quadro K4200.
As additional tests, while keeping the ROI at 300 x 350 x 350 I tried to change the default parameters. It ran just fine with SP shape (5,5,5) and (20,20,20) when used on the raw data, while it ran fine with SP shape (5,5,5) but crashed with (20,20,20) when I tried to use it on the gaussian filtered data. In this crash, the error message was not specified:

create_supervoxels

• Loading data into memory
• Computing supervoxels
  [GPU] Reusing device (0) with 1024 threads
  .

I haven't played with the spacing parameter, as I don't believe I will need it. I have tried to change compactness to 15 with both SP shape (5,5,5) and (20,20,20), the behaviour was the same as above (only one crash, on the gaussian filtered data with the larger size).

Regarding the googling, what I meant was that I just googled for "QObject::~QObject: Timers cannot be stopped from another thread" (without quotes) and found some threads both on Qt Forum and stackoverflow (nothing specifically related to SuRVoS, though). The proposed solutions all seemed to require debugging and modifying of the code, while I was hoping that it was just a dependency issue which I could fix without a lot of trouble. Apologies for the misunderstanding!

Yes, the requirements.txt that I used was the one from issue #105. I didn't have PyQt, which I believe was causing the issues with the GUI buttons.

All the best,
Niccolò

[OllyK]

Hi Niccolò,

You're welcome. Thanks for the reply. Yes unfortunately there are some issues with supervoxel calculation in terms of the data size limit (which will depend on your graphics card memory size) and some combinations of parameters that cause issues. We are working to fix them for SuRVoS2 which we hope to release in the New Year. Unfortunately we're not making many changes to this version of SuRVoS anymore but if we resolve some of the issues you found I'll put a bug fix in.
Thanks for the info on the requirements file, I'd tested things on Linux, perhaps Windows Anaconda is doing something different. I'll look into it,

Best Wishes,

Olly

[nclprz]

Hi Olly,

thank you for the reply. It seems odd for it to be a hardware limit, as a 350 x 350 x 350 volume with 16-bit depth should be only 82 MB in size. My GPU is a bit old, but it shouldn't be that bad (it's supposed to have 4GB of memory). Is the supervoxel calculation that demanding on the GPU?
In any case, thank you for your help. I look forward to SuRVoS2.

Best,
Niccolò

[OllyK]

Hi Niccolò,

It's not quite as simple as just the data size, since there seems to be a limit on the number of supervoxels. You might be able to process larger data if you use larger supervoxels. Also the GPU based supervoxel algorithm is inefficient in some ways - multiple searches are made in parallel for each pixel but this is still faster than using a CPU.

Best Wishes,

Olly