Run demo.py, while only loops twice and ends

[psiydown]

The result was that the file was not saved and there were no error messages. What is the reason?
Tracked to, second loop, \dpvo\dpvo.py: if self.n>0 and not self.is_ Initialized: Print out: n=1, is_ Initialized=False, motion_probe()=NULL, The return has ended.
@lahavlipson

[lahavlipson]

The most likely reason is that the input data was not found. Can you double check that the video_stream or image_stream is actually loading the images?

[psiydown]

Hello @lahavlipson
I have tested that the images and videos are correct and inputted, and can print out the shape of the images. What I want to emphasize is that installing DPVO on Windows does not support the long type. I need to change the long type of the following three files to the long long type in order to compile and install them successfully.
correlation_kernel.cu
ba.cpp
ba_cuda.cu

I'm not sure if changing the long long type is causing it to not work properly, or if DPVO is not compatible with Windows. Can you identify the cause of the problem or test compatibility with Windows? Thank you very much!

[lahavlipson]

I wasn't familiar with this issue, thanks for letting me know; I will investigate.

[psiydown]

Hello, @lahavlipson
I found through testing that, It exited while executing the 'motion_probe()' function, and did not reach the return step. I am not familiar with the purpose of this function, Is there a solution to replace this function?
https://github.com/princeton-vl/DPVO/blob/main/dpvo/dpvo.py#L376
https://github.com/princeton-vl/DPVO/blob/main/dpvo/dpvo.py#L205-220

[carlosedubarreto]

I did go further and with me it was exiting on this line (I'm also on windows and had to compile hanging the source from long to int64_t (i think it was this):

DPVO/dpvo/dpvo.py

Line 188 in 5833835

coords = pops.transform(SE3(self.poses), self.patches, self.intrinsics, ii, jj, kk)

it stops running wihtout any error.

[psiydown]

@carlosedubarreto Yes, you are correct. It actually exited at this line:

DPVO/dpvo/projective_ops.py

Line 60 in 5833835

Gij = poses[:, jj] * poses[:, ii].inv()

motion_probe()->reproject()->pops.transform()

[carlosedubarreto]

Even more specific, while debugging, the error happens at exactly this point

poses[:, ii].inv()

If i watch this part, the process crashes also while debugging giving no error.

I'm starting to think it might be something related with pytorch. I plan to test on other pytorch version. Since I was using to process wham, I'm with pytorch 1.11 cuda 11.3

[psiydown]

@carlosedubarreto
I am also using WHAM, pytorch 1.11 cuda 11.3 is the same as your version.

I captured the entire process of error occurrence using python -X faulthandler demo.py,

looking from bottom to top:

Current thread 0x000044f4 (most recent call first):
  File "N:\Python\WHAM-main\third-party\DPVO\dpvo\lietorch\group_ops.py", line 14 in forward
  File "N:\Python\WHAM-main\third-party\DPVO\dpvo\lietorch\groups.py", line 129 in apply_op
  File "N:\Python\WHAM-main\third-party\DPVO\dpvo\lietorch\groups.py", line 147 in inv
  File "N:\Python\WHAM-main\third-party\DPVO\dpvo\projective_ops.py", line 60 in transform
  File "N:\Python\WHAM-main\third-party\DPVO\dpvo\dpvo.py", line 188 in reproject
  File "N:\Python\WHAM-main\third-party\DPVO\dpvo\dpvo.py", line 212 in motion_probe
  File "N:\Python\WHAM-main\third-party\DPVO\dpvo\dpvo.py", line 376 in __call__
  File "N:\Python\WHAM-main\third-party\DPVO\demo.py", line 51 in run
  File "N:\anaconda3\lib\site-packages\torch\autograd\grad_mode.py", line 27 in decorate_context
  File "N:\Python\WHAM-main\third-party\DPVO\demo.py", line 92 in <module>

[carlosedubarreto]

great finding @psiydown .

so probably the guy that is causing problems is the compilation of lietorch...
I'll try to compile it using some other option instead of the "int64_t" replacong the "long" word.

[psiydown]

@carlosedubarreto
I have tried changing 'long' to 'long long' type instead of ‘int64_t’ type, the result is the same.

[lahavlipson]

I will investigate this issue. But if the issue is lietorch specifically, then in the meantime you can replace the lietorch.SE3 type with the SE3 type from pypose. The data types are virtually identical, but the latter is written entirely in python. You should only need to replace SE3.exp -> pp.Exp and .inv() -> .Inv()

[carlosedubarreto]

@lahavlipson , I made a lot of changes to use the pypose library, but it didnt work too.
Nover thought it would be this hard.

thanks for the help

[psiydown]

@lahavlipson @carlosedubarreto

Regarding this issue, lietorch cannot be used on Windows 10.He suggest "managed to fix this on Windows by making all the CUDA kernels in \lietorch\src\lietorch_gpu.cu accept regular pointers where currently they accept const pointers. " But I don't know how to change accept regular pointers, hope your know how to do so.

[carlosedubarreto]

Another great finding @psiydown .
I dont know much about compiling, but I'll tkae a look on that possibility. thanks a lot.

[carlosedubarreto]

@psiydown I was able to make it work.

I changes the files from long to int64_t and also removed the "const" from the lietorch_gpu.cu

Here are the files I changed to be able to compile and not crash.

fastba.zip

lietorch_gpu.zip

correlation_kernel.zip

[psiydown]

@carlosedubarreto
This is a great discovery, I have succeeded, thank you very much for your help!

Next, I plan to implement real-time camera WHAM inference. If you are interested, if you encounter any problems during the implementation process, we can communicate via email and exchange opinions. My email: [email protected]