How to implement cross-camera ReID

[JQZhai]

Describe what you want to do

1. What input videos you will provide, if any:
   Two rtsp video streams
2. What outputs you are expecting:
   Re-identify the same person in different cameras
3. Ask your questions here, if any:
   How should I modify app.py

Your Environment

• NVIDIA Jetson
  • Which Jetson?
  • xavier
  • Jetpack version:
  • 4.4.1
  • Ran install_jetson.sh?
  • yes
  • Reinstalled OpenCV from Jetpack?
  • no

Any idea?Thanks.

[GeekAlexis]

Do your two RTSP streams have non overlapping views and the same frame rate?

[JQZhai]

Do your two RTSP streams have non overlapping views and the same frame rate?

Hi,GeekAlexis:
yes,the two RTSP streams have non overlapping views,and their frame rates are the same.

[GeekAlexis]

Your case is doable but it requires a nontrivial amount of code changes. Track history needs to be shared across different MOT objects. I'll try to create a demo script for multi-camera tracking. The initial version might only support non-overlapping views. It would also help if anyone can provide sample videos to test this.

[JQZhai]

Thank you for your excellent work, I am looking forward to the demo script for multi-camera tracking.

[weilenglxl]

Hi,Alexis.I'm also looking forward to this demo, and the dataset here can be helpful:
https://www.epfl.ch/labs/cvlab/data/data-pom-index-php/
https://bitbucket.org/merayxu/multiview-object-tracking-dataset/src/master/
Thanks for your excellent work again.

[weilenglxl]

Hi,GeekAlexis.If the function is like running four cameras at the same time in deepstream as shown above, how should it be added to FastMOT?I think it will be helpful for cross-camera ReID task.

[weilenglxl]

deepstream_multcamera.zip
Please refer to this code from Deepstream with py.Thanks a lot!

[GeekAlexis]

I'm not familiar with the Deep Stream SDK. You might need to add an appsink in the middle of your pipeline to feed the frames into FastMOT. Related issue: #132

[weilenglxl]

OK, I got it,thanks. Now if I don't use Deep Stream SDK. How do I implement four rtsp streams at the same time in FastMOT?The device I use is a jetson NX that can support four network ports.
At the beginning, I considered using four NX devices to run FastMOT separately, sending features to the data center server for feature matching, and then transferring the results to NXs, thus realizing cross-camera ReID, but I was not sure if it could be done in real time, and if data correlation could be achieved in one set of NX, it also solved the cost and resource overhead.

[GeekAlexis]

You can create 4 fastmot.VideoIO objects.

[weilenglxl]

Oh,that is a good idea,I will try it,thanks!

[weilenglxl]

Hi Alex, is it possible for me to store the features embenddings features extracted by the OsNet model in the host's database by setting up a kafka server on the host to create topics? For example mysql etc. If so where should I put the kafka code in your code? Thanks.

[CompliceU]

@GeekAlexis Hello, if the camera I am using has overlapping areas, I don't know how to deal with this situation better.