Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Could results be different between this repo's MDv5 and the older MD repo MDv5? #571

Open
1 task done
VLucet opened this issue Feb 16, 2025 · 3 comments
Open
1 task done
Labels
question Further information is requested

Comments

@VLucet
Copy link
Contributor

VLucet commented Feb 16, 2025

Search before asking

  • I have searched the Pytorch-Wildlife issues and found no similar bug report.

Question

Would it be possible for detection results of MDv5 from this repo to be different from the results produced by https://github.com/agentmorris/MegaDetector ?

Additional

Because providing a minimally repeatable example would be quite difficult (as one needs to do the setup required by both repositories), I thought I would ask whether it is possible to get two different results from the two implementation of the predictions from MD in the two repos. I have this repo's version set up on a VM and it seems to give me different results from the other repo's results.

@VLucet VLucet added the question Further information is requested label Feb 16, 2025
@danielaruizl1
Copy link
Collaborator

Hi @VLucet The models for MDv5 in both repositories are the same and therefore the results should be comparable. However, we are currently validating all our results because other users have also reported inconsistencies for the results we have reported for MDv5. We will be updating this information soon, so stay tuned for our announcements!

Additionally, we would appreciate it if you could provide more details on the differences you have found between the results from both repositories. Thanks for helping us improve!

@VLucet
Copy link
Contributor Author

VLucet commented Feb 20, 2025

Thanks for answering @danielaruizl1. I seem to be getting different scores for the bounding boxes. I seem to get the same detections, but with different scores. I will attempt to small make a docker image with both environments and a simple script that demonstrates this. In the meantime, here is an example of the differences:

This image, generated from the code in https://github.com/agentmorris/MegaDetector, gives me a score of 0.4

Image

The same image in your gradio demo, gets a score of 0.3.

Image

@danielaruizl1
Copy link
Collaborator

Hi @VLucet thanks for the example. Could you share the input image with us so we can test it on our end?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants