Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Save all scores from prediction results #635

Open
wants to merge 52 commits into
base: main
Choose a base branch
from

Conversation

mihow
Copy link
Collaborator

@mihow mihow commented Nov 28, 2024

@todo

  • More testing with live ML backend
  • Test migration of existing detections & classifications (they shouldn't have any data in the affected fields)
  • Add Algorithm & AlgorithmCategoryMap schemas to ML backends
  • Ensure the schemes in the example ML backend and live ML backend are the same
  • Update tests
  • Add new tests for category maps
  • Add tests for creation of algorithms & category maps (document how to update either)
  • Fix detection of already processed images (since we now return moth/non-moth classifications, all detections are seen as already processed by every pipeline)
  • Fix occurrences using moth/non-moth algorithm as the determination (likely because score is higher)
  • Fix algorithms being registered multiple times
  • Fix category maps from being registered multiple times
  • Fix reprocessing - always is processing all images
  • Test on fresh export of data from production
  • Verify that full category map + scores map to expected taxa results
  • Fix tests

Benefits of this change:

  • Can apply post-processing filters of species by region (filter and re-weight the scores)
  • Can do genus & higher taxon roll-ups
  • Can get the top N (top 3, 5, 10) suggestions instead of just the first
  • Labels are now entities with GBIF key, synonyms, etc. instead of just a text label - So can better map to Taxon entities in the Antenna database
  • Can use confidence score algorithms other than softmax on the Antenna side (e.g. temperature calibrated)
  • Allow seeing and agreeing with results from multiple models
  • Uses best score from all algorithms to determine the species determination of an occurrence (not just the latest)
  • Ready for detection tracking across frames!
  • Can better send current results back to backend to determine reprocessing needs

While troubleshooting I should have fixed most of the issues reported in #310

Copy link

netlify bot commented Nov 28, 2024

Deploy Preview for ami-dev canceled.

Name Link
🔨 Latest commit e8eb341
🔍 Latest deploy log https://app.netlify.com/sites/ami-dev/deploys/6765eafee3c6570008ea1d32

Base automatically changed from feat/improve-initial-start to main November 28, 2024 01:21
@mihow mihow force-pushed the feat/more-predictions-data branch from 7a615a5 to 9f10aa6 Compare November 28, 2024 01:31
@mihow mihow self-assigned this Nov 28, 2024
@mihow mihow force-pushed the feat/more-predictions-data branch 2 times, most recently from 39f974c to d9604c0 Compare December 7, 2024 02:34
@mihow mihow marked this pull request as ready for review December 19, 2024 05:43
@mihow mihow added this to the ML pipeline enhancements milestone Dec 19, 2024
@mihow mihow force-pushed the feat/more-predictions-data branch from 9d7285a to 89ad145 Compare December 20, 2024 02:12
mihow added 27 commits December 20, 2024 13:22
@mihow mihow force-pushed the feat/more-predictions-data branch from 109b707 to a79f177 Compare December 20, 2024 21:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant