-
Notifications
You must be signed in to change notification settings - Fork 15
Open
Labels
AWMLUpdates for AWMLUpdates for AWML
Description
Description
It seems that the AP computation is slightly different from the AP computation from nuScenes, where it computes and interpolates a precision for 101 recall bins (0.0 - 1.0) to smooth the PR curve. However, it simply gets the maximum precision for a recall bin, which I believe it is similar to 11-point interpolated AP
I am suggesting that we should also add a feature to filter out min_recall/min_precision, and also TP errors, for example, trans_error, orientation error, therefore, it can decreases the discrepancy if someone evaluates models based on nuScenes metrics.
Purpose
- To reduce discrepancy for models that evaluate in different metrics
Possible approaches
- Add a class
NusceneAP
for the 101 recall bins, and makeMap
to support it
Definition of done
- Implement the nuScene metrics as described
ktro2828
Metadata
Metadata
Assignees
Labels
AWMLUpdates for AWMLUpdates for AWML