Skip to content

[TODO]: Add nuScene metric computation #203

@KSeangTan

Description

@KSeangTan

Description

It seems that the AP computation is slightly different from the AP computation from nuScenes, where it computes and interpolates a precision for 101 recall bins (0.0 - 1.0) to smooth the PR curve. However, it simply gets the maximum precision for a recall bin, which I believe it is similar to 11-point interpolated AP

I am suggesting that we should also add a feature to filter out min_recall/min_precision, and also TP errors, for example, trans_error, orientation error, therefore, it can decreases the discrepancy if someone evaluates models based on nuScenes metrics.

Purpose

  1. To reduce discrepancy for models that evaluate in different metrics

Possible approaches

  1. Add a class NusceneAP for the 101 recall bins, and make Map to support it

Definition of done

  1. Implement the nuScene metrics as described

Metadata

Metadata

Assignees

Labels

AWMLUpdates for AWML

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions