add a column of self-reported precision/numerics to MLPerf results table

[christ1ne]

Currently there is no easy way for the table users to tell whether it is a FP32, INT8 or other numerical being used.
This essentially discourages any FP32 submission but most inference are still in FP32. Adding the numerics will help the users to further determine the relevance of the results to their needs.

@tjablin @DilipSequeira FYI.

[christ1ne]

WG:

• concerns: self-reporting will lead to messy outcome
• objective should be define the efforts required to produce this result
• please add your thoughts offline to this issue. we will discuss again in Jan, 2021.

[christ1ne]

WG: need a specific example AR: @christ1ne

[christ1ne]

precision_example.xlsx
@DilipSequeira please let me know what you think? This precision info is from the json in the measurement directories.

[DilipSequeira]

The format looks reasonable (I would slightly prefer colors to postfix characters but I expect there are UI accessibility concerns.)
A larger problem is having rules for transforming from what is essentially a freeform text field (a submitter could reasonably write there e.g. "INT8/BF16" if they're using mixed-precision) to a precision annotation. How would we handle that case?

[christ1ne]

We allow all numerics but we should have a good list to take care of different capitalization, naming nuances.

[christ1ne]

WG:
concerns:

1. symbols are hard to work with in a numerics table.
2. The list of precision choices could be hard to determine. -> we can defer the list of precision choices to the review period

However the WG think we should do this and we will find a UI solution in the review period. Possibly a different view of the results table in a different link.