… compliance tests (mlcommons#1576)

* Fix offline_min_samples in submission checker and mlcommons#1569

* Removed mlperf.conf from llama2 directory to avoid confusion

* Update submission_checker.py

* Fixes for 4.0

* Cleanup compliance dir check for models without compliance tests

…emote 'compliance/sources_checksums.json' (mlcommons#1582)

Co-authored-by: mlcommons-bot <null>

Co-authored-by: Miro <[email protected]>

Co-authored-by: Miro <[email protected]>

…'compliance/check.py' (mlcommons#1587)

Co-authored-by: mlcommons-bot <null>

* Ignore trailing whitespace lines in spl.txt files.

* Remove fix from sync'ed power_checker.py.

* Reformat according to black.

)

…mlcommons#1591)

* Add support to dump 10 compliance images during accuracy run for SDXL

* Fix typo

* Dump caption.txt in the same path

…_log_sampling_target is enabled (mlcommons#1599)

* Fix loadgen token metrics latency constrains

* Update perf constraints check for token metrics

* Add equal issue mode for LLMs models

* Add sample length check to test06

* Remove spaces in token metrics recomendation

* Add important item to Llama readme

* Fix Bug: number of tokens logged before computing them

* Fix typo: lenght -> length

* Enable equal issue mode for LLM benchmarks

* Reduce min_query_count to 1 for server/MS/SS

* Remove scenario

* Remove min_query_count so default is used; revoke padding change for equal issue offline

* Pad min_queries, not samples_per_query for non-offline

* Add documentation to the sample equal issue

Co-authored-by: Miro <[email protected]>

* Update README.md

No longer need custom fork as the relevant changes are in the inference repository

* Update dataset.py

---------

Co-authored-by: Miro <[email protected]>

Co-authored-by: Miro <[email protected]>

…and dlrmv2 models (mlcommons#1604)

* Update README.md

Add CM commands to download Stable diffusion models

* Update README.md

* Update README.md

* Turn equal issue mode off for Llama2 TEST06

* Add TEST06 to the output dir

* Fix submission checker and TEST06 for Llama2

* Remove redundant line

* Move test_dir check

…UNet) (mlcommons#1624)

Currently 3D-UNet is the only workload using equal-issue mode on Offline scenario. 
Recent code change on LLM equal-issue mode caused 3D-UNet accuracy run to run more than 1 queries, causing the accuracy log to bloat and fail the accuracy checking script.
This change fixes the problem described above.

* Hotfix: DLRMv2 Audit Test01 fallback failure 

DLRMv2 Audit TEST01 may go to fallback route and the accuracy check script (accuracy-dlrm.py) didn't expect this to happen. It always expects entire sample set to be in the accuracy log while Audit TEST01 would generate subset only.

This fixes the Audit TEST01 failure described above.

* typo fix

…g low accuracy results (mlcommons#1627)