Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add exrmetrics tool #1769

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

peterhillman
Copy link
Contributor

Add tool to measure read/write times of OpenEXR images and compression performance, by reading entire file into memory, then writing it back out again. Zip and DWA compression level can be adjusted. Currently only regular and deep scanline images are supported; tiled images are not implemented yet.

multithreading is not used because that would give inconsistent timings (and is dependent on threading strategy)

The expected usage would be to run exrmetrics twice for each file: first, reading the source image and writing a temporary file with -z <test_codec>, to get write speed of the and file size of the codec on the compressed file, then use that output as input for a run with -z none to get the read speed of the test codec and baseline file size and write speed.

std::string compressionNames;
getCompressionNamesString ("/", compressionNames);
stream
<< "Read a part OpenEXR image from infile, write an identical copy to outfile"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you mean "Read an OpenEXR image part from infile..."?

"\n"
"Options:\n"
"\n"
" -p n part number to copy (only one part will be written to output file)\n"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason to not just do all parts? Seems like in the field you'd like to run this on all parts, and observe the effect on the files as a whole, even if the compression stats are specific to individual parts.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's certainly doable, but there are a bunch of issues: since the parts can have different types, and different compression codecs, generating meaningful output gets messy. The C++ API doesn't give access to the size of individual compressed tiles/scanlines, so the total file size is quoted instead. It would be tricky to quote the compressed size of each part, but some data only makes sense if it's part-specific. A custom stream, or the Core API, might do a better job of that.

Currently, for all types, exrmetrics reads all the data in one go, then writes them all out, timing the total read and write time, since that should give the most reliable statistics. That gets messy to implement if it has to read data all parts first, then write them all out again, using part-specific code for each type.

@peterhillman peterhillman changed the title add exrmetrics tool (no tile support yet) add exrmetrics tool Jun 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants