Profiling ONNX-Runtime Web (WASM)

[mram0509]

How to go about profiling the inference performance when using ONNX run time in the browser?

[haydn-jones]

This is working for me, though I haven't really done much with it so I could be off base. Logs the profiling results to the console.

const profilerConfig = {maxNumberEvents: 65536};
const session = await ort.InferenceSession.create(..., {
    executionProviders: ["wasm"],
    ...
    profiler: profilerConfig,
    enableProfiling: true
});

session.startProfiling();
const results = await session.run(feeds);
session.endProfiling();

[mram0509]

Thanks! Does not output anything to the console. I had read somewhere that profiling hadn't been implemented - maybe that's still the case.

[haydn-jones]

Oops, it's quite possible that its working for me because I'm using the master branch of the runtime, not the release on NPM. I'll check against the NPM version later today.

[haydn-jones]

Yep, looks like its not working for me on 1.8.0. 1.9.0 looks to be right around the corner or you can use master.

[mram0509]

Thanks for confirming.

This article ( https://cloudblogs.microsoft.com/opensource/2021/09/02/onnx-runtime-web-running-your-machine-learning-model-in-browser/) says SIMD is enabled. But in the distro for v1.8, there is no binary for SIMD and SIMD does not seem to be supported. Looks like that also will be supported in v1.9. Is that right? Or am I missing something with v1.8?

[haydn-jones]

I believe that SIMD is not working in v1.8. I can confirm that in master it DOES work, so it should be working in 1.9. Likewise, proxy was not working for me in v1.8 (so a large model was hanging the web page and making the browser angry), but it is working in master.

[haydn-jones]

@CACUser is the snipped I provided working for you now that 1.9.0 is out?

[mram0509]

Yes! it works! Thanks.

Now I am trying to verify SIMD works...

[mram0509]

With v1.9 - ort-wasm-simd.wasm is being loaded with Chrome (Windows) on an x86 system I am using BERT models for Q&A.
Using Chrome dev tools, when I look under Application->Frames->WebAssembly->ort-wasm-simd.wasm, I see the following in the right pane.
Unknown operator: 0xfd57

What is this error?