LiteRT vs TFLite C vs TFLite C++

[bojeckkk]

Hey,

What's the current status performance/size/memory wise of LiteRT when compared to TFLite with C API and TFLite with C++ API?

In the past I've chosen TFLite with C API due to lower overall library size. Right now I'm looking into updating TFLite version (from 2.10) and considering switching to LiteRT. Also the official docs are the same as old ones, with just TFLite word replaced by LiteRT: https://ai.google.dev/edge/litert/build/cmake . These docs still point to the whole Tensorflow github repository, which is confusing.

Also I do crosscompile with CMake and as I've experienced already, LiteRT has trouble building with CMake: #3379

Best,

[gaikwadrahul8]

Hi, @bojeckkk
I apologize for the delay in my response, AFAIK performance wise LiteRT matches or slightly exceeds TFLite’s C++ and C APIs on CPU workloads. For example, MobileNetV1-float runs in ~14 ms and quantized MobileNetV1 in ~5 ms on Pixel 4 using 4 threads. New MLDrift GPU delegates in LiteRT deliver up to 15% faster GPU inference and early‐access NPU delegates can boost throughput up to 3× versus CPU. [Ref-1], [Ref-2]

Binary size wise TFLite C API static lib: ~300 KB, TFLite C++ API static lib: ~500 KB and LiteRT base static lib: ~600–700 KB (includes delegate management and zero-copy buffers) You can reduce LiteRT’s footprint by disabling unused ops/delegates (-DTFLITE_ENABLE_XNNPACK=OFF, selective‐ops builds).

Please refer these official documentations for more information LiteRT CMake build guide, Binary size tuning, Performance measurement, LiteRT: Maximum performance, simplified

The LiteRT build failure with CMake appears to be caused by version incompatibilities with Protobuf or other dependent libraries but our relavant team is working on it to fix as soon as possible. Thank you for your cooperation and patience.

[gaikwadrahul8]

Hi, @bojeckkk
As mentioned by @terryheo in the comment, the steps in the original documentation are for LiteRT v1. For building LiteRT v2, please refer to the updated instructions here: LiteRT v2 Build Instructions

Could you please give it a try with these updated steps and let us know if you’re still facing any issues? Thank you for your cooperation and patience.

[github-actions]

Marking this issue as stale since it has been open for 7 days with no activity. This issue will be closed if no further activity occurs.

[github-actions]

This issue was closed because it has been inactive for 14 days. Please post a new issue if you need further assistance. Thanks!