Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TFJS basic operations are extremely slow in comparison to Native JS #70252

Open
botcs opened this issue Jun 22, 2024 · 2 comments
Open

TFJS basic operations are extremely slow in comparison to Native JS #70252

botcs opened this issue Jun 22, 2024 · 2 comments
Assignees
@sushreebarsa
Labels
comp:apis Highlevel API related issues stat:awaiting response Status - Awaiting response from author type:performance Performance Issue

Comments

@botcs
Copy link

botcs commented Jun 22, 2024

Issue type

Performance

Have you reproduced the bug with TensorFlow Nightly?

Yes

Source

source

TensorFlow version

tfjs

Custom code

No

OS platform and distribution

No response

Mobile device

No response

Python version

No response

Bazel version

No response

GCC/compiler version

No response

CUDA/cuDNN version

No response

GPU model and memory

No response

Current behavior?

I was just shocked when I saw that even basic operations like passing an array to tf.tensor is 60% slower than allocating a tf.buffer, setting each value one by one and calling toTensor(). When it comes to tf.randomNormal, which I believe is called many many times, there is a major slow down, even though the same Box-Muller algorithm is used.

Standalone code to reproduce the issue

<!DOCTYPE html>
<html>
<head>
    <title>Random Number Benchmark</title>
    <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs"></script>
</head>
<body>
    <script>
        async function benchmarkTfjsRandomNormal(numSamples) {
            const start = performance.now();

            // Generate random floats using TensorFlow.js
            const randomFloats = tf.randomNormal([numSamples]);

            // Ensure the tensors are evaluated
            await randomFloats.data();

            const end = performance.now();
            tf.dispose([randomFloats]); // Clean up tensors
            return end - start;
        }

        function generateBoxMullerNormal() {
            let u = 0, v = 0;
            while (u === 0) u = Math.random(); // Converting [0,1) to (0,1)
            while (v === 0) v = Math.random();
            return Math.sqrt(-2.0 * Math.log(u)) * Math.cos(2.0 * Math.PI * v);
        }

        function benchmarkNativeRandomNormal(numSamples) {
            const start = performance.now();

            const randomFloats = new Float32Array(numSamples);
            for (let i = 0; i < numSamples; i++) {
                randomFloats[i] = generateBoxMullerNormal();
            }

            const end = performance.now();
            return end - start;
        }

        async function runBenchmarks() {
            const sampleSizes = [1000, 10000, 100000, 1000000];
            for (const numSamples of sampleSizes) {
                const tfjsTime = await benchmarkTfjsRandomNormal(numSamples);
                console.log(`TensorFlow.js (${numSamples} samples): ${tfjsTime.toFixed(2)} ms`);

                const nativeTime = benchmarkNativeRandomNormal(numSamples);
                console.log(`Native JavaScript (${numSamples} samples): ${nativeTime.toFixed(2)} ms`);

                const speedup = nativeTime / tfjsTime;
                console.log(`Speedup: ${speedup}`);
                console.log('');
            }
        }

        // Run the benchmarks and log the results
        runBenchmarks();
    </script>
</body>
</html>

Relevant log output

TensorFlow.js (1000 samples): 11.00 ms
test.html:49 Native JavaScript (1000 samples): 0.40 ms
test.html:52 Speedup: 0.03636363690549677
test.html:53 
test.html:46 TensorFlow.js (10000 samples): 2.40 ms
test.html:49 Native JavaScript (10000 samples): 0.90 ms
test.html:52 Speedup: 0.3750000015522043
test.html:53 
test.html:46 TensorFlow.js (100000 samples): 5.90 ms
test.html:49 Native JavaScript (100000 samples): 5.20 ms
test.html:52 Speedup: 0.8813559292925052
test.html:53 
test.html:46 TensorFlow.js (1000000 samples): 64.80 ms
test.html:49 Native JavaScript (1000000 samples): 58.20 ms
test.html:52 Speedup: 0.8981481482120248
test.html:53 
test.html:46 TensorFlow.js (10000000 samples): 648.90 ms
test.html:49 Native JavaScript (10000000 samples): 591.00 ms
test.html:52 Speedup: 0.9107720758122536
@google-ml-butler google-ml-butler bot added the type:performance Performance Issue label Jun 22, 2024
@botcs
Copy link
Author

botcs commented Jun 23, 2024

I am not sure if warmup needed for these experiments

@sushreebarsa
Copy link
Contributor

@botcs TensorFlow.js uses the WebGL backend for faster computations. Please ensure WebGL is enabled and utilized.
For any further queries please raise a ticket in TFJS repository.
Thank you!

@sushreebarsa sushreebarsa added comp:apis Highlevel API related issues stat:awaiting response Status - Awaiting response from author labels Jun 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sushreebarsa sushreebarsa

Labels
comp:apis Highlevel API related issues stat:awaiting response Status - Awaiting response from author type:performance Performance Issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@botcs @sushreebarsa