You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I want to use trino to get data from multiple backend, and the using the data to do some data science in jupyter notebook.
however, I found the speed to acquire data is quite slow. e.g. it could reach less than 15MB/s while I have a 10Gb/s network.
I use the memory connector to find out the bottleneck and the trino worker could talk with my backend database at a speed about more than 100MB/s.
because I found the bottleneck is at trino itself, rather than the database, so it cannot be optimized with the help of alluxio and caches like it.
so is there any method to make it faster, and what's the underlying bottleneck in trino that limit the speed. great thanks.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
I want to use trino to get data from multiple backend, and the using the data to do some data science in jupyter notebook.
however, I found the speed to acquire data is quite slow. e.g. it could reach less than 15MB/s while I have a 10Gb/s network.
I use the
memory connector
to find out the bottleneck and the trino worker could talk with my backend database at a speed about more than 100MB/s.because I found the bottleneck is at trino itself, rather than the database, so it cannot be optimized with the help of alluxio and caches like it.
so is there any method to make it faster, and what's the underlying bottleneck in trino that limit the speed. great thanks.
Beta Was this translation helpful? Give feedback.
All reactions