Skip to content

ocramz/trie-perf

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

23 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

trie-perf

Build Status

Performance shootout of various prefix tree ("trie") implementations.

"space" benchmarking performed with weigh and "time" benchmarks done with criterion.

Currently comparing two unoptimized implementations taken (but slightly modified) from didactic blogposts (https://alexandersgreen.wordpress.com/2010/09/13/prefix-trees-in-haskell/ and https://blog.jle.im/entry/tries-with-recursion-schemes.html) and four available on Hackage (generic-trie, bytestring-trie, text-trie, and trie-simple).

generic-trie seems to be the best choice, at least for a "lookup - fromList" pair, but I was curious to see how very diverse implementation techniques, notably one based on recursion schemes and another using an arrow type internally, lead to different space and time scaling behaviours.

Contributing

Pull requests that extend and/or improve these benchmarks are very welcome.

Results obtained on a 2017 MacBook Pro using stack bench:

Benchmark space: RUNNING...

AG

  Case    Allocated  GCs
  small       1,208    0
  medium     12,968    0
  large     133,456    0

JL

  Case    Allocated  GCs
  small       5,976    0
  medium     59,928    0
  large     645,264    0

generic-trie

  Case    Allocated  GCs
  small       4,320    0
  medium     21,992    0
  large     221,760    0

bytestring-trie

  Case    Allocated  GCs
  small       2,224    0
  medium     52,416    0
  large   2,600,752    2

text-trie

  Case    Allocated  GCs
  small       2,536    0
  medium     62,264    0
  large   3,071,280    2

trie-simple

  Case     Allocated  GCs
  small        3,088    0
  medium     201,448    0
  large   18,897,064   18
Benchmark space: FINISH
Benchmark time: RUNNING...
benchmarking AG/small
time                 135.4 ns   (132.6 ns .. 138.7 ns)
                     0.994 R²   (0.989 R² .. 0.998 R²)
mean                 133.4 ns   (130.8 ns .. 138.2 ns)
std dev              11.43 ns   (6.546 ns .. 17.15 ns)
variance introduced by outliers: 87% (severely inflated)

benchmarking AG/medium
time                 4.207 μs   (4.111 μs .. 4.371 μs)
                     0.989 R²   (0.981 R² .. 0.997 R²)
mean                 4.364 μs   (4.246 μs .. 4.522 μs)
std dev              452.0 ns   (337.3 ns .. 582.7 ns)
variance introduced by outliers: 88% (severely inflated)

benchmarking AG/large
time                 70.83 μs   (68.15 μs .. 73.93 μs)
                     0.991 R²   (0.986 R² .. 0.997 R²)
mean                 69.11 μs   (67.75 μs .. 71.10 μs)
std dev              5.418 μs   (3.809 μs .. 7.481 μs)
variance introduced by outliers: 75% (severely inflated)

benchmarking JL/small
time                 887.7 ns   (866.9 ns .. 918.7 ns)
                     0.993 R²   (0.983 R² .. 0.999 R²)
mean                 886.1 ns   (874.1 ns .. 924.3 ns)
std dev              64.19 ns   (32.05 ns .. 129.6 ns)
variance introduced by outliers: 81% (severely inflated)

benchmarking JL/medium
time                 12.11 μs   (11.89 μs .. 12.40 μs)
                     0.996 R²   (0.993 R² .. 0.999 R²)
mean                 12.25 μs   (12.09 μs .. 12.48 μs)
std dev              638.9 ns   (425.2 ns .. 969.5 ns)
variance introduced by outliers: 62% (severely inflated)

benchmarking JL/large
time                 68.27 μs   (65.93 μs .. 71.35 μs)
                     0.991 R²   (0.982 R² .. 1.000 R²)
mean                 67.17 μs   (66.33 μs .. 69.05 μs)
std dev              4.090 μs   (1.513 μs .. 7.208 μs)
variance introduced by outliers: 63% (severely inflated)

benchmarking generic-trie/small
time                 486.8 ns   (477.0 ns .. 499.4 ns)
                     0.996 R²   (0.993 R² .. 0.998 R²)
mean                 478.3 ns   (470.7 ns .. 489.4 ns)
std dev              30.40 ns   (20.92 ns .. 48.38 ns)
variance introduced by outliers: 77% (severely inflated)

benchmarking generic-trie/medium
time                 4.786 μs   (4.620 μs .. 4.987 μs)
                     0.994 R²   (0.988 R² .. 1.000 R²)
mean                 4.677 μs   (4.627 μs .. 4.773 μs)
std dev              222.9 ns   (99.69 ns .. 392.9 ns)
variance introduced by outliers: 60% (severely inflated)

benchmarking generic-trie/large
time                 52.39 μs   (50.40 μs .. 53.86 μs)
                     0.994 R²   (0.992 R² .. 0.998 R²)
mean                 50.30 μs   (49.64 μs .. 51.10 μs)
std dev              2.537 μs   (1.805 μs .. 3.436 μs)
variance introduced by outliers: 55% (severely inflated)

benchmarking bytestring-trie/small
time                 258.9 ns   (255.1 ns .. 263.4 ns)
                     0.999 R²   (0.997 R² .. 1.000 R²)
mean                 257.7 ns   (255.8 ns .. 261.0 ns)
std dev              7.759 ns   (5.639 ns .. 11.23 ns)
variance introduced by outliers: 44% (moderately inflated)

benchmarking bytestring-trie/medium
time                 3.325 μs   (3.256 μs .. 3.426 μs)
                     0.994 R²   (0.990 R² .. 0.998 R²)
mean                 3.354 μs   (3.301 μs .. 3.433 μs)
std dev              209.7 ns   (134.9 ns .. 298.7 ns)
variance introduced by outliers: 73% (severely inflated)

benchmarking bytestring-trie/large
time                 73.46 μs   (72.62 μs .. 74.44 μs)
                     0.994 R²   (0.987 R² .. 0.999 R²)
mean                 74.85 μs   (72.91 μs .. 79.93 μs)
std dev              9.677 μs   (3.383 μs .. 16.95 μs)
variance introduced by outliers: 89% (severely inflated)

benchmarking text-trie/small
time                 282.0 ns   (278.1 ns .. 286.3 ns)
                     0.996 R²   (0.992 R² .. 0.999 R²)
mean                 289.9 ns   (282.4 ns .. 304.0 ns)
std dev              32.29 ns   (16.36 ns .. 51.40 ns)
variance introduced by outliers: 92% (severely inflated)

benchmarking text-trie/medium
time                 3.279 μs   (3.237 μs .. 3.340 μs)
                     0.997 R²   (0.992 R² .. 0.999 R²)
mean                 3.396 μs   (3.311 μs .. 3.534 μs)
std dev              352.3 ns   (223.2 ns .. 507.7 ns)
variance introduced by outliers: 89% (severely inflated)

benchmarking text-trie/large
time                 74.49 μs   (73.21 μs .. 76.11 μs)
                     0.996 R²   (0.993 R² .. 0.998 R²)
mean                 78.65 μs   (76.51 μs .. 82.87 μs)
std dev              10.06 μs   (6.959 μs .. 14.33 μs)
variance introduced by outliers: 89% (severely inflated)

benchmarking trie-simple/small
time                 274.3 ns   (270.3 ns .. 279.6 ns)
                     0.997 R²   (0.995 R² .. 0.999 R²)
mean                 272.6 ns   (269.8 ns .. 277.2 ns)
std dev              12.07 ns   (7.266 ns .. 18.85 ns)
variance introduced by outliers: 63% (severely inflated)

benchmarking trie-simple/medium
time                 23.87 μs   (23.52 μs .. 24.49 μs)
                     0.995 R²   (0.991 R² .. 0.999 R²)
mean                 24.86 μs   (24.37 μs .. 25.73 μs)
std dev              2.058 μs   (1.379 μs .. 3.075 μs)
variance introduced by outliers: 79% (severely inflated)

benchmarking trie-simple/large
time                 16.70 ms   (15.57 ms .. 17.95 ms)
                     0.977 R²   (0.956 R² .. 0.995 R²)
mean                 16.51 ms   (16.05 ms .. 17.14 ms)
std dev              1.378 ms   (1.037 ms .. 1.926 ms)
variance introduced by outliers: 40% (moderately inflated)

Benchmark time: FINISH
Completed 2 action(s).

About

Performance shootout of various trie implementations

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published