Benchmarking Haskell extensible-effect libraries.
BSD-3-CLAUSE License
This is a set of memory and time benchmarks for the various Haskell effects libraries, including fused-effects
, polysemy
, freer-simple
, and extensible-effects
. Benchmarks for equivalent mtl
and handwritten computations are included for comparison purposes. It uses Cabal's new Backpack feature to save a great deal of boilerplate code.
fused-effects
and mtl
achieve roughly commensurate performance. polysemy
is within one to three orders of magnitude from fused-effects
and mtl
. freer-simple
and extensible-effects
are considerably slower.
On a 15-inch 2017 MacBook Pro (3.1GHz Intel i7, 16GB RAM), here are results for space benchmarks:
Countdown
Case Allocated
fused-effects 40
mtl 40
polysemy 1,304
freer-simple 1,096
extensible-effects 1,304
shallow 232
Countdown + exc
Case Allocated GCs
fused-effects 336
mtl 336
polysemy 792
freer-simple 864
extensible-effects 984
shallow 320
(The fact that fused-effects
and mtl
have the same allocations is not an error; fused-effects
was designed to have almost identical performance characteristics to mtl
.)
Time (State
-based countdown):
benchmarked Countdown/Put/fused-effects
time 5.692 s (5.570 s .. 5.835 s)
benchmarked Countdown/Put/mtl
time 5.500 s (5.386 s .. 5.719 s)
benchmarked Countdown/Put/polysemy
time 1.903 ms (1.845 ms .. 1.954 ms)
benchmarked Countdown/Put/freer-simple
time 1.324 ms (1.282 ms .. 1.361 ms)
benchmarked Countdown/Put/extensible-effects
time 1.383 ms (1.343 ms .. 1.422 ms)
benchmarked Countdown/Put/shallow
time 275.1 s (270.1 s .. 281.6 s)
Time (State
+ Error
countdown):
benchmarked Countdown/Put+Exc/fused-effects
time 9.829 s (9.645 s .. 10.04 s)
benchmarked Countdown/Put+Exc/mtl
time 9.968 s (9.779 s .. 10.15 s)
benchmarked Countdown/Put+Exc/polysemy
time 418.1 s (408.9 s .. 427.1 s)
benchmarked Countdown/Put+Exc/freer-simple
time 1.157 ms (1.120 ms .. 1.199 ms)
benchmarked Countdown/Put+Exc/extensible-effects
time 1.232 ms (1.212 ms .. 1.262 ms)
benchmarked Countdown/Put+Exc/shallow
time 55.12 s (54.34 s .. 56.29 s)
Time (custom HTTP DSL):
benchmarked HTTP/fused-effects
time 8.311 s (8.060 s .. 8.592 s)
benchmarked HTTP/polysemy
time 14.99 ms (14.26 ms .. 15.80 ms)
benchmarked HTTP/extensible-effects
time 1.612 ms (1.578 ms .. 1.654 ms)
benchmarked HTTP/Deep embedding
time 2.027 ms (1.876 ms .. 2.184 ms)
benchmarked HTTP/Shallow embedding
time 15.94 s (15.49 s .. 16.46 s)
benchmarked HTTP/freer-simple
time 1.478 ms (1.443 ms .. 1.515 ms)
Writer
, logging, grammar
, pipes
.