Fast and easy parallel mapreduce on HPC clusters
MIT License
Parallel mapreduce and other helpful functions for HPC, meant primarily for embarassingly parallel operations that often require one to split up a list of tasks into subsections that may be processed on individual cores.
Install the package using
pkg> add ParallelUtilities
julia> using ParallelUtilities
Just replace mapreduce
by pmapreduce
in your code and things should work the same.
julia> @everywhere f(x) = (sleep(1); x^2); # some expensive calculation
julia> nworkers()
2
julia> @time mapreduce(f, +, 1:10) # Serial
10.021436 seconds (40 allocations: 1.250 KiB)
385
julia> @time pmapreduce(f, +, 1:10) # Parallel
5.137051 seconds (863 allocations: 39.531 KiB)
385
See the documentation for examples and the API.