Tool for writing large xarray datasets to zarr stores with independent processes
MIT License
Dask tutorials for Big Data Analysis and Machine Learning as Jupyter notebooks
Run many functions (adaptively) on many cores (>10k-100k) using mpi4py.futures, ipyparallel, loky...
Efficiently generate and analyse high dimensional data.
Assign tasks to pools of workers in dask
Provides a sharded Zarr store
Fast dataset format and loader
Distributed segmentation for bio-image-analysis
Bounded-memory serverless distributed N-dimensional array processing
A low-impact profiler to figure out how much memory each task in Dask is using
Making sbatch more user-friendly (for python users of Jean-Zay).
Tool to easily start up an IPython cluster on different schedulers.
An implementation of chunked, compressed, N-dimensional arrays for Python.
Turn any collection of files into a dataset
Create dask.delayed objects that only run once per worker
Python 3.8+ toolbox for submitting jobs to Slurm