Simple, safe way to store and distribute tensors
APACHE-2.0 License
Published by Narsil almost 2 years ago
SAFETENSORS_FAST_GPU=1
on Windows : https://github.com/huggingface/safetensors/pull/140
Published by Narsil almost 2 years ago
First Rust release.
Published by Narsil almost 2 years ago
Published by Narsil almost 2 years ago
convert.py
to be more general.Published by Narsil almost 2 years ago
Fast CPU load on Pytorch
Fast GPU load on Python (unsafe, need to use SAFETENSORS_FAST_GPU=1
environement variable to use.
Published by Narsil almost 2 years ago
Remove importib
. This was bogus and it crashes on Python2.7.
Published by Narsil almost 2 years ago
device
argument to safe_open
to reduce overhead of sending tensors to GPU.Published by Narsil about 2 years ago
math.prod
is not supported for Python < 3.8. Putting back np.prod
in there.
Published by Narsil about 2 years ago
Added safe_open
.
Added __metadata__
special key in the format (ability to store layout information that could be useful to transformers.)
Removed some API surface.
Cleaned up TF to be actually TF and not jax anymore.
Very shallow changelog, this is still a lot in exploration mode.
Published by Narsil about 2 years ago
Initial release with barebone features just to get a PoC running.
Main features missing: