Hyperbolic Learning Rate Scheduler
MIT License
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
🦁 Lion, new optimizer discovered by Google Brain using genetic algorithms that is purportedly bet...
Implement DNN or ML models and advanced policies with PyTorch.(Include experiment)
Distributed Asynchronous Hyperparameter Optimization in Python
Code repository of the paper Learning Long-Term Dependencies in Irregularly-Sampled Time Series
Time series forecasting with PyTorch
🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed conf...
Train TensorFlow Keras models with cosine annealing and save an ensemble of models with no additi...
DeepHyper: Scalable Asynchronous Neural Architecture and Hyperparameter Search for Deep Neural Ne...
Run many functions (adaptively) on many cores (>10k-100k) using mpi4py.futures, ipyparallel, loky...
Software Architecture for ML engineers
ESGD-M is a stochastic non-convex second order optimizer, suitable for training deep learning mod...
Schedule-Free Optimization in PyTorch
Reinforcement Learning in PyTorch
2-2000x faster ML algos, 50% less memory usage, works on all hardware - new and old.