fairscale

What's Changed

• [docs][fix] Update example to use offload_model by @anj-s in https://github.com/facebookresearch/fairscale/pull/806
• Switch default branch from master to main by @tmarkstrum in https://github.com/facebookresearch/fairscale/pull/807
• [FairScale] Remove refs to "cpu_offload" in code comments by @rohan-varma in https://github.com/facebookresearch/fairscale/pull/814
• [chore] Remove deprecated THCudaCheck by @anj-s in https://github.com/facebookresearch/fairscale/pull/818
• [feat] layer memory tracking by @QuentinDuval in https://github.com/facebookresearch/fairscale/pull/808
• [chore] Add log for the new experimental memory tracker feature. by @anj-s in https://github.com/facebookresearch/fairscale/pull/819
• [chore] Update the PyTorch version that we run CPU tests with by @anj-s in https://github.com/facebookresearch/fairscale/pull/809
• [chore] Update the PyTorch version that we run benchmarks with. by @anj-s in https://github.com/facebookresearch/fairscale/pull/823
• Extend auto shard capabilities to work around torch.fx edge cases. by @EugenHotaj in https://github.com/facebookresearch/fairscale/pull/817
• [fix] Update golden data for account for the speed regression by @anj-s in https://github.com/facebookresearch/fairscale/pull/825
• [chore] Fix main breakage temporarily by relaxing constraints by @anj-s in https://github.com/facebookresearch/fairscale/pull/828
• Use correct node names for param counting in auto_shard. by @EugenHotaj in https://github.com/facebookresearch/fairscale/pull/830
• [chore] Update requirements file to reflect latest config by @anj-s in https://github.com/facebookresearch/fairscale/pull/832
• [fix]: Fixes an issue with pre_backward hook registering by @min-xu-ai in https://github.com/facebookresearch/fairscale/pull/833
• [feature] Skip creating the CPU grad tensor when training by @anj-s in https://github.com/facebookresearch/fairscale/pull/821
• [test] improve a test's coverage by @min-xu-ai in https://github.com/facebookresearch/fairscale/pull/798
• [fix] Decouple move_params_to_cpu from the mixed_precision. by @anj-s in https://github.com/facebookresearch/fairscale/pull/822
• [fix] fix test on main by @min-xu-ai in https://github.com/facebookresearch/fairscale/pull/835
• [feature] Add the low level SSD APIs by @anj-s in https://github.com/facebookresearch/fairscale/pull/829
• [feat] [FSDP]: add experimental support to shared weights by @min-xu-ai in https://github.com/facebookresearch/fairscale/pull/836
• update nightly torch and test the flaky test by @min-xu-ai in https://github.com/facebookresearch/fairscale/pull/837
• [chore] Fix broken main due to updated github URL requirements by @anj-s in https://github.com/facebookresearch/fairscale/pull/838
• [chore] Update Sphinx version in docs requirements file by @vtantia in https://github.com/facebookresearch/fairscale/pull/841
• [feat] experimental MEVO layer by @min-xu-ai in https://github.com/facebookresearch/fairscale/pull/840
• [feat] Gossip/SlowMo by @blefaudeux in https://github.com/facebookresearch/fairscale/pull/378
• [feature]Add support for SSD offload with FSDP for eval workloads by @anj-s in https://github.com/facebookresearch/fairscale/pull/839
• [chore] 0.4.2 release by @anupambhatnagar in https://github.com/facebookresearch/fairscale/pull/846
• CI config changes by @anupambhatnagar in https://github.com/facebookresearch/fairscale/pull/847
• Setup pre-commit github action and apply pre-commit to all files by @anupambhatnagar in https://github.com/facebookresearch/fairscale/pull/849
• Allow sharded grad scaler to cpu offload with FSDP by @anupambhatnagar in https://github.com/facebookresearch/fairscale/pull/831
• Update changelog, removed meta.yml and requirements cleanup by @anupambhatnagar in https://github.com/facebookresearch/fairscale/pull/853
• [feature] Add a OffloadConfig object to specify offloading params to disk. by @anj-s in https://github.com/facebookresearch/fairscale/pull/855
• [POC] Testing Manual dispatch by @anupambhatnagar in https://github.com/facebookresearch/fairscale/pull/859
• [fix] [MEVO]: make mevo work with eval and optim_state checkpointing by @min-xu-ai in https://github.com/facebookresearch/fairscale/pull/851
• [chore] 0.4.3 release by @min-xu-ai in https://github.com/facebookresearch/fairscale/pull/860

New Contributors

• @rohan-varma made their first contribution in https://github.com/facebookresearch/fairscale/pull/814
• @EugenHotaj made their first contribution in https://github.com/facebookresearch/fairscale/pull/817
• @vtantia made their first contribution in https://github.com/facebookresearch/fairscale/pull/841

Full Changelog: https://github.com/facebookresearch/fairscale/compare/v0.4.1...v0.4.3

Released version 0.4.1 for FairScale.

[0.3.4] - 2021-04-13

Added

• FSDP: Add no broadcast optim state option (#560)

Fixed

• ShardedDDP: Properly handle .eval() mode (#587)
• ShardedDDP: Handle model being moved back to CPU prior to state consolidation (#573)
• FSDP: much faster state consolidation (#595)
• FSDP: Add gradient pre-divide to prevent overflow with large world sizes (#565)
• Offload: (experimental) Fix activation offloading to CPU (#588