dvc

🐛 Bug Fixes

• fix making too many ssh connections when pushing run cache (#4732) (#4733) @peper0
• cli: config: check name format (#5090) @efiop

🐛 Bug Fixes

• system: _reflink_darwin: use fsencode for path (#5085) @efiop

Changes

🔨 Maintenance

• build: pin numpy for windows build (#5012) @skshetry

Changes

🔨 Maintenance

• snap: publish to v1/stable channel

Changes

🔨 Maintenance

• snap: publish to v1 channel

Refer to https://dvc.org/doc/install for installation instructions.

Changes

🚀 New Features and Enhancements

• s3: provide more helpful messages on common errors (#4480) @efiop
• diff: Add sort logic to sort diff ouput irrespective of output format… (#4459) @PuneethaPai
• status: no warnings for missing state (#4470) @nik123
• Outputs as target supporting for dvc status (#4433) @karajan1001

🏇 Optimizations

• Speed up dvc files load (#4417) @Suor

🐛 Bug Fixes

• experiments: only show queued experiments for the current commit (#4462) @pmrowla
• experiments: fix inconsistent dvc exp checkout behavior (#4461) @pmrowla

🔨 Maintenance

• changelog: categorize changelog entry using labels (#4474) @skshetry
• output: dont rely on the default hash type (#4466) @efiop

Thanks again to @PuneethaPai, @Suor, @efiop, @karajan1001, @nik123, @pmrowla and @skshetry for the contributions! 🎉

• gdrive: fix multi-remote workflow, cont. cleanup (#3686) @shcheklein
• setup: relax python-dateutil pip version constraint to include v2.8.2. #3701 (#3702) @dchichkov
• tag: getting rid of it (#3699) @skshetry
• remote.ssh: suppress paramiko logging (#3675) @pmrowla
• dvc: implement params support for pipeline file (#3694) @skshetry
• refactor: dvc/output class names unification (#3695) @nik123
• repo: use reverse post-order DFS in repro --downstream (#3689) @pmrowla
• remote: use string paths over PathInfo for performance reasons (#3672) @pmrowla
• dvc: implement multistage dvcfile (#3676) @skshetry
• refactor: dvc/dependency class names unification (#3688) @nik123
• refactor: dvc/remotes class names unification (#3684) @nik123
• progress: add main bars (#3594) @casperdcl
• git: is_tracked: use ls_files instead of walking the index (#3677) @efiop
• flake8: fix ambiguous var names (#3674) @pmrowla
• Convert all paths in config to posix-style (#3665) @charlesbaynham
• output: use strings instead of PathInfo for performance reasons (#3663) @pmrowla
• metrics: rework metrics implementation (#3658) @efiop
• Fix repro duplicating effort while checking steps #3644 (#3645) @charlesbaynham
• metrics diff error message in --no-scm projects (#3650) @karajan1001
• gc: don't forget to load remote index (#3660) @efiop
• remote: locally index list of checksums available on cloud remotes (#3634) @pmrowla
• refactor: extract stage loading outside of dvcfile (#3655) @skshetry
• tests: test_directories (#3649) @nik123
• Specify cmd arg names (metavars), et al. (#3626) @jorgeorpinel
• scm: git: clone: add progress (#3595) @casperdcl
• dvc: implement multi-stage dvcfile (#3584) @skshetry
• tests: add some DvcTree unit tests (#3640) @efiop
• dvc: introduce DvcTree (#3639) @efiop
• remote: don't use PathInfo.from_posix (#3637) @efiop
• remote config validation (#3628) @karajan1001
• params: diff: introduce --all (#3631) @efiop
• remote: use .dir checksum existence to infer file contents existence (#3632) @pmrowla
• refactor: extract stage's relationship with Dvcfile out from Stage (#3619) @skshetry
• Updates latest version of PyDrive2 (#3624) @mribeirodantas
• stage: resolve stage path before creating the stage (#3630) @efiop
• run: remove deprecated --cwd (#3623) @efiop

remote: use progress bar for remote cache query status during dvc gc (#3559) @pmrowla

• test: scm: convert to dir helpers (#3116) @efiop
• get: refactor handling of git files and git-only repos (#3114) @efiop
• test: get_url: convert to dir helpers (#3115) @efiop
• test: init: convert to dir helpers (#3117) @efiop
• tests: external_repo: convert to dir helpers (#3118) @efiop
• add space for import help text (#3126) @pared
• test: refactor ssh related functions in test helper classes (#3123) @skshetry
• summon: fixes and dvcx prereq (#3101) @Suor
• gdrive: add support for teamDriveId (#3016) @efiop
• rwlock: flush and fsync (#3076) @efiop
• get: handle non-DVC repositories (#3097) @fabiosantoscode
• run: tests: migrate to dir helpers (#3094) @pared
• state: tests: migrate to dir helpers (#3096) @pared
• update: tests: migrate to dir helpers (#3100) @pared
• stage: tests: migrate to dir helpers (#3099) @pared
• tests: remove redundant native ssh tests (#3104) @efiop
• py3: use links from os module (#3090) @mroutis
• repro: tests: migrate to dir helpers (#3091) @pared
• dvc: optimize Git.is_tracked() (#3053) @Suor
• Move fs utilities to fs.py from init.py (#3093) @algomaster99
• import url: test: migrate to dir helpers (#3072) @pared
• output: tests: migrate to dir helpers (#3083) @pared
• py3: remove code related to py2 limitations (#3034) @mroutis
• remote: test: migrate to dir helpers (#3086) @pared
• pipeline: test: migrate to dir helpers (#3085) @pared
• install: tests: migrate to dir helpers (#3074) @pared
• setup: get rid of py2 reqs (#3082) @efiop
• py3: use format_exc instead of print_exception (#3078) @mroutis
• rwlock: check that .dvc/lock is locked (#3075) @efiop
• ui: warn about ignored options for dvc status and a grammar fix (#3027) @Suor
• ui: don't show exception for "Too many files" error (#3058) @efiop
• Write test for walk_files to check if it accepts Path-like and str objects (#3035) @algomaster99
• fix snap deploy stage (#3064) @casperdcl
• dvc: update gc to remove unpacked dir (#3054) @sharidas
• test: merge _should_test_aws and _get_aws_url inside S3 test helper (#3073) @skshetry
• test: merge _should_test_oss and _get_oss_url inside OSS test helper (#3071) @skshetry
• http/ imp tests - migrate do dir helpers (#3070) @pared
• dependency: test: migrate to dir helpers (#3068) @pared
• cache: test: migrate to dir helpers (#3067) @pared
• analytics: tests: migrate to dir helpers (#3066) @pared
• tests: api: convert to dir helpers (#3044) @pared
• Snap classic deployment (#3063) @casperdcl
• completion: fix --relink problem (#3061) @mroutis
• local-related methods should verify that WorkingTree is in use (#3018) @pared
• tests: remote: local: migrate to dir fixtures (#3049) @pared
• import: allow importing from non-DVC git repositories (#3020) @chatcannon
• test: Merge _should_test_azure and _get_azure_url inside Azure test helper (#3055) @skshetry
• test: add _should_test_gdrive and _get_gdrive_url inside GDrive (#3056) @skshetry
• py3: use yield from where appropriate (#3052) @Suor
• test: do not bootstrap repo template and version files in erepo_dir (#3045) @Suor
• dvc: optimize and clean up stages collection (#3048) @Suor
• dvc: autowrap repo tree into CleanTree (#3047) @Suor
• test: add @TmpDir.chdir() (#3043) @Suor
• dvc: clean brancher (#3046) @Suor
• py3: refactor exceptions (#3042) @mroutis
• py3: use new magic from PEP 3135 (#3039) @mroutis
• Restyle fix for base None value (#3041) @restyled-io
• Ensure relpath accepts str and Path-like objects (#3037) @algomaster99
• metrics: don't show working tree if it is clean (#3025) @Suor
• ui: add checkout --relink to autocomplete scripts (#3026) @Suor
• py3: migrate utils/compat.py to compat.py (#3032) @mroutis
• py3: remove imports from utils.compat (#3031) @mroutis
• py3: remove csv_reader and makedirs from utils/compat.py (#3030) @mroutis
• py3: remove methods for casting bytes in python 2 (#3029) @mroutis

Fix inflect version.

Fixed pypi package deployment.

Fix windows build.

• dvc get leaves dangling symlinks if source repo is set to use symlinks as its cache type (#2745) @shcheklein

1. Refactored URLInfo-related logic;
2. Added support for moving between different drives for SSH remotes;
3. Removed "Multi-threaded" prefix from progress bars; Kudos @kurianbenoy 🎉
4. Added unit tests for get_mtime_and_size; Kudos @algomaster99 🎉
5. Removed cask installation detection; Kudos @ATR-oCiTy 🎉
6. Fixed typos: https://github.com/iterative/dvc/pull/2712, https://github.com/iterative/dvc/pull/2713, https://github.com/iterative/dvc/pull/2714, https://github.com/iterative/dvc/pull/2717, https://github.com/iterative/dvc/pull/2716, https://github.com/iterative/dvc/pull/2715; Kudos @Pandemic179 @eshaan-jeckyll @ATR-oCiTy @the-thirteenth-star @GeorgeSabu-mec @johntharian 🎉 🎉 🎉
7. Added pkg manager to dvc version output; Kudos @kss682 🎉
8. Stopped printing multiple "Need help?" footers when encountering multiple exceptions; Kudos @n3hrox 🎉

1. Introduced additional tests for s3 ETags;
2. Removed obsoleted warning; Kudos @sadielbartholomew 🎉
3. Fixed bug in dvc get-url;

1. Added additional unit tests for fs utils;
2. Temporarily hidden --all-commits flag for gc;
3. Refactored backtics and single-quotes usage in the code;
4. Added a workaround for flufl.lock bug;
5. Fixed bug when pulling imported directories;

1. Reverted path.lstrip("/") patch for dvc get/import