dvc

Refer to https://dvc.org/doc/install for installation instructions.

Changes

🐛 Bug Fixes

• fs: s3: don't forget to pass verify option (#6298) @kareemjano

Thanks again to @efiop and @kareemjano for the contributions! 🎉

Refer to https://dvc.org/doc/install for installation instructions.

Changes

🚀 New Features and Enhancements

• Fix #6205: Fixed diff summary to only show states with values. (#6229) @hadasarik
• add: reduce os.fspath calls on recursive add (#6287) @skshetry
• add: fix progress bar on single target (#6288) @skshetry
• ui: Prevent tqdm from breaking on output (#6280) @skshetry
• cleaning up add, progress bars and status (#6224) @skshetry

🏇 Optimizations

• staging: optimize the number of info() calls (#6281) @isidentical
• objects: hash_file: retrieve size from hash_info (#6284) @isidentical

🐛 Bug Fixes

• ui: Prevent tqdm from breaking on output (#6280) @skshetry

🔨 Maintenance

• Fix typos discovered by codespell (#6294) @cclauss
• add: reduce os.fspath calls on recursive add (#6287) @skshetry
• cleaning up add, progress bars and status (#6224) @skshetry

Thanks again to @cclauss, @efiop, @hadasarik, @isidentical and @skshetry for the contributions! 🎉

Refer to https://dvc.org/doc/install for installation instructions.

Changes

🚀 New Features and Enhancements

• add: do not delete stage files before add (#6239) @skshetry
• separate validation and transformation of args from the add func (#6220) @skshetry
• import: error message when imported file does not exist (#6240) @mmeendez8

🏇 Optimizations

• add: do not delete stage files before add (#6239) @skshetry

🐛 Bug Fixes

• ssh: don't use path_info for connections (#6259) @efiop
• output: optimize obj collection (#6277) @efiop
• pygit: handle annotated tags in resolve_refish (#6274) @pmrowla
• fsspec: loosen the prefix check to cover both None and False (#6246) @isidentical
• push params in the experiments table to the right (#6254) @skshetry

🔨 Maintenance

• separate validation and transformation of args from the add func (#6220) @skshetry
• build(deps): bump google-cloud-storage from 1.39.0 to 1.40.0 (#6261) @dependabot

Thanks again to @dependabot, @dependabot[bot], @efiop, @isidentical, @kurianbenoy, @mmeendez8, @pmrowla and @skshetry for the contributions! 🎉

Refer to https://dvc.org/doc/install for installation instructions.

Changes

• [pre-commit.ci] pre-commit autoupdate (#6243) @pre-commit-ci
• Loosen versions for fsspec and pygtrie (#6242) @Erotemic

🐛 Bug Fixes

• git: Keep trailing slash when checking if directory is gitignored (#6232) @tpietruszka

Thanks again to @Erotemic, @efiop, @pre-commit-ci, @pre-commit-ci[bot] and @tpietruszka for the contributions! 🎉

Refer to https://dvc.org/doc/install for installation instructions.

Changes

🏇 Optimizations

• obj: staging: remove redundant exists() call (#6217) @efiop

🐛 Bug Fixes

• output: don't lower the etag field on the schema when deserializing (#6238) @isidentical
• utils: normalize case in path_isin functions (#6236) @pmrowla

🔨 Maintenance

• build(deps): bump crc32c from 2.2 to 2.2.post0 (#6234) @dependabot
• build(deps): bump urllib3 from 1.26.5 to 1.26.6 (#6235) @dependabot
• setup: update fsspec to 2021.6.1 (#6219) @efiop
• build(deps): bump mypy from 0.902 to 0.910 (#6212) @dependabot
• build(deps): bump google-cloud-storage from 1.38.0 to 1.39.0 (#6211) @dependabot

Thanks again to @dependabot, @dependabot[bot], @efiop, @isidentical and @pmrowla for the contributions! 🎉

Refer to https://dvc.org/doc/install for installation instructions.

Changes

• Bump fsspec version (#6214) @skshetry
• Create CODEOWNERS (#6213) @skshetry

Thanks again to @efiop and @skshetry for the contributions! 🎉

Refer to https://dvc.org/doc/install for installation instructions.

Changes

• fix webdav tests and then add some (#6189) @skshetry

🚀 New Features and Enhancements

• pass size to upload_fobj (#6187) @skshetry
• serialize: Wrap UnicodeDecodeErrors (#6193) @duijf
• azure: add allow_anonymous_login option to the schema (#6190) @isidentical
• webdav: minor updates (#6155) @skshetry
• http/webdav: support specifying custom certs in ssl_verify (#6142) @skshetry
• webdav4: migrate to new library (#6099) @skshetry

🐛 Bug Fixes

• repo: preserve meta field on move (#6201) @yehiyam
• add, import-url: preserve size for to-remote/to-cache (#6162) @isidentical
• s3, gs, azure: re-enable prefix-based search optimization (#6151) @isidentical
• stage: monitor: don't forget to unprotect outs (#6134) @efiop
• ls: specify rev as kwarg (#6121) @efiop
• add: prevent crash while getting relpath (#5837) @tomrutter

🔨 Maintenance

• [pre-commit.ci] pre-commit autoupdate (#6167) @pre-commit-ci
• odb: treat external repo dependencies as objects (#6109) @pmrowla
• build(deps): bump pytest-xdist from 2.2.1 to 2.3.0 (#6188) @dependabot
• build(deps): bump pytest-docker from 0.10.1 to 0.10.3 (#6172) @dependabot
• build(deps): bump mypy from 0.901 to 0.902 (#6153) @dependabot
• setup: bump fsspec/s3fs/gcsfs to 2021.06.0 (#6137) @isidentical
• build(deps): bump mypy from 0.812 to 0.901 (#6140) @dependabot
• build(deps): bump codecov/codecov-action from 1.5.0 to 1.5.2 (#6139) @dependabot
• [pre-commit.ci] pre-commit autoupdate (#6135) @pre-commit-ci
• build(deps): bump rtCamp/action-slack-notify from 2.1.3 to 2.2.0 (#6128) @dependabot
• ci: move to ubuntu-18 from ubuntu-16 (#6101) @isidentical

Thanks again to @dependabot, @dependabot[bot], @dmpetrov, @duijf, @efiop, @isidentical, @pmrowla, @pre-commit-ci, @pre-commit-ci[bot], @skshetry, @tomrutter and @yehiyam for the contributions! 🎉

Refer to https://dvc.org/doc/install for installation instructions.

Changes

🚀 New Features and Enhancements

• config, remote: Made S3 CA bundle customizable (#6018) @rgvanwesep
• fs: don't automatically create buckets (#6086) @efiop
• azure: better error messages / allow_anonymous_login option (#5833) @isidentical

🐛 Bug Fixes

• fs: temporarily disable prefix-based search for fsspec-implementations (#6096) @isidentical

🔨 Maintenance

• build(deps): bump pytest-cov from 2.12.0 to 2.12.1 (#6095) @dependabot
• pin networkx to 2.5 (#6085) @skshetry

Thanks again to @dependabot, @dependabot[bot], @efiop, @isidentical, @rgvanwesep and @skshetry for the contributions! 🎉

Refer to https://dvc.org/doc/install for installation instructions.

Changes

🚀 New Features and Enhancements

• respect $PAGER env vars, make -FRSX default in less (#5906) @skshetry
• commit,completion: Add stages completion for dvc commit (#5963) @ciaochiaociao

🏇 Optimizations

• git: update dulwich to 0.20.22 (#6049) @pmrowla

🐛 Bug Fixes

• fix empty string parsing for the command (#6074) @skshetry
• git: update dulwich to 0.20.22 (#6049) @pmrowla
• dvcfs: open: raise FileNotFoundError on missing version info (#6017) @efiop
• azure: use selector event loop for windows (#5983) @isidentical
• plots: handle NoRemoteError for dirs (#5976) @efiop
• s3: respect credentialpath/configpath (#5973) @isidentical
• azure: use a custom event loop for authentication (#5958) @isidentical
• move: validate new stage before move (#5953) @pmrowla
• dvcfile/stage: check if dvcfile would be gitignored on stage creation (#5907) @pmrowla

🔨 Maintenance

• fix errors raised by pylint (#6081) @skshetry
• build(deps): bump actions/cache from 2.1.5 to 2.1.6 (#6073) @dependabot
• build(deps): bump urllib3 from 1.26.4 to 1.26.5 (#6069) @dependabot
• setup: bump fsspec/s3fs/gcsfs to 2021.5.0 (#6056) @isidentical
• build(deps): bump pydocstyle from 6.0.0 to 6.1.1 (#6027) @dependabot
• build(deps): bump pytest-cov from 2.11.1 to 2.12.0 (#6022) @dependabot
• build(deps): bump rtCamp/action-slack-notify from 2 to 2.1.3 (#5993) @dependabot
• build(deps): bump actions/setup-ruby from 1 to 1.1.3 (#5994) @dependabot
• Decoupling dvcignore and fs (#5812) @karajan1001
• setup: freeze aiobotocore/botocore/boto3 versions (#5987) @isidentical
• build(deps): bump pytest-mock from 3.6.0 to 3.6.1 (#5981) @dependabot
• run gdrive tests in a single job, run all on schedule (#5974) @skshetry
• build(deps): bump pytest from 6.2.3 to 6.2.4 (#5968) @dependabot
• s3: fsspec migration (#5683) @isidentical
• build(deps): bump moto from 2.0.5 to 2.0.6 (#5955) @dependabot
• build(deps): bump pygments from 2.8.1 to 2.9.0 (#5956) @dependabot
• build(deps): bump codecov/codecov-action from v1.4.1 to v1.5.0 (#5957) @dependabot
• tests: mark test_stash_pop as xfail on linux (#5952) @skshetry

Thanks again to @ciaochiaociao, @dependabot, @dependabot[bot], @dmpetrov, @efiop, @isidentical, @karajan1001, @pmrowla, @shcheklein and @skshetry for the contributions! 🎉

Refer to https://dvc.org/doc/install for installation instructions.

Changes

🚀 New Features and Enhancements

• exp run: error out when trying to resume from intermediate checkpoint w/no modifications (#5829) @pmrowla
• exp show: separate ui representation of exp names and identifiers (#5885) @skshetry
• diff: use ui.table for --show-md (#5887) @skshetry
• refactor exp show: use new abstractions (#5880) @skshetry
• refactor diff and show to use new table ui and new data structure (#5872) @skshetry
• Add full stops in CLI help (#5867) @jankrepl
• exp run: dvc commit DVC-tracked data deps when stashing an experiment (#5859) @pmrowla
• html: enable providing custom html page templates (#5694) @pared
• exp: use exp baseline for DVC commands that compare to HEAD during exp run (#5818) @pmrowla
• Add support for kerberos token based auth (#5809) @naveen24691
• dvc: add -A flag for * --all-commits (#5783) @jorgeorpinel
• exp run: disallow --rev for workspace runs (#5831) @pmrowla
• exp run: suppress "finished run" output on --dry (#5830) @pmrowla
• list: add --show-json option (#5888) @pared

🏇 Optimizations

• fs: introduce fs.find() (#5879) @isidentical
• objects: don't recompute hash on odb.add() if verify is False (#5860) @isidentical
• update: only update the missing hashes with --to-remote (#5773) @isidentical

🐛 Bug Fixes

• pygit: force release of odb contexts after checkout/merge (#5849) @pmrowla
• output: relative paths outside of repo (#5893) @efiop
• metrics: pass --precision to _show_metrics (#5839) @isidentical

🔨 Maintenance

• dependabot: use "maintenance" label (#5898) @efiop
• fs: introduce fs.find() (#5879) @isidentical
• exp show: separate ui representation of exp names and identifiers (#5885) @skshetry
• build(deps): bump google-cloud-storage from 1.37.1 to 1.38.0 (#5883) @dependabot
• build(deps): bump pylint from 2.8.1 to 2.8.2 (#5884) @dependabot
• dagascii: remove unused pager (#5876) @skshetry
• build(deps): bump pre-commit/action from v2.0.2 to v2.0.3 (#5856) @dependabot
• build(deps): bump pylint-pytest from 1.0.3 to 1.1.2 (#5850) @dependabot
• build(deps): bump pytest-mock from 3.5.1 to 3.6.0 (#5874) @dependabot
• build(deps): bump pylint from 2.7.4 to 2.8.1 (#5873) @dependabot
• output: introduce BaseOutput.transfer (#5861) @isidentical
• build(deps): bump codecov/codecov-action from v1.3.2 to v1.4.0 (#5838) @dependabot
• build(deps): bump codecov/codecov-action from v1.4.0 to v1.4.1 (#5857) @dependabot

Thanks again to @bhagone, @dependabot, @dependabot[bot], @efiop, @isidentical, @jankrepl, @jorgeorpinel, @naveen24691, @pared, @pmrowla and @skshetry for the contributions! 🎉

Refer to https://dvc.org/doc/install for installation instructions.

Changes

• build(deps): bump actions/cache from v2.1.4 to v2.1.5 (#5805) @dependabot
• build(deps): bump actions/setup-python from v2.2.1 to v2.2.2 (#5804) @dependabot
• params: read from ParamsDependency if possible (#5550) @pared
• build(deps): bump moto from 2.0.4 to 2.0.5 (#5799) @dependabot
• fsspec: update to 0.9 (including remotes) (#5788) @isidentical
• build(deps): bump codecov/codecov-action from v1.3.1 to v1.3.2 (#5770) @dependabot
• build(deps): bump google-cloud-storage from 1.37.0 to 1.37.1 (#5771) @dependabot
• gha: packages: don't build snap (#5774) @efiop
• setup: fix all fsspec remote versions (#5769) @isidentical
• build(deps): bump pre-commit/action from v2.0.0 to v2.0.2 (#5767) @dependabot
• build(deps): bump pytest from 6.2.2 to 6.2.3 (#5764) @dependabot
• use sys.executable instead of hardcoded python (#5763) @skshetry
• Update pycryptodome to 3.10 (#5762) @skshetry

🚀 New Features and Enhancements

• exp show: fix bug where params from commits were merged with workspace values (#5813) @pmrowla
• ui: unify tables (#5756) @skshetry
• azure: use account_name with default credentials (#5725) @friggog
• exp show: Allow for wildcard patterns in include and exclude params/m… (#5720) @daavoo

🐛 Bug Fixes

• repo: handle rev without scm (#5822) @efiop
• exp show: fix bug where params from commits were merged with workspace values (#5813) @pmrowla
• remote: use configured jobs number (#5747) @pared
• exp run: jobs should default to 1 (#5785) @pmrowla
• exp run: --reset no longer implies --force (#5751) @pmrowla

Thanks again to @daavoo, @dependabot, @dependabot[bot], @efiop, @friggog, @isidentical, @pared, @pmrowla, @shcheklein and @skshetry for the contributions! 🎉

Refer to https://dvc.org/doc/install for installation instructions.

Changes

• setup: bump adlfs to 0.7 (#5760) @isidentical

Thanks again to @efiop and @isidentical for the contributions! 🎉

Refer to https://dvc.org/doc/install for installation instructions.

Changes

• build(deps): bump moto from 2.0.3 to 2.0.4 (#5755) @dependabot
• build: fix fpm build zsh-completion path (#5743) @pmrowla
• build(deps): bump moto from 2.0.2 to 2.0.3 (#5744) @dependabot

🚀 New Features and Enhancements

• exp show: fix queued exp display (#5752) @pmrowla
• Added SSL verification option (#5732) @NigelVanHattum

🏇 Optimizations

• status: collect stages only once (#5741) @skshetry

🐛 Bug Fixes

• exp show: fix queued exp display (#5752) @pmrowla
• webdav: use root url to check and on mkdir to find recurse depth (#5742) @skshetry
• dvcignore: ignore file '.git' by default (#5745) @pmrowla

Thanks again to @NigelVanHattum, @dependabot, @dependabot[bot], @efiop, @pmrowla and @skshetry for the contributions! 🎉

Refer to https://dvc.org/doc/install for installation instructions.

Changes

• dvc: fix breakage with rich==10.0.0 (#5733) @alopezz

🚀 New Features and Enhancements

• ui: Use sys.stdout and sys.stderr dynamically (#5729) @skshetry

🐛 Bug Fixes

• parsing: resolve relative paths when loading parameters (#5734) @skshetry

Thanks again to @alopezz, @efiop and @skshetry for the contributions! 🎉

Refer to https://dvc.org/doc/install for installation instructions.

Changes

• git: update dulwich to 0.20.21 (#5730) @pmrowla
• build(deps): bump pylint from 2.7.2 to 2.7.4 (#5726) @dependabot

Thanks again to @dependabot, @dependabot[bot], @efiop and @pmrowla for the contributions! 🎉

Refer to https://dvc.org/doc/install for installation instructions.

Changes

• deps: limit rich version to <10 (#5723) @skshetry

Thanks again to @efiop and @skshetry for the contributions! 🎉

Refer to https://dvc.org/doc/install for installation instructions.

Changes

• dependencies: pin urllib3 to 1.26.4 (#5714) @skshetry
• build(deps): bump pytest-mock from 1.11.2 to 3.5.1 (#5705) @dependabot
• build(deps): update pytest requirement from <6.2.2,>=6.0.1 to ==6.2.2 (#5706) @dependabot
• test requirements: remove pytest-tap (#5713) @skshetry
• build(deps): bump google-cloud-storage from 1.19.0 to 1.37.0 (#5711) @dependabot
• build(deps): update urllib3 requirement from <1.26.0 to <1.27.0 (#5707) @dependabot
• build(deps): bump moto from 1.3.16.dev122 to 2.0.2 (#5709) @dependabot
• build(deps): bump jaraco-windows from 3.9.2 to 5.5.0 (#5700) @dependabot
• build(deps): bump actions/setup-python from v2.1.4 to v2.2.1 (#5702) @dependabot
• build(deps): bump actions/upload-release-asset from v1.0.1 to v1.0.2 (#5704) @dependabot
• build(deps): bump beautifulsoup4 from 4.4.0 to 4.9.3 (#5703) @dependabot
• build(deps): bump actions/cache from v2 to v2.1.4 (#5699) @dependabot
• build(deps): bump actions/checkout from v2.3.2 to v2.3.4 (#5698) @dependabot
• build(deps): bump codecov/codecov-action from v1.0.13 to v1.3.1 (#5701) @dependabot

🏇 Optimizations

• pull: skip isdir() checks on downloading files (#5696) @isidentical

🔨 Maintenance

• dvc: update precommit hooks (#5716) @skshetry
• Add dependabot to update github-actions and python requirements (#5697) @skshetry
• Move tests requirements to test_requirements.txt (#5674) @skshetry

Thanks again to @dependabot, @dependabot[bot], @efiop, @isidentical and @skshetry for the contributions! 🎉

Refer to https://dvc.org/doc/install for installation instructions.

Changes

🔨 Maintenance

• build: add osx signing/notarization (#5695) @efiop

Thanks again to @efiop for the contributions! 🎉

Refer to https://dvc.org/doc/install for installation instructions.

Changes

🔨 Maintenance

• scripts: fpm: install zsh competion to install / "share"

Refer to https://dvc.org/doc/install for installation instructions.

Changes

🔨 Maintenance

• build: osxpkg: build lxml (#5687) @efiop

Thanks again to @efiop for the contributions! 🎉