Security fixes

1. Updated Golang version to 1.22.3 to address security vulnerabilities

New Features

1. Workload Identity authentication is now available (#2619)
2. azcopy list now supports a --location flag, to support ambiguous URIs (#2595)
3. azcopy list now properly supports --output-type=json for users in automated scenarios. (#2629)

Bug Fixes

1. Fixed a bug where AzCopy was not reporting performance info in -output-type=json (#2636)
2. Fixed a bug AzCopy would crash when operating on extremely large (16.5+TB) managed disks (#2635)
3. Fixed a bug with hash-based sync where the directory structure would not be replicated when using --local-hash-storage-mode=HiddenFiles with --hash-meta-dir (#2611)
4. Fixed a bug where attempting to use a non-S3/GCP/Azure storage URI would result in treating the URI as a local path (#2652)

Documentation changes

1. Updated inaccurate helptext and filled in missing helptext (#2649)
2. Many important errors now include a link to relevant documentation to assist users in troubleshooting AzCopy (#2647)
3. Ambiguous flags (such as --cpk-by-value) have improved clarity in documentation (#2615)
4. A clearer error message is provided when failing a transfer due to authorization. (#2644)
5. A special error has been created when performing an Azure Files to Azure Blob Storage transfer, indicating present lack of service-side support (#2616)

Version 10.25.0-Preview-1

Security fixes

1. Updated version of GoLang used to 1.21 to address security vulnerabilities.

This release can be downloaded at the following links.

 Darwin AMD64: https://azcopyvnext.azureedge.net/releases/release-10.25.0-Preview-1-20240403/azcopy_darwin_amd64_10.25.0-Preview-1.zip
Darwin ARM64: https://azcopyvnext.azureedge.net/releases/release-10.25.0-Preview-1-20240403/azcopy_darwin_arm64_10.25.0-Preview-1.zip
Linux AMD64: https://azcopyvnext.azureedge.net/releases/release-10.25.0-Preview-1-20240403/azcopy_linux_amd64_10.25.0-Preview-1.tar.gz
Linux ARM64: https://azcopyvnext.azureedge.net/releases/release-10.25.0-Preview-1-20240403/azcopy_linux_arm64_10.25.0-Preview-1.tar.gz
Windows 386: https://azcopyvnext.azureedge.net/releases/release-10.25.0-Preview-1-20240403/azcopy_windows_386_10.25.0-Preview-1.zip
Windows AMD64: https://azcopyvnext.azureedge.net/releases/release-10.25.0-Preview-1-20240403/azcopy_windows_amd64_10.25.0-Preview-1.zip

Version 10.24.0

New Features

1. Print summary logs at lower log levels and add BytesTransferred to the output in the jobs show command. (#1319)
2. Added a flag --put-blob-size-mb to copy, sync and bench commands to specify the maximum size of a blob to be uploaded using PutBlob. (#2561)
3. Added support for latest put blob service limits. Block blob put blob size can now be set up to 5000MB. (#2569)
4. Updated all SDK dependencies to their latest version. (#2599)
5. Updated summary logs to use consistent wording across all commands. (#2602)

Bug Fixes

1. Fixed an issue where AzCopy would fail to auto login with the AZCOPY_AUTO_LOGIN_TYPE environment variable set to PSCRED on certain Linux and MacOS environments. (#2491)(#2555)
2. Fixed a bug where page blob download optimizer would behave incorrectly on highly fragemented blobs if the service times out. (#2445)
3. Ignore 404 errors on retried deletes. (#2554)
4. Fixed a bug where the VersionID property would not be honored on the list command. (#2007)
5. Fixed a bug where ADLS Gen2 paths with encoded special characters would fail to transfer. (#2549)
6. Fixed an issue where ACL copying would fail when specifying an ADLS Gen2 account with the blob endpoint. (#2546)
7. Fixed an issue where the snapshot ID would not be preserved when testing which authentication type to use for managed disks. (#2547)
8. Fixed an issue where copy would panic if a root directory is specified as the destination. (#2036)

Documentation

1. Removed the azcopy login/logout deprecation notice. (#2589)
2. Added a warning for customers using Shared Key for Azure Datalake transfers to indicate that Shared Key authentication will be deprecated and removed in a future release. (#2569)
3. Updated the list help text to clearly indicate the services and authentication types supported.(#2563)

Security fixes

1. Updated dependencies to address security vulnerabilities.

Version 10.23.0

New Features

1. Added support to ignore the error and output a summary if a cancelled job has already completed through the use of the --ignore-error-if-completed flag. (#2519)
2. Added support for high throughput append blob. Append blob block size can now be set to up to 100 MB. (#2480)
3. Added support to exclude containers when transferring from account to account through the use of the --exclude-container flag. (#2504)

Bug Fixes

1. Fixed an issue where specifying AZCOPY_AUTO_LOGIN_TYPE in any form other than uppercase would be incorrectly parsed. (#2499)
2. Fixed an issue where a failure to rename a file from the temporary prefix to the file name would not be considered to be a failed transfer. (#2481)
3. Fixed an issue where closing the log would panic for benchmark jobs. (#2537)
4. Fixed an issue where --preserve-posix-properties would not work on download. (#2497)
5. Fixed an issue where --decompress would not be honored in Linux. (#2392)
6. Fixed an issue where log files would miss the .log extension. (#2529)
7. Fixed an issue where AzCopy would fail to set metadata properties on a read only directory when using the --force-if-read-only flag. (#2515)
8. Fixed an issue where the AzCopy log location on resumed jobs would be reported incorrectly. (#2466)
9. Fixed an issue with preserving SMB properties in Linux. (#2530)
10. Fixed an issue where long-running service to service copies using OAuth at the source would result in the token expiring too early. (#2513)
11. Fixed an issue where AzCopy would try to create folders that already existed, resulting in many unnecessary requests. (#2511)

Documentation

1. Updated --include-directory-stub inline help to match public documentation. (#2488)

Version 10.22.2

1. Fixed an issue where AzCopy operations pointed at a snapshot or version object would operation on the base object instead.
2. Fixed an issue where AzCopy would download only the base blob when the --list-of-versions flag was used.

Version 10.22.1

Bug Fixes

1. Fixed a regression with Azurite support. (#2485)
2. Fixed an issue where AZCOPY_OAUTH_TOKEN_INFO would be refreshed too often. (#2503)
3. Fixed an issue where commands would lag for multiple seconds. (#2482)
4. Fixed an issue where azcopy version would crash. (#2483)

Documentation

1. Updated documentation to include AZCLI and PSCRED auto login types. (#2494)

Security fixes

1. Updated dependencies to address security vulnerabilities.

Version 10.22.0

New Features

1. Migrated to the latest azdatalake SDK.
2. Added support for OAuth when performing File -> File and Blob -> File copy/sync and File make/list/remove (#2302).
3. Added support to set tier on premium block blob accounts. (#2337)
4. Added support to cache latest AzCopy version and check the remote version every 24 hours instead of every run. (#2426)
5. Updated all SDK dependencies to their latest version and the default service version to 2023-08-03 for all services. (#2402)
6. Added support to rotate AzCopy logs. (#2213)
7. Added support to authenticate with Powershell and Azure CLI credentials. (#2433)

Bug Fixes

1. Fixed an issue where http headers and access tier would sometimes be sent as empty headers.
2. Fixed an issue where AzCopy would panic when passing an un-parseable URL. (#2404)
3. Fixed an issue where Object ID would be set as Resource ID when using MSI. (#2395)
4. Fixed an issue where the percent complete stat could round incorrectly. (#1078)
5. Fixed an issue where no transfers were scheduled would be logged as an error, it is now logged as a warning. (#874)
6. Fixed an issue where non canonicalized headers would not be printed in the log. (#2454)
7. Fixed an issue where cold tier would not be recognized as an allowed tier. (#2447)
8. Fixed an issue where s2s append blob copies would fail with AppendPositionConditionNotMet error on retry after first experiencing a service timeout error. (#2430)
9. Fixed an issue where AZCOPY_OAUTH_TOKEN_INFO would not be refreshed. (#2434)

Documentation

1. Updated --preserve-permissions documentation to indicate the correct roles necessary to perform the operation. (#2440)
2. Updated help message for sync and copy to include all ADLS Gen2 supported auth methods. (#2440)

Security fixes

1. Updated dependencies to address security vulnerabilities.

Version 10.21.2

Security fixes

1. Updated dependencies to address security vulnerabilities.

Version 10.21.1

Bug Fixes

1. Fixed an issue where validating destination length would fail a job instead of logging the error if read permissions are not provided.

Version 10.22.0-Preview

New Features

1. Migrated to the latest azdatalake SDK.

Bug Fixes

1. Fixed an issue where http headers and access tier would sometimes be sent as empty headers.
2. Fixed an issue where AzCopy would panic when passing an un-parseable URL. (#2404)

Security fixes

1. Updated dependencies to address security vulnerabilities.

Change Log

Version 10.21.0

New Features

1. Migrated to the latest azblob SDK.
2. Migrated to the latest azfile SDK.
3. Migrated from deprecated ADAL to MSAL through the latest azidentity SDK.
4. Added support for sync with Azure Data Lake Storage Gen2. (#2376)

Bug Fixes

1. Fixed an issue where ACL data would not copy when specifying *.dfs.core.windows.net endpoints (#2347).
2. Fixed an issue where Sync would incorrectly log that all files, even those that didn't get overwritten, would be overwritten. (#2372)

Documentation

1. Updated --dry-run documentation to indicate the effects of --overwrite are ignored. (#2325)

Special notes

1. Due to the migration from ADAL to MSAL, tenant ID must now be set when authorizing with single tenant applications created after 10/15/2018.

Version 10.21.0-Preview

New Features

1. Migrated to the latest azblob SDK.
2. Migrated to the latest azfile SDK.
3. Migrated from deprecated ADAL to MSAL through the latest azidentity SDK.
4. Deprecated support for object IDs in MSI. Client ID or Resource ID can be used as an alternative.

Special notes

1. Due to the migration from ADAL to MSAL, tenant ID must now be set when authorizing with single tenant applications created after 10/15/2018.

Version 10.20.1

Bug Fixes

1. Fixed an issue where LMT data is not returned on list command for Azure Files.

Change Log

Version 10.20.0

New Features

1. Mac M1/M2 Support
2. Force small blobs to use PutBlob for any source.
3. Support to delete CPK encrypted blobs.
4. Support to follow symlinks when --preserve-smb-permissions is enabled.
5. Support to return LMT data on list command for Azure Files.

Bug Fixes

1. Fixed an issue where source trailing dot header was passed when source for a S2S copy is not file service
2. Gracefully handle File Share trailing dot paths to Windows/Blob (that do not support trailing dot) by skipping such files
3. Allow trailing dot option to be ignored instead of erroring out in situations it does not apply.
4. Issue #2186 where AzCopy would panic when using --include-before and --include-after flags on remove file share resources.
5. Issue #2183 where AzCopy would panic when providing Azure Files URLs with no SAS token.
6. Fixed a bug where AzCopy would automatically assume a HNS path to be a file if the path did not have a slash terminator.
7. Fixed an issue where --skip-version-check would not be honored for login, logout, help commands. #2299

Documentation

1. Add a log for LMTs when a mismatch is encountered.
2. Added documentation indicating the login and logout commands will be deprecated in the future.

Security fixes

1. Updated dependencies to address security vulnerabilities.

What's Changed

• Mac M1 Beta Release by @tasherif-msft in https://github.com/Azure/azure-storage-azcopy/pull/2239
• Code coverage and tests on e2e tests by @gapra-msft in https://github.com/Azure/azure-storage-azcopy/pull/2225
• Resolved a few trailing dot issues by @gapra-msft in https://github.com/Azure/azure-storage-azcopy/pull/2248
• Make pipeline fail when e2e tests fail by @gapra-msft in https://github.com/Azure/azure-storage-azcopy/pull/2261
• Prevent panic on remove with include before param on file share accounts by @gapra-msft in https://github.com/Azure/azure-storage-azcopy/pull/2270
• Return error on failure to split URL by @gapra-msft in https://github.com/Azure/azure-storage-azcopy/pull/2271
• Updated dependencies by @gapra-msft in https://github.com/Azure/azure-storage-azcopy/pull/2286
• Force small blobs to use PutBlob for all sources by @nakulkar-msft in https://github.com/Azure/azure-storage-azcopy/pull/2250
• Handle DFS directories without path separator by @nakulkar-msft in https://github.com/Azure/azure-storage-azcopy/pull/2280
• chore: unnecessary use of fmt.Sprintf by @testwill in https://github.com/Azure/azure-storage-azcopy/pull/2277
• Add additional log when LMT mismatch is seen by @nakulkar-msft in https://github.com/Azure/azure-storage-azcopy/pull/2306
• Added support to delete CPK encrypted blobs by @gapra-msft in https://github.com/Azure/azure-storage-azcopy/pull/2304
• Added skip version check before calling beginDetectNewVersion by @gapra-msft in https://github.com/Azure/azure-storage-azcopy/pull/2301
• Bump google.golang.org/grpc from 1.51.0 to 1.53.0 by @dependabot in https://github.com/Azure/azure-storage-azcopy/pull/2284
• chore: slice replace loop by @testwill in https://github.com/Azure/azure-storage-azcopy/pull/2278
• Support follow-symlinks SMB properties by @adreed-msft in https://github.com/Azure/azure-storage-azcopy/pull/2258
• Pull properties to support extra data on Azure Files by @adreed-msft in https://github.com/Azure/azure-storage-azcopy/pull/2256
• Deprecate login command by @nakulkar-msft in https://github.com/Azure/azure-storage-azcopy/pull/2310
• Updated version and changelog for 10.20.0 release by @gapra-msft in https://github.com/Azure/azure-storage-azcopy/pull/2312

New Contributors

• @testwill made their first contribution in https://github.com/Azure/azure-storage-azcopy/pull/2277
• @dependabot made their first contribution in https://github.com/Azure/azure-storage-azcopy/pull/2284

Full Changelog: https://github.com/Azure/azure-storage-azcopy/compare/v10.19.0...v10.20.0

A native build of 10.19.0 targeting Apple Silicon Macs has been released.

A temporary download link is available here.

Documentation located at https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10 will be updated shortly to include references to the preview build.

Version 10.19.0

New Features

1. Support for new Cold Tier feature for Azure Blobs (--block-blob-tier=Cold)
2. Support preserving a trailing dot ('.') in names of files and directories in Azure Files (default is --trailing-dot=Enable)
3. Alternate modes to preserve hash for hash-based sync (#2214) (default is OS-dependent, either --local-hash-storage-mode=XAttr on MacOS/Linux or --local-hash-storage-mode=AlternateDataStreams on Windows)
   • OS-specific hashing modes are expected to be available on all filesystems the source would traverse. (user_xattr enabled on filesystems on Unix systems, FILE_NAMED_STREAMS flag expected on Windows volumes)
   • HiddenFiles provides an OS-agnostic method to store hash data; to prevent "dirtying" the source, also specify --hash-meta-dir directing AzCopy to store & read hidden hash metadata files elsewhere.
4. Support 'force-if-readonly' flag for Sync. (false by default)
5. Preserve posix properties while uploading or downloading from HNS enabled accounts (--preserve-posix-properties, false by default.)

Bug Fixes

1. Fix situation where large-files would hang infinitely with low value for 'cap-mbps'
2. Issue #2074 where AzCopy would hang after cancelling
3. Issue #1888 where directories with empty name are incorrectly handled.
4. Cancel HNS delete jobs #2117
5. Fix issue where large chunks could not be scheduled #2228
6. Fixed segfault on MacOS #1790
7. Fixed panic on attempt to create AzCopy dir #2191

Version 10.18.1

Bug fixes

1. Fixed a data race when utilizing hash-based sync. (Issue 2146)
2. Fixed the destination naming behavior for container-to-container transfers while using --preserve-permissions (Issue 2141)
3. Temporarily disabled hostname lookup before enumeration (Issue 2144)

Documentation

1. Modified --from-to flag to be more clear (Issue 2153)

Version 10.18.0

New features

1. Added support for Content-MD5 in list command. User can now list the MD5 hash of the blobs in the target container.
2. Added support to resume incomplete blobs. User can now resume the upload of a blob which was interrupted in the middle.
3. Added support for download of POSIX properties.
4. Added support for persisting symlink data.

Bug fixes

1. Fixed Issue 2120
2. Fixed Issue 2062
3. Fixed Issue 2046
4. Fixed Issue 1762

Documentation

1. Added example for --include-pattern.
2. Added documentation for --compare-hash.

Security fixes

1. CPK-related headers are now sanitized from the logs.
2. Updated dependencies to address security vulnerabilities.

New features

1. Added support for hash-based sync. AzCopy sync can now take the new flag --compare-hash, which user will be able to transfer only those files which differ in their MD5 hash.

Bug fixes

1. Fixed issue 1994: Error in calculation of block size
2. Fixed issue 1957: Repeated Authentication token refresh
3. Fixed issue 1870: Fixed issue where CPK would not be injected on retries
4. Fixed issue 1946: Fixed Metadata parsing
   5: Fixed issue 1931

Bug Fixes

1. Fixed an issue where sync would always re-download files as we were comparing against the service LMT, not the SMB LMT
2. Fixed a crash when copying objects service to service using a user delegation SAS token
3. Fixed a crash when deleting folders that may have a raw path string