The new Azure Storage data transfer utility - AzCopy v10
MIT License
Bot releases are hidden (Show)
Published by JohnRusk over 4 years ago
Published by zezha-msft over 4 years ago
--preserve-smb-permissions
flag, sometimesPublished by JohnRusk over 4 years ago
azcopy copy
now supports the persistence of ACLs between supported resources (Windows and Azure Files) using the --persist-smb-permissions
flag.azcopy copy
now supports the persistence of SMB property info between supported resources (Windows and Azure Files) using the --persist-smb-info
flag. The information that can be preserved is Created Time, Last Write Time and Attributes (e.g. Read Only).SeBackupPrivilege
and SeRestorePrivilege
. Most admin-level accounts have these privileges in a deactivated state, as do all members of the "Backup Operators" security group. If you run AzCopy as one of those users and supply the new flag --backup
, AzCopy will activate the privileges. (Use an elevated command prompt, if running as Admin). At upload time, this allows AzCopy to read files which you wouldn't otherwise have permission to see. At download time, it works with the --preserve-smb-permissions
flag to allow preservation of permissions where the Owner is not the user running AzCopy. The --backup
flag will report a failure if the privileges cannot be activated.copy
, sync
, jobs list
, and jobs status
now contains information about folders.--output-type json
is used.azcopy rm
on Azure Files.copy
,sync
and list
. To use, add a sharesnapshot
parameter at end of URL for your Azure Files source. Remember to separate it from the existing query string parameters (i.e. the SAS token) with a &
. E.g.https://<youraccount>.file.core.windows.net/sharename?st=2020-03-03T20%3A53%3A48Z&se=2020-03-04T20%3A53%3A48Z&sp=rl&sv=2018-03-28&sr=s&sig=REDACTED&sharesnapshot=2020-03-03T20%3A24%3A13.0000000Z
--include-pattern
that contains only *
wildcards will be performance optimized when querying blob storage without the recursive flag. The section before the first *
will be used as a server-side prefix, to filter the search results more efficiently. E.g. --include-pattern abc*
will be implemented as a prefix search for "abc". In a more complex example, --include-pattern abc*123
, will be implemented as a prefix search for abc
, followed by normal filtering for all matches of abc*123
. To non-recursively process blobs contained directly in a container or virtual directory include /*
at the end of the URL (before the query string). E.g. http://account.blob.core.windows.net/container/*?<SAS>
.--cap-mbps
parameter now parses floating-point numbers. This will allow you to limit your maximum throughput to a fraction of a megabit per second.2019-02-02
. Users targeting local emulators, Azure Stack, or other private/special instances of Azure Storage may need to intentionally downgrade their service revision using the environment variable AZCOPY_DEFAULT_SERVICE_API_VERSION
. Prior to this release, the default service revision was 2018-03-28
.azcopy login
, the resulting token will only be sent to locations that appear in the list. The list is:*.core.windows.net;*.core.chinacloudapi.cn;*.core.cloudapi.de;*.core.usgovcloudapi.net
.--trusted-microsoft-suffixes
. For security, you should only add Microsoft Azure domains.SkippedFileAlreadyExists
has been renamed SkippedEntityExists
and may now be used both for when files are skipped and for when the setting of folder properties is skipped. This affects the input and output of azcopy jobs show
and the status values shown in the JSON output format from copy
and sync
.--force-if-read-only
flag.)from-to
flag is typically needed when using such sources or destinations. E.g. --from-to BlobLocal
if downloading from a blob storage emulator to local disk.:
can now safely be downloaded on Windows and uploaded to Azure Files+
can now safely be used in imported S3 object namescheck-length
flag is now exposed in benchmark mode, so that length checking can be turned off for more speed, when benchmarking with small file sizes. (When using large file sizes, the overhead of the length check isazcopy rm
against ADLS Gen2 endpoints. Previously include/exclude patterns were disallowed, but exclude-path was not. That was incorrect. All should have been disallowed because none (other than include-path) are respected.Published by zezha-msft almost 5 years ago
azcopy list
is now supported on Azure Files and ADLS Gen 2, in addition to Blob Storage.--exclude-path
flag is now supported in the sync
command.AZCOPY_USER_AGENT_PREFIX
to allow a prefix to be appended to the user agent strings.azcopy list
command no longer repeats parts of its output. (Previously it would sometimes repeat itself and show the same blob multiple times in the output.)--aad-endpoint
parameter is now visible, instead of hidden. It allows use of Azure Active Directory--check-length=false
.Published by zezha-msft almost 5 years ago
Published by zezha-msft about 5 years ago
Published by zezha-msft about 5 years ago
*
character is no longer supported as a wildcard in URLs, except for the two exceptions/*
is still allowed at the very end of the "path" section of ahttps://account/container/virtualDir?SAS
andhttps://account/container/virtualDir/*?SAS
. The former copies the virtual directoryvirtualDir
by creating a folder of that name at the destination. The latter copies thevirtual
dir directly into the target without creating a folder named--include
and --exclude
parameters have been replaced by --include-pattern
and--exclude-pattern
(for filenames) and --include-path
and --exclude-path
(for paths,*
wildcard is supported in the pattern parameters, but not in the path ones.sync
command's output in JSON has changed for consistency reasons,copy
and sync
has changed its name from Exit
to EndOfJob
.--list-of-files
parameter is changed. (It remains--include-pattern
and --include-path
parameters instead.)sync
is supported from Blob Storage to Blob Storage, and from Azure Files to Azure Files.copy
is supported from Azure Files to Azure Files, and from Blob Storage to Azure Files.bench --help
for details. This feature is inAZCOPY_JOB_PLAN_LOCATION
. (If you move the plan files and also move the log files using the existingAZCOPY_LOG_LOCATION
, then AzCopy will not store anything under your home directory on Linux andc:\users\<username>\.azcopy
)jobs rm
andjobs clean
commands.jobs show
, the status of each job is included in the output.--overwrite
parameter now supports the value of "prompt" to prompt the user on aAZCOPY_CONCURRENCY_VALUE
can now be set to "AUTO". This is expected to beAZCOPY_BUFFER_GB
.Content-Encoding
of gzip
or deflate
. To enable this behaviour, supply the --decompress
AZCOPY_CONCURRENT_FILES
environment variable. This is an advanced setting, which generallyAZCOPY_CONCURRENCY_VALUE
.*
in the name of a blob or Azure Files file, e.g. for a blob*
using standard URL escaping. To do this, replace the *
with the following--decompress
flag removes an extension but--content-type
on the command line it's no longer necessary to also specify--no-guess-mime-type
.sync
d:\
) no longer causes an error.remove
command no longer hangs when removing blobs that have snapshots. Instead it will fail toPublished by zezha-msft over 5 years ago
Published by zezha-msft over 5 years ago
--log-level=none
now drops no logs, and has a listing in --help
.mattn/go-ieproxy
.azcopy login
-- check azcopy env
for details on client secrets/cert passwords.azcopy cp -h
for details.Published by adreed-msft over 5 years ago
Files with Content-Encoding: gzip
are now downloaded in compressed form. Previous versions tried to save a
decompressed version of the file. But they incorrectly truncated it at the original compressed length, so the
downloaded file was not complete.
By changing AzCopy to save the compressed version, that problem is solved, and Content-MD5 checks now work for such files. (It is
assumed that the Content-MD5 hash is the hash of the compressed file.)
azcopy copy --help
to see command line parameterPublished by zezha-msft over 5 years ago
Published by zezha-msft over 5 years ago
--block-size
parameter has been replaced by --block-size-mb
. The old parameter took a number of bytes; the--output
is now --output-type
--md5-validation
is now called --check-md5
--s2s-source-change-validation
is now called --s2s-detect-source-changed
--s2s-invalid-metadata-handle
is is now called --s2s-handle-invalid-metadata
--quota
(in the make
command) is now called --quota-gb
. Note that the values were always in GB, the new name--overwrite
parameter is clarified in the in-application help.Published by zezha-msft over 5 years ago
--put-md5
on the command line.--delete-destination
flag is specified).remove
commandPublished by zezha-msft over 5 years ago
Published by zezha-msft over 5 years ago
Published by zezha-msft over 5 years ago
Published by zezha-msft almost 6 years ago
Published by zezha-msft almost 6 years ago
sync
command's performanceenv
command to show possible configurations (environment variables)Published by zezha-msft about 6 years ago
Published by zezha-msft about 6 years ago