azure-data-lake-store-python

Disclaimer
This is a pre-release or preview version of the azure-datalake-store package and should be treated as such. As a result, there will be fairly rapid development and bug fixing, which might result in breaking changes from release to release.

What's new in this release

• Address a request to indicate that files being downloaded are in progress until download completes
• All files will have a suffix .inprogress until they are fully downloaded, at which point the suffix will be removed.

Full pip release location
https://pypi.python.org/pypi/azure-datalake-store/0.0.11

Disclaimer
This is a pre-release or preview version of the azure-datalake-store package and should be treated as such. As a result, there will be fairly rapid development and bug fixing, which might result in breaking changes from release to release.

What's new in this release

• Allow users to explicitly use or invalidate the internal, local cache of the filesystem that is built up from previous ls calls. It is now set to always call the service instead of the cache by default.
• Update to properly create the wheel package during build to ensure all pip packages are available.
• Update folder upload/download to properly throw early in the event that the destination files exist and overwrite was not specified. NOTE: target folder existence (or sub folder existence) does not automatically cause failure. Only leaf node existence will result in failure.
• Fix a bug that caused file not found errors when attempting to get information about the root folder.

Full pip release location
https://pypi.python.org/pypi/azure-datalake-store/0.0.10

Disclaimer
This is a pre-release or preview version of the azure-datalake-store package and should be treated as such. As a result, there will be fairly rapid development and bug fixing, which might result in breaking changes from release to release.

What's new in this release

• Addresses issue: https://github.com/pyca/pyopenssl/issues/625, ensuring consistent performance of upload and download regardless of which underlying packages are installed.

Full pip release location
https://pypi.python.org/pypi/azure-datalake-store/0.0.9

Disclaimer
This is a pre-release or preview version of the azure-datalake-store package and should be treated as such. As a result, there will be fairly rapid development and bug fixing, which might result in breaking changes from release to release.

What's new in this release

• Enabled more robust throttling handling and back-off retry support. This change helps ensure that upload and download will more reliably complete successfully in the event of intermittent server-side throttling through the use of a back-off retry strategy. Note that this strategy is a best effort and will "give up" if the server continues to reject requests for an extended period of time, ultimately returning the error back to the user. In most cases, though, the retry strategy will work and allow for the operation to complete successfully.

Full pip release location
https://pypi.python.org/pypi/azure-datalake-store/0.0.8

Disclaimer
This is a pre-release or preview version of the azure-datalake-store package and should be treated as such. As a result, there will be fairly rapid development and bug fixing, which might result in breaking changes from release to release.

What's new in this release

• Updated how packages are built to enable a more reliable pip package and cleaner install and uninstall.

Full pip release location
https://pypi.python.org/pypi/azure-datalake-store/0.0.7

Disclaimer
This is a pre-release or preview version of the azure-datalake-store package and should be treated as such. As a result, there will be fairly rapid development and bug fixing, which might result in breaking changes from release to release.

What's new in this release

• Fixed an issue with lib.auth() credential expiration and refresh. It should now properly refresh.
• Fixed ADLUploader and ADLDownloader to handle large folders with higher performance.

Full pip release location
https://pypi.python.org/pypi/azure-datalake-store/0.0.6

Disclaimer
This is a pre-release or preview version of the azure-datalake-store package and should be treated as such. As a result, there will be fairly rapid development and bug fixing, which might result in breaking changes from release to release.

What's new in this release

• Credentials and lib.auth() now use the generic msrest.authentication class. This means that any Authentication object that cotnains a signed_session() method that returns an Authorization header can be used.
• Fix ADLDownloader to do validation of the source path before creating local files.
• Adds set_expiry and ACL management functions.
• Adds support for API versions
• Fixes an issue where exceptions were not bubbled up on all transfer failures. Now if there is any failed transfer an exception will be raised.

Full pip release location
https://pypi.python.org/pypi/azure-datalake-store/0.0.5

Disclaimer
This is a pre-release or preview version of the azure-datalake-store package and should be treated as such. As a result, there will be fairly rapid development and bug fixing, which might result in breaking changes from release to release.

What's new in this release

• Fix for overwriting a folder during upload when the target folder is not empty
• Disabled verbose logging by default, which increases performance without the need for explicitly disabling verbose output.

Full pip release location
https://pypi.python.org/pypi/azure-datalake-store/0.0.4

This release is identical to release v0.0.2, but fixes the PyPi package issues reported as #139 and #136

Disclaimer
This is a pre-release or preview version of the azure-datalake-store package and should be treated as such. As a result, there will be fairly rapid development and bug fixing, which might result in breaking changes from release to release.

What's new in this release

• Better support for two factor authentication through lib.auth()
• Fixes to upload and download stability.
  • Overwrite will now properly overwrite files and folders on upload and download
• Support for upload and download of empty files
• General documentation fixes and cleanup to accurately reflect the client features. This is an ongoing effort and we will continue to improve the documentation as we move towards a stable release.
• Introduction of a samples directory, which includes the following:
  • A sample CLI implementation that shows some common ways to use the SDK
  • A sample benchmarking script which showcases how to use the ADLUploader and ADLDownloader classes, as well as giving the user the opportunity to measure upload and download performance in a consistent way.

Full pip release location
https://pypi.python.org/pypi/azure-datalake-store/0.0.2

Disclaimer
This is a pre-release or preview version of the azure-datalake-store package and should be treated as such. As a result, there will be fairly rapid development and bug fixing, which might result in breaking changes from release to release.

What's new in this release

• First release of the package (so it's all new!)
• Ability to interact with the Azure Data Lake Store filesystem in python:
  • File and folder management (create, append, concatenate, get, list, move and delete)
  • Permissions management (add, remove, get and set permissions on files and folders)
• High performance file and folder upload and download
• Proof of concept CLI implementation on top of the SDK:
  • How to authenticate
  • How to setup logging
  • How to exercise the various components of the SDK
• Robust test scripts for benchmarking ingress and egress (bring your own data to test with).

Full pip release location
https://pypi.python.org/pypi/azure-datalake-store/0.0.1