The Metaflow 2.7.12 release is a minor release

What's Changed

• Make plugins.airflow.plumbing a well-formed module by @savingoyal in https://github.com/Netflix/metaflow/pull/1148

Full Changelog: https://github.com/Netflix/metaflow/compare/2.7.11...2.7.12

The Metaflow 2.7.11 release is a minor release

Fixes

• Fix DeprecationWarning on invalid escape sequence by @tommybrecher in https://github.com/Netflix/metaflow/pull/1133
• fix cpu value formatting for aws batch/sfn by @oavdeev in https://github.com/Netflix/metaflow/pull/1140

New Contributors

• @tommybrecher made their first contribution in https://github.com/Netflix/metaflow/pull/1133

Full Changelog: https://github.com/Netflix/metaflow/compare/2.7.10...2.7.11

What's Changed

• Card bug fix when task-ids are non-unique by @valayDave in https://github.com/Netflix/metaflow/pull/1126
• Bump version to 2.7.10 to prepare for release by @romain-intel in https://github.com/Netflix/metaflow/pull/1136

Full Changelog: https://github.com/Netflix/metaflow/compare/2.7.9...2.7.10

What's Changed

• Fix issue with S3 URLs for packages by @savingoyal in https://github.com/Netflix/metaflow/pull/1130

Full Changelog: https://github.com/Netflix/metaflow/compare/2.7.8...2.7.9

What's Changed

• Support airflow with metaflow on azure by @valayDave in https://github.com/Netflix/metaflow/pull/1127
• Fix issue with S3 invocation for conda bootstrap by @savingoyal in https://github.com/Netflix/metaflow/pull/1128

Full Changelog: https://github.com/Netflix/metaflow/compare/2.7.7...2.7.8

Metaflow 2.7.7 Release Notes

The Metaflow 2.7.7 release is a minor release

Fixes:

• Fix an issue with get_cards not respecting a Task's ds-root in https://github.com/Netflix/metaflow/pull/1111
• more robust resource type conversions for aws batch/sfn in https://github.com/Netflix/metaflow/pull/1118

Full Changelog: https://github.com/Netflix/metaflow/compare/2.7.6...2.7.7

What's Changed

• Fix another issue with the escape hatch and paths by @romain-intel in https://github.com/Netflix/metaflow/pull/1105
• Bump to 2.7.6 by @romain-intel in https://github.com/Netflix/metaflow/pull/1106

Full Changelog: https://github.com/Netflix/metaflow/compare/2.7.5...2.7.6

What's Changed

• Fix for env_escape bug when importing local packages by @hunsdiecker in https://github.com/Netflix/metaflow/pull/1100
• Bump to 2.7.5 by @romain-intel in https://github.com/Netflix/metaflow/pull/1102

New Contributors

• @hunsdiecker made their first contribution in https://github.com/Netflix/metaflow/pull/1100

Full Changelog: https://github.com/Netflix/metaflow/compare/2.7.4...2.7.5

What's Changed

• Fix docstrings for the upcoming API reference (no functional changes!) by @tuulos in https://github.com/Netflix/metaflow/pull/1076
• Move a sys.path modification in s3op to main by @romain-intel in https://github.com/Netflix/metaflow/pull/1095
• Airflow Support by @valayDave in https://github.com/Netflix/metaflow/pull/1094
• Move sys.path insert earlier in s3op.py by @romain-intel in https://github.com/Netflix/metaflow/pull/1098
• bump version to 2.7.4 for release by @savingoyal in https://github.com/Netflix/metaflow/pull/1099

Full Changelog: https://github.com/Netflix/metaflow/compare/2.7.3...2.7.4

Metaflow 2.7.3 Release Notes

The Metaflow 2.7.3 release is a minor release

Fixes

• Fix fractional resoure handling for batch in https://github.com/Netflix/metaflow/pull/1089

Improvements

• Metadata version check flag by @mrfalconer in https://github.com/Netflix/metaflow/pull/1088

Full Changelog: https://github.com/Netflix/metaflow/compare/2.7.2...2.7.3

Metaflow 2.7.2 Release Notes

Metaflow 2.7.2 is a minor release

Features

• Support M1 Macs for @conda in https://github.com/Netflix/metaflow/pull/1077

Full Changelog: https://github.com/Netflix/metaflow/compare/2.7.1...2.7.2

This is a patch release addressing a behavior of the environment escape mechanism.

Bug Fixes

• Previously, if the environment escape mechanism provided a package, a failure would occur if that package was also present in the inner environment. This is now changed and, in that case, the package present is used and the environment escape mechanism is not used.

Full Changelog: https://github.com/Netflix/metaflow/compare/2.7.0...2.7.1

This is a minor release which primarily adds the ability to do runtime tagging.

Features

• Adds the ability to mutate a run's tags during or after a run. A CLI tool is provided (tag) as well as methods in the client add_tags, replace_tags and remove_tags. If using the Metaflow Metadata service, a version greater than 2.3.0 is required to use this feature.

Full Changelog: https://github.com/Netflix/metaflow/compare/2.6.3...2.7.0

Metaflow 2.6.3 Release Notes

The Metaflow 2.6.3 release is a minor release

Bug Fixes

• Fix instance metadata calls for IMDSV2 in https://github.com/Netflix/metaflow/pull/1053

Full Changelog: https://github.com/Netflix/metaflow/compare/2.6.2...2.6.3

Metaflow 2.6.2 Release Notes

The Metaflow 2.6.2 release is a minor release

Features

• Support setting default secrets for @kubernetes (#1048 ). Metaflow allows you mount secrets in Kubernetes containers created by tasks. Now you can specify a set of secrets to be mounted by default via METAFLOW_KUBERNETES_SECRETS configuration option, in addition to existing @kubernetes(secrets="...") API.

Fixes

• When using --run-id-file, the file is now written prior to execution when resuming a flow (#1051). That matches how run command behaves already.

Full Changelog: https://github.com/Netflix/metaflow/compare/2.6.1...2.6.2

Metaflow 2.6.1 Release Notes

The Metaflow 2.6.1 release is a minor release.

Features

• Proper support for custom S3 endpoints. This enables using S3-compatible object storages like MinIO or Dell EMC-ECS as data stores for Metaflow ( https://github.com/Netflix/metaflow/pull/1045 )

Bug fixes

• Fixed card rendering for tables with some NaN values (https://github.com/Netflix/metaflow/issues/1023) in https://github.com/Netflix/metaflow/pull/1025
• current.pathspec to return None when used outside Flow in https://github.com/Netflix/metaflow/pull/1033
• Fixed bug in the card list command in https://github.com/Netflix/metaflow/pull/1044
• Fixed issues with S3 get and ranges in https://github.com/Netflix/metaflow/pull/1034
• Fix _new_task calling bug in LocalMetadataProvider in https://github.com/Netflix/metaflow/pull/1046

Full Changelog: https://github.com/Netflix/metaflow/compare/2.6.0...2.6.1

Metaflow 2.6.0 Release Notes

The Metaflow 2.6.0 release is a minor release and introduces Metaflow's integration with Kubernetes and Argo Workflows

• Features
  • Add capability to launch Metaflow tasks on Kubernetes and schedule Metaflow flows with Argo Workflows.
  • Expose tags in current object.

Features

Add capability to launch Metaflow tasks on Kubernetes and schedule Metaflow flows with Argo Workflows.

This release enables brand new capabilities for Metaflow on top of Kubernetes. You can now run --with kubernetes all or parts of any Metaflow flow on top of any Kubernetes cluster from your workstation. To execute your flow asynchronously, you can deploy the flow to Argo Workflows (a Kubernetes-native workflow scheduler) with a single command - argo-workflows create.

To get started, take a look at the deployment guide for Kubernetes. Your feedback and feature requests are highly appreciated! - please reach out to us at slack.outerbounds.co

PR #992 addressed issue #50.

Expose tags in current object.

Metaflow tags are now available as part of the current singleton object.

@step
def my_step(self):
    from metaflow import current
    tags = current.tags
    ...

PR #1019 fixed issue #1007.

Metaflow 2.5.4 Release Notes

The Metaflow 2.5.4 release is a minor release.

Bug Fixes

• Card bug fixes (https://github.com/Netflix/metaflow/pull/989, https://github.com/Netflix/metaflow/pull/981 )
• importlib_metadata fixes for Python 3.5 (https://github.com/Netflix/metaflow/pull/988 )
• Configurable temp root when pulling artifacts from s3 by @kgullikson88 in https://github.com/Netflix/metaflow/pull/991

Full Changelog: https://github.com/Netflix/metaflow/compare/2.5.3...2.5.4

Metaflow 2.5.3 Release Notes

The Metaflow 2.5.3 release is a minor release.

Fixes

• Fix "Too many symbolic links" error when using Conda + Batch on MacOS by @bishax in https://github.com/Netflix/metaflow/pull/972
• Emit app tag for AWS Batch jobs ( https://github.com/Netflix/metaflow/pull/970 )

Full Changelog: https://github.com/Netflix/metaflow/compare/2.5.2...2.5.3

Metaflow 2.5.2 Release Notes

The Metaflow 2.5.2 release is a minor release.

Improvements

• follow symlinks when creating code packages https://github.com/Netflix/metaflow/pull/958

Full Changelog: https://github.com/Netflix/metaflow/compare/2.5.1...2.5.2