zenml

ZenML 🙏: Build portable, production-ready MLOps pipelines. https://zenml.io.

APACHE-2.0 License

Downloads
44.5K
Stars
3.6K
View Code on GitHub Visit Website View on X
Ecosystems: PyTorch, Tensorflow

Bot releases are visible (Hide)

zenml - 0.44.4

Published by stefannica 11 months ago

This patch release backports some important fixes that have been introduced in more recent versions
of ZenML to the 0.44.x release line.

What's Changed

Full Changelog: https://github.com/zenml-io/zenml/compare/0.44.3...0.44.4

zenml - 0.46.1 [YANKED]

Published by stefannica 11 months ago

THIS RELEASE HAS BEEN YANKED

  • Due to an unforeseen alembic migration issue and missing dashboard, this release has been yanked
  • 0.47.0 is out to fix the bugs that have been uncovered

0.46.1

The 0.46.1 release introduces support for Service Accounts and API Keys that
can be used to authenticate with the ZenML server from environments that do not
support the web login flow, such as CI/CD environments, for example.

Also included in this release are some documentation updates and bug fixes,
notably moving the database migration logic deployed with the Helm chart out of
the init containers and into a Kubernetes Job, which makes it possible to scale
out the ZenML server deployments without the risk of running into database
migration conflicts.

What's Changed

Full Changelog: https://github.com/zenml-io/zenml/compare/0.46.0...0.46.1

zenml - 0.46.0

Published by strickvl 12 months ago

This release brings some upgrades, documentation updates and bug fixes. Notably,
our langchain integration now supports more modern versions and has been
upgraded to a new version at the lower edge of supported packages on account of
a security vulnerability.

Other fixes related to the Model Control Plane which was updated to support the
deletion of model versions via the CLI, for example.

Breaking Change

We removed the llama_index integration in this release. This related to
unsolvable dependency clashes that relate to sqlmodel and our database. We
expect these clashes to be resolved in the future and then we will add our
integration back in. If you were using the llama_index materializer that was
part of the integration, you will have to use a custom materializer in the
meanwhile. We apologize for the inconvenience.

What's Changed

Full Changelog: https://github.com/zenml-io/zenml/compare/0.45.6...0.46.0

zenml - 0.45.6

Published by stefannica 12 months ago

This release brings an array of enhancements and refinements. Notable improvements include
allowing for disconnecting service connectors from stack components, adding connector support to the
sagemaker step operator, turning synchronous mode on by default for all orchestrators, and enabling
server-side component config validation.

What's Changed

New Contributors

Full Changelog: https://github.com/zenml-io/zenml/compare/0.45.5...0.45.6

zenml - 0.45.5

Published by strickvl 12 months ago

This minor release contains bugfixes and documentation improvements. Notably,
our sqlmodel dependency has been pinned to 0.0.8 which fixes installation
errors following the release of 0.0.9.

What's Changed

Full Changelog: https://github.com/zenml-io/zenml/compare/0.45.4...0.45.5

zenml - 0.45.4

Published by avishniakov 12 months ago

This minor update fixes a database migration bug that you could potentially encounter while upgrading your ZenML version and relates to use of the ExternalArtifact object.
If you are upgrading from <0.45.x version, this is the recommended release.

PROBLEMS?: If you upgraded to ZenML v0.45.2 or v0.45.3 and are experiencing issues with your database, please consider upgrading to v0.45.4 instead.

What's Changed

Full Changelog: https://github.com/zenml-io/zenml/compare/0.45.3...0.45.4

zenml - 0.45.3

Published by strickvl about 1 year ago

This minor update fixes a database migration bug that you could potentially encounter while upgrading your ZenML version and relates to use of the ExternalArtifact object.

PROBLEMS?: If you upgraded to ZenML v0.45.2 and are experiencing issues with your database, please either reach out to us on Slack directly or feel free to use this migration script that will manually fix the issue. (Please do backup your database before using the migration script so as to prevent any data loss!)

This release also includes a bugfix from @cameronraysmith relating to the
resolution of our Helm chart OCI location. Thank you!

What's Changed

New Contributors

Full Changelog: https://github.com/zenml-io/zenml/compare/0.45.2...0.45.3

zenml - 0.45.2

Published by strickvl about 1 year ago

This release replaces 0.45.0 and 0.45.1, and fixes the major migration bugs that were in
that yanked release. Please upgrade directly to 0.45.2 and avoid upgrading to
0.45.0 to avoid unexpected migration issues.

Note that 0.45.0 and 0.45.1 were removed from PyPI due to an issue with the
alembic versions + migration which could affect the database state. This release
fixes that issue.

If you have already upgraded to 0.45.0 please let us know in Slack and we'll happy to assist in rollback and recovery.

This release introduces a major upgrade to ZenML, featuring a new authentication mechanism, performance improvements, the introduction of the model control plane, and internal enhancements.

New Authentication Mechanism (#4303)

Our improved authentication mechanism offers a more secure way of connecting to the ZenML server. It initiates a device flow that prompts you to log in via the browser dashboard:

zenml connect --url <YOUR_SERVER_URL>

This eliminates the need for explicit credential input. The previous method (zenml connect --url <URL> --username <USERNAME> --password <PASSWORD>) remains operational but is less recommended due to security concerns.

Critical This change disrupts existing pipeline schedules. After upgrading, manually cancel and reschedule pipelines using the updated version of ZenML.

For more information, read about the device flow in our documentation.

Performance enhancements (#3207)

Internal API adjustments have reduced the footprint of ZenML API objects by up to 35%. This will particularly benefit users with large step and pipeline configurations. Further reductions will be implemented in our next release.

Model Control Plane debut (#5648)

ZenML now includes a preliminary version of the model control plane, a feature for registering models and their metadata on a single ZenML dashboard view. Future releases will provide more details. To test this early version, follow this example.

Breaking Changes

  • Environment variables ZENML_AUTH_TYPE and ZENML_JWT_SECRET_KEY have been renamed to ZENML_SERVER_AUTH_SCHEME and ZENML_SERVER_JWT_SECRET_KEY, respectively.
  • All ZenML server-issued JWT tokens now include an issuer and an audience. After the server update, current scheduled pipelines become invalidated. Reset your schedules and reconnect all clients to the server to obtain new tokens.
  • UnmaterializedArtifact has been relocated to zenml.artifacts. Change your import statement from from zenml.materializers import UnmaterializedArtifact to from zenml.artifacts.unmaterialized_artifact import UnmaterializedArtifact.

Deprecations

  • zenml.steps.external_artifact.ExternalArtifact has moved to zenml.artifacts.external_artifact.ExternalArtifact.

And the rest:

New Contributors

Full Changelog: https://github.com/zenml-io/zenml/compare/0.44.3...0.45.0

zenml - 0.45.1

Published by htahir1 about 1 year ago

0.45.1

Update: It's one of those days! This release is also yanked due to alembic migration issues. We are working hard to solve it. In the meanwhile please use 0.44.3

This release replaces 0.45.0, and fixes the major migration bugs that were in that yanked release. Please upgrade directly to 0.45.1 and avoid upgrading to 0.45.0 to avoid unexpected migration issues.

What's changed

This release introduces a major upgrade to ZenML, featuring a new authentication mechanism, performance improvements, the introduction of the model control plane, and internal enhancements.

New Authentication Mechanism (#4303)

Our improved authentication mechanism offers a more secure way of connecting to the ZenML server. It initiates a device flow that prompts you to log in via the browser dashboard:

zenml connect --url <YOUR_SERVER_URL>

This eliminates the need for explicit credential input. The previous method (zenml connect --url <URL> --username <USERNAME> --password <PASSWORD>) remains operational but is less recommended due to security concerns.

Critical This change disrupts existing pipeline schedules. After upgrading, manually cancel and reschedule pipelines using the updated version of ZenML.

For more information, read about the device flow in our documentation.

Performance enhancements (#3207)

Internal API adjustments have reduced the footprint of ZenML API objects by up to 35%. This will particularly benefit users with large step and pipeline configurations. Further reductions will be implemented in our next release.

Model Control Plane debut (#5648)

ZenML now includes a preliminary version of the model control plane, a feature for registering models and their metadata on a single ZenML dashboard view. Future releases will provide more details. To test this early version, follow this example.

Breaking Changes

  • Environment variables ZENML_AUTH_TYPE and ZENML_JWT_SECRET_KEY have been renamed to ZENML_SERVER_AUTH_SCHEME and ZENML_SERVER_JWT_SECRET_KEY, respectively.
  • All ZenML server-issued JWT tokens now include an issuer and an audience. After the server update, current scheduled pipelines become invalidated. Reset your schedules and reconnect all clients to the server to obtain new tokens.
  • UnmaterializedArtifact has been relocated to zenml.artifacts. Change your import statement from from zenml.materializers import UnmaterializedArtifact to from zenml.artifacts.unmaterialized_artifact import UnmaterializedArtifact.

Deprecations

  • zenml.steps.external_artifact.ExternalArtifact has moved to zenml.artifacts.external_artifact.ExternalArtifact.

And the rest:

New Contributors

Full Changelog: https://github.com/zenml-io/zenml/compare/0.44.3...0.45.0

zenml - 0.45.0

Published by safoinme about 1 year ago

What change's

This release introduces a major upgrade to ZenML, featuring a new authentication mechanism, performance improvements, the introduction of the model control plane, and internal enhancements.

New Authentication Mechanism (#4303)

Our improved authentication mechanism offers a more secure way of connecting to the ZenML server. It initiates a device flow that prompts you to log in via the browser dashboard:

zenml connect --url <YOUR_SERVER_URL>

This eliminates the need for explicit credential input. The previous method (zenml connect --url <URL> --username <USERNAME> --password <PASSWORD>) remains operational but is less recommended due to security concerns.

Critical This change disrupts existing pipeline schedules. After upgrading, manually cancel and reschedule pipelines using the updated version of ZenML.

For more information, read about the device flow in our documentation.

Performance enhancements (#3207)

Internal API adjustments have reduced the footprint of ZenML API objects by up to 35%. This will particularly benefit users with large step and pipeline configurations. Further reductions will be implemented in our next release.

Model Control Plane debut (#5648)

ZenML now includes a preliminary version of the model control plane, a feature for registering models and their metadata on a single ZenML dashboard view. Future releases will provide more details. To test this early version, follow this example.

Breaking Changes

  • Environment variables ZENML_AUTH_TYPE and ZENML_JWT_SECRET_KEY have been renamed to ZENML_SERVER_AUTH_SCHEME and ZENML_SERVER_JWT_SECRET_KEY, respectively.
  • All ZenML server-issued JWT tokens now include an issuer and an audience. After the server update, current scheduled pipelines become invalidated. Reset your schedules and reconnect all clients to the server to obtain new tokens.
  • UnmaterializedArtifact has been relocated to zenml.artifacts. Change your import statement from from zenml.materializers import UnmaterializedArtifact to from zenml.artifacts.unmaterialized_artifact import UnmaterializedArtifact.

Deprecations

  • zenml.steps.external_artifact.ExternalArtifact has moved to zenml.artifacts.external_artifact.ExternalArtifact.

And the rest:

New Contributors

Full Changelog: https://github.com/zenml-io/zenml/compare/0.44.3...0.45.0

zenml -

Published by safoinme about 1 year ago

This release, introduces SkyPilot, a new VM orchestrator for ZenML that lets users run pipelines on their choice of cloud provider VMs, offering a GPU option without Kubernetes or serverless orchestrators. This release also brings bug fixes and improvements, including a streamlined 'connect' command, interactive configuration for 'zenml stack deploy,' and enhanced documentation covering SageMaker, GCP, and service connectors with MFA.

New Orchestrator: SkyPilot (#1765)

This release introduces a new orchestrator called SkyPilot. SkyPilot is a VM orchestrator
that can be used to run ZenML pipelines on a VM of choice in one of the three supported
cloud providers. It is an excellent choice for users who want to run ZenML pipelines on a GPU
instance, but don't want to use Kubernetes or serverless orchestrators like SageMaker.

Fixes and Improvements

This release fixes several bugs and improves the user experience of the CLI and the
documentation. The most notable changes are:

  • The new connect command that allows connecting all stack components within a stack to a
    service connector with a single command.
  • Adding an interactive flow to the zenml stack deploy command that allows users to
    configure their stack in a guided manner.
  • Add documentation on how to debug the SageMaker orchestrator, how to get started with
    a quick cloud stack on GCP, and documentation on using service connectors with
    enabled MFA.

What's Changed

New Contributors

Full Changelog: https://github.com/zenml-io/zenml/compare/0.44.2...tes

zenml - 0.44.2

Published by fa9r about 1 year ago

This release contains updates for some of the most popular integrations, as well as several bug fixes and documentation improvements.

Minor Default Behavior Changes

  • The default page size for zenml list commands was reduced to 20 (from 50) to speed up the runtime of such commands.
  • Simultaneous connection to local and remote ZenML servers is no longer possible since this caused several unexpected behaviors in the past.

Integration Updates

  • The mlflow integration now supports the newest MLflow version 2.6.0.
  • The evidently integration now supports the latest Evidently version 0.4.4.
  • The SageMaker orchestrator of the aws integration now supports authentication via service connectors.

What's Changed

Full Changelog: https://github.com/zenml-io/zenml/compare/0.44.1...0.44.2

zenml - 0.44.1

Published by safoinme about 1 year ago

This release brings various improvements over the previous version, mainly focusing on using the newly refactored mlstacks package, ZenML's logging module and the changes in our analytics.

IMPORTANT NOTICE

Note: 0.44.0 was removed from Pypi due to an issue with the alembic versions which could affect the database state. A branch occurred in the versions: 0.42.1 -> [0.43.0, e1d66d91a099] -> 0.44.0. This release fixes the issue.
The primary issue arises when deploying version 0.44.0 using a MySQL backend. Although the alembic migration executes all tasks up to 0.44.0, the alembic version represented in the database remains at 0.43.0. This issue persists irrespective of the measures taken, including trying various versions after 0.43.0.
This imbalance leads to failure when running a second replica migration because the database's state is at 0.44.0 while the alembic version remains at 0.43.0. Similarly, attempts to run a second replica or restart the pod fail as the alembic tries to migrate from 0.43.0 to 0.44.0, which is not possible because these changes already exist in the database.
Please note: If you encounter this problem, we recommend you roll back to previous versions and upgrade to 0.43.0. If you still experience difficulties, please join our Slack community at https://zenml.io/slack. We're ready to help you work through this issue.

What's Changed

Full Changelog: https://github.com/zenml-io/zenml/compare/0.43.0...0.44.1

zenml - 0.44.0 (YANKED [SEE RELEASE NOTES])

Published by bcdurak about 1 year ago

IMPORTANT NOTICE

This release has been removed from pypi due to an issue with the alembic versions which could affect the database state. A branch occurred in the versions: 0.42.1 -> [0.43.0, e1d66d91a099] -> 0.44.0.

The primary issue arises when deploying version 0.44.0 using a MySQL backend. Although the alembic migration executes all tasks up to 0.44.0, the alembic version represented in the database remains at 0.43.0. This issue persists irrespective of the measures taken, including trying various versions after 0.43.0.

This imbalance leads to failure when running a second replica migration because the database's state is at 0.44.0 while the alembic version remains at 0.43.0. Similarly, attempts to run a second replica or restart the pod fail as the alembic tries to migrate from 0.43.0 to 0.44.0, which is not possible because these changes already exist in the database.

Please note: If you encounter this problem, we recommend that you rollback to previous versions and then upgrade to 0.43.0. If you still experience difficulties, please join our Slack community at https://zenml.io/slack. We're ready to help you work through this issue.

Older Notes

This release brings various improvements over the previous version, mainly
focusing on the usage of newly refactored mlstacks package, ZenML's logging
module and the changes in our analytics.

What's Changed

Full Changelog: https://github.com/zenml-io/zenml/compare/0.43.0...0.44.0

zenml - 0.43.0

Published by avishniakov about 1 year ago

This release brings limited support for Python 3.11, improves quickstart experience with the fully reworked flow, enhances the user experience while dealing with ZenML docs, offers new extended templates for projects and fixes GCP connector creation issue.

Limited support for Python 3.11

This release adds limited support for Python 3.11.
The following integrations are currently not tested and may behave unexpectedly with Python 3.11:

  • gcp
  • kubeflow
  • tekton

This is because:

  • GCP packages that support Python 3.11 are not compatible with KFP 1
  • Upgrade to KFP 2 is blocked by the fact that Tekton doesn't have any release compatible with KFP 2 yet (https://github.com/zenml-io/zenml/pull/1697)

Breaking Changes

A minor breaking change in CLI for zenml init:

  • previously supported flag --starter
  • new flag --template-with-defaults
  • behavior remains the same - flag is responsible for usage of default settings in the template

What's Changed

New Contributors

Full Changelog: https://github.com/zenml-io/zenml/compare/0.42.0...0.43.0

zenml - 0.42.1

Published by stefannica about 1 year ago

0.42.1

This is a minor release that fixes a couple of minor issues and improves the
quickstart example.

Breaking Changes

Disable Implicit Auth Methods for Service Connectors by Default

The implicit authentication methods supported by cloud Service Connectors method
may constitute a security risk, because they can give users access to the same
cloud resources and services that the ZenML Server itself is allowed to access.

For this reason, the default behavior of ZenML Service Connectors has been
changed to disable implicit authentication methods by default. If you try to
configure any of the AWS, GCP or Azure Service Connectors using the implicit
authentication method, you will now receive an error message.

To enable implicit authentication methods, you have to set the
ZENML_ENABLE_IMPLICIT_AUTH_METHODS environment variable or the ZenML helm
chart enableImplicitAuthMethods configuration option to true.

What's Changed

Full Changelog: https://github.com/zenml-io/zenml/compare/0.42.0...0.42.1

zenml - 0.42.0

Published by fa9r about 1 year ago

This release brings major user experience improvements to how ZenML logs are managed and displayed, removes Python 3.7 support, and fixes the Python 3.10 PyYAML issues caused by the Cython 3.0 release.

Improved Logging UX

The log messages written by ZenML when running pipelines or executing ZenML CLI commands are now more concise and easier to digest and the log message colors were adjusted to be more intuitive. Additionally, all log messages, including custom prints to stdout, now show up as step logs in the dashboard.

Breaking Changes

Python 3.7 Support Dropped

Python 3.7 reached its end of life on on June 27th, 2023. Since then, several MLOps tools have stopped supporting Python 3.7. To prevent dependency issues with our integrations and other open-source packages, ZenML will also no longer support Python 3.7 starting from this release.

Dependency and Integration Version Updates

ZenML now requires PyYAML 6 since older versions are broken under Python 3.10. Subsequently, the following integrations now require a higher package version:

  • Kubeflow now requires kfp==1.8.22
  • Tekton now requires kfk-tekton==1.7.1
  • Evidently now requires evidently==0.2.7 or evidently==0.2.8

What's Changed

New Contributors

Full Changelog: https://github.com/zenml-io/zenml/compare/0.41.0...0.42.0

zenml - 0.41.0

Published by fa9r over 1 year ago

ZenML release 0.41.0 comes with a second round of updates to the pipeline and step interface with major changes in how step outputs are defined, how information about previous runs can be fetched programmatically, and how information about the current run can be obtained.

See this docs page for an overview of all pipeline interface changes introduced since release 0.40.0 and for more information on how to migrate your existing ZenML pipelines to the latest syntax.

Fetching Runs Programmatically (#1635)

The entire syntax of fetching previous runs programmatically was majorly redesigned. While the overall user flow is still almost identical, the new approach does not contain pipeline-versioning-related inconsistencies, has a more intuitive syntax, and is also easier for users to learn since the new syntax uses the ZenML Client and response models natively instead of requiring the zenml.post_execution util functions and corresponding ...View wrapper classes.

Accessing Current Run Information (#1648)

How to fetch information about the current pipeline run from within the run has been majorly redesigned:

  • Instead of being an argument of the step function, the StepContext is now a singleton that can be accessed via the new zenml.get_step_context() function.
  • The StepContext is now decoupled from the StepEnvironment and the StepEnvironment is deprecated.
  • The StepContext now contains the full PipelineRunResponseModel and StepRunResponseModel so all information about the run is accessible, not just the name / id / params.

Defining Step Outputs (#1653)

Instead of using the zenml.steps.Output class to annotate steps with multiple outputs, ZenML can now handle Tuple annotations natively and output names can now be assigned to any step output using typing_extensions.Annotated.

What's Changed

New Contributors

Full Changelog: https://github.com/zenml-io/zenml/compare/0.40.3...0.41.0

zenml - 0.40.3

Published by stefannica over 1 year ago

This is a minor ZenML release that introduces a few new features around Service Connectors and Dashboard improvements:

  • Service Connectors can now also be managed through the ZenML Dashboard

  • the Azure Service Connector is now available in addition to the AWS and GCP ones. It can be used to connect ZenML and Stack Components to Azure cloud infrastructure resources like Azure Blob Storage, Azure Container Registry and Azure Kubernetes Service.

  • added ability to view pipeline run logs in the ZenML Dashboard

  • adds zenml secret export CLI command to export secrets from the ZenML Secret Store to a local file

  • adds the ability to create/update ZenML secrets from JSON/YAML files or command line arguments (courtesy of @bhatt-priyadutt)

In addition to that, this release also contains a couple of bug fixes and improvements, including:

What's Changed

New Contributors

Full Changelog: https://github.com/zenml-io/zenml/compare/0.40.3...0.40.2

zenml - 0.40.2

Published by htahir1 over 1 year ago

Minor release with a few docs changes and preparation of the MLOps Platform Sandbox

What's Changed

Full Changelog: https://github.com/zenml-io/zenml/compare/0.40.1...0.40.2

Package Rankings
Top 2.72% on Pypi.org
Badges
Extracted from project README
PyPi PyPi PyPi Contributors License Star this project Open In Colab Introductory Youtube Video Star this project
Related Projects
ml-workspace

🛠 All-in-one web-based IDE specialized for machine learning and data science.

27 May 2019 3,406
mlreef

The collaboration workspace for Machine Learning

20 Jul 2020 1,442