prefect

When running a server with this version, the client must be the same version. This does not apply to clients connecting to Prefect Cloud.

This release attempted to fix a bug affecting a subset of users on 2.4.3 where block schemas cannot be saved. This release did not address the issue as intended and is fixed in 2.4.5.

When running a server with this version, the client must be the same version. This does not apply to clients connecting to Prefect Cloud.

This release includes several performance enhancements, CLI quality of life improvements, and numerous fixes - full release notes here

This release includes some fixes to bugs with blocks in 2.4.1, see the release notes for details.

Thanks to our users who found and helped fix these issues!

This release includes bug fixes and a lot of enhancements. We're sure you'll be excited about:

• Microsoft Teams notifications
• Improved orchestration for duplicate flow runs
• A CLI command to generate a Kubernetes manifest for agents
• Performance improvements for blocks

Check out the release notes to see all of the changes.

Changes

Enhancements

• Add get_latest to TaskRunView — #6749
• Update agents to allow logs to be disabled with PREFECT__CLOUD__SEND_FLOW_RUN_LOGS — #6833

Fixes

• Fix bug where scheduler_comm.close_rpc() was not awaited in DaskExecutor — #6793
• Fix bug where the Prefect diagnostics logger could duplicate output — #6832

Task library

• Add validation for node type id and instance pool id in Databricks NewCluster — #6853

This release includes updates to deployments for using custom infrastructure blocks as well as many enhancements and bugfixes.

Read the full release notes for details.

Changes

Fixes

• Fix bug where flows with emoji characters pass flow.validate(), but fail to extract from file - #6375
• Fix FlowRunView task run query errors when all task runs in the flwo run are cached - #6572
• Update FlowRunView to avoid retrieval of cached static tasks during latest calls - #6572
• Update FlowRunView to be robust to multithreaded usage - #6572

Task Library

• Add state handler to log to Snowflake - #5671
• Fix default idempotency token for DatabricksSubmitMultitaskRun - #6412
• Fix custom tag type for Databricks NewCluster - #6573
• Fix empty file handling in S3List task - #6028

Contributors

• Dennis Hinnenkamp
• Edmondo Porcu
• Mate Hricz
• Stéphan Taljaard

Releases some bug fixes and minor enhancements. Check out the release notes for more info.

This release includes some important bug fixes and minor enhancements. Check out the release notes for details.

There are some exciting features in this release:

• Add support for deploying flows stored in Docker images — #6574
• Add support for deploying flows stored on GitHub — #6598
• Add file system block for reading directories from GitHub — #6517
• Add a context manager to disable the flow and task run loggers for testing — #6575

The list of changes is way to big to fit here! Check out the full release notes.

Exciting New Features 🎉

• Added automatic detection of static arguments to Task.map in #6513

Fixes

• Updated deployment flow run retry settings with runtime values in #6489
• Updated media queries for flow-run-filter in #6484
• Added empirical_policy to flow run update route in #6486
• Updated flow run policy retry settings to be nullable in #6488
• Disallowed extra attribute initialization on Deployment objects in #6505
• Updated deployment build to raise an informative error if two infrastructure configs are provided in #6504
• Fixed calling async subflows from sync parents in #6514

Fixes an issue where deployment flow run's would fail if they did not specify a value for parameters that have a default value on the flow function. A few minor fixes are included as well, see the release notes for more details.

See the release notes for details about what's new in this release.

Simplified deployments

The deployment experience has been refined to remove extraneous artifacts and make configuration even easier. In particular:

• prefect deployment build no longer generates a manifest.json file. Instead, all of the relevant information is written to the deployment.yaml file.
• Values in the deployment.yaml file are more atomic and explicit
• Local file system blocks are no longer saved automatically
• Infrastructure block values can now be overwritten with the new infra_overrides field

Start custom flow runs from the UI

Now, from the deployment page, in addition to triggering an immediate flow run with default parameter arguments, you can also create a custom run. A custom run enables you to configure the run's parameter arguments, start time, name, and more, all while otherwise using the same deployment configuration. The deployment itself will be unchanged and continue to generate runs on its regular schedule.

Improvements and bug fixes

• Made timeout errors messages on state changes more intuitive
• Added debug level logs for task run rehydration
• Added basic CLI functionality to inspect Blocks; more to come
• Added support for filtering on state name to prefect flow-run ls
• Refined autogenerated database migration output

New Contributors

• @hugolmn made their first contribution in https://github.com/PrefectHQ/prefect/pull/6275

Full Changelog: https://github.com/PrefectHQ/prefect/compare/2.0.3...2.0.4

2.0.3

This release contains a number of bug fixes and documentation improvements.

Introducing prefect-dbt

We've released prefect-dbt - a collection of Prefect integrations for working with dbt in your Prefect flows. This collection has been built as part of a partnership with dbt Labs to ensure that it follows best practices for working with dbt.

Improvements and bug fixes

• Azure storage blocks can use .prefectignore
• Resolved bugs and improved interface in the Orion client.
• Resolved a bug in Azure storage blocks that would cause uploads to get stuck.
• Resolved a bug where calling a flow in a separate thread would raise an exception.
• Resolved issues with loading flows from a deployment.
• Corrected some erroneous type annotations.
• Better handling of database errors during state transition validation.
• Better sanitization of labels for Kubernetes Jobs.
• Fixes --manifest-only flag of prefect deployment build command to ensure that using this flag, the manifest gets generated, but the upload to a storage location is skipped.
• Added support for multiple YAML deployment paths to the prefect deployment apply command.

Enhancements

• Allow custom job name for Kubernetes flow runsto be set in job template - #5819

Task Library

• Adds container parameter to ReadNamespacedPodLogs - #5979
• Add PowershellTask for running commands with Microsoft Powershell - #6035
• Move azure-mgmt-datafactory requirement to 'extras', instead of 'orchestration_extras' - #6018
• Fix default initialization of polling_interval in azure.datafactory.PipelineRun - #6021

Contributors

• Alvaro Duran Tovar
• Chinmay Bhat

2.0.2

This release implements a number of improvements and bug fixes in response to continued engagement by members of our community. Thanks, as always, to all who submitted ideas on how to make Prefect 2 even better.

Introducing .prefectignore files

.prefectignore files allow users to omit certain files or directories from their deployments. Similar to other .ignore files, the syntax supports pattern matching, so an entry of *.pyc will ensure all .pyc files are ignored by the deployment call when uploading to remote storage. Prefect provides a default .prefectignore file, but users can customize it to their needs.

Improvements and bug fixes

• Users can now submit bug reports and feature enhancements using our issue templates.
• Block deletion is now more performant.
• Inconsistencies in UI button copy have been removed.
• Error messaging is clearer in the deployment build CLI command.
• Resolved timeout errors that occurred when using async task functions inside synchronous flows.

The response to Prefect 2 has been overwhelming in the best way possible. Thank you to the many community members who tried it out and gave us feedback! Thanks in particular to the students at this week's Prefect Associate Certification Course (PACC) in San Jose for their thoughtful recommendations. This release is a compilation of enhancements and fixes that make for a more resilient, performant, and refined Prefect experience.

Improvements and bug fixes

• Schedules set via the API or UI are now preserved when building deployments from the CLI
• JSON types are now coerced to none, following Javascript convention and supporting standards compatibility
• The prefect deployment execute command has been removed to avoid confusion between running a flow locally from a Python script and running it by an agent using prefect deployment run
• This repository now includes templates for pull requests and issues to make bug reports and community contributions easier
• The scheduler and flow-run-notifications LoopServices have been made more resilient
• Log inserts have been made more performant through smaller log batches
• Local file system blocks created from the UI now point to the right base_path
• Support for unmapped values to Task.map has been added as requested by Club42 members
• The deployment build command now supports an optional output flag to customize the name of the deployment.yaml file, to better support projects with multiple flows

We're thrilled to announce that, with this release, Prefect 2.0 has exited its public beta! Hopefully, this release comes as no surprise. It is the culmination of nearly a year of building in public and incorporating your feedback. Prefect 2.0 is now the default version of the open source prefect framework provided upon installation. We will continue enhancing Prefect 2.0 rapidly, but future breaking changes will be less frequent and more notice will be provided.

Prefect 2.0 documentation is now hosted at docs.prefect.io. Prefect 1.0 documentation is now hosted at docs-v1.prefect.io.

Upgrading from Prefect 1.0

Flows written with Prefect 1.0 will require modifications to run with Prefect 2.0. If you're using Prefect 1.0, please see our guidance on Discourse for explicitly pinning your Prefect version in your package manager and Docker, so that you can make the transition to Prefect 2.0 when the time is right for you. See our migration page to learn more about upgrading.

Upgrading from earlier versions of Prefect 2.0

We have shipped a lot of breaking changes to Prefect 2.0 over the past week. Most importantly, recent changes to deployments required that schedules for all previously created deployments be turned off. You can learn more about the changes via the deployments concept documentation, the tutorial, or the discourse guide.

Changelog

Fixes

• Fix bug where S3Result could fail to deserialize if pickled with mismatched Prefect versions - #5978

Task library

• Fix sftp-task imports - #5971
• Fix definition of Databricks GitSource at task initialization - #5974
• Add container parameter to ReadNamespacedPodLogs - #5979

Contributors

• Bral Bral
• Edmondo Porcu
• Alvaro Duran Tovar