prefect

Prefect is a workflow orchestration tool empowering developers to build, observe, and react to data pipelines

APACHE-2.0 License

Downloads
8.4M
Stars
14.5K
Committers
148
View Code on GitHub Visit Website
Ecosystems: Python

Bot releases are hidden (Show)

prefect - Release 2.16.2

Published by zzstoatzz 8 months ago

โ›‘๏ธ Fixing a logging bug introduced by 2.16.1

In 2.16.1, a regression related to logging non-str objects (when a PREFECT_API_KEY was set) was introduced. The fix is included in 2.16.2.

See https://github.com/PrefectHQ/prefect/pull/12151 for implementation details.

๐ŸŽฏ Use a module path entrypoint when using .serve or .deploy 

from prefect_dbt.cloud.jobs import trigger_dbt_cloud_job_run_and_wait_for_completion

from prefect.deployments import EntrypointType

if __name__ == "__main__":
    trigger_dbt_cloud_job_run_and_wait_for_completion.deploy(
        name="demo-deploy-from-module",
        work_pool_name="olympic",
        entrypoint_type=EntrypointType.MODULE_PATH,
        job_variables={"env": { "EXTRA_PIP_PACKAGES": "prefect-dbt" } }
    )

See https://github.com/PrefectHQ/prefect/pull/12134 for implementation details.

Full Changelog: https://github.com/PrefectHQ/prefect/compare/2.16.1...2.16.2

See the release notes for more!

prefect - Release 2.16.1

Published by zzstoatzz 8 months ago

Enhanced multiple schedule support ๐Ÿ“…

prefect.yaml now supports specifying multiple schedules via the schedules key.

This allows you to define multiple schedules for a single deployment, and each schedule can have its own cron, interval, or rrule configuration:

 ...
 schedules:
    - cron: "0 0 * * *"
      active: false
    - interval: 3600
      active: true
    - rrule: "FREQ=YEARLY"
      active: true

In addition you can specify multiple schedules via arguments to prefect deploy:

prefect deploy ... --cron '4 * * * *' --cron '1 * * * *' --rrule 'FREQ=DAILY'

We've also added support for multiple schedules to flow.serve, flow.deploy and prefect.runner.serve. You can provide multiple schedules by passing a list to the cron, interval, or rrule arguments:

import datetime
import random

from prefect import flow


@flow
def trees():
    tree = random.choice(["๐ŸŒณ", "๐ŸŒด", "๐ŸŒฒ", "๐ŸŒต"])
    print(f"Here's a happy little tree: {tree}")

if __name__ == "__main__":
    trees.serve(
        name="trees",
        interval=[3600, 7200, 14400],
    )

This will create a deployment with three schedules, one that runs every hour, one that runs every two hours, and one that runs every four hours. For more advanced cases, use the schedules argument.

    trees.serve(
        name="trees",
        schedules=[
            IntervalSchedule(interval=datetime.timedelta(minutes=30)),
            {"schedule": RRuleSchedule(rrule="FREQ=YEARLY"), "active": True},
            MinimalDeploymentSchedule(schedule=CronSchedule(cron="0 0 * * *"), active=False),
        ]
    )

Dive into these new scheduling capabilities today and streamline your workflows like never before.

For implementation details, see the following pull request:
- https://github.com/PrefectHQ/prefect/pull/12107

New Contributors ๐ŸŒŸ

  • Shoutout to @jrbourbeau for their first contribution!

See the release notes for more details!

prefect - Release 2.16.0

Published by abrookins 8 months ago

๐Ÿ•ข Deployments now support multiple schedules ๐Ÿ•

With todayโ€™s release, weโ€™re excited to roll out initial support for using multiple schedules with Deployments! You can now use multiple schedules in the following ways:

  • Specifying schedules in a Deployment YAML file
  • Creating Python-based Deployments with the Deployment class
  • New CLI commands: prefect deployment schedule <create, delete, pause, resume, ls, clear>
  • New UI components aware of multiple schedules

Coming soon, weโ€™ll round out support for multiple schedules in other areas, such as:

  • When running a flow with flow.serve() and flow.deploy()
  • When using prefect deploy

The easiest way to get started with multiple schedules is to try out the new CLI commands:

$ prefect deployment schedule ls happy-flow/my-deployment
                       Deployment Schedules
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”“
โ”ƒ ID                                   โ”ƒ Schedule        โ”ƒ Active โ”ƒ
โ”กโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”ฉ
โ”‚ c7d3ddc4-9a5a-4dec-bd59-eed282ae55d5 โ”‚ cron: 0 0 1 * 1 โ”‚ True   โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

$ prefect deployment schedule create happy-flow/my-deployment --interval 60
Created deployment schedule!

$ prefect deployment schedule ls happy-flow/my-deployment
                         Deployment Schedules
โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”“
โ”ƒ ID                                   โ”ƒ Schedule           โ”ƒ Active โ”ƒ
โ”กโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”ฉ
โ”‚ 3638ed58-cab2-4462-a680-2f92fcf6c797 โ”‚ interval: 0:01:00s โ”‚ True   โ”‚
โ”‚ c7d3ddc4-9a5a-4dec-bd59-eed282ae55d5 โ”‚ cron: 0 0 1 * 1    โ”‚ True   โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

Enhancements

Fixes

Documentation

Experimental

Contributors

  • @NodeJSmith

Full release notes: https://github.com/PrefectHQ/prefect/blob/main/RELEASE-NOTES.md

prefect - 2.15.0

Published by desertaxle 8 months ago

๐Ÿ”ง Task runs now execute on the main thread

We are excited to announce that task runs are now executed on the main thread!

When feasible, task runs are now executed on the main thread instead of a worker thread. Previously, all task runs were run in a new worker thread. This allows objects to be passed to and from tasks without worrying about thread safety unless you have opted into concurrency. For example, an HTTP client or database connection can be shared between a flow and its tasks now (unless synchronous concurrency is used). Some asynchronous and sequential use cases may see performance improvements.

Consider the following example:

import sqlite3
from prefect import flow, task

db = sqlite3.connect("threads.db")

try:
    db.execute("CREATE TABLE fellowship(name)")
except sqlite3.OperationalError:
    pass
else:
    db.commit()

db.execute("DELETE FROM fellowship")
db.commit()

cur = db.cursor()


@task
def my_task(name: str):
    global db, cur

    cur.execute('INSERT INTO fellowship VALUES (?)', (name,))

    db.commit()


@flow
def my_flow():
    global db, cur

    for name in ["Frodo", "Gandalf", "Gimli", "Aragorn", "Legolas", "Boromir", "Samwise", "Pippin", "Merry"]:
        my_task(name)

    print(cur.execute("SELECT * FROM fellowship").fetchall())

    db.close()


if __name__ == "__main__":
    my_flow()

In previous versions of Prefect, running this example would result in an error like this:

sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 7977619456 and this is thread id 6243151872.

But now, with task runs executing on the main thread, this example will run without error! We're excited this change makes Prefect even more intuitive and flexible!

See the following pull request for implementation details:
- https://github.com/PrefectHQ/prefect/pull/11930

๐Ÿ”ญ Monitor deployment runs triggered via the CLI

You can monitor the status of a flow run created from a deployment via the CLI. This is useful for observing a flow run's progress without navigating to the UI.

To monitor a flow run started from a deployment, use the --watch option with prefect deployment run:

prefect deployment run --watch <slugified-flow-name>/<slugified-deployment-name>

See the following pull request for implementation details:
- https://github.com/PrefectHQ/prefect/pull/11702

Enhancements

Fixes

Experimental

Documentation

Contributors

  • @zanieb

All changes: https://github.com/PrefectHQ/prefect/compare/2.14.21...2.15.0

prefect - 2.14.21

Published by zangell44 8 months ago

Release 2.14.21

Introducing work queue status

We're excited to unveil the new status indicators for work queues in Prefect's UI, enhancing your ability to oversee and control flow run execution within our hybrid work pools.

Work queues will now display one of three distinct statuses:

  • Ready - one or more online workers are actively polling the work queue
  • Not Ready - no online workers are polling the work queue, signaling a need for intervention
  • Paused - the work queue is intentionally paused, preventing execution

With the introduction of work queue status, you'll notice the absence of deprecated work queue health indicators in the UI.

See the documentation on work queue status for more information.

For now, this is an experimental feature, and can be enabled by running:

prefect config set PREFECT_EXPERIMENTAL_ENABLE_WORK_QUEUE_STATUS=True

See the following pull request for implementation details:
- https://github.com/PrefectHQ/prefect/pull/11829

Fixes

Documentation

Experimental

All changes: https://github.com/PrefectHQ/prefect/compare/2.14.20...2.14.21

prefect - Release 2.14.20

Published by collincchoy 9 months ago

Hotfix

All changes: https://github.com/PrefectHQ/prefect/compare/2.14.19...2.14.20

prefect - 2.14.19

Published by bunchesofdonald 9 months ago

Dynamic descriptions for paused and suspended flow runs

You can now include dynamic, markdown-formatted descriptions when pausing or suspending a flow run for human input. This description will be shown in the Prefect UI alongside the form when a user is resuming the flow run, enabling developers to give context and instructions to users when they need to provide input.

from datetime import datetime
from prefect import flow, pause_flow_run, get_run_logger
from prefect.input import RunInput

class UserInput(RunInput):
    name: str
    age: int

@flow
async def greet_user():
    logger = get_run_logger()
    current_date = datetime.now().strftime("%B %d, %Y")

    description_md = f"""
**Welcome to the User Greeting Flow!**
Today's Date: {current_date}

Please enter your details below:
- **Name**: What should we call you?
- **Age**: Just a number, nothing more.
"""

    user_input = await pause_flow_run(
        wait_for_input=UserInput.with_initial_data(
            description=description_md, name="anonymous"
        )
    )

    if user_input.name == "anonymous":
        logger.info("Hello, stranger!")
    else:
        logger.info(f"Hello, {user_input.name}!")

See the following PR for implementation details:

Enhancements

Fixes

Documentation

Experimental

New Contributors

prefect - 2.14.18

Published by zzstoatzz 9 months ago

Release 2.14.18

This release includes:

Fixes

Documentation

New Contributors

See the release notes for details!

prefect - Release 2.14.17

Published by desertaxle 9 months ago

๐Ÿงช Experimental: Non-blocking submission of flow runs to the Runner web server

You can now submit runs of served flows without blocking the main thread, from inside or outside a flow run. If submitting flows from inside a parent flow, these submitted runs will be tracked as subflows of the parent flow run.

In order to use this feature, you must:

  • enable the experimental Runner webserver endpoints via 
    prefect config set PREFECT_EXPERIMENTAL_ENABLE_EXTRA_RUNNER_ENDPOINTS=True
  • ensure the Runner web server is enabled, either by:
    • passing webserver=True to your serve call
    • enabling the webserver via
    prefect config set PREFECT_RUNNER_SERVER_ENABLE=True

You can then submit any flow available in the import space of the served flow, and you can submit multiple runs at once. If submitting flows from a parent flow, you may optionally block the parent flow run from completing until all submitted runs are complete with wait_for_submitted_runs().

import time

from pydantic import BaseModel

from prefect import flow, serve, task
from prefect.runner import submit_to_runner, wait_for_submitted_runs


class Foo(BaseModel):
    bar: str
    baz: int


class ParentFoo(BaseModel):
    foo: Foo
    x: int = 42

@task
def noop():
    pass

@flow(log_prints=True)
async def child(foo: Foo = Foo(bar="hello", baz=42)):
    print(f"received {foo.bar} and {foo.baz}")
    print("going to sleep")
    noop()
    time.sleep(20)


@task
def foo():
    time.sleep(2)

@flow(log_prints=True)
def parent(parent_foo: ParentFoo = ParentFoo(foo=Foo(bar="hello", baz=42))):
    print(f"I'm a parent and I received {parent_foo=}")

    submit_to_runner(
        child, [{"foo": Foo(bar="hello", baz=i)} for i in range(9)]
    )
    
    foo.submit()
    
    wait_for_submitted_runs() # optionally block until all submitted runs are complete
    

if __name__ == "__main__":
    # either enable the webserver via `webserver=True` or via
    # `prefect config set PREFECT_RUNNER_SERVER_ENABLE=True`
    serve(parent.to_deployment(__file__), limit=10, webserver=True)

This feature is experimental and subject to change. Please try it out and let us know what you think!

See the PR for implementation details.

๐Ÿ‘ Enhancements

๐Ÿงฐ Fixes

โœ๏ธ Documentation

๐Ÿง‘โ€๐Ÿคโ€๐Ÿง‘ Contributors

  • @sgbaird

All changes: https://github.com/PrefectHQ/prefect/compare/2.14.16...2.14.17

prefect - Release 2.14.16

Published by urimandujano 9 months ago

๐ŸŽ‰ Support for access block fields in prefect.yaml templating

You can now access fields on blocks used in your prefect.yaml files. This enables you to use values stored in blocks to provide dynamic configuration for attributes like your work_pool_name and job_variables.

Here's what it looks like in action:

deployments:
- name: test
  version: 0.1
  tags: []
  description: "Example flow"
  schedule: {}
  entrypoint: "flow.py:example_flow"
  parameters: {}
  work_pool:
    name: "{{ prefect.blocks.json.default-config.value.work_pool }}"
    work_queue: "{{ prefect.blocks.json.default-config.value.work_queue }}"

In the above example, we use fields from a JSON block to configure which work pool and queue we deploy our flow to. We can update where our flow is deployed to by updating the referenced block without needing to change our prefect.yaml at all!

Many thanks to @bjarneschroeder for contributing this functionality! Check out this PR for implementation details: https://github.com/PrefectHQ/prefect/pull/10938

๐Ÿ‘๐Ÿผ Enhancements

โ›‘๏ธ Fixes

โœ๏ธ Documentation

๐Ÿฅ‡ New Contributors

prefect - Release 2.14.15

Published by znicholasbrown 9 months ago

What's Changed

Hotfix

Full Changelog: https://github.com/PrefectHQ/prefect/compare/2.14.14...2.14.15

prefect - Release 2.14.14

Published by collincchoy 9 months ago

โœจ Support for custom prefect.yaml deployment configuration files

You can now specify a prefect.yaml deployment configuration file while running prefect deploy by using the
--prefect-file command line argument. This means that your configuration files can be in any directory
and can follow your own naming conventions. Using this feature provides more flexibility in defining
and managing your deployments.

โœ… Toggle Deployment Schedule Status via prefect.yaml

You can now toggle your deployment schedules between active and inactive in your prefect.yaml configuration file. This enables you to create deployments with initially inactive schedules, allowing for thorough testing or staged rollouts!

๐Ÿ Support for Python 3.12

You can now install prefect using Python 3.12! This support is experimental and will be hardened in future releases.

๐Ÿง‘โ€๐Ÿคโ€๐Ÿง‘ Contributors

A big thanks to @brett-koonce, @jitvimol, @oz-elhassid, @Zyntogz, and @Andrew-S-Rosen for making their first contributions in this release.

See the release notes for more details.

prefect - Release 2.14.13

Published by serinamarie 10 months ago

This release includes a few very exciting enhancements! ๐Ÿช„

Access default work pool configurations in an air-gapped environment ๐ŸŒ

Those who run Prefect server in an environment where arbitrary outbound internet traffic is not allowed were previously unable to retrieve up-to-date default work pool configurations (via the UI or otherwise). You can now access the worker metadata needed to access the corresponding work pool configurations in your server even in such an air-gapped environment. Upon each release of prefect, the most recent version of this worker metadata will be embedded in the prefect package so that it can be used as a fallback if the outbound call to retrieve the real-time metadata fails.

Introducing conditional task retries for enhanced workflow control ๐Ÿ”

In this release, we're excited to introduce the ability to conditionally retry tasks by passing in an argument to retry_condition_fn in your task decorator, enabling more nuanced and flexible retry mechanisms. This adds a significant level of control and efficiency, particularly in handling complex or unpredictable task outcomes. For more information on usage, check out our docs!

Other enhancements and fixes ๐Ÿ› ๏ธ

We've also added the ability to open your current Prefect Cloud workspace in the browser from the CLI, display work queue status details via CLI, and much more! See the release notes for details.

Contributors ๐Ÿง‘โ€๐Ÿคโ€๐Ÿง‘

A big thanks to @dominictarro and @ConstantinoSchillebeeckx for contributing enhancements to this release, as well as to @yifanmai, who made their first contribution.

prefect - Release 2.14.12

Published by serinamarie 10 months ago

This release brings some exciting enhancements and fixes!

Enhanced UI Customization ๐Ÿ“†

  • Now featuring relative time spans like "Past 7 days" for better insight into current and upcoming flow runs across the UI.
  • Flow Runs Page: Enhanced with customizable date and time filters. Save custom filters with specific date ranges for tailored views
  • Flows Page: Updated with similar date and time filtering capabilities

See a demonstration here!
short loom demo

Enhancements and Fixes ๐Ÿ› ๏ธ

  • Fixed environment variable parsing during deployment and prompt and role assignment in the ContainerInstanceProvisioner
  • Enhanced dashboard header to adapt to various tag and date input sizes
  • Addressed an error encountered when deploying flows with options

See the release notes for details!

prefect - Release 2.14.11

Published by desertaxle 10 months ago

Release 2.14.11

๐Ÿช„ โœจ Customize resource names when provisioning infrastructure for push work pools

In the past few releases, we've added the ability to provision infrastructure for push work pools via the CLI. This release adds the ability to customize the name of the resources created in your cloud environment when provisioning infrastructure for push work pools so you can follow your organization's naming conventions.

To customize your resource names when provisioning infrastructure for a push work pool, follow the interactive prompts:

? Proceed with infrastructure provisioning with default resource names? [Use arrows to move; enter to select]
โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”“
โ”ƒ    โ”ƒ Options:                                                                  โ”ƒ
โ”กโ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ฉ
โ”‚    โ”‚ Yes, proceed with infrastructure provisioning with default resource names โ”‚
โ”‚ >  โ”‚ Customize resource names                                                  โ”‚
โ”‚    โ”‚ Do not proceed with infrastructure provisioning                           โ”‚
โ””โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
? Please enter a name for the resource group (prefect-aci-push-pool-rg): new-rg
? Please enter a name for the app registration (prefect-aci-push-pool-app): new-app
? Please enter a prefix for the Azure Container Registry (prefect): newregistry
? Please enter a name for the identity (used for ACR access) (prefect-acr-identity): new-identity
? Please enter a name for the ACI credentials block (new-work-pool-push-pool-credentials): new-aci-block
โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
โ”‚ Provisioning infrastructure for your work pool new-work-pool will require:                โ”‚
โ”‚                                                                                           โ”‚
โ”‚     Updates in subscription: Azure subscription 1                                         โ”‚
โ”‚                                                                                           โ”‚
โ”‚         - Create a resource group in location: eastus                                     โ”‚
โ”‚         - Create an app registration in Azure AD: new-app                                 โ”‚
โ”‚         - Create/use a service principal for app registration                             โ”‚
โ”‚         - Generate a secret for app registration                                          โ”‚
โ”‚         - Create an Azure Container Registry with prefix newregistry                      โ”‚
โ”‚         - Create an identity new-identity to allow access to the created registry         โ”‚
โ”‚         - Assign Contributor role to service account                                      โ”‚
โ”‚         - Create an ACR registry for image hosting                                        โ”‚
โ”‚         - Create an identity for Azure Container Instance to allow access to the registry โ”‚
โ”‚                                                                                           โ”‚
โ”‚     Updates in Prefect workspace                                                          โ”‚
โ”‚                                                                                           โ”‚
โ”‚         - Create Azure Container Instance credentials block: new-aci-block                โ”‚
โ”‚                                                                                           โ”‚
โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ
Proceed with infrastructure provisioning? [y/n]: y
Creating resource group
Resource group 'new-rg' created successfully
Creating app registration
App registration 'new-app' created successfully
Generating secret for app registration
Secret generated for app registration with client ID '03923189-3151-4acd-8d59-76483752cd39'
Creating ACI credentials block
ACI credentials block 'new-aci-block' created in Prefect Cloud
Assigning Contributor role to service account
Service principal created for app ID '25329389-3151-4acd-8d59-71835252cd39'
Contributor role assigned to service principal with object ID '483h4c85-4a8f-4fdb-0394-bd0f0b1202d0'
Creating Azure Container Registry
Registry created
Logged into registry newregistry1702538242q2z2.azurecr.io
Creating identity
Identity 'new-identity' created
Provisioning infrastructure. โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” 100% 0:00:00
Your default Docker build namespace has been set to 'newregistry1702538242q2z2.azurecr.io'.
Use any image name to build and push to this registry by default:

โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ example_deploy_script.py โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
โ”‚ from prefect import flow                                                       โ”‚
โ”‚ from prefect.deployments import DeploymentImage                                โ”‚
โ”‚                                                                                โ”‚
โ”‚                                                                                โ”‚
โ”‚ @flow(log_prints=True)                                                         โ”‚
โ”‚ def my_flow(name: str = "world"):                                              โ”‚
โ”‚     print(f"Hello {name}! I'm a flow running on an Azure Container Instance!") โ”‚
โ”‚                                                                                โ”‚
โ”‚                                                                                โ”‚
โ”‚ if __name__ == "__main__":                                                     โ”‚
โ”‚     my_flow.deploy(                                                            โ”‚
โ”‚         name="my-deployment",                                                  โ”‚
โ”‚         work_pool_name="my-work-pool",                                         โ”‚
โ”‚         image=DeploymentImage(                                                 โ”‚
โ”‚             name="my-image:latest",                                            โ”‚
โ”‚             platform="linux/amd64",                                            โ”‚
โ”‚         )                                                                      โ”‚
โ”‚     )                                                                          โ”‚
โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ
Infrastructure successfully provisioned for 'new-work-pool' work pool!
Created work pool 'new-work-pool'!

Using a push work pool with automatic infrastructure provisioning is a great way to get started with a production-level Prefect set up in minutes! Check out our push work pool guide for step-by-step instructions on how to get started!

See the following pull requests for implementation details:

๐Ÿ“† ๐Ÿ• An updated date time input on the workspace dashboard

We've added a new date and time filter to the workspace dashboard that gives greater control over the dashboard. You can now filter by days, hours, and even minutes. You can also specify a specific date and time range to filter by. You can also go backwards and forwards in time using that time window, for example, you can scroll through by hour.

See it in action!
Demo of updated time input in the Prefect UI

See the following pull requests for implementation details:

๐Ÿ’ฅ Enhancements

๐Ÿงฐ Fixes

๐Ÿงช Experimental

๐Ÿง  Documentation

prefect - Release 2.14.10

Published by urimandujano 11 months ago

๐Ÿช„ โœจ Azure Container Instance push pool infrastructure provisioning via the CLI

We're introducing an enhancement to the Azure Container Instance push pool experience. You can now conveniently provision necessary Azure infrastructure with the --provision-infra flag during work pool creation, automating the provisioning of various Azure resources essential for ACI push pools, including resource groups, app registrations, service accounts, and more.

To provision Azure resources when creating an ACI push pool:

โฏ prefect work-pool create my-work-pool --provision-infra --type azure-container-instance:push
? Please select which Azure subscription to use: [Use arrows to move; enter to select]
โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”“
โ”ƒ    โ”ƒ Name                 โ”ƒ Subscription ID                      โ”ƒ
โ”กโ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ฉ
โ”‚    โ”‚ Engineering          โ”‚ 123                                  โ”‚
โ”‚ >  โ”‚ Azure subscription 1 โ”‚ 234                                  โ”‚
โ””โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
โ”‚ Provisioning infrastructure for your work pool my-work-pool will require:             โ”‚
โ”‚                                                                                       โ”‚
โ”‚     Updates in subscription Azure subscription 1                                      โ”‚
โ”‚                                                                                       โ”‚
โ”‚         - Create a resource group in location eastus                                  โ”‚
โ”‚         - Create an app registration in Azure AD                                      โ”‚
โ”‚         - Create a service principal for app registration                             โ”‚
โ”‚         - Generate a secret for app registration                                      โ”‚
โ”‚         - Assign Contributor role to service account                                  โ”‚
โ”‚         - Create Azure Container Instance                                             โ”‚
โ”‚                                                                                       โ”‚
โ”‚     Updates in Prefect workspace                                                      โ”‚
โ”‚                                                                                       โ”‚
โ”‚         - Create Azure Container Instance credentials block aci-push-pool-credentials โ”‚
โ”‚                                                                                       โ”‚
โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ
Proceed with infrastructure provisioning? [y/n]: y
Creating resource group
Provisioning infrastructure... โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”   0% -:--:--Resource group 'prefect-aci-push-pool-rg' created in location 'eastus'
Creating app registration
Provisioning infrastructure... โ”โ”โ”โ”โ”โ”โ”โ”โ•บโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”  20% -:--:--App registration 'prefect-aci-push-pool-app' created successfully
Generating secret for app registration
Provisioning infrastructure... โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•บโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”  40% 0:00:06Secret generated for app registration with client ID 'abc'
ACI credentials block 'aci-push-pool-credentials' created
Assigning Contributor role to service account...
Provisioning infrastructure... โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•บโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”  60% 0:00:06Contributor role assigned to service principal with object ID 'xyz'
Creating Azure Container Instance
Provisioning infrastructure... โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•บโ”โ”โ”โ”โ”โ”โ”  80% 0:00:04Container instance 'prefect-acipool-container' created successfully
Creating Azure Container Instance credentials block
Provisioning infrastructure... โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” 100% 0:00:00
Infrastructure successfully provisioned for 'my-work-pool' work pool!
Created work pool 'my-work-pool'!

This marks a step forward in Prefect's Azure capabilities, offering you an efficient and streamlined process for leveraging Azure Container Instances to execute their workflows.

See the following pull request for implementation details:
โ€” https://github.com/PrefectHQ/prefect/pull/11275

๐Ÿช„ ๐ŸŽฉ ๐Ÿฐ Introducing the provision-infra sub-command for enhanced push work pool management

This enhancement allows you to directly provision infrastructure for existing push work pools. Rather than recreating a work pool, you can provision necessary infrastructure and
update the existing work pool base job template with the following command:

โฏ prefect work-pool provision-infra my-work-pool
? Please select which Azure subscription to use: [Use arrows to move; enter to select]
โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”“
โ”ƒ    โ”ƒ Name                 โ”ƒ Subscription ID                      โ”ƒ
โ”กโ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ฉ
โ”‚    โ”‚ Engineering          โ”‚ 13d                                  โ”‚
โ”‚ >  โ”‚ Azure subscription 1 โ”‚ 6h4                                  โ”‚
โ””โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
โ”‚ Provisioning infrastructure for your work pool my-work-pool will require:                                      โ”‚
โ”‚                                                                                                                โ”‚
โ”‚     Updates in subscription Azure subscription 1                                                               โ”‚
โ”‚                                                                                                                โ”‚
โ”‚         - Create a resource group in location eastus                                                           โ”‚
โ”‚         - Create an app registration in Azure AD prefect-aci-push-pool-app                                     โ”‚
โ”‚         - Create/use a service principal for app registration                                                  โ”‚
โ”‚         - Generate a secret for app registration                                                               โ”‚
โ”‚         - Assign Contributor role to service account                                                           โ”‚
โ”‚         - Create Azure Container Instance 'aci-push-pool-container' in resource group prefect-aci-push-pool-rg โ”‚
โ”‚                                                                                                                โ”‚
โ”‚     Updates in Prefect workspace                                                                               โ”‚
โ”‚                                                                                                                โ”‚
โ”‚         - Create Azure Container Instance credentials block aci-push-pool-credentials                          โ”‚
โ”‚                                                                                                                โ”‚
โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ
Proceed with infrastructure provisioning? [y/n]: y
...

This PR bolsters support for efficient work pool management across diverse cloud environments, delivering a tool for seamless infrastructure setup.

See the following pull request for implementation details:

๐Ÿ’ฅ Enhancements

๐Ÿงฐ Fixes

๐Ÿง  Documentation

prefect - Release 2.14.9

Published by bunchesofdonald 11 months ago

๐Ÿช„ Automatic infrastructure provisioning for ECS push work pools

Following the introduction of automatic project configuration for Cloud Run push pools last week, we've added the ability to automatically provision infrastructure in your AWS account and set up your Prefect workspace to support a new ECS push pool!

You can create a new ECS push work pool and provision infrastructure in your AWS account with the following command:

prefect work-pool create --type ecs:push --provision-infra my-pool

Using the --provision-infra flag will automatically set up your default AWS account to be ready to execute flows via ECS tasks:

โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
โ”‚ Provisioning infrastructure for your work pool my-work-pool will require:                                         โ”‚
โ”‚                                                                                                                   โ”‚
โ”‚          - Creating an IAM user for managing ECS tasks: prefect-ecs-user                                          โ”‚
โ”‚          - Creating and attaching an IAM policy for managing ECS tasks: prefect-ecs-policy                        โ”‚
โ”‚          - Storing generated AWS credentials in a block                                                           โ”‚
โ”‚          - Creating an ECS cluster for running Prefect flows: prefect-ecs-cluster                                 โ”‚
โ”‚          - Creating a VPC with CIDR 172.31.0.0/16 for running ECS tasks: prefect-ecs-vpc                          โ”‚
โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ
Proceed with infrastructure provisioning? [y/n]: y
Provisioning IAM user
Creating IAM policy
Generating AWS credentials
Creating AWS credentials block
Provisioning ECS cluster
Provisioning VPC
Creating internet gateway
Setting up subnets
Setting up security group
Provisioning Infrastructure โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” 100% 0:00:00
Infrastructure successfully provisioned!
Created work pool 'my-pool'!

If you have yet to try using an ECS push pool, now is a great time!

If you use Azure, don't fret; we will add support for Azure Container Instances push work pools in a future release!

See the following pull request for implementation details:
โ€” https://github.com/PrefectHQ/prefect/pull/11267

๐Ÿš€ Enhancements

๐Ÿ› ๏ธ Fixes

prefect - Release 2.14.8

Published by zhen0 11 months ago

This release is a follow-up to 2.14.7 which never made it to PyPI because of an issue with our Github workflow.

Documentation

All changes: https://github.com/PrefectHQ/prefect/compare/2.14.7...2.14.8

prefect - Release 2.14.7

Published by zhen0 11 months ago

This release fixes a bug introduced in 2.14.6 where deployments with default Docker image builds looked for images tagged v2.14.6 instead of 2.14.6. Users of 2.14.6 should upgrade if planning to create deployments with an image other than a custom image.

๐ŸŒŸ Enhancements

๐Ÿ”ง Fixes

๐Ÿ“š Documentation

๐Ÿง‘โ€๐Ÿคโ€๐Ÿง‘ New Contributors

For more details see the release notes.

prefect - 2.14.6

Published by zzstoatzz 11 months ago

This release includes:

๐Ÿ”œ View the next run for a deployment at a glance

  • View the next run for a deployment in the Runs tab of the Deployments page in the Prefect UI! Upcoming runs are now in a dedicated tab, making the most relevant running and completed flow runs more apparent.

๐Ÿ”ง Automatic project configuration for Cloud Run push work pools

  • We've enhanced the prefect work-pool create CLI command to automatically configure your GCP project and set up your Prefect workspace to use a new Cloud Run push pool immediately

๐Ÿ“š Updated Guides

  • Instructions for turning off the flow run logger to the unit testing guide

๐Ÿง‘โ€๐Ÿคโ€๐Ÿง‘ Contributors

  • A big thanks to @ConstantinoSchillebeeckx for their continued support and contributions.

See the the release notes for more details!

Package Rankings
Top 0.45% on Pypi.org
Top 4.06% on Proxy.golang.org
Top 4.91% on Conda-forge.org
Related Projects
prefect-tutorial

Python Prefect tutorial

01 Jan 2022 9
devops-resources

DevOps resources - Linux, Jenkins, AWS, SRE, Prometheus, Docker, Python, Ansible, Git, Kubernetes...

07 Jan 2019 8,538
fastapi_cloudrun_pubsub

Python API, queues and workers on GCP: FastAPI + CloudRun + PubSub

22 Aug 2023 3
echodataflow

Orchestrated sonar data processing workflow

21 Jan 2022 4
runhouse

The fastest way to iterate and deploy AI workloads on your own infra. Unobtrusive, debuggable, Py...

10 May 2022 707