Flyte v0.17.3-b2

Change additional Admin volumes and volume mounts to list

Flyte v0.17.3-b1

Just picking up some minor changes around the admin deployment.yaml file

Flyte v0.17.2

Platform

• Added default buckets in sandbox helm
• Fixed bug in helm chart propeller config. logs url will not update correctly
• Added cluster-admin read only roles for kubernetes dashboard

Flyte v0.17.1

Platform

• Updated helm chart to include minio & kubernetes Dashboard nodePort

Flyte v0.17.0

Platform

1. Recovery Mode: Executions that fail due to external system failures (e.g. external system being down) can now be rerun in recovery mode (flytectl --recover docs). It's also available in the UI:

Flytekit

1. Great Expectations Integration (docs).
2. Access to durable blob stores (AWS/GCS/etc) are now pluggable.
3. Local task execution has been updated to also trigger the type engine.
4. Tasks that have cache=True should now be cached when running locally as well (docs).

Please see the flytekit release for the full list and more details.

UI

1. Shiny new Graph UX. The graph rendering has been revamped to be more functional and accessible. More updates are coming for better visualization for nested executions and branches.

2. JSON Validation for json-based types in the UI.

Before	After

1. Enum support in UI

   

FlyteCtl

1. flytectl upgrade to automatically upgrade itself (docs).
2. --dryRun is available in most commands with server-side-effects to simulate the operations before committing any changes.

And various stabilization fixes!

Flyte v0.16.0

FlyteKit

• Redesigned Flyte Remote
• Parse out descriptions of inputs/outputs for workflows and tasks (issue)
• Improvements to exception handling
• Reference launch plan support in dynamic tasks

Please see the flytekit release for the full list and more details.

Platform

• Allow download of unlimited size in copilot (issue)
• Allow overriding of various configs in flytectl command line (e.g. --storage.type=s3) (issue)
• Allow further customization of the namespaces in which workflows run (issue)

Documentation

• Improvements to getting started.
• Contributing to Flyte guide to help understand the components and their responsibilities (available here).

And various stabilization fixes!

Flyte v0.15.1

Platform

• Updated helm chart to include missing ingress paths

Flyte v0.15.0

Platform

• Enum type support throughout the system
• Stabilization improvements for conditionals
• Support AWS Secrets Manager as a source for secrets injector
• Support Map tasks over Pod tasks
• Support max parallelism to limit how many nodes can be allowed to run in parallel.
• Add Athena flytekit plugin and examples
• Add BigQuery plugin

Flytekit

• Support Schema of Dataclasses
• Support node resource overrides

Please see the flytekit release for the full list and more details.

flytectl

• flytectl sandbox start to start a sandbox cluster locally.
• flytectl get workflow .... -o dot to visualize a workflow graph locally.
• Add Bash completion support

Flyte v0.14.0

Platform

• Update to the Flyte Compiler, to supported nested branches and more. Better
  regression tests
• support for iam roles and k8s serviceaccounts
• customizable pod specs for pod tasks (add labels and annotations)
• copilot improvements
• Bug-fixes, visibility improvements
• support for non Cloud provider emailers - like Sendgrid
• performance improvement for dynamic workflows

Flyteconsole

• Bug fixes
• More updates coming soon

Flytekit

• Support for nested conditionals
• DoltTable and Dolt plugins and integration
• Better context management (ready for more work)
• Support for pre-built container plugins in flytekit. this makes it possible
  to library plugins and users do not need to build containers
• More control plane class features
• See full release notes
  Coming soon:
  • Great Expectations integration
  • More use case driven examples in flytesnacks

flytectl

• flytectl is ready for BETA. check it out - https://docs.flyte.org/projects/flytectl/en/latest/

Please see the flytekit release for the full list and more details.

Fixes a race condition with checking availability status on flyte pods

Wait for deployments to be created using rollout before checking for availability

Signed-off-by: Prafulla Mahindrakar [email protected]

https://github.com/flyteorg/flyte/pull/979

Platform

• Oauth2 support with SingleSignOn and configuration examples for popular IDP's now available in Flyte.
  Please see the updated information and description of the feature, and the setup information
  Attention: If using Auth already - this is a BREAKING change. refer to the migration guide to update configuration to ensure Admin continues to work. (No migration needed if auth is not turned on.)

• Backend improvements to support dynamic workflow visualization (in future releases).
• Lot of features added to flytectl .
• Documentation site overhaul and redesign (more in progress)

Flytekit

The first two features should be considered beta and subject to change

• First cut of the control plane classes to replace the old Sdk... classes. These classes provide programmatic access to a Flyte backend.
• New paradigm for flytekit-only plugins
• SqlAlchemy/Dolt plugins.

Please see the flytekit release for the full list and more details.

General

• Added CoPilot configuration to sandbox manifest
• Updated and streamlined documentation navigation and themes. Better content organization coming soon!

Performance

• Improved execution performance including:
  • Reduced cache lookups
  • Improved GetWorkflowExecution performance
  • Capped max number of nodes in each propeller round
  • Misc. propeller performance tweaks
• TaskTemplate offloading

Housekeeping

• Migrated Datacatalog protobuf definitions to flyteidl [thanks @tnsetting]
• Upgraded stow version used in flytestdlib
• Moved off lyft kubernetes forks and onto official kubernetes library dependencies
• Revamped pod tasks to use official kubernetes python client library for defining PodSpecs

Events

• Richer event metadata for task executions
• Better merging of custom info across task events

Bug fixes

• Resolved non-backwards protobuf role changes that prevented launching single task executions [thanks @kanterov]
• Better handling of large workflows and errors in flytepropeller

Flytekit (Python)

• Access to secrets
• Bug fixes around the 0.16 release.
  • Use original FlyteFile/FlyteDirectory
  • Fix serialization of pod specs in pod plugin [thanks @jeevb]
  • Accept auth role arg in single task execution
  • Fixed task resolver in map task
  • Requests and limits added to ContainerTask [thanks @migueltol22]

Flyte Platform

• New to flyte? https://start.flyte.org takes you through first run experience. (Thanks to @jeevb)
• Grafana templates for monitoring Flyte System and User Workflows.
• Extend Flyte docs.
• FlyteIdl Docs are published! You can learn about the core language that makes it all work.
• Additional knob for fine tuning flyte propeller performance that speeds up executions drastically.
• OidC support for Google Idp (And other OidC compliant Idps)
• Various stabilization bugs.

Flytekit

Since v0.16.0a2, the last flytekit milestone release, all effort has been towards stabilizing the new API. Please see the individual releases for detailed information. The highlights are

• Serialization/registration processes have been firmed up and utilities to ease that process introduced (not having to build a container to serialize for instance).
• Plugins structure revamped (eventually we'll move to a separate new repo entirely)
• User-facing imports have been organized into three top-level subpackages (flytekit, flytekit.testing, and flytekit.extend)
• Retries added to read-only Admin calls in client
• Lots of cleanup and additions to the cookbook and documentation generally.
• Bug fixes.