Flyte v1.10.6-b0 Release

Beta release.

Flyte v0.19.2-b1

Resolving the minio port conflict.

Fixes a race condition with checking availability status on flyte pods

Wait for deployments to be created using rollout before checking for availability

Signed-off-by: Prafulla Mahindrakar [email protected]

https://github.com/flyteorg/flyte/pull/979

Platform

• Oauth2 support with SingleSignOn and configuration examples for popular IDP's now available in Flyte.
  Please see the updated information and description of the feature, and the setup information
  Attention: If using Auth already - this is a BREAKING change. refer to the migration guide to update configuration to ensure Admin continues to work. (No migration needed if auth is not turned on.)

• Backend improvements to support dynamic workflow visualization (in future releases).
• Lot of features added to flytectl .
• Documentation site overhaul and redesign (more in progress)

Flytekit

The first two features should be considered beta and subject to change

• First cut of the control plane classes to replace the old Sdk... classes. These classes provide programmatic access to a Flyte backend.
• New paradigm for flytekit-only plugins
• SqlAlchemy/Dolt plugins.

Please see the flytekit release for the full list and more details.

General

• Added CoPilot configuration to sandbox manifest
• Updated and streamlined documentation navigation and themes. Better content organization coming soon!

Performance

• Improved execution performance including:
  • Reduced cache lookups
  • Improved GetWorkflowExecution performance
  • Capped max number of nodes in each propeller round
  • Misc. propeller performance tweaks
• TaskTemplate offloading

Housekeeping

• Migrated Datacatalog protobuf definitions to flyteidl [thanks @tnsetting]
• Upgraded stow version used in flytestdlib
• Moved off lyft kubernetes forks and onto official kubernetes library dependencies
• Revamped pod tasks to use official kubernetes python client library for defining PodSpecs

Events

• Richer event metadata for task executions
• Better merging of custom info across task events

Bug fixes

• Resolved non-backwards protobuf role changes that prevented launching single task executions [thanks @kanterov]
• Better handling of large workflows and errors in flytepropeller

Flytekit (Python)

• Access to secrets
• Bug fixes around the 0.16 release.
  • Use original FlyteFile/FlyteDirectory
  • Fix serialization of pod specs in pod plugin [thanks @jeevb]
  • Accept auth role arg in single task execution
  • Fixed task resolver in map task
  • Requests and limits added to ContainerTask [thanks @migueltol22]

Flyte Platform

• New to flyte? https://start.flyte.org takes you through first run experience. (Thanks to @jeevb)
• Grafana templates for monitoring Flyte System and User Workflows.
• Extend Flyte docs.
• FlyteIdl Docs are published! You can learn about the core language that makes it all work.
• Additional knob for fine tuning flyte propeller performance that speeds up executions drastically.
• OidC support for Google Idp (And other OidC compliant Idps)
• Various stabilization bugs.

Flytekit

Since v0.16.0a2, the last flytekit milestone release, all effort has been towards stabilizing the new API. Please see the individual releases for detailed information. The highlights are

• Serialization/registration processes have been firmed up and utilities to ease that process introduced (not having to build a container to serialize for instance).
• Plugins structure revamped (eventually we'll move to a separate new repo entirely)
• User-facing imports have been organized into three top-level subpackages (flytekit, flytekit.testing, and flytekit.extend)
• Retries added to read-only Admin calls in client
• Lots of cleanup and additions to the cookbook and documentation generally.
• Bug fixes.

Flytekit

• Alpha release of Flytekit (v0.16.0a2). Please check the release for the detailed changelog.
• Alpha A revamp of the flytekit examples can be found here.

FlyteConsole

• Show all executions across project and domain
• Yielding launch plans from dynamic tasks will now show links for executions in the UI
• Ability to pass role/service account for single task executions in the launch form
• If auth is enabled, Console will automatically attempt login, saving you a click
• Bug and stability fixes

Flyte Platform

• SparkOperator v1beta2 support
• Bug Fixes

Flytekit

• Fast register - code only changes no longer need a rebuild of the container to run (read the docs!)
• Project archive/activate command in flyte-cli
• Bug fixes and misc improvements

Please take a look at the newest alpha release of Flytekit (v0.16.0a0) which introduces native Python based typing, and a new way of workflow authoring that enables local workflow execution.
For examples of the native typed SDK usage refer to: Cookbook Examples

FlyteConsole

• Better Single task launch support
• Launch form with more types supported
• Bug and stability fixes

Flyte Platform

• SparkOperator v1beta2 support
• Archive/Unarchive projects
• Bug Fixes

Flyte v0.8.0

Core Platform

• Metadata available for every task execution (First one: if the task was executed on interruptible nodes, useful when running on spot instances)
• Support for Cron schedules with offset (only on supported schedulers - e.g. Styx)
• Plugin overrides using Admin (docs coming soon)
• Custom model support in Sagemaker for single node and distributed training

Console

• Full inputs and outputs from Execution data
• Timestamp rendering always in UTC

Flytekit

• Dynamic overridable Spark configuration for spark tasks
• New Tensorflow task
• Formally removed python 2.x support. Supports only 3.6+
• Refactor - getting ready for enhanced auto-typed flytekit