kestra

Infinitely scalable, event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code.

APACHE-2.0 License

Downloads
228.4K
Stars
7.2K
View Code on GitHub Visit Website View on X
Ecosystems: Java

Bot releases are visible (Hide)

kestra - v0.18.0 Latest Release

Published by github-actions[bot] 3 months ago

Features

Bug Fixes

Documentation

Code Refactoring

Tests

Continuous Integration

Chores

Commits

  • da500b3: feat(core & webserver): bring lightweight version of namespaces (brian.mulier) #4219
  • 735ed6e: fix(core, jdbc): purge deleted executions (Loïc Mathieu) #4282
  • f39a6ac: feat(core, jdbc): add a PurgeLogs task (Loïc Mathieu) #4298
  • 2416760: feat(core,jdbc,webserver): find trigger by worker id (Loïc Mathieu) #4336
  • 0125d50: feat(core, jdbc, webserver): allow to filter logs by triggerId (Loïc Mathieu) #4337
  • a5dc8e4: feat(core, script): use a volume instead of a bind for handling files in the Docker task runner (Loïc Mathieu) #4349
  • f78a765: Revert "feat(cli): provide descriptive server start error (#4171)" (YannC)
  • 24e0063: closes https://github.com/kestra-io/kestra/issues/4371 (Anna Geller)
  • 5bab548: fix(ui):hide switch if trigger is disabled from source code (#4422) (YannC) #4422
  • 89af5cb: Fix/waitfor issue (#4421) (YannC) #4421
  • 68a7527: related to https://github.com/kestra-io/kestra/issues/4396 (Anna Geller)
  • 54aa618: feat(core,jdbc,webserver): delete flow logs (Loïc Mathieu) #4438
  • ui: regular dependency updates (#4469) #4469 (Miloš Paunović)
  • 3462815: feat(core, webserver): delete execution related logs, metrics and storage (Loïc Mathieu) #4427
  • uploadfiles fix (#4484) #4484 (YannC)
  • 69a1e8a: Revert "chore(deps): bump kafkaVersion from 3.7.1 to 3.8.0" (#4515) (YannC) #4515
  • c73d832: fix https://github.com/kestra-io/kestra/issues/4371 (Anna Geller)
  • 3aead5b: fix(core, script): Docker and Proces should always be a new instance (Loïc Mathieu)

Breaking Changes

Docker image tags

Most of our users leverage a Kestra version with all plugins included. We've changed our Docker image tags to ensure that the default Docker image includes all plugins. To use a lightweight version of Kestra without plugins, you will now have to explicitly use the suffix *-no-plugins.

Before upgrading to Kestra 0.18.0, make sure to adjust the image in your deployment as follows:

  • kestra/kestra:latest to always use the latest stable version incl. all plugins
  • kestra/kestra:latest-no-plugins to always use the latest stable version of Kestra without plugins (e.g. only your custom plugins).

If your deployment strategy is to pin the version, make sure to change the image to:

  • kestra/kestra:v0.18.0 if you want to have all plugins included in the image
  • kestra/kestra:v0.18.0-no-plugins if you prefer to use only your custom plugins.

If you are using the develop images that include the most recently developed (but not yet released) features and bug fixes, make sure to change the image to:

  • kestra/kestra:develop if you want to have all plugins included in the image
  • kestra/kestra:develop-no-plugins if you prefer to use only your custom plugins.

TL;DR of what you need to do on your end:

  • If you use the develop-full or latest-full image with all plugins, cut the -full suffix from your Docker image tag, and you're good to go!
  • If you use the develop or latest image without plugins, add -no-plugins suffix to the image tag.

We believe that this change will simplify your deployments. If you have any questions or need help with this change, please reach out via Slack.

The kestra-ee binary has been renamed to kestra

If you are using the kestra-ee CLI, note that it has been renamed to kestra. This change is intended to avoid confusion between the open-source and Enterprise Edition binaries.

The runner property is deprecated in favor of the taskRunner

Starting with Kestra 0.18.0, the taskRunner property has precedence over the runner property. In Kestra 0.18.0, task runners are out of Beta, and runner is now deprecated. You can still use the deprecated runner property when explicitly specified, but remember that Docker-type taskRunner is now the default way of running script tasks in Kestra. We recommend switching any existing pluginDefaults using runner to taskRuner.

Changes in JSON function and filter (non-breaking)

We've made several improvements to JSON and ION handling. To avoid confusion between the json filter and json() function, we've renamed:

  1. the json filter to toJson — it converts an object or data structure into a JSON string e.g. {{ [1, 2, 3] | toJson }}
  2. the json() function to fromJson — it converts a JSON string into a data structure or object so that you can access further attributes using a dot notation e.g. {{ fromJson(kv('YOUR_JSON_KEY')).some_attribute }}

This renaming has been done with an alias to the old name:

  • The json filter has been renamed to toJson — using the old filter will emit a warning in the server logs
  • The json() function has been renamed to fromJson — using the old function will emit a warning in the server logs

New mechanism of the Purge tasks (non-breaking)

We've changed the mechanism of the Purge tasks to make it more performant and reliable:

  • io.kestra.plugin.core.storage.Purge has been renamed to io.kestra.plugin.core.execution.PurgeExecutions to reflect that it only purges data related to executions (e.g. not including trigger logs, to purge those you should use the PurgeLogs task) — we've added an alias so that using the old task type will still work but it will emit a warning. We recommend using the new task type.
  • io.kestra.plugin.core.storage.PurgeExecution has been renamed to io.kestra.plugin.core.storage.PurgeCurrentExecutionFiles to reflect that it can purge all execution-related data incl. inputs to an Execution and Execution outputs — we've added an alias so that using the old task type will still work but it will emit a warning. Also here, we recommend adjusting your flow code to match the new task type.

Starting with the 0.18.0 release, the recommended way to clean internal storage files, executions and logs is using a combination of io.kestra.plugin.core.execution.PurgeExecutions but without logs and the newly added io.kestra.plugin.core.log.PurgeLogs task that removes all logs (both Execution logs and Trigger logs) in a performant batch operation. Combining those two together will give you the same functionality as the old io.kestra.plugin.core.storage.Purge task but in a more performant and reliable way (roughly 10x faster as compared to the old task).

id: purge
namespace: company.team
description: |
  This flow will remove all executions and logs older than one month.
  We recommend running this flow daily to avoid running out of disk space.

tasks:
  - id: purge_executions
    type: io.kestra.plugin.core.execution.PurgeExecutions
    endDate: "{{ now() | dateAdd(-1, 'MONTHS') }}"
    purgeLog: false
    
  - id: purge_logs
    type: io.kestra.plugin.core.log.PurgeLogs
    endDate: "{{ now() | dateAdd(-1, 'MONTHS') }}"

triggers:
  - id: daily
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "@daily"

KV Store permission must be added manually for now (non-breaking)

Starting with the 0.18.0 release, we've introduced a new KVSTORE permission that allows users to access the new KV Store functionality. This permission needs to be added manually to all Roles that need access to the KV Store.

If you have any Users and Groups that want to leverage the new KV Store functionality, make sure to update their RBAC to include the new KVSTORE permission.

The latest kestra terraform provider now uses plugin_defaults instead of task_defaults

The 0.18.0 version of the Terraform Provider now uses the property plugin_defaults instead of task_defaults in the kestra_namespace resource. Simply replace task_defaults with plugin_defaults in your Terraform configuration before you upgrade your Kestra Terraform provider.

kestra - v0.17.17

Published by github-actions[bot] 3 months ago

Bug Fixes

  • ui: properly update low code editor on metadata editing (#4464) (Miloš Paunović)
  • ui: properly set key attribute to log-line component to render required content (#4468) (Miloš Paunović)
  • ui: filter out falsy values for icons list (#4481) (Miloš Paunović)
  • ui: handle invalid task run when showing logs fro execution topology view (#4482) (Miloš Paunović)
  • ui: amend editor height under namespace tabs (#4492) (Miloš Paunović)
  • ui: show tabs on blueprints for custom ones (#4493) (Miloš Paunović)
  • ui: casting boolean values on resume dialog (#4498) (Miloš Paunović)
  • allow create flow and allow save when namespace is allowed (#4512) (YannC)

Chores

  • upgrade to 0.17.17 (YannC)
kestra - v0.17.16

Published by github-actions[bot] 3 months ago

Chores

kestra - v0.17.15

Published by github-actions[bot] 3 months ago

Bug Fixes

  • ui: show proper code value in blueprint detail editor (#4444) (Miloš Paunović)

Chores

Commits

  • f2a33eb: Revert "fix(script): task runner override" (Loïc Mathieu)
kestra - v0.17.14

Published by github-actions[bot] 3 months ago

Bug Fixes

Chores

  • ui: enhanced email and password length constraints in basic authentication (#4261) (Frank Tianyu Zeng)
  • update to version 0.17.14 (YannC)
kestra - v0.16.12

Published by github-actions[bot] 3 months ago

Bug Fixes

Chores

kestra - v0.17.13

Published by github-actions[bot] 3 months ago

Bug Fixes

  • core: keep key order when rendering a Map (Yoann Vernageau)
  • ui: properly handle array input types (#4331) (Miloš Paunović)
  • ui: proprely handle json type inputs when prefilling executions (#4333) (Loïc Mathieu)
  • ui: handle file names without extensions (#4352) (Miloš Paunović)
  • allow null enum when required is false (YannC)

Chores

kestra - v0.16.11

Published by github-actions[bot] 3 months ago

Bug Fixes

Chores

kestra - v0.17.12

Published by github-actions[bot] 3 months ago

Features

Bug Fixes

Chores

Commits

  • f09fa74: fix(core, jdbc): purge deleted executions (Loïc Mathieu)
kestra - v0.17.11

Published by github-actions[bot] 4 months ago

Chores

kestra - v0.17.10

Published by github-actions[bot] 4 months ago

Bug Fixes

Chores

kestra - v0.17.9

Published by github-actions[bot] 4 months ago

Features

  • ui: humanize cron expressions (#4233) (yuri)

Bug Fixes

Chores

kestra - v0.17.7

Published by github-actions[bot] 4 months ago

Bug Fixes

  • core: use the flow revision from the condition context instead of the trigger context (Loïc Mathieu)
  • core: flows from one tenant don't erase those from others (brian.mulier)
  • ui: only append date values to inputs form if they exist (#4135) (Miloš Paunović)
  • ui: properly handle view type on namespace files (#4138) (Miloš Paunović)

Chores

kestra - v0.17.8

Published by github-actions[bot] 4 months ago

Chores

Commits

  • c3b1ceb: Revert "fix(ui): properly handle view type on namespace files (#4138)" (Loïc Mathieu)
kestra - v0.17.6

Published by github-actions[bot] 4 months ago

Bug Fixes

  • core: the Request task can crash the worker (Loïc Mathieu)
  • jdbc: add timezone in JDBC url as it was using default JVM timezone (#4114) (YannC)

Chores

kestra - v0.16.10

Published by github-actions[bot] 4 months ago

Chores

kestra - v0.17.5

Published by github-actions[bot] 4 months ago

Bug Fixes

  • core: add secret consumer when rendering variables for subflows (brian.mulier)
  • core: possible NPE on namespace files usage (Loïc Mathieu)
  • ui: amended editor content when switching from topology view (#4091) (Loïc Mathieu)
  • script: bad merge (Loïc Mathieu)

Chores

kestra - v0.17.4

Published by github-actions[bot] 4 months ago

Features

Bug Fixes

  • *****: add tenant id to namespace identifier for skip execution by namespace (brian.mulier)
  • core: nullable tenants & executions for execution skips (brian.mulier)
  • namespaces files working in script tasks on Windows (YannC)

Chores

kestra - v0.15.17

Published by github-actions[bot] 4 months ago

Features

Bug Fixes

  • *****: add tenant id to namespace identifier for skip execution by namespace (brian.mulier)
  • core: nullable tenants & executions for execution skips (brian.mulier)

Chores

kestra - v0.17.3

Published by github-actions[bot] 4 months ago

Features

  • ui: better handle of no permission (#4004) (YannC)

Bug Fixes

  • core: add unique prefix identifier for output files (#3991) (Florian Hussonnois)
  • core: decrypt outputs for tasks within WorkingDirectory (#4001) (brian-mulier-p)
  • core: properly inject pluginConfiguration for WorkingDirectory task (#4006) (Florian Hussonnois)
  • replace Windows \ for / in LocalStorage (YannC)
  • ui: properly parsing json files in the editor (#4007) (Miloš Paunović)
  • ui: improve the readability of error message in flows (#3901) (Frank Tianyu Zeng)
  • webserver: trim the flow when importing (Loïc Mathieu)
  • ui: allow colon mark in label value (#4027) (yuri)
  • ui: only throw flow/execution not found on sse error when execution is not populated (YannC)
  • ui: no stringify of json inputs (already string as coming form string input) (YannC)
  • webserver: add plugin alias icons (Loïc Mathieu)

Chores

Package Rankings
Top 22.7% on Repo1.maven.org
Top 11.19% on Pypi.org
Top 7.48% on Proxy.golang.org
Badges
Extracted from project README
Star the Repo