Build data pipelines, the easy way 🛠️
APACHE-2.0 License
Bot releases are hidden (Show)
Published by ricklamers over 3 years ago
We are very proud of this release and we call it: The Big One 🔥.
This release introduces a massive new feature to Orchest: full URL deep linking support. Easy back/forward browser navigation to open the right views in your browser. Reloading the URL and directly opening the right view.
Furthermore, environment build dependent actions (e.g. running a job, or interactive pipeline editing) are more automated: no more hard choices to make when builds complete.
In addition, we now support running an exhaustive debug dump using the simple orchest debug
command. This will make troubleshooting much easier for both the users & developers.
Many tests were added to the orchest-api
test suite for much improved reliability thanks mainly to @fruttasecca.
Finally, many smaller bug fixes and performance improvements were added. Have fun building in Orchest!
Published by ricklamers over 3 years ago
This release introduces the concept of Environment variables 📇 to Orchest! With its introduction we're deprecating and removing Data sources.
We've been testing Orchest and listening to feedback 📣. As a result we discovered everybody handles data sources slightly differently in their workflow. We want Orchest to be helpful, but allow for flexibility in for example the choice of the data source connector (e.g. Python offers various Postgres database connectors).
Environment variables allow for simple and effective secret passing (e.g. hostnames, passwords, usernames) to make it easy to connect and configure your favorite data sources.
In addition, this release includes many UI/UX improvements. Such as a dedicated project settings pane, and a cleaned up ✨ job pane.
Published by ricklamers over 3 years ago
This release brings some great quality of life 💎 features!
For those of you that prefer configuring your IDEs to your liking (🕶️ dark mode anyone?) - with this release we make JupyterLab configuration persistent. In addition to user config (settings you modify in the JupyterLab Settings pane) we also make client side JupyterLab extensions persistent. Support for server side extensions will be added later.
@howie6879 pointed out that it would be nice to have pipeline parameters. It would alleviate the need to repeat yourself in pipeline steps for global pipeline level parameterization. Hence, you can now configure pipeline parameters 🥳. Thanks @howie6879! Pipeline parameters can also be overridden for scheduled jobs.
In addition, some minor logging, performance, and UI improvements were made.
Published by fruttasecca over 3 years ago
Hotfix missing import in orchest-webserver
.
Published by ricklamers over 3 years ago
This release introduces a massive new feature: cron job scheduling ⏱.
From now on you can create jobs that run regularly specified by a cron schedule.
Gain maximal use out of your pipelines by batch scheduling them in your local 💻 or cloud ☁️ environment.
Published by yannickperrenet over 3 years ago
Published by ricklamers almost 4 years ago
This release mostly includes some minor bug fixes in the application web server (client & server side).
In addition we include some quality of life improvements like deletion of steps & connections in the pipeline editor through Delete and Backspace. Credits to @obulat!
Lastly, this release includes improved logging so that we can better analyze Orchest's runtime behavior which should speed up future development.
Published by ricklamers almost 4 years ago
Published by ricklamers almost 4 years ago
userdir/
- massive increase in performance (no more daemon 🎉).Heads up: this requires updating your old code. In short:
orchest.output(data)
becomesorchest.output(data, name)
.
Published by ricklamers almost 4 years ago
Published by ricklamers almost 4 years ago