kamu-cli

[0.166.1] - 2024-03-14

Fixed

• Allow OData adapter to skip fields with unsupported data types instead of chasing

[0.166.0] - 2024-03-14

Added

• Experimental support for OData protocol

Fixed

• Pulling datasets by account in multi-tenant workspace

[0.165.0] - 2024-03-12

Updated

• Extended flow history:
  • start condition update event now holds a snapshot of the start condition
  • input dataset trigger now returns a queryable Dataset object instead of simply identifier
  • dependent dataset flows should not be launched after Up-To-Date input flow

Fixed

• Zero value handling for per_page value in GraphQL
• Flow history: more corrections to show natural time of flow events and keep flow system testable
• Fixed metadata chain scanning regression

[0.164.2] - 2024-03-07

Changed

• Using cargo udeps to detect unused dependencies during linting

Fixed

• Flow history no longer produces duplicate flow abortion events
• Flow history no longer shows initiation time that is earlier than the next event's time

[0.164.1] - 2024-03-06

Changed:

• Flow status Cancelled was finally replaced with Aborted, unifying cancellation types for simplicity

Fixed

• Flow system now pauses flow configuration, even when cancelling an already completed flow

[0.164.0] - 2024-03-06

Added

• Added support of wildcard patterns for kamu pull and kamu push commands
• Added {dataset}/tail and /query REST API endpoints

[0.163.1] - 2024-03-01

Fixed

• Fixed mapboxgl dependency issue in Jupyter image

[0.163.0] - 2024-03-01

Changed

• Simplified flow statuses within Flow System (no more Queued or Scheduled status)
• Extended flow start conditions with more debug information for UI needs
• Simplified flow cancellation API:
  • Cancelling in Waiting/Running states is accepted, and aborts the flow and it's associated tasks
  • Cancelling in Waiting/Running states also automatically pauses flow configuration

[0.162.1] - 2024-02-28

Added

• kamu system check-token command for token debugging

Fixed

• kamu system api-server startup failure

[0.162.0] - 2024-02-28

Added

• Flow system now fully supports batching conditions for derived datasets:
  • not launching excessive flows unless minimal number of input records is accumulated
  • not waiting on the batching condition over a limit of 24h, if at least something accumulated

Fixed

• kamu login no longer requires workspace in --user scope (#525)
• Sync will correctly select smart ODF protocol when pushing/pulling via repository alias (#521)
• Fixed kamu verify crash under verbose logging (#524)
• Increased default number of results returned by kamu search command

[0.161.0] - 2024-02-26

Added

• kamu search command now works with ODF repositories

[0.160.0] - 2024-02-26

Changed

• Upgraded to latest datafusion v36.0.0
• Upgraded to latest jupyter

Added

• New kamu system generate-token command useful for debugging node interactions
• New --check flag for kamu login command to validate token with the remote and exit

[0.159.0] - 2024-02-16

Added

• New --exists-ok flag for kamu init command

Fixed

• Ignoring the trailing slash during inference of a dataset name from pull URL

[0.158.0] - 2024-02-13

Added

• Flows API now reports number of ingested/transformed blocks & records to improve UI informativity
• Support of --recursive flag for kamu delete and kamu verify commands

Changed

• The state when all flows of the given dataset are paused should be queryable via GraphQL API
• Added caching of metablock chains to improve performance within transactions

[0.157.0] - 2024-02-12

Added

• Complete support for arm64 architecture (including M-series Apple silicon)
  • kamu-cli now depends on multi-platform Datafusion, Spark, Flink, and Jupyter images allowing you to run data processing at native CPU speeds

Changed

• Spark engine is upgraded to latest version of Spark 3.5
• Spark engine is using ANSI mode by default which produces helpful errors instead of silently returning null in many built-in functions

Fixed

• Modeling mistake in Smart Transfer Protocol that creates unexpected push/pull transfer path for metadata blocks

[0.156.3] - 2024-02-09

Added

• Native support for arm64 architecture (including M-series Apple silicon) in kamu-cli and kamu-engine-datafusion
  • Note: Flink and Spark engine images still don't provide arm64 architecture and continue to require QEMU

Changed

• Flow system scheduling rules improved to respect system-wide throttling setting and take last successful run into account when rescheduling a flow or after a restart

Changes in this version:

[0.156.2] - 2024-02-07

Changed

• Using unique container and network names to prevent collisions when running concurrent kamu processes
• Improved error handling for all subprocesses