optimus

Changelog

• f7ebafa5 fix: Remove sensor poke failure event (#520)

Changelog

• 6309c5cc fix: ignore deleted job when getting inferred dependencies (#517) (#518)
• c3c48af4 fix: use a single deployTimeout in optimus deploy cli (#515) (#516)

Changelog

• e55c5683 fix: job_end_event trigger set to all_success (#510) (#511)

Changelog

• 11698cd2 fix: bug breaking deploy on resource manager failure (#500) (#508)

Changelog

• fe3bb2ab fix: set depends on past value set by users in transformation task level (#488) (#509)

Changelog

• 5b3896d refactor: change external optimus sensor task id similar with internal sensor task id (#491)
• 65af2ca fix: bug job deployment failed and cancelled not returning error (#485)
• 606b5d4 fix: bug unable to decode GetJobSpecificationsResponse for jobs with hook (#478)
• bc0a5eb fix: always load configs from env variables regardless filepath exist (#473)
• 878ddda fix: capture scheduler exception errors in Optimus notifications (#457)
• ea50e90 fix: bug external sensor created when internal sensor with same urn already resolved (#466)
• 80df76a fix: bug unable to deploy job without dependency mod (#459)
• b4c9bba feat: support optimus sensor (#438)
• aeda4bb fix: load config from env var (#447)
• bd5f932 fix: race condition on resource deploy (#448)
• 779ca46 feat: add Optimus dashboard (#439)
• e69feb9 feat: revamp config overriding (#431)
• 00de128 fix: pass context to plugin call (#402)
• 0aba386 refactor: GetJobSpecificationsResponse to also include project and namespace names (#436)
• 6a6835a refactor: persist job and resource dependency (#401)
• 3cf7239 feat: job monitoring (#395)
• 59aaefa fix: use go releaser token to push artifacts
• e82229d fix: use job_spec api to get schedule interval for sensors (#426)
• a418c90 fix: use right execution_dates to fetch airflow job runs for varying schedule intervals (#396)
• a976283 fix: increase send and receive size limit (#427)
• 05fd4c9 feat: provide api related to optimus sensor (#406)
• de15c74 fix: temporary fix to avoid failing of sensor fast in case of shedule run is not present
• 79fdefa feat: resource apply changes only if different (#351)
• 7b1198b fix: format name for read config through file (#403)
• 609b62d feat: async job deployment (#350)
• 36a5111 fix: use right execution_dates to fetch airflow job runs for varying schedule intervals (#396)
• bb61b05 fix: initialize cli colors (#361)
• 243f0c0 feat: improve telemetry for external system interactions (#393)
• 3b283db feat: expose telemetry address to plugin and propagate metadata downstream (#388)
• 3a4ea5e fix: server crash when accepting improper resource create request (#374)
• 453568a refactor: extension (#328)
• 54f0784 fix: add missing logs on job deployment and fix job refresh cli (#358)
• 6d63be3 refactor: merge refactoring changes to main (#373)
• 19ff6d6 refactor: split interfaces for backup and replay (#312)
• 85a7f47 refactor: command (#313)
• df0ca11 fix: resolve bug when refreshing jobs with cross project dependency (#349)

Changelog

• 59aaefa fix: use go releaser token to push artifacts
• a418c90 fix: use right execution_dates to fetch airflow job runs for varying schedule intervals (#396)

Changelog

• dc3fcc5 Api to delete secret for project (#160)
• fcfddc6 Enable all linters, disable only the problamatic ones (#159)
• c07c545 Namespace name during saving secret is optional (#158)
• ed76281 feat : add external http sensor at optimus (#150)
• a8da326 feat: Split runtime in multiple files (#153)
• 76be7c4 feat: Use secrets in macros (#155)
• 008f5a8 feat: Use services in runtime instead of repo-factories (#149)
• 49a76e4 feat: add api to list secrets (#141)
• b5277d5 feat: add dagrun_timeout to airflow dag (#142)
• 500e3cd feat: create and update secrets through cli & api (#129)
• 01db921 feat:Print Basic Details of Image Version & Optimus Client & Server Versions (#143)
• 7c7fcda fix : maintain single optimus host variable across environment (#147)
• 431059b fix(optimus): fix issue related to newline of resources (#140)
• a2a0f91 fix: avoid unmarshall if external dependencies is null (#214)
• a759f84 fix: fix issues for staticcheck linter (#191)
• 3d6f277 fix: remove or rename unused params (#190)
• bfec874 fix: remove reference to make generate in release (#212)
• bcd7324 fix: set CrossTenantDependancySensor mode from default poke to reschedule (#145)
• ecccf4b fix: update the grpc limits for the client side (#234)
• 1ec74c3 refactor: Use optimus config struct instead of interface (#162)
• 9e7d04b refactor: change command for running test in makefile (#194)
• a993cf4 refactor: clean unused code and methods (#161)
• 9b7a598 refactor: fix errors reported by magic number lint (#199)
• d775eb1 refactor: fix issues reported by stylecheck and enable the linter (#193)
• 31b6c90 refactor: reduce the number of global vars (#195)
• 28e6062 refactor: refactor and cleanup for secrets management (#202)
• 8521ef1 refactor: remove deprecated package "github.com/pkg/errors" (#200)

Changelog

• dc3fcc5a Api to delete secret for project (#160)
• fcfddc6a Enable all linters, disable only the problamatic ones (#159)
• c07c5452 Namespace name during saving secret is optional (#158)
• ed76281d feat : add external http sensor at optimus (#150)
• a8da3263 feat: Split runtime in multiple files (#153)
• 76be7c4b feat: Use secrets in macros (#155)
• 008f5a8e feat: Use services in runtime instead of repo-factories (#149)
• 49a76e48 feat: add api to list secrets (#141)
• b5277d5c feat: add dagrun_timeout to airflow dag (#142)
• 500e3cdd feat: create and update secrets through cli & api (#129)
• 01db9214 feat:Print Basic Details of Image Version & Optimus Client & Server Versions (#143)
• 7c7fcda7 fix : maintain single optimus host variable across environment (#147)
• 431059b6 fix(optimus): fix issue related to newline of resources (#140)
• a2a0f91d fix: avoid unmarshall if external dependencies is null (#214)
• a759f840 fix: fix issues for staticcheck linter (#191)
• 3d6f2771 fix: remove or rename unused params (#190)
• bfec8741 fix: remove reference to make generate in release (#212)
• 2ce5ba4f fix: sensor to use scheduled_time & all the window params (#211)
• bcd7324b fix: set CrossTenantDependancySensor mode from default poke to reschedule (#145)
• 1ec74c35 refactor: Use optimus config struct instead of interface (#162)
• 9e7d04b2 refactor: change command for running test in makefile (#194)
• a993cf46 refactor: clean unused code and methods (#161)
• 9b7a5987 refactor: fix errors reported by magic number lint (#199)
• d775eb10 refactor: fix issues reported by stylecheck and enable the linter (#193)
• 31b6c90e refactor: reduce the number of global vars (#195)
• 28e60627 refactor: refactor and cleanup for secrets management (#202)
• 8521ef10 refactor: remove deprecated package "github.com/pkg/errors" (#200)

Changelog

• 0a8d332 fix: add instance_type prefix in airflow lib (#136)
• dcad5af fix: make job run data backward compatible to old runs
• c36818b fix: use scheduled at instead of current time for empty jobrun data

Changelog

• 0a8d332 fix: add instance_type prefix in airflow lib (#136)
• dcad5af fix: make job run data backward compatible to old runs
• c36818b fix: use scheduled at instead of current time for empty jobrun data

Changelog

• c840243 fix: airflow2 api url path
• a32e35a fix: empty metadata during job creation panics the server (#135)
• 3a021be fix: rerunning an old job cause task and hook execution time to differ (#133)
• 93d12fa fix: unknown project dependency should not raise an error
• 22696aa refactor(cli): make project/namespace configs required in deploy cmd (#134)

Changelog

• e8b3363 feat(cli): add command grouping (#132)
• 02efc17 fix(cli): log formatting for optimus job render command
• 839e39f fix: serialize instance type to protobuf enum
• 94adc17 refactor: rename v1 to v1beta1 go package for proto api

Changelog

• 7645d51 feat: progress bar for long running operations in cli (#130)
• 5f34b27 feat: support for specifying execution resources using metadata (#120)
• b547d74 refactor: protobuf lint issues of Runtime/Plugins/Cluster RPCs (#106)

Changelog

• 4e3ec40 feat: add extension supports
• d84d85b feat: allow manually triggered job using empty scheduler interval (#122)
• 448bfd7 feat: endpoint/rpc for fetching task details of a job (#107)
• cc106f3 feat: enhance replay & backup to support multiple namespaces jobs (#105)
• 4e7c152 feat: fetch info for requested backup runs (#112)
• 9dc68d8 fix: dependency resolution during duplicate destinations of job (#118)
• 330889d fix: deployment fails even if broken job belong to different namespace (#110)
• 1414cd8 fix: priority resolution should not fail because of missing jobs (#117)
• ef394fc refactor: project and namespace configuration to support multiple namespaces (#103)

Changelog

a908f4e feat: add backup dry run (#77)
4622182 feat: add backup feature (#88)
86c3e9b feat: add replay list sub command (#67)
56cbf97 feat: add replay syncer and get replay status (#48)
cf8c7bb feat: custom scheduler for running jobs on demand (#65)
94d5619 feat: pass authorization http header from cli if available (#90)
6da97d0 feat: support alert for the latest binary release (#46)
7150c57 feat: support static inter project dependencies (#93)
d1001c8 feat: telemetry for api traces, database calls and runtime stats (#102)
1ac1718 fix: cross entity sensor in airflow not failing if condition not met (#99)
1ac23a8 fix: cross entity sensor in airflow should use base job windows (#84)
f853468 fix: deploy grpc stream should be concurrent safe (#86)
407057b fix: instances should reuse execution time across a complete run (#96)
5d0fdae fix: skip dependency checks for non DR mods (#54)
ca5771e fix: skip using a prefixed bucket if gcs path is empty
3db9fa5 fix: validation for empty project config during deploy
6b760d4 refactor: airflow scheduler should upload jobs in parallel
587ace4 refactor: inject logger using salt in the required packages (#55)
089e4a6 refactor: optimise ListJob operation for airflow (#95)
2d3c693 refactor: rename dryrun replay methods (#78)
f98a15a refactor: use gorm v2 for postgres storage (#101)
25f4492 refactor: use updated buf lib for inmemory proto generation (#76)

Docker images

• docker pull docker.io/odpf/optimus:0.0.3
• docker pull docker.io/odpf/optimus:v0.0.3-amd64

Changelog

872e8a1 fix: plugin discovery based on plugin type

Docker images

• docker pull docker.io/odpf/optimus:0.0.3-alpha.2
• docker pull docker.io/odpf/optimus:v0.0.3-alpha.2-amd64

Changelog

7aca79c feat: breaking plugins to mods for easier composition (#47)
4bbe355 feat: google sheets external table support for bigquery datastore (#45)
d5c9b08 feat: validate replay request before execution (#35)
bc9912a refactor: pass raw resource with spec yaml parsing errors (#43)
506dff7 refactor: replay errors, replay failure test cases, airflow and replay manager (#37)

Docker images

• docker pull docker.io/odpf/optimus:0.0.3-alpha.1
• docker pull docker.io/odpf/optimus:v0.0.3-alpha.1-amd64

Changelog

d4f27ee chore: update cobra version to v1.2.1 (#29)
987d7d9 feat: support slack based alerts on failure and sla miss (#25)
58ebfc3 fix: return error when fail to parse yaml spec for resource

Docker images

• docker pull docker.io/odpf/optimus:0.0.2
• docker pull docker.io/odpf/optimus:v0.0.2-amd64

Changelog

4cdee6a feat(cli): ask for directories from user before creating new specification (#15)
d11711f feat(cli): support default values inferred from job name (#21)
57a3c9f feat: implement replay without backup and validation (#12)
a577ede fix: cross entity sensor should use updated job metadata api (#23)
ea6cf6a fix: using m instead of h while requesting window dates in airflow sensor

Docker images

• docker pull docker.io/odpf/optimus:0.0.2-alpha.3
• docker pull docker.io/odpf/optimus:v0.0.2-alpha.3-amd64