Collect, aggregate, and visualize a data ecosystem's metadata
APACHE-2.0 License
Bot releases are hidden (Show)
#2770
@phixMeCOMPLETE
/FAIL
events within streaming jobs #2768
@pawel-big-lebowskijob_version
is not created for a streaming job terminal event with no dataset information and existing version is kept.
Published by merobi-hub 8 months ago
Published by merobi-hub 9 months ago
#2714
@davidsharp7#2705
@davidsharp7#2716
@davidsharp7Published by merobi-hub 10 months ago
Published by merobi-hub 10 months ago
RunDao
SQL query #2685
@sophiely#2683
@sophielyDatasetEvent
#2641
#2654
@pawel-big-lebowskiDatasetEvent
event type. Includes optimization of the lineage query.
JobEvent
#2661
@pawel-big-lebowskiJobEvent
event type.
#2682
@pawel-big-lebowski#2658
@julienledemDELETE
endpoint for dataset tags #2698
@davidsharp7datasets_tag_mapping
to supply a way to delete a tag from a dataset via the API.
#2672
@davidsharp7#2693
@davidjgossMarquezPathV1::encode
to use the UrlEscapers
path segment escaper, which does proper URI encoding.
#2655
@merobi-hub#2679
@phixMe#2695
@davidjgossbuild.gradle
that was bringing some transitive vulnerabilities.
Published by merobi-hub about 1 year ago
Client: add Java client method for dataset/job lineage #2623
@davidjgoss
To add a method for the dataset/job-level endpoint (GET /lineage
) to the Java SDK, this adds a new method to the MarquezClient
for the endpoint, along with tests, and the necessary new subclasses of NodeData
for datasets and jobs.
Web: add IO tab #2613
@phixme
Improves experience with large graphs by adding a new tab to move between graph elements without looking at the graph itself.
Web: add hover-over Tag tooltip to datasets #2630
@davidsharp7
For parity with columns in the GUI, this adds a Tag tooltip to datasets.
Docker: upgrade to Docker Compose V2 #2644
@merobi-hub
Docker Compose V1 has been at EOL since June, but docker/up.sh uses the V1 format. This upgrades the up
command in up.sh to V2.
API: drop table job_contexts
and usage #2621
@wslulciuc
Removes usage of job_contexts
, which has been replaced by OpenLineage facets, and adds a migration to drop the table.
API: remove usage of current_job_context_uuid
column #2622
@wslulciuc
Removes usage of job_context_uuid
and current_job_context_uuid
. Column to be removed in 0.43.0.
Web: fix Unix epoch time display for null endedAt
values #2647
@merobi-hub
Fixes the issue of the GUI displaying Unix epoch time (midnight on January 1, 1970) in the case of running jobs/null endedAt
values.
Published by merobi-hub about 1 year ago
SearchDao
#2556
@tati @wslulciucYYYY-MM-DD
for query params, use YYYY-MM-DD
as LocalDate
, and support the following query params:
namespace
- matches jobs or datasets within the given namespace.before
- matches jobs or datasets before YYYY-MM-DD
.after
- matches jobs or datasets after YYYY-MM-DD
.#2614
@phixme#2612
@davidsharp7#2606
@davidsharp7#2601
@davidsharp7#2615
@phixmejobdao
query #2609
@algorithmy114
#2607
@wslulciuc./docker/down.sh
to clean existing volumes.
#2600
@davidjgoss@NonNull
annotation from the client class and the @NotNull
from the model class.
Published by merobi-hub about 1 year ago
#2577
@phixme#2579
@terrpanchart/templates/helpers
to use the global.imageRegistry
input value for the wait-for-db
container to improve performance on private registries.
#2578
@AmandaYao00idFromValue()
and idFromValueAndType()
#2581
@wslulciucnull
values in these functions in EventTypeResolver
with defaults.
#2575
@davidjgossPublished by merobi-hub about 1 year ago
#2569
@jlukenoff#2562
@merobi-hub#2563
@Xavier-Cliquennois#2573
@phixmerunInfo
facets search #2557
@merobi-hubrunInfo
and search.
#2571
@phixmehttp-proxy-middleware
.
Published by merobi-hub about 1 year ago
#2486
@wslulciucdbRetention
config in marquez.yml
for enabling a retention policy, and a db-retention
command for executing a policy.
#2535
@phixmeDbRetentionJob(Jdbi, DbRetentionConfig)
#2549
@wslulciuc@Positive
to DbRetentionConfig
instance variables for validating DbRetentionConfig
properties internally within the class.
#2551
@wslulciucDbRetentionJob
.
#2534
@phixme#2554
@hloomupgrade#2559
@Nisarg-ChokshiDataset
& Event
route rendering to remove pagination in the case of no content.
Published by merobi-hub over 1 year ago
#2495
@pawel-big-lebowskiEventTypeResolver
for using the schemaURL
field to decode POST
requests to /lineage
with LineageEvent
s, DatasetEvent
s or JobEvent
s, as the first step in implementing static lineage support.
#2531
@pawel-big-lebowski#2529
@jlukenoffPublished by merobi-hub over 1 year ago
#2525
@jlukenoffuuid
for etl_menus
in seed data #2519
@wslulciucrunID
collision creating an invalid lineage graph when the seed command is used.
#2516
@Nisarg-Chokshi#2524
@jlukenoff#1419
does not revert.
Published by merobi-hub over 1 year ago
#2512
@merobi-hub#2506
@jlukenoffPublished by merobi-hub over 1 year ago
#2488
@wslulciucPublished by merobi-hub over 1 year ago
inputFacets
and outputFacets
from Openlineage specification #2417
@pawel-big-lebowskiinputFacets
/ outputFacets
sent within datasets, exposing them through the Marquez API as part of the Run
resource.
simple_name
for job updates #2457
collado-mikesimple_name
and updates the relevant test.
#2461
collado-mikerun_uuid
via a new temp table for tracking and sorting runs.
#2464
collado-mike#2472
collado-mikejobs_fqn
table in #2448
.
#2458
tito12#2458
tito12run.jobVersion
in DatasetInfo.tsx
to fix rendering issues. [#2471
] (https://github.com/MarquezProject/marquez/pull/2471) perttus#2467
perttusPublished by merobi-hub over 1 year ago
#2407
@pawel-big-lebowskiDatasetDao
and DatasetVersionDao
.
#2430
@thomas-delrueMARQUEZ_PORT
as .Values.marquez.port (80) in the Helm Chart so the Marquez Web component can communicate with the API.
MqCode
#2454 @JDarDagranDatasetInfo
and RunInfo
pages when no SqlJobFacet
exists.
#2373
@JDarDagranSQLJobFacet
.
jobs_fqn
table and move FQN into jobs directly #2448
@collado-mikePublished by merobi-hub over 1 year ago
#2336
@tito12#2384
@tito12#2388
@merobi-hubFontAwesomeIcon
s to enable a built-in accessibility feature.
jobs_fqn
table using namespace_name
and job_fqn
columns #2357
@collado-mikecolumn_lineage
, dataset_facets
, job_facets
tables #2419
@pawel-big-lebowski#2400
@phixmefields
property the same for datasets and dataset versions, allowing type-generating systems to treat them the same way.
#2409
@tito12Published by merobi-hub over 1 year ago
#2076
@wslulciucOptimize query performance for OpenLineage facets
.
#2293
@pawel-big-lebowski @tito12#2343
@tito12lineage_events
table to dataset_facets
, run_facets
, and job_facets
tables. 2350
, 2355
, 2359
#2380
@rossturkdocker-compose down
.
#2381
@rossturkColumnLineageDatasetFacet
JSON snippets to docker/metadata.json
to seed data for column-level lineage facets.
RunLink
and JobLink
#2342
@pawel-big-lebowskiParentRunFacet
to avoid NullPointerException
s in the case of empty run sections.
docker-compose.web.yml
as base compose file #2360
@wslulciucdocker/up.sh
so the script uses docker-compose.web.yml
with overrides for dev
set via docker-compose.web-dev.yml
.
#2353
@merobi-hub#2374
@perttus#2389
@phixmelimit
and offset
to the openapi.yml spec file as query parameters.
Published by merobi-hub almost 2 years ago
created_at
column to lineage events table https://github.com/MarquezProject/marquez/pull/2299 @prachim-collab
iconSearchArrow.svg
https://github.com/MarquezProject/marquez/pull/2280 @wslulciuc
RunDao.updateStartState()
https://github.com/MarquezProject/marquez/pull/2319 @wslulciuc