amundsen

Adds TypeMetadata as a possible ResourceType to allow implementing metadata service endpoints like put_resource_description which take a ResourceType.

Minor cleanups in TypeMetadata utils.

• fix: dashboard query #1762

Minor improvements to nested column type models and parsing.

Adds the TypeMetadata model and makes it an optional field of Column. TypeMetadata will allow treating types and subtypes as first class citizens with their own descriptions/badges.

Changes the verbiage of the new TypeMetadata feature, removing references to the word "subtype".

• feat: Databuilder support for nested columns #1695

• ✅ Add Elasticsearch 8 compatibility
• ❌ Remove Elasticsearch 6 compatibility

• ✅ Add Elasticsearch 8 compatibility
• ❌ Remove Elasticsearch 6 compatibility

• ✅ Add Elasticsearch 8 compatibility
• ❌ Remove Elasticsearch 6 compatibility

• fix: finalized cypher queries for new ES publishing task #1717

• fix: small error when using given queries #1715

• feat: general document mappings for ES #1637

• Changes neo4j stale remover task to delete targets which are < (instead of <>) to a given job_publish_tag. This may remove fewer targets than previously but prevents deleting fresh data.
• Allows neo4j stale remover task to take an optional retain_data_with_no_publisher_metadata config. This will prevent deleting targets without a job_publish_tag or publisher_last_updated_epoch_ms property. Targets without these properties occur through UI actions, such as editing a description or adding an owner. The default behavior continues to be to delete them.

Thanks to @kristenarmes for these improvements.

• refactors generic usage extractor for easier subclassing

• refactors hive table metadata extractor for easier subclassing
• refactors hive table last updated extractor for easier subclassing

• fixes issues with db mapping of superset extractor

• feat: Support for ES 7, ES field names containing dots, ES field sort order #1648

NEW SEARCH BW INCOMPATIBLE
Related RFC

• chore: bumped search to 3.0.0 (#1630)
• fix: add missing fields for dashboard uri (#1631)
• fix: added missing description fields to mappings (#1629)
• feat: v2/document API (#1611)
• fix: error on search with pagination (#1622)
• fix: new search API fixes (#1594)
• fix: added page size argument for compatibility (#1593)
• fix: add es v2 client config (#1592)
• fixed response schema issue (#1585)
• feat: search v2 API (#1556)
• chore: deprecate Atlas in Search (#1530)

Notes:

• Check out amundsen-frontend 4.0.0 release
• Make sure to configure PROXY_CLIENT = PROXY_CLIENTS['ELASTICSEARCH_V2']

NEW SEARCH BW INCOMPATIBLE
Related RFC

• feat: Use new search endpoint in FE and add search filter operation selection (#1623)
• feat: FE document update with v2/document search service endpoint use (#1619)

Notes:

• Make sure to add allowableOperation to custom filter configurations
• Check out amundsen-search 3.0.0 release