openverse

Improvements

• Remove proxy service and make API NGINX setup similar to the frontend (#4605) by @dhruvkb

Internal Improvements

• Expand ov documentation and replace environment setup docs with ov instructions (#4567) by @sarayourfriend
• Update pook to v2 (#4582) by @sarayourfriend
• Decrease link validation timeout (#4598) by @sarayourfriend

Improvements

• Make the sources tables responsive using CSS (#4543) by @szymon-polaczy

Internal Improvements

• Make running Playwright to update snapshots easier (#4585) by @madewithkode
• Remove unnecessary locales downloads from CI (#4627) by @obulat
• Update pnpm and use corepack (#4628) by @obulat
• Expand ov documentation and replace environment setup docs with ov instructions (#4567) by @sarayourfriend

New Features

• Add catalog indexer worker (#4330) by @stacimc

Improvements

• Expand ov documentation and replace environment setup docs with ov instructions (#4567) by @sarayourfriend
• Split batched_update DAG into automated and manual DAGs (#4552) by @AetherUnbound
• Add linting for Dockerfiles (#4540) by @dhruvkb
• Explicitly include Filter Data step in ingestion server removal IP (#4524) by @AetherUnbound
• Changes all sensible occurrences of the just commands to have them run using ov (#4483) by @szymon-polaczy

Internal Improvements

• Create a dataclass for indexer worker TaskStatus (#4601) by @akshay-km
• Update ingestion server removal IP with EC2 approach (#4615) by @stacimc
• Update dependency apache-airflow to v2.9.3 [SECURITY] (#4629) by @openverse-bot
• Expand ov documentation and replace environment setup docs with ov instructions (#4567) by @sarayourfriend
• Update pook to v2 (#4582) by @sarayourfriend
• Specify pull policy for openverse- images (#4568) by @dhruvkb
• Add CI/CD and PDM to new indexer worker (#4497) by @sarayourfriend
• Remove trim_and_deduplicate_tags DAG after successful run (#4557) by @sarayourfriend
• Fix separators in catalog and dev-env images and dev-env volume (#4526) by @dhruvkb
• Bump requests from 2.31.0 to 2.32.2 in /indexer_worker (#4532) by @dependabot
• Update dependency apache-airflow to v2.9.2 [SECURITY] (#4502) by @openverse-bot

Bug Fixes

• Fix Docker build warnings (#4573) by @dhruvkb
• Specify pull policy for openverse- images (#4568) by @dhruvkb
• Ensure plpython3u exists in live db when using it (#4555) by @sarayourfriend
• Add DAG to decode and deduplicate image tags with escaped literal unicode sequences (#4475) by @sarayourfriend
• Fix placing test S3 data into MinIO (#4495) by @krysal
• Fix trim and deduplicate tags deduplication (#4473) by @sarayourfriend

Internal Improvements

• Update ingestion server removal IP with EC2 approach (#4615) by @stacimc
• Bump sentry-sdk from 2.7.1 to 2.8.0 in /ingestion_server (#4631) by @dependabot

New Features

• Add catalog indexer worker (#4330) by @stacimc

Improvements

• Expand ov documentation and replace environment setup docs with ov instructions (#4567) by @sarayourfriend
• Split batched_update DAG into automated and manual DAGs (#4552) by @AetherUnbound
• Add linting for Dockerfiles (#4540) by @dhruvkb
• Explicitly include Filter Data step in ingestion server removal IP (#4524) by @AetherUnbound
• Changes all sensible occurrences of the just commands to have them run using ov (#4483) by @szymon-polaczy

Internal Improvements

• Update dependency apache-airflow to v2.9.3 [SECURITY] (#4629) by @openverse-bot
• Expand ov documentation and replace environment setup docs with ov instructions (#4567) by @sarayourfriend
• Update pook to v2 (#4582) by @sarayourfriend
• Specify pull policy for openverse- images (#4568) by @dhruvkb
• Add CI/CD and PDM to new indexer worker (#4497) by @sarayourfriend
• Remove trim_and_deduplicate_tags DAG after successful run (#4557) by @sarayourfriend
• Fix separators in catalog and dev-env images and dev-env volume (#4526) by @dhruvkb
• Bump requests from 2.31.0 to 2.32.2 in /indexer_worker (#4532) by @dependabot
• Update dependency apache-airflow to v2.9.2 [SECURITY] (#4502) by @openverse-bot

Bug Fixes

• Fix Docker build warnings (#4573) by @dhruvkb
• Specify pull policy for openverse- images (#4568) by @dhruvkb
• Ensure plpython3u exists in live db when using it (#4555) by @sarayourfriend
• Add DAG to decode and deduplicate image tags with escaped literal unicode sequences (#4475) by @sarayourfriend
• Fix placing test S3 data into MinIO (#4495) by @krysal
• Fix trim and deduplicate tags deduplication (#4473) by @sarayourfriend

Internal Improvements

• Update dependency elasticsearch to v8.14.0 (#4579) by @openverse-bot

Internal Improvements

• Bump schemathesis to v3.31.0 and silence warnings on API tests (#4593) by @krysal
• Drop status column from media reports (#4590) by @krysal

Internal Improvements

• Update dependency @swc/cli to ^0.4.0 (#4578) by @openverse-bot

Bug Fixes

• Fix TypeError by checking if duration is not Finite before setting cu… (#4584) by @spran180

Improvements

• Add linting for Dockerfiles (#4540) by @dhruvkb
• Include xml in frontend attribution options (https://github.com/WordPress/openverse/pull/4499) by @madewithkode

Internal Improvements

• Update dependency prettier-plugin-tailwindcss to v0.6.5 (#4574) by @openverse-bot
• Update dependency typescript to v5.5.2 (#4581) by @openverse-bot
• Update dependency @playwright/test to v1.45.0 (#4577) by @openverse-bot
• Update dependency prettier to v3.3.2 (#4580) by @openverse-bot
• Add CI/CD and PDM to new indexer worker (#4497) by @sarayourfriend

Bug Fixes

• Fix Docker build warnings (#4573) by @dhruvkb
• Remove volta and engines.pnpm settings (#4583) by @obulat

New Features

• Include xml in frontend attribution options (#4499) by @madewithkode

Improvements

• Add linting for Dockerfiles (#4540) by @dhruvkb

Internal Improvements

• Specify pull policy for openverse- images (#4568) by @dhruvkb
• Remove backfillmoderationdecision management command after production run (#4554) by @sarayourfriend
• Shorten PDM hash to first 8 characters (#4551) by @dhruvkb

Bug Fixes

• Fix Docker build warnings (#4573) by @dhruvkb
• Specify pull policy for openverse- images (#4568) by @dhruvkb
• Do not use unstable pook reference for API (#4536) by @sarayourfriend
• Create sensitive and deleted media models for decisions (#4544) by @dhruvkb

New Features

• Upload Ingestion Server's TSV files to AWS S3 (skip tags) (#4529) by @krysal

Internal Improvements

• Remove single quotes in values of Ingestion Server's TSV files (#4471) by @krysal
• Bump urllib3 from 2.2.1 to 2.2.2 in /ingestion_server (#4519) by @dependabot

New Features

• Include xml in frontend attribution options (#4499) by @madewithkode

Improvements

• Replace "Over..." language with more precise "Top..." (#4509) by @sarayourfriend

Internal Improvements

• Add caching to frontend Nginx configuration (#4516) by @sarayourfriend

Bug Fixes

• Fix possible TypeError when accessing properties of route.value (#4523) by @krysal
• Fix flaky VCollectionHeader snapshot tests (#4559) by @sarayourfriend
• Re-add tags page text (#4549) by @obulat

Improvements

• Display generated tags separately (#4291) by @obulat
• Changes all sensible occurrences of the just commands to have them run using ov (#4483) by @szymon-polaczy
• Stop opening links in a new tab (#4446) by @obulat

Bug Fixes

• Update works count on the frontend (#4486) by @szymon-polaczy

New Features

• Add catalog indexer worker (#4330) by @stacimc

Internal Improvements

• Log dead link verification request timings (#4508) by @sarayourfriend

Bug Fixes

• Drop FK constraint on media_obj in MediaDecisionThrough, update backfillmoderationdecision command (#4530) by @stacimc

New Features

• Added documentations for how to run DAGs in development alongside how to add new documentations. (#4388) by @madewithkode
• Add new data refresh factory (#4259) by @stacimc

Improvements

• Capture thumbnails during europeana ingestion (#4447) by @madewithkode
• Add verbose logging option to ProviderDataIngester (#4068) by @nicoepp
• Add catalog media properties documentation (#4366) by @obulat
• Use .venv for catalog virtualenv instead of venv (#4369) by @thegreendrinker
• Add variable to disable removing sql source files during ingestion (#4216) by @madewithkode
• Round decimal record difference percent to three decimals after zeros (#4065) by @mattfergoda
• Make links in docstring display as hyperlinks in docs site (#4195) by @madewithkode
• Add instructions how to run ingestion script from command line (#4083) by @kuephi

Internal Improvements

• Add verbose logging option to ProviderDataIngester (#4068) by @nicoepp
• Added documentations for how to run DAGs in development alongside how to add new documentations. (#4388) by @madewithkode
• Always assume special urgency for contributor PR pings (#4385) by @AetherUnbound
• Update dependency apache-airflow to v2.9.1 [SECURITY] (#4378) by @openverse-bot
• Convert longer media varchar fields to text in the catalog db (#4357) by @AetherUnbound
• Use .venv for catalog virtualenv instead of venv (#4369) by @thegreendrinker
• Set up airflow variable defaults with descriptions automatically (#4297) by @madewithkode
• Remove popularity & matview timeouts from data refresh configurations (#4353) by @johan-naizu
• Update bucket names to use openverse-catalog and remove openverse-storage (#4359) by @Wencho8
• Remove temporary Science Museum DAG now that it is no longer necessary (#4314) by @stacimc
• Rename old data refresh to legacy_data_refresh (#4260) by @stacimc
• Add variable to disable removing sql source files during ingestion (#4216) by @madewithkode
• Update dependency pytest-socket to v0.7.0 (#4244) by @openverse-bot
• Modify add_license_url DAG for more specific null check (#4124) by @krysal

Bug Fixes

• Update the 'updated_on' column during popularity refresh (#4460) by @stacimc
• Moved by tag from the fuzzy match group to exact match (#4481) by @szymon-polaczy
• Add DAG to trim and deduplicate tags (#4429) by @sarayourfriend
• Modify add_license_url DAG to use batched_update (#4370) by @krysal
• Fix Slack message formatting for ES health alert (#4345) by @AetherUnbound
• Update Science museum urls (#4276) by @stacimc
• Fix Science Museum index out of range error (#4253) by @stacimc
• Fix advanced provider options for dags that override ingest records (#4214) by @stacimc
• Extend update_license_url tasks timeout to a day and a half (#4209) by @krysal
• Don't show sources if their count has not changed (#4176) by @AetherUnbound
• Add DAG run link, explanation of missing failures to agg failure report (#4180) by @AetherUnbound

New Features

• Add backfillmoderationdecision management command (#4415) by @krysal

Bug Fixes

• More precisely handle waveform generation failures (#4444) by @sarayourfriend

New Features

• Add favicon to Storybook (#4441) by @dhruvkb

Internal Improvements

• Update dependency elasticsearch to v8.13.2 (#4418) by @openverse-bot

Bug Fixes

• Revert "Save cleaned data of Ingestion Server to AWS S3 (#4163)" (#4443) by @krysal

Internal Improvements

• Update docker.io/redis Docker tag to v7.2.5 (#4419) by @openverse-bot

Bug Fixes

• Handle tags without provider in media admin view (#4440) by @dhruvkb

Internal Improvements

• Update pnpm to v9.2.0 (#4445) by @openverse-bot
• Update dependency @playwright/test to v1.44.1 (#4424) by @openverse-bot
• Update dependency eslint-plugin-tsdoc to ^0.3.0 (#4425) by @openverse-bot
• Update dependency prettier-plugin-tailwindcss to v0.6.1 (#4426) by @openverse-bot

Bug Fixes

• Delete frontend/src/stories/ directory (#4437) by @dhruvkb
• Ensure required DB extension is installed before attempting to setup plausible (#4428) by @madewithkode