pgqueuer

What's Changed

• Update classifiers to Production/Stable by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/116
• Update PGQueuer in docs by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/117

Full Changelog: https://github.com/janbjorge/pgqueuer/compare/v0.8.8...v0.8.9

What's Changed

• Illformed docker build by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/115

Full Changelog: https://github.com/janbjorge/pgqueuer/compare/v0.8.7...v0.8.8

What's Changed

• Log exception now says job-id by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/112
• Use uv as package and project manager by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/111
• Push docker latest tag by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/113
• Refactor qm run, use context manager for setup/teardown by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/114

Full Changelog: https://github.com/janbjorge/pgqueuer/compare/v0.8.6...v0.8.7

What's Changed

• Gracefull shutdown benchmarking script by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/106
• Use async.Event for alive flagging by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/107
• Add tm to db-drivers by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/108
• Bugfix: Canceled logged as successful by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/110

Full Changelog: https://github.com/janbjorge/pgqueuer/compare/v0.8.5...v0.8.6

New Feature: Concurrency Limiting! 🚀

Im excited to announce Concurrency Limiting in pgqueuer, enhancing your ability to manage job processing effectively.

• Concurrency Limiting: Controls the number of jobs that can be processed concurrently per entrypoint.

Example Usage:

from pgqueuer.qm import QueueManager
from pgqueuer.models import Job

# Setup QueueManager
qm = QueueManager()

@qm.entrypoint("data_processing", concurrency_limit=5)
async def process_data(job: Job):
    # Job processing logic
    pass

What's Changed

• Entrypoint concurrency limiting (with tests & docs) by @colingavin in https://github.com/janbjorge/pgqueuer/pull/103
• Bugfix: Invoking .clear on events to eagerly by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/104

Full Changelog: https://github.com/janbjorge/pgqueuer/compare/v0.8.4...v0.8.5

Shutout to @colingavin for another great submission ❤️

🚀 New Feature: Prometheus Metrics Integration in pgqueuer!

Im thrilled to unveil a new enhancement to the pgqueuer library — Prometheus Metrics Integration! This feature brings monitoring capabilities, enabling you to track and analyze real-time performance metrics of your job queues seamlessly.

Quick Setup Using Docker:

Setting up the Prometheus metrics service is straightforward using Docker and can be done in just a few steps. Here's how:

1. Pull the Docker Image:
   Ensure you have the latest version of the Prometheus service by pulling the image from the GitHub Container Registry:

   # Pull the latest image
   docker pull ghcr.io/janbjorge/pgq-prometheus-service:latest
   

2. Run the Docker Container:
   Use the following command to run the service, ensuring to replace the environment variables with your specific PostgreSQL connection details:

   # Run the service
   docker run -p 8000:8000 -e PGHOST=your-postgres-host -e PGDATABASE=your-database -e PGPASSWORD=your-password -e PGUSER=your-username -e PGPORT=5432 ghcr.io/janbjorge/pgq-prometheus-service
   

   This command starts the Prometheus metrics service and makes it accessible at http://localhost:8000/metrics, ready for Prometheus to scrape.

Full Changelog: https://github.com/janbjorge/pgqueuer/compare/v0.8.3...v0.8.4

Full Changelog: https://github.com/janbjorge/pgqueuer/compare/v0.8.2...v0.8.3

What's Changed

• Buffer contains a refernce to queries class, removed callback by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/99
• Standardize imports by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/100
• Prometheus metrics service by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/97

Full Changelog: https://github.com/janbjorge/pgqueuer/compare/v0.8.1...v0.8.2

What's Changed

• Add configuration arguments to CLI run subcommand by @colingavin in https://github.com/janbjorge/pgqueuer/pull/96

Full Changelog: https://github.com/janbjorge/pgqueuer/compare/v0.8.0...v0.8.1

Breaking Change

Before using this version, it is essential to run the database schema upgrade to ensure compatibility with the new features and improvements introduced. Please execute the following command before deploying the new version:

python -m pgqueuer upgrade

This step updates the database schema, including necessary modifications for the job cancellation feature and other enhancements that are critical for the new version to function correctly.

What's Changed

• Bugfix: TM gather exits on first exception by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/90
• Standard query format by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/92
• Add JobExecutor class by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/93
• Use Counter for entrypoint_tally by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/94
• Feature: Job cancellation by @janbjorge in https://github.com/janbjorge/pgqueuer/pull/91
• Normalize line endings when building test docker image by @colingavin in https://github.com/janbjorge/pgqueuer/pull/95

New Contributors

• @colingavin made their first contribution in https://github.com/janbjorge/pgqueuer/pull/95

Full Changelog: https://github.com/janbjorge/pgqueuer/compare/v0.7.2...v0.8.0

Full Changelog: https://github.com/janbjorge/pgqueuer/compare/v0.7.1...v0.7.2

Full Changelog: https://github.com/janbjorge/pgqueuer/compare/v0.7.0...v0.7.1

Change

Breaking Change

• Renamed PgQueuer -> pgqueuer - Changed the package name to lowercase to adhere to community standards, as suggested by @skhaz in https://github.com/janbjorge/pgqueuer/discussions/85. This modification might require updates in dependent codes.

Full Changelog: https://github.com/janbjorge/pgqueuer/compare/v0.6.6...v0.7.0

Full Changelog: https://github.com/janbjorge/pgqueuer/compare/v0.6.4...v0.6.6

Full Changelog: https://github.com/janbjorge/pgqueuer/compare/v0.6.4...v0.6.5

Full Changelog: https://github.com/janbjorge/PgQueuer/compare/v0.6.3...v0.6.4

What's Changed

• Update query builder doc-strings by @janbjorge in https://github.com/janbjorge/PgQueuer/pull/77
• Add optional dependencies asyncpg/psycopg by @janbjorge in https://github.com/janbjorge/PgQueuer/pull/79
• Update dev dependencies by @janbjorge in https://github.com/janbjorge/PgQueuer/pull/80

Full Changelog: https://github.com/janbjorge/PgQueuer/compare/v0.6.2...v0.6.3

What's Changed

• Fix a few annoyances in docs. by @janbjorge in https://github.com/janbjorge/PgQueuer/pull/75

Full Changelog: https://github.com/janbjorge/PgQueuer/compare/v0.6.1...v0.6.2

What's Changed

• CI test with more pg versions by @janbjorge in https://github.com/janbjorge/PgQueuer/pull/69
• Job model id uses job-id type by @janbjorge in https://github.com/janbjorge/PgQueuer/pull/71
• Make ruff max line length 100 by @janbjorge in https://github.com/janbjorge/PgQueuer/pull/72
• Improve documentation by @poksiala in https://github.com/janbjorge/PgQueuer/pull/70

New Contributors

• @poksiala made their first contribution in https://github.com/janbjorge/PgQueuer/pull/70

Full Changelog: https://github.com/janbjorge/PgQueuer/compare/v0.6.0...v0.6.1

Release Notes

What's Changed

• apg vs. psy benchmark section by @janbjorge in https://github.com/janbjorge/PgQueuer/pull/61
• Remove superfluous use of dsn func by @janbjorge in https://github.com/janbjorge/PgQueuer/pull/62
• Update docker compose ci by @janbjorge in https://github.com/janbjorge/PgQueuer/pull/64
• Use relative imports by @janbjorge in https://github.com/janbjorge/PgQueuer/pull/66
• Entrypoint rate limit by @janbjorge in https://github.com/janbjorge/PgQueuer/pull/65

Breaking Changes

Important: Before upgrading to this version, you must run the following command:

python3 -m PgQueuer upgrade

Alternatively, you can manually update the trigger function before booting up the new version.

    CREATE OR REPLACE FUNCTION fn_pgqueuer_changed() RETURNS TRIGGER AS $$
    DECLARE
        to_emit BOOLEAN := false;  -- Flag to decide whether to emit a notification
    BEGIN
        -- Check operation type and set the emit flag accordingly
        IF TG_OP = 'UPDATE' AND OLD IS DISTINCT FROM NEW THEN
            to_emit := true;
        ELSIF TG_OP = 'DELETE' THEN
            to_emit := true;
        ELSIF TG_OP = 'INSERT' THEN
            to_emit := true;
        ELSIF TG_OP = 'TRUNCATE' THEN
            to_emit := true;
        END IF;

        -- Perform notification if the emit flag is set
        IF to_emit THEN
            PERFORM pg_notify(
                'ch_pgqueuer',
                json_build_object(
                    'channel', 'ch_pgqueuer',
                    'operation', lower(TG_OP),
                    'sent_at', NOW(),
                    'table', TG_TABLE_NAME,
                    'type', 'table_changed_event'
                )::text
            );
        END IF;

        -- Return appropriate value based on the operation
        IF TG_OP IN ('INSERT', 'UPDATE') THEN
            RETURN NEW;
        ELSIF TG_OP = 'DELETE' THEN
            RETURN OLD;
        ELSE
            RETURN NULL; -- For TRUNCATE and other non-row-specific contexts
        END IF;

    END;
    $$ LANGUAGE plpgsql;

Full Changelog: https://github.com/janbjorge/PgQueuer/compare/v0.5.7...v0.6.0