pgqueuer

PgQueuer is a Python library leveraging PostgreSQL for efficient job queuing.

MIT License

Downloads
2.6K
Stars
950

Bot releases are visible (Hide)

pgqueuer - v0.5.8

Published by janbjorge 3 months ago

pgqueuer - v0.5.7

Published by janbjorge 3 months ago

What's Changed

Full Changelog: https://github.com/janbjorge/PgQueuer/compare/v0.5.6...v0.5.7

pgqueuer - v0.5.6

Published by janbjorge 3 months ago

What's Changed

Full Changelog: https://github.com/janbjorge/PgQueuer/compare/v0.5.5...v0.5.6

pgqueuer - v0.5.5

Published by janbjorge 3 months ago

What's Changed

Full Changelog: https://github.com/janbjorge/PgQueuer/compare/v0.5.4...v0.5.5

pgqueuer - v0.5.4

Published by janbjorge 3 months ago

What's Changed

Full Changelog: https://github.com/janbjorge/PgQueuer/compare/v0.5.3...v0.5.4

pgqueuer - v0.5.3

Published by janbjorge 4 months ago

What's Changed

Full Changelog: https://github.com/janbjorge/PgQueuer/compare/v0.5.2...v0.5.3

pgqueuer - v0.5.1

Published by janbjorge 4 months ago

What's Changed

Full Changelog: https://github.com/janbjorge/PgQueuer/compare/v0.5.1...v0.5.2

pgqueuer - v0.5.1

Published by janbjorge 4 months ago

pgqueuer - v0.5.0

Published by janbjorge 4 months ago

Release Notes

Breaking

  • Simplified Database Connection: The system no longer requires a connection pool. Instead, a single database connection is now used.

Major

  • Database Abstraction Layer: Introduction of db.py to provide database driver abstractions, supporting both AsyncPG and Psycopg drivers for PostgreSQL.
  • Performance Boost: Introducing the JobBuffer class has significantly increased job processing performance, now handling up to 18,000 jobs per second, up from 5,000 jobs per second.
  • Buffer Management: The JobBuffer class to handle job accumulation and flushing based on capacity or timeout, improving job processing efficiency.

Minor

  • Dependency Management: Adjustments in pyproject.toml to include and better manage dependencies such as asyncpg and psycopg.
  • Queue Manager Updates: Significant enhancements in QueueManager to integrate the new JobBuffer, optimizing job handling and logging.
  • CLI Improvements: Enhancements to cli.py to support the new driver implementations and streamline command execution.
  • Benchmarking: Updates to benchmarking scripts for better performance tracking and configuration options.

These changes aim to enhance performance, improve code maintainability.

Full Changelog: https://github.com/janbjorge/PgQueuer/compare/0.4.1...v0.5.0

pgqueuer - 0.4.1

Published by janbjorge 5 months ago

What's Changed

Full Changelog: https://github.com/janbjorge/PgQueuer/compare/0.4.0...0.4.1

pgqueuer - 0.4.0

Published by janbjorge 5 months ago

Notice of Breaking Change

The latest update introduces a breaking change due to the addition of a new updated column to the queue table with a NOT NULL constraint. Users will need to run the upgrade command to update their database schema accordingly. Please ensure you execute the following command after upgrading to avoid any disruptions.

python -m PgQueuer upgrade

This will trigger the below addition a column and creation of an index.

ALTER TABLE pgqueuer ADD COLUMN updated TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW();
CREATE INDEX pgqueuer_updated_id_id1_idx ON pgqueuer (updated ASC, id DESC) INCLUDE (id) WHERE status = 'picked';

What's Changed

Full Changelog: https://github.com/janbjorge/PgQueuer/compare/v0.3.6...0.4.0

pgqueuer - v0.3.6

Published by janbjorge 5 months ago

What's Changed

Full Changelog: https://github.com/janbjorge/PgQueuer/compare/v0.3.5...v0.3.6

pgqueuer - v0.3.5

Published by janbjorge 5 months ago

What's Changed

Full Changelog: https://github.com/janbjorge/PgQueuer/compare/v0.3.4...v0.3.5

pgqueuer - v0.3.4

Published by janbjorge 5 months ago

What's Changed

Full Changelog: https://github.com/janbjorge/PgQueuer/compare/v0.3.3...v0.3.4

pgqueuer - v0.3.3

Published by janbjorge 6 months ago

What's Changed

Full Changelog: https://github.com/janbjorge/PgQueuer/compare/v0.3.2...v0.3.3

pgqueuer - v0.3.2

Published by janbjorge 6 months ago

pgqueuer - v0.3.1

Published by janbjorge 6 months ago

What's Changed

Full Changelog: https://github.com/janbjorge/PgQueuer/compare/v0.3.0...v0.3.1

pgqueuer - v0.3.0

Published by janbjorge 6 months ago

pgqueuer - v0.2.4

Published by janbjorge 6 months ago

pgqueuer - v0.2.2

Published by janbjorge 6 months ago

What's Changed

Full Changelog: https://github.com/janbjorge/PgQueuer/compare/v0.2.2...v0.2.3