MySQL to PostgreSQL replica system
BSD-2-CLAUSE License
Bot releases are hidden (Show)
Published by the4thdoctor over 7 years ago
Pg_chameleon is a replication tool from MySQL to PostgreSQL developed in Python 2.7 and Python 3.3+
The system relies on the mysql-replication library to pull the changes from MySQL and covert them into a jsonb object.
A plpgsql function decodes the jsonb and replays the changes into the PostgreSQL database.
The tool requires an initial replica setup which pulls the data from MySQL in read only mode.
This is done by the tool running FLUSH TABLE WITH READ LOCK; .
The tool can pull the data from a cascading replica when the MySQL slave is configured with log-slave-updates.
Published by the4thdoctor over 7 years ago
Pg_chameleon is a replication tool from MySQL to PostgreSQL developed in Python 2.7 and Python 3.3+
The system relies on the mysql-replication library to pull the changes from MySQL and covert them into a jsonb object.
A plpgsql function decodes the jsonb and replays the changes into the PostgreSQL database.
The tool requires an initial replica setup which pulls the data from MySQL in read only mode.
This is done by the tool running FLUSH TABLE WITH READ LOCK; .
The tool can pull the data from a cascading replica when the MySQL slave is configured with log-slave-updates.
Published by the4thdoctor over 7 years ago
Pg_chameleon is a replication tool from MySQL to PostgreSQL developed in Python 2.7 and Python 3.3+
The system relies on the mysql-replication library to pull the changes from MySQL and covert them into a jsonb object.
A plpgsql function decodes the jsonb and replays the changes into the PostgreSQL database.
The tool requires an initial replica setup which pulls the data from MySQL in read only mode.
This is done by the tool running FLUSH TABLE WITH READ LOCK; .
The tool can pull the data from a cascading replica when the MySQL slave is configured with log-slave-updates.
For working properly you should use virtualenv for installing the requirements via pip
The script should be executed in a screen session to keep it running. Currently there's no respawning of the process on failure nor failure detector.
The psycopg2's pip installation requires the python development files and postgresql source code.
Please refer to your distribution for fulfilling those requirements.
DDL and DML mixed in the same transaction are not decoded in the right order. This can result in a replica breakage caused by a wrong jsonb descriptor if the DML change the data on the same table modified by the DDL. I know the issue and I'm working on a solution.
Test please!
Please submit the issues you find.
Bear in mind this is an alpha release. if you use the software in production keep an eye on the process to ensure the data is correctly replicated.
Published by the4thdoctor almost 8 years ago
This is the second alpha release.
The system comes with the following limitations.
Several fixes in the DDL replica and add support for CHANGE statement.
Add support for check if process is running already, in order to avoid two replica processes run at the same time.
Port to python 3.6. This is still experimental. Any feedback is more than welcome.
For working properly you should use virtualenv for installing the requirements via pip
The script should be executed in a screen session to keep it running. Currently there's no respawning of the process on failure nor failure detector.
The psycopg2's pip installation requires the python development files and postgresql source code.
Please refer to your distribution for fulfilling those requirements.
DDL and DML mixed in the same transaction are not decoded in the right order. This can result in a replica breakage caused by a wrong jsonb descriptor if the DML change the data on the same table modified by the DDL. I know the issue and I'm working on a solution.
Test please!
Please submit the issues you find.
Bear in mind this is an alpha release. if you use the software in production keep an eye on the process to ensure the data is correctly replicated.
Published by the4thdoctor almost 8 years ago
This is the first alpha release.
The system comes with the following limitations.
For working properly you should use virtualenv for installing the requirements via pip
The script should be executed in a screen session to keep it running. Currently there's no respawning of the process on failure nor failure detector.
The psycopg2's pip installation requires the python development files and postgresql source code.
Please refer to your distribution for fulfilling those requirements.
DDL and DML mixed in the same transaction are not decoded in the right order. This can result in a replica breakage caused by a wrong jsonb descriptor if the DML change the data on the same table modified by the DDL. I know the issue and I'm working on a solution.
Test please!
Please submit the issues you find.
Bear in mind this is an alpha release. if you use the software in production keep an eye on the process to ensure the data is correctly replicated.