Addax

Addax is a versatile open-source ETL tool that can seamlessly transfer data between various RDBMS and NoSQL databases, making it an ideal solution for data migration.

APACHE-2.0 License

Stars
1.1K
Committers
25

Bot releases are visible (Hide)

Addax -

Published by wgzhao over 3 years ago

  • 24bdf314 Remove duplicate string
  • 7b11021b Httpreader support sock proxy (#177)
  • 6507fb9f bugfix: txtreader process blank or null value throws null point exception
  • 9961d5ea bugfix: Incorrect conversion of mysql year type
  • 548adf7f bugfix: convert json to string will filter null value #171 (#172)
  • 74a5ca10 Use connection pool for improve connection stable

Known issues

Addax -

Published by wgzhao over 3 years ago

6c829813 Migrates common-lang to common-lang3
b68f42c2 Use connection pool for improve connection stable
4b618933 Remove un-necessary username and password
c5b308db Extracts IP/Host from all kinds of jdbc url
151314ae Merge branch 'master' of github.com:wgzhao/DataX
eb2ddc66 Add support for reading compress files in jsonfilereader plugin
96c2f9bd Add detail info when hive type cast failure
c8ebaeb3 Merge remote-tracking branch 'github/master'
24082b3c Fixed read only the first line via each file
ce07a1bf improve retention-policy implement in InfluxDBWriter plugin #162
125f9d1e influxdb: fix column maybe not have time (#160)
9bebd7ae Add retentionPolicy option in plugin InfluxDBWriter #162
810965e7 Fixed #157 javadoc syntax error
a04ccc3c Merge branch 'master' of github.com:wgzhao/DataX
f89ee158 Fixed #158 main class not found in Windows and in-correct encoding in Windows
0e3f4e8e Fixed protobuf in-compliant version
8e7f910f Fixed printf error
e10a0f08 Fixed javadoc syntax error #157
339ae8fa 1. Fixed #156 exception occurred when writing Parquet file with null value 2. Optimize Decimal type
55079bac Fixed #155 jdbc java runtime version compliant exception

Addax -

Published by wgzhao over 3 years ago

General Changes

  • Bump protobuf-java from 2.5.0 to 3.15.5
  • Bump mssql-jdbc from 8.4.0.jres8 to 9.2.1.jre15
  • Add missing ElasticSearchReader package
  • Add support for get version in datax.py

RDBMS Reader

  • Fixed get XML data as string instead of SQLXML handler

RDBMS Writer

  • Add support XML data type

Stream Reader

  • Add field value incrment function

InfluxDB Reader

  • Fixed job template file
  • Add timeout configure items
  • Use native influxDB API instead of influxdb-java (#153)

InfluxDB Writer

  • Add timeout configure items

Textfile Reader

  • Add support for more compress format (#154)
Addax -

Published by wgzhao over 3 years ago

  1. Fixed: unknown host exception when writing parquet file #135
  2. Documentation streamreader and streamwriter
  3. Add support for DorisDB writer #131
  4. Add support more than one channel setup for InfluxDB #130
  5. Add support for InfluxDB TAG data type #125
  6. Add support for TDengine reader/writer #119
  7. Add flattend index type
Addax -

Published by wgzhao over 3 years ago

Addax -

Published by wgzhao over 3 years ago

repackage to com.wgzhao

Addax -

Published by wgzhao over 3 years ago

General Changes

  • Add HTTP reader plugin
  • make package.xml more general
Addax -

Published by wgzhao almost 4 years ago

3.1.8

General Changes

  • Add Kudu reader plugin
  • Add kudu writer plugin
  • Add Greenplum writer plugin
  • Add InfluxDB reader plugin
  • Add InfluxDB Writer plugin
  • Change loglevel to reduce console output
  • Add more exit code
Addax -

Published by wgzhao almost 4 years ago

General Changes

  • Add support for more data type such as Array, UUID, JSON, XML
  • Add more deme job files
  • Add support for HBase 2.0 reader ( hbase20xreader )
  • Fixes for some minor issues
  • Update some sample job json-string for documentation

ClickHouse reader

  • Add support for DateTime, DateTime64 data type

ClickHouse writer

  • Add support for DateTime, DateTime64 data type

PostgreSQL reader

  • Add support for UUID,JSON, INET,MACADDR data type

PostgreSQL writer

  • Add support for UUID,JSON, INET,MACADDR data type

SQLServer reader

  • Add support for image data type

SQLServer writer

  • Add support for image data type

Oracle writer

  • Fixed incorrect column cuased by duplicate variable initialization (#92)

Transformer

  • Add power expressions (^) for dx_map function
Addax -

Published by wgzhao almost 4 years ago

General Changes

  • Add tar.gz release package
  • Takes item password to optional instead of required one

Redis writer

  • Fixed incorrect target packaging folder (#81)

Oracle writer

  • Add support for merge into statement via configuring wirteMode item (#82)

PostgreSQL writer

  • Add support for insert into ... on conflict statement via configuring wirteMode item (#82)
Addax - 3.1.5

Published by wgzhao almost 4 years ago

Release Notes

General Changes

  • Various code clean

DBF reader

  • Reconstruct this plugin with 3rd-party jar package
  • Add support for Date type
  • Fix for the occasional null pointer exception

DBF writer

  • Add support for Date type

HDFS reader

  • Fix read failure when the filetype is text (#66)

Elasticsearch writer

  • Fixed missing dependency jars (#68)
Addax - 3.1.4

Published by wgzhao almost 4 years ago

This is an emergency fix version

Addax -

Published by wgzhao almost 4 years ago

There are serious problems with this version, do not use it!

Release 3.1.3 (16 Nov 2020)

General Changes

  • Various code clean
  • Misc bugs fixed

Redis reader

  • Delete temporary local file
  • Only parse redis String data type, other types will be ignore

HDFS reader

  • Add support for reading Parquet file (#54)
Addax - release 3.1.2

Published by wgzhao about 4 years ago

Changelog

General

  • Does not parse the -m command line argument, it doesn't really do anything!

SQLServer reader

  • Add support for datetime data type with old SQLServer edition

HBase11xsql writer

  • Add support for truncating table before inserting records
  • Add support for specifing zookeeper's port, if not presents, it uses 2181 as default port

HDFS writer

  • Add support for timestamp data type

MongoDB reader

  • Add support for json data type
Addax - 3.1.1

Published by wgzhao about 4 years ago

3.1.1

General

  • Transformer add column's basic operation
  • Use prestosql's hadoop and hive jars instead of apache's
  • various misc codes optimize

dbffilereader

  • remove supported for reading compressed dbf file

jsonreader

  • fixed parse non-string type value

dbffilewriter

  • fixed boolean type writing error

hdfswriter

  • Use keyword parquest indicates support parquet format, old keyword par is not used