go-etl

go-etl is a toolset for data extraction, transformation and loading。(go-etl是一个集数据源抽取,转化,加载的工具集,提供强大的数据同步能力)

APACHE-2.0 License

Stars
119
Committers
3

Bot releases are hidden (Show)

What's Changed

  • feat & docs: improve go1.16 to go 1.20, change package sqlserver to github.com/microsoft/go-mssqldb,and Update all packages to the latest version. by @Breeze0806 in https://github.com/Breeze0806/go-etl/pull/41. In this pull request, we use the sqlserver driver library maintained by Microsoft.
  • feat: in package element add NewBigIntColumnValueFromUint64 and NewDe… by @Breeze0806 in https://github.com/Breeze0806/go-etl/pull/42。 In this pull request, the most importance lies in preserving precision as much as possible for data types of float64 and float32.
  • fix & style: fix the issue that datax output too much log, replace th… by @Breeze0806 in https://github.com/Breeze0806/go-etl/pull/47. We fix https://github.com/Breeze0806/go-etl/issues/46.
  • replace interface{} to any, add fuzz test and fi… …x bug -0 by @Breeze0806 in https://github.com/Breeze0806/go-etl/pull/48. In addition to fixing bugs, in this new version, we have adopted fuzz testing and the corresponding generic any type to ensure compatibility with Go 1.20, and have also deprecated the io/ioutil library among other changes.
  • feat: upgrade jodaTime by @Breeze0806 in https://github.com/Breeze0806/go-etl/pull/49. Since the original jodaTime supports the Getlayout function, we have deprecated Breeze0806/jodaTime.
  • Officially releasing the release version using go build -ldflags="-s -w" in order to reduce the program's size.

Full Changelog: https://github.com/Breeze0806/go-etl/compare/v0.1.9...v0.2.0

go-etl - v0.2.0: upgrade to go1.20, fix some issues.

Published by Breeze0806 2 months ago

What's Changed

Full Changelog: https://github.com/Breeze0806/go-etl/compare/v0.1.9...v0.2.0

go-etl - v0.1.9: The last version for v0.1, fix some bugs.

Published by Breeze0806 5 months ago

What's Changed

New Contributors

Full Changelog: https://github.com/Breeze0806/go-etl/compare/v0.1.8...v0.1.9

What's Changed

Full Changelog: https://github.com/Breeze0806/go-etl/compare/v0.1.7...v0.1.8

go-etl - v0.1.7: add no db2 release and add trimChar option in database reader

Published by Breeze0806 about 1 year ago

What's Changed

Full Changelog: https://github.com/Breeze0806/go-etl/compare/v0.1.6...v0.1.7

go-etl - v0.1.7: 新增不依赖db2的Linux程序和修复问题

Published by Breeze0806 about 1 year ago

What's Changed

Full Changelog: https://github.com/Breeze0806/go-etl/compare/v0.1.6...v0.1.7

go-etl - v0.1.6: 升级db2驱动库至v0.4.4

Published by Breeze0806 over 1 year ago

What's Changed

Full Changelog: https://github.com/Breeze0806/go-etl/compare/v0.1.5...v0.1.6

go-etl - v0.1.5: 修复db2驱动的问题以及querySql特性

Published by Breeze0806 over 1 year ago

What's Changed

Full Changelog: https://github.com/Breeze0806/go-etl/compare/v0.1.4...v0.1.5

go-etl - v0.1.4: 新增querySQL特性和调试监控端口

Published by Breeze0806 over 1 year ago

What's Changed

Full Changelog: https://github.com/Breeze0806/go-etl/compare/v0.1.3...v0.1.4

go-etl - v0.1.3: 新增监控端口和用户手册

Published by Breeze0806 over 1 year ago

What's Changed

Full Changelog: https://github.com/Breeze0806/go-etl/compare/v0.1.2...v0.1.3

go-etl - v0.1.2: 新增流控特性和简单监控特性

Published by Breeze0806 over 1 year ago

  1. 新增流控特性,可以控制通道字节数和消息数
  2. 新增简单流控特性,输出单个任务的当前全部字节数和消息数
  3. 新增切分特性最大值最小值预置值,减少最大值最小值的查询时间
  4. 修复xlsx reader和writer的部分配置不生效的问题
go-etl - v0.1.1: 新增切分键和写数据前后执行SQL功能

Published by Breeze0806 almost 2 years ago

新增特性:
1.新增Reader切分键以完成快速同步
2.新增Writer写数据前后执行SQL功能
3.新增datax批量配置向导和脚本生成
4.新增datax版本写入data version

go-etl - v0.1.0: 初次发布go-etl

Published by Breeze0806 almost 2 years ago

  1. 支持db2数据库读写数据源。
  2. 支持sql server数据库读写数据源。
  3. 支持oracle数据库读写数据源。
  4. 支持mysql数据库读写数据源。
  5. 支持postgresql数据库读写数据源。
  6. 支持cvs文件读写数据源。
  7. 支持xlsx文件读写数据源。
  8. 支持关系数据库数据源写重试。
  9. 支持批量生成数据同步配置集。
  10. 发布windows和linux 64位操作系统的二进制程序。