equivalent to kafka-connect for nodejs
MIT License
The framework can be used to build connectors,
that transfer data to
and from
Apache Kafka and Databases,
very easily. If you are looking for already implemented connectors
for you favorite datastore, take a look at the Available Connector Implementations
below.
kafka -> datastore
& datastore -> kafka
data movements.If you are using the native mode (config: { noptions: {} }
).
You will have to manually install node-rdkafka
alongside kafka-connect.
(This requires a Node.js version between 9 and 12 and will not work with Node.js >= 13, last tested with 12.16.1)
On Mac OS High Sierra / Mojave:
CPPFLAGS=-I/usr/local/opt/openssl/include LDFLAGS=-L/usr/local/opt/openssl/lib yarn add --frozen-lockfile [email protected]
Otherwise:
yarn add --frozen-lockfile [email protected]
(Please also note: Doing this with npm does not work, it will remove your deps, npm i -g yarn
)
yarn add kafka-connect
const source = new TestSourceConfig(config,
TestSourceConnector,
TestSourceTask,
[TestConverter]);
source.run().then();
const sink = new TestSinkConfig(config,
TestSinkConnector,
TestSinkTask,
[TestConverter]);
sink.run().then();
DEBUG=kafka-connect:*
to debug the sink configuration.