akka-stream-kafka-template.g8
Template for Akka Streams & Kafka. Default impl: mirror a topic onto another one
Stars
14
Batteries included!
Using akka-streams, mirror one Kafka topic to another. Serve as a base building block for any Kafka Stream applications using Scala. Comes with an SBT config, and a .travis.yml that can publish straight to Docker.
Usage
$ sbt new ScalaWilliam/akka-stream-kafka-template.g8
And then look at README.md.
kafka-mirror on Docker Hub
kafka-mirror is automatically built from this template and published to Docker Hub:
Technical choices & relevant documentation
- Configuration options are parsed by Typesafe Config, the defacto configuration library for Scala.
-
akka-stream-kafka Consumer wraps the
Kafka Consumer
in a much more usable way. - The record goes through akka-streams with an offset of the input record. Akka streams
provides very powerful streaming abstractions that can be used with many connectors, including MQTT, AMQP, SQS. - Once the akka-stream-kafka Producer pushes the message the consumer offset is committed automatically. This enables resiliency.
- SBT compiles our source code. Read Essential SBT to get more familiar with SBT.
- sbt-native-packager packages the Scala application into a deployable unit with its dependencies.
- The sbt-native-packager Docker Plugin will go the extra mile to build your Docker image and publish it where you ask it to.
- By default Docker (What is Docker?) will publish to the Docker Hub which you should sign up to.
- And finally, for logging, we use logback.
Also, for sbt new support:
- Using the Giter8 templating.
-
sbt newdocumentation. - One example: https://github.com/typesafehub/stack-examples.g8
- And for reference: https://github.com/lloydmeta/scalameta.g8
Other notes
To regenerate TOC we use markdown-toc:
$ npm install -g markdown-toc
$ markdown-toc -i README.md
Licence
- MIT. Copyright (2016) Apt Elements Ltd. William Narmontas