Event sourcing journal implementation using Kafka as main storage
MIT License
Stream data from two sources where one is eventually consistent and the other one loses its tail
This library provides ability to use Kafka as storage for events.
Kafka is a perfect fit in case you want to have streaming capabilities for your events.
However, it also uses Cassandra to keep data access performance on acceptable level and
overcome Kafka's retention policy
.
Cassandra is a default choice, but you may use any other storage which satisfies following interfaces:
Performance of reading events depends on finding the closest offset to the marker as well on replication latency (time difference between the moment event has been written to Kafka and the moment when event becomes available from Cassandra).
Same Kafka consumer may be shared for many simultaneous recoveries.
read
+ write
Kafka and read
Cassandraread
Kafka and read
+ write
CassandraHence, we recommend configuring access rights accordingly.
Kafka client tends to log some exceptions at error
level, however in reality those are harmless in case if operation
retried successfully. Retriable exceptions usually extend RetriableException.
List of known "error" cases which may be ignored:
In order to use kafka-journal
as akka persistence plugin
you have to add following to your *.conf
file:
akka.persistence.journal.plugin = "evolutiongaming.kafka-journal.persistence.journal"
Unfortunately akka persistence snapshot
plugin is not implemented, yet.
addSbtPlugin("com.evolution" % "sbt-artifactory-plugin" % "0.0.2")
libraryDependencies += "com.evolutiongaming" %% "kafka-journal" % "3.4.0"
libraryDependencies += "com.evolutiongaming" %% "kafka-journal-persistence" % "3.4.0"
libraryDependencies += "com.evolutiongaming" %% "kafka-journal-replicator" % "3.4.0"
libraryDependencies += "com.evolutiongaming" %% "kafka-journal-eventual-cassandra" % "3.4.0"
To run unit-test, have to have Docker environment running (Docker Desktop, Rancher Desktop etc.). Some tests expect
to have /var/run/docker.sock
available. In case of Rancher Desktop, one might need to amend local setup with:
sudo ln -s $HOME/.rd/docker.sock /var/run/docker.sock