Lightning Memory Database (LMDB) for scala ZIO
OTHER License
Why ZIO-lmdb ? Because I wanted a very simple embedded (in the same process) ACID database for small applications while keeping deployment, maintenance, upgrades as simple as possible.
ZIO-lmdb is based on the powerful lmdb-java library and bring a higher level API in order to enhance the developer experience.
So ZIO-lmdb is an embedded key/value database, with an easy to use opinionated API, choices have been made to make the developer experience as simple as possible :
API is designed to not lie, all functions signatures describe precisely what you must expect from them, thanks to ZIO and Scala3.
For a better understanding, this library use a slightly different vocabulary from LMDB original one :
Configuration is based on the standard ZIO config mechanism, the default configuration provider uses environnment variables or java properties to resolve this library configuration parameters.
Configuration key | Environment variable | Description | Default value |
---|---|---|---|
lmdb.name | LMDB_NAME | Database name, which will be also used as the directory name | default |
lmdb.home | LMDB_HOME | Where to store the database directory | $HOME/.lmdb |
lmdb.sync | LMDB_SYNC | Synchronize the file system with all database write operations | false |
lmdb.maxReaders | LMDB_MAXREADERS | The maximum number of readers | 100 |
lmdb.maxCollections | LMDB_MAXCOLLECTIONS | The maximum number of collections which can be created | 10_000 |
lmdb.mapSize | LMDB_MAPSIZE | The maximum size of the whole database including metadata | 100_000_000_000L |
//> using scala "3.3.1"
//> using dep "fr.janalyse::zio-lmdb:1.8.0"
//> using javaOpt "--add-opens", "java.base/java.nio=ALL-UNNAMED", "--add-opens", "java.base/sun.nio.ch=ALL-UNNAMED"
import zio.*, zio.lmdb.*, zio.json.*
import java.io.File, java.util.UUID, java.time.OffsetDateTime
case class Record(uuid: UUID, name: String, age: Int, addedOn: OffsetDateTime) derives JsonCodec
object SimpleExample extends ZIOAppDefault {
override def run = example.provide(LMDB.liveWithDatabaseName("lmdb-data-simple-example"), Scope.default)
val collectionName = "examples"
val example = for {
examples <- LMDB.collectionCreate[Record](collectionName, failIfExists = false)
recordId <- Random.nextUUID
dateTime <- Clock.currentDateTime
record = Record(recordId, "John Doe", 42, dateTime)
_ <- examples.upsertOverwrite(recordId.toString, record)
gotten <- examples.fetch(recordId.toString).some
collected <- examples.collect()
_ <- Console.printLine(s"collection $collectionName contains ${collected.size} records")
_ <- ZIO.foreach(collected)(record => Console.printLine(record))
lmdb <- ZIO.service[LMDB]
_ <- Console.printLine("""LMDB standard tools can be used to manage the database content : sudo apt-get install lmdb-utils""")
_ <- Console.printLine(s"""To get some statistics : mdb_stat -s $collectionName ${lmdb.databasePath}/""")
_ <- Console.printLine(s"""To dump collection content : mdb_dump -p -s $collectionName ${lmdb.databasePath}/""")
} yield ()
}
SimpleExample.main(Array.empty)
To run the previous logic, you'll have to provide the LMDB layer, two layers are available :
LMDB.live
: Fully configurable using standard zio-configLMDB.liveWithDatabaseName("chosen-database-name")
: to override/force the database nameLMDB standard tools can be used to manage the databases content : sudo apt-get install lmdb-utils
mdb_stat -a database_directory_path/
mdb_dump -a -p database_directory_path/
mdb_dump -s collectionName -p database_directory_path/
mdb_load
which uses the same format as for mdb_dump
As zio-lmdb is using json format, dumps are just text, which can be edited and then loaded back. So simple data migration is straightforward.
When LVMDB is used as persistence store with recent JVM it requires some JVM options :
--add-opens java.base/java.nio=ALL-UNNAMED
--add-opens java.base/sun.nio.ch=ALL-UNNAMED