Persistent and in-memory key-value storage engine for JVM that scales on a single machine.
APACHE-2.0 License
Bot releases are hidden (Show)
MultiPrepareBuilder
for creating MultiPrepare
instance for MultiMap
from Java.Bag.Less
to Glass
and Bag.less
instance to Bag.glass
. Glass
being a transparent type.BAG[_]
type param from getKeyValueDeadline
functionsPublished by simerplaha about 4 years ago
Implemented a faster NavigableMap
(for SequentialOrder
writes) backed by Array
and binary-search which can be enabled for applications that only require sequential writes and do not need random writes.
The following shows benchmarks of SequentialOrder
(Array) vs RandomOrder
(java.ConcurrentSkipListMap) for sequential writes.
Default setting optimises writes for RandomOrder
. The following code samples optimise writes for SequentialOrder
.
initialSkipListLength
sets the length of the Array
since SequentialOrder
is backed by Array
.
Map<Integer, String, Void> map =
MemoryMap
.functionsOff(Default.intSerializer(), Default.stringSerializer())
.setOptimiseWrites(OptimiseWrites.sequentialOrder(10000))
.get();
val map =
memory.Map[Int, String, Nothing, Bag.Less](
optimiseWrites = OptimiseWrites.SequentialOrder(initialSkipListLength = 10000)
)
Previously SkipList
required copying in-memory key-values for atomicity (#124). Now LevelZero
performs atomic reads without copying.
Default setting disables Atomic
which can be enabled with the following.
Map<Integer, String, Void> map =
MemoryMap
.functionsOff(Default.intSerializer(), Default.stringSerializer())
.setAtomic(Atomic.on())
.get();
val map =
memory.Map[Int, String, Nothing, Bag.Less](atomic = Atomic.On)
Previously reads were creating a snapshot which provided the state of write-ahead-log files in LevelZero
for each read. Now snapshotting happen lazily (Stream like) as the read progresses. This change was aimed to further minimise GC workload.
Published by simerplaha about 4 years ago
MultiMap
and SetMap
are now implemented for Java. Java API is now complete.
MultiMap
allows creating deeply nested Map
.
//create a root map
MultiMap<String, Integer, String, Void> root =
MemoryMultiMap.functionsOff(
stringSerializer(), //nested Map key type
intSerializer(), //key type
stringSerializer() //value type
).get();
//first child map
MultiMap<String, Integer, String, Void> child1 = root.child("first child");
child1.put(1, "child1's value 1");
SetMap
provides Map
like API for Set
. Useful for application that always fetch the value with the key. It stores both key & value in the same location for faster seeks and reduced IOps.
SetMap<Integer, String> setMap =
MemorySetMap
.config(intSerializer(), stringSerializer())
.get();
setMap.put(1, "one");
MultiMap
previously stored parent key bytes within child key bytes. Now deeply nested Map
do not have any extra storage cost over a root MultiMap. This also improved read and write performance.
Deeply nested MultiMap
has constant performance regardless of the depth.
//get all children of root map
Stream<MultiMap<String, Integer, String, Void>> children = root.children();
//get all children of root map and all it's grand children.
Stream<MultiMap<String, Integer, String, Void>> child = root.childrenFlatten();
Create functions without need to specify the return type. All java functions types are under PureFunctionJava
.
//function that appends "updated" to old value
OnValue<Integer, String> myFunction =
(String value) ->
Apply.update(value + " updated");
Register the function as a List
.
Map<Integer, String, PureFunction<Integer, String, Apply.Map<String>>> map =
PersistentMap
.functionsOn(
Paths.get("my_map"), //directory
Default.intSerializer(), //key serialiser
Default.stringSerializer(), //value serializwr
Arrays.asList(myFunction) //functions list
).get();
Syntax shortened for enabling functions - PureFunction.Map[Int, String]
. All Scala function types are under PureFunctionScala
.
val map = persistent.Map[Int, String, PureFunction.Map[Int, String], Bag.Less]("my_map")
To register function supply implicit
functions List.
//sample function that appends "updated" to old value
val myFunction: OnValue[String] = (value: String) => Apply.Update(value + " updated")
implicit val myFunctions = Functions[PureFunction.Map[Int, String]](myFunction)
OnKey
- reads the keyOnKeyDeadline
(Scala) & OnKeyExpiration
(Java) - reads the key and deadlineOnKeyValue
- reads the key and valueOnValue
- reads the valueOnValueDeadline
(Scala) & OnValueExpiration
(Java) - reads the value and deadlineOnKeyValueValueDeadline
(Scala) & OnKeyValueValueExpiration
(Java) - read the key, value and deadline.Use Slice<T>
to easily build custom serialisers. ByteOps.Java
removes the need to cast java.lang.Byte
to scala.Byte
during runtime.
new Serializer<String>() {
@Override
public Slice<Byte> write(String data) {
return Slice.writeStringUTF8(data, ByteOps.Java());
}
@Override
public String read(Slice<Byte> slice) {
return slice.readStringUTF8(ByteOps.Java());
}
};
Easily serialise case class
es using Bookpickle
import swaydb._
import boopickle.Default._
case class MyClass(id: Int, value: String)
implicit val serialiser = swaydb.serializers.BooPickle[MyClass]
val set = memory.Set[MyClass, Nothing, Bag.Less]()
Scala functions can be shared with Java and vice versa. Scala instance can also be accessed from Java.
Set<Integer, Void> mySet =
MemorySet
.functionsOff(intSerializer())
.get();
mySet.asScala(); //get scala set
Missing applied functions reported on re-boot to ensure safe startups.
swaydb.Exception$MissingFunctions: Missing 1 function. List("myFunction").
Functions that are applied can be cleared by setting the clearAppliedFunctionsOnBoot
flag.
val map = persistent.Set[Int, PureFunction.Set[Int], Bag.Less]("my_set", clearAppliedFunctionsOnBoot = true)
Set<Integer, PureFunction<Integer, Void, Apply.Set<Void>>> mySet =
PersistentSet
.functionsOn(Paths.get("my_set"), intSerializer(), Arrays.asList())
.setClearAppliedFunctionsOnBoot(true)
.get();
Incompatible SwayDB version files get reported on boot-up.
Eg: Current v0.16 is incompatible with older versions so start v0.16 on older versions will result in the following error
Incompatible versions! SwayDB v0.16 is not compatible with files created by v0.15
Directories are type-checked. Eg: starting Set
on a Map
directory will yield
Invalid data type Set for the directory of type Map.
Published by simerplaha about 4 years ago
Map
, Set
, Queue
... ) use the bag from within the data-type.eventually.persistent.SetMap
added eventually.persistent.Queue
build.info
file which stores the version and used to perform compatibility checks.MultiMap
is not experimental anymore.Breaking change - with the addition of build.info
file, previous versions are incompatible with this version.
Published by simerplaha about 4 years ago
Resolved issues
#260 - Rename MapConfig, SetConfig & QueueConfig in Java to reflect their storage types
#259 - Remove Function type param from SetMap
#258 - Java's Set.stream is returning Stream instead of Source
#257 - Add asJava to interop with java.util.Map
#256 - Terminate Bag on close
#255 - Implement Source for Stream
#253 - Add Stream.materializeBuilder for custom collection type
#252 - SwayDBExpireSpec test failing
Published by simerplaha about 4 years ago
This release fixes issues relating to memory-mapped files not working on Windows and other important issues.
#156 - program won't end (resolved) & java.nio.file.AccessDeniedException
on Windows
#251 - [WINDOWS] - MappedByteBuffer.force
is extremely slow on Windows Performance
#250 - Compaction does not progress on reboot - ArrayIndexOutOfBoundsException
#249 - ByteBufferSweeper Actor can be a basic Actor
See issue #156 for more details on new ForceSave
and MMAP
configurations to handle memory-mapped files on Windows.
Published by simerplaha about 4 years ago
Resolves important issues.
Published by simerplaha about 4 years ago
#240 - java
and data-java
modules should not disable cross builds.
Reverts change made to java artifact id in previous release - v0.14.3.
<dependency>
<groupId>io.swaydb</groupId>
<artifactId>java_2.13</artifactId>
<version>0.14.4</version>
</dependency>
Published by simerplaha about 4 years ago
Resolves following bugs.
#238 - LevelZero's compaction is not updating Maps.currentMapsSlice safely
#236 - Stream returns missing results when transitioning level0 entries to level1 for databases with mapSize = 1.byte
#237 - Actor pauses processing messages
#239 - Convenience functions for Java to overwrite default Configs parameters from Scala.
Java's artifactId changed to java
instead of java_2.13
Maven dependency looks like
<dependency>
<groupId>io.swaydb</groupId>
<artifactId>java</artifactId>
<version>0.14.3</version>
</dependency>
Published by simerplaha about 4 years ago
Fixes
#237 - Actor pauses processing messages
Published by simerplaha about 4 years ago
#236 - Stream returns missing results when transitioning level0 entries to level1 for databases with mapSize = 1.byte
Published by simerplaha about 4 years ago
Major release for MultiMap
type which allows SQL's Table
like structure.
#227 - Implement MultiMap feature
#233 - Change all data types to return the BAG type?
#230 - SegmentReadState should be invalidated bug
#229 - Bump versions
#228 - delete does not delete all database files bug
Published by simerplaha over 4 years ago
Java API improvements
Published by simerplaha over 4 years ago
Published by simerplaha over 4 years ago
Published by simerplaha over 4 years ago
Published by simerplaha over 4 years ago
persistent.Queue[T]
and memory.Queue[T]
#47comparableKey
should be an internal API #197IO
when providing KeyOrder
or KeyComparator
#198Queue
should not use SetMap
#204String
to name functionId instead of Slice[Byte]
#203MapSet
data type #196Queue
#200Published by simerplaha over 4 years ago
Closed
Published by simerplaha over 4 years ago
Fixed #190 - Simplify syntax for defining function type Improvement java
Published by simerplaha over 4 years ago
Closes - Synchronous Steps should also suspend operations to recover from failures #192