aodbm - Append Only Database Manager

An append only database manager in the style of dbm. It has both a C and a Python interface and is ACID compliant. Internally it uses a B+ Tree. (Being append only doesn't mean that you can't replace records or delete keys).

Getting Started

$ make

This will build a static and a shared library.

Python interface

import the module

import aodbm

create a database object

db = aodbm.AODBM('testdb')

get the current version of the database

version = db.current_version()

versions updated either in place or by creating new versions

in place:

version['hello'] = 'world' version['hello'] "world" list(version) [('hello', 'world')] del version['hello']

new versions

new_version = version.set_record('hello', 'world') new_version['hello'] "world" version['hello'] Traceback (most recent call last): File "", line 1, in File "aodbm.py", line 56, in getitem raise KeyError() KeyError newest_version = new_version.del_key('hello')

commiting a new version is simple

db.commit(new_version)

it returns whether the operation was successful

True

the operation may fail if you try to commit a version that isn't based on the

the current version.

C interface

You'll need to link against either the static or shared library and include the header file aodbm.h in order to use the C API.

Before you can do anything, you will need a handle for a database. This is simple to acquire, simply call aodbm_open passing the filename as a NULL terminated string, the second argument is for flags. There are no flags at the moment so just pass 0. This will return an "aodbm *", when you are done with the handle then close the database using aodbm_close. These functions do not do any filelocking so ensure that only one handle exists for a given database file at any time.

Once you have a handle, the next step is to obtain a reference to the most current version of the database. Versions are represented as "aodbm_version"s. Under the hood these are just "uint64_t"s, so don't worry about freeing them. The most current version can be found using aodbm_current.

By explicitly dealing with versions of the database, you can be sure that other threads will not interfere during database operations.

There are four database operations: get, set, delete and has. They appear in the respective functions aodbm_get, aodbm_get, aodbm_del and aodbm_has. Each deals with "aodbm_data *"s. The aodbm_data structure represents a piece of data and is defined like so:

struct aodbm_data { char *dat; size_t sz; };

Each of the operations' function's first three arguments are the database handle, the version of the database on which to operate on and the key to which the operation relates to. aodbm_get return a "aodbm_data *" which is populated with the value at the key or NULL if the record doesn't exist. When you have finished using the value from this operation, you should free it with aodbm_free_data. aodbm_set takes one extra argument, the value to store at the given key. aodbm_set and aodbm_del both return the new version of the database that you have created.

The aodbm_data objects that you provide are not modified in any way.

Iteration is also possible. First you must create an iterator using aodbm_new_iterator, passing in the database handle and the version to iterate over. Then call aodbm_iterator_next, passing in the database handle and the iterator. This call is not thread-safe. It returns an aodbm_record that will be filled with data (a key and a value) that you should free using aodbm_free_data when you are finished using them. The key and value will be set to NULL when the end of the database is reached. When you are finished using an iterator, whether you have reached the end of the database or not, you should free the iterator with aodbm_free_iterator. The records will be given in the order they are stored in. The less than operator is defined like so:

Given 2 keys, a and b. if a is shorter than b return true, if b is shorter than a return false, return the lexicographic ordering

Having modified the database, you will likely want to commit the changes. Commiting the changes means that when future requests for the current version are made, your new version of the database will be returned. To commit your changes, pass your new version as the second argument to aodbm_commit. This will return a boolean value which indicates whether the commit was successful. A commit will fail if you try to commit a version that is not based on the current latest version.

This only leaves two functions that haven't been covered in the public API. aodbm_is_based_on and aodbm_previous_version. They both do exactly what you think they'd do. aodbm_is_based_on takes two arguments in addition to the database handle and they are both aodbm_versions. It returns whether the first is based on the second.

Documentation

You can find more information at https://sourceforge.net/apps/mediawiki/aodbm

Current Progress

It is not yet suitable for production usage. There are a few known bugs in the delete function and I'm going to be refactoring much of the code. I consider it to be of (early) beta quality.

The goal of the project is to write a storage backend for a future DBMS. It should therefore be:

• Fast,
• ACID compliant,
• As simple as possible.

Reporting Bugs

If you find a bug then please write a unit test (in Python if possible) that fails and send it to me (Daniel) at [email protected] .

Thanks