irmin

CHANGES:

Added

• irmin-server
  • Added irmin-server package (#2031, @zshipko)
• irmin-client
  • Added irmin-client package to connect to irmin-server instances (#2031,
    @zshipko)
• irmin
  • Add pretty printers for Commit, Tree, Info, Status, Branch when
    using utop (@metanivek, #1839)

Fixed

• irmin-pack
  • Fix index integrity check for v3 stores (#2267, @metanivek)

Removed

• irmin-http
  • Removed irmin-http since it is not compatible with generic keys.
    irmin-grapqhl or irmin-server should be used instead. (#1902, @zshipko)
• irmin
  • Removed stream proofs. We now only have Merkle tree proofs. This simplifies
    the maintenance of that part of the code, as ensuring the correct order of
    cached IO operations was tricky for stream proofs (#2275, @samoht)

Changed

• irmin-git
  • Moved lower bounds to git.3.14.0 to use new function (#2277, @metanivek)

CHANGES:

Added

• irmin-server
  • Added irmin-server package (#2031, @zshipko)
• irmin-client
  • Added irmin-client package to connect to irmin-server instances (#2031,
    @zshipko)
• irmin
  • Add pretty printers for Commit, Tree, Info, Status, Branch when
    using utop (@metanivek, #1839)

Fixed

• irmin-pack
  • Fix index integrity check for v3 stores (#2267, @metanivek)

Removed

• irmin-http
  • Removed irmin-http since it is not compatible with generic keys.
    irmin-grapqhl or irmin-server should be used instead. (#1902, @zshipko)
• irmin
  • Removed stream proofs. We now only have Merkle tree proofs. This simplifies
    the maintenance of that part of the code, as ensuring the correct order of
    cached IO operations was tricky for stream proofs (#2275, @samoht)

Changed

• irmin-git
  • Moved lower bounds to git.3.14.0 to use new function (#2277, @metanivek)

CHANGES:

Added

• irmin

  • Change behavior of Irmin.Conf.key to disallow duplicate key names by
    default. Add allow_duplicate optional argument to override. (#2252,
    @metanivek)

• irmin-pack

  • Add maximum memory as an alternative configuration option, lru_max_memory,
    for setting LRU capacity. (@metanivek, #2254)

Changed

• irmin

  • Lower bound for mtime is now 2.0.0 (#2166, @patricoferris)

• irmin-mirage-git

  • Lower bound for mirage-kv is now 6.0.0 (#2256, @metanivek)

Fixed

• irmin-cli
  • Changed --store irf to --store fs to align the CLI with what is
    published on the Irmin website (#2243, @wyn)

CHANGES:

Fixed

• irmin

  • Fix performance issue in Tree.length (#2258, @art-w)

• irmin-pack

  • Fix snapshot export when using lower layer (#2257, @metanivek)

CHANGES:

Fixed

• irmin-pack
  • Fix issue when migrating v2 stores to use lower layer (@metanivek, #2241)
  • Fix issue when calling GC for a commit in the lower after migration
    (@metanivek, #2242)

CHANGES:

Added

• irmin

  • Add Conf.pp and Conf.equal to print and compare configuration values
    (#2227, @samoht)
  • Add a clear optional arguments to all function that adds a new commit:
    Commit.v, set, set_tree, remove, test_and_set,
    test_and_set_tree, test_set_and_get, test_set_and_get_tree, merge,
    merge_tree and with_tree. This new argument allows to control whether
    the tree caches are cleared up after objects are exported to disk during
    the commit. (#2225, @samoht)

• irmin-pack

  • Add configuration option, lower_root, to specify a path for archiving data
    during a GC. (#2177, @metanivek)
  • Add is_split_allowed to check if a store allows split. (#2175, @metanivek)
  • Add add_volume to allow creating new empty volume in lower layer. (#2188,
    @metanivek)
  • Add a behaviour function to the GC to check wether the GC will archive or
    delete data. (#2190, @Firobe)
  • Add a migration on open_rw to move the data to the lower_root if
    the configuration was enabled (#2205, @art-w)

Changed

• irmin
  • Expose type equality for Schema.Info to avoid defining the info function
    multiple times when using similar stores (#2189, #2193, @samoht)
• irmin-pack
  • GC now changes its behaviour depending on the presence of a lower layer.
    (#2190, @Firobe)
  • Split now raises an exception if it is not allowed. It is not allowed on
    stores that do not allow GC. (#2175, @metanivek)
  • GC now supports stores imported V1/V2 stores, in presence of a lower layer
    only. (#2190, @art-w, @Firobe)
  • Upgrade on-disk format to version 5. (#2184, @metanivek)
  • Archive to lower volume does not copy orphaned commits. (#2215, @art-w)

Fixed

• irmin-pack
  • Unhandled exceptions in GC worker process are now reported as a failure
    (#2163, @metanivek)
  • Fix the silent mode for the integrity checks. (#2179, @icristescu)
  • Fix file descriptor leak caused by mmap. (#2232, @art-w)

CHANGES:

Fixed

• irmin-pack
  • Clear LRU when calling reload after a GC (#2200, @metanivek)

CHANGES:

Fixed

• irmin-pack
  • Clear LRU when calling reload after a GC (#2200, @metanivek)

CHANGES:

Changed

• irmin-pack
  • Improve GC reachability traversal to optimize memory, speed and remove
    the need for temporary files. (#2085, @art-w)

CHANGES:

Fixed

• irmin-pack
  • Integrity check of a commit works on stores using the minimal indexing
    strategy. (#2160, @icristescu)

CHANGES:

Added

• irmin-pack
  • Add Irmin_pack_unix.Stats.Latest_gc which is now the parameter of GC's
    finished callback (#2089, @Ngoguey42)
  • Add Gc.oldest_live_commit which returns the key of the commit on which the
    latest gc was called on. (#2110, @icristescu)
  • Add split to create a new suffix chunk. Subsequent writes will append to
    this chunk until split is called again. (#2118, @icristescu)
  • Add create_one_commit_store to create a new store from the existing one,
    containing only one commit. (#2125, @icristescu)

Changed

• irmin-pack
  • Upgraded on-disk format to version 4. (#2110, @icristescu)
  • Detecting control file corruption with a checksum (#2119, @art-w)
  • Change on-disk layout of the suffix from a single file to a multiple,
    chunked file design (#2115, @metanivek)
  • Modify GC to work with new chunked suffix. See examples/gc.ml for a
    demonstration of how it works with the new split function. (#2126,
    @metanivek)

Fixed

CHANGES:

Fixed

• irmin-pack
  • Fix read-only opening flags of mapping for read-only opening of stores that
    do not have read-write rights on the files. (#2121, @Ngoguey42)

CHANGES:

Added

• irmin

  • Add test_set_and_get* functions to retrieve commit associated with an update to the store (#2075, @patricoferris)

• irmin-graphql

  • Expose test_set_and_get function as a new mutation (#2075, @patricoferris)
  • Add contents_hash function to get a value's hash (#2099, @patricoferris)

• irmin-pack

  • Expose Gc.cancel to abort a running GC (#2101, @art-w)

• irmin-tezos-utils

  • Add package irmin-tezos-utils containing a graphical tool for manual pack
    files analysis. (#1939, @clecat)

Changed

• irmin-pack
  • irmin_pack_mem no longer exposes disk specifics functions (#2081,
    @icristescu)
  • Move unix specific details for Pack_key and Pack_value from irmin-pack
    to irmin-pack.unix (#2084, @metanivek)
  • Remove unnecessary files at open_rw and after a failed GC (#2095, @art-w)

Fixed

• irmin-pack
  • Fix data race in RO instances when reading control file (#2100, @Ngoguey42)
  • Fix bugs in gc related to commits that share the same tree. (#2106,
    @icristescu)
  • Fix the traverse pack files commands in the irmin-tezos CLI to work with
    gced stores. (#1919, @icristescu)

CHANGES:

Added

• irmin

  • Add Storage module for creating custom storage layers (#2047, @metanivek)

• irmin-pack

  • Add Gc.is_allowed (#2076, @icristescu)
  • Add a weight parameter in the LRU implementation to bound
    memory usage (#2050, @samoht)

Changed

• irmin
  • Removed Irmin_unix.set_listen_dir_hook (#2071, @zshipko)

Fixed

• irmin-pack
  • Fix the behaviour of irmin-pack regarding hashes and keys to GCed objects.
    It used to not correctly ignore these entries, which could have resulted in
    various bugs. E.g. the impossibility to append an object that used to be
    dead and that has its hash in index. (#2070, @Ngoguey42)

CHANGES:

Added

• irmin

  • Add Tree.seq to Tree's public API (#1923, @metanivek)

• irmin-fs

  • Add unix subpackage, irmin-fs.unix (#1948, @metanivek)

• irmin-git

  • Add unix subpackage, irmin-git.unix (#1948, @metanivek)

• irmin-graphql

  • Add unix subpackage, irmin-graphql.unix (#1948, @metanivek)

• irmin-http

  • Add unix subpackage, irmin-http.unix (#1948, @metanivek)

• irmin-cli

  • Add new package for irmin command-line tool (#1951, @metanivek)

• irmin-pack

  • Add a garbage collection feature, allowing a user to discard commits older
    than a specified commit. This feature is only enabled for stores that use
    Indexing_strategy.minimal. The primary API is Store.Gc.run,
    Store.Gc.wait, and Store.Gc.is_finished. See examples/gc.ml for a
    demonstration of usage.
  • Add a consistency check for the files on disk when opening store (#2004,
    @icristescu)

Changed

• irmin

  • Replaced Tree.node_fn type with more general Tree.folder type to
    represent the different ways to use Tree.fold (#1918, @metanivek)

• irmin-unix

  • Removed the irmin-unix package. Unix backends are now subpackages of their
    relevant backend (see irmin-fs.unix and irmin-git.unix). The CLI tool is
    in irmin-cli. For common unix utilities, see irmin.unix. (#1953, @metanivek)

• irmin-graphql

  • Updated to graphql.0.14.0 (#1843, @patricoferris, @zshipko)

Fixed

• irmin
  • Fix a bug in Irmin.LRU.clear that disables the cache completly
    after a clear. This is not used in any production code as only
    the GC is clearing LRU so far (#1998, @samoht)

CHANGES:

Fixed

• Add a consistency check for the files on disk when opening store
  (#2004, #2007, #@icristescu)
• Fix irmin-tezos CLI and ./tezos-node storage to work with
  v2 inodes (#1903, @icristescu, @Ngoguey42)

CHANGES:

Fixed

• irmin-pack
  • Fix topology irregularities on disk which may lead to post-gc crashes.
    (#1925, @Ngoguey42, @icristescu)

CHANGES:

Added

• irmin

  • Add Metrics module to describe metric gathering in irmin.
    (#1817, @maiste)
  • Add Repo.config to access config used to create repo
    (#1886, @zshipko)

• irmin-unix

  • Add --plugin flag to load Dynlink plugins that can register new
    contents, hash and store types (#1808, @zshipko)

• irmin-pack

  • Add use_fsync, dict_auto_flush_threshold and
  • suffix_auto_flush_threshold in store configuration. (#1865, @Ngoguey42)
  • Add no_migrate in store configuration. (#1893, @zshipko)

Changed

• irmin-pack
  • Move Irmin_pack.Pack_store.Indexing_strategy to
    Irmin_pack.Indexing_strategy and the rest of Pack_store
    to Irmin_pack_unix (#1833, @Ngoguey42)
  • Different repos opened using the same store module no longer share caches
    and file descriptors (#1852, @Ngoguey42)
  • Snapshot.Import.close requires a repo as additional argument (#1872,
    @icristescu)
  • Upgraded on-disk format to version 3 to support better synchronisation
    mechanism between readwrite and readonly instances. This change is not
    backwards-compatible with existing stores using irmin-pack.x.x < 3.3.0
    versions. A migration done by the readwrite instance is necessary to open
    older stores with irmin-pack.3.3.0. It is not forwards compatible. (#1865)
  • Rename Store.sync to Store.reload (#1900, @Ngoguey42).
  • Add Pack_error exception that centralizes many error cases alongside
    RO_not_allowed and Unix.Unix_error (#1899, @Ngoguey42)

Fixed

• irmin-pack

  • Allow snapshot export to work on indexed root nodes (#1845, @icristescu)

• irmin

  • Fix Tree.export for nodes exported twice using different repos. (#1795,
    @Ngoguey42)

CHANGES:

Fixed

• irmin-pack
  • Allow snapshot export to work on indexed root nodes (#1846, @icristescu)

CHANGES:

• Support all version of cmdliner (#1803, @samoht)