hash_ring

hash_ring

Stars
14

Bot releases are visible (Hide)

hash_ring - hotfix - pytest in setup.py Latest Release

Published by jhgg over 6 years ago

Removes that for now, so it can install.

hash_ring - HashRing.add_nodes

Published by jhgg over 6 years ago

Merge #2
Rebuilding the ring each time the ring changes turns out to be a really slow operation. This hopes to address this by creating an add_nodes operation that can be performed on an empty ring that is faster than individually calling add_node for each node in the ring. So far, the benchmarks are promising, the add_nodes operation taking 2% the wall time of calling add_node individually for larger ring sizes. Right now, this operation will run in the event loop of our API servers, so making it go faster is a good way to reduce the impact of re-building the ring.

Here is a result of the benches of adding nodes in the slow (add_node) vs fast (add_nodes) path, for 512 replicas per node:

image

  • add HashRing.add_nodes(nodes) method.
  • add HashRing.get_num_items() method for debugging.
  • add HashRing.get_node(node_name) method for debugging.
  • add hash_ring_sort method that will sort the ring items array after we're done adding stuff to
    it.
  • add hash_ring_ensure_items_size which will allow us to pre-allocate the items array, so we do not need to needlessly realloc it to grow, if we can compute the target size before hand.
  • mod hash_ring_add_node to take a doSort param, to defer sorting of the items array.
  • formalize benchmarks with pytest-bench
  • test coverage around this.
hash_ring - v0.0.2

Published by jhgg over 6 years ago

Add support for dynamic replicas. See #1 for more details.