Priam

Co-Process for backup/recovery, Token Management, and Centralized Configuration management for Cassandra.

APACHE-2.0 License

Stars
1K
Committers
59
Priam - Forgotten files and new API to consume configuration

Published by arunagrawal84 about 6 years ago

  • Forgotten files: Increment metric when a forgotten file is detected in C* data folder. For future, we need to move forgotten files to lost+found directory based on configuration.

  • Expose Priam config via HTTP and persist to disk

    • This provides two new HTTP APIs for external tools to access Priam
      configuration. These APIs allow external tools to work with Priam to
      manage Cassandra while still using Priam's flat configuration as a
      source of truth for the Cassandra instance
      GET /v1/config/structured/all -> returns all Priam IConfiguration fields
      GET /v1/config/unstructured/X?default=Y -> returns the property X, or Y if
      it is not found (404 will be raised if no default and the value doesn't
      exist)
    • In addition to the HTTP apis there is also now a persister thread which
      makes it so that every minute (by default) configuration is written out
      to a directory in tmp by Priam. This way tools can access config even if
      Priam is dead. The file permissions are 600 so that only the Priam user
      can access it.
Priam - Alpha release 7: 4.x

Published by sumanth-pasupuleti about 6 years ago

No new features. Trying to fix the broken release.

Priam - Alpha release 6: 4.x

Published by arunagrawal84 about 6 years ago

No new features. Trying to fix the broken release.

Priam - Alpha release 5: 4.x

Published by arunagrawal84 about 6 years ago

Do not use this in prod. This is to test 4.x Cassandra branch only.

Priam - 4th alpha for 4.x

Published by sumanth-pasupuleti about 6 years ago

Updated travis build link in readme

Priam - 3rd alpha for 4.x

Published by sumanth-pasupuleti about 6 years ago

Includes fix for bootstrap issue

  • Fix handling of cluster startup.

Previously we would provide the local node as a seed even if
auto_bootstrap was set to true, which meant that when doubling we could
potentially stream no data. Providing the local node as a seed was added
to work around the (imo buggy) behaviour introduced in CASSANDRA-10134
where the shadow round fails unless the node is a seed which even if
autobootstrap was turned off.

  • Test for InstanceIdentity properly handling auto bootstrap conditions

Previously we could double and have nodes join without data, now when
auto bootstrap is true we do not return the current node as a seed ever.

  • Provide an overwrite method to force Priam to replace a particular ip

This allows us to work around the A->B->C replacement problem where
Priam get's confused about who to replace

Priam - Fix bootstrap issue

Published by arunagrawal84 about 6 years ago

  • Fix handling of cluster startup.

Previously we would provide the local node as a seed even if
auto_bootstrap was set to true, which meant that when doubling we could
potentially stream no data. Providing the local node as a seed was added
to work around the (imo buggy) behaviour introduced in CASSANDRA-10134
where the shadow round fails unless the node is a seed which even if
autobootstrap was turned off.

  • Test for InstanceIdentity properly handling auto bootstrap conditions

Previously we could double and have nodes join without data, now when
auto bootstrap is true we do not return the current node as a seed ever.

  • Provide an overwrite method to force Priam to replace a particular ip

This allows us to work around the A->B->C replacement problem where
Priam get's confused about who to replace

Priam - Second alpha for 4.x

Published by jolynch about 6 years ago

Removes support for streaming_socket_timeout_in_ms and replaces it with streaming_keep_alive_period_in_secs. This option was removed in 4.x

Priam - FIrst alpha for 4.x

Published by jolynch about 6 years ago

NOTE: this build is experimental and not recommended for production use (hence the -alpha tag). We recommend against running this in production as we have not yet vetted any 4.x release in production yet.

Cassandra 4.x breaks a lot (as is expected in a major version). The following breaking changes were introduced:

  • All thrift related functionality has been removed
  • The seed provider now returns InetAddressAndPort, should work fine with Cassandra 4.x, but will not work with 3.11 clusters.
  • Jvm options files are now located at jvm-server.options rather than jvm.options, the default location in priam has been updated accordingly but if you had a custom location specified this needs to change
  • Various cassandra.yaml options have been removed (mostly rpc and thrift related)

Support for the following features were added in a backwards compatible way:

  • Support for tuning arbitrary properties file e.g. cassandra-rackdc.properties. As the EC2 multi region snitch now uses different naming (https://issues.apache.org/jira/browse/CASSANDRA-7839) if upgrading a cluster from 4.0 you should tune the cassandra-rackdc.properties file either by setting priam.propertyOverrides.cassandra-rackdc to ec2_naming_scheme=legacy or switching to gossiping property file snitch and setting dc and rack manually by setting the same property to dc=${dc},rack=${rac} (which will do auto-substitution).

This artifact internally compiles against a checked in cassandra-4.0 jar until Cassandra releases official jars. To compile against this you need to do the same.

Priam - Metric collection change (Breaking Release)

Published by arunagrawal84 about 6 years ago

WARNING THIS IS A BREAKING RELEASE

New Feature

  • (#707) Move to spectator to collect Metrics. This is a breaking change, if you were depending on MetricPublisher to collect metrics.
Priam - Metric collection change (Breaking Release)

Published by arunagrawal84 about 6 years ago

WARNING THIS IS A BREAKING RELEASE

  • (#706) Move to spectator to collect Metrics. This is a breaking change, if you were depending on MetricPublisher to collect metrics.
Priam - Snapshot Meta Service

Published by arunagrawal84 about 6 years ago

  • (#671) Compactions on CRON: Added ability to schedule compactions for columnfamilies on CRON.
  • (#691) Snapshot Meta Service: This will form the basis of backup version 2. This service will run on CRON and will ensure we upload a meta file containing the list of SSTables on disk.
  • (#703) Making post restore hook heartbeat timeout and heartbeat check frequency configurable
  • (#671) Deprecate flush hour and backup hour.
Priam - Snapshot Meta Service

Published by arunagrawal84 about 6 years ago

This will form the basis of backup version 2. This service will run on CRON and will ensure we upload a meta file containing the list of SSTables on disk.

Priam - Compactions on CRON

Published by arunagrawal84 about 6 years ago

New Feature

  • (#699) Compactions on CRON: Added ability to schedule compactions of columnfamilies on CRON.
  • (#699) Move to cassandra-all:3.0.17

Deprecation

  • (#699) Deprecate flush hour and backup hour.
Priam - Making post restore hook heartbeat timeout and heartbeat check frequency configurable

Published by sumanth-pasupuleti about 6 years ago

Priam - Moving restore-finished status to after PostRestoreHook execution

Published by sumanth-pasupuleti about 6 years ago

Priam - Moving restore-finished status to after PostRestoreHook execution

Published by sumanth-pasupuleti about 6 years ago

Priam - Adding partitioner endpoint to cassadmin resource to get C* partitioner name

Published by sumanth-pasupuleti over 6 years ago

Priam - Adding partitioner endpoint to cassadmin resource to get C* partitioner name

Published by sumanth-pasupuleti over 6 years ago

Extend StandardTuner to allow custom yaml properties, generic JMXNodeTool method to return mbeanserverconnection