monstache

a go daemon that syncs MongoDB to Elasticsearch in realtime. you know, for search.

MIT License

Stars
1.3K
Committers
16

Bot releases are visible (Hide)

monstache - monstache v3.22.1

Published by rwynn over 5 years ago

  • Upgrade golang to 1.12
  • Fix a panic under some conditions when processing the result of a golang plugin Map function
monstache - monstache v4.15.0

Published by rwynn over 5 years ago

  • This release adds the ability for MongoDB 4+ users to open change streams against entire
    databases or entire deployments. See the documentation for the change-stream-namespaces
    option for details.

  • The resume, resume-from-timestamp, replay, and cluster-name options can now be used
    in conjunction with change-stream-namespaces if you have MongoDB 4 or greater.

  • Improved support for connecting directly to shards when authorization is required on the connection.
    This is only applicable if you are not using change streams and you are connecting to a sharded
    MongoDB deployment. In that case Monstache needs to discover and connect directly to shards. In this
    case it will reuse the login info from the initial connection to the mongos server when connecting to
    the shards.

monstache - monstache v3.22.0

Published by rwynn over 5 years ago

  • This release adds the ability for MongoDB 4+ users to open change streams against entire
    databases or entire deployments. See the documentation for the change-stream-namespaces
    option for details.

  • The resume, resume-from-timestamp, replay, and cluster-name options can now be used
    in conjunction with change-stream-namespaces if you have MongoDB 4 or greater.

  • Improved support for connecting directly to shards when authorization is required on the connection.
    This is only applicable if you are not using change streams and you are connecting to a sharded
    MongoDB deployment. In that case Monstache needs to discover and connect directly to shards. In this
    case it will reuse the login info from the initial connection to the mongos server when connecting to
    the shards.

monstache - monstache v4.14.2

Published by rwynn over 5 years ago

  • change stream performance improvements
monstache - monstache v3.21.2

Published by rwynn over 5 years ago

  • change stream performance improvements
monstache - monstache v4.14.1

Published by rwynn over 5 years ago

  • Fix for regression in previous release where the MongoDB initial connection timeout
    was not being honored
  • Added a new setting relate-buffer which is the maximum number of relate events to
    allow in queue before skipping the relate and printing an error. This was added to
    prevent the scenario where a large number of relate events stall the pipeline. The
    default number of relates to allow in queue is 1000.
monstache - monstache v3.21.1

Published by rwynn over 5 years ago

  • Fix for regression in previous release where the MongoDB initial connection timeout
    was not being honored
  • Added a new setting relate-buffer which is the maximum number of relate events to
    allow in queue before skipping the relate and printing an error. This was added to
    prevent the scenario where a large number of relate events stall the pipeline. The
    default number of relates to allow in queue is 1000.
monstache - monstache v4.14.0

Published by rwynn over 5 years ago

  • Performance and reliability improvements
  • Fix for issue #168 by adding new TOML only configs elasticsearch-healthcheck-timeout-startup and elasticsearch-healthcheck-timeout. These are in seconds.
  • Fix a panic occurring when an empty array was returned from a pipeline javascript function.
  • Default read/write timeouts changed to 30s, up from 7s.
monstache - monstache v3.21.0

Published by rwynn over 5 years ago

  • Performance and reliability improvements
  • Fix for issue #168 by adding new TOML only configs elasticsearch-healthcheck-timeout-startup and elasticsearch-healthcheck-timeout. These are in seconds.
  • Fix a panic occurring when an empty array was returned from a pipeline javascript function.
  • Default read/write timeouts changed to 30s, up from 7s.
monstache - monstache v4.13.4

Published by rwynn over 5 years ago

  • Improvements to change-stream-namespaces
monstache - monstache v3.20.4

Published by rwynn over 5 years ago

  • Improvements for change-stream-namespaces
monstache - monstache v4.13.3

Published by rwynn over 5 years ago

  • Improvements in recovery from failed connections and errors
monstache - monstache v3.20.3

Published by rwynn over 5 years ago

  • Improvements in recovery from failed connections and errors
monstache - monstache v4.13.2

Published by rwynn almost 6 years ago

  • Fix namespace parsing for collections with dots in the name in find calls in Javascript
  • Add additional validation when certs are appended to the root store to make sure it was successful
  • Ensure the mechanism is set to MONGODB-X509 when logging in with an X509 certificate
monstache - monstache v3.20.2

Published by rwynn almost 6 years ago

  • Fix namespace parsing for collections with dots in the name in find calls in Javascript
  • Add additional validation when certs are appended to the root store to make sure it was successful
  • Ensure the mechanism is set to MONGODB-X509 when logging in with an X509 certificate
monstache - monstache v4.13.1

Published by rwynn almost 6 years ago

  • Fix issue #157 related to the relate config
  • Improve reliability for issue #153 by ensuring direct reads are more resilient in cluster mode
  • Breaking: removes the dynamic nature of finding the oplog collection. Now defaults to oplog.rs. If you are still using MongoDB with master mode instead of replica sets then you now need to explicitly set mongo-oplog-collection-name to oplog.$main.
monstache - monstache v3.20.1

Published by rwynn almost 6 years ago

  • Fix issue #157 related to the relate config
  • Improve reliability for issue #153 by ensuring direct reads are more resilient in cluster mode
  • Breaking: removes the dynamic nature of finding the oplog collection. Now defaults to oplog.rs. If you are still using MongoDB with master mode instead of replica sets then you now need to explicitly set mongo-oplog-collection-name to oplog.$main.
monstache - monstache v4.13.0

Published by rwynn almost 6 years ago

  • Fixed issue where keep-src was not being honored for relate configs
  • Added mongo-x509-settings config option to allow x509 auth when connecting
  • Added ability to see field level changes (updateDescriptoin) if available in javascript and golang
  • Handle delete events if possible such that they trigger resync of related docs when a relationship exists
  • Remove the requirement that golang plugins must implement a Map function
  • Added the ability to override the _id sent to Elasticsearch in the mapping phase. For scripts the _meta_monstache key to use is id.
monstache - monstache v3.20.0

Published by rwynn almost 6 years ago

  • Fixed issue where keep-src was not being honored for relate configs
  • Added mongo-x509-settings config option to allow x509 auth when connecting
  • Added ability to see field level changes (updateDescriptoin) if available in javascript and golang
  • Handle delete events if possible such that they trigger resync of related docs when a relationship exists
  • Remove the requirement that golang plugins must implement a Map function
  • Added the ability to override the _id sent to Elasticsearch in the mapping phase. For scripts the _meta_monstache key to use is id.
monstache - monstache v4.12.5

Published by rwynn almost 6 years ago

  • Fix regression preventing resume-name from being set correctly
  • Add configuration option config-database-name to configure the MongoDB database under which monstache stores metadata. Previously hard coded to monstache, the default value.