Bot releases are hidden (Show)
Published by mreiferson over 10 years ago
Upgrading from 0.2.25: No backwards incompatible changes.
IMPORTANT: this is a quick bug-fix release to address a regression identified in 0.2.25
where statsd
prefixes were broken when using the default (or any) prefix that contained a %s
for automatic host replacement.
nsqd
--statsd-prefix
when using %s
host replacementPublished by mreiferson over 10 years ago
IMPORTANT: please upgrade to v0.2.26
to resolve a potentially important regression.
Upgrading from 0.2.24: No backwards incompatible changes.
This release adds several commonly requested features.
First, thanks to @elubow you can now configure your clients to sample the stream they're subscribed to. To read more about the details of the implementation see #286 and the original discussion in #223. Eric also contributed an improvement to nsq_tail
to add the ability to tail the last N
messages and exit.
We added config file support (TOML) for nsqd
, nsqlookupd
, and nsqadmin
- providing even more deployment flexibility. Example configs are in the contrib
directory. Command line arguments override the equivalent option in the config file.
We added the ability to pause a topic (it is already possible to pause individual channels). This functionality stops all message flow from topic to channel for all channels of a topic, queueing at the topic level. This enables all kinds of interesting possibilities like atomic channel renames and trivial infrastructure wide operations.
Finally, we now compile the static assets used by nsqadmin
into the binary, simplifying deployment. This means that --template-dir
is now deprecated and will be removed in a future release and you can remove the templates you previously deployed and maintained.
New Features / Enhancements:
IDENTIFY
option to sample a % of messagesnsqd
, nsqlookupd
, and nsqadmin
nsqadmin
binarynsq_tail
support for -n #
to get recent # messagesIDENTIFY
attributes in nsqadmin
(sample rate, TLS, compression)nsqadmin
nsq_to_nsq
JSON message filtering optionsPublished by mreiferson almost 11 years ago
Upgrading from 0.2.23: No backwards incompatible changes. However, as you'll see below, quite a few command line flags to the utility apps (nsq_to_http
, nsq_to_file
, nsq_to_http
) were deprecated and will be removed in the next release. Please use this release to transition over to the new ones.
NOTE: we are now publishing additional binaries built against go1.2
The most prominent addition is the tracking of end-to-end message processing percentiles. This measures the amount of time it's taking from PUB
to FIN
per topic/channel. The percentiles are configurable and, because there is some overhead in collecting this data, it can be turned off entirely. Please see the section in the docs for implementation details. It looks like this:
Additionally, the utility apps received comprehensive support for all configurable reader options (including compression, which was previously missing). This necessitated a bit of command line flag cleanup, as follows:
--gzip-compression
in favor of --gzip-level
--verbose
in favor of --reader-opt=verbose
--throttle-fraction
in favor of --sample
--http-timeout-ms
in favor of --http-timeout
(which is a--verbose
in favor of --reader-opt=verbose
--max-backoff-duration
in favor of--reader-opt=max_backoff_duration=X
--verbose
in favor of --reader-opt=verbose
--max-backoff-duration
in favor of--reader-opt=max_backoff_duration=X
New Features / Enhancements:
Published by mreiferson almost 11 years ago
Upgrading from 0.2.22: No backwards incompatible changes.
We now use godep in order to achieve reproducible builds with pinned dependencies. If you're on go1.1+ you can now just use godep get github.com/bitly/nsq/...
.
This release includes nsqd
protocol compression feature negotiation. Snappy and Deflate are supported, clients can choose their preferred format.
--statsd-prefix
can now be used to modify the prefix for the statsd
keys generated by nsqd
. This is useful if you want to add datacenter prefixes or remove the default host prefix.
Finally, this release includes a "bug" fix that reduces CPU usage for nsqd
with many clients by choosing a more reasonable default for a timer used in client output buffering. For more details see #236.
New Features / Enhancements:
Bug Fixes:
Published by mreiferson about 11 years ago
Upgrading from 0.2.21: message timestamps are now officially nanoseconds. The protocol docs always stated this however nsqd
was actually sending seconds. This may cause some compatibility issues for client libraries/clients that were taking advantage of this field.
This release also introduces support for TLS feature negotiation in nsqd
. Clients can optionally enable TLS by using the appropriate handshake via the IDENTIFY
command. See #227.
Significant improvements were made to the HTTP publish endpoints and in flight message handling to reduce GC pressure and eliminate memory abuse vectors. See #242, #239, and #245.
This release also includes a new utility nsq_to_nsq
for performant, low-latency, copying of an NSQ topic over the TCP protocol.
Finally, a whole suite of debug HTTP endpoints were added (and consolidated) under the /debug/pprof
namespace. See #238, #248, and #252. As a result nsqd
now supports direct profiling via Go's pprof
tool, ie:
$ go tool pprof --web http://ip.address:4151/debug/pprof/heap
New Features / Enhancements:
nsqadmin
single node view (with GC/mem graphs)nsq_to_nsq
utility for copying a topic over TCPnsq_to_http
takes --content-type
flag (thanks @michaelhood)nsqadmin
displays tombstoned topics in the /nodes
list/mput
Bug Fixes:
tombstone_topic_producer
action in nsqadmin
missing node infonsqd
metadata on topic/channel changesnsqlookupd
/nsqadmin
displays inactive nodes in /nodes
Published by mreiferson over 11 years ago
Upgrading from 0.2.15: there are no backward incompatible changes in this release.
However, this release introduces the IDENTIFY
command (which supersedes sending metadata along with SUB
) for clients of nsqd
. The old functionality will be removed in a future release.
Published by mreiferson over 11 years ago
Upgrading from 0.2.16: IDENTIFY and SUB now return success responses (they previously only responded to errors). The official Go and Python libraries are forwards/backwards compatible with this change however 3rd party client libraries may not be.
Upgrading from 0.2.15: in #132 deprecations in SUB were removed as well as support for the old, line oriented, nsqd
metadata file format. For these reasons you should upgrade to 0.2.16
first.
New Features / Enhancements:
Bug Fixes:
Published by mreiferson over 11 years ago
Upgrading from 0.2.17: all V2 clients of nsqd now receive heartbeats (previously only clients that subscribed would receive heartbeats, excluding TCP producers).
Upgrading from 0.2.16: follow the notes in the 0.2.17 changelog for upgrading from 0.2.16.
Beyond the important note above regarding heartbeats this release includes nsq_tail
, an extremely useful utility application that can be used to introspect a topic on the command line. If statsd is enabled (and graphite in nsqadmin
) we added the ability to retrieve rates for display in nsqadmin
.
We resolved a few critical issues with data consistency in nsqlookupd
when channels and topics are deleted. First, deleting a topic would cause that producer to disappear from nsqlookupd
for all topics. Second, deleting a channel would cause that producer to disappear from the topic list in nsqlookupd
.
New Features / Enhancements:
Bug Fixes:
Published by mreiferson over 11 years ago
Upgrading from 0.2.18: there are no backward incompatible changes in this release.
This release is a small release that introduces one major client side feature and resolves one critical bug.
nsqd
clients can now configure their own heartbeat interval. This is important because as of 0.2.18
all clients (including producers) received heartbeats by default. In certain cases receiving a heartbeat complicated "simple" clients that just wanted to produce messages and not handle asynchronous responses. This gives flexibility for the client to decide how it would like behave.
A critical bug was discovered where emptying a channel would leave client in-flight state inconsistent (it would not zero) which limited deliverability of messages to those clients.
New Features / Enhancements:
Bug Fixes:
Published by mreiferson over 11 years ago
Upgrading from 0.2.19: there are no backward incompatible changes in this release.
This release adds a couple of convenient features (such as adding the ability to empty a topic) and continues our work to reduce garbage produced at runtime to relieve GC pressure in the Go runtime.
nsqd
now has two new flags to control the max value clients can use to set their heartbeat interval as well as adjust a clients maximum RDY count. This is all set/communicated via IDENTIFY
.
nsqadmin
now displays nsqd
-> nsqlookupd
connections in the "nodes" view. This is useful for visualizing how the topology is connected as well as situations where --broadcast-address
is being used incorrectly.
nsq_to_http
now has a "host pool" mode where upstream state will be adjusted based on successful/failed requests and for failures, upstreams will be exponentially backed off. This is an incredibly useful routing mode.
As for bugs, we fixed an issue where "fatal" client errors were not actually being treated as fatal. Under certain conditions deleting a topic would not clean up all of its files on disk. There was a reported issue where the --data-path
was not writable by the process and this was only discovered after message flow began. We added a writability check at startup to improve feedback. Finally. deferred_count
was being sent as a counter value to statsd, it should be a gauge.
New Features / Enhancements:
Bug Fixes:
Published by mreiferson over 11 years ago
Upgrading from 0.2.20: there are no backward incompatible changes in this release.
This release introduces a significant new client feature as well as a slew of consistency and recovery improvements to diskqueue.
First, we expanded the feature negotiation options for clients. There are many cases where you want different output buffering semantics from nsqd
to your client. You can now control both output buffer size and the output buffer timeout via new fields in the IDENTIFY
command. You can even disable output buffering if low latency is a priority.
You can now specify a duration between fsyncs via --sync-timeout
. This is a far better way to manage when the process fsyncs messages to disk (vs the existing --sync-every
which is based on # of messages). --sync-every
is now considered a deprecated option and will be removed in a future release.
Finally, 0.2.20
introduced a significant regression in #176 where a topic would not write messages to its channels. It is recommended that all users running 0.2.20
upgrade to this release. For additional information see #217.
New Features / Enhancements:
Bug Fixes: