Features

• [TSDB] Support for present_over_time , group and sgn functions PromQL function.
• [TSDB] Support to enable partial results for metadata queries. This fixes all or nothing scenarios for metadata queries where incomplete results are acceptable over no results. Responds with HTTP code 206 instead of 200 when results are partial.
• [TSDB] Performant Host Monitoring let us define “target-schemas” allows performant querying by colocating frequently joined data. More details on this feature to come soon in a blog post.
• [TSDB] Support subqueries in FILODB.
• [TSDB] Support for simple OR regex on shared label values:
  • sum(
    container_memory_working_set_bytes{
    ws="aci-kubernetes",
    ns="us-east-1a",
    exported_namespace=~"(kube-namespace-1|kube-namespace-2)"
    }
    )

Bug Fixes

• [TSDB] SinglePartitionPlanner bug preventing joins on raw and recording rules.
• [TSDB] InternalError shard(s?) should have dropped into "error" state and been reassigned
• [TSDB] Label-values query with !~ regex filters out too many results
• [TSDB] Compliance in behavior with Prometheus when != is used with a non existent label.
• [TSDB] Check for query timeouts in long-lived aggregations and joins so expensive queries are stopped in time.
• [TSDB] Fix metadata query routing when a regex filter is/isn’t given in the query.
• [TSDB] Make label-names api PromQL-compatible.
• [TSDB] Fix inconsistent results when binary joins operate on data both older and newer than now-7d.
• [TSDB] Failed Index lookup caused by large regex. Label regex now capped to 1 KB.

Performance enhancements

• [TSDB] Added support for stateful sets in FiloDB enabling faster shard recovery times by using block storage for index storage over container local storage.
• [TSDB] Label values query performance has been improved significantly with Lucene faceting. At high load, shard level latency for this query decreased from 5s to 16ms (300x).
• [TSDB] More performant label-values queries: queries for workspace and/or namespace values will now always bypass the storage engine when filters include only workspace and/or namespace labels.
• [TSDB] Optimization of binary joins by reducing data export when queried time range is beyond recent 7d.
• [TSDB] Track and log query stats for expensive queries. Future releases will enforce strict limits and block queries that scan/export large volume of data

Release 0.9.9

Release Notes

New features / Enhancements

• Support __name__ in group by, without, on and ignoring [PR 711]
• Apply data size restrictions per query to ensure very large queries don’t destabilize TSDB [PR 748]

Reliability Improvements

• Add utility method to verify histogram column [PR 731]
• Utility to convert ColumnFilters in LogicalPlan [PR 751]
• Timeout query immediately if it stayed in actor mailbox for too long [PR 761]

Bug Fixes

• Fix premature memory reclamation when data is fetched from Cassandra, which can sometimes cause incorrect query results when memory is saturated (various PRs)
• Queries requiring full paging of data from Cassandra sometimes returned incomplete results [PR 774]
• Addressed long lookback queries returning incomplete results [PR 743]

• MultiSchema support. Each metric will now have a type: counter, gauge, histogram, or summary. Types are used for optimizing downsampling and queries, and optimizing storage for histograms. Metadata queries will return a new type label describing the time series type, and filtering by metric type is possible. Type is auto-detected in the open source gateway when using Telegraf as input source.
• Support for first class histograms: transports and stores histogram buckets as a single entity and time series, rather than a separate time series for each bucket. Leads to huge savings for storage as well as Kafka. See the README notes about First-Class Histograms for more details on query changes.
• Many PromQL functions added, including predict_linear, stddev, stdvar, and others
• More efficient index and chunk persistence and querying to/from Cassandra

Due to changes in the format of keys, indices and other tables in Cassandra, this new release will not be able to read from older Cassandra tables and truncation or deploy to new environment will be necessary.

Release 0.8.7

New features / Enhancements

• [PromQL] Rate and increase functions for HistogramColumn
• [PromQL] Operator precedence
• [PromQL] Support for “changes” function
• [PromQL] Sort and sort desc functions
• [TSDB] More progress towards full multi-schema support
• [TSDB] Ability to respond to gauge-related PromQL queries from downsampled data
• [TSDB] Routing queries based on failure time ranges across pods/cluster, and routing for instant queries

Bug fixes

• [TSDB] Release IndexSearchers after each use, reducing disk usage and host termination
• [TSDB] More fixes for memory management issues which can lead to “match errors”, which in turn cause queries to fail
• [Gateway] Honor order in copyTags

Reliability / Tooling improvements

• [TSDB] Additional memory management tracing for diagnosing “match error” corruption issues
• [TSDB] Trusty build in travis

For more details, please view the git change log.

Release 0.8.5

New features / Enhancements

• Optimize Aggregate over vector function by 2x for no grouping
• Allow metric name as tag in promQL and allow escape chars in tag values
• ExplainPlan support in FiloDB HTTP
• Add multiline query support
• Query Support for Label Join
• Spread override and non metric shard keys based spread support
• Made FiloDB and Query Service connection more reliable

Bug fixes

• Fix very rare partition ID collision
• Fix edge case for DDV encoding where first value != the rest
• Including missed time bucket during index recovery
• Fix DeltaDelta bug where slope(int) multiply overflowed
• Fix corner cases in indexing of intermittent time series
• Fix race conditions with internal bloom filter
• Schedule flush tasks in the correct thread, to preventing lock conflicts with ingestion
• Improve handling of ingestion recovery exceptions
• Binary join having ignoring/on bug fix
• Comparison operator returns empty metric bug fix
• Topk/bottomk function bug fix

Reliability / Tooling improvements

• Detailed ingestion logging for matching subset of records
• Full shard state command to resync shard mapping
• Use shared ingestion buffer memory, to prevent out of memory errors when shards are imbalanced
• Return extra WriteBuffer memory back for reuse
• Add shared lock debugging checks for ingestion thread

For more details, please view the git change log.

Release 0.8.4

Bug fixe(s)

• Added batching logic on lucene query

Release 0.8.4

New features / Enhancements

• Foundational features for non-increasing type HistogramColumn along with relevant query functions
• Improved support for intermittently publishing time series
• Support for stitching query results when spread changes
• Query Support for Label Replace, Comparison Operators
• Optimize index recovery for long term retention

Bug fixes

• Various bug fixes and improvements in Query Engine
• Demand page from Cassandra only if Lucene index indicates that there is data

Reliability / Tooling improvements

• Reduced query bottlenecks by further parallelizing query steps
• Several stabilization fixes for memory handling and locking

For more details, please view the git change log.

Release 0.8.3

New features / Enhancements

• Simplify in-memory chunk index data structure

• Bug fixes

• Fix cases where binary joins could return incorrect results

• Handle no data cases in {aggregate}_over_time functions

• Fix minor bug in resets function

• Reliability / Tooling improvements

• Added bounds check to correctly access off-heap memory

• Several stabilization fixes for memory handling and locking

• Fixes for edge cases that could potentially corrupt data

• Integrate with tracing infrastructure

For more details, please view the git change log.

This release covers 89 commits total.

• New query functionality: histogram_quantile, resets, deriv, idelta, irate
• New metadata query capabilities, support for listing time series matching certain criteria
• Major query performance improvements, especially for *_over_time functions. Functions which use last sample windowing and *_over_time run up to 7.5x faster.
• Many fixes to query functionality
• Fixes and stabilization around memory handling, locking, and preventing data corruption
• Internal debugging query function: _filodb_chunkmeta_all()