Published by squat almost 4 years ago
thanos tools bucket rewrite
, enabling the deletion of series from given block.labels
and series
API endpoints.thanos tools bucket replicate
can now copy particular blocks by ID.Series
call can now be limited using a CLI flag.hints
field to LabelNamesRequest
and LabelValuesRequest
. Hints are an opaque data structure that can be used to carry additional information from the store and its content is implementation-specific.thanos tools bucket rewrite
command allowing to delete series from given block.--receive.hashrings
alternative to receive.hashrings-file
flag (lower priority). The flag expects the literal hashring configuration in JSON format.thanos_objstore_bucket_operation_failures_total
alert.no such host error
when using dnssrv+
or dnssrvnoa+
.segments/
if the file is bigger than 1GB
.meta.json
file doesn't cause block cleanup anymore. This has a potential to generate corrupted blocks under specific conditions. Partial block is left in bucket for later cleanup.Published by metalmatze almost 4 years ago
Published by metalmatze almost 4 years ago
thanos query-frontend
now supports query splitting and retries for label names and label values APIs.thanos tools bucket replicate
can now copy particular blocks by time range.thanos query
is now using dynamic lookback delta when downsampled data is used by default.no-compact-mark.json
file to bucket block directory.thanos tools bucket mark
for marking blocks with no compact
or delayed deletion
marker.--store.enable-index-header-lazy-reader
flag), Store Gateway will mmap only those blocks caches that are needed for the queries. This option will be set to default in next release.thanos_querier_...
to thanos_query_...
and thanos_compactor_...
to thanos_compact_...
--compact.cleanup-interval
time period. By default it is 5 minutes.--query.http-method
is set GET
.query-frontend.org-id-header
flag to specify HTTP header(s) to populate slow query log (e.g. X-Grafana-User).--store.enable-index-header-lazy-reader
flag, the store-gateway will load into memory an index-header only once it's required at query time. Index-header will be automatically released after --store.index-header-lazy-reader-idle-timeout
of inactivity.
thanos tools bucket mark
command that allows to mark given block for deletion or for no-compactexperimental.enable-index-cache-postings-compression
flag.thanos_compactor_blocks_marked_for_deletion_total
to thanos_compactor_blocks_marked_total
with marker
label.thanos_replicate_origin_.*
metrics.
thanos_replicate_origin_meta_loads_total
can be replaced by blocks_meta_synced{state="loaded"}
.thanos_replicate_origin_partial_meta_reads_total
can be replaced by blocks_meta_synced{state="failed"}
.thanos_compactor
to thanos_compact
, thanos_querier
to thanos_query
and thanos_ruler
to thanos_rule
.Published by metalmatze almost 4 years ago
thanos query-frontend
now supports query splitting and retries for label names and label values APIs.thanos tools bucket replicate
can now copy particular block s by ID or time range.thanos query
is now using dynamic lookback delta when downsampled data is used by default.no-compact-mark.json
file to bucket block directory.thanos tools bucket mark
for marking blocks with no compact
or delayed deletion
marker.--store.enable-index-header-lazy-reader
flag), Store Gateway will mmap only those blocks caches that are needed for the queries. This option will be set to default in next release.thanos_querier_...
to thanos_query_...
and thanos_compactor_...
to thanos_compact_...
--compact.cleanup-interval
time period. By default it is 5 minutes.--query.http-method
is set GET
.query-frontend.org-id-header
flag to specify HTTP header(s) to populate slow query log (e.g. X-Grafana-User).--store.enable-index-header-lazy-reader
flag, the store-gateway will load into memory an index-header only once it's required at query time. Index-header will be automatically released after --store.index-header-lazy-reader-idle-timeout
of inactivity.
thanos tools bucket mark
command that allows to mark given block for deletion or for no-compactthanos_compactor_blocks_marked_for_deletion_total
to thanos_compactor_blocks_marked_total
with marker
label.thanos_replicate_origin_.*
metrics.
thanos_replicate_origin_meta_loads_total
can be replaced by blocks_meta_synced{state="loaded"}
.thanos_replicate_origin_partial_meta_reads_total
can be replaced by blocks_meta_synced{state="failed"}
.thanos_compactor
to thanos_compact
, thanos_querier
to thanos_query
and thanos_ruler
to thanos_rule
.Published by bwplotka almost 4 years ago
Highlights:
storeMatch[]
arguments with __address__
matchers.--web.prefix-header
is used.web.external-prefix
for Compactor UI.log_queries_longer_than
to log-queries-longer-than
.storeMatch[]
parameter to queries.storeMatch[]
to Labels APIs; Time range metadata based store filtering is supported on Labels APIs.thanos_sidecar_prometheus_store_received_frames
histogram metric.query.metadata.default-time-range
flag to specify the default metadata time range duration for retrieving labels through Labels and Series API when the range parameters are not specified. The zero value means range covers the time since the beginning.cache-compression-type
flag to use compression in the query frontend cache./debug/fgprof
endpoint on HTTP port allowing to get off-CPU profiles as well.Cache-Control
HTTP response header which controls caching behaviour. So far no-store
value is supported and it makes the response skip cache.tools bucket cleanup
CLI tool that deletes all blocks marked to be deleted.thanos_sidecar_reloader_config_apply_operations_total
and rename metric thanos_sidecar_reloader_config_apply_errors_total
to thanos_sidecar_reloader_config_apply_operations_failed_total
.thanos_query_gate_queries_max
. Remove metric thanos_query_concurrent_selects_gate_queries_in_flight
.thanos_bucket_store_queries_concurrent_max
to thanos_bucket_store_series_gate_queries_max
.thanos_objstore_bucket_operation_failures_total
.thanos_sidecar_reloader_config_apply_operations_total
and rename metric thanos_sidecar_reloader_config_apply_errors_total
to thanos_sidecar_reloader_config_apply_operations_failed_total
.Published by bwplotka about 4 years ago
Published by bwplotka about 4 years ago
Published by kakkoyun about 4 years ago
Thanks to all contributors! 🤗
--web.external-prefix
and --web.route-prefix
.--receive.local-endpoint
.thanos_compact_downsample_total
metric for downsample of 5m resolution blocks.--store.grpc.series-sample-limit
implementation. The limit is now applied to the sum of all samples fetched across all queried blocks via a single Series call, instead of applying it individually to each block./api/v1/rules
is called and the time zone is not UTC; rules
field is an empty array now if no rules have been defined in a rule group./api/v1/rules
endpoint no longer returns the old, deprecated partial_response_strategy
. The old, deprecated value has been fixed to WARN
for quite some time. Please use partialResponseStrategy
.maxSourceResolution
when auto downsampling is enabled.--query.lookback-delta
flag to override the default lookback delta in PromQL. The flag should be lookback delta should be set to at least 2 times of the slowest scrape interval. If unset it will use the PromQL default of 5m./api/v1/status/flags
, /api/v1/status/runtimeinfo
and /api/v1/status/buildinfo
) are now available on all components with a HTTP API.start
and end
fields to Store API LabelNamesRequest
and LabelValuesRequest
.reloader_config_apply_errors_total
metric. Add new flags --reloader.watch-interval
, and --reloader.retry-interval
.thanos_bucket_store_cached_series_fetch_duration_seconds
and thanos_bucket_store_cached_postings_fetch_duration_seconds
.storeMatch[]
to select matching stores when debugging the querier. See documentation
thanos_bucket_store_cached_postings_compression_time_seconds
to thanos_bucket_store_cached_postings_compression_time_seconds_total
.--tsdb.no-lockfile
) to override behavior.operation
label value getrange
changed to get_range
for thanos_store_bucket_cache_operation_requests_total
and thanos_store_bucket_cache_operation_hits_total
to be consistent with bucket operation metrics.encrypt_sse: true
option is now deprecated in favour of sse_config
. If you used encrypt_sse
, the migration strategy is to set up the following block:sse_config:
type: SSE-S3
Published by kakkoyun about 4 years ago
Published by bwplotka about 4 years ago
Published by kakkoyun over 4 years ago
bucket tool bucket verify
--id-whitelist
flag to --id
.index.cache.json
. The hidden flag --store.disable-index-header
was removed.Thanks all for contributing! 🤗
tsdb.MultiError
.af-south-1
and eu-south-1
regions./api/v1/stores
now guarantees to return a string in the lastError
field.index.cache.json
. The hidden flag --store.disable-index-header
was removed.chunk_object_size_ttl
to chunk_object_attrs_ttl
.--index.generate-missing-cache-file
and the metric thanos_compact_generated_index_total
were removed.--resolution
is now in Go duration format.bucket tool bucket verify
--id-whitelist
flag to --id
.--compaction
and --resolution
flags.--block-viewer.global.sync-block-interval
to configure metadata sync interval for the bucket UI.Published by kakkoyun over 4 years ago
/api/v1/stores
requested.
Thanks all for contributing! 🤗
Published by kakkoyun over 4 years ago
bucket tool bucket verify
--id-whitelist
flag to --id
.index.cache.json
. The hidden flag --store.disable-index-header
was removed.Thanks all for contributing! 🤗
tsdb.MultiError
.af-south-1
and eu-south-1
regions./api/v1/stores
now guarantees to return a string in the lastError
field.index.cache.json
. The hidden flag --store.disable-index-header
was removed.chunk_object_size_ttl
to chunk_object_attrs_ttl
.--index.generate-missing-cache-file
and the metric thanos_compact_generated_index_total
were removed.--resolution
is now in Go duration format.bucket tool bucket verify
--id-whitelist
flag to --id
.--compaction
and --resolution
flags.--block-viewer.global.sync-block-interval
to configure metadata sync interval for the bucket UI.Published by bwplotka over 4 years ago
Thanks all for contributing! 🤗
rate
with deduplication enabled.SeriesResponse
protobuf message that may be added in the future.Query
component.inspect --sort-by
doesn't work correctly in all cases.irate
and resets
use now counter downsampling aggregations.af-south-1
and eu-south-1
regions.hints
field to SeriesResponse
. Hints in an opaque data structure that can be used to carry additional information from the store and its content is implementation specific.thanos_sidecar_reloader_reloads_failed_total
, thanos_sidecar_reloader_reloads_total
, thanos_sidecar_reloader_watch_errors_total
, thanos_sidecar_reloader_watch_events_total
and thanos_sidecar_reloader_watches
metrics./graph
endpoint is migrated.--store.caching-bucket.config=<yaml content>
(or --store.caching-bucket.config-file=<file.yaml>
) for experimental caching bucket, that can cache chunks into shared memcached. This can speed up querying and reduce number of requests to object storage.--deduplication.replica-label
, assign first replica-label
with value deduped
.thanos_store_node_info
metric.thanos bucket
commands to thanos tools bucket
, alsothanos check rules
to thanos tools rules-check
. thanos tools rules-check
also takes rules by --rules
repeated flag not argumentlabel=~"a|b|c"
matchers.--deduplication.replica-label
, assign first replica-label
with value deduped
.Published by bwplotka over 4 years ago
rate
with deduplication enabled.SeriesResponse
protobuf message that may be added in the future.Query
component.inspect --sort-by
doesn't work correctly in all cases.irate
and resets
use now counter downsampling aggregations.af-south-1
and eu-south-1
regions.hints
field to SeriesResponse
. Hints in an opaque data structure that can be used to carry additional information from the store and its content is implementation specific.thanos_sidecar_reloader_reloads_failed_total
, thanos_sidecar_reloader_reloads_total
, thanos_sidecar_reloader_watch_errors_total
, thanos_sidecar_reloader_watch_events_total
and thanos_sidecar_reloader_watches
metrics./graph
endpoint is migrated.--store.caching-bucket.config=<yaml content>
(or --store.caching-bucket.config-file=<file.yaml>
) for experimental caching bucket, that can cache chunks into shared memcached. This can speed up querying and reduce number of requests to object storage.--deduplication.replica-label
, assign first replica-label
with value deduped
.thanos_store_node_info
metric.thanos bucket
commands to thanos tools bucket
, alsothanos check rules
to thanos tools rules-check
. thanos tools rules-check
also takes rules by --rules
repeated flag not argumentlabel=~"a|b|c"
matchers.--deduplication.replica-label
, assign first replica-label
with value deduped
.Published by bwplotka over 4 years ago
rate
with deduplication enabled.SeriesResponse
protobuf message that may be added in the future.Query
component.inspect --sort-by
doesn't work correctly in all cases.irate
and resets
use now counter downsampling aggregations.af-south-1
and eu-south-1
regions.hints
field to SeriesResponse
. Hints in an opaque data structure that can be used to carry additional information from the store and its content is implementation specific.thanos_sidecar_reloader_reloads_failed_total
, thanos_sidecar_reloader_reloads_total
, thanos_sidecar_reloader_watch_errors_total
, thanos_sidecar_reloader_watch_events_total
and thanos_sidecar_reloader_watches
metrics./graph
endpoint is migrated.--store.caching-bucket.config=<yaml content>
(or --store.caching-bucket.config-file=<file.yaml>
) for experimental caching bucket, that can cache chunks into shared memcached. This can speed up querying and reduce number of requests to object storage.--deduplication.replica-label
, assign first replica-label
with value deduped
.thanos_store_node_info
metric.thanos bucket
commands to thanos tools bucket
, alsothanos check rules
to thanos tools rules-check
. thanos tools rules-check
also takes rules by --rules
repeated flag not argumentlabel=~"a|b|c"
matchers.--deduplication.replica-label
, assign first replica-label
with value deduped
.Published by bwplotka over 4 years ago
Highlights:
rate
with deduplication enabled, thanks to @SuperQ who shared raw data, that reproducing this!rate
with deduplication enabled.SeriesResponse
protobuf message that may be added in the future.Query
component.inspect --sort-by
doesn't work correctly in all caseshints
field to SeriesResponse
. Hints in an opaque data structure that can be used to carry additional information from the store and its content is implementation-specific.thanos_sidecar_reloader_reloads_failed_total
, thanos_sidecar_reloader_reloads_total
, thanos_sidecar_reloader_watch_errors_total
, thanos_sidecar_reloader_watch_events_total
and thanos_sidecar_reloader_watches
metrics./graph
endpoint is migrated.--store.caching-bucket.config=<yaml content>
(or --store.caching-bucket.config-file=<file.yaml>
) for experimental caching bucket, that can cache chunks into shared memcached. This can speed up querying and reduce number of requests to object storage.thanos_store_node_info
metric.thanos bucket
commands to thanos tools bucket
, alsothanos check rules
to thanos tools rules-check
. thanos tools rules-check
also takes rules by --rules
repeated flag not argumentlabel=~"a|b|c"
matchers.--deduplication.replica-label
, assign first replica-label
with value deduped
.Published by squat over 4 years ago