Release Highlights

This release includes a fix for a potential shard deadlock.

All changes

2022-03-04 - 6d90f6540- Fix potential shard deadlock issue(#2570)

Details about v1.15.0 release can be found here.

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use tag 1.15.2)

Server
Server With Auto Setup (what is Auto-Setup?)
Admin-Tools

Release Highlights

This release includes a fix for a potential shard deadlock.

All changes

2022-03-04 - f911933b0 - Fix potential shard deadlock issue(#2570)

Details about v1.14.0 release can be found here.

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use tag 1.14.6)

Server
Server With Auto Setup (what is Auto-Setup?)
Admin-Tools

Release Highlights

This release includes TLS supports for multi-clusters and some fixes of config updating, task failure handling, invalid workflow ID. See full list of patches below.

All changes

2022-02-28 - 9d892c634 - Return NOT_FOUND for invalid workflow ID (#2559)
2022-02-25 - ba35d7e13 - Allow customizing retry behavior for timeout failure (#2524)
2022-02-25 - aa9a8b1b5 - Bump server and cli version to 1.15.1 (#2554)
2022-02-25 - a9b223b98 - Backoff failed workflow task (#2548)
2022-02-25 - 24eb5cfbb - reduce unnecessary logging (#2549)
2022-02-24 - 749d2c602 - Preserve the info when updating is_global_namepsace_enabled (#2540)
2022-02-24 - 3b6d58602 - Check is global ns enabled in handler (#2541)
2022-02-23 - e7852f73e - Treat enablement of TLS separately for server and client config (#2501)
2022-02-23 - 66ebbd49d - Add config and handling for remote cluster cert (#2475)
2022-02-22 - 1a1aa2f75 - Adds ExecutionInfo reference to InternalWorkflowMutation #2530 (#2531)
2022-02-14 - faf1b3e2c - Update go version of goreleaser to to 1.17.6 (#2506)
2022-02-14 - 1d3caadcb - Split goreleaser archive in two:cgo/nocgo (#2505)

Details about v1.15.0 release can be found here.

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use tag 1.15.1)

Server
Server With Auto Setup (what is Auto-Setup?)
Admin-Tools

Release Highlights

tctl

• A new sub-command is added that lists all Cluster information: tctl admin cluster list

API

• Option to enable auto-forwarding over all APIs.
• De-duplicate SignalWithStartWorkflowExecution requests on RequestID.

Namespace promotion

• Support local Namespace promotion and handle Workflow version correctly.
• Allow updating enableGlobalNamepsace config from false to true .
• Allow Namespace update operations from non-master Clusters.
• Allow removal of Clusters from the replication Cluster list.

Cron

• Update cron to v3.0 and add CronSchedule Timezone support.

Metrics and logging

• Several metrics and logging improvements in History client and Matching client.
• Fix History client and Matching client error logging with nil return error.

Visibility

• Replace Cassandra visibility TTL with explicit DELETE.
• Support visibility dual-write to multiple Elasticsearch indices.

Other

• Upgrade to go 1.17.6.
• Upgrade SDK to v1.13.
• Switch scanner to scan over visibility.
• Update OpenTelemetry to v1.2.0 and OpenTelemetry Metrics to v0.25.0.

Improvements and fixes

• Several test improvements on coverage and flaky tests.
• Fix SQLite concurrency bug fixes.
• Improvements in database schema setup and update tooling.
• Several improvements on handling loss of shard ownership.
• Improvements to sync matches and task dispatch in Matching engine.
• Handle namespaceNotActive error correctly with ParentChildPolicy Workflows.
• Correctly handle “History not found” error in History Archival.
• Handle SQL duplicate-key case in fetching History.
• Apply ParentClosePolicy to Child Workflow only.
• Trigger History node validation in Workflow updates.
• Detect stale cache when RecordChildExecutionCompleted.
• Use latest Worker identity for pending Activity.
• Disallow marking an Activity as cancelled without a prior request to cancel the Activity.
• Initialize shard controller with fx.
• Initialize shard correctly in ExecutionMutableStateSuite.
• Fix fail Workflow Task and generate new Workflow Task in Workflow Heartbeats.
• Notify new Tasks when Workflow is potentially updated.
• Use next-page token when listing Cluster metadata from database.
• Fix transient Workflow Task check in getRawHistory.
• Fix reference count in DeleteHistoryBranch.
• New Task flush buffer in Matching engine.
• Fix custom Search Attribute for Child Workflow and Continue-As-New.

Internal refactoring

• Extend Cassandra test utility methods so that they can be executed from outside.
• Separate History engine factory from handler.
• Move Workflow deletion logic to DeleteManager.
• Add ExecutionInfo field to InternalWorkflowSnapshot.
• Rename InternalWorkflowSnapshot.ExecutionInfo to ExecutionInfoBlo.
• Simplify LRU cache implementation.

All changes

2022-02-08 - fb8a5e0c3 - Fix admin handler getTaskType implementation (#2474)
2022-02-08 - cdb4fa01c - Use throttled logger and use error type in metrics client (#2477)
2022-02-08 - 670d99c95 - Move search attribute validator and mapper calls to frontend service (#2476)
2022-02-08 - 5fcfe252e - Remove unused queue processor (#2473)
2022-02-08 - 0e9f41593 - Add error type for logging in clients (#2480)
2022-02-08 - 01d071abe - Fix conflict error handling (#2469)
2022-02-05 - b2c7ee12f - Update invalid command message (#2468)
2022-02-05 - 8f88254b1 - Fix matching client error logging (#2471)
2022-02-05 - 16871d7fb - fix error logging for history client (#2472)
2022-02-04 - e64202091 - Upgrade base images (#2467)
2022-02-04 - 8c81dbcd6 - Handle history not found error when archiving history (#2465)
2022-02-04 - 89c17cc7f - Remove dynamic config translation map (#2462)
2022-02-04 - 6e7489a1c - Order delete workflow execution steps and document the order (#2466)
2022-02-03 - a7abf8490 - Updated README for running BuildKite tests locally (#2459)
2022-02-02 - d2d1e3d76 - Add more test for namespace migration (#2450)
2022-02-02 - 9454db231 - Update developer tools (#2454)
2022-02-02 - 515900d41 - Updated SetUpCassandraDatabase to take an actual logger instead of creating a NoopLogger. (#2453)
2022-02-02 - 4d41efc85 - Set task queue name in DescribeWorkflowExecutionResponse (#2455)
2022-02-02 - 4670e98ad - Update go dependencies (#2452)
2022-02-02 - 0447a17d3 - Assign LastFirstTransactionID correctly (#2438)
2022-02-01 - fe89dd4aa - Update SDK version to 1.13.0 (#2445)
2022-02-01 - a9ebd010b - Use namespace function to check active cluster (#2449)
2022-02-01 - 9fc6b63a3 - Address late review comments from #2365 (#2441)
2022-02-01 - 3eb7730fb - Convert ConverterError to InvalidArgument in standard visibility parser (#2437)
2022-02-01 - 2503c8469 - Added GetVersionedSchemaFilesInOrder helper function to cassandra_test_util.go (#2436)
2022-02-01 - 0e547dc39 - Remove unused resource struct (#2444)
2022-01-31 - 8c30d1da9 - Remove hacky force flush buffer check (#2439)
2022-01-31 - 736fabf05 - Single binary server (SQLite) and concurrency bugfixes for in memory storages (#2114)
2022-01-31 - 460071313 - Add RawTokenKeyProvider (#2420)
2022-01-31 - 426621d7c - Handle buffered events during namespace migration / promotion (#2440)
2022-01-28 - f4c4ab66e - Fix task queue user latency (#2431)
2022-01-28 - c0cac88f6 - Allow global namespace update from all clusters (#2429)
2022-01-28 - a7ab9a2e0 - Allow remove cluster from namespace cluster list (#2432)
2022-01-28 - 6c1f2ffee - Set git revision at build from env variable (#2428)
2022-01-28 - 46376c0f8 - Integration test: retry with empty task response (#2433)
2022-01-28 - 31dc2f81c - Moved Cassandra test utility methods to a new file so they can be used outside of temporal repo (#2434)
2022-01-27 - fcd3148ba - Add resource_exhausted_cause metrics tag (#2423)
2022-01-27 - f9c6ddf27 - Add DeleteWorkflowExecution API to history service (#2311)
2022-01-27 - b80a54fa6 - Updated schema tool to work with full semantic versions, i.e., MAJOR.MINOR.PATCH (#2417)
2022-01-27 - aceda146c - Update XDC integration test to use add remote cluster api (#2424)
2022-01-27 - 73887831f - Allow update initial failover version and version increment (#2425)
2022-01-26 - e5c7c7c51 - Handle namespaceNotActive error in parentClosePolicy workflow (#2394)
2022-01-26 - b87d60512 - Fix fake cluster for empty version (#2376)
2022-01-26 - b04265051 - Update replication fetcher lifecycle (#2421)
2022-01-26 - a69c45c31 - Expose pending workflow task information (#2416)
2022-01-26 - 735e05757 - Add metrics on replication lag and replication task apply latency (#2404)
2022-01-26 - 6ebaa80d4 - Add dynamic config for force search attributes refresh (#2422)
2022-01-26 - 0bf97af00 - Remove namespace from GetSystemInfo call (#2418)
2022-01-25 - 57cdbe3e0 - Correct branch name in docker image build pipeline (#2413)
2022-01-25 - 2a887797e - Add pipeline to trigger Temporal docker images build (#2399)
2022-01-24 - ab519c925 - Replace Cassandra visibility TTL with explicit DELETE (#2387)
2022-01-24 - 87dc0efeb - Handle SQL duplicate key case (#2410)
2022-01-24 - 410b5933d - Separate history engine factory from handler (#2409)
2022-01-24 - 3377e6432 - Verify run IDs in archival int test (#2411)
2022-01-21 - aff2a8299 - Use base-server:1.5.0 after recent update (#2402)
2022-01-21 - 65e4958c8 - Check SDK versions via version check call (#2365)
2022-01-21 - 43b453127 - Extract workflow execution delete code to workflow.DeleteManager (#2388)
2022-01-20 - e16c038f1 - Added ExecutionInfo field to InternalWorkflowSnapshot (#2392)
2022-01-20 - c74cb4989 - Fix c symbol in caCert variable name (#2397)
2022-01-20 - b8a36d64e - Switch force-replication to scan over visibility (#2386)
2022-01-20 - aa69178fd - Build docker image using Buildkite (#2400)
2022-01-20 - 7e9a7897b - Remove go mod init from dockerize build in base-server.Dockerfile (#2393)
2022-01-20 - 1e191789e - Add validation for invalid starttime filter for standard visibility (#2401)
2022-01-20 - 17f330dac - Wire up cluster metadata in start fx DI (#2398)
2022-01-20 - 0ff1b16d7 - Enable cgo by default and disable it explicitly for docker image build (#2396)
2022-01-20 - 07cda1bdb - Apply parentClosePolicy to child workflow only (#2390)
2022-01-19 - 28f78d54c - Separate SQL session management into a dedicated file (#2391)
2022-01-18 - 26d1806ee - Update OpenTelemetry to v1.2.0 and OpenTelemetry Metrics to v0.25.0 (#2385)
2022-01-18 - 24ca4e8f2 - Renamed InternalWorkflowSnapshot.ExecutionInfo to ExecutionInfoBlob. (#2384)
2022-01-14 - 773e18a6c - [Docker build] fix defaults with variables in templates (#2378)
2022-01-14 - 71093d7c5 - Trigger history node validation & trimming logic within write path (#2379)
2022-01-13 - eb95443d4 - Add admin api to list all cluster information (#2377)
2022-01-13 - 72e5b67b3 - Remove dummy registrations in sdk integration test (#2349)
2022-01-12 - f3780d0c5 - Option to enable forwarding all APIs (#2367)
2022-01-12 - ba30aa8ab - Check max taskID instead of max read level when process task (#2371)
2022-01-12 - 434ec0e3d - Favor local config value on enable global namespace (#2368)
2022-01-12 - 12bbccb68 - Add more tests to history cache (#2373)
2022-01-12 - 00a49ab57 - Remove removal function away from LRU cache for simplicity (#2372)
2022-01-11 - dc208e95e - add basic rate limit to force replication workflow (#2364)
2022-01-11 - c61e2c8bd - Support visibility dual write to different ES indices (#2359)
2022-01-11 - b3c858698 - Include more API to Cassandra read after write inconsistency protection (#2366)
2022-01-11 - aa6286665 - Unload shard if loaded task ID is greater than max transfer ID (#2361)
2022-01-11 - 86605c4d1 - [Docker build] add separate visibility env vars for different db instance (#2362)
2022-01-11 - 6e4cdc0cc - Remove cluster metadata dual write (#2358)
2022-01-10 - b33d24199 - make tests more extensible/reusable by other tests (#2357)
2022-01-07 - d01b1c7d1 - Dedup SignalWithStart based on RequestId (#2356)
2022-01-07 - b0e2c05a8 - Reduce heartbeat timeout for flaky test (#2351)
2022-01-07 - 16bf19dcb - Use actual worker in onebox setup. (#2352)
2022-01-06 - ffc60714b - Unload shard when ownership lost (#2355)
2022-01-06 - fde1e4e39 - Detecting stale cache when RecordChildExecutionCompleted (#2354)
2022-01-06 - cf6c03edc - Use latest worker identity for pending activity (#2350)
2022-01-06 - b13fccf6c - Set Fossa to run non-blocking (#2348)
2022-01-06 - 9ed152f0b - Fix LRU cache eviction behavior (#2353)
2022-01-06 - 46eafc910 - add link to explain docker builds and auto setup (#2346)
2022-01-06 - 327591b5a - Use error message for tiered storage (#2347)
2022-01-05 - ec65088fe - Disallow marking activity as cancelled without activity being request cancelled first (#2344)
2022-01-05 - 7a695e53e - Replace abandoned dgrijalva/jwt-go with golang-jwt/jwt (#2339)
2022-01-05 - 412035f20 - Rewrite TestCronWorkflow_Failed/Succeeded as SDK integration tests (#2345)
2022-01-04 - dc0c75a0a - Add missing err returns (#2250)
2022-01-04 - 7ae02e317 - Add logging to matching / history client (#2340)
2022-01-03 - f689eeba7 - Cleans up some comments for ServerOptions #2337
2022-01-03 - 4d9faa2c8 - Initialize shard controller with fx (#2319)
2021-12-30 - 733a0525a - Fix fail workflow task and generate new workflow task (#2335)
2021-12-30 - 633e45693 - Initialize shard correctly in ExecutionMutableStateSuite (#2336)
2021-12-30 - 05f584cc5 - Notify new tasks when workflow is potentially updated (#2334)
2021-12-29 - e9a1cc956 - Update failover version for namespace promote (#2332)
2021-12-29 - 26278fd4d - Add metrics to ns migration workflows (#2326)
2021-12-29 - 1e0266437 - Use page token when list cluster metadata from DB (#2331)
2021-12-28 - c191883ea - Add lag and pending tasks metrics for visibility queue (#2328)
2021-12-28 - 94ef18411 - Fix remote sync match for standby namespaces (#2327)
2021-12-28 - 81ce60406 - Fix queue diff metrics for disabled clusters (#2329)
2021-12-27 - e1fa8f2a9 - Fix transient workflow task check in getRawHistory (#2324)
2021-12-27 - d61a45474 - Fix reference count in DeleteHistoryBranch (#2323)
2021-12-27 - 097fa7960 - Fix flaky Test_ActivityTimeouts (#2325)
2021-12-20 - f61e90f21 - Update cron to v3.0 + CronSchedule Timezone support (#2215)
2021-12-20 - e61e8fe73 - Update DB task manager (#2310)
2021-12-20 - bdbfb605f - Use --target for docker builds (#2273)
2021-12-20 - 929c1cd37 - Add tzdata to server base image and binary (#2277)
2021-12-20 - 0f2bcc83c - New GetSystemInfo RPC (#2309)
2021-12-18 - c80744dff - Revert persistence rate limit error type (#2312)
2021-12-15 - c95a34eb3 - Matching sync match task should not have completion function (#2307)
2021-12-15 - a619fd4a6 - Make matching service executeWithRetry function stateless (#2305)
2021-12-15 - 8e9a8d49f - Use metrics.Client for sdk.metrics.Handler (#2302)
2021-12-15 - 8750b1cfe - Deflake/rewrite TestActivityHeartbeatDetailsDuringRetry test (#2304)
2021-12-15 - 67b06d647 - Add basic metrics unit tests (#2271)
2021-12-14 - adf5747f9 - Use persistence config consistency for schema version check (#2291)
2021-12-14 - 18d1d068b - New matching task flush buffer (#2286)

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use tag 1.15.0)

Server
Server With Auto Setup (what is Auto-Setup?)
Admin-Tools

Release Highlights

This release includes fix for golang security vulnerability CVE-2021-44716 (#2451) and minor bug fixes. See full list of patches below.

All changes

2022-02-10 - 7788b134b - Update versionChecker to 1.14.5
2022-02-10 - ae637fa90 - Upgrade base images (#2467)
2022-02-10 - c41819a76 - Move search attribute validator and mapper calls to frontend service (#2476)
2022-02-10 - f8ba073d0 - Handle SQL duplicate key case (#2410)

Details about v1.14.0 release can be found here.

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use tag 1.14.5)

Server
Server With Auto Setup (what is Auto-Setup?)
Admin-Tools

Release Highlights

This release includes fix for golang security vulnerability CVE-2021-44716 (#2451) and minor bug fixes. See full list of patches below.

All changes

2022-02-10 - 3532e47da - Update versionChecker to 1.13.4
2022-02-10 - 81166f637 - Move search attribute validator and mapper calls to frontend service (#2476)
2022-02-10 - c35109634 - Handle SQL duplicate key case (#2410)
2022-02-10 - f7488a045 - Upgrade base images (#2467)

Details about v1.13.0 release can be found here.

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use tag 1.13.4)

Server
Server With Auto Setup (what is Auto-Setup?)
Admin-Tools

Release Highlights

This release includes minor bug fixes for adding remote cluster. See full list of patches below.

All changes

2022-01-21 - 5f418b916 - Update version checker to 1.14.4
2022-01-21 - 75bf59f55 - Favor local config value on enable global namespace (#2368)
2022-01-21 - 4dd443267 - Add validation for invalid starttime filter for standard visibility (#2401)
2022-01-21 - 2e94883ac - Fix c symbol in caCert variable name (#2397)
2022-01-21 - 2dbf144ac - Wire up cluster metadata in start fx DI (#2398)
2022-01-20 - fc9390e61 - Add logging to matching / history client (#2340)

Details about v1.14.0 release can be found here.

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use tag 1.14.4)

Server
Server With Auto Setup (what is Auto-Setup?)
Admin-Tools

Release Highlights

This release includes a series of fixes:

• Fixed bugs in history service that could potentially miss tasks in failure mode when history pods are bouncing around.
• Updated docker config to allow different DB address for main and visibility DB.
• Support visibility dual write to 2 advanced visibility persistence.
• Apply trim logic for stale history events to PollWorkflowTaskQueue.

All changes

2022-01-18 - 71c697917 - Update versionChecker for 1.14.3
2022-01-13 - c2124f736 - [Docker build] fix defaults with variables in templates (#2378)
2022-01-12 - 72aba0e5b - Unload shard if loaded task ID is greater than max transfer ID (#2361)
2022-01-12 - 3c351f233 - [Docker build] add separate visibility env vars for different db instance (#2362)
2022-01-12 - 319861d4b - Support visibility dual write to different ES indices (#2359)
2022-01-12 - 2457d11b9 - Check max taskID instead of max read level when process task (#2371)
2022-01-12 - 1a3cf49e5 - Include more API to Cassandra read after write inconsistency protection (#2366)

Details about v1.14.0 release can be found here.

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use tag 1.14.3)

Server
Server With Auto Setup (what is Auto-Setup?)
Admin-Tools

Release Highlights

This release includes hotfixes for matching service when dealing with large number of low traffic task queues.
See detailed change list below.

All changes

2022-01-12 - c0436c842 - Update versionChecker to 1.13.3
2022-01-12 - 429291f96 - Move ackLevel past gaps in task ids (#2281)

Details about v1.13.0 release can be found here.

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use tag 1.13.3)

Server
Server With Auto Setup
Admin-Tools

Release Highlights

This release contains bug that could potentially lose tasks in failure scenario when history pods are not stable and keep bouncing between multiple nodes. Please upgrade to latest patch release 1.14.3 or later.

This release includes a series of fixes:

• Disallow marking activity as cancelled without activity being request cancelled first
• Fix incorrect LRU cache eviction behavior
• Detect stale cache when RecordChildExecutionCompleted
• Eagerly unload shard when ownership is lost

All changes

2022-01-10 - b535b87fb - Update CLI and ServerVersion in version checker for v1.14.2
2022-01-07 - 9ca8598a9 - Detecting stale cache when RecordChildExecutionCompleted (#2354)
2022-01-07 - 94b29c3b9 - Disallow marking activity as cancelled without activity being request cancelled first (#2344)
2022-01-07 - 80cd5f557 - Fix LRU cache eviction behavior (#2353)
2022-01-07 - 5efdf07a4 - Unload shard when ownership lost (#2355)

Details about v1.14.0 release can be found here.

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use tag 1.14.2)

Server
Server With Auto Setup (what is Auto-Setup?)
Admin-Tools

Release Highlights

This release contains a bug that could potentially lose tasks in failure scenario when history pods are not stable and keep bouncing between multiple nodes. Please upgrade to latest patch release 1.14.3 or later.

This release includes a hotfix for persistence stability at high loads.

All changes

2021-12-14 - baf3b94b4 - Use persistence config consistency for schema version check (#2291)
2021-12-20 - f9c1eba85 - Revert persistence rate limit error type (#2312) (#2315)
2021-12-20 - f7abe2db4 - Update CLI and Server versions in version checker (#2317)

Details about v1.14.0 release can be found here.

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use tag 1.14.1)

Server
Server With Auto Setup (what is Auto-Setup?)
Admin-Tools

Release Highlights

This release includes a hotfix for persistence stability at high loads.

All changes

2021-12-20 - ce005f641 - Revert persistence rate limit error type (#2312) (#2314)
2021-12-20 - 5fee1718c - Update CLI and Server versions in version checker (#2316)

Details about v1.13.0 release can be found here.

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use tag 1.13.2)

Server
Server With Auto Setup
Admin-Tools

Schema changes

This release contains a bug that could potentially lose tasks in failure scenario when history pods are not stable and keep bouncing between multiple nodes. Please upgrade to latest patch release 1.14.3 or later.

Before upgrading to your Temporal Cluster to this release 1.14, you must upgrade your storage schema version to the following:

• MySQL schema version 1.7
• PostgreSQL schema version 1.7
• Cassandra schema version 1.6

Use the Schema Upgrade Tool to upgrade your schema version.
How to use the Schema Upgrade Tool: https://docs.temporal.io/docs/server/versions-and-dependencies/#upgrade-your-version-of-temporal.

Note that schema version is NOT the same as the storage type (database) version, see “Latest SQL database support” below.

tctl

• tctl batch - Added a flag to limit amount of goroutines: --concurrency.
• tctl admin - Removed direct database connection and switched to Admin API. Removed flags related to database connection.
• Added a cluster command to upsert or remove remote cluster. This feature is still in development and may change in future release.
• Renamed --min/max_read_level flag to --min/max_task_id when listing cluster tasks.
• tctl namespace - Added a flag to promote a local namespace to a global namespace: --promote-namespace.
• Removed commands related to Elasticsearch.
• Removed cluster metadata command.

Metrics

• Added option to configure histogram buckets per unit type (milliseconds, bytes, dimensionless). Existing metrics now have assigned relevant unit types. Example.
• Added config option to exclude reported metric tags. Example.
• Upgraded uber-go/Tally to v4.0.1.
• Upgraded opentelemetry version to v1.1.0.

Latest SQL database support

• Latest version of MySQL and PostgreSQL were tested. Temporal now officially supports MySQL 8 starting from version 8.0.19 and PostgreSQL 13 starting from version 13.5.

Multi-Cluster

• Cluster metadata definition is migrating from application static config file to a new DB table. Cluster metadata will be initialized once when the service first starts up. Future remote cluster metadata modifications need to be done via tctl admin cluster tool. Example.

Other

• Server will fail to start if dynamic config is malformed. Previously, default config was used if errors occurred.
• Upgraded golang to 1.17.
• Added safety check for empty JWT source URIs.
• Added Handover ReplicationState for graceful failover. This feature is still in development and will change in a future release.

Improvements/Fixes

• Fix unexpected matching GetTasks call growth on low-traffic task queues.
• Reduced overhead due to unloading history shards during short database unavailability.
• Limited Elasticsearch document _id to 512 bytes (#2112).

Internal refactoring

• Adopted uber/fx for server initialization and dependency injection.

All changes

2021-12-13 - 074d8ebb1 - Fix rollup metrics aggregation (#2298)
2021-12-10 - 29e81571b - New future implementation (#2265)
2021-12-09 - f684386fe - Rename simpleStore to standardStore (#2278)
2021-12-09 - 7feb6b3a5 - Replication handover state (#2280)
2021-12-09 - 2eaadeaad - Move ackLevel past gaps in task ids (#2281)
2021-12-08 - c4976c9ea - Only update maxReadLevel after successful re-acquire of shard (#2272)
2021-12-08 - 0e844bc3b - Add additional UTF-8 validation (#2276)
2021-12-07 - 88c6eb5d7 - patch tally histogram (#2270)
2021-12-06 - f430f8573 - Stop server start up on malformed dynamic config (#2268)
2021-12-06 - be0df0d3e - Use correct histogram type for dimensionless (#2264)
2021-12-06 - 93de50aa4 - Add custom buckets to rollup metrics (#2266)
2021-12-06 - 3f4de5243 - Fix Elasticsearch bulk processor metrics units (#2267)
2021-12-03 - beb9a5a15 - Remove namespace APIs from adminservice (#2261)
2021-12-03 - 8a409cd25 - Add option to configure per metric unit histogram buckets. (#2251)
2021-12-03 - 827c92225 - Fix flaky TestVisibility (#2259)
2021-12-03 - 3e19fff5c - Fix batcher (#2260)
2021-12-02 - f338d10d1 - Remove DB record version config (#2253)
2021-12-02 - afc775a7b - Fix onebox worker shutdown (#2255)
2021-12-02 - 822eed7f5 - handover workflow (#2258)
2021-12-02 - 725730697 - Use history event as signal payload data (#2254)
2021-12-02 - 68498e122 - Remove --state from namespace update command (#2257)
2021-12-02 - 27c97af32 - Add shard lifetime context (#2218)
2021-12-02 - 06bd292b5 - Add UpdateNamespace tests for DELETED namespace state (#2185)
2021-12-01 - c9315c3e4 - Add dedicated persistence execution mutable state integration tests (#2244)
2021-12-01 - c099c7963 - Support limited list workflow queries for standard  visibility (#2197)
2021-12-01 - aa17a4aba - Matching cleanup (#2246)
2021-12-01 - 873f28085 - Add handover namespaces to GetReplicationStatus response (#2248)
2021-12-01 - 815bfd7d0 - Fix error messages (#2247)
2021-12-01 - 799a3c1e3 - Inserted retries to avoid flaky persistence (#2232)
2021-12-01 - 4f28e32b4 - Add dedicated persistence execution mutable state integration tests (#2245)
2021-12-01 - 2a47cdf0e - Replace ldflags with go:embed (#2085)
2021-12-01 - 0d9fcd3eb - Deprecate static cluster metadata (#2231)
2021-11-30 - d49000975 - Upgrade golang to 1.17.3 (#2240)
2021-11-30 - be7482878 - Add dedicated persistence execution mutable state integration tests (#2241)
2021-11-30 - b5e343cea - Remove resource.Resource from temporal services (#2229)
2021-11-30 - 4a4cc58b4 - Add Makefile targets for arm64 (#2201)
2021-11-30 - 1edb0a6b1 - Add dedicated persistence execution mutable state integration tests (#2242)
2021-11-30 - 171d228fc - Remove duplicated history size from execution info (#2239)
2021-11-29 - f888c2840 - Namespace Handover (#2228)
2021-11-29 - a1a237022 - Fix forward / backward compatibility issue (#2237)
2021-11-29 - 6daf3b6fb - Remove unnecessary CreateWorkflowModeContinueAsNew mode (#2235)
2021-11-29 - 6c139012d - Fix ParentClosePolicy not respected cross-namespace (#2224)
2021-11-29 - 1c52e90b6 - Remove unnecessary start version from current execution table (#2236)
2021-11-24 - e741533d4 - Add testhelper package to create temp file and dir (#2214)
2021-11-24 - a0e896524 - Add interleaved weighted round robin scheduler (#2209)
2021-11-24 - 876f75ce7 - Introduce dynamic cluster metadata component (#2208)
2021-11-24 - 182195552 - Namespace state update fix (#2227)
2021-11-24 - 0f2da592b - Add enable remote connection tctl (#2213)
2021-11-24 - 0baca26c8 - Use proper logger for worker service (#2226)
2021-11-23 - fc4024331 - Switch to Admin API in tctl list task queue tasks (#2222)
2021-11-23 - c3fdcb813 - Extract common struct for service provider args (#2223)
2021-11-23 - 90e89dd7c - Create admin API to list shard tasks (#2200)
2021-11-23 - 78acd4d7b - Use context for shard acquisition timeout (#2219)
2021-11-23 - 769b69f41 - Switch to admin API in tctl list shard tasks by category (#2217)
2021-11-23 - 6c86649b9 - Create admin API to get task queue tasks (#2221)
2021-11-23 - 1bbc3fbd7 - Refactor tctl admin show workflow raw history, switch to admin API (#2225)
2021-11-22 - 69959bc5e - Allow namespace state update (#2220)
2021-11-22 - 17b9d4cae - Pin MySQL version to 8.0.19 (#2210)
2021-11-19 - c9579c2ef - Handle missing metrics config (#2205)
2021-11-19 - c24eaa4be - Add List Cluster Metadata API (#2206)
2021-11-19 - 67da8a3b9 - Use gopkg.in/yaml.v3 instead of v2 (#2203)
2021-11-19 - 21531bab5 - Cleanup serverOptions depency and add server metrics client (#2187)
2021-11-19 - 1dd76d3f7 - Wait replications (#2202)
2021-11-18 - e5740fa22 - Use t.Setenv function to set env in unit tests (#2196)
2021-11-18 - 66780893b - Update development environment to MySQL 8.0 (#2193)
2021-11-18 - 6257ac319 - Change shard persistence interface to get-or-create (#2199)
2021-11-18 - 590d9dc98 - Implement GetReplicationStatus (#2195)
2021-11-18 - 39bd317c7 - Update development environment to PostgreSQL 13.5 (#2194)
2021-11-18 - 2eb40fe73 - Add fan out to history client for GetReplicationStatus (#2198)
2021-11-18 - 00cb5194c - Add tiered storage queue processor (#2189)
2021-11-17 - ca556868a - Add GetReplicationStatus to history service (#2186)
2021-11-17 - 9a4982999 - Remove Elasticsearch admin commands (#2181)
2021-11-17 - 14fe930b3 - Switch to admin API in tctl register namespace (#2176)
2021-11-16 - e6757b9ed - Add try lock APIs to priority mutex (#2183)
2021-11-16 - de83ad266 - Add NamespaceValidator interceptor (#2169)
2021-11-16 - 407184d92 - Add IsLocked to priority mutex (#2180)
2021-11-16 - 196b57afc - Create admin API to update namespace (#2182)
2021-11-15 - b9b0615a5 - Create admin API to register namespace (#2175)
2021-11-15 - 58e380a9e - Make cluster metadata schema backward compatible (#2177)
2021-11-15 - 58af5da4d - Header support for signals and queries (#2149)
2021-11-15 - 43ac47ffb - Report acked replication task timestamp (#2173)
2021-11-15 - 170bde6ac - Use admin API for tctl list cluster members (#2162)
2021-11-14 - 6bfe955d7 - Support concurrency flag in the tctl batch command (#2168)
2021-11-12 - f1cbac2de - Disable fx logs (#2091)
2021-11-12 - b9edf16e6 - Create admin API to list namespaces (#2170)
2021-11-12 - a439a679e - Wire up cluster metadata APIs (#2174)
2021-11-12 - 5d6bb62b1 - Update opentelemetry version and add gauge cache. (#2161)
2021-11-12 - 59c571530 - Use admin API for tctl list namespaces (#2171)
2021-11-12 - 519bffb7d - Move DeleteWorkflowExecution into shard context (#2138)
2021-11-12 - 1198962e9 - Create admin API to list cluster members (#2159)
2021-11-11 - f141698a0 - Initialize SignalRequestedIDs correctly (#2167)
2021-11-11 - dc60fffe3 - Fix test behavior (#2166)
2021-11-11 - 725a4de90 - Add new admin APIs to manage remote cluster info (#2163)
2021-11-11 - 60fb3cd65 - Add protection for empty JWT key source URIs (#2142)
2021-11-11 - 3f1eb310e - Set VisibilityTimestamp on ReplicationTask (#2165)
2021-11-11 - 13082f322 - Migrate cluster metadata store to use cluster metadata info table (#2155)
2021-11-11 - 0f70d6192 - Matching GetTasks call reduction (#2164)
2021-11-10 - 8c308a0d7 - Add custom grpc inspectors (#2156)
2021-11-10 - 62ce8c584 - Move namespace name validation to RegisterNamespace (#2160)
2021-11-10 - 590b6eaeb - Add GetAllKeys to keyResolver (#2152)
2021-11-09 - e25f5dc7b - Remove command tctl admin cluster metadata (#2151)
2021-11-08 - f35895c9b - Add RequestID validation for ResetWorkflow (#2141)
2021-11-08 - e9f8078d3 - Extend admin Describe Cluster API response with cluster metadata (#2150)
2021-11-08 - c5baafa11 - Add new table for cluster metadata info and update cluster metadata struct (#2147)
2021-11-08 - 5561f2a48 - Add support exclusion of tags from metrics (#2137)
2021-11-08 - 31a576749 - Fix potential nil pointer use in shard context (#2148)
2021-11-07 - 4fb8100a9 - fix flaky TestActivityHeartbeatFailover (#2145)
2021-11-06 - f8fc78efd - Refactor worker for addsearchattributes using fx (#2144)
2021-11-05 - 62e85b420 - Update error type (#2135)
2021-11-05 - 1e161ebc9 - Add gen-replication workflow (#2139)
2021-11-04 - 6ad2edaf0 - Fix GenerateLastHistoryReplicationTask to use shard.AddTasks() (#2136)
2021-11-04 - 203ccb085 - Refactor shard context lifecycle (#2102)
2021-11-04 - 0917c302b - Remove sampled retention (#2133)
2021-11-03 - f463f0dbc - Move generation of replication task logic to task generator (#2127)
2021-11-03 - 851aaec89 - Use correct absolute paths to scripts in dockerfile (#2123)
2021-11-03 - 67b606425 - Add generic task processor (#2120)
2021-11-03 - 60a52ee1d - update cli usage (#2124)
2021-11-03 - 5acdf8654 - Add GenerateLastHistoryReplicationTasks to history service (#2129)
2021-11-03 - 577de1ad8 - Add worker manager to worker service (#2126)
2021-11-01 - ea45be2d7 - Move timer gate to common package (#2121)
2021-11-01 - c28665dd1 - Promote local namespace to global namespace (#2105)
2021-11-01 - 7e8b9650d - Specify the entrypoint absolutely (#2118)
2021-10-29 - f253ddfc0 - Patch tally import (#2115)
2021-10-29 - ed0743d90 - Remove dependency on resource from worker (#2098)
2021-10-29 - ae959113d - Add http status metric tag to bulk processor errors (#2113)
2021-10-29 - 744f26d28 - Use strong types for namespace Name and Id (#2082)
2021-10-29 - 48f78b781 - Add metrics for loaded task queue count as gauge (#2111)
2021-10-29 - 181ad20ae - Bump Web to v1.13.0 (#2110)
2021-10-28 - fc422185f - Limit Elasticsearch document _id to 512 bytes (#2112)
2021-10-28 - d2405acd2 - Make SQL CLI tool more idempotent (#2104)
2021-10-28 - 3f6253069 - Extract SearchSource object in Elasticsearch client (#2106)
2021-10-28 - 28db61ac2 - Added ability to specify dataconverter port via flag (#2093)
2021-10-28 - 0e67cd904 - Use PPROF_PORT env to configure pprof port in config_template.yaml (#2109)
2021-10-27 - e93f4b23c - Finish task serialization / deserialization (#2083)
2021-10-27 - 88473687e - Update submodules and go.temporal.io/api (#2103)
2021-10-27 - 1503e924e - Upgrade SDK and Tally (#2100)
2021-10-26 - ede54d90f - Add in flight poll activity / workflow task queue metrics (#2095)
2021-10-26 - cb9462ab2 - Shard context cleanups: private eventsCache, remove SetEngine (#2086)
2021-10-26 - af181b404 - Add per namespace burst limit control (#2067)
2021-10-26 - 9c64e2e8d - Remove deprecated task ID field from replication task (#2078)
2021-10-26 - 98814a00e - Remove half done task processing restructure from Cadence (#2074)
2021-10-26 - 8b4936a24 - Remove unnecessary method on history tasks (#2072)
2021-10-26 - 3ce382a90 - Shard context cleanups: [R]Lock/Unlock should be private (#2087)
2021-10-26 - 03fde05be - Move history tasks definition to history package (#2070)
2021-10-26 - 00388db7f - Rename workflow identifier to workflow key (#2071)
2021-10-25 - d83e601c0 - Return deserialization tasks as is from persistence layer (#2042)
2021-10-25 - d7a2d5609 - Post-release tasks (#2092)
2021-10-25 - d6771efef - Pass exit signals to temporal-server (#2069)
2021-10-25 - bdeff26cf - Move from io/ioutil to io and os package (#2062)
2021-10-25 - b3731a43b - Shard context cleanups: split two remove shard paths, don't hold engine in workflow context (#2088)
2021-10-25 - 9a3c57741 - Admin API GetShard (#2075)
2021-10-25 - 92a42adb8 - Allow port forwarding to the tctl web dataconverter (#2089)
2021-10-25 - 779f64591 - Init server via uber/fx (#2034)

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use tag 1.14.0)

Server
Server With Auto Setup
Admin-Tools

Release Highlights

This release includes a hotfix for long Elasticsearch document _id field values. You don't need to upgrade if your Workflow Ids are shorter than 475 bytes.

2021-10-28 - fc422185f - Limit Elasticsearch document _id to 512 bytes (#2112)

Details about v1.13.0 release can be found here.

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use tag 1.13.1)

Server
Server With Auto Setup
Admin-Tools

Release Highlights

This release does not includes any schema changes.

Notable changes

Expose Closed status of Temporal Cron Workflow Executions and Workflow Executions that retry

Previously Temporal Cron Workflow Executions and Workflow Executions that retried always ended with a "Continued-As-New" status, now they have a final event like any other, with a "Completed" or "Failed" or "Timed out" status. If you have code that depends on the "Continued-As-New" status you'll need to update it.

Custom search attributes changes

String search attribute type was renamed to Text to better reflect its nature and match Elasticsearch type. String is still supported as deprecated alias to Text and will be removed.

Visibility changes

Many small improvements and bug fixes in visibility.

The docker container now runs as a non-root user

You should make sure you're using https://github.com/temporalio/helm-charts/releases/tag/v1.13.0 which contains a corresponding change to the Kubernetes securityContext.

Dynamic config validator

If dynamic config is malformed an error message will be written to the log. In the next version, a malformed dynamic config will prevent the server from starting. If you see error message about dynamic config, please fix it.

SQLite persistence

Many thanks to Jacob LeGrone (DataDog) for his contribution of the SQLite driver (#1968, #2029). NOTE: The SQLite driver is still under development.

Metrics

• Added task_schedule_to_start_latency metric.

Bug fixes

• Correctly set shard ID when range complete replication tasks (#2056)
• Correctly set last event task ID as part of events replication (#2055)
• Fix match task queue rate calculation issue (#1910)

All changes

2021-10-20 - 6cbfa2a3a - Avoid acquiring shared lock recursively (#2076)
2021-10-18 - 92081a5a8 - Handle case of empty string CAs in TLS config (#2063)
2021-10-18 - 87a89e682 - Adding snipsync wrappers to setup commands (#2051)
2021-10-18 - 83c060ac7 - Remove read-through from namespace.Registry (#2066)
2021-10-18 - 7f6a954e0 - Rename INDEXED_VALUE_TYPE_STRING to INDEXED_VALUE_TYPE_TEXT (#2061)
2021-10-18 - 7c83fa4b0 - Avoid generating empty string in tls cert array (#2064)
2021-10-18 - 077d39c77 - Remove awk dependency in auto-setup.sh (#2068)
2021-10-15 - f955633ed - Make shard info to 0 unix nano time, instead of default time.Time (#2052)
2021-10-15 - c121a7d3b - Correctly set shard ID when range complete replication tasks (#2056)
2021-10-15 - 1d7b63b10 - Added php and typescript sdks to supportedClients (#2058)
2021-10-14 - 13fef8a74 - Correctly set last event task ID as part of events replication (#2055)
2021-10-14 - 10df97be0 - Adding temporal-cla and update link (#2053)
2021-10-14 - 0a92fc6d7 - Sync Elasticsearch index create script between Makefile and auto-setup.sh (#2047)
2021-10-14 - 043c97498 - add optional explicit ForceTLS option to ClientTLS config (#2036)
2021-10-13 - 861df80f1 - Unify workflow completion metrics (#2050)
2021-10-12 - cbe6c18c5 - Add metrics for workflow_continued_as_new counter (#2048)
2021-10-12 - c18e7cc30 - Use uber/fx to init worker service (#2041)
2021-10-12 - 4c2fa1794 - Remove duplicate code (#2046)
2021-10-12 - 467e5291a - Persistence rate limit error (#2049)
2021-10-12 - 1a811e079 - Refactor visibility manager (#2045)
2021-10-11 - 4ee976855 - Use time.Unix(0, 0) instead of time.Time (#2044)
2021-10-11 - 00f8c7cb6 - Use Match query instead of MatchPhrase for Elasticsearch (#2037)
2021-10-09 - e08cd0541 - Revert "Use uber/fx to init worker service (#2032)" (#2039)
2021-10-08 - f687b5b34 - Use time.Unix(0, 0) instead of time.Time (#2038)
2021-10-08 - a3649987e - Use uber/fx to init worker service (#2032)
2021-10-08 - 86d715f22 - Fix buffered query timeout issue (#2033)
2021-10-08 - 53f438a5f - Use uber/fx to init frontend (#2023)
2021-10-08 - 3e5196b1b - Log error message during server startup if dynamic config is misconfigured (#2019)
2021-10-08 - 08b48711b - Add schema setup and namespace registration utilities for SQLite (#2029)
2021-10-07 - fdea07050 - Remove video.png (#2028)
2021-10-07 - b621a6546 - Convert namespace.Registry to read-through (#2020)
2021-10-07 - 99355bfb1 - Bump Web to v1.12.0 (#2031)
2021-10-07 - 6848bf1ba - Remove unused field (#2027)
2021-10-07 - 25fec58e8 - Use di for matching integration tests (#2024)
2021-10-05 - ad4dc3f1f - Remove task polling recursion (#2022)
2021-10-04 - bf2e5a653 - Remove namespace/ dependence on cluster metadata (#2011)
2021-10-04 - 2c52fd617 - Increase add-search-attributes tctl admin command timeout to 30s (#2021)
2021-10-04 - 25f551b59 - Separate Cassandra persistence related structs (#2016)
2021-10-04 - 06b863741 - Namespace type renames (#2018)
2021-10-03 - 8fdc8992e - Escape password in connection url (#2015)
2021-10-01 - fd4e2ca2d - Use DI for matching service initialization (#1995)
2021-10-01 - f831481fc - Fix bulk processor NPE (#2013)
2021-10-01 - e80e8c1a0 - unlimited MaxConnectionAge (#1998)
2021-10-01 - d9bc815d9 - Make compare function easy to use for task key (#2008)
2021-10-01 - c7b568e97 - Allow setting workflow run timeout in tctl (#2009)
2021-10-01 - 531cc9664 - Use StartedIdentity as LastWorkerIdentity for PendingActivity if no retry (#2007)
2021-10-01 - 37799989a - Fix elasticsearch_bulk_processor_wait_add_latency metric (#2012)
2021-10-01 - 27e9ff249 - Namespace Cache refactor to take advantage of immutability (#2006)
2021-10-01 - 16b126699 - Revert "unlimited MaxConnectionAge (#1998)" (#2010)
2021-09-30 - fff6fbb85 - Add elasticsearch_bulk_processor_duplicate_request metric (#2003)
2021-09-30 - f5ad4da3c - Add SDK commands sequence validation (#1993)
2021-09-30 - e0eb82daf - Use DI to inject new cache provider (#1997)
2021-09-30 - dc4cd740d - Remove namespace.CacheEntry.GetInfo() (#2000)
2021-09-30 - 6d8338aa2 - Remove advanced visibility specific errors (#2002)
2021-09-30 - 6120b46c4 - Remove namespace.CacheEntry.GetConfig() (#1991)
2021-09-30 - 57985cbdf - Improve mapper tests and comment message (#2005)
2021-09-30 - 45a3b798c - Correctly set non timer task key (#2004)
2021-09-30 - 26882f188 - Remove cache.Cache from workflow.Cache (#2001)
2021-09-29 - ae8c7978a - Remove CacheEntry.GetReplicationConfig (#1976)
2021-09-29 - 8df4cc86e - Improve message on injected errors (#1973)
2021-09-29 - 64ef4e223 - Add SQLite storage driver (#1968)
2021-09-29 - 3f29b8f7b - Add workflow identifier to all tasks (#1989)
2021-09-29 - 2e8f38e5c - Make workflow.Cache interface (#1990)
2021-09-27 - f31bc0b8a - Unify task serialization / deserialization (#1985)
2021-09-27 - bcf518bd9 - Utilize task serialization / deserialization util (#1987)
2021-09-27 - ab32caf07 - Add persistence/visibility store info to GetClusterInfo (#1984)
2021-09-27 - a63db6682 - Improve invalid search attribute error message (#1986)
2021-09-27 - 7af4d6577 - Move task definition from persistence layer to business logic layer (#1983)
2021-09-27 - 180db9247 - Update proto submodule (#1982)
2021-09-24 - f0d4f89a5 - Make new events for retry compatible with older SDK versions (#1949)
2021-09-24 - c7994e228 - Reorganize go.mod file to follow 1.17 convention (#1974)
2021-09-24 - 60a6a61ca - Add feature flag to enable Ringpop TLS (#1967)
2021-09-24 - 57938c28b - Initialize resource via DI (#1970)
2021-09-24 - 07cae17d8 - Add more details to invalid command attributes error (#1978)
2021-09-23 - ed49957ce - Update dashboards (#1969)
2021-09-23 - cc1370f3c - Remove default page size from Elasticsearch visibility store (#1971)
2021-09-23 - a38dbd10c - Use internode mTLS for ringpop (#1902)
2021-09-23 - 3f1aaa252 - Convert Elasticsearch BadRequest error to InvalidArgument error (#1573)
2021-09-23 - 0652f4106 - Set timeout for all Elasticsearch read calls (#1972)
2021-09-22 - d79e55874 - Return memo serialization error instead of logging (#1948)
2021-09-22 - d23e605eb - Construct history.Service via DI (#1964)
2021-09-22 - 81db9bf02 - Add more metrics for mutable state update (#1960)
2021-09-22 - 6356d64e9 - Move Elasticsearch client config to elasticsearch/client package (#1950)
2021-09-22 - 3cd95674b - Plumb fault injection for use in integration tests (#1957)
2021-09-22 - 302d03ba1 - Revert "Remove deprecated metrics config code as per release 1.11 note (#1959)" (#1961)
2021-09-22 - 220795cf3 - Integrate dashboards submodule (#1965)
2021-09-21 - f769507d6 - Move SearchAttributesManager to searchattribute package (#1953)
2021-09-21 - f2c216c44 - Consistent TaskQueueManager fatal signaling (#1933)
2021-09-21 - b20a256e8 - Make ResilientSyncMatch the default behavior (#1955)
2021-09-21 - 923a45c3f - Remove standard logger dependency (#1951)
2021-09-21 - 8dfbb7405 - Clean up worker README (#1954)
2021-09-21 - 67b416cc9 - Use internal error to indicate code bug / contract breach (#1958)
2021-09-21 - 584f28272 - Don't refresh search attributes cache if database is down (#1956)
2021-09-21 - 183739dcc - Remove deprecated metrics config code as per release 1.11 note (#1959)
2021-09-20 - d9162f358 - Fix coverage profile generator (#1947)
2021-09-20 - d6c9b4af8 - Add custom logger option to schema tool (#1943)
2021-09-20 - ce4f0364b - Use internal error to indicate code bug / contract breach (#1945)
2021-09-20 - 58393935e - Refactor visibility stores and managers (#1942)
2021-09-20 - 1ec6e9f1a - Ignore goveralls exit code (#1946)
2021-09-17 - ddb37af33 - Use persistence mutable state mutation / snapshot binary size util (#1935)
2021-09-17 - 8b71d20d9 - Remove deprecated history events from mutable state (#1937)
2021-09-17 - 08a15e384 - Fix comments for NewExecutionStore (#1939)
2021-09-16 - c73460c27 - Always return nil value for NextPageToken if there are no more pages (#1936)
2021-09-16 - 8b1993b7e - Use internal error to indicate code bug / contract breach (#1924)
2021-09-16 - 7223e7a53 - Update integration tests (#1934)
2021-09-16 - 463b75820 - Remove unused code (#1938)
2021-09-15 - ebe540796 - Use internal error to indicate code bug / contract breach (#1919)
2021-09-15 - d588c3ef3 - Simplify some uses of payloads.Decode in integration tests (#1889)
2021-09-15 - ba3943e86 - Remove taskQueueManager by instance rather than queue ID (#1931)
2021-09-15 - 9bac95d6d - Merge persistence API for append history and conflict resolve (#1926)
2021-09-15 - 847c449b3 - Add persistence mutable state mutation / snapshot binary size util (#1925)
2021-09-15 - 845c4e7fc - Extract VisibilityManager from Resources (#1932)
2021-09-15 - 6ed17e325 - Reset gauge on scan complete (#1930)
2021-09-15 - 340949ff3 - Unify visibility metrics (#1928)
2021-09-15 - 1ce94933b - Update event blog size warning message (#1923)
2021-09-15 - 117fd3ae1 - Set timeout for TestCluster (#1927)
2021-09-14 - fa4a34dc7 - Add schedule to start task metric (#1915)
2021-09-14 - ba4197f40 - Use point in time with Elasticsearch v7 only if it is available (#1918)
2021-09-14 - ab8c0f36d - Update SDK to v1.10.0 (#1921)
2021-09-14 - 032ce4e05 - Remove dead code (#1920)
2021-09-13 - 96de6faec - Unload taskQueueManager by instance, not queue ID (#1917)
2021-09-13 - 469b4e616 - Use serviceerror.Unavailable error type instead of serviceerror.Internal (#1916)
2021-09-11 - ba5be8004 - Ignore InvalidArgument error for unknown custom search attributes (#1911)
2021-09-11 - 09e682a9c - Remove missing keyword support in visibility queries (#1912)
2021-09-10 - 90d08e4de - Make TestContinueAsNewWorkflow_Timeout faster (#1908)
2021-09-10 - 56d1f9aa3 - Fix match task queue rate calculation issue (#1910)
2021-09-10 - 4cb5ae03c - Wait for namespace cache refresh without sleeping in integration tests (#1907)
2021-09-09 - e84ef4084 - Adds docs and examples to the goro package (#1900)
2021-09-09 - 12e6347af - Add metrics to bulk processor (#1909)
2021-09-09 - 0d6006624 - Additional doc updates for package goro (#1906)
2021-09-08 - ef889362a - Ack existing visibility task in case of duplicates (#1905)
2021-09-08 - b113b214d - Add goro.Group and use it in taskReader (#1894)
2021-09-08 - 32b36068c - Remove openssl from base server image (#1901)
2021-09-07 - ee93ab0c7 - Switch memo flags to StringSlice (#1859)
2021-09-07 - e81657bd3 - Fix TaskWriter panic when shutdown (#1892)
2021-09-07 - 9016e5dc4 - Remove retry from DeleteExecution (#1896)
2021-09-03 - f2b317b56 - Add searchattribute.Mapper interface for custom search attributes (#1880)
2021-09-03 - 4fb055503 - Move namespace cache into namespace package (#1884)
2021-09-03 - 263797637 - Paul MS cache update (#1887)
2021-09-02 - a447f2e1a - Correctly set current version if workflow is finished (#1879)
2021-09-02 - 32fd7ddd0 - Refactor Elasticsearch visibility store (#1888)
2021-09-02 - 24a081a2b - Remove search attributes errors logs (#1886)
2021-09-01 - 9509717d8 - Remove non-retriable case from fault injection (#1877)
2021-09-01 - 42bf6ef02 - Change parseESDoc and generateESDoc to return error instead of logging (#1878)
2021-08-31 - b37a91d91 - Add non-root user to run temporal server (#1814)
2021-08-31 - 988d865f8 - Patch metrics UserScope (#1876)
2021-08-31 - 811cc8bef - Expose status of cron and retry workflow executions (#1866)
2021-08-31 - 7fc934022 - Change custom search attributes (#1870)
2021-08-31 - 67c950d70 - Update dependencies (#1874)
2021-08-31 - 52be73e49 - Bump current version to 1.13.0 (#1873)
2021-08-31 - 24898d059 - Fix TestCronWorkflow and TestCronChildWorkflowExecution integration tests (#1872)

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use tag 1.13.0)

Server
Server With Auto Setup
Admin-Tools

Release Highlights

• Elasticsearch v7 is now supported: #1164 (see details below).
• Internal visibility queue is used by default (not Kafka): #1235 (see details below).
• Metrics improvements: #1216, #1223, #1227, #1231, #1232, #1245.
• Schema changes.
• Bug fixes.

Schema changes

This release includes schema changes.

Before upgrading your Temporal Server cluster, run the schema upgrate tool to support version 1.4.
Schema update tool instructions are here: https://docs.temporal.io/docs/server-versions-and-dependencies/#upgrade-your-version-of-temporal.

Elasticsearch v7 Support

Upgrading to Elasticsearch v7 (ES7) from ES6 includes a number of breaking changes.
The most important one is the removal of types.
Therefore the ES6 Go client is not compatible with ES7 and upgrading the Temporal Server is a multi step process.

Fresh install

If you are installing a fresh version of the Temporal Server, use ES7.
Install the latest ES7 version, and set the ES version in the Server's static config:

persistence:
  datastores:
    es-visibility:
      elasticsearch:
        version: "v7"

Or, set ES_VERSION env to v7 if you are using pre-build docker image.

Upgrade with temporary cluster shutdown

If you can afford any Temporal Server downtime while upgrading Elasticsearch, then simply shutdown your Temporal cluster and upgrade ES6 to ES7 following the steps from the Elasticsearch documentation.
If you are on the latest 6.8.x, just perform a rolling upgrade of Elasticsearch.

Rolling upgrade of the Temporal Server

You can also upgrade the Temporal Server cluster with some minimal downtime in enhanced visibility only.

1. Update the Server to this v1.7.0 release.
2. Add the following to the dynamic config:

history.visibilityProcessorEnabled:
  - value: false

3. Restart the Server.
   Workflow visibility information won't be updated.

4. Upgrade ES6 to ES7 following steps from the Elasticsearch documentation.
   If you are on the latest 6.8.x, just perform a rolling upgrade of Elasticsearch.

5. Start ES7.
   Visibility read queries will temporarily generate errors.
   Write queries are blocked because the processor is disabled.

6. Switch to ES7 in the Server's static config:

persistence:
  datastores:
    es-visibility:
      elasticsearch:
        version: "v7"

Or, set ES_VERSION env to v7 if you are using pre-build docker image.

7. Remove this from the dynamic config:

history.visibilityProcessorEnabled:
  - value: false

8. Restart the Server one more time.

Managed AWS Elasticsearch version

During our internal testing we found out that the latest AWS Elasticsearch version 7.9 has issues and can't be used.
If you are using AWS Elasticsearch upgrade to 7.7.

Internal visibility queue is used by default

This release uses internal visibility queues by default, instead of Kafka.
You may lose unprocessed Kafka messages if you are upgrading from version other than v1.6.4.
To prevent this, upgrade to v1.6.4 first or follow the upgrade steps from the v1.5.0 release.

In the next release all remaining Kafka code will be completely removed.

Change details

2021-01-21 - 1428a3da - Set default ES bulk processor ack timeout to 1 minute (#1210)
2021-01-21 - b46fbe3f - Add ES logger (#1209)
2021-01-22 - 3aa0d683 - Update Go SDK to 1.4.0 (#1211)
2021-01-22 - 95c1b5bc - Remove support for locking down admin operations (#1194)
2021-01-22 - d2fc34b6 - Improve PostgreSQL config (#1214)
2021-01-22 - f6225dea - Fix NDC metrics reporting (#1216)
2021-01-25 - 08252071 - Add more logging to ES (#1222)
2021-01-25 - 1e7c1903 - Fix metrics scope namespace tag initialization (#1223)
2021-01-25 - a4d1428c - Bugfix: activity heartbeat timeout (#1220)
2021-01-26 - 089f746c - Improve ES logging (#1228)
2021-01-26 - 30aed765 - Fix metrics tag name (#1227)
2021-01-26 - ba7be16d - Fix replication failure metrics (#1232)
2021-01-26 - cce754ac - Fix parent close metrics definition (#1231)
2021-01-26 - ec338589 - Bugfix: close execution request goes to Kafka even when visibility queue type is set to internalWithDualProcessor (#1229)
2021-01-27 - 485ae998 - Accept empty Kafka config when visibility queue is set to internal (#1238)
2021-01-27 - 532b3181 - Cleanup DB schema (#1130)
2021-01-27 - e12c7a15 - Switch visibility queue default to internal (#1235)
2021-01-28 - d3bc2a92 - Bugfix: NoopResolver was always used for version compatibility check (#1233)
2021-01-29 - ce445321 - Add password masker (#1217)
2021-02-01 - 23e5b8b0 - Fix distribution metrics issue (#1245)
2021-02-01 - 43954fdc - Adjust number of histogram buckets (#1248)
2021-02-01 - 520de19f - Correctly initialize prometheus reporter (#1250)
2021-02-02 - 69da5619 - Bugfix: Allow enabling TLS for sql tool without requiring key/cert. (#1252)
2021-02-02 - 9d9d8bf4 - Add server option for custom Elasticsearch HTTP client (#1246)
2021-02-02 - f3e33a95 - Set search.idle.after to 365 days (#1254)
2021-02-03 - 10884f0a - Deprecate token bucket (#1253)
2021-02-03 - 3e9e452d - Replace deprecated NewExitError with Exit method (#1257)
2021-02-03 - f7f9ac1c - Simplify matching engine rate limit logic (#1260)
2021-02-04 - 10205def - Improve matching engine error type (#1261)
2021-02-05 - c5ee4750 - Add logger to matching service (#1264)
2021-02-08 - e831c9d6 - Refactor TLS tests (#1266)
2021-02-09 - 7ea0ee04 - Add background executions table scanner (#1244)
2021-02-09 - 83018043 - Fix update schema command for temporal_visibility (#1268)
2021-02-09 - f6e2c136 - Create non-transient workflow task when version changes (#1272)
2021-02-10 - cf204719 - Set connection timeout for cassandra tool (#1277)
2021-02-11 - afc00012 - Add quotes to config_tempate.yaml (#1284)
2021-02-11 - c977c8b8 - Rename ArchiveVisibilityRequest to VisibilityRecord (#1281)
2021-02-12 - 4363aca9 - Use document Id to match request and response in ES processor afterFunc callback (#1265)
2021-02-12 - ee845110 - Copy visibilityAckLevel in copyShardInfo (#1287)
2021-02-16 - 289e1de9 - Enable dynamic update of TLS certificates (#1288)
2021-02-16 - e3496b1c - Add protection on child workflow ID (#1293)

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use tag 1.7.0)

Server
Server With Auto Setup
CLI (tctl)
Admin-Tools

Release Highlights

This release includes hotfix for the long Elasticsearch document _id field value. You don't need to upgrade if your Workflow Ids are shorter that 475 bytes.

2021-10-28 - fc422185f - Limit Elasticsearch document _id to 512 bytes (#2112)

Details about v1.12.0 release can be found here.

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use tag 1.12.4)

Server
Server With Auto Setup
CLI (tctl)
Admin-Tools

Release Highlights

This release includes hotfix for the matching engine and few improvements for the visibility metrics.

2021-10-07 - 037892933 - Unify visibility metrics (#1928)
2021-10-07 - 11f35c837 - Add metrics to bulk processor (#1909)
2021-10-07 - 63fe119da - Fix elasticsearch_bulk_processor_wait_add_latency metric (#2012)
2021-10-07 - 9c5468e37 - Fix bulk processor NPE (#2013)
2021-10-07 - d10b98be7 - Add elasticsearch_bulk_processor_duplicate_request metric (#2003)
2021-10-07 - f225894c9 - Remove task polling recursion (#2022)

Details about v1.12.0 release can be found here.

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use tag 1.12.3)

Server
Server With Auto Setup
CLI (tctl)
Admin-Tools

Release Highlights

This release includes small bug fix for custom search attributes custom mapper. You don't need it if you are not using custom mapper.

2021-09-11 - ba5be8004 - Ignore InvalidArgument error for unknown custom search attributes (#1911)

Details about v1.12.0 release can be found here.

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use tag 1.12.2)

Server
Server With Auto Setup
CLI (tctl)
Admin-Tools

Release Highlights

This release includes bug fixes for matching service and advanced visibility:

2021-09-16 - d8a538da2 - Bump current version to 1.12.1
2021-09-16 - b4b0ce8dd - Use point in time with Elasticsearch v7 only if it is available (#1918)
2021-09-16 - 7704dd2ee - Remove taskQueueManager by instance rather than queue ID (#1931)
2021-09-16 - 75a1a7d5e - Unload taskQueueManager by instance, not queue ID (#1917)
2021-09-16 - eb606bba0 - Fix match task queue rate calculation issue (#1910)
2021-09-10 - 077dd42c6 - Ack existing visibility task in case of duplicates (#1905)
2021-09-10 - 7af6ad332 - Fix TaskWriter panic when shutdown (#1892)
2021-09-10 - bb10f2d97 - Refactor Elasticsearch visibility store (#1888)
2021-09-10 - 1de4b2fac - Remove search attributes errors logs (#1886)
2021-09-10 - 69515153a - Add searchattribute.Mapper interface for custom search attributes (#1880)
2021-09-10 - 59c3ccbb8 - Change parseESDoc and generateESDoc to return error instead of logging (#1878)
2021-09-10 - 2b415aa6f - Patch metrics UserScope (#1876)
2021-09-10 - d817cb1c6 - Change custom search attributes (#1870)

Details about v1.12.0 release can be found here.

Helpful links to get you started with Temporal

Temporal Docs
Server
Docker Compose
Helm Chart

Docker images for this release (use tag 1.12.1)

Server
Server With Auto Setup
CLI (tctl)
Admin-Tools