Release notes

• Updates Gundeck to remove unused APNS_VOIP related code.
• This supersedes 2024-02-12 (Chart Release 4.40.0)

Release notes

• The settings setDisabledAPIVersions (brig) and disabledAPIVersions (in cannon, cargohold, galley, gundeck, proxy, and spar) are now required.
  The default defined in charts/<service>/values.yaml is set to [ development ] and disables all development API versions.
  For more information see https://docs.wire.com/developer/reference/config-options.html#disabling-api-versions (#3772)

• The mls team feature now has a lock status. If the current installation has default settings defined in wire-server's values.yaml, the lockStatus has to be added, e.g.:

  mls:
    defaults:
      status: enabled
      config:
        protocolToggleUsers: []
        defaultProtocol: mls
        allowedCipherSuites: [1]
        defaultCipherSuite: 1
        supportedProtocols: [proteus, mls] # must contain defaultProtocol
      lockStatus: locked
  

(#3681)

• • Replace fake-sqs with ElasticMQ

  ElasticMQ is an actively maintained project, fake-sqs hasn't seen a commit since
  2018. This is not expected to have any noticeable effect on deployments that
  don't have any extra configurations for the SQS queues. If the fake-aws-sqs
  chart had configured custom queue names, they have couple of extra limitations:

  • The queue names must only contain alphanumeric characters and hyphens.
  • The FIFO queue names must end in .fifo. (#3750)

• Upgrade team-settings version to 4.15.1-v0.31.19-0-ee1dbce (#2180)

• Upgrade webapp to 2023-12-11-production.0-v0.31.17-0-1e91445

  Beside using up-to-date versions in Helm charts is generally beneficial,
  this version also provides multi-ingress support. (#3803)

• Upgrade webapp version to 2024-01-22-production.1-v0.31.17-0-7f83dbe (#2302)

API changes

• Un-verified users can no longer upload assets (#3604)

• • Remove IP request header from add-client endpoint.
  • Remove longitude and latitude response fields from add-client, list-clients, and get-client endpoints.

  These are not considered breaking changes, since clients are not using this information. (#3792)

• Create new API version V6 and finalize V5 (#3815)

• Block changes of userDisplayName, userHandle in mlsE2EI-enabled teams on the backend without SCIM; report "managed_by" == "scim" in GET /self, but only there (#3827)

• The federation API can now be versioned. Multiple versions of an RPC can be defined on the same path. After version negotiation, the federation client now sets the X-Wire-API-Version header, and federator propagates it to the destination service. (#3762)

• Improved formatting of federation errors. No extra copy of the response body, and nested errors are now part of the JSON structure, not quoted inside the message. (#3742)

• New endpoint for replacing MLS key packages in bulk: PUT /mls/key-packages/self/:client. It replaces all existing key packages that match the given ciphersuites with the new key packages provided in the body. (#3654)

Features

• The lifetime of conversation guest links is now configurable (#3796)

• Events for a member update, join and leave are not sent to everyone in the team any longer. Only team admins get them. (#3703)

• Allowlist for who on cloud can connect to on-prem:

  • Internal API to configure allowlist
  • Restrict federated user search according to team federation policy
    (#3697, #3732, #3758)

• The mls team feature now has a lock status (#3681)

• add a uniform timeout to the integration test-suite set by the environment variable TEST_TIMEOUT_SECONDS with a default of 10 seconds if the variable isn't set (#3692)

• Apply team-level federation policies when establishing and updating user connections (#3774)

• Introduce a feature flag that controls whether the limited event fanout should be used when a team member is deleted (#3797)

• Send a conversation.member-leave event to team admins for each conversation the deleted team member used to be part of (#3790)

• Allow the configuration of TLS-secured connections to Cassandra. TLS is used
  when a certificate is provided. This is either done with
  --tls-ca-certificate-file for cli commands or the configuration attribute
  cassandra.tlsCa for services. In Helm charts, the certificate is provided as
  literal PEM string; either as attribute cassandra.tlsCa (analog to service
  configuration) or by a reference to a secret (cassandra.tlsCaSecretRef.) (#3587)

• [SFT chart] Add option to enable serviceMonitor to scrape prometheus metrics (#3770)

Bug fixes and other updates

• galley's DB migrations fixed (#3680)

• The X509 client identity parser supports a new format: wireapp://{userid}!{deviceid}@{host} (#3808)

• Updated demo-smtp Helm chart from deprecated docker image namshi/smtp to ixdotai/smtp (#3791)

• External partners search restriction enforced by backend (#3708)

• File upload size is now limited to 100 MiB (#3752)

• Fix a bug where non-team conversation members that are remote would not get a conversation.member-leave event (#3745, #3764)

• Enforce external partner permissions on the backend (#3788)

• Various improvements around LH policy conflict detection:

  • Fix LH policy conflict detection logic when posting messages
  • Better policy conflict error messages (distinguish between old clients and missing consent)
  • Add first LH scaffolding and tests to /integration
  • Annotate some API functions in /integration with links to openapi3 docs (#3773)

• Do not match on the Accept header for service provider endpoints with no response body (#3766)

• Guests should not be added to conversations that are under legalhold (#3853)

• Intra-service calls from brig to galley's public API are now aware of disabled API versions (#3863)

• fix Helm pretty-printer for disabledAPIVersions (#3877)

• Adjust the requested memory and upper bound limit of nginz pods in the related Helm chart. (We experienced OOM errors with the old settings.) (#3821)

• don't use shell when communicating with mls-test-cli, move flaking brig tests over to new integration testsuite (#3701)

• set notificationTimeOut to 28 days, make it legible (#3714)

• Update coturn image with bugfix to its prestop-hook from https://github.com/wireapp/coturn/pull/10 to allow coturn pods to terminate once their traffic has drained. (#3872)

• Extra remove proposals were being sent when a user was removed from a conversation (#3672)

• Remove client check for subconversations (#3677)

• Ensure that SCIM can find users even after the team admin has changed the SAML issuer for the user. (#3747)

• addClient used the internal brig API in the integration testsuite when it should use the public one (#3869)

• Ensure that HTTP 1.1 connections are grafully closed

  To fix this warp had to be patched to fix the bug upstream: https://github.com/yesodweb/wai/pull/958 (#3775)

Documentation

• Fix missing code sections on docs.wire.com, notably on "configuring TLS" page. (#3839)

• Swagger generation no longer adds tags containing information about federation calls.

  Added the federation calling graph to the Federation API Conventions page. (#3674)

• Backend-to-backend OpenApi Docs added (#3666)

• Documentation for creating a new API version updated (#3817)

• Update documentation of MLS group ID (#3705)

• Turn long summaries in openapi documentation into descriptions (#3706)

• update the build instructions for wire-server (#3854)

Internal changes

• stern/backoffice PUT /teams/{teamId}/features/conferenceCalling fixed (#3723)

• Removed client ID conversion round trip (#3727)

• Migrate to Servant the Galley conversation internal endpoints (#3718)

• The development API version is now disabled by default (#3772)

• Attempt to fix flaky integration test provider.service.delete (#3689)

• The fedcalls tool no longer walks the Swagger/OpenAPI structure when generating call graphs. These graphs are now generated directly from the Servant API types. (#3674, #3691)

• Increased ingress payload size from 256k to 512k (#3756)

• Request tracing across federated requests (#3765)

• upgrade nixpkgs to upgrade haskell-language-server (#3650)

• upgrade the GHC version to GHC 9.4 (#3679)

• Removed APNS_VOIP code. (APNS_VOIP is a native push notification channel which we aren't using anymore.) (#3695)

• Improve error logs (#3782)

• Migrating tests for Cargohold to the new integration test suite. (#3741)

• Fix calendar integration setting in backoffice / stern (#3761)

• Reply-Nonce is added to Access-Control-Expose-Headers (#3729)

• Add custom feature flag; only supported for some on-prem installations; locked & disabled by default (#3779)

• Improved how tests are automatically extracted from the integration test suite.

  The test extractor parser has been improved to handle block comments, and to more strictly check for Haddock documentation for each test. (#3749)

• Additional logging on user/team suspension (#3795)

• cleanup the haskell-pins

  • remove many pins
  • remove many overrides
  • restructure the files such that it's easier to see what is going on (#3814)

• Version of rusty-jwt-tools bumped to v0.8.0 (#3805)

• Feature enforceFileDownloadLocation lockstatus can be set with basic auth on staging (#3802)

• Version of rusty-jwt-tools bumped to v0.8.5 (#3820)

• Translate integration tests: manually add / delete LH device (#3830)

• adds a new executable, hs-run, to quickly run haskell scripts (#3716)

• Represent client IDs as Word64 internally (#3713)

• Allow to install the coturn chart multiple times in multiple namespaces on the same cluster. (#3698)

• For some rust packages (cryptobox and libzauth-c), we now use crate2nix as a build tool, rather than the more coarse and FOD-based nixpkgs rustPlatform.buildRustPackage approach. (#3686)

• Delete shell.nix. It has been broken for quite some time. The supported way to get a development nix environment is to use direnv. (#3726)

• Deploy a backend with federation API V0 while setting up services for local testing (#3719)

• Improve integration test coverage (#3757)

• Increase timeout for waiting for SQS notifications in galley's integration tests (#3699)

• Simplify process spawning of dynamic backends in integration tests (#3759)

• More robust consuming of MLS messages: the behaviour of sendAndConsumeMessage and sendAndConsumeCommitBundle is changed to actually wait for those messages on the client's websocket (#3671)

• Update group state after application message (#3678)

• bump the nixpkgs version to allow updating curl (#3781)

• Simplify the definition of the servant notification API (#3685)

• Start refactoring code into subsystems, first subsystem being the NotificationSubsystem. (#3786)

• Remove apply-refact from CI image

  This gets rid of GHC in the image, making the image smaller. (#3712)

• Refactor getOptions (#3707)

• Restored Brig memory quota to 512mb down from 1gb. (prev bump #3751) (#3806)

• Add tool to analyse test results in junit/ant xml format (#3652)

• updated annotation for enabling Topology Aware Routing to service.kubernetes.io/topology-mode for k8s 1.27+ (#3878)

• replace runAsNonRoot with runAsUser and runAsGroup 1000 (#3826)

• Update SFTD default to 4.0.10 and its nginx to 1.25.3. (#3768)

• add a Makefile target to make it possible to upload a bom of all services to s3 on every CI run (#3744)

• Upload bill-of-material (BOM) files directly to the Dependency Tracker via REST.
  This eases the life of the security team and prevents cluttering our release
  artifact page. (#3810)

• Passively migrate user passwords from scrypt to argon2id.

  By passively we mean that whenever a user re-enters their passwords, if it was hashed using scrypt, it is then rehashed using argon2id and stored as such.
  If that user has a legacy short password (under 8 characters in length), it does not migrate to argon2id. (#3720)

Federation changes

• Define a few tests for adding members to an MLS conversation when unreachable backends are involved (#3673)

• Make sure that remote users can be added to both a Proteus and an MLS conversation when other users are unreachable (#3688)

Release notes

• New field for Supported protocols in Galley's MLS feature config

  Galley will refuse to start if the list supportedProtocols does not contain
  the value of the field defaultProtocol. Galley will also refuse to start if
  MLS migration is enabled and MLS is not part of supportedProtocols.

  The default value for supportedProtocols is:

  [proteus, mls]
  

(#3374)

API changes

• The JSON schema of NonConnectedBackends has changed to have its single field now called non_connected_backends. (#3518)

• Remove de-federation (to avoid a scalability issue). (#3582)

• Replace the placeholder self conversation id with the qualified conversation id for welcome events. (#3335)

• Add new endpoint DELETE /mls/key-packages/self/:client (#3295)

• Introduce an endpoint for deleting a subconversation (#2956, #3119, #3123)

• Remove MLS endpoints from API v4 and finalise it (#3545)

• Add new endpoint GET /conversations/one2one/:domain/:uid to fetch the MLS 1-1 conversation with another user (#3345)

• Introduce a subconversation GET endpoint (#2869, #2995)

• Add GET /conversations/:domain/:cid/subconversations/:id/groupinfo endpoint to fetch the group info object for a subconversation (#2932)

• Introduce v5 development version (#3527)

• It is now possible to use PUT /conversation/:domain/:id/protocol to transition from Mixed to MLS (#3334)

• Report a failure to add remote users to an MLS conversation (#3304)

• The key package API has gained a ciphersuite query parameter, which should be the hexadecimal value of an MLS ciphersuite, defaulting to 0x0001. The ciphersuite parameter is used by the claim and count endpoints. For uploads, the API is unchanged, and the ciphersuite is taken directly from the uploaded key package. (#3454)

• Add MLS migration feature config (#3299)

• Switch to MLS draft 20. The following endpoints are affected by the change:

  • All endpoints with message/mls content type now expect and return draft-20 MLS structures.
  • POST /conversations does not require creator_client anymore.
  • POST /mls/commit-bundles now expects a "stream" of MLS messages, i.e. a sequence of TLS-serialised messages, one after the other, in any order. Its protobuf interface has been removed.
  • POST /mls/welcome has been removed. Welcome messages can now only be sent as part of a commit bundle.
  • POST /mls/message does not accept commit messages anymore. All commit messages must be sent as part of a commit bundle. (#3172)

• Key packages and leaf nodes with x509 credentials are now supported (#3532)

Features

• Add reason field to conversation.member-leave (#3640)

• Support deleting a remote subconversation (#2964)

• Introduce support for resetting a subconversation (#2956)

• Introduce a "mixed" conversation protocol type. A conversation of "mixed" protocol functions as a Proteus converation as well as a MLS conversations. It's intended to be used for migrating conversations from Proteus to MLS. (#3258)

• Added support for post-quantum ciphersuite 0xf031. Correspondingly, MLS groups with a non-default ciphersuite are now supported. The first commit in a group determines the group ciphersuite. (#3454)

• Remove conversation size limit for MLS conversations (#3468)

• Added support for MSL 1-1 conversations (#3360)

• MLS application messages for older epochs are now rejected (#3438)

• The public key in an x509 credential is now checked against that of the client (#3542)

• Add federated endpoints to get subconversations (#2952)

• Add Helm chart (rabbitmq-external) to interface RabbitMQ instances outside of the Kubernetes cluster. (#3626)

• Removing or kicking a user from a conversation also removes the user's clients from any subconversation. (#2942)

• Add support for subconversations in POST /mls/commit-bundles (#2932)

• Implement endpoint for leaving a subconversation (#2969, #3080, #3085, #3107)

Bug fixes and other updates

• Fix nix derivations for rust packages (#3628)

• Ensure benchmarking dependencies are provided by nix development environment (#3628)

• Disable a guest user from creating a group conversation (#3622)

• Adding users to a conversation now enforces that all federation domains that will be in the conversation are federated with each other. (#3514)

• Fix ES migration script. (#3558)

• Fixed add user to conversation when one of the other participating backends is offline (#3585)

• Create a new http2 connection in every federator client request instead of using a shared connection. (#3602)

• list-clients returns with partial success even if one of the remote backends is unreachable (#3611)

• Defederation notifications, federation.delete and federation.connectionRemoved, now deduplicate the user list so that we don't send them more notifications than required. (#3515)

• Fix memory and TCP connection leak in brig, galley, caroghold and background-worker. (#3663)

• Fix bug where notifications for MLS messages were not showing up in all notification streams of clients (#3610)

• Map the MLS self-conversation creator's key package reference in Brig (#3055)

• This fixes a bug where a remote member is removed from a conversation while their backend is unreachable, and the backend does not receive the removal notification once it is reachable again. (#3537)

• Welcome messages are not sent anymore to the creator of an MLS group on the first commit (#3392)

Documentation

• Fix: support api versions other than v0 in swagger docs. (#3619)

• Updating the route documentation from Swagger 2 to OpenAPI 3. (#3570)

• Elaborate on internal user creation in prod (#3596)

• Adding a testing config entry to the PR guidelines. (#3624)

Internal changes

• remove leaving clients immediately from subconversations (#3096)

• Servantify internal end-points: brig/teams (#3634)

• add conversation type to group ID serialisation (#3344)

• Do not cache federation remote configs on non-brig services (#3612)

• JSON derived schemas have been changed to no longer pre-process record fields to drop prefixes that were required to disambiguate fields.
  Prefix processing still exists to drop leading underscores from field names, as we are using prefixed field names with makeLenses.
  Code has been updated to use OverloadedRecordDot with the changed field names. (#3518)

• Updating the route documentation library from swagger2 to openapi3.

  This also introduced a breaking change in how we track what federation calls each route makes.
  The openapi3 library doesn't support extension fields, and as such tags are being used instead in a similar way. (#3570)

• • Extending the information returned in errors for Federator. Paths and response bodies, if available, are included in error logs.
  • Prometheus metrics for outgoing and incoming federation requests added. They can be enabled by setting metrics.serviceMonitor.enabled, like in other charts. (#3556)

• CLI tool to consume messages from a RabbitMQ queue (#3589, #3655)

• Removed user and client threshold fields from mls migration feature. (#3364)

• Include timestamp in s3 upload path for test logs (#3621)

• Migrating the following routes to the Servant API form.

  POST /provider/services
  GET /provider/services
  GET /provider/services/:sid
  PUT /provider/services/:sid
  PUT /provider/services/:sid/connection
  DELETE /provider/services/:sid
  GET /providers/:pid/services
  GET /providers/:pid/services/:sid
  GET /services
  GET /services/tags
  GET /teams/:tid/services/whitelisted
  POST /teams/:tid/services/whitelist (#3554)

• Provider API has been migrated to servant (#3547)

• background-worker: Get list of domains from RabbitMQ instead of brig for pushing backend notifications (#3588)

• Avoid including MLS application messages in the sender client's event stream. (#3379)

• Avoid empty pushes when chunking pushes in galley (#3646)

• Introduce a Galley DB table for subconversations (#2869)

• Support mapping MLS group IDs to subconversations (#2869)

• change version and conversation type to 16 bit in group ID serialisation (#3353)

• Brig does not perform key package ref mapping anymore. Claimed key packages are simply removed from the mls_key_packages table. The mls_key_package_refs table is now unused, and will be removed in the future. (#3172)

• Add intermediate "mixed" protocol for migrating from Proteus to MLS (#3292)

• • Do not perform client checks for add and remove proposals in mixed conversations
  • Restrict protocol updates to team conversations
  • Disallow MLS application messages in mixed conversations
  • Send remove proposals when users leave mixed conversations (#3303)

• New cron job to save data usable to watch the progress of the Proteus to MLS migration in S3 bucket.

  IMPORTANT: This cron job is not meant for general use! It can leak data about one team to other teams. (#3579)

• Subconversations are now created on their first commit (#3355)

• Propagate messages in MLS subconversations (#2937)

• Move some MLS tests to new integration suite (#3286)

• Check validity of notification IDs in the notification API (#3550)

• stern: Optimize RAM usage of /i/users/meta-info (#3522)

• Additional integration test for federated connections (#3538)

• The bot API is now migrated to servant (#3540)

• rusty-jwt-tools is upgraded to version 0.5.0 (#3572)

• Refactored schema version tracking from manually managed to automatic. (#3643)

• Avoid unnecessary error logs on service shutdown (#3592)

• Introduce an effect for subconversations (#2869)

• Via the update path update the key package of the committer in epoch 0 of a subconversation (#2975)

• Add more tests for joining a subconversation (#2974)

• Added /tools/db/repair-brig-clients-table to clean up after the fix in #3504 (#3507)

• Distinguish between update and upsert cassandra commands (follow-up to #3504) (#3513)

• Truncate galley.mls_group_member_client table and drop galley.member_client table.

  The data in mls_group_member_client could contain nulls from client testing in prod. So, its OK to truncate it.
  The member_client table is unused. (#3648)

• All integration tests can generate XML reports.

  To generate the report in brig-integration, galley-integration,
  cargohold-integration, gundeck-integration, stern-integration and the new
  integration suite pass --xml=<outfile> to generate the XML file.

  For spar-integration and federator-integration pass -f junit and set
  JUNIT_OUTPUT_DIRECTORY and JUNIT_SUITE_NAME environment variables. The XML
  report will be generated at $JUNIT_OUTPUT_DIRECTORY/junit.xml.

  (#3568, #3633)

Federation changes

• Add subconversation ID to onMLSMessageSent request payload. (#3270)

• Derive group ID from qualified conversation ID and, if applicable,
  subconversation ID.

  Retire mapping from group IDs to conversation IDs. (group_id_conv_id)

  Remove federation endpoints

  • on-new-remote-conversation,
  • on-new-remote-subconversation, and
  • on-delete-mls-conversation
    which were used to synchronise the group to conversation mapping. (#3309)

• Reorganise the federation API such that queueing notification endpoints are separate from synchronous endpoints. Also simplify queueing federation notification endpoints. (#3647)

• Introduce an endpoint for resetting a remote subconversation (#2964)

• Split federation endpoint into on-new-remote-conversation and on-new-remote-subconversation
  Call on-new-remote-subconversation when a new subconversation is created
  Call on-new-remote-subconversation for all existing subconversations when a new backend gets involved
  Call on-new-remote-subconversation when a subconversation is reset (#2997)

• federator: Allow setting TCP connection timeout for HTTP2 requests

  The helm chart defaults it to 5s which should be best for most installations. (#3595)

• Constrain which federation endpoints can be used via the queueing federation client (#3629)

• There is a breaking change in the "on-mls-message-sent" federation endpoint due to queueing. Now that there is retrying because of queueing, the endpoint can no longer respond with a list of unreachable users. (#3629)

• Remote MLS messages get queued via RabbitMQ (#3635)

Bug fixes and other updates

• Fix syntax error in cassandra update to brig.client. (#3508)

API changes

• Conversation creation endpoints can now return unreachable_backends error responses with status code 533 if any of the involved backends are unreachable. The conversation is not created in that case. (#3486)

Bug fixes and other updates

• Make sure cassandra updates do not re-introduce removed content. (#3504)

Federation changes

• Return unreachable_backends error when some backends of newly added users to a conversation are not reachable (#3496)

Release notes

• federation only Introduce background-worker

  This release introduces a new component: background-worker. This is currently
  only used to federation-related tasks. Enabling federation in
  the wire-server helm chart automatically installs this component.

  When federation is enabled, wire-server will require running RabbitMQ. The helm
  chart in rabbitmq can be used to install RabbitMQ. Please refer to the
  documentation at https://docs.wire.com to install RabbitMQ in Kubernetes. These
  new configurations are required:

  brig:
    config:
      rabbitmq:
        host: rabbitmq
        port: 5672
        vHost: /
    secrets:
      rabbitmq:
        username: <YOUR_USERNAME>
        password: <YOUR_PASSWORD>
  galley:
    config:
      rabbitmq:
        host: rabbitmq
        port: 5672
        vHost: /
    secrets:
      rabbitmq:
        username: <YOUR_USERNAME>
        password: <YOUR_PASSWORD>
  background-worker:
    config:
      rabbitmq:
        host: rabbitmq
        port: 5672
        vHost: /
        adminPort: 15672
    secrets:
      rabbitmq:
        username: <YOUR_USERNAME>
        password: <YOUR_PASSWORD>
  

  The above are the default values (except for secrets, which do not have
  defaults), if they work they are not required to be configured.
  (#3276, #3314, #3333, #3366, #3383, #3391)

• Federation only A few helm values related to federation have been renamed, no action is required if federation was disabled.
  If federation was enabled these values must be renamed in the wire-server chart:

  • tags.federator -> tags.federation
  • brig.enableFederator -> brig.enableFederation
  • galley.enableFederator -> galley.enableFederation
  • cargohold.enableFederator -> galley.enableFederation

  So, an old config which looked like this:

  tags:
    federator: true
  brig:
    enableFederator: true
  galley:
    enableFederator: true
  cargohold:
    enableFederator: true
  

  would now look like this:

  tags:
    federation: true
  brig:
    enableFederation: true
  galley:
    enableFederation: true
  cargohold:
    enableFederation: true
  

  (#3236)

• Federation only From this release on, remote connections can be configured via an
  internal REST API; the remote connections configured in the
  values.yaml file(s) will be honored for a transition period, but will
  be ignored starting in some future release.

  YOU NEED TO UPDATE YOUR BRIG HELM VALUES BEFORE DEPLOYING THIS RELEASE.

  Add the following to brig:

  brig:
    config:
      optSettings:
        setFederationStrategy: allowNone # [allowAll | allowDynamic | allowNone]
        setFederationDomainConfigsUpdateFreq: 10 # seconds
  

  allowNone is equivalent to allowList with empty list; allowAll
  remains the same as before; allowDynamic is allowList, but the
  list is now stored in cassandra, not the config file.

  If your federator config values contain something like this:

      federationStrategy:
        allowedDomains:
        - red.example.com
        - blue.example.com
  

  you need to make sure that the following lines are part of your brig
  config (after the upgrade and until you have loaded the data into
  casssandra, federation with those domains won't possible if you forget
  this):

  brig:
    config:
      optSettings:
        setFederationDomainConfigs:
        - domain: red.example.com
          search_policy: full_search
        - domain: blue.example.com
          search_policy: no_search
  

  The search policy for a remote backend can be:

  • no_search: No users are returned by federated searches. default.
  • exact_handle_search: Only users where the handle exactly matches are returned.
  • full_search: Additionally to exact_handle_search, users are found by a freetext search on handle and display name.

  Once the new release is deployed, you need to copy all the data from
  the config files into brig.federation_remotes in cassandra internal
  CRUD
  API;
  look for /i/federation/remotes).

  Once the upgrade has been deployed and cassandra has been filled
  with the temporary contents of
  brig.config.optSettings.setFederationDomainConfigs, it is safe to
  remove the latter and the above lines from the federator config.

  See also. (#3260, #3384, #3389)

• Upgrade team-settings version to 4.15.0-v0.31.16-0-8138d2e (#2180)

• Upgrade webapp version to 2023-07-13-production.0-v0.31.16-0-a9b67c6 (#2302)

• Update email templates from https://github.com/wireapp/wire-emails (#3386)

• Removed brig configuration value from gundeck. (#3404)

API changes

• Updating conversation meta-data APIs to be fault tolerant of unavailable federation servers. (#3229)

• Adding users in Proteus will only succeed if all federated backends hosting the
  users are available. Otherwise, the endpoint will fail with a Federation error,
  enumerating all unavailable domains. (#3449)

• Added a new notification event type, "federation.delete". (#3397)
  This event contains a single domain for a remote server that the local server is de-federating from.
  This notification is sent twice during de-federation. Once before and once after cleaning up and removing references to the remote server from the local database.

• list unavailable backends as JSON on federation-unreachable-domains-error

  • extend federation-unreachable-domains-error by FederationErrorData
  • add domains field in FederationErrorData, containing the list of failing
    domains
  • deprecate domain field in FederationErrorData which now contains the first
    element of domains (#3407)

• Throw when remote users to be added to an MLS conversation are unreachable (#3322)

• The connection-update internal Brig endpoint now has a different JSON format for its request body. See the swagger documentation for details. (#3458)

• Client objects have gained an optional last_active field. Whenever a client fetches notifications via GET /notifications, as long as it provides a client parameter, the last_active field of that client is updated, and set to the current timestamp, rounded to the next multiple of a week. (#3409)

• The POST /conversations endpoint now in case of the Proteus protocol gives a 503 error response listing unreachable backends in case there were any, instead of a 2xx response by adding only members from reachable backends. (#3479)

• User objects have gained a supported_protocols field. Users can set it to any subset of ["proteus", "mls"] using PUT /self/supported-protocols. There is also a new endpoint GET /users/:domain/:id/supported-protocols. The backend does not assign any semantics to this field, but it is intended to be used to coordinate migration to MLS across the clients of a user, as well as between two users participating in a 1-1 conversation. (#3326)

• Several federation Galley endpoints have a breaking change in their response types: "leave-conversation", "update-conversation" and "send-mls-message". They have been extended with information related to unreachable users. (#3248)

Features

• Add federation options to the coturn Helm chart including DTLS support. The options themselves are strongly inspired by the restund Helm chart. (#3283)

• Let cargohold redirect to different s3 download endpoints according to a multiIngress configuration. This is part of a larger multi-ingress story where one backend can pretend to be multiple ones by using different domains for different users. (#3264)

• Introduce nginx_conf.additional_external_env_domains (nginz and cannon) setting to configure CORS headers for multiple domains. (#3368)

• Add configuration options to setup instances of the nginx-ingress-services chart to act as additional ingresses (with sourrounding infrastructure) to provide additional domains for the same backend. (#3375)

• Nonce base 64 encoding is now unpadded (#3255)

• MlsE2EIdConfig does now contain an ACME discovery URL and verificationExpiration is now a duration. (#3237, #3244)

• Functionality to determine the federation status between federating remote backends (#3290)

• Prevent conversation creation if any two federated backends are not connected to each other (#3382)

• Improve gundeck performance: notifications to multiple recipients are stored in a normalized manner. (#3403)

• When a proteus message is send and a remote user's backend is offline, the message will be enqueued and reported as failed_to_confirm_clients (#3460, #3474)

• Check if remote backends are connected on adding conversation members (#3483)

• In a setting where remote participants are included in a freshly created Proteus conversation, the backend now sends a conversation.create and a conversation.member-join event per user once all remote participants are confirmed. This fixes a bug where remote conv members would get false entries in the member lists in these events. (#3359)

• Enable indexed billing members by default and remove the feature flag (#3434)

• stern/backoffice: read, update, delete domain login redirects to custom backends (#3471)

Bug fixes and other updates

• If role is not set ([], null, or field missing) in scim-put-user, do not change role to default in brig (#3488)

• Do not accept federation traffic from not-federating backends (#3484)

• Bump coturn default image to upstream coturn 4.6.2 + custom Wire code including a bugfix for a bug that resulted in unstable operation during higher load. (#3250)

• Get the correct domain for DPoP access token generation (#3255)

• Correct http host is passed to proxy request (#3263)

• Use backend domain for DPoP access token request (#3267)

• The DPoP access token is now base64 encoded (once) (#3269)

• Fix nginx.conf for local integration tests (#3362)

• Fix cross domain user search (#3420)

• backoffice/stern

  • Fixed /i/user/meta-info (#3436)
  • Fixed /i/user/meta-info (#3281)
  • Register/Update OAuth client via backoffice/stern (#3305)

• Fix: When defederating, don't crash on already-deleted conversations. (#3478)

• No conversation.delete event is sent to users during de-federation clean up (#3485)

Documentation

• Improve the cassandra developer guidelines under https://docs.wire.com/developer/developer/cassandra-interaction.html (#3342)

• Document crypto library dependencies and sources of randomness (#3254)

• Add 'grepinclude' sphinx directive to document with some code snippets. (#3256)

• swagger:

  • Render Named names as "internal route ID" in swagger UI. (#3319)
  • Make /api/swagger{-ui,.json} TOC html pages to all versions (#3259)
  • Explain links to swagger docs better on docs.wire.com (#3388)
  • Swagger docs for custom backends (#3415)

• SSO Faq entry on CSP (#PR_NOT_FOUND)

Internal changes

• Export Data.String.Conversions.cs from Imports (#3320)

• Metrics for federator are available at GET /i/metrics for both the internal and external servers. (#3467)

• Add the status endpoint to both federator ports (#3443)

• Better errors in golden tests (#3370)

• In CI integration tests, use redis-ephemeral in master mode (may be reverted in the future, see PR details) (#3446)

• Containers now run as non-root, to improve compatibility with default PodSecurityPolicies in more recent versions of Kubernetes. (#3352)

• By default, the coturn helm chart will no longer log verbosely. This can be enabled if desired. (#3238)

• Delete libraries api-bot and api-client. Also delete tools from api-simulation. (#3395)

• Use feature singletons in TeamFeatureStore (#3308)

• Adding a new internal API to Brig and Galley to defederate domains. Background-Worker has been reworked to seperate AMQP channel handling from processing. This was done to allow a defederation worker to share the same connection management process with notification pusher. (#3378)

• Improved websocket tests:

  • better error reporting
  • choose the correct backend when establishing a websocket connection (#3393)

• /integration (#3293)

  • Add convenience getJSON and getBody functions (#3293)
  • baseRequest now adds Z headers automatically (#3293)
  • Add liftIO versions of putStrLn etc (#3293)
  • Add Show instances for MLSState (#3293)
  • Implement test listing (#3301)
  • Port MLS test framework (#3288)
  • Support spawning multiple dynamic backends (#3316)
  • Split App module in integration package (#3273)
  • Test swagger docs (#3367)
  • Add parametrised tests (#3296)

• On CI runs, provide additional context when 'helmfile install' fails. (#3400)

• [hscim] make jsonLower fail on duplicate fields (#3346)

• Clean up output and logs (#3371)

  • integration: Remove debug messages from ModService tests
  • Do not log rabbit MQ connection failures on async exceptions
  • cannon: Do not print uncaught SignalledToExit exceptions to stdout

• End-to-end test for creating a DPoP access token for the E2EID client certificate enrollment (#3255)

• backoffice/stern

  • more integration tests and fixes (#3232, #3239)
  • stern is added to the new run-services implementation for the integration tests (#3425)

• Fixed eventually function in test for potentially less flakiness (#3240)

• Script to bulk-change/-repair user's scim and brig email address (#3321, #3331)

• Servantify brig internal api (#3346, #3338, #3339)

• Updated rusty-jwt-tools and error mapping (#3348)

• Reuse HTTP2 connections from brig, galley, cargohold and federator (#3120, #3233)

• Add combinator for maps with arbitrary keys in schema-profunctor (#3372)

• Introduce SearchContacts permission (#3252)

• All wire-server containers now run in a restricted securityContext when run on k8s >= 1.24 (#3351)

• Adding graceful shutdown handling to background-worker to allow it to finish processing its current message before the service quits. (#3421)

• In (the unlikely) case your server config file contains setWhitelist:, you need to change this before the upgrade! It used to refer to a whitelisting service, which is now replaced with a local list of allowed domains and phone numbers. See docs for details. Migration path: add new config fields; upgrade, remove old config fields. (#3043)

• The coturn Helm chart has been promoted to beta level stability. (#3078)

API changes

• API v3 is now supported. The new MLS endpoints introduced in API v3 have been removed, and are now only available under v4. (#3122)

Features

• Add internal endpoints of cargohold, galley, legalhold and spar to the Swagger docs for internal endpoints. (#3007)

• The coturn container image included in the coturn Helm chart was updated to
  version 4.6.0-wireapp.4.

  With this version of coturn, the Prometheus metrics endpoint has been
  updated, and the turn_active_allocations metric label has been renamed to
  turn_total_allocations. (#3078)

• Better error message for invalid ID in a credential when uploading MLS key packages (#3102)

• Add Swagger documentation for internal endpoints. It's reachable at the path /v<n>/api-internal/swagger{-ui,.json}. (#3003)

• Render one Swagger page per internal endpoint. This superseeds the previous Swagger docs page for all internal endpoints. (#3094)

• Feature flag for Outlook calendar integration (#3025)

• Team feature setting for MLS end-to-end identity was added and server setting setEnableMls is exposed via new authorized endpoint GET /system/settings (#3082)

Bug fixes and other updates

• The container image used for handling online TLS certificate updates in the
  coturn Helm chart was updated to a version with metadata compatible with
  containerd. (#3078)

• Fix a bug in the helm chart's nginx-ingress-services / federator Ingress resource introduced in the last release. (#3034)

• Remove overly restricte api check (#3131)

• Typing indicators not working accross federated backends (#3118)

Documentation

• Extend the docs on the federation error type (#3045)

• Update SAML/SCIM docs (#3038)

Internal changes

• • use exponential backoff for retrying requests to Amazon
  • also retry in case of server-side rate limiting by Amazon (#3121)

• Also run the 'backoffice' pod in CI (to test it can successfully start) (#3130)

• Make brig-schema a little faster by merging the first 34 schema migrations on fresh installations. (#3099)

• Deflake integration test: metrics (#3053)

• Document in code a function that sends remote Proteus messages (#PR_NOT_FOUND)

• Lower the log level of federator inotify (#3056)

• use Wai's settings for graceful shutdown (#3069)

• CI integration setup time should be reduced: tweak the way cassandra-ephemeral is started (#3052)

• charts: Mark all service/secret/configmap test resources to be re-created by defining them as helm hooks (#3037, #3049)

• New integration test script with support for running end2end tests locally (#3062)

• Bump nixpkgs to latest commit on nixpkgs-unstable branch (#3084)

• Add config to allow to run helm tests for different services in parallel; improve integration test output logs (#3040)

• Run brig and galley integration tests concurrently (#2825)

• Add wrapper for bitnami/postgresql chart. (#3012)

• Branch on performAction tags for finer-grained CallsFed constraints (#3030)

• Fixed broken stern endpoint POST i/user/meta-info (#3035)

• Make stern fail on startup if supported backend api version needs bumping (#3035)

• Automatically track CallsFed constraints via a GHC plugin (#3083)

• Rust library rusty-jwt-tools upgraded to latest version (#3112)

• Fixed test of jwt-tools Rust FFI (#3125)

• Enabling warnings for redundant constraints and removing the redundant
  constraints. (#3009)

• Migrate /teams/notifications to use the Servant library. (#3020)

• Split polysemy Members constraints into multiple Member constraints (#3093)

Federation changes

• Use HsOpenSSL instead of tls for federation communication. (#3051)

Release notes

• In (the unlikely) case your server config file contains setWhitelist:, you need to change this before the upgrade! It used to refer to a whitelisting service, which is now replaced with a local list of allowed domains and phone numbers. See docs for details. Migration path: add new config fields; upgrade, remove old config fields. (#3043)

• The coturn Helm chart has been promoted to beta level stability. (#3078)

API changes

• API v3 is now supported. The new MLS endpoints introduced in API v3 have been removed, and are now only available under v4. (#3122)

Features

• Add internal endpoints of cargohold, galley, legalhold and spar to the Swagger docs for internal endpoints. (#3007)

• The coturn container image included in the coturn Helm chart was updated to
  version 4.6.0-wireapp.4.

  With this version of coturn, the Prometheus metrics endpoint has been
  updated, and the turn_active_allocations metric label has been renamed to
  turn_total_allocations. (#3078)

• Better error message for invalid ID in a credential when uploading MLS key packages (#3102)

• Add Swagger documentation for internal endpoints. It's reachable at the path /v<n>/api-internal/swagger{-ui,.json}. (#3003)

• Render one Swagger page per internal endpoint. This superseeds the previous Swagger docs page for all internal endpoints. (#3094)

• Feature flag for Outlook calendar integration (#3025)

• Team feature setting for MLS end-to-end identity was added and server setting setEnableMls is exposed via new authorized endpoint GET /system/settings (#3082)

Bug fixes and other updates

• The container image used for handling online TLS certificate updates in the
  coturn Helm chart was updated to a version with metadata compatible with
  containerd. (#3078)

• Fix a bug in the helm chart's nginx-ingress-services / federator Ingress resource introduced in the last release. (#3034)

• Remove overly restricte api check (#3131)

• Typing indicators not working accross federated backends (#3118)

Documentation

• Extend the docs on the federation error type (#3045)

• Update SAML/SCIM docs (#3038)

Internal changes

• • use exponential backoff for retrying requests to Amazon
  • also retry in case of server-side rate limiting by Amazon (#3121)

• Also run the 'backoffice' pod in CI (to test it can successfully start) (#3130)

• Make brig-schema a little faster by merging the first 34 schema migrations on fresh installations. (#3099)

• Deflake integration test: metrics (#3053)

• Document in code a function that sends remote Proteus messages (#PR_NOT_FOUND)

• Lower the log level of federator inotify (#3056)

• use Wai's settings for graceful shutdown (#3069)

• CI integration setup time should be reduced: tweak the way cassandra-ephemeral is started (#3052)

• charts: Mark all service/secret/configmap test resources to be re-created by defining them as helm hooks (#3037, #3049)

• New integration test script with support for running end2end tests locally (#3062)

• Bump nixpkgs to latest commit on nixpkgs-unstable branch (#3084)

• Add config to allow to run helm tests for different services in parallel; improve integration test output logs (#3040)

• Run brig and galley integration tests concurrently (#2825)

• Add wrapper for bitnami/postgresql chart. (#3012)

• Branch on performAction tags for finer-grained CallsFed constraints (#3030)

• Fixed broken stern endpoint POST i/user/meta-info (#3035)

• Make stern fail on startup if supported backend api version needs bumping (#3035)

• Automatically track CallsFed constraints via a GHC plugin (#3083)

• Rust library rusty-jwt-tools upgraded to latest version (#3112)

• Fixed test of jwt-tools Rust FFI (#3125)

• Enabling warnings for redundant constraints and removing the redundant
  constraints. (#3009)

• Migrate /teams/notifications to use the Servant library. (#3020)

• Split polysemy Members constraints into multiple Member constraints (#3093)

Federation changes

• Use HsOpenSSL instead of tls for federation communication. (#3051)

Release notes

• wire-server helm charts using Ingress resources are now compatible with kubernetes versions 1.22, 1.23 and 1.24 (but remain compatible with older versions of kubernetes).

  If you upgrade to this version of helm charts and/or you upgrade your version of kubernetes while wire-server is deployed, you may find that helm update or helmfile apply/sync gives an error like this:

  Error: UPGRADE FAILED: current release manifest contains removed kubernetes api(s) for this kubernetes version and it is therefore unable to build the kubernetes objects for performing the diff. error from kubernetes: unable to recognize "": no matches for kind "Ingress" in version "extensions/v1beta1"
  In which case you can use the helm mapkubeapis plugin to upgrade an existing release with the following command:

  # install plugin version 0.1.0 (more recent may not work)
  helm plugin install --version v0.1.0 https://github.com/helm/helm-mapkubeapis
  # adjust helm release name and namespace as required
  helm mapkubeapis --namespace wire nginx-ingress-services
  

  Alternatively, if a few minutes of downtime are not a problem; you can helm delete a release and re-install it again, which will work without the above plugin. (#3002)

• Upgrade team-settings version to 4.14.0-v0.31.9-0-bf82b46 (#2180)

• Upgrade webapp version to 2023-01-24-production.0-v0.31.9-0-17b742f (#2302)

API changes

• The unqualified GET /conversations/:id endpoint has been removed from API v3, and is restored to the previous behaviour of returning a Conversation using the v2 schema. Similarly, its qualified counterpart GET /conversations/:domain/:id now returns a v2 Conversation when accessed through API v2. (#2992)

Bug fixes and other updates

• Fix pagination in team user search (make search key unique) (#2968)

• Update inbucket (fake smtp server) chart dependency: The prior version relied on an image that has been removed from docker hub. Thus, our own inbucket chart could not be deployed anymore. (#2998)

Documentation

• Add sphinx-copybutton plugin to make copying snippets of code from docs.wire.com easier. (#2900)

• Hook federated API call documentation into docs.wire.com (manually). (#2988)

• Tool for dumping fed call graphs (dot/graphviz and csv); see README for details (#2973)

Internal changes

• Add Helm chart to configure clusters managed by k8ssandra-operator for test environments. (#2981)

• Fix kind setup for running end-to-end federation tests locally. (#3008)

• Fix Makefile target kind-restart-all. (#3015)

• Add combinators for creating mocked federator responses in integration tests (#3014)

• Add two integration tests arounds last prekeys (#2694)

• Fix make clean (#2965, #2978)

• Make ID tags more readable by expanding abbreviations to full names. (#2991)

• Unused old swagger code removed from stern and team features (#3017)

• Refactor Writetime from Int64 to wrapper of UTCTime (#2994)

• Restructure docs.wire.com (#2986)

• Fixed flaky team user search integration test (#2996)

Release notes

• This realease migrates data from galley.member_client to galley.mls_group_member_client. When upgrading wire-server no manual steps are required. (#2859)

• Upgrade webapp version to 2022-12-19-production.0-v0.31.9-0-6b2f2bf (#2302)

API changes

• • The endpoints POST /conversations/list and GET /conversations have been removed. Use POST /conversations/list-ids followed by POST /conversations/list instead.
  • The endpoint PUT /conversations/:id/access has been removed. Use its qualified counterpart instead.
  • The field access_role_v2 in the Conversation type, in the request body of POST /conversations, and in the request body of PUT /conversations/:domain/:id/access has been removed. Its content is now contained in the access_role field instead. It replaces the legacy access role, previously contained in the access_role field.
  • Clients implementing the V3 API must be prepared to handle a change in the format of the conversation.access_update event. Namely, the field access_role_v2 has become optional. When missing, its value is to be found in the field access_role. (#2841)

• Added a domain parameter to the typing indicator status update API (#2892)

• Support MLS self-conversations via a new endpoint GET /conversations/mls-self. This removes the PUT counterpart introduced in #2730 (#2839)

• List the MLS self-conversation automatically without needing to call GET /conversations/mls-self first (#2856)

• Fail early in galley when the MLS removal key is not configured (#2899)

• Introduce a flag in brig to enable MLS explicitly. When this flag is set to false or absent, MLS functionality is completely disabled and all MLS endpoints fail immediately. (#2913)

• Conversation events may have a "subconv" field for events that originate in a MLS subconversation (#2933)

• GET /system/settings/unauthorized returns a curated set of system settings from brig. The endpoint is reachable without authentication/authorization. It's meant to be used by apps to adjust their behavior (e.g. to show a registration dialog if registrations are enabled on the backend.) Currently, only the setRestrictUserCreation flag is exported. Other options may be added in future (in consultation with the security department.) (#2903)

Features

• The coturn Helm chart now has a .tls.ciphers option to allow setting
  the cipher list for TLS connections, when TLS is enabled. By default,
  this option is set to a cipher list which is compliant with BSI
  TR-02102-2. (#2924)

• Nginz helm chart: The list of upstreams is split into nginx_conf.upstreams and
  nginx_conf.extra_upstreams. Extra upstreams are disabled by default. They can
  be enabled by adding their name (entry's key) to
  nginx_conf.enabled_extra_upstreams. nginx_conf.ignored_upstreams is only
  applied to upstreams from nginx_conf.upstreams. In the default configuration
  of nginz extra upstreams are ibis, galeb, calling-test and proxy. If one
  of those is deployed, its name has be be added to
  nginx_conf.enabled_extra_upstreams (otherwise, it won't be reachable). Unless
  nginx_conf.upstreams hasn't been changed manually (overriding its default),
  this should be the only needed migration step. (#2849)

• A team member's role can now be provisioned via SCIM (#2851, #2855)

• Team search endpoint now supports pagination (#2898, #2895)

• Introduce optional disabledAPIVersions configuration setting (#2951)

• Add more logs to SMTP mail sending. Ensure that logs are written before the application fails due to SMTP misconfiguration. (#2818)

• Added typing indicator status propagation to federated environments (#2892)

• Allow vhost style addressing for S3 as path style is not supported for newer buckets.

  More info: https://aws.amazon.com/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story/ (#2955)

Bug fixes and other updates

• Fix typo for Servicemonitor enable var in default values for helm charts. (#2896)

• The parser for the AWS/SNS error message to explain that an endpoint is already in use was incorrect. This lead to an "invalid token" error when registering push tokens for multiple user accounts (user ids) instead of updating the SNS endpoint with an additional user id. (#2921)

• Avoid client deletion edge case condition which can lead to inconsistent data between brig and galley's clients tables. (#2830)

• Conversations inside events are now serialised using the format of API V2 (#2971)

• Do not throw 500 when listing conversations and MLS is not configured (#2893)

• Do not list MLS self-conversation in client API v1 and v2 if it exists (#2872)

• Limit 2FA code retries to 3 attempts (#2960)

• Fix bug in MLS user removal from conversation: the list of removed clients has to be compared with those in the conversation, not the list of all clients of that user (#2817)

• Due to sftd changing how configuration is handled for "multi-SFT" calling (starting with version 3.1.10), new options have been added to the sftd Helm chart for compatibility with these newer versions. (#2886)

• For sftd/coturn/restund, fixed a bug in external ip address lookup, in case Kubernetes Node Name doesn't equal hostname. (#2837)

• Requesting a new token with the client_id now works correctly when the old token is part of the request (#2860)

Documentation

• Add extra section to the deeplink docs to explain the socks proxy support while login. (#2885)

• Describe the auth cookie throttling mechanism. And overhaul the description of auth cookies in general. (#2941)

• PR guidelines docs are updated with correct helm configuration syntax (#2889)

Internal changes

• Log AWS / SNS invalid token responses. This is helpful for native push notification debugging purposes. (#2908)

• Add tests for invitation urls in team invitation responses. These depend on the settings of galley. (#2797)

• brig: Allow multiple threads to run simultaneously (#2972)

• Remove support for compiling local docker images with buildah. Nix is used to build docker images these days (#2822)

• Nix-created docker images: add some debugging tools in the containers, and add 'make build-image-' for convenience (#2829)

• Added typeclasses to track uses of federated calls across the codebase. (#2940)

• Split galley API routes and handler definitions into several modules (#2820)

• Default intraListing to true. This means that the list of clients, so far saved in both brig's and galley's databases, will still be written to both, but only read from brig's database. This avoids cases where these two tables go out of sync. Brig becomes the source of truth for clients. In the future, if this holds, code and data for galley's clients table can be removed. (#2847)

• Introduce the MakesFederatedCall Servant combinator (#2950)

• Bump nixpkgs to latest unstable. Stop using forked nixpkgs. (#2828)

• Optimize memory usage while creating large conversations (#2970)

• Reduce Polysemy-induced high memory requirements (#2947)

• Brig calling API is now migrated to servant (#2815)

• Fixed flaky feature TTL integration test (#2823)

• Brig teams API is now migrated to servant (#2824)

• Add 'inconsistencies' tool to check for, and repair certain kinds of data inconsistencies across different cassandra tables. (#2840)

• Backoffice Swagger 2.x docs is exposed on / and the old Swagger has been removed. Backoffice helm chart only runs stern without an extra nginx. (#2846)

• Give proxy service a servant routing table for swagger (not for replacing wai-route; see comments in source code) (#2848)

• Stern API endpoint GET ejpd-info has now the correct HTTP method (#2850)

• External commits: add additional checks (#2852)

• Golden tests for conversation and feature config event schemas (#2861)

• Add startup probe to brig helm chart. (#2878)

• Track federated calls in types across the codebase. (#2940)

• Update nix pins to point at polysemy-1.8.0.0 (#2949)

• Add MakesFederatedCall combinators to Galley (#2957)

• Fix make clean; allow new data constructors in ToSchema Version instance (#2965)

• Refactor and simplify MLS message handling logic (#2844)

• Remove cassandra queries to the user_keys_hash table, as they are never read anymore since 'onboarding' / auto-connect was removed in https://github.com/wireapp/wire-server/pull/1005 (#2902)

• Replay external backend proposals after forwarding external commits.
  One column added to Galley's mls_proposal_refs. (#2842)

• Remove an unused effect for remote conversation listing (#2954)

• Introduce types for subconversations (#2925)

• Use treefmt to ensure consistent formatting of .nix files, use for shellcheck too (#2831)

Federation changes

• Honour MLS flag in brig's federation API (#2946)

• Split the Proteus and MLS message sending requests into separate types. The MLS request now supports MLS subconversations. This is a federation API breaking change. (#2925)

• Injects federated calls into the x-wire-makes-federated-calls-to extension of the swagger Operations (#2950)

Bug fixes and other updates

• Prevention of storing unnecessary data in the database if adding a bot to a conversation fails. (#2870)

Internal changes

• bump nginx-module-vts from v0.1.15 to v0.2.1 (#2827)
• Build nginz and nginz_disco docker images using nix (#2796)

Release notes

• If you have not upgraded to release 2021-03-21 (Chart Release 2.103.0) yet, please do that now!

  NB: we only support releases 6 months back, so this should not be an issue. But in this particular case we are positive that things will break if you don't do an intermediate upgrade. (#2768)

• Build docker images using nix derivations instead of Dockerfiles (#2331, #2771, #2772, #2775, #2776)

• Upgrade team-settings version to 4.13.0-v0.31.5-0-4754212 (#2180)

• Upgrade webapp version to 2022-11-02-production.0-v0.31.9-0-337e400 (#2302)

• The experimental wire-server-metrics helm chart has been removed.

  These were mostly a wrapper around prometheus operator. It makes more sense to
  refer to the upstream docs of Prometheus Operator or Grafana Agent Operator for
  installation instead. (#2740)

API changes

• Do not expose swagger-ui on prod systems (to minimize attack surface) (#2800)

• Change mime type of body of /v3/mls/commit-bundles endpoint (#2773)

• Stop rate-limiting asset-signed-url requests on /assets/.* (#2786)

• The /access endpoint now takes an optional client_id query parameter. The first time it is provided, a new user token will be generated containing the given client ID. Successive invocations of /access will ignore the client_id parameter. Some endpoints can now potentially require a client ID as part of the access token. When trying to invoke them with an access token that does not contain a client ID, an authentication error will occur. (#2764)

Features

• Introduce support for external commits in MLS (#2765)

• The GET /teams/{tid}/members endpoint now supports pagination (#2802)

Bug fixes and other updates

• Clients without any prekeys are not deleted completely (#2758)

Documentation

• tentatively allow GET /api/event-notification-schemas for json schemas of server-initiated events (missing pieces tracked in https://wearezeta.atlassian.net/browse/FS-1008) (#2739)

• Fix copyright date on docs.wire.com (#2792)

• Improve and cross-link documentation on SNS / push notifications. (#PR_NOT_FOUND)

• Add extension sphinx-reredirects and configuration to generate simple JavaScript based redirects to new locations of previously inconsistently named files/URLs. (#2811)

Internal changes

• Convert brig's auth endpoints to servant (#2750)

• Remove deprecated table for storing scim external_ids.

  Data has been migrated away in release 2021-03-21 (Chart Release 2.103.0) (see /services/spar/migrate-data/src/Spar/DataMigration/V1_ExternalIds.hs); last time it has been touched in production is before upgrade to release 2021-03-23 (Chart Release 2.104.0). (#2768)

• Refactor some internal Scim user tests (#2762)

• Reduce the payload size of internal client.delete event (#2807, #2816)

• Bump servant-swagger-ui package. (#2747)

• Increase charts/galley memory limit to 500M. (#2798)

• Add RPC, ServiceRPC and GalleyProvider effects to brig (#2653)

• Use locally build schema binaries for db migrations and execute them right before running integration tests. (#2791)

• Rename the make targets from db-migrate-package and db-reset-package to db-migrate and db-reset and allow migrating and resetting all keyspaces. (#2791)

• Add a Make target for ghci (#2749)

• Upgrade nginz/nginx to 1.22.1 (#2777)

• The dev environment provided by nix now contains all the haskell packages
  compiled by nix. This could casue linker errors while compiling haskell code in
  this repo. One way to get resolve them is to delete the 'dist-newstyle'
  directory. (#2331)

• Implemented a new intersperse combinator for Polysemy (#2767)

• Add a Concurrency effect for Polysemy (#2748)

• Don't fail client deletion when mls remove key is undefined (#2738)

• Migrate stern to swagger2-ui (remaining backwards compatible with circulating backoffice images) (see also #2742 from last release) (#2744)

• Gundeck push token API and notification API is migrated to Servant (#2769)

• Delete deploy/services-demo directory (#2789)

• Upgrade Servant to 0.19 (#2809)

Release notes

• Upgrade webapp version to 2022-10-04-production.0-v0.31.2-0-a438b30
  (#2302)

API changes

• Remove /legalhold/conversation alias from v2 (#2734)

• Make v2 a supported version and start v3 (#2734)

Features

• Allow deletion of MLS team conversations (#2733)

Bug fixes and other updates

• Revert synchronous semantics of client deletion endpoint (#2737)

Documentation

• JCT-146 - update outdated info SER-211 - update new info regarding
  nodetool use (#2736)

Internal changes

• Skeleton implementation of new endpoint for JWT DPoP access token
  generation (#2652, #2686)

• Add swagger2-ui to stern (#2742 …)

Release notes

• For users of the (currently alpha) coturn Helm chart, manual action is
  required when upgrading to this version. The labels applied to the Kubernetes
  manifests in this chart have changed, in order to match the conventions used
  in the wire-server charts. However, this may mean that upgrading with Helm can
  fail, due to changes to the StatefulSet included in this chart -- in this
  case, the StatefulSet must be deleted before the chart is upgraded. (#2677)

• wire-server helm charts: Adjust default CPU/Memory resources: Remove CPU limits to avoid CPU throttling; adjust request CPU and memory based on observed values. Overall this decreases the amount of CPU/memory that the wire-server chart needs to install/schedule pods. (#2675)

• Upgrade team-settings version to 4.12.1-v0.31.5-0-0167ea4 (#2180)

• Upgrade webapp version to 2022-09-20-production.0-v0.31.2-0-7f74074 (#2302)

API changes

• Add new endpoint /mls/commit-bundles for submitting MLS CommitBundles. A CommitBundle is a triple consisting of a commit message, an optional welcome message and a public group state. (#2688)

• MLS: Store and expose group info via GET /conversations/:domain/:id/groupinfo (#2721)

• Add /mls/public-keys to nginz chart (#2676)

• Users being kicked out results in member-leave events originating from the user who caused the change in the conversation (#2724)

• Leaving an MLS conversation is now possible using the regular endpoint DELETE /conversations/{cnv_domain}/{cnv}/members/{usr_domain}/{usr}. When a user leaves, the backend sends external remove proposals for all their clients in the corresponding MLS group. (#2667)

• Validate remotely claimed key packages (#2692)

Features

• The coturn chart now has support for exposing its metric endpoint with a
  ServiceMonitor, which can be ingested by third-party metrics collection tools. (#2677)

• Deleting clients creates MLS remove proposals (#2674)

• External remove proposals are now sent to a group when a user is deleted (#2650)

• Allow non-admins to commit add proposals in MLS conversations (#2691)

• Optionally add invitation urls to the body of /teams/{tid}/invitations. This allows further processing; e.g. to send those links with custom emails or distribute them as QR codes. See docs for details and privacy implications. (#2684)

Bug fixes and other updates

• SCIM user deletion suffered from a couple of race conditions. The user in now first deleted in spar, because this process depends on data from brig. Then, the user is deleted in brig. If any error occurs, the SCIM deletion request can be made again. This change depends on brig being completely deployed before using the SCIM deletion endpoint in brig. In the unlikely event of using SCIM deletion during the deployment, these requests can be retried (in case of error). (#2637)

• The 2nd factor password challenge team feature is disabled for SSO users (#2693)

• Less surprising handling of SIGINT, SIGTERM for proxy, stern. Increase grace period for shutdown from 5s to 30s for all services. (#2715)

Documentation

• Drop Client model (unused) from old swagger.
  Add a description and example data for mls_public_keys field in new swagger. (#2657)

• Document user deactivation (aka suspension) with SCIM. (#2720)

• Monitoring page showed wrong wrong configuration charts. Updated prometheus-operator to kube-prometheus-stack chart in the documentation. (#2708)

Internal changes

• Make client deletion asynchronous (#2669)

• Allow external add proposals without previously uploading key packages. (#2661)

• Allow legalhold tokens access to /converations/<uuid> endpoint (#2682, #2726)

• Move Brig.Sem.* modules to Brig.Effects (consistency) (#2672)

• The labels applied to resources in the coturn chart have been changed to
  reflect the conventions in the wire-server charts. (#2677)

• Drop the managed column from team_conv table in Galley (#2127)

• Fix link in PR template (#2673)

• In Gundeck's 'notifications' cassandra table, switch to TWCS compaction strategy, which should be more efficient for this workload, and possibly bring performance benefits to latencies.
  It may be beneficial to run a manual compaction before rolling out this
  change (but things should also work without this manual operation).
  In case you have time, run the following from a cassandra machine before deploying this update: nodetool compact gundeck notifications. (#2615)

• Add regular expression support to libzauth ACL language (#2714)

• Make test API calls point to the most recent version by default (#2695)

• Clients and key package refs in an MLS conversation are now stored in their own table. (#2667)

• Refactor MLS test framework (#2678)

• Update mls-test-cli to version 0.5 (#2685)

• Added rusty-jwt-tools to docker images (#2686)

• The account API is now migrated to servant. (#2699, #2700, #2701, #2702, #2703, #2704, #2705, #2707)

• Update nginz and cannon ACLs to match api-versioned paths (#2725)

• For wire-server cloud, on kubernetes 1.21+, favour topology-aware routing, which reduces unnecessary inter-availability-zone traffic, reducing latency and cloud provider cross-AZ traffic costs. (#2723)

Release notes (Chart Release 4.23.0)

• The internal endpoint GET i/mls/clients has been changed, and it now returns a list of ClientInfo instead of a list of ClientId. (#2631)

API changes

• Fix key package error description (#2651)

• Expose MLS public keys in a new endpoint GET /mls/public-keys. (#2602)

Features

• The coturn chart now supports exposing the control port over TLS. (#2620)

• Forward all MLS default proposal types (#2628)

• New endpoints HEAD and GET /nonce/clients to request new nonces for client certificate requests (coming up soon). (#2641, #2655)

Bug fixes and other updates

• Fix cql-io bug where restarting whole cassandra cluster could cause downtime. Upstream changes in https://gitlab.com/twittner/cql-io/-/merge_requests/20 (#2640)

• Improve client check when adding clients to MLS conversations (#2631)

Documentation

• Move developer docs onto docs.wire.com (instead of exposing them on github only) (#2622, #2649)

• Add build instructions for developers (#2621)

• Make target audience explicit on docs.wire.com (#2662)

Internal changes

• Support for external Add proposals (#2567)

• Add additional checks on incoming MLS messages:

  • if the sender matches the authenticated user
  • if the sender of message to a remote conversation is a member
  • if the group ID of a remote conversation matches the local mapping (#2618)

• Apply changes introduced by cabal-fmt. (#2624)

• Remove some redudant constraints in brig (#2638)

• Brig Polysemy: Port UserPendingActivationStore to polysemy (#2636)

• Add make target delete-cache-on-linker-errors to delete all Haskell compilation related caches. This is useful in cases where the development environment gets into an inconsistent state. (#2623)

• Move Paging effect from galley into polysemy-wire-zoo (#2648)

• Fix broken hls-hlint-plugin in nix env (#2629)

• Adjust developer PR template and document config and API procedures in-tree. (#2617)

• Add mls-test-cli to builder image (#2626)

• Add mls-test-cli to deps image (#2630)

• mls-test-cli: Use Cargo.lock file when building (#2634)

• Move common Arbitrary instances to types-common package for compilation speed (#2658)

• LoginId migrated to schema-profunctor (#2633, #2645)

• Improve cleaning rules in Makefile. (#2639)

• Fix typos, dangling reference in source code haddocs, etc. (#2586)

• Update the Elastic Search version used for running integration tests to the one that is delivered by wire-server-deploy. (#2656)

Federation changes

• Add mlsPrivateKeyPaths setting to galley (#2602)

API changes

• Drop the deprecated member removal endpoint (#2593)

Features

• charts/cannon: Ensure HSTS headers are set for all endpoints (#2574)

• Expired MLS key packages are deleted from the database (#2582)

• Add support for MLS Remove proposals (#2561)

• Human readable names for SAML IdPs (#2565)

• The preferredLanguage field from SCIM now maps to the user locale in BRIG and will be set and updated on post SCIM user and on update SCIM user using SAML. (#2605)

• For TLS1.2, by default, remove ECDHE-ECDSA-AES128-GCM-SHA256 and ECDHE-RSA-AES128-GCM-SHA256 ciphers for ingress traffic. (#2528)

Bug fixes and other updates

• Allow deleting existing splash screens in `PUT /teams/:tid (see also PR#2474 in Release 4.18.0) (#2588)
• Backoffice: Fix an issue where in some deployments ibis/galeb (Wire Cloud internal services) are unreachable from backoffice if deployed in a different namespace. (#2610)
• Fix an issue for larger client requests on e.g. /list-users and /list-conversations, which were giving 413 errors for some users. Allow client requests of 256k by default (was 64k). (#2579)

Internal changes

• Add shellcheck, libstdc++ to nix env; handle emacs auto-save files better (#2609)
• Allow features to be set with HTTP method PATCH. This reflects a prior behavior
  that is used by Ibis. Additionally, it's more consistent when all setters can be
  called with PUT and PATCH. As this will fix calls by Ibis, the deployment order
  doesn't matter. (#2575)
• Brig Polysemization: introduce BlacklistStore and BlacklistPhonePrefixStore effects (#2590)
• Add cabal-fmt development tool (#2601)
• Reformat all cabal files with cabal-fmt (#2603)
• Delete tools: bonanza and makedeb (#2600)
• No more package.yaml / hpack, and stick with cabal files as the single (and only) source of truth (#2596)
• Port Brig SearchAPI and UserRichInfo endpoints to Servant (#2580)
• Added TTL data to stern feature flag GET endpoint. (#2564)
• Prepare removing deprecated non-binding teams (no more used in integration tests) (#2514, #2607)
• Add internal endpoint in Brig to update clients' key package refs in DB upon committing.
  Brig should be deployed before Galley. (#2604)
• Improved the resilience of provisioning new users via SAML by combining two persistence calls into one, preventing a creation failure from locking a user handle with no corresponding user. (#2526)

Federation changes

• Fix TBS field in MLS Message type (#2599)

Release notes

• Users of the (currently alpha) coturn Helm chart must manually update
  their configuration due to changes in how the chart handles authentication
  secrets. Please see below for further details. (#2553)

API changes

• The response to POST /mls/messages adds a timestamp (#2560)

Features

• charts/wire-server: default log format everywhere to StructuredJSON format (introduced in #1951 and #1959) (#2559)

• The coturn chart now supports multiple authentication secrets, which permits
  multiple backend instances to use the same TURN servers without needing to
  share authentication secrets between the backend instances.

  Correspondingly, the .Values.secrets.zrestSecret configuration option, which
  took a single authentication secret as its argument, has been replaced with the
  option .Values.secrets.zrestSecrets (note spelling!), which instead takes a
  list of authentication secrets as its argument. (#2553)

• Add support for bare MLS proposals (#2436)

Bug fixes and other updates

• Fix a bug in charts/cannon. It's now possible to use a custom TLS certificate when enabling cannon's nginz sidecar container. (Previously only letsencrypt certificates worked, and were tested) (#2558)

• Minor fixes in helmcharts:

  • charts/nginz: Rate limit SSO endpoints less
  • charts/nginz: Ensure rate limiting isn't commented out
  • charts/galley: Honour .setttings.httpPoolSize
  • charts/galley: Fix typo in settings.featureFlags.validateSAMLEmails
  • charts/gundeck: Remove aws.connectionLimit
  • charts/brig: Fix default brandLabelUrl and remove brandLabel (#2563)

Internal changes

• Port brig UserHandle API to servant (#2556)

• Bump timeout for integration tests to 15 minutes (from 10 minutes), as 10 minutes is no longer enough. (#2570)

• Internal endpoints to PATCH feature status (#2555)

• Change the proposal hold time to 28 days (#2568)

Release notes

• Deploy spar before galley (#2543)

• Upgrade team-settings version to 4.11.0-v0.31.1-0-9e64150 (#2180)

API changes

• Restore PUT /v2/connections endpoint (#2539)

Features

• 2nd factor authentication code generation is rate limited now (#2522)

• The team member CSV export now fills created_on for SCIM users (#2543)

Internal changes

• Add Helm chart for inbucket. Inbucket is a SMTP server that does not relay, but instead display received mail in a webapp and make them accessible via an API. (#2544)

• Bump saml2-web-sso (#2545, #2546)

• use checkedConnectCluster to avoid dropping requests to Redis when Gundeck reconnects to the Redis cluster (#2542)

• Do not log polysemy errors in Galley (#2531)

• Remove old crypto-cli tool from the ubuntu image (#2538)

Federation changes

• Added new-remote-conversation RPC, used to notify a backend of a remote conversation the first time any user from that backend is added to it. (#2378)

• Added federation endpoint send-mls-message used to send messages to remote converesations (#2378)

Release notes

• For users of the (currently alpha) coturn Helm chart:
  manual intervention may be required when upgrading to
  this version of the chart from a prior version, due to a bug in
  Kubernetes which
  may interfere with applying changes to pod and service port configuration
  correctly.

  If, after updating this chart, the coturn pods do not have both a coturn-udp
  port and a coturn-tcp port, then the coturn StatefulSet must be manually
  deleted from the cluster, and then recreated by re-running Helm. Similarly, if
  the coturn Service does not have both a coturn-udp port and a coturn-tcp
  port, this Service must also be deleted and recreated. (#2500)

• The nginz{-tcp,-http} services have been unified into a nginz service, and
  moved into the nginz chart.

  The nginz-ingress-services chart simply targets the nginz service, so there's
  no need to set matching service.nginz.external{Http,Tcp}Port inside the
  nginx-ingress-services chart anymore.

  The config.http.httpPort and config.ws.wsPort values in the nginz chart
  still configure the ports the nginz service is listening on.

  Metrics were moved from config.http.httpPort to a new http-metrics port.

  The nginz chart also gained support for metrics.serviceMonitor.enabled,
  creating a ServiceMonitor resource to scrape metrics, like for other wire
  services.

  (#2476)

• Upgrade webapp version to 2022-06-30-production.0-v0.30.5-0-3e2aaf6 (#2302)

• In the helm charts, the wireService label has been removed.

  In some cases, we were already setting the app label too.

  Now we consistently use the app label to label different wire services.

  The wireService label was also used in the spec.selector.matchLabels field
  on existing Deployment / StatefulSet resources.
  As these fields being immutable, changing them isn't possible without recreation.

  If you encounter an issue like

  field is immutable && cannot patch "*" with kind *

  you need to manually delete these StatefulSet and Deployment resources, and apply helm again, which will recreate them.

  This means downtime, so plan a maintenance window for it.

  The wire-server-metrics chart was previously running some custom
  configuration to automatically add all payloads with a wireService label into
  metrics scraping.

  With the removal of the wireService label, this custom configuration has been
  removed.

  Instead, all services that expose metrics will now create ServiceMonitor
  resources, if their helm chart is applied with metrics.serviceMonitor.enable
  set to true.

  This prevents scraping agents from querying services that don't expose metrics
  at /i/metrics unnecessarily.

  Additionally, makes it easier to run other metric scraping operators, like
  grafana-agent-operator, without the need to also create some custom
  wireService label config there.

  Generally, if you have any monitoring solution installed in your cluster that
  uses the Prometheus CRDs, set metrics.serviceMonitor.enable for the following charts:

  • brig
  • cannon
  • cargohold
  • galley
  • gundeck
  • proxy
  • spar (#2413)

API changes

• The request body of POST /conversations endpoint can now contain an optional creator_client field. The creator_client field is only relevant for MLS conversations, in which case it must be set to the ID of the client making the request. (#2486)

• Retire deprecated feature config API endpoints for API version V2 (#2492)

Features

• Prevent race conditions in concurrent MLS commit requests. (#2525)

• charts/wire-server: Optionally include backoffice (#2490)

• The coturn chart has new functionality to enable graceful pod termination, by
  waiting for all active allocations on a coturn instance to drain first. When
  combined with a suitable external service discovery mechanism which can steer
  client traffic away from terminating coturn pods, this can be used to implement
  graceful rolling restarts of clusters of coturn instances. (#2456)

• ./deploy/services-demo/create_team_members.sh creates users with given roles now (#2137)

• MLS implementation progress:

  • Remote users can be added to MLS conversations
  • MLS messages (both handshake and application) are now propagates to remote
    conversation participants. (#2415)

• charts/nginz: Serve swagger-ui for viewing swagger-1.2 docs (#2466)

• GET teams/:tid response now contains an optional field splash_screen which contains the asset key of the team's splash screen. PUT teams/:tid now supports updating the splash screen asset key. (#2474)

• Missing feature config mapping added (#2494)

• Add MLS team feature configuration (#2499)

• Team feature API now includes endpoints to get and set the searchVisibilityInbound feature (#2503)

Bug fixes and other updates

• charts/backoffice: Fix version of frontend and auto-bump version of stern on every release (#2490)

• The service definitions in the coturn Helm chart were missing the control plane
  UDP port used by coturn. (#2500)

• In nginx-ingress-services chart, when enabling useCertManager, now correctly creates the required issuer by default. (#2532)

• Fix handling of creator client in MLS conversations (#2486)

• Fix all clients having the same MLS public key (#2501)

• A user now cannot delete an identity provider that they are authenticated with any more (#2519)

Internal changes

• brig-types: remove all re-exports (#2505)

• Fixed flakiness of email update test, related to the test user account being suspended, causing subsequent runs of the test to fail. (#2497)

• galley-types: remove all re-exports (#2504)

• Enforce some IdP invariants (#2533)

• Switch to new MLS test CLI (https://github.com/wireapp/mls-test-cli) (#2508)

• Forward /i/users/:uid/features/:feature to brig (#2468)

• charts/nginz: Forward /i/legalhold/whitelisted-teams to galley instead of brig (#2460)

• make the ldap-scim-bridge chart deployable once per team, and improve docs. (#1843)

• Refactored and simplified the feature config API (#2435)

• Removed deprecated internal feature config API endpoints (#2496)

• Deactivated gundeck's integration tests for local steps (make ci). (#2510)

• retry gundeck's Redis connection in case of network errors such as IP changes or network outages (#2512)

• Add AWS security token metrics to all services (#2473)

Release notes

• Upgrade team-settings version to 4.10.0-v0.29.7-0-3be8ca3 (#2180)

• Upgrade webapp version to 2022-06-13-production.0-v0.29.7-0-2819b90 (#2302)

Documentation

• Docs for guest links server and team feature settings added (#2480)

Internal changes

• All feature configs like guest links e.g. can now be overridden in the helm configuration, so that they can be disabled/enabled and configured server wide (#2479)