linkerd2

edge-22.7.1

This release includes a security improvement. When a user manually specified the
policyValidator.keyPEM setting, the value was incorrectly included in the
linkerd-config configmap. This means that this private key was erroneously
exposed to service accounts with read access to this configmap. Practically,
this means that the Linkerd proxy-injector, identity, and heartbeat pods
could read this value. This should not have exposed this private key to
other unauthorized users unless additional role bindings were added outside of
Linkerd. Nevertheless, we recommend that users who manually set control plane
certificates update the credentials for the policy validator after upgrading
Linkerd.

Additionally, the linkerd-multicluster extensions has several fixes related to
fail fast errors during link watch restarts, improper label matching for
mirrored services, and properly cleaning up mirrored endpoints in certain
situations.

Lastly, the proxy can now retry gRPC requests that have responses with a
TRAILERS frame. A fix to reduce redundant load balancer updates should also
result in less connection churn.

• Changed unit tests to use newly introduced prommatch package for asserting
  expected metrics (thanks @krzysztofdrys!)
• Fixed Docker container runtime check to only during linkerd install rather
  than linkerd check --pre
• Changed linkerd-multicluster's remote cluster watcher to assume the gateway is
  alive when starting—fixing fail fast errors from occurring during restarts
  (thanks @chenaoxd!)
• Added matchLabels and matchExpressions to linkerd-multicluster's Link CRD
• Fixed linkerd-multicluster's label selector to properly select resources that
  match the expected label value, rather than just the presence of the label
• Fixed linkerd-multicluster's cluster watcher to properly clean up endpoints
  belonging to remote headless services that are no longer mirrored
• Added the HttpRoute CRD which will be used by future policy features
• Fixed CNI plugin event processing where file updates could sometimes be
  skipped leading to the update not being acknowledged
• Fixed redundant load balancer updates in the proxy that could cause
  unnecessary connection churn
• Fixed gRPC request retries for responses that contain a TRAILERS frame
• Fixed the dashboard's linkerd check due to missing RBAC for listing pods in
  the cluster
• Fixed API check that ensures access to the Server CRD (thanks @aatarasoff!)
• Changed linkerd authz to match the labels of pre-fetched Pods rather than
  the multiple API calls it was doing—resulting in significant speed-up (thanks
  @aatarasoff!)
• Unset policyValidtor.keyPEM in linkerd-config ConfigMap

stable-2.11.3

This release pulls in several control plane and proxy fixes from the main
development branch. The linkerd-multicluster extension has several fixes
regarding incorrect label matching and resource cleanup. Additionally, a long
standing panic has been fixed in the proxy.

• Fixed an error in linkerd multicluster allow which resulted in broken YAML
  output
• Fixed a potential panic in the proxy's outbound load balancer that could be
  triggered when the balancer processes many service discovery updates in a
  short period of time.
• Fixed a class of DNS errors by ensuring the proxy falls back to A records when
  SRV resolution fails
• Fixed an issue where the proxy would pass along illegal headers from CONNECT
  responses
• Fixed several Helm labels to follow the Helm standards recommendation which
  were sometimes resulting chart generation errors
• Fixed an issue where linkerd check did not skip Pods with a NodeShutdown
  status resulting in incorrect errors
• Fixed the Docker container runtime check to only occur during linkerd install rather than linkerd check
• Fixed a class of fail fast errors that were occurring with
  linkerd-multicluster due to delayed gateway liveness probes
• Fixed linkerd-multicluster Endpoints not being deleted when their remote
  Service was no longer mirrored
• Fixed linkerd-multicluster's label selector to properly match the value of
  mirror.linkerd.io/exported rather than just its presence

edge-22.6.2

This edge release bumps the minimum supported Kubernetes version from v1.20
to v1.21, introduces some new changes, and includes a few bug fixes. Most
notably, a bug has been fixed in the proxy's outbound load balancer that could
cause panics, especially when the balancer would process many service discovery
updates in a short period of time. This release also fixes a panic in the
proxy-injector, and introduces a change that will include HTTP probe ports in
the proxy's inbound ports configuration, to be used for policy discovery.

• Fixed a bug in the proxy's outbound load balancer that could cause panics
  when many discovery updates were processed in short time periods
• Added runtimeClassName options to Linkerd's Helm chart (thanks @jtcarnes!)
• Introduced a change in the proxy-injector that will configure the inbound
  ports proxy configuration with the pod's probe ports (HTTPGet)
• Added godoc links in the project README file (thanks @spacewander!)
• Increased minimum supported Kubernetes version to v1.21 from v1.20
• Fixed an issue where the proxy-injector would not emit events for resources
  that receive annotation patches but are skipped for injection
• Refactored PublicIPToString to handle both IPv4 and IPv6 addresses in a
  similar behavior (thanks @zhlsunshine!)
• Replaced the usage of branch with tags, and pinned cosign-installer action
  to v1 (thanks @saschagrunert!)
• Fixed an issue where the proxy-injector would panic if resources have an
  unsupported owner kind

edge-22.6.1

This edge release fixes an issue where Linkerd injected pods could not be
evicted by Cluster Autoscaler. It also adds the --crds flag to linkerd check
which validates that the Linkerd CRDs have been installed with the proper
versions.

The previously noisy "cluster networks can be verified" check has been replaced
with one that now verifies each running Pod IP is contained within the current
clusterNetworks configuration value.

Additionally, linkerd-viz is no longer required for linkerd-multicluster's
gateways command — allowing the Gateways API to marked as deprecated for
2.12.

Finally, several security issues have been patched in the Docker images now that
the builds are pinned only to minor — rather than patch — versions.

• Replaced manual IP address parsing with functions available in the Go standard
  library (thanks @zhlsunshine!)
• Removed linkerd-multicluster's gateway command dependency on the linkerd-viz
  extension
• Fixed issue where Linkerd injected pods were prevented from being evicted by
  Cluster Autoscaler
• Added the dst_target_cluster metric to linkerd-multicluster's service-mirror
  controller probe traffic
• Added the --crds flag to linkerd check which validates that the Linkerd
  CRDs have been installed
• Removed the Docker image's hardcoded patch versions so that builds pick up
  patch releases without manual intervention
• Replaced the "cluster networks can be verified check" check with a "cluster
  networks contains all pods" check which ensures that all currently running Pod
  IPs are contained by the current clusterNetworks configuration
• Added IPv6 compatible IP address generation in certain control plane
  components that were only generating IPv4 (thanks @zhlsunshine!)
• Deprecated linkerd-viz's Gateways API which is no longer used by
  linkerd-multicluster
• Added the promm package for making programatic Prometheus assertions in
  tests (thanks @krzysztofdrys!)
• Added the runAsUser configuration to extensions to fix a PodSecurityPolicy
  violation when CNI is enabled

edge-22.5.3

This edge release fixes a few proxy issues, improves the upgrade process, and
introduces proto retries to Service Profiles. Also included are updates to the
bash scripts to ensure that they follow best practices.

• Polished the shell scripts (thanks @joakimr-axis)
• Introduced retries to Service Profiles based on the idempotency option of the
  method by adding an isRetryable function to the proto definition
  (thanks @mahlunar)
• Fixed proxy responses to CONNECT requests by removing the content-length
  and/or transfer-encoding headers from the response
• Fixed DNS lookups in the proxy to consistently use A records when SRV records
  cannot be resolved
• Added dynamic policy discovery to the proxy by evaluating traffic on ports
  not included in the LINKERD2_PROXY_INBOUND_PORTS environment variable
• Added logic to require that the linkerd CRDs are installed when running
  the linkerd upgrade command

edge-22.5.2

This edge release ships a few changes to the chart values, a fix for
multicluster headless services, and notable proxy features. HA functionality,
such as PDBs, deployment strategies, and pod anti-affinity, have been split
from the HA values and are now configurable for the control plane. On the proxy
side, non-HTTP traffic will now be forwarded on the outbound side within the
cluster when the proxy runs in ingress mode.

• Updated ingress-mode proxies to forward non-HTTP traffic within the cluster
  (protocol detection will always be attempted for outbound connections)
• Added a new proxy metric process_uptime_seconds_total to keep track of the
  number of seconds since the proxy started
• Fixed an issue with multicluster headless service mirroring, where exported
  endpoints would be mirrored with a delay, or when changes to the export label
  would be ignored
• Split HA functionality, such as PodDisruptionBudgets, into multiple
  configurable values (thanks @evan-hines-firebolt for the initial work)

edge-22.5.1

This edge release adds more flexibility to the MeshTLSAuthentication and
AuthorizationPolicy policy resources by allowing them to target entire
namespaces. It also fixes a race condition when multiple CNI plugins are
installed together as well as a number of other bug fixes.

• Added support for MeshTLSAuthentication resources to target an entire
  namespace, authenticating all ServiceAccounts in that namespace
• Fixed a panic in linkerd install when the --ignore-cluster flag is passed
• Fixed issue where pods would fail to start when enablePSP and
  proxyInit.runAsRoot are set
• Added support for AuthorizationPolicy resources to target namespaces, applying
  to all Servers in that namespace
• Fixed a race condition where the Linkerd CNI configuration could be
  overwritten when multiple CNI plugins are installed
• Added test for opaque ports using Service and Pod IPs (thanks @krzysztofdrys!)
• Fixed an error in the linkerd-viz Helm chart in HA mode

edge-22.4.1

In order to support having custom resources in the default Linkerd installation,
the CLI install flow is now always a 2-step process where
linkerd install --crds must be run first to install CRDs only and then linkerd install is run
to install everything else. This more closely aligns the CLI install flow with
the Helm install flow where the CRDs are a separate chart. This also applies to
linkerd upgrade. Also, the config and control-plane sub-commands have been
removed from both linkerd install and linkerd upgrade.

On the proxy side, this release fixes an issue where proxies would not honor the
cluster's opaqueness settings for non-pod/service addresses. This could cause
protocol detection to be peformed, for instance, when using off-cluster
databases.

This release also disables the use of regexes in Linkerd log filters (i.e., as
set by LINKERD2_PROXY_LOG). Malformed log directives could, in theory, cause a
proxy to stop responding.

The helm.sh/chart label in some of the CRDs had its formatting fixed, which
avoids issues when installing/upgrading through external tools that make use of
it, such as recent versions of Flux.

• Added --crds flag to install/upgrade and remove config/control-plane stages
• Allowed the AuthorizationPolicy CRD to have an empty
  requiredAuthenticationRefs entry that allows all traffic
• Introduced nodeAffinity config in all the charts for enhanced control on the
  pods scheduling (thanks @michalrom089!)
• Introduced resources, nodeSelector and tolerations configs in the
  linkerd-multicluster-link chart for enhanced control on the service mirror
  deployment (thanks @utay!)
• Fixed formatting of the helm.sh/chart label in CRDs
• Updated container base images from buster to bullseye
• Added support for spaces in the config.linkerd.io/opaque-ports annotation

stable-2.11.2

This release pulls in many small fixes and improvements from the main
development branch. It features changes to the multicluster extension to
support the new linkerd-failover extension so that clients can
failover across services hosted on remote clusters.

• CLI

  • Updated check to avoid checking the proxy version of uninjected pods
  • Updated check to skip evicted pods
  • Updated extension install commands to support the --ignore-cluster flag

• Core

  • Fixed a bug in the destination controller that could prevent service
    endpoint updates from being sent to the proxy
  • Updated the destination controller to honor Server resources when
    determining an endpoint's opaqueness
  • Updated the proxy to correctly honor opaque protocol hints for
    non-Kubernetes targets, i.e., when a workload's
    config.linkerd.io/enable-external-profiles annotation is set to true
  • Updated controller webhook servers to ensure that TLS v1.2 or greater is
    used
  • Disabled pprof in control plane admin endpoints by default
  • Updated controllers to ensure that user input is quoted & escaped
    in log messages
  • Updated the proxy's linkerd-await post-start hook to timeout after 2
    minutes. This makes it easier to debug proxies that fail to become ready
  • Updated the proxy init container to support JSON log formatting
  • Added a config.linkerd.io/skip-subnets workload annotation that can be
    used to configure the proxy-init to skip rewriting all traffic to a given
    subnet. This is primarily intended to support docker-in-docker deployments
  • Updated the policy controller to use an openssl backend for its admission
    controller server on x86_64 to improve interopability with more exotic
    Kubernetes server configurations
  • Updated the policy controller to dynamically reload its webhook server
    credentials without restarting
  • Updated the Server CRD to relax OpenAPI schema validation requirements
  • Updated the policy controller webhook server to enforce validation of
    Server and ServerAuthorization resources
  • Added a proxyInit.runAsRoot helm variable that may be set to false to run
    the proxy-init container as a non-root user
  • Updated controller servers to limit the amount of data that may be buffered
    to guard against malicious clients
  • Removed use of the deprecated beta.kubernetes.io/node label

• Jaeger

  • Upgraded jaeger to v1.31 and opentelemetry-collector to v0.43 to support
    ARM

• Multicluster

  • Updated service mirrors so that local services reflect the
    readiness of the remote service. When the remote service has no ready
    endpoints or when its gateway is unavailable, the mirrored local service
    will also have no ready endpoints
  • Fixed a configuration issue that prevented multicluster gateways from
    running on ARM nodes
  • Updated multicluster service mirrors to only create mirrored services when
    the service's namespace already exists in the local cluster
  • Fixed a bug that prevented WebSocket requests from being routed by gateways
  • Updated the linkerd-multicluster-link Helm chart so that a RoleBinding
    is created for each target cluster. This role binding is now only created
    when the enablePSP helm value is set to true
  • Added a linkerd multicluster install --ha flag to run gateways with
    multiple replicas, pod disruption budgets, anti-affinity settings, etc

edge-22.3.5

This edge release introduces new policy CRDs that allow for more generalized
authorization policies.

The AuthorizationPolicy CRD authorizes clients that satisfy all the required
authentications to communicate with the Linkerd Server that it targets.
Required authentications are specified through the new MeshTLSAuthentication
and NetworkAuthentication CRDs.

A MeshTLSAuthentication defines a list of authenticated client IDs—specified
directly by proxy identity strings or referencing resources such as
ServiceAccounts.

A NetworkAuthentication defines a list of client networks that will be
authenticated.

Additionally, to support the new CRDs, policy-related labels have been changed
to better categorize policy metrics. A srv_kind label has been introduced
which splits the current srv_name value—formatted as kind:name—into separate
labels. The saz_name label has been removed and is replaced by the new
authz_kind and authz_name labels.

• Introduced the srv_kind label which allowed splitting the value of the
  current srv_name label
• Removed the saz_name label and replaced it with the new authz_kind and
  authz_name labels
• Fixed an issue in the destination controller where an update would not be sent
  after an endpoint was discovered for a currently empty service
• Introduced the following custom resource types to support generalized
  authorization policies: AuthorizationPolicy, MeshTLSAuthentication,
  NetworkAuthentication
• Deprecated the --proxy-version flag (thanks @importhuman!)
• Updated linkerd-viz to use new policy CRDs

edge-22.3.4

• Disabled pprof endpoints on Linkerd control plane components by default
• Fixed an issue where mirror service endpoints of headless services were always
  ready regardless of gateway liveness
• Added server side validation for ServerAuthorization resources
• Fixed an "origin not allowed" issue when using the latest Grafana with the
  Linkerd Viz extension

edge-22.3.3

This edge release ensures that in multicluster installations, mirror service
endpoints have their readiness tied to gateway liveness. When the gateway for a
target cluster is not alive, the endpoints that point to it on a source cluster
will properly indicate that they are not ready.

• Fixed tap controller logging errors that were succeptible to log forgery by
  ensuring special characters are escaped
• Fixed issue where mirror service endpoints were always ready regardless of
  gateway liveness
• Removed unused namespace entry in linkerd-control-plane chart

edge-22.3.2

This edge release includes a few fixes and quality of life improvements. An
issue has been fixed in the proxy allowing HTTP Upgrade requests to work
through multi-cluster gateways, and the init container's resource limits and
requests have been revised. Additionally, more Go linters have been enabled and
improvements have been made to the devcontainer.

• Changed linkerd-init resource (CPU/memory) limits and requests to ensure by
  default the init container does not break a pod's Guaranteed QOS class
• Added a new check condition to skip pods whose status is NodeShutdown
  during validation as they will not have a proxy container
• Fixed an issue that would prevent proxies from sending HTTP Upgrade requests
  (used in websockets) through multi-cluster gateways

edge-22.3.1

This edge release includes updates to dependencies, CI, and rust 1.59.0. It also
includes changes to the linkerd-jaeger chart to ensure that namespace labels
are preserved and adds support for imagePullSecrets, along with improvements
to the multicluster and policy functionality.

• Added note to multicluster link command to clarify that the link is
  one-direction
• Introduced imagePullSecrets to Jaeger Helm chart
• Updated Rust to v1.59.0
• Fixed a bug where labels can be overwritten in the linkerd-jaeger chart
• Fix broken mirrored headles services after repairEndpoints runs
• Updated Server CRD to handle an empty PodSelector

edge-22.2.4

This edge release continues to address several security related lints and
ensures they are checked by CI.

• Add linkerd check warning for clusters that cannot verify their
  clusterNetworks due to Nodes missing the podCIDR field
• Changed Server CRD to allow having an empty PodSelector
• Modified linkerd inject to only support https URLs to mitigate security
  risks
• Fixed potential goroutine leak in the port forwarding used by several CLI
  commands and control plane components
• Fixed timeouts in the policiy validator which could lead to failures if
  failurePolicy was set to Fail

edge-22.2.3

This edge release fixes some Instant-related proxy panics that occur on Amazon
Linux. It also includes many behind the scenes improvements to the project's
CI and linting.

• Removed the --controller-image-version install flag to simplify the way that
  image versions are handled. The controller image version can be set using the
  --set linkerdVersion flag or Helm value
• Lowercased logs and removed redundant lines from the Linkerd2 proxy init
  container
• Prevented the proxy from logging spurious errors when its pod does not define
  any container ports
• Added workarounds to reduce the likelihood of Instant-related proxy panics
  that occur on Amazon Linux

edge-22.2.2

This edge release updates the jaeger extension to be available in ARM
architectures and applies some security-oriented amendments.

• Upgraded jaeger and the opentelemetry-collector to their latest versions,
  which now support ARM architectures
• Fixed linkerd multicluster check which was reporting false warnings
• Started enforcing TLS v1.2 as a minimum in the webhook servers
• Had the identity controller emit SHA256 certificate fingerprints in its
  logs/events, instead of MD5

edge-22.2.1

This edge release removed the disableIdentity configuration now that the proxy
no longer supports running without identity.

• Added a privileged configuration to linkerd-cni which is required by some
  environments
• Fixed an issue where the TLS credentials used by the policy validator were not
  updated when the credentials were rotated
• Removed the disableIdentity configurations now that the proxy no longer
  supports running without identity
• Fixed an issue where linkerd jaeger check would needlessly fail for BYO
  Jaeger or collector installations
• Fixed a Helm HA installation race condition introduced by the stoppage of
  namespace creation

edge-22.1.5

This edge release adds support for per-request Access Logging for HTTP inbound
requests in Linkerd. A new annotation i.e. config.linkerd.io/access-log is added,
which configures the proxies to emit access logs to stderr. apache and json
are the supported configuration options, emitting access logs in Apache Common
Log Format and JSON respectively.

Special thanks to @tustvold for all the initial work around this!

• Updated injector to support the new config.linkerd.io/access-log annotation
• Added a new LINKERD2_PROXY_ACCESS_LOG proxy environment variable to configure
  the access log format (thanks @tustvold)
• Updated service mirror controller to emit relevant events when
  mirroring is skipped for a service
• Updated various dependencies across the project (thanks @dependabot)

edge-22.1.4

This edge release features a new configuration annotation, support for
externally hosted Grafana instances, and other improvements in the CLI,
dashboard and Helm charts. To learn more about using an external Grafana
instance with Linkerd, you can refer to our
docs.

• Added a new annotation to configure skipping subnets in the init container
  (config.linkerd.io/skip-subnets). This configuration option is ideal for
  Docker-in-Docker (dind) workloads (thanks @michaellzc!)
• Added support in the dashboard for externally hosted Grafana instances
  (thanks @jackgill!)
• Introduced resource block to linkerd-jaeger Helm chart (thanks
  @yuriydzobak!)
• Introduced parametrized datasource (DS_PROMETHEUS) in all Grafana
  dashboards. This allows pointing to the right Prometheus datasource when
  importing a dashboard
• Introduced a consistent --ignore-cluster flag in the CLI for the base
  installation and extensions; manifests will now be rendered even if there is
  an existing installation in the current Kubernetes context (thanks
  @krzysztofdrys!)
• Updated the service mirror controller to skip mirroring services whose
  namespaces do not yet exist in the source cluster; previously, the service
  mirror would create the namespace itself.