antrea

Changed

• Document the limit of maximum receiver group number on a Linux Node for multicast. (#4850, @ceclinux)

Fixed

• Fix Service not being updated correctly when stickyMaxAgeSeconds or InternalTrafficPolicy is updated. (#4845, @tnqn)
• Fix EndpointSlice API availablility check to resolve the issue that AntreaProxy always falls back to the Endpoints API when EndpointSlice is enabled (#4852, @tnqn)
• Fix the Antrea Agent crash issue when large amount of multicast receivers with different multicast IPs on one Node start together.(#4870, @ceclinux)

Changed

• Upgrade Antrea base image to ubuntu:22.04. (#4459 #4499, @antoninbas)

Fixed

• Ensure NO_FLOOD is always set for IPsec tunnel ports and TrafficControl ports. (#4419 #4654 #4674, @xliuxu @tnqn)
• Fix Service routes being deleted on Agent startup on Windows. (#4470, @hongliangl)
• Fix route deletion for Service ClusterIP and LoadBalancerIP when AntreaProxy is enabled. (#4711, @tnqn)
• Fix OpenFlow Group being reused with wrong type because groupDb cache was not cleaned up. (#4592, @ceclinux)
• Add a periodic job to rejoin dead Nodes to fix Egress not working properly after long network downtime. (#4491, @tnqn)
• Fix Agent crash in dual-stack clusters when any Node is not configured with an IP address for each address family. (#4480, @hongliangl)
• Fix potential deadlocks and memory leaks of memberlist maintenance in large-scale clusters. (#4469, @wenyingd)
• Fix connectivity issues caused by MAC address changes with systemd v242 and later. (#4428, @wenyingd)
• Fix a ClusterInfo export bug when Multi-cluster Gateway changes. (#4412, @luolanzone)
• Fix OpenFlow rules not being updated when Multi-cluster Gateway updates. (#4388, @luolanzone)
• Set no-flood config with ports for TrafficControl after Agent restarting. (#4318, @hongliangl)

Changed

• Add OVS connection check to Agent's liveness probes for self-healing on OVS disconnection. (#4126, @tnqn)
• Upgrade Antrea base image to ubuntu:22.04. (#4459 #4499, @antoninbas)

Fixed

• Ensure NO_FLOOD is always set for IPsec tunnel ports and TrafficControl ports. (#4419 #4654 #4674, @xliuxu @tnqn)
• Fix Service routes being deleted on Agent startup on Windows. (#4470, @hongliangl)
• Fix route deletion for Service ClusterIP and LoadBalancerIP when AntreaProxy is enabled. (#4711, @tnqn)
• Add a periodic job to rejoin dead Nodes to fix Egress not working properly after long network downtime. (#4491, @tnqn)
• Fix Agent crash in dual-stack clusters when any Node is not configured with an IP address for each address family. (#4480, @hongliangl)
• Fix potential deadlocks and memory leaks of memberlist maintenance in large-scale clusters. (#4469, @wenyingd)
• Fix connectivity issues caused by MAC address changes with systemd v242 and later. (#4428, @wenyingd)
• Fix OpenFlow rules not being updated when Multi-cluster Gateway updates. (#4388, @luolanzone)
• Set no-flood config with ports for TrafficControl after Agent restarting. (#4318, @hongliangl)
• Fix packet resubmission issue when AntreaProxy is enabled and AntreaPolicy is disable. (#4261, @GraysonWu)
• Fix data race when Multi-cluster controller reconciles ServiceExports concurrently. (#4305, @Dyanngg)
• Fix multicast group not removed from cache when it is uninstalled. (#4176, @wenyingd)
• Fix nil pointer error when there is no ClusterSet found during MemberClusterAnnounce validation. (#4154, @luolanzone)
• Remove redundant Openflow messages when syncing an updated group to OVS. (#4160, @hongliangl)

Fixed

• Fix race conditions in NetworkPolicyController. (#4028, @tnqn)
• Ensure NO_FLOOD is always set for IPsec tunnel ports and TrafficControl ports. (#4419 #4654 #4674, @xliuxu @tnqn)
• Fix Service routes being deleted on Agent startup on Windows. (#4470, @hongliangl)
• Fix Agent crash in dual-stack clusters when any Node is not configured with an IP address for each address family. (#4480, @hongliangl)
• Fix route deletion for Service ClusterIP and LoadBalancerIP when AntreaProxy is enabled. (#4711, @tnqn)

• The EndpointSlice feature is graduated from Alpha to Beta and is therefore enabled by default.

Added

• Add the following capabilities to Antrea-native policies:
  • ClusterSet scoped policy rules now support with the namespaces field. (#4571, @Dyanngg)
  • Layer 7 policy rules now support traffic logging. (#4625, @qiyueyao)
  • The implementation of FQDN policy rules has been extended to process DNS packets over TCP. (#4612 #4732, @GraysonWu @tnqn)
• Add the following capabilities to the AntreaProxy feature:
  • Graduate EndpointSlice from Alpha to Beta; antrea-agent now listens to EndpointSlice events by default. (#4634, @hongliangl)
  • Support ProxyTerminatingEndpoints in AntreaProxy. (#4607, @hongliangl)
  • Support rejecting requests to Services without available Endpoints. (#4656, @hongliangl)
• Add the following capabilities to Egress policies:
  • Support limiting the number of Egress IPs that can be assigned to a Node via new configuration option egress.maxEgressIPsPerNode or Node annotation "node.antrea.io/max-egress-ips". (#4593 #4627, @tnqn)
  • Add antctl get memberlist CLI command to get memberlist state. (#4611, @Atish-iaf)
• Support "noEncap", "hybrid", and "networkPolicyOnly" in-cluster traffic encapsulation modes with Multi-cluster Gateway. (#4407, @luolanzone)
• Enhance CI to validate Antrea with Rancher clusters. (#4496, @jainpulkit22)

Changed

• Ensure cni folders are created when starting antrea-agent with containerd on Windows. (#4685, @XinShuYang)
• Decrease log verbosity value for antrea-agent specified in the Windows manifest for containerd from 4 to 0. (#4676, @XinShuYang)
• Bump up cni and plugins libraries to v1.1.1. (#4425, @wenyingd)
• Upgrade OVS version to 2.17.5. (#4742, @antoninbas)
• Extend the message length limitation in the Conditions of Antrea-native policies to 256 characters. (#4574, @wenyingd)
• Stop using ClusterFirstWithHostNet DNSPolicy for antrea-agent; revert it to the default value. (#4548, @antoninbas)
• Perform Service load balancing within OVS for Multi-cluster Service traffic, when the local member Service of the Multi-cluster Service is selected as the destination. (#4693, @luolanzone)
• Rename the multicluster.enable configuration parameter to multicluster.enableGateway. (#4533, @jianjuns)
• Add the multicluster.enablePodToPodConnectivity configuration parameter for antrea-agent to enable Multi-cluster Pod-to-Pod connectivity. (#4605, @hjiajing)
• No longer install Whereabouts CNI to host. (#4617, @jianjuns)
• Add an explicit Secret for the vm-agent ServiceAccount to the manifest for non-Kubernetes Nodes. (#4560, @wenyingd)
• Change the toService.scope field of Antrea ClusterNetworkPolicy to an enum. (#4562, @GraysonWu)

Fixed

• Fix route deletion for Service ClusterIP and LoadBalancerIP when AntreaProxy is enabled. (#4711, @tnqn)
• Fix Service routes being deleted on Agent startup on Windows. (#4470, @hongliangl)
• Avoid duplicate Node Results in Live Traceflow Status. (#4715, @antoninbas)
• Fix OpenFlow Group being reused with wrong type because groupDb cache was not cleaned up. (#4592, @ceclinux)
• Ensure NO_FLOOD is always set for IPsec tunnel ports and TrafficControl ports. (#4654 #4419, @xliuxu)
• Fix Agent crash in dual-stack clusters when any Node is not configured with an IP address for each address family. (#4480, @hongliangl)
• Fix antctl not being able to talk with GCP kube-apiserver due to missing platforms specific imports. (#4494, @luolanzone)

The main purpose of this pre-release is to validate the updated release workflow.

Added

• Add L7NetworkPolicy feature which enables users to protect their applications by specifying how they are allowed to communicate with others, taking into account application context. (#4380 #4406 #4410, @hongliangl @qiyueyao @tnqn)
  • Layer 7 NetworkPolicy can be configured through the l7Protocols field of Antrea-native policies.
  • Refer to this document for more information about this feature.
• Add SupportBundleCollection feature which enables a CRD API for Antrea to collect support bundle files on any K8s Node or ExternalNode, and upload to a user-defined file server. (#4184 #4338 #4249, @wenyingd @mengdie-song @ceclinux)
  • Refer to this document for more information about this feature.
• Add support for NetworkPolicy for cross-cluster traffic. (#4432 #3914, @Dyanngg @GraysonWu)
  • Setting scope of an ingress peer to clusterSet expands the scope of the podSelector or namespaceSelector to the entire ClusterSet.
  • Setting scope of toServices to clusterSet selects a Multi-cluster Service. (#4397, @Dyanngg)
  • Refer to this document for more information about this feature.
• Add the following capabilities to the ExternalNode feature:
  • Containerized option for antrea-agent installation on Linux VMs. (#4413, @Nithish555)
  • Support for RHEL 8.4. (#4323, @Nithish555)
• Add support for running antrea-agent as DaemonSet when using containerd as the runtime on Windows. (#4279, @XinShuYang)
• Add documentation for Antrea Multicast. (#4339, @ceclinux)

Changed

• Extend antctl mc get joinconfig to print member token Secret. (#4363, @jianjuns)
• Improve support for Egress in Traceflow. (#3926, @Atish-iaf)
• Add NodePortLocalPortRange field for AntreaAgentInfo. (#4379, @wenqiq)
• Use format "namespace/name" as the key for ExternalNode span calculation. (#4401, @wenyingd)
• Enclose Pod labels with single quotes when uploading CSV record to S3 in the FlowAggregator. (#4334, @dreamtalen)
• Upgrade Antrea base image to ubuntu 22.04. (#4459 #4499, @antoninbas)
• Update OVS to 2.17.3. (#4402, @mnaser)
• Reduce confusion caused by transient error encountered when creating static Tiers. (#4414, @tnqn)

Fixed

• Add a periodic job to rejoin dead Nodes, to fix Egress not working properly after long network downtime. (#4491, @tnqn)
• Fix potential deadlocks and memory leaks of memberlist maintenance in large-scale clusters. (#4469, @wenyingd)
• Fix connectivity issues caused by MAC address changes with systemd v242 and later. (#4428, @wenyingd)
• Fix error handling when S3Uploader partially succeeds. (#4433, @heanlan)
• Fix a ClusterInfo export bug when Multi-cluster Gateway changes. (#4412, @luolanzone)
• Fix OpenFlow rules not being updated when Multi-cluster Gateway updates. (#4388, @luolanzone)
• Delete Pod specific VF resource cache when a Pod gets deleted. (#4285, @arunvelayutham)
• Fix OpenAPI descriptions for AntreaAgentInfo and AntreaControllerInfo. (#4390, @tnqn)

Changed

• Upgrade Antrea base image to ubuntu 22.04. (#4459, @antoninbas)
• Add OFSwitch connection check to Agent's liveness probes. (#4126, @tnqn)
• Improve install_cni_chaining to support updates to CNI config file. (#4012, @antoninbas)

Fixed

• Add a periodic job to rejoin dead Nodes to fix Egress not working properly after long network downtime. (#4491, @tnqn)
• Fix connectivity issues caused by MAC address changes with systemd v242 and later. (#4428, @wenyingd)
• Fix potential deadlocks and memory leaks of memberlist maintenance in large-scale clusters. (#4469, @wenyingd)
• Fix Windows AddNodePort parameter error. (#4103, @XinShuYang)
• Set no-flood config with ports for TrafficControl after Agent restarting. (#4318, @hongliangl)
• Fix multicast group not removed from cache when it is uninstalled. (#4176, @wenyingd)
• Remove redundant Openflow messages when syncing an updated group to OVS. (#4160, @hongliangl)
• Fix Antrea Octant plugin build. (#4107, @antoninbas)

Added

• Add the following capabilities to the Multi-cluster feature:
  • Add support for Pod-to-Pod connectivity across clusters. (#4219, @hjiajing)
  • Add active-passive mode high availability support for Gateway Nodes. (#4069, @luolanzone)
  • Allow Pod IPs as Endpoints of Multi-cluster Service; option endpointIPType is added to the Multi-cluster Controller ConfigMap to specify the Service Endpoints type. (#4198, @luolanzone)
  • Add antctl mc get joinconfig command to print ClusterSet join parameters. (#4299, @jianjuns)
  • Add antctl mc get|delete membertoken commands to get/delete member token. (#4254, @bangqipropel)
• Add rule name to Audit Logging for Antrea-native policies. (#4178, @qiyueyao)
• Add Service health check similar to kube-proxy in antrea-agent; it provides HTTP endpoints <nodeIP>:<healthCheckNodePort>/healthz for querying number of local Endpoints of a Service. (#4120, @shettyg)
• Add S3Uploader as a new exporter of Flow Aggregator, which periodically exports expired flow records to AWS S3 storage bucket. (#4143, @heanlan)
• Add scripts and binaries needed for running Antrea on non-Kubernetes Nodes (ExternalNode) in release assets. (#4266 #4113, @antoninbas @Anandkumar26)

Changed

• AntreaProxy now supports more than 800 Endpoints for a Service. (#4167, @hongliangl)
• Add OVS connection check to Agent's liveness probes for self-healing on OVS disconnection. (#4126, @tnqn)
• antrea-agent startup scripts now perform cleanup automatically on non-Kubernetes Nodes (ExternalNode) upon Node restart. (#4277, @Anandkumar26)
• Make tunnel csum option configurable and default to false which avoids double encapsulation checksum issues on some platforms. (#4250, @tnqn)
• Use standard value type for k8s.v1.cni.cncf.io/networks annotation for the SecondaryNetwork feature. (#4146, @antoninbas)
• Update Go to v1.19. (#4106, @antoninbas)
• Add API support for reporting Antrea NetworkPolicy realization failure. (#4248, @wenyingd)
• Update ResourceExport's json tag to lowerCamelCase. (#4211, @luolanzone)
• Add clusterUUID column to S3 uploader and ClickHouseExporter to support multiple clusters in the same data warehouse. (#4214, @heanlan)

Fixed

• Fix nil pointer error when collecting support bundle from Agent fails. (#4306, @tnqn)
• Set no-flood config for TrafficControl ports after restarting Agent to prevent ARP packet loops. (#4318, @hongliangl)
• Fix packet resubmission issue when AntreaProxy is enabled and AntreaPolicy is disable. (#4261, @GraysonWu)
• Fix ownerReferences in APIExternalEntities generated from ExternalNodes. (#4259, @wenyingd)
• Fix the issue that "MulticastGroup" API returned wrong Pods that have joined multicast groups. (#4240, @ceclinux)
• Fix inappropriate route for IPv6 ClusterIPs in the host network when proxyAll is enabled. (#4297, @tnqn)
• Fix log spam when there is any DNS based LoadBalancer Service. (#4234, @tnqn)
• Remove multicast group from cache when group is uninstalled. (#4176, @wenyingd)
• Remove redundant Openflow messages when syncing an updated group to OVS. (#4160, @hongliangl)
• Fix nil pointer error when there is no ClusterSet found during MemberClusterAnnounce validation. (#4154, @luolanzone)
• Fix data race when Multi-cluster controller reconciles ServiceExports concurrently. (#4305, @Dyanngg)
• Fix memory leak in Multi-cluster resource import controllers. (#4251, @Dyanngg)
• Fix Antrea-native policies for multicast traffic matching IGMP traffic unexpectedly. (#4206, @liu4480)
• Fix IPsec not working in UBI-based image. (#4244, @xliuxu)
• Fix antctl mc get clusterset command output when a ClusterSet's status is empty. (#4174, @luolanzone)

Added

• Add ExternalNode feature which enables Antrea to manage security policies for non-Kubernetes Nodes (like virtual machines or bare-metal servers). (#4110, @wenyingd @mengdie-song @Anandkumar26)
  • It introduces the ExternalNode CRD; each resource of this kind represents a virtual machine or bare-metal server and supports specifying which network interfaces on the external Node are expected to be protected with Antrea-native policies.
  • An ExternalEntity resource will be created for each network interface specified in the ExternalNode resource. Antrea-native policies are applied to an external Node by using the ExternalEntity selector.
  • Refer to this document for more information about this feature.
• Add the following capabilities to Antrea-native policies:
  • Add Audit Logging support for K8s Networkpolicy. (#4047, @qiyueyao)
  • Support applying Antrea ClusterNetworkPolicy to NodePort Services for securing ingress traffic. (#3997, @GraysonWu)
  • Introduce the Group CRD to logically group different network endpoints and reference them together in Antrea NetworkPolicy. (#2438, @qiyueyao @abhiraut)
• Release new Antrea Helm chart version for each Antrea release. (#3935 #3952, @antoninbas @yanjunz97)
  • Refer to this document for Helm installation method. (#3989, @antoninbas)
• Support TopologyAwareHints in AntreaProxy. (#3515, @hongliangl)
• Add encap mode support for the Multicast feature. (#3947, @wenyingd)
• Support configurable Geneve, VXLAN, or STT port number for encap mode. (#4065, @Jexf)
• Add Status field to the IPPool CRD: it is used to report usage information for the pool (total number of IPs in the pool and number of IPs that are currently assigned). (#3072 #4088, @ksamoray @tnqn)
• Support updating configuration at runtime for flow-aggregator via antctl or by updating the ConfigMap. (#3642, @yuntanghsu)
• Add antctl commands to set up and delete Multi-cluster ClusterSet. (#3992, @hjiajing)
• Add documentation to set up Multi-cluster ClusterSet with antctl. (#4096, @jianjuns)

Changed

• Antrea now uses OpenFlow 1.5 to program OVS. (#3770, @wenyingd @ashish-varma)
• Rename Windows script Start.ps1 to Start-AntreaAgent.ps1, and rename Stop.ps1 to Stop-AntreaAgent.ps1. (#3904, @wenyingd)
• Unify NodePortLocal behavior across Linux and Windows. Linux agents now support allocating different Node ports for different protocols even when the Pod port number is the same. (#3936, @XinShuYang)
• Antrea IPAM now uses the name of the uplink interface to name the host internal port, and the uplink interface will be renamed with a ~ suffix, e.g. eth0~. (#3938, @gran-vmv)
• Send Neighbor Advertisement messages after creating Pods in an IPv6 cluster. (#3998, @gran-vmv)
• Add an output formatter "raw" to better display multi-line string responses for antctl. (#3589, @Atish-iaf)
• Add new ports to network requirement doc. (#4063, @luolanzone)
• Windows OVS installation script now installs required SSL library if missing. (#4029, @XinShuYang)
• Upgrade whereabouts CNI to v0.5.4 and provide required pluginArgs when invoking the CNI binary. (#3987, @arunvelayutham)
• Remove Grafana flow collector files in the Antrea repo (as they were moved to the Theia repo). (#4048, @dreamtalen)
• Make the following changes to the Multi-cluster feature:
  • Add columns of kubectl outputs for Multi-cluster custom resources. (#3923, @jianjuns)
  • Use hostNetwork for Multi-cluster controller. (#3965, @luolanzone)
  • Update ClusterClaim CRD to v1alpha2. (#3755, @bangqipropel)
  • Update GatewayIPPrecedence to support the "external/internal" options. (#3930, @luolanzone)
  • Disable metrics API and change the health binding address port to 8080. (#4101, @luolanzone)
  • Improve CRD validation. (#4062 #4090 #4043, @luolanzone)
  • Auto create MemberClusterAnnounce and update ClusterSet in leader cluster for each member cluster. (#3956 #4054 #4026, @hjiajing @luolanzone)
  • Add Multi-cluster Gateway descriptions in the Multi-cluster architecture document. (#3638 #3899, @luolanzone @jianjuns)

Fixed

• Fix reconnection issue between Agent and OVS. (#4091, @wenyingd)
• Fix the wrong DNAT IP used by AntreaProxy for serving NodePort traffic on Windows Nodes. (#4103, @XinShuYang)
• Fix Antrea Octant plugin build. (#4107, @antoninbas)
• Fix Pod-to-external traffic on EKS in policyOnly mode. (#3975, @antoninbas)
• Fix problems caused by Node restart on EKS in policyOnly mode. (#4012 #4042, @antoninbas)
• Fix race conditions in NetworkPolicyController. (#4028, @tnqn)
• Fix FlowExporter memory bloat when export process is dead. (#3994, @wsquan171)
• Fix socket leak in an IPv6 cluster. (#4104, @wenyingd)
• Fix ClickHouse client race during batch commit. (#4071, @wsquan171)
• Retry when retrieval of PodCIDRs fails to avoid Agent crash due to the delay in allocating PodCIDRs for the Node. (#3950, @ksamoray)
• Fix nil pointer issue when ClusterSet is deleted in leader cluster. (#3915, @luolanzone)
• Clean up ResourceExport if the exported Service has no available Endpoints. (#4056, @luolanzone)

Fixed

• Fix FlowExporter memory bloat when export process is dead. (#3994, @wsquan171)
• Fix Pod-to-external traffic on EKS in policyOnly mode. (#3975, @antoninbas)
• Use uplink interface name for host interface internal port to support DHCP client. (#3938, @gran-vmv)

The main purpose of this pre-release is to validate Antrea Helm chart releases.

The main purpose of this pre-release is to validate Antrea Helm chart releases.

Added

• Add TrafficControl feature to control the transmission of Pod traffic; it allows users to mirror or redirect traffic originating from specific Pods or destined for specific Pods to a local network device or a remote destination via a tunnel of various types. (#3644 #3580 #3487, @tnqn @hongliangl @wenqiq)
  • Refer to this document for more information about this feature.
  • Refer to this cookbook for more information about using this feature to provide network-based intrusion detection service to your Pods.
• Add support for the IPsec Certificate-based Authentication. (#3778, @xliuxu)
  • Add an Antrea Agent configuration option ipsec.authenticationMode to specify authentication mode. Supported options are "psk" (default) and "cert".
  • Add an Antrea Controller configuration option ipsecCSRSigner.autoApprove to specify the auto-approve policy of Antrea CSR signer for IPsec certificates management. By default, Antrea will auto-approve the CertificateSingingRequest (CSR) if it is verified.
  • Add an Antrea Controller configuration option ipsecCSRSigner.selfSignedCA to specify whether to use auto-generated self-signed CA certificate. By default, Antrea will auto-generate a self-signed CA certificate.
• Add the following capabilities to Antrea-native policies:
  • Add support for matching ICMP traffic. (#3472, @GraysonWu)
  • Add support for matching multicast and IGMP traffic. (#3660, @liu4480)
  • Add support for rule-level statistics for multicast and IGMP traffic. (#3449, @ceclinux)
• Add the following capabilities to the Multicast feature:
  • Add antctl get podmulticaststats command to query Pod-level multicast traffic statistics in Agent mode. (#3449, @ceclinux)
  • Add "MulticastGroup" API to query Pods that have joined multicast groups; kubectl get multicastgroups can generate requests and output responses of the API. (#3354 #3449, @ceclinux)
  • Add an Antrea Agent configuration option multicast.igmpQueryInterval to specify the interval at which the antrea-agent sends IGMP queries to Pods. (#3819, @liu4480)
• Add the following capabilities to the Multi-cluster feature:
  • Add the Multi-cluster Gateway functionality which supports routing Multi-cluster Service traffic across clusters through tunnels between the Gateway Nodes. It enables Multi-cluster Service access across clusters, without requiring direct reachability of Pod IPs between clusters. (#3689 #3463 #3603, @luolanzone)
  • Add a number of antctl mc subcommands for bootstrapping Multi-cluster; refer to the Multi-cluster antct document for more information. (#3474, @hjiajing)
• Add the following capabilities to secondary network IPAM:
  • Add support for IPAM for Pod secondary networks managed by Multus. (#3529, @jianjuns)
  • Add support for multiple IPPools. (#3606, @jianjuns)
  • Add support for static addresses. (#3633, @jianjuns)
• Add support for NodePortLocal on Windows. (#3453, @XinShuYang)
• Add support for Traceflow on Windows. (#3022, @gran-vmv)
• Add support for containerd to antrea-eks-node-init.yml. (#3840, @antoninbas)
• Add an Antrea Agent configuration option disableTXChecksumOffload to support cases in which the datapath's TX checksum offloading does not work properly. (#3832, @tnqn)
• Add support for InternalTrafficPolicy in AntreaProxy. (#2792, @hongliangl)
• Add the following documentations:
  • Add documentation for the Antrea Agent RBAC permissions and how to restrict them using Gatekeeper/OPA. (#3694, @antoninbas)
  • Add quick start guide for Antrea Multi-cluster. (#3853, @luolanzone @jianjuns)
  • Add documentation for the AntreaProxy feature. (#3679, @antoninbas)
  • Add documentation for secondary network IPAM. (#3634, @jianjuns)

Changed

• Optimize generic traffic performance by reducing OVS packet recirculation. (#3858, @tnqn)
• Optimize NodePort traffic performance by reducing OVS packet recirculation. (#3862, @hongliangl)
• Improve validation for IPPool CRD. (#3570, @jianjuns)
• Improve validation for egress.to.namespaces.match of AntreaClusterNetworkPolicy rules. (#3727, @qiyueyao)
• Deprecate the Antrea Agent configuration option multicastInterfaces in favor of multicast.multicastInterfaces. (#3898, @tnqn)
• Reduce permissions of Antrea Agent ServiceAccount. (#3691, @xliuxu)
• Create a Secret in the Antrea manifest for the antctl and antrea-agent ServiceAccount as K8s v1.24 no longer creates a token for each ServiceAccount automatically. (#3730, @antoninbas)
• Implement garbage collector for IP Pools to clean up allocations and reservations for which owner no longer exists. (#3672, @annakhm)
• Preserve client IP if the selected Endpoint is local regardless of ExternalTrafficPolicy. (#3604, @hongliangl)
• Add a Helm chart for Antrea and use the Helm templates to generate the standard Antrea YAML manifests. (#3578, @antoninbas)
• Make "Agent mode" antctl work out-of-the-box on Windows. (#3645, @antoninbas)
• Truncate SessionAffinity timeout values of Services instead of wrapping around. (#3609, @antoninbas)
• Move Antrea Windows log dir from C:\k\antrea\logs\ to C:\var\log\antrea\. (#3416, @GraysonWu)
• Limit max number of data values displayed on Grafana panels. (#3812, @heanlan)
• Support deploying ClickHouse with Persistent Volume. (#3608, @yanjunz97)
• Remove support for ELK Flow Collector. (#3738, @heanlan)
• Improve documentation for Antrea-native policies. (#3512, @Dyanngg)
• Update OVS version to 2.17.0. (#3591, @antoninbas)

Fixed

• Fix Egress not working with kube-proxy IPVS strictARP mode. (#3837, @xliuxu)
• Fix intra-Node Pod traffic bypassing Ingress NetworkPolicies in some scenarios. (#3809, @hongliangl)
• Fix FQDN policy support for IPv6. (#3869, @tnqn)
• Fix multicast not working if the AntreaPolicy feature is disabled. (#3807, @liu4480)
• Fix tolerations for Pods running on control-plane for Kubernetes >= 1.24. (#3731, @xliuxu)
• Fix DNS resolution error of antrea-agent on AKS by using ClusterFirst dnsPolicy. (#3701, @tnqn)
• Clean up stale routes installed by AntreaProxy when ProxyAll is disabled. (#3465, @hongliangl)
• Ensure that Service traffic does not bypass NetworkPolicies when ProxyAll is enabled on Windows. (#3510, @hongliangl)
• Use IP and MAC to find virtual management adapter to fix Agent crash in some scenarios on Windows. (#3641, @wenyingd)
• Fix handling of the "reject" packets generated by the Antrea Agent to avoid infinite looping. (#3569, @GraysonWu)
• Fix export/import of Services with named ports when using the Antrea Multi-cluster feature. (#3561, @luolanzone)
• Fix Multi-cluster importer not working after leader controller restarts. (#3596, @luolanzone)
• Fix Endpoint ResourceExports not cleaned up after corresponding Service is deleted. (#3652, @luolanzone)
• Fix pool CRD format in egress.md and service-loadbalancer.md. (#3885, @jianjuns)
• Fix infinite looping when Agent tries to delete a non-existing route. (#3827, @hongliangl)
• Fix race condition in ConntrackConnectionStore and FlowExporter. (#3655, @heanlan)

Fixed

• Fix export/import of Services with named ports when using the Antrea Multi-cluster feature. (#3561, @luolanzone)
• Fix handling of the "reject" packets generated by the Antrea Agent to avoid infinite looping. (#3569, @GraysonWu)
• Fix DNS resolution error of Antrea Agent on AKS by using ClusterFirst dnsPolicy. (#3701, @tnqn)
• Fix tolerations for Pods running on control-plane for Kubernetes >= 1.24. (#3731, @xliuxu)
• Reduce permissions of Antrea Agent ServiceAccount. (#3691, @xliuxu)

Added

• Add documentation for the Antrea Agent RBAC permissions and how to restrict them using Gatekeeper/OPA. (#3694, @antoninbas)

Fixed

• Clean up stale routes installed by AntreaProxy when ProxyAll is disabled. (#3465, @hongliangl)
• Fix export/import of Services with named ports when using the Antrea Multi-cluster feature. (#3561, @luolanzone)
• Fix handling of the "reject" packets generated by the Antrea Agent to avoid infinite looping. (#3569, @GraysonWu)
• Fix DNS resolution error of Antrea Agent on AKS by using ClusterFirst dnsPolicy. (#3701, @tnqn)
• Fix tolerations for Pods running on control-plane for Kubernetes >= 1.24. (#3731, @xliuxu)
• Reduce permissions of Antrea Agent ServiceAccount. (#3691, @xliuxu)
• [Windows] Ensure that Service traffic does not bypass NetworkPolicies when ProxyAll is enabled. (#3510, @hongliangl)
• Fix Antrea wildcard FQDN NetworkPolicies not working when NodeLocal DNSCache is enabled. (#3510, @hongliangl)

Changed

• Use iptables-wrapper in Antrea container. Now antrea-agent can work with distros that lack the iptables kernel module of "legacy" mode (ip_tables). (#3276, @antoninbas)
• Reduce permissions of Antrea ServiceAccount for updating annotations. (#3393, @tnqn)
• [Windows] Use uplink MAC as source MAC when transmitting packets to underlay network from Windows Nodes. Therefore, MAC address spoofing configuration like "Forged transmits" in VMware vSphere doesn't need to be enabled. (#3516, @wenyingd)

Fixed

• Fix DNS resolution error of antrea-agent on AKS by using ClusterFirst dnsPolicy. (#3701, @tnqn)
• Fix status report of Antrea-native policies with multiple rules that have different AppliedTo. (#3074, @tnqn)
• Upgrade Go version to 1.17 to pick up security fix for CVE-2021-44716. (#3189, @antoninbas)
• Fix NetworkPolicy resources dump for Agent's supportbundle. (#3083, @antoninbas)
• Fix gateway interface MTU configuration error on Windows. (#3043, @lzhecheng) [Windows]
• Fix initialization error of antrea-agent on Windows by specifying hostname explicitly in VMSwitch commands. (#3169, @XinShuYang) [Windows]
• Ensure that the Windows Node name obtained from the environment or from hostname is converted to lower-case. (#2672, @shettyg) [Windows]
• Fix typos in the example YAML in antrea-network-policy doc. (#3079 #3092, @antoninbas @Jexf)
• Fix ipBlock referenced in nested ClusterGroup not processed correctly. (#3383, @Dyanngg)
• Fix NetworkPolicy may not be enforced correctly after restarting a Node. (#3467, @tnqn)
• Fix antrea-agent crash caused by interface detection in AKS/EKS with NetworkPolicyOnly mode. (#3219, @wenyingd)
• Fix locally generated packets from Node net namespace might be SNATed mistakenly when Egress is enabled. (#3430, @tnqn)

• The Egress feature is graduated from Alpha to Beta and is therefore enabled by default.
• The support for proxying all Service traffic by Antrea Proxy (enabled by antreaProxy.proxyAll) is now Beta.

Added

• Add the following capabilities to the Antrea IPAM feature:
  • Support pre-allocating continuous IPs for StatefulSet. (#3281, @annakhm)
  • Support specifying VLAN for IPPool. Traffic from Pods whose IPPools are configured with a VLAN ID will be tagged when leaving the Node uplink. (#3247, @gran-vmv)
• Add the following capabilities to the Antrea Multi-cluster feature:
  • Support Antrea ClusterNetworkPolicy replication in a ClusterSet. (#3363, @Dyanngg)
  • Add antctl mc subcommand for Antrea Multi-cluster resources. (#3341 #3287, @hjiajing @bangqipropel)
• Add the following capabilities to the AntreaPolicy feature:
  • Add Node selector in Antrea-native policies to allow matching traffic originating from specific Nodes or destined to specific Nodes. (#3038, @wenqiq)
  • Add ServiceAccount selector in Antrea-native policies to allow selecting Pods by ServiceAccount. (#3044, @GraysonWu)
  • Support Pagination for ClusterGroupMembership API. (#3183, @qiyueyao)
  • Add Port Number to Audit Logging. (#3277, @qiyueyao)
• [Flow Visibility] Add Grafana Flow Collector as the new visualization tool for flow records.
  • Add Grafana dashboards, Clickhouse data schema, deployment files, and doc. (#3063 #3525, @heanlan @zyiou @dreamtalen)
  • Add support for exporting flow records to ClickHouse from Flow Aggregator. (#3196 #3526, @wsquan171 @dreamtalen)
  • Add ClickHouse monitor to ensure data retention for in-memory ClickHouse deployment. (#3244 #3498, @yanjunz97)
• [Multicast] Support IGMPv3 leave action. (#3389, @wenyingd)
• [Windows] Add support for EndpointSlices on Windows Nodes. (#3321, @XinShuYang)
• Add SKIP_CNI_BINARIES environment variable to support skipping the installation of specified CNI plugins. (#3454, @jainpulkit22)
• Support UBI8-based container image to run Antrea. (#3273, @ksamoray)
• Add the following documentations:
  • Add documentation for ServiceExternalIP feature and Service of type LoadBalancer. (#3322, @hty690)
  • Add documentation for deploying Antrea to Minikube cluster. (#3391, @jainpulkit22)
  • Add documentation for antctl Multi-cluster commands. (#3414, @bangqipropel)
  • Add documentation for Multiple-VLAN support. (#3507, @gran-vmv)
  • Add upgrade guide for Multi-cluster. (#3374, @luolanzone)
  • Add a per-rule example for NetworkpolicyStats docs. (#3356, @ceclinux)

Changed

• Remove all legacy (*.antrea.tanzu.vmware.com) APIs. (#3299, @antoninbas)
• Remove Kind-specific manifest and scripts. Antrea now uses OVS kernel datapath for Kind clusters. (#3413, @antoninbas)
• [Windows] Use uplink MAC as source MAC when transmitting packets to underlay network from Windows Nodes. Therefore, MAC address spoofing configuration like "Forged transmits" in VMware vSphere doesn't need to be enabled. (#3516, @wenyingd)
• Add an agent config parameter "enableBridgingMode" for enabling flexible IPAM (bridging mode). (#3297 #3365, @jianjuns)
• Use iptables-wrapper in Antrea container to support distros that runs iptables in "nft" mode. (#3276, @antoninbas)
• Install CNI configuration files after installing CNI binaries to support container runtime cri-o. (#3154, @tnqn)
• Upgrade packaged Whereabouts version to v0.5.1. (#3511, @antoninbas)
• Upgrade to go-ipfix v0.5.12. (#3352, @yanjunz97)
• Upgrade Kustomize from v3.8.8 to v4.4.1 to fix Cronjob patching bugs. (#3402, @yanjunz97)
• Fail in Agent initialization if GRE tunnel type is used with IPv6. (#3156, @antoninbas)
• Refactor the OpenFlow pipeline for future extensibility. (#3058, @hongliangl)
• Validate IP ranges of IPPool for Antrea IPAM. (#2995, @ksamoray)
• Validate protocol in the CRD schema of Antrea-native policies. (#3342, @KMAnju-2021)
• Validate labels in the CRD schema of Antrea-native policies and ClusterGroup. (#3331, @GraysonWu)
• Reduce permissions of Antrea ServiceAccounts. (#3393, @tnqn)
• Remove --k8s-1.15 flag from hack/generate-manifest.sh. (#3350, @antoninbas)
• Remove unnecessary CRDs and RBAC rules from Multi-cluster manifest. (#3491, @luolanzone)
• Update label and image repo of antrea-mc-controller to be consistent with antrea-controller and antrea-agent. (#3266 #3466, @luolanzone)
• Add clusterID annotation to ServiceExport/Import resources. (#3359, @luolanzone)
• Do not log error when Service for Endpoints is not found to avoid log spam. (#3256, @tnqn)
• Ignore Services of type ExternalName for NodePortLocal feature. (#3114, @antoninbas)
• Add powershell command replacement in the Antrea Windows documentation. (#3264, @GraysonWu)

Fixed

• Add userspace ARP/NDP responders to fix Egress and ServiceExternalIP support for IPv6 clusters. (#3318, @hty690)
• Fix incorrect results by antctl get networkpolicy when both Pod and Namespace are specified. (#3499, @Dyanngg)
• Fix IP leak issue when AntreaIPAM is enabled. (#3314, @gran-vmv)
• Fix error when dumping OVS flows for a NetworkPolicy via antctl get ovsflows. (#3335, @jainpulkit22)
• Fix IPsec encryption for IPv6 overlays. (#3155, @antoninbas)
• Add ignored interfaces names when getting interface by IP to fix NetworkPolicyOnly mode in AKE. (#3219, @wenyingd)
• Fix duplicate IP case for NetworkPolicy. (#3467, @tnqn)
• Don't delete the routes which are added for the peer IPv6 gateways on Agent startup. (#3336 #3490, @Jexf @xliuxu)
• Fix pkt mark conflict between HostLocalSourceMark and SNATIPMark. (#3430, @tnqn)
• Unconditionally sync CA cert for Controller webhooks to fix Egress support when AntreaPolicy is disabled. (#3421, @antoninbas)
• Fix inability to access NodePort in particular cases. (#3371, @hongliangl)
• Fix ipBlocks referenced in nested ClusterGroup not processed correctly. (#3383, @Dyanngg)
• Realize Egress for a Pod as soon as its network is created. (#3360, @tnqn)
• Fix NodePort/LoadBalancer issue when proxyAll is enabled. (#3295, @hongliangl)
• Do not panic when processing a PacketIn message for a denied connection. (#3447, @antoninbas)
• Fix CT mark matching without range in flow exporter. (#3348, @hongliangl)
• [Windows] Enable IP forwarding of the Windows bridge local interface to fix support for Service of type LoadBalancer. (#3137, @hongliangl)

Fixed

• Fix NetworkPolicy may not be enforced correctly after restarting a Node. (#3467, @tnqn)
• Fix antrea-agent crash caused by interface detection in AKS/EKS with NetworkPolicyOnly mode. (#3219, @wenyingd)
• Fix locally generated packets from Node net namespace might be SNATed mistakenly when Egress is enabled. (#3430, @tnqn)

Changed

• Use iptables-wrapper in Antrea container. Now antrea-agent can work with distros that lack the iptables kernel module of "legacy" mode (ip_tables). (#3308, @antoninbas)
• Reduce permissions of Antrea ServiceAccount for updating annotations. (#3408, @tnqn)

Fixed

• Fix NodePort/LoadBalancer Service cannot be accessed when externalTrafficPolicy changed from Cluster to Local with proxyAll enabled. (#3330, @hongliangl)
• Fix initial egress connections from Pods may go out with node IP rather than Egress IP. (#3378, @tnqn)
• Fix NodePort Service access when an Egress selects the same Pod as the NodePort Service. (#3397, @hongliangl)
• Fix ipBlock referenced in nested ClusterGroup not processed correctly. (#3405, @Dyanngg)