kube-router

Bug fix release. Fixes for regressions found in v1.0.0-rc5

thanks @eeeeeta for reporting and fixing the regression

Breaking changes and knows issues:

If you are upgrading from v1.0.0-rc4 or earlier version please see release notes for v1.0.0-rc5. Same breaking changes and known issues apply for this release as well

Changelog

4f9a7949 Merge pull request #931 from cloudnativelabs/pr914-feedback
1bec864a avoide listing a chain if the rule already exists
309c8032 Merge pull request #928 from eeeeeta/fix-generate-fwmark
a2ac2f00 fix unintentional Sprint of two-argument generateFwmark() call
a23017d5 Merge pull request #927 from cloudnativelabs/bgppolicies
81d717d9 fix false negative errors in creating BGP defined sets

This release has serveral improvements to network policies implementation in kube-router and cleanup of code base to fix all go lint errors and few bug fixes.

Thanks @mrueg @aauren @liuxu623 for your the PR's.

Thanks @aauren for reporting regression in v1.0.0-rc4 and critical feedback on network policy implementation some of which are addressed in this release.

Note: This release has following breaking changes:

• The way network policies are configured using iptables on the nodes has been modified to keep built in chains cleaner. You need to flush the iptables filter table or reboot the node before running this version of kube-router
• if you have egress network policies applied to workload, you need to ensure proper value for service-cluster-ip-range and service-node-port-range configured to ensure pod's can access service cluster IP's and NodePort services

Note: This release has following known issues:

• please see #934

Changelog

e858e262 change ACCEPT to RETURN with mark when a netpol is matched so that we run through (#915)
4d6b0b81 whitelist traffic to cluster IP and node ports in INPUT chain to bypass netwrok policy enforcement (#914)
210dc3d5 avoids adding kube-router specific rules to enforce network policies in (#909)
8f5c9594 full sync when namespace labels change (#917)
12674d5f Add golangci-lint support (#895)
4a08e118 Dockerfile: Update to alpine:3.11 (#918)
cb48a7f8 fix(network_routes): missing node ip -> error log (#904)
d2178da5 fix(ecmp_vip): check for nil nodename (#903)

Thanks to @aauren for your contributions!

Changelog

837554bf Fix Memory Consumption in network_policy_controller (#902)

Changelog

#900 - Fix Network Policy Cleanup Code
#894 - .goreleaser.yml: Multiarch build
#898 - Use same image for container and initContainer

Thanks @mrueg & @cfrantsen for your contributions!

We are excited to release rc2 candidate for v1.0 release.

thanks @rmb938 @CertainLach @mrueg @ufou @ldx @bumyongchoi @filintod @aauren @paulbsch for your contribution

Changelog

f695c752 Merge pull request #892 from cloudnativelabs/proxy-healtchecks
e04ac66f ensure hearbeats are sent during sync done for add/delete/update events of service, endpoints
361d6feb outbound traffic from pod should be intercepted in filter table INPUT chain (#891)
df40aa5e push multi-arch images to the dev registry (#890)
1af329c4 nflog the packet that will be dropped by network policy enforcement (#889)
f3ea1a6f Merge pull request #888 from CertainLach/master
86ebd286 Fix for same issue as #750, but for network_routes_controller
21ea5a5a Add multi-arch support for container images. (#885)
24621375 .travis.yml: Update manifest-tool to 1.0.2 (#886)
ea4f2db6 Merge pull request #747 from ufou/enable_bgp_restart_default
6640c65f mount host /run/xtables.lock to kube-router container which will be (#884)
ffad3388 Handle missing routing tables (#865)
f5db29e3 honor the ClientIP session affinity timeout when set. (#882)
7777b9a8 use Spec.PolicyTypes for the type of network policy (#883)
0f21f87f withdraw external IP from advertisement only if the deleted service is the last service using external IP (#850)
3e671595 Update selectors to allow matchexpressions as well as matlabels (#881)
b5e9bd30 intercept pod egress traffic going through the OUTPUT chain of filter table and run through the (#875)
4c764f54 handle DeletedFinalStateUnknown objects in DeleteFunc handlers (#856)
19e56370 switch --set to less ambiguous --match-set (#874)
2c4911b9 Fix unit test failure due to switch of listing node API objects from (#869)
d8382537 Add Numberly to USERS.md (#867)
33724aac read the necessary API objects from local cache instead of listing from the API server (#864)
5c5dc411 add Globo.com to USERS.md (#858)
945a8ca9 Update USERS.md (#857)
3b9f22b4 add enix as user (#855)
c857f5d4 add DigitalOcean to USERS.md (#852)
97ec4dda adding kube-router users list (#851)
0857436e use endpoint (IP, port) tuple to track active endpoints of a service in use. Currently only endpoint IP (#842)
4f627bc8 Enable ppc64le builds (#847)
8f0bcfb2 Enabling --bgp-graceful-restart by default when the router component is deployed via daemonset

Note: Please note behaviour change that is introduced by 13421da5. Functionally service proxy will remain same but kube-router now will internally use SNAT instead of MASQUERADE

Changelog

9db9a498 populate pod CID in network routing controler to simulate reading from node spec once at begining (#844)
148736b3 fix gofmt
459e52eb fix unhealthy on api server down (#813)
97c682e6 Ignore deletion of unknown IPVS rules (#830)
13421da5 Use SNAT instead of MASQUERADE to source NAT outbound IPVS traffic (#668)

Changelog

230ff155 restrict externalTrafficPolicy=Local interpretation only to NodePort and LoadBalancer services (#836)

Changelog

5671c3ab fix .goreleaser.yml (#837)
53e0571c fix broken CI (#823)
f01a9a57 Revert "restrict externalTrafficPolicy=Local interpretation only to NodePort and LoadBalancer services (#819)" (#835)
27ec314e restrict externalTrafficPolicy=Local interpretation only to NodePort and LoadBalancer services (#819)
c160e907 [FIX] Don't ignore silently service proxy errors. (#796)
8bcd166c Fix connection resets during firewall sync (#807)
3a0da2bf fix build break due to commit 05d03e76861ab677e2edf12a4ce887af730c1e8b (#817)
52e338d8 Add PriorityClass and docs update (#816)
05d03e76 #797 Conditionally disable "Allow All" input/chain on IPVS KUBE-ROUTER-SERVICES (#809)
ff6a024d set cniVersion in 10-kuberouter.conf (#811)
a339d8a5 remove stale project sponsorships (#805)

Changelog

d6f9f31a Fix: Send BGP Withdrawals for Service VIPs Upon Service Deletion (#756)
3aacd488 fix clusteripprefixset import policy (#771)
803bd902 Allow setting the BGP graceful restart deferral time. See RFC4724 4.1 (#753)
b54b80cb update to apps/v1 and add selector (#759)
4afd6d6d Updated the kube-proxy cleanup command to use the newer version (#762)
94fd7b6d Send heartbeats during NetworkPolicy and NetworkService sync. (#741)
6470795d Use x/sys/unix epoll (#737)
8fe9f70d Add Import Policy for Service VIPs (#721)
4be51ba1 First stab at pushing multiarch releases (#735)

WARNING: this release has a regression where service VIPs may not be advertised properly. Please use v0.4.0-rc1 instead.

Changelog

42a046b5 Send heartbeats during NetworkPolicy and NetworkService sync. (#741)

Bug fix release. Fixes regression introduced in 0.3 while adding support for --overlay-type=fulll/subnet

Changelog

d6a93d44 handle null *route (#732)
8bb50d5a do not setup ipip tunneling when --enable-overlay is false (#722)

Changelog

New Features

54eedcd0 Issue 572 - Graceful termination + Update to go-1.10.8, alpine-3.9 (#706)
8f9729a0 Introduces the option --overlay-type={subnet,full}, to be able to always generate IPIP tunnels regardless of node subnets (#666)
736757d9 Support named port of network policy (#679)

Bug Fixes

7181d6fa Prefer node PodCIDR from an annotation (#720)
e2301761 docs,pkgs: change 'can not' to 'cannot' (#701)
c2f893f6 default cni config to list format (#690)
375ccc27 Minor typo fix in logs (#700)
fac06635 add Jimmy to maintainers (#687)
70969a3a Add iptables rules for accessing tunneled services from node (#682)
42997cb9 Delete iptables rule if --masquerade-all is false (#665)
961d8ab8 fix #639 (#670)
7b20ae9f document workaround for cloudnativelabs/kube-router#676 (#677)
a63d386a fix typo (#673)
ffc37f77 remove gitter and add slack as community forum
00824cd8 Fix typo (#661)
52127e6c Fix ’make test’ when GOPATH contains multiple paths (#658)
34666a15 Run ‘go generate’ from %_moq.go target in docker when BUILD_IN_DOCKER is true (#660)

This release has two security fixes (#648, #649) and other small fixes

Thanks to all the contributors.

Changelog

18769938 Removes IPv6 address insertion into BGP IPv4-only nexthop field. (#606)
e99b6941 make gobgp grpc server listen only nodeip and 127.0.0.1 (#649)
62d0e866 handle network policies with named ports gracefully (#648)
a93dec21 fix: broken links in contributing guide (#650)
0599a27e Add iptables INPUT rules for tunneled services (#610)
f07ec535 avoid duplicate peer pods in npc rules variables (#634)
bdfdc127 when use multiple registries for pulling images in container runtime, we need specify which registry will use exacly (#645)
a968b2b4 cleanup local routes if nexthop moves outside host subnet (#629)
11ae253f Validate the presence of port definitions before attempting to access (#643)

Apart from bug fixes some nice enhancements went in to the release

thanks to @asteven for adding support for

• #575 annotation that let your selectively advertise service VIP's
• #618 prevent access to the nodes through service IP, permit traffic to the only required ports

thanks to @uablrek @Arvinderpal for continued IPv6 incremental updates

• #578

thanks @bazuchan @eric @zerkms @mk01 for your contributions

Changelog

10ddc095 Fixed typo in Global External BGP Peers example (#627)
d7a7a6d7 Add missing ip6tables package to docker container (#631)
4da8ee70 [RFC] prevent host services from being accessible through service IPs (#618)
4efc6cce Add documentation on dependency management using dep. (#621)
34270e42 Periodicaly sync iptables MASQUERADE rules (#619)
c63e71a5 Enable net.bridge.bridge-nf-call-ip6tables for IPv6. (#608)
48e2c7b7 Add iptables input rules for ipvs services (#604)
c38e8f66 Change append to insert for iptables rules (#596)
853b75b1 Periodicaly sync default forward rules (#603)
6cdc2373 Make ipv6 routing to pods (CNI routing) work for ipv6 (#578)
7b9291aa fix docs
1a30f9e2 implement per-service annotations to control IP advertisment (#575)
e5d599b1 Roffe/metrics polish (#595)
0cdaa436 docs/bgp.md: change example to use printf (#594)
46f8265e docs: how to configure explicit proxy (#582)

IPVS throughput fixes and enable arp_ignore and arp_announce

Changelog

c39c13b No reason to restrict Peer ASN's to private only. (#576)
87718c9 make NSC set net.ipv4.vs.conn_reuse_mode=0 (#577)
5bfab47 unified function to set sysctl values and enable arp_ignore and arp_announce(#580)

Apart from support IPVS maglev hashing rest are bug fixes. No breaking changes.

Changelog

4d6b7faa Fixes regression in BGP route reflector functionality. (#573)
cf9bf47d Integrate ip_vs_mh scheduler into kube-router (#564)
3723d822 fix typo on docs/bgp.md (#568)
535fcc5a Added "--router-id=" parameter. (#563)
2a820355 Add mount of /lib/modules to kube-router kubeadm setup doc (#565)
f95cdedf Improved detect in ipv6IsEnabled() (#555)
827bbbcd infer endpoint is local from endpoints "subset.addresses.nodeName" (#560)
d9570c58 all toleration for nodes with taint node.kubernetes.io/not-ready (#558)
bf636c0c Added ipv6 documentation (#551)
0416e07c Change IMAGE_TAG -> IMG_TAG in developing guide (#550)

We are excited to bring a new release with great enhancements. Finally kube-router is starting to add support for
IPv6. Its still WIP progress but pretty close to full working functionality. Also kube-router now fully support network policy semantics with addition of support for ipblock and except.

Shout-out to @uablrek for leading the IPv6 effort in kube-router and @jimmy-zh for filling the last remaining gaps to network policies.

Changelog

077ff86b Ipv6; BGP peering (#545)
a47e0f45 Add support for 'except' feature of network policy rule (#543)
05907d8d Ipv6; Support ipset with "family inet6" (#538)
77459ddb Add CLI option to toggle disabling of source-dest-check in EC2 (#541)
cadba6c8 Use ipset to manage multiple CIDRs in a network policy rule (#529)
cd4ad6f3 update docker build image to go1.10.3 (#535)
c10a6155 update vendored gobgp to latest release (#533)
b479f25d Added support for ARCH=s390x (#532)
1b7ae13e make the comments of the iptables rules more accurate and reasonable (#527)

We are excited to bring the new release version of kube-router. There were 10 pre-releases from v0.1.0

Here is the quick highlight of enhancements in v0.2.0 from version v0.1.0

• kube-router is refactored to be usable to advertise just service VIP's. Kube-router can used with other CNI's like cillium (for e.g https://docs.cilium.io/en/stable/kubernetes/install/kube-router/), Weave etc
• support for advertising service load-balancer IPs with flag --advertise-loadbalancer-ip
• various fixes/enhancements to intelligently advertise/withdraw service VIP's (cluster IP, external IP and load balancer IP's) in case of service with service.Spec.ExternalTrafficPolicy=Local
• support to control on which local IP of the node GoBGP will listen
• ability to enable/disable advertising its pod CIDR to external BGP peers
• bug fixes to BGP graceful restart functionality integrated in kube-router. data-path is not impacted when performing rolling upgrade of kube-router
• better support for the nodes with multiple physical interfaces. Kube-router configures GoBGP such that next hop advertised for a router is an appropriate local IP

Changelog

7496b00d dont shutdown gobgp server if graceful restart is enabled (#526)
02eb11ba Sponsorship update (#524)
468f16b5 Delete CHANGELOG.md (#520)

Enhancements

• Support to choose a different port for BGP than default value of 179. This opens up possibility to run a different BGP implementation (like for e.g. BIRD) in conjunction with GoBGP running as part of kube-router

   An e.g use-case: https://kubernetes.slack.com/archives/C8DCQGTSB/p1533650593000416
  

• In case of multiple uplinks to different external peers, next-hop used should be appropriate local IP (instead of hard code node IP as earlier). --override-nexthop basically overrides the next hop configured in local RIB with appropriate local ip as next hop when advertising routes to the BGP peers.

• Support for user configurable list of local IP address on which BGP server should listen. This is important in case of nodes with multiple interfaces and multiple external peers.

Changelog

01ec8837 prevent IPIP tunnel creation when --override-nexthop=ture (#518)
1db83adf Added support for custom BGP ports with 179 still being default (#492) (#493)
86ba7840 Introduces the option --override-nexthop, to override the next hop used in advertised routes (#502)
b76d22f0 [jjo] ipAddrDel(): also delete VIP local rt addition (#514)
624c74f8 issue-385: make it optional on which ip address BGP server listens (#473)
94e163b5 update BGP export policies on endpoints add event (#508)
85d8df42 Improve health check for cache synchronization (#498)
e2ee6a76 Fix blackholing of traffic when using local traffic policy / annotation (#495)
8bed56fb processing k8s version for NPC (#488)
f340218f fix case where 1 min unintended delay is added when checking for tunnel interface to come ip in pod (#472)

thanks @jjo @johanot @jimmy-zh @jdconti for the contributions.