Open, Multi-Cloud, Multi-Cluster Kubernetes Orchestration
APACHE-2.0 License
Bot releases are hidden (Show)
Published by RainbowMango over 1 year ago
Introduced FederatedHPA
API to address the requirements that scale workloads across clusters.
FederatedHPA
works similarly to HPA in a single cluster. Karmada aggregates metrics from multiple clusters through the karmada-metrics-adaptor
component and then scales Pod replicas. The scaled replicas will be distributed to multiple clusters according to the declaration of PropagationPolicy
or ClusterPropagationPolicy
.
See FederatedHPA Proposal for more details.
(Feature contributor: @chaunceyjiang @jwcesign @Poor12)
In a multi-cluster scenario, the fault may come from the cluster as a whole, or it may be because the application cannot adapt to a certain cluster. Users are now able to declare strategies of application failover about when and how to migrate the unhealthy application in PropagationPolicy
and ClusterPropagationPolicy
. Karmada will automatically migrate unhealthy applications to other available clusters to improve their availability.
See application-level failover for more details.
(Feature contributor: @Poor12, @RainbowMango)
Karmada provides CLI tools and Helm Charts for installation and deployment in previous releases. In this release, the Karmada operator is provided as another declarative deployment method.
The Karmada operator is a method for installing, upgrading, and deleting Karmada instances. It builds upon the basic Karmada resource and controller concepts and provides convenience to centrally manage the entire lifecycle of Karmada instances in a global cluster. With the operator, users can extend Karmada with custom resources (CRs) to manage their instances not only in local clusters but also in remote clusters.
See quick start for more details.
(Feature contributor: @calvin0327, @lonelyCZ, @Poor12)
Karmada's Resource Interpreter Framework is designed for interpreting resource structure. It consists of built-in and customized interpreters. Karmada has bundled the following open-sourced resources so that users can save the effort to customize them, including Argo Workflow, Flux CD, Kyverno, and OpenKruise. They have been verified by the community.
(Feature contributor: @yike12, @chaunceyjiang, @Poor12)
AffinityName
in PropagationPolicy
now is restricted to [1, 32], and must be a qualified name. (#3442, @chaunceyjiang)wk
for resource Work
. (#3468, @yanfeng1992)karmada-webhook
: Introduced validation to ensure the .spec.placement.spreadConstraints.maxGroups/minGroups
in PropagationPolicy is declared with a reasonable value. (#3232, @whitewindmills )karmada-webhook
: Validated the predicate path for imageOverride. (#3397, @chaunceyjiang)karmada-webhook
: Added the missing federatedresourcequota validation config. (#3523, @chaunceyjiang)karmada-controller-manager
: Fixed the issue that RB/CRB labels were not merged when syncing new changes. (#3239, @lxtywypc)karmada-controller-manager
: Fixed the issue that control plane endpointslices cannot be deleted. (#3348, @wenchezhao)karmada-controller-manager
: Corrected the issue of adding duplicate eviction tasks. (#3456, @jwcesign)karmada-controller-manager
: Fixed a corner case that when there were tasks in the GracefulEvictionTasks queue, graceful-eviction-controller would not work after restarting karmada-controller-manager. (#3475, @chaunceyjiang)karmada-controller-manager
: Fixed the panic issue in the case that the grade number of resourceModel is less than the number of resources. (#3591, @sunbinnnnn)karmadactl
: Resolved the failure to view the options of karmadactl addons enable/disable
. (#3298, @Poor12)karmada-scheduler
: Resolved unexpected re-scheduling due to mutating informer cache issue. (#3393, @whitewindmills)karmada-scheduler
: Fixed the issue of inconsistent Generation and SchedulerObservedGeneration. (#3455, @Poor12)karmada-search
: Fixed the paging list issue in karmada search proxy in large-scale member clusters. (#3402, @ikaven1024)karmada-search
: Fixed a panic in ResourceRegistry controller caused by receiving DeletedFinalStateUnknown object from the cache. (#3478, @xigang)karmada-search
: Fixed contecnt-type header issue in HTTP responses. (#3505, @callmeoldprince)None
karmadactl
: Introduced --image-pull-secrets
flag to init
command to specify the secret. (#3237, @my-git9)karmadactl
: Introduced --force
flag to addons disable
command. (#3266, @my-git9)karmadactl
: Introduced support for running init
within a pod. (#3338, @lonelyCZ)karmadactl
: Introduced --host-cluster-domain
flag to command init
and addons
to specify the host cluster domain. (#3292, @tedli)karmadactl
: Introduced --private-image-registry
flag to addons
command to specify image registry. (#3345, @my-git9)karmadactl
: Introduced --purge-namespace
flag for deinit
command to skip namespace deletion during uninstallation. (#3326, @my-git9)karmadactl
: Introduced --auto-create-policy
and --policy-name
flags for promote
command to customize the policy during the promotion. (#3494, @LronDC)karmada-aggregated-apiserver
: Increased .metadata.generation
once the desired state of the Cluster
object is changed. (#3241, @XiShanYongYe-Chang)karmada-controller-mamager
: Provided support for Lua's built-in string function in ResourceInterpreterCustomization. (#3256, @chaunceyjiang)karmada-controller-manager
: The overriders commandsOverrider
and argOverride
in OverridePolicy
now support Job
resources. (#3414, @chaunceyjiang)karmada-controller-manager
: Allowed setting wildcards for --skippedPropagatingNamespaces
flag. (#3373, @chaunceyjiang)karmada-controller-manager
/karmada-agent
: Supported connection to resourceInterpretWebhook without DNS Service. (#2999, @lxtywypc)karmada-controller-manager
: The --skipped-propagating-namespaces
flags now can take regular expressions to represent namespaces and defaults to kube-*
. (#3433, @chaunceyjiang)karmada-controller-manager
: Introduced --concurrent-propagation-policy-syncs
/--concurrent-cluster-propagation-policy-syncs
flags to specify concurrent syncs for PropagationPolicy and ClusterPropagationPolicy. (#3511, @zach593)karmada-search
: Introduced unified-auth support for proxy. (#3279, @XiShanYongYe-Chang)karmada-search
: Returned the actual resource list from search API. (#3312, @tedli )karmada-search
: Fixed the problem that ResourceVersion base64 encrypted repeatedly when starting multiple informers to watch resources. (#3376, @niuyueyang1996)karmada-search
: Supported namespace filters in RR for search proxy. (#3527, @ikaven1024)karmada-scheduler
: Optimized the region selection algorithm. (#3259, @whitewindmills)karmada-scheduler
: Introduced clusterEviction
plugin to skip the clusters that are in the process of eviction. (#3469)karmada-webhook
: Inroduced validation for MultiClusterIngress
objects. (#3516, @XiShanYongYe-Chang)controllers
config for karmada-controller-manager
. (#3240, @Poor12)helm uninstall
). (#3473, @7sunarni)None
Thank you to everyone who contributed to this release!
Users whose commits are in this release (alphabetically by username)
Published by RainbowMango over 1 year ago
karmada-search
: Fixed the problem that ResourceVersion base64 encrypted repeatedly when starting multiple informers to watch resource. (#3387, @niuyueyang1996)karmada-search
: Fixed paging list in karmada search proxy in large-scale member clusters issue. (#3449, @ikaven1024)karmada-search
: Fixed contecnt-type header issue in HTTP response. (#3513, @callmeoldprince)karmada-controller-mamager
: Fixed Lua's built-in string function can not be used issue in ResourceInterpreterCustomization. (#3282, @chaunceyjiang)karmada-controller-manager
: Fixed the control plane endpointslices cannot be deleted issue. (#3354, @wenchezhao)karmada-controller-manager
: Fixed a corner case that when there are tasks in the GracefulEvictionTasks queue, graceful-eviction-controller will not work after restarting karmada-controller-manager. (#3490, @chaunceyjiang)karmada-scheduler
: Fixed unexpected re-scheduling due to mutating informer cache issue. (#3428, @whitewindmills)karmada-scheduler
: Fixed the issue of inconsistent Generation and SchedulerObservedGeneration. (#3477, @Poor12)karmadactl
: Fixed unable to view the options of karmadactl addons enable/disable
issue. (#3305, @lonelyCZ)Published by RainbowMango over 1 year ago
karmada-search
: support pod subresource (attach, exec, port-forward) through global proxy. (#3100, @ikaven1024)karmada-search
: Fixed the problem that ResourceVersion base64 encrypted repeatedly when starting multiple informers to watch resource. (#3388, @niuyueyang1996)karmada-search
: Fixed paging list in karmada search proxy in large-scale member clusters issue. (#3450, @ikaven1024)karmada-search
: Fixed contecnt-type header issue in HTTP response. (#3514, @callmeoldprince)karmada-controller-manager
: Fixed the issue that RBAC resources whose name contains uppercase characters can not be propagated. (#3215, @whitewindmills)karmada-controller-mamager
: Fixed Lua's built-in string function can not be used issue in ResourceInterpreterCustomization. (#3301, @chaunceyjiang)karmada-controller-manager
: Fixed the control plane endpointslices cannot be deleted issue. (#3353, @wenchezhao)karmadactl
: Fixed unable to view the options of karmadactl addons enable/disable
issue. (#3306, @lonelyCZ)Published by RainbowMango over 1 year ago
karmada-search
: support pod subresource (attach, exec, port-forward) through global proxy. (#3116, @XiShanYongYe-Chang)karmada-search
: Fixed the problem that ResourceVersion base64 encrypted repeatedly when starting multiple informers to watch resource. (#3391, @niuyueyang1996)karmada-search
: Fixed http response add content-type header. (#3515, @callmeoldprince)karmada-search
: Fiexed paging list in karmada search proxy in large-scale member clusters. (#3451, @ikaven1024)karmada-controller-manager
: Fixed the issue that RBAC resources whose name contains uppercase characters can not be propagated. (#3214, @whitewindmills)karmada-controller-manager
: Fixed the problem that ServiceAccount can be interpreted as a dependency, but may not be delivered. (#3222, @whitewindmills)karmadactl
: Fixed unable to view the options of karmadactl addons enable/disable
issue. (#3307, @lonelyCZ)Published by RainbowMango over 1 year ago
karmada-controller-manager
: Fixed the issue that RBAC resources whose name contains uppercase characters can not be propagated. (#3123, @whitewindmills)Published by RainbowMango over 1 year ago
Users are now able to declare multiple groups of clusters to both PropagationPolicy
and ClusterPropagationPolicy
by leveraging the newly introduced ClusterAffinities
field. The scheduler will evaluate these groups one by one in the order they appear in the specification until it finds the one that satisfies scheduling restrictions.
This feature allows the Karmada scheduler to first schedule applications to lower-cost clusters or migrate applications from a primary cluster to backup clusters in the case of cluster failure.
See multiple scheduling group proposal for more info.
(Feature contributor: @XiShanYongYe-Chang @RainbowMango)
The default scheduler is now able to work with any number of third-party customized schedulers. Similar to Kubernetes, the workloads will be scheduled by the default scheduler if the scheduler name is not declared in PropagationPolicy
or ClusterPropagationPolicy
.
See customize scheduler for more details.
(Feature contributor: @Poor12)
Affinities
to both PropagationPolicy
and ClusterPropagationPolicy
. (#3105, @RainbowMango)Placement
to the ResoureBinding
/ClusterResourceBinding
API. (#2702, @Poor12)SchedulerObservedAffinityName
to both ResourceBinding
and ClusterResourceBinding
. (#3163, @RainbowMango)karmadactl
: Fixed the issue that karmada-agent
installed by the register
command cannot delete works due to lack of permissions. (#2902, @lonelyCZ)karmadactl
: Fixed the issue that the default ValidatingWebhookConfiguration for resourceinterpreterwebhook
was not working. (#2915, @whitewindmills)karmadactl
: Fixed the issue that the default ValidatingWebhookConfiguration for resourceinterpretercustomizations
was not working. (#2916, @chaunceyjiang)karmadactl
: Fixed the error of resources whose name contains colons failed to be created when using karmadactl apply
. (#2919, @Poor12)karmadactl init
: Granted karmada-agent permissions to access resourceinterpretercustomizations. (#2984, @jwcesign)karmada-controller-manager
: Generated PolicyRules from given subjects for impersonation deduplicate. (#2911, @yanfeng1992)karmada-controller-manager
/karmada-agent
: Fixed the failure to sync work status due to the informer being accidentally shut down. (#2930, @Poor12)karmada-controller-manager
/karmada-agent
: Fixed misjudgment of deployment and statefuleset health status. (#2928, @Fish-pro)karmada-controller-manager
: Fixed LabelsOverrider
and AnnotationsOverrider
failures to add new items in the case of null label
/annotation
. (#2971, @chaunceyjiang)karmada-controller-manager
: labelsOverrider/annotationsOverrider
supports composed-labels
, like testannotation/projectId: . (#3037, @chaunceyjiang)karmada-controller-manager
: Fixed the issue that RBAC resources whose name contains uppercase characters cannot be propagated. (#3201, @whitewindmills)karmada-agent
: Check whether the resource exists before creating it. Sometimes the resource is created in advance, giving less privilege to Karmada. (#2988, @jwcesign)karmada-scheduler
: Fixed a corner case that re-scheduling was skipped in the case that the cluster becomes not fit. (#2912, @jwcesign)karmada-search
: Avoided proxy request block when member clusters were down. (#3027, @ikaven1024)karmada-webhook
: Validated replicaSchedulingType and replicaDivisionPreference. (#3014, @chaunceyjiang)karmada-webhook
: Fixed the issue that the InterpretDependency operation cannot be registered. (#3052, @whitewindmills)karmada-search
: Supported pod subresource (attach, exec, port-forward) through global proxy. (#3098, @ikaven1024)karmadactl
: Introduced --kube-image-tag
flag to the init
command to specify the Kubernetes image version. (#2840, @helen-frank)karmadactl
: The --cluster-context
flag of join
command now takes current-context
by default. (#2956, @helen-frank)karmadactl
: Added edit mode for interpret commands. (#2831, @ikaven1024)--cert-validity-period
for init
to make the validity period of cert configurable. (#3156, @lonelyCZ)karmada-controller-manager
: Now the OverridePolicy
and ClusterOverridePolicy
will be applied by implicit priority order. The one with the lower priority will be applied before the one with the higher priority. (#2609)karmada-controller-manager
: Users are now able to apply multiple dependencies interpreter configurations. (#2884, @XiShanYongYe-Chang)karmada-controller-manager
: Built-in interpreter supports StatefulSets. (#3009, @chaunceyjiang)karmada-controller-manager
: Retained the labels added to resources by member clusters. (#3088, @chaunceyjiang)karmada-controller-manager
: Default interpreter supports CronJob aggregated status. (#3129, @chaunceyjiang)karmada-controller-manager
: Supports PodDisruptionBudget resource in default interpreter. (#2997, @a7i)karmada-controller-manager
: Support for removing annotations/labels propagated through karmada. (#3099, @chaunceyjiang)karmada-webhook
: Added validation for policy.spec.placement.orderedClusterAffinities. (#3164, @XiShanYongYe-Chang)karmada-webhook
: Validated the fieldSelector of overridepolicy. (#3193, @chaunceyjiang)alpine
now has been promoted from alpine:3.15.1
to alpine:3.17.1
. (#3045, @fengshunli)Instrumentation
: Introduced the pool_get_operation_total
, pool_put_operation_total
metrics to karmada-controller-manager
and karmada-agent
. (#2883, @ikaven1024)Thank you to everyone who contributed to this release!
Users whose commits are in this release (alphabetically by username)
Published by RainbowMango over 1 year ago
karmada-controller-manager
: Fixed LabelsOverrider
and AnnotationsOverrider
failed to add new items issue in case of label
/annotation
is nil. (#2972, @chaunceyjiang)karmada-controller-manager
: labelsOverrider/annotationsOverrider
supports composed-labels
, like testannotation/projectId: . (#3047, @chaunceyjiang)karmadactl
: Grant karmada-agent permission to access resourceinterpretercustomizations for init
command. (#2986, @jwcesign)karmada-agent
: Check if the resource exists before creating it. Sometimes the resource is created in advance, to give less privilege to Karmada. (#3002, @jwcesign)karmada-search
: filter out not ready clusters. (#3016, @yanfeng1992)karmada-search
: avoid proxy request block when member cluster down. (#3030, @ikaven1024)karmada-webhook
: Fixed the issue that the InterpretDependency operation can't be registered. (#3074, @XiShanYongYe-Chang)Published by RainbowMango over 1 year ago
karmada-search
: filter out not ready clusters. (#3019, @yanfeng1992)karmada-search
: avoid proxy request block when member cluster down. (#3031, @ikaven1024)karmada-webhook
: Fixed the issue that the InterpretDependency operation can't be registered. (#3076, @XiShanYongYe-Chang)Published by RainbowMango over 1 year ago
karmada-search
: filter out not ready clusters. (#3018, @yanfeng1992)karmada-webhook
: Fixed the issue that the InterpretDependency operation can't be registered. (#3077, @XiShanYongYe-Chang)Published by RainbowMango almost 2 years ago
karmadactl
: Fixed karmada-agent
installed by the register
command can not delete works due to lack of permission issue. (#2904, @lonelyCZ)karmadactl
: Fixed the default ValidatingWebhookConfiguration for resourceinterpreterwebhook
not working issue. (#2924, @qingwave)karmadactl
: Fixed the default ValidatingWebhookConfiguration for resourceinterpretercustomizations
not working issue. (#2927, @chaunceyjiang)karmadactl
: Fixed the error of resources whose name contains colons failing to be created when using karmadactl apply
. (#2931, @Poor12)karmada-controller-manager
/karmada-agent
: Fixed misjudgment of deployment and statefuleset health status. (#2944, @Fish-pro)karmada-controller-manager
/karmada-agent
: Fixed failed to sync work status issue due to the informer being accidentally shut down. (#2937, @Poor12)karmada-scheduler
: Fixed a corner case that re-schedule be skipped in case of the cluster becomes not fit. (#2955, @jwcesign)Published by RainbowMango almost 2 years ago
karmada-scheduler
: Fixed a corner case that re-schedule be skipped in case of the cluster becomes not fit. (#2959, @jwcesign)karmada-controller-manager
/karmada-agent
: Fixed failed to sync work status issue due to the informer being accidentally shut down. (#2938, @Poor12)karmada-controller-manager
/karmada-agent
: Fixed misjudgment of deployment and statefuleset health status. (#2945, @Fish-pro)karmadactl
: Fixed the error of resources whose name contains colons failing to be created when using karmadactl apply
. (#2932, @Poor12)karmadactl
: Fixed the default ValidatingWebhookConfiguration for resourceinterpreterwebhook
not working issue. (#2923, @qingwave)karmadactl
: Fixed karmada-agent
installed by the register
command can not delete works due to lack of permission issue. (#2905, @lonelyCZ)karmada-webhook
: Fixed failed to set resource selector default namespace if the relevant OverridePolicy and PropagationPolicy with namespace unset issue. (#2872, @carlory)Published by RainbowMango almost 2 years ago
karmada-controller-manager
/karmada-agent
: Fixed failed to sync work status issue due to the informer being accidentally shut down. (#2939, @Poor12)karmadactl
: Fixed the error of resources whose name contains colons failing to be created when using karmadactl promote
. (#2933, @Poor12)karmadactl
: Fixed the default ValidatingWebhookConfiguration for resourceinterpreterwebhook
not working issue. (#2922, @qingwave)karmada-webhook
: Fixed failed to set resource selector default namespace if the relevant OverridePolicy and PropagationPolicy with namespace unset issue. (#2873, @carlory )Published by RainbowMango almost 2 years ago
The Interpreter Framework is designed for interpreting the structure of arbitrary resource types. It consists of built-in
and customized
interpreters, this release introduced another brand-new customized interpreter.
With the newly introduced declarative
interpreter, users can quickly customize resource interpreters for both Kubernetes resources and CRD resources by the rules declared in the ResourceInterpreterCustomization
API specification. Compared with the interpreter customized by the webhook, it gets the rules from the declarative specifications instead of requiring an additional webhook component.
The new command named interpret
in the karmadactl
could be used to test the rules before applying them to the system.
Some examples are provided to help users better understand how this interpreter can be used.
(Feature contributor: @jameszhangyukun @ikaven1024 @chaunceyjiang @XiShanYongYe-Chang @RainbowMango)
Users are now able to declare the priorities for both PropagationPolicy
and ClusterPropagationPolicy
. A policy will be applied for the matched resource templates if there are no other policies with higher priority at the point the resource template be processed.
The system administrator can use priorities to manage and control policies. Refer to Configure PropagationPolicy priority for more details.
(Feature contributor: @Garrybest @jwcesign)
This release enhanced observability significantly through metrics and events.
The metrics can be queried by the endpoint(/metrics
) of each component using an HTTP scrap, and they are served in Prometheus format. Events are reported to the relevant resource objects respectively.
Refer to events and metrics for more details.
(Feature contributor: @Poor12)
LabelsOverrider
and AnnotationsOverrider
for overriding labels and annotations.(#2584, @chaunceyjiang)karmada-search
: Fixed concurrent map writes panic while listing objects via proxy. (#2483, @ikaven1024)karmada-search
: Fixed the issue that the returned ResourceVersion by proxy was not stable. (#2746, @cmicat)karmada-controller-manager
/karmada-agent
: Fixed the issue that pod information cannot be collected when building resource summary. (#2489, @Poor12)karmada-controller-manager
: Used cluster secret ref namespace in unified-auth-controller when generating a ClusterRoleBinding. (#2516, @XiShanYongYe-Chang)karmada-controller-manager
: Fixed the failure of creating resources whose name contains colons. (#2549, @XiShanYongYe-Chang)karmada-controller-manager
: Fixed the panic when cluster ImpersonatorSecretRef is nil. (#2675, @stingshen)karmada-controller-manager
: Fixed serviceaccount continual regeneration by service account controller. (#2578, @Poor12)karmada-controller-manager
: Disabled the preemption matching of pp/cpp priority. (#2734, @XiShanYongYe-Chang)karmada-controller-manager
: Fixed the issue that clusterOverridePolicy and overridePolicy with nil resource selector could not work. (#2771, @wuyingjun-lucky)karmada-controlle-managerr
: Ignored resources that do not match with a policy before applying the policy. (#2786, @XiShanYongYe-Chang)karmada-agent
: Fixed the issue that ServiceExport
controller cannot report endpointSlices
(due to the missing create
permission). (#2515, @lonelyCZ)karmadactl
: Fixed init
cannot honor IPv6 addresses when generating the kubeconfig file. (#2450, @duanmengkk)karmadactl
: Fixed --karmada-data
directory was not initialized in init
command. (#2548, @jwcesign)karmadactl
: Fixed the issue that init
commands print incorrect register command. (#2707, @Songjoy)karmadactl
: Fixed the issue that the namespace already exists. (#2505, @cleverhu)karmada-webhook
: Fixed the failure to set resource selector default namespace when the relevant OverridePolicy and PropagationPolicy are not configured with namespaces. (#2858, @carlory)Security
: Added limitReader to io.ReadAll
which could limit the memory request and avoid DoS attacks. (#2765, @Poor12)karmadactl
: Improved karmada init help output. (#2342, @my-git9)karmadactl
: karmadactl prohibits input extra arguments for init
command. (#2497, @helen-frank)karmadactl
/chart
: The init
no longer creates redundant ServiceAccounts for components except karmada-agent
. (#2523, @carlory)karmadactl
: Fixed the issue that options of deinit
cannot be shown. (#2540, @helen-frank)karmadactl/chart
: If Karmada is installed by using karmadactl or a Helm chart with default configurations, the image tag of karmada-kube-controller-manager/karmada-apiserver will be kube-controller-manager:v1.25.2/kube-apiserver:v1.25.2. (#2539, @jwcesign)karmadactl
: Introduced --karmada-apiserver-advertise-address
flag to specify Karmada APIserver's address to the init
sub-command. (#2550, @wuyingjun-lucky)karmadactl
: Introduced --enable-cert-rotation option to register commands. (#2596, @lonelyCZ)karmadactl
: Added dryrun for uncordon. (#2760, @helen-frank)karmadactl
: Added validation on cluster existence for karmadactl get. (#2787, @helen-frank)karmadactl
: Added liveness probe into the kube-controller-manager component. (#2817, @carlory)karmadactl
: Added --image-registry
flags for init
. (#2655, @helen-frank)karmadactl
: Added interpreter command for resource interpretation customizations. (#2750, @ikaven1024)karmadactl
: Added execute mod for the interpret command. (#2824, @ikaven1024)karmada-search
: Objects returned by proxy will have resource.karmada.io/cached-from-cluster
annotation to indicate which member cluster it comes from. (#2469, @ikaven1024)karmada-search
: Users can get the real resource request metrics while using the proxy. (#2481, @ikaven1024)karmada-search
: Users now can use --disable-search
and --disable-proxy
options to disable search and proxy features (both enabled by default). (#2650, @ikaven1024)karmada-controller-manager
: Added implicit priority for PropagationPolicy. (#2267, @Garrybest)karmada-controller-manager
: Introduced resource label namespace.karmada.io/skip-auto-propagation: "true"
for namespaces that should be skipped from auto propagation. (#2696, @jwcesign)karmada-controller-manager
: Allowed users to update the .spec.resourceSelectors
field of PropagationPolicy/ClusterPropagationPolicy
. (#2562, @XiShanYongYe-Chang)karmada-scheduler-estimator
: Leveraged scheduler cache to estimate replicas. (#2704, @Garrybest)karmada-controller-manager
: Stop propagating finalizers to member clusters. (#2870, @stingshen)karmada-scheduler
/karmada-scheduler-descheduler
: Introduced --scheduler-estimator-service-prefix
flag for discovery estimators. (#2527, @carlory)karmada-scheduler
: Added scheduling diagnosis. (#2302, @Garrybest)karmada-agent
: Introduced auto certificate rotation function. (#2596, @lonelyCZ)karmada-webhook
: Prevented modifying and creating ResourceInterpreterCustomization
using the same interpretation rules. (#2755, @chaunceyjiang)Helm Chart
: Added descheduler name suffix of chart deployment manifest. (#2330, @calvin0327)HelmChart
: Fixed liveness probe misconfiguration which caused kube-controller-manager to always be CrashLoopBackup
. (#2277, @calvin0327 )chart
: Fixed the issue that using custom certs leads to failed post-install-job and kube-controller-manager crash due to missing /etc/karmada/pki/server-ca.key. (#2637, @631068264)Instrumentation
: Introduced the GetDependenciesSucceed
and GetDependenciesFailed
to resource templates. Introduced the SyncScheduleResultToDependenciesSucceed
and SyncScheduleResultToDependenciesFailed
to resourceBinding
object. (#2773, @Poor12 )Instrumentation
: Introduced EvictWorkloadFromClusterSucceed
and EvictWorkloadFromClusterFailed
to the binding
object and its reference. Refactored the event name of TaintManagerEviction
. (#2835, @Poor12)Instrumentation
: Introduced metrics resource_find_matched_policy_duration_seconds
, resource_apply_policy_duration_seconds
, policy_apply_attempts_total
, binding_sync_work_duration_seconds
, and work_sync_workload_duration_seconds
. (#2868, @Poor12 )Instrumentation
: Introduced the CreateExecutionSpaceSucceed
and RemoveExecutionSpaceSucceed
events to Cluster
object. (#2688, @Poor12)Instrumentation
: Introduced the ApplyOverridePolicySucceed
and ApplyOverridePolicyFailed
events to workloads. (#2764, @Poor12)Instrumentation
: Introduced the ReflectStatusToWorkSucceed
, ReflectStatusToWorkFailed
, InterpretHealthSucceed
and InterpretHealthFailed
events to work
object. (#2770, @Poor12)Instrumentation
: Introduced the SyncImpersonationConfigSucceed
and SyncImpersonationConfigFailed
to the cluster
object. (#2796, @Poor12)Instrumentation
: Applied AggregateStatusFailed
, AggregateStatusSucceed
, SyncWorkSucceed
, and SyncWorkFailed
to federatedResourceQuota
object. (#2812, @Poor12)Instrumentation
: Introduced SyncDerivedServiceSucceed
and SyncDerivedServiceFailed
to serviceImport
object. (#2830, @Poor12)Instrumentation
: Introduced the cluster_ready_info
, cluster_node_number
, cluster_ready_node_number
, cluster_memory_allocatable_bytes
, cluster_cpu_allocatable_number
, cluster_pod_allocatable_number
, cluster_memory_allocated_bytes
, cluster_cpu_allocated_number
, cluster_pod_allocated_number
, cluster_sync_status_duration
to record the cluster status in karmada-controller-manager
and karmada-agent
. (#2496, @Poor12)Instrumentation
: Introduced metrics framework_extension_point_duration_seconds
and plugin_execution_duration_seconds
for karmada-scheduler
. (#2087, @Poor12)Thank you to everyone who contributed to this release!
Users whose commits are in this release (alphabetically by username)
Published by RainbowMango almost 2 years ago
karmada-controller-manager
: Fixed the panic when cluster ImpersonatorSecretRef is nil. (#2690, @stingshen)karmada-controller-manager
: Disable the preemption matching of pp/cpp priority. (#2738, @XiShanYongYe-Chang)karmada-controller-manager
: Fix clusterOverridePolicy and overridePolicy with nil resource selector could not work. (#2788, @wuyingjun-lucky)karmada-controller-manager
: ignore resource that does not match the policy before applying the policy. (#2813, @XiShanYongYe-Chang)HelmChart
: Fixed liveness probe misconfiguration which caused kube-controller-manager to always CrashLoopBackup
. (#2836, @calvin0327)karmadactl
: Fixed init
commands print incorrect register command issue. (#2744, @Songjoy)karmada-search
: Fixed returned ResourceVersion by proxy not stable issue. (#2816, @cmicat)Security
: Add limitReader to io.ReadAll
which could limit the memory request and avoid DoS attacks. (#2774, @Poor12)Published by RainbowMango almost 2 years ago
karmada-controller-manager
: Fixed the panic when cluster ImpersonatorSecretRef is nil. (#2691, @stingshen)karmada-controller-manager
: Disable the preemption matching of pp/cpp priority. (#2739, @XiShanYongYe-Chang)karmada-controller-manager
: Fix clusterOverridePolicy and overridePolicy with nil resource selector could not work. (#2798, @wuyingjun-lucky)karmada-controller-manager
: ignore resource that does not match the policy before applying the policy. (#2814, @XiShanYongYe-Chang)Security
: Add limitReader to io.ReadAll
which could limit the memory request and avoid DoS attacks. (#2775, @Poor12)Published by RainbowMango almost 2 years ago
karmada-controller-manager
: Fixed the panic when cluster ImpersonatorSecretRef is nil. (#2692, @stingshen)karmada-controller-manager
: Disable the preemption matching of pp/cpp priority. (#2740, @XiShanYongYe-Chang)karmada-controller-manager
: Fix ClusterOverridePolicy and OverridePolicy with nil resource selector could not work. (#2800, @wuyingjun-lucky)karmada-controller-manager
: ignore resource that does not match the policy before applying policy. (#2815, @XiShanYongYe-Chang)Security
: Add limitReader to io.ReadAll
which could limit the memory request and avoid DoS attacks. (#2776, @Poor12)Published by RainbowMango about 2 years ago
karmada-controller-manager
/karmada-agent
: Fixed pod information can not be collected issue when building resource summary. (#2500, @Poor12)karmada-search
: fix concurrent map writes panic while list via proxy; And add cache source annotation to returned objects. (#2514, @ikaven1024)karmada-agent
: Fixed ServiceExport
controller can not report endpointSlices
issue(due to miss create
permission). (#2517, @lonelyCZ)karmadactl
: Fixed init
can not honor IPV6 address issue when generating kubeconfig file. (#2520, @duanmengkk)karmada-controller-manager
: use cluster secret ref namespace in unified-auth-controller when generate ClusterRoleBinding. (#2530, @XiShanYongYe-Chang)karmadactl
: Fixed --karmada-data
directory not initialization issue in init
command. (#2563, @jwcesign)karmadactl
: Fixed options of deinit
can not be shown issue. (#2567, @helen-frank)Published by RainbowMango about 2 years ago
karmada-controller-manager
/karmada-agent
: Fixed a resource status can not be collected issue in case of Resource Interpreter returns an error. (#2429, @zhuwint)karmada-sechduler
: Fixed a panic issue when replicaDivisionPreference
is Weighted
and WeightPreference
is nil. (#2453, @XiShanYongYe-Chang)karmadactl
: Fixed init
can not honor IPV6 address issue when generating kubeconfig file. (#2529, @duanmengkk)karmada-controller-manager
: use cluster secret ref namespace in unified-auth-controller when generate ClusterRoleBinding. (#2531, @XiShanYongYe-Chang)karmadactl
: Fixed options of deinit
can not be shown issue. (#2566, @helen-frank)Published by RainbowMango about 2 years ago
Helm Chart
: Fixed karmada-apiserver
always pending in case of upgrade issue. (#2436, @Poor12)karmadactl
: Fixed init
can not honor IPV6 address issue when generating kubeconfig file. (#2535, @duanmengkk)karmada-controller-manager
: use cluster secret ref namespace in unified-auth-controller when generate ClusterRoleBinding.(#2538, @XiShanYongYe-Chang )Published by RainbowMango about 2 years ago
We introduced a new controller named taint manager
which aims to evict workloads from faulty clusters after a grace period.
Then the scheduler would select new best-fit clusters for the workloads. In addition, if the feature GracefulEviction
is enabled,
the eviction will be very smooth, that is, the removal of evicted workloads will be delayed until the workloads are available on
new clusters or reach the maximum grace period. For more details please refer to Failover Overview.
(Feature contributor: @Garrybest, @XiShanYongYe-Chang)
We introduced a new proxy feature to karmada-search
that allows users to access resources in multiple clusters in a way just like accessing resources in a single cluster. No matter whether the resources are managed by Karmada, by leveraging the proxy, users
can manipulate the resources from the Karmada control plane.
For more details please refer to Global Resource Proxy.
(Feature contributor: @ikaven1024, @XiShanYongYe-Chang)
To provide a more accurate scheduling basis for the scheduler, we introduced a way to model the cluster's available resources.
The cluster status controller will model the resources as per the customized resource models, which is more accurate than the general resource summary. For more details please refer to Cluster Resource Modeling.
(Feature contributor: @halfrost, @Poor12)
Now for clusters in Pull
mode, we provide a way for them to register with the Karmada control plane. By leveraging the commands token
and register
in kubectl
, the registration process including deploying the karmada-agent
can be completed very easily. For more details please refer to Register cluster with Pull mode.
(Feature contributor: @lonelyCZ )
We improved the system scalability, such as:
cachedRESTMapper
(#2187)With these improvements, Karmada can easily manage hundreds of huge clusters. The detailed test report will be released soon.
Cluster
API is added optional field ID
to uniquely identify the cluster. (@RainbowMango, #2180)Cluster
API is added optional field ProxyHeader
to specify the HTTP header required by the proxy server. (@mrlihanbo, #1874)Cluster
API is added optional field named ResourceModels
to specify resource modeling. (@halfrost, #2386)Work
and ResourceBinding
/ClusterResourceBinding
APIs are added field health
to represent the state of workload. (@XiShanYongYe-Chang, #2351)karmadactl
: Fixed issue that Kubernetes v1.24 cannot be joined. (@zgfh, #1972)karmadactl
: Fixed a panic issue when retrieving resources from an unknown cluster(karmadactl get xxx --cluster=not-exist
). (@my-git9, #2171)karmadactl
: Fixed failed promoting if a resource with another kind using the same name has been promoted before. (@wuyingjun-lucky, #1824)karmada-search
: Fixed panic when the resource annotation is nil. (@XiShanYongYe-Chang, #1921)karmada-search
: Fixed panic comparing uncomparable type cache.ResourceEventHandlerFuncs
. (@liys87x, #1951)karmada-search
: Fixed failed query on a single namespace (@luoMonkeyKing, #2227)karmada-controller-manager
: Fixed that Job
status might be incorrectly marked as Completed
. (@Garrybest, #1987)karmada-controller-manager
: Fixed returning err when the interpreter webhook returns nil patch and nil patchType. (@CharlesQQ, #2161)karmada-controller-manager
: Fixed that Argo CD cannot assess Deployment health status. (@xuqianjins, #2241)karmada-controller-manager
: Fixed that Argo CD cannot assess StatefulSet/DaemonSet health status. (@RainbowMango, #2252)karmada-controller-manager
/karmada-agent
: Fixed an resource status can not be collected issue in case of Resource Interpreter returns an error. (@XiShanYongYe-Chang, #2428)karmada-sechduler
: Fixed a panic issue when replicaDivisionPreference
is Weighted
and WeightPreference
is nil. (@XiShanYongYe-Chang, #2451)karmadactl
: Added --force
flag to deinit
to skip confirmation. (@zgfh, #2016)karmadactl
: The flag -c
of sub-command promote
now has been changed to uppercase -C
. (@Fish-pro, #2140): Introduced
--cluster-zone` and `--cluster-region` flags to `join` command to specify the zone and region of joining cluster. (@chaunceyjiang, #2048)karmadactl
: Introduced --namespace
flag to exec
command to specify the workload namespace. (@carlory, #2092)karmadactl
: Allowed reading namespaces from the context field of karmada config for get
command. (@carlory, #2148)karmadactl
: Introduced apply
subcommand to apply a configuration to a resource by file name or stdin. (@carlory, #2000)karmadactl
: Introduced --namespace
flag to describe
command to specify the namespace the workload belongs to. (@TheStylite, #2153)karmadactl
: Introduced --cluster
flag for apply
command to allow users to select one or many member clusters to propagate resources. (@carlory, #2192)karmadactl
: Introduced options subcmd to list global command-line options. (@lonelyCZ, #2283)karmadactl
: Introduced the token
command to manage bootstrap tokens. (@lonelyCZ, #2399)karmadactl
: Introduced the register
command for joining PULL mode cluster. (@lonelyCZ, #2388)karmada-scheduler
: Introduced --enable-empty-workload-propagation
flag to enable propagating empty workloads. (@CharlesQQ, #1720)karmada-scheduler
: Allowed extended plugins in an out-of-tree mode. (@kerthcet, #1663)karmada-scheduler
: Introduced --disable-scheduler-estimator-in-pull-mode
flag to disable scheduler-estimator for clusters in pull mode. (@prodanlabs, #2064)karmada-scheduler
: Introduced --plugins
flag to enable or disable scheduler plugins. (@chaunceyjiang, #2135)karmada-scheduler
: Now the scheduler starts to re-schedule in case of cluster state changes. (@chaunceyjiang, #2301)karmada-search
: The search API supports searching for resources according to labels. (@XiShanYongYe-Chang, #1917)karmada-search
: The annotation cluster.karmada.io/name
which is used to represent the source of cache now has been changed to resource.karmada.io/cached-from-cluster
. (@calvin0327, #1960)karmada-search
: Fixed panic issue when dumping error info. (@AllenZMC, #2231)karmada-controller-manager/karmada-agent
: Cluster state controller now able to collect partial API list in the case of discovery failure. (@duanmengkk, #1968)karmada-controller-manager/karmada-agent
: Introduced --cluster-success-threshold
flag to specify cluster success threshold. Default to 30s. (@dddddai, #1884)karmada-controller-manager/karmada-agent
: Added CronJob support to the default resource interpreter framework. (@chaunceyjiang, #2060)karmada-controller-manager
/karmada-agent
: Introduced --leader-elect-lease-duration
, --leader-elect-renew-deadline
and --leader-elect-retry-period
flags to specify leader election behaviors. (@CharlesQQ, #2056)karmada-controller-manager
/karmada-agent
: Fixed panic issue when dumping error infos. (@AllenZMC, #2117)karmada-controller-manager
/karmada-agent
: Supported interpreting health state by levering Resource Interpreter Framework
. (@zhuwint, #2329)karmada-controller-manager
/karmada-agent
: Introduced --enable-cluster-resource-modeling
flag to enable or disable cluster modeling feature. (@RainbowMango, #2387)karmada-controller-manager
/karmada-agent
: Now able to retain .spec.claimRef
field of PersistentVolume
. (@Garrybest, #2415)karmada-controller-manager
: interpreter framework
starts to support Pod state aggregation. (@xyz2277, #1913)karmada-controller-manager
: interpreter framework
starts to support PVC state aggregation. (@chaunceyjiang, #2070)karmada-controller-manager
: Stopped reporting and refreshing lease
for clusters in Push
mode. (@dddddai, #2033)karmada-controller-manager
: interpreter framework
starts to support Pod Failded
and Succeeded
aggregation. (@chaunceyjiang, #2146)karmada-controller-manager
: namespace
controller starts to apply ClusterOverridePolicy
during propagation namespaces. (@zirain, #2263)karmada-controller-manager
: Propagation dependencies support propagating ServiceAccounts. (@chaunceyjiang, #2035)karmada-agent
: Introduced --metrics-bind-address
flag to specify the address for serving Prometheus metrics. (@1953, #1953)karmada-agent
: Introduced --report-secrets
flag to allow secrets to be reported to the Karmada control plane during registering. (@CharlesQQ, #1990)karmada-agent
: Introduced --cluster-provider
and --cluster-region
flags to specify cluster-provider and cluster-region during registering. (@CharlesQQ, #2152)karmada-webhook
: Added default tolerations, defaultNotReadyTolerationSeconds, and defaultUnreachableTolerationSeconds, for (Cluster)PropagationPolicy. (@Garrybest, #2284)karmada-webhook
: The '.spec.ttlSecondsAfterFinished' field of the Job resource will be removed before propagating to member clusters. (@chaunceyjiang, #2294)karmada-agent
/karmadactl
: Now an error will be reported when registering the same cluster to Karmada. (@yy158775, #2369)Helm chart
: Updated default kube-apiserver
from v1.21 to v1.22. (@AllenZMC, #1941)Helm Chart
: Added missing APIService
configuration for karmada-aggregated-apiserver
. (@zhixian82, #2258)Helm Chart
: Fixed the webhook service mismatch issue in the case of customized release name. (@calvin0327, #2275)Helm Chart
: Introduced --cluster-api-endpoint
for karmada-agent
. (@my-git9, #2299)Helm Chart
: Fixed misconfigured MutatingWebhookConfiguration
. (@zhixian82, #2401)karmadactl
: Removed --dry-run
flag from describe
, exec
and log
commands. (@carlory, #2023)karmadactl
: Removed the --cluster-namespace
flag for get
command. (@carlory, #2190)karmadactl
: Removed the --cluster-namespace
flag for promote
command. (@carlory, #2193)Thank you to everyone who contributed to this release!
Users whose commits are in this release (alphabetically by user name)