Recently, after upgrading the Docker version, some pods remained in the ‘pending’ state, and it was found that the reason for the pods’ pending status was the inability to obtain an IP address. After investigation, it was discovered that the Docker version upgrade was performed incorrectly, leading to an IP leak in kubenet, which resulted in no available IPs for allocation.

The Kubernetes version used was 1.18.8, with the network mode set to kubenet, a maximum of 125 pods per node, and a pod CIDR of 25.

Kubernetes has many feature functionalities, and these features are generally associated with Kubernetes Enhancement Proposals (KEPs). Feature functionalities go through stages of development such as alpha, beta, GA (Generally Available), and deprecated. Alpha signifies an unstable phase, beta is relatively stable but may contain bugs, GA indicates full stability and usability, and deprecated means the feature is being phased out. The lifecycle of a feature typically involves the proposal of a KEP, alpha phase, beta phase, GA phase, and eventual deprecation. Features in the alpha phase are not enabled by default, while features in beta are enabled by default. For more information on feature gates, visit the Feature Gates documentation and KEP documentation.

Feature gates are used to control whether a specific feature is enabled or disabled. It’s important to note that GA features cannot be disabled. In this article, we’ll use the kubelet source code as an example to analyze how feature gates work.

Kubernetes defaults to using the klog logging library, which is managed by the k8s.io/component-base/logs library and is a forked version of glog. This fork was created because glog had several issues, especially when running in containers, and was not easy to test. As a result, Kubernetes maintains its own version of klog.

Overview

The kubelet command-line has 177 parameters, making it the Kubernetes component with the most configuration options.

Kubelet’s options can be categorized into three types: kubeletFlags (those that cannot be changed after startup and can only be configured via the command line, such as IP addresses and paths), kubeletConfiguration (those that can be modified after startup and can be set via the command line, configuration files, or dynamic Kubelet config - configmap; configuring via the command line is deprecated, and some options can only be set through configuration files, e.g., NodeStatusReportFrequency), and globalFlags (those that can only be configured via the command line, such as log levels, log paths, and versions).

The Taint Manager is triggered by pod and node events. It examines whether nodes bound to pods or nodes themselves have a “NoExecute” taint. If such taints are present, the Taint Manager proceeds to delete all pods on the affected node(s) or a specific pod.

In the previous article about the Node Lifecycle Controller, if the Taint Manager is enabled, it is initialized using NewNoExecuteTaintManager.

Within the Node Lifecycle Controller, handlers are defined for pod and node events. When the Taint Manager is enabled, these events are added to the nodeUpdateQueue and podUpdateQueue within the Taint Manager.

Additionally, a goroutine is launched in the Node Lifecycle Controller to execute taintManager.Run(stopCh).

The Node Lifecycle Controller Manager decides whether to evict pods on a node or set taints based on node leases and node status update times. It also sets the node ready condition to “unknown” when needed. Additionally, it adjusts the node eviction rate based on the overall cluster state and the number of unready nodes in a zone, enabling it to add taints or execute pod evictions as necessary.