When troubleshooting an issue in a cluster, there are generally two ways to investigate problems. One is to check the logs of various components to identify the issue, while the other is to retrieve event events through the API server and analyze what actions a component has taken based on those events.

The event event mechanism makes it easy for us to troubleshoot problems. An event is an API object that records what action a component took at a certain time. Each component acts as a client, sending requests to the API server to create or update event events. By default, the API server retains event events for one hour.

1 Overview

In the cloud, to allow external traffic to access your Kubernetes cluster, the typical approach is to use a LoadBalancer-type service, which directs external traffic through a load balancer into the Ingress Controller and then distributes it to various pods.

Ingress controllers often need updates or configuration changes, which usually require the program to be restarted. So, what happens when an Ingress Controller needs a rolling update? Will Ingress access be interrupted?

In the previous analysis, we explored the initialization of kubelet’s command-line. Now, we will delve into the execution of kubelet’s Cobra Command. The Execute() function in Cobra essentially involves command-line parsing (unless DisableFlagParsing: true is set), followed by passing the remaining arguments to the Command.Run function for execution.

This analysis is based on Kubernetes version 1.18.6. For the source code and related readings, please visit the source code analysis repository.

Recently, after upgrading the Docker version, some pods remained in the ‘pending’ state, and it was found that the reason for the pods’ pending status was the inability to obtain an IP address. After investigation, it was discovered that the Docker version upgrade was performed incorrectly, leading to an IP leak in kubenet, which resulted in no available IPs for allocation.

The Kubernetes version used was 1.18.8, with the network mode set to kubenet, a maximum of 125 pods per node, and a pod CIDR of 25.

Kubernetes has many feature functionalities, and these features are generally associated with Kubernetes Enhancement Proposals (KEPs). Feature functionalities go through stages of development such as alpha, beta, GA (Generally Available), and deprecated. Alpha signifies an unstable phase, beta is relatively stable but may contain bugs, GA indicates full stability and usability, and deprecated means the feature is being phased out. The lifecycle of a feature typically involves the proposal of a KEP, alpha phase, beta phase, GA phase, and eventual deprecation. Features in the alpha phase are not enabled by default, while features in beta are enabled by default. For more information on feature gates, visit the Feature Gates documentation and KEP documentation.

Feature gates are used to control whether a specific feature is enabled or disabled. It’s important to note that GA features cannot be disabled. In this article, we’ll use the kubelet source code as an example to analyze how feature gates work.

Kubernetes defaults to using the klog logging library, which is managed by the k8s.io/component-base/logs library and is a forked version of glog. This fork was created because glog had several issues, especially when running in containers, and was not easy to test. As a result, Kubernetes maintains its own version of klog.

Overview

The kubelet command-line has 177 parameters, making it the Kubernetes component with the most configuration options.

Kubelet’s options can be categorized into three types: kubeletFlags (those that cannot be changed after startup and can only be configured via the command line, such as IP addresses and paths), kubeletConfiguration (those that can be modified after startup and can be set via the command line, configuration files, or dynamic Kubelet config - configmap; configuring via the command line is deprecated, and some options can only be set through configuration files, e.g., NodeStatusReportFrequency), and globalFlags (those that can only be configured via the command line, such as log levels, log paths, and versions).