kubelet启动--命令行初始化

概述

kubelet命令行参数有177个,是kubernetes组件中配置项最多的。

kubelet的选项分为三类,一类是kubletFlags–(启动之后不能随时改变的,只能通过命令行配置,比如ip地址、路径等);另一类是kubeletConfiguration–(可以在启动之后进行修改,可以通过命令行设置或配置文件或者dynamic Kubelet config–configmap,通过命令行配置是DEPRECATED,一些选项只能通过配置文件设置–比如NodeStatusReportFrequency);最后是globalFlags–(只能通过命令行配置,比如日志级别、日志路径、版本)。

kubelet是如何启动、初始化并识别这么多选项的?本文就是讲kubelet启动过程中的命令初始化,至于各个组件的运行过程,后续文章再来分析。

本文基于kubernetes 1.18.6版本,请访问源代码阅读仓库

kubelet入口文件在cmd\kubelet\kubelet.go,主要流程

  1. 初始化随机因子
  2. 创建一个cobra Command
  3. 初始化日志
  4. 执行cobra Command

这个阶段进行命令行选项的定义、默认值的设置、command的各个字段定义、命令帮助函数和命令使用函数。

主要流程

  • 这里新建一个name为kubelet的flagset,为了拥有干净的flagset(不会被CommandLine的flagset污染),而不用cobra的命令行解析。

  • 同时设置flagset的NormalizeFunc–将flag名包含下划线"_“转为横杆”-“,比如–iptables-drop-bit和–iptables_drop_bit是一样的。

  • 新建一个kubeletFlags里面包含了一些初始选项的默认值:

    具体的field对应的命令行选项见下面kubeletFlags选项,NewKubeletFlags()定义在cmd\kubelet\app\options\options.go

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    
    // NewKubeletFlags will create a new KubeletFlags with default values
    func NewKubeletFlags() *KubeletFlags {
        remoteRuntimeEndpoint := ""
        if runtime.GOOS == "linux" {
            remoteRuntimeEndpoint = "unix:///var/run/dockershim.sock"
        } else if runtime.GOOS == "windows" {
            remoteRuntimeEndpoint = "npipe:////./pipe/dockershim"
        }
    
        return &KubeletFlags{
            EnableServer:                        true,
            ContainerRuntimeOptions:             *NewContainerRuntimeOptions(),
            CertDirectory:                       "/var/lib/kubelet/pki",
            RootDirectory:                       defaultRootDir, // defaultRootDir为/var/lib/kubelet
            MasterServiceNamespace:              metav1.NamespaceDefault, // NamespaceDefault为default
            MaxContainerCount:                   -1,
            MaxPerPodContainerCount:             1,
            MinimumGCAge:                        metav1.Duration{Duration: 0},
            NonMasqueradeCIDR:                   "10.0.0.0/8",
            RegisterSchedulable:                 true,
            ExperimentalKernelMemcgNotification: false,
            RemoteRuntimeEndpoint:               remoteRuntimeEndpoint,
            NodeLabels:                          make(map[string]string),
            VolumePluginDir:                     "/usr/libexec/kubernetes/kubelet-plugins/volume/exec/",
            RegisterNode:                        true,
            SeccompProfileRoot:                  filepath.Join(defaultRootDir, "seccomp"),
            // prior to the introduction of this flag, there was a hardcoded cap of 50 images
            NodeStatusMaxImages:         50,
            EnableCAdvisorJSONEndpoints: false,
        }
    }
    

    其中NewContainerRuntimeOptions()定义在cmd\kubelet\app\options\container_runtime.go

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    
    const (
        // When these values are updated, also update test/utils/image/manifest.go
        defaultPodSandboxImageName    = "k8s.gcr.io/pause"
        defaultPodSandboxImageVersion = "3.2"
    )
    
    var (
        defaultPodSandboxImage = defaultPodSandboxImageName +
            ":" + defaultPodSandboxImageVersion
    )
    
    // NewContainerRuntimeOptions will create a new ContainerRuntimeOptions with
    // default values.
    func NewContainerRuntimeOptions() *config.ContainerRuntimeOptions {
        dockerEndpoint := ""
        if runtime.GOOS != "windows" {
            dockerEndpoint = "unix:///var/run/docker.sock"
        }
    
        return &config.ContainerRuntimeOptions{
            ContainerRuntime:           kubetypes.DockerContainerRuntime, //DockerContainerRuntime为docker
            RedirectContainerStreaming: false,
            DockerEndpoint:             dockerEndpoint,
            DockershimRootDirectory:    "/var/lib/dockershim",
            PodSandboxImage:            defaultPodSandboxImage,
            ImagePullProgressDeadline:  metav1.Duration{Duration: 1 * time.Minute},
            ExperimentalDockershim:     false,
    
            //Alpha feature
            CNIBinDir:   "/opt/cni/bin",
            CNIConfDir:  "/etc/cni/net.d",
            CNICacheDir: "/var/lib/cni/cache",
        }
    }
    
  • 创建一个kubeletConfiguration,用于设置kubeletConfiguration的flag选项的默认值,它通过创建v1beta1的KubeletConfiguration 然后通过scheme.Default()生成默认的配置,再通过scheme.Convert转换成内部的KubeletConfiguration。

    具体的field对应的命令行选项见下面kubeletConfigurationFlag选项

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    
     KubeletConfiguration{
    		SyncFrequency: metav1.Duration{Duration: 1 * time.Minute},
    		FileCheckFrequency: metav1.Duration{Duration: 20 * time.Second},
    		HTTPCheckFrequency: metav1.Duration{Duration: 20 * time.Second},
    		Address: "0.0.0.0",
    		Port: 10250,
    		Authentication.Anonymous.Enabled: true, // by applyLegacyDefaults, by default false,
    		Authentication.Webhook.Enabled: false, // by applyLegacyDefaults, by default true,
    		Authentication.Webhook.CacheTTL: metav1.Duration{Duration: 2 * time.Minute},
    		Authorization.Mode: "AlwaysAllow", // by applyLegacyDefaults, by default "webhook",
    		Authorization.Webhook.CacheAuthorizedTTL: metav1.Duration{Duration: 5 * time.Minute},
    		Authorization.Webhook.CacheUnauthorizedTTL: metav1.Duration{Duration: 30 * time.Second},
    		ReadOnlyPort: 10255, // by applyLegacyDefaults,
    		RegistryPullQPS: 5,
    		RegistryBurst: 10,
    		EventRecordQPS: 5,
    		EventBurst: 10,
    		EnableDebuggingHandlers: true,
    		HealthzPort: 10248,
    		HealthzBindAddress: "127.0.0.1",
    		OOMScoreAdj: -999,
    		StreamingConnectionIdleTimeout: metav1.Duration{Duration: 4 * time.Hour},
    		NodeStatusReportFrequency: metav1.Duration{Duration: 5 * time.Minute},
    		NodeStatusUpdateFrequency: metav1.Duration{Duration: 10 * time.Second},
    		NodeLeaseDurationSeconds: 40,
    		ImageMinimumGCAge: metav1.Duration{Duration: 2 * time.Minute},
    		ImageGCHighThresholdPercent: 85,
    		ImageGCLowThresholdPercent: 80,
    		VolumeStatsAggPeriod: metav1.Duration{Duration: time.Minute},
    		CgroupsPerQOS: true,
    		CgroupDriver: "cgroupfs",
    		CPUManagerPolicy: "none",
    		CPUManagerReconcilePeriod: metav1.Duration{Duration: 10 * time.Second},
    		TopologyManagerPolicy: "none",
    		RuntimeRequestTimeout: metav1.Duration{Duration: 2 * time.Minute},
    		HairpinMode: "promiscuous-bridge",
    		MaxPods: 110,
    		PodPidsLimit: -1,
    		ResolverConfig: "/etc/resolv.conf",
    		CPUCFSQuota: true,
    		CPUCFSQuotaPeriod: metav1.Duration{Duration: 100 * time.Millisecond},
    		MaxOpenFiles: 1000000,
    		ContentType: "application/vnd.kubernetes.protobuf",
    		KubeAPIQPS: 5,
    		KubeAPIBurst: 10,
    		SerializeImagePulls: true,
    		EvictionHard: map[string]string{
    			"memory.available":  "100Mi",
    			"nodefs.available":  "10%",
    			"nodefs.inodesFree": "5%",
    			"imagefs.available": "15%",
    		},
    		EvictionPressureTransitionPeriod: metav1.Duration{Duration: 5 * time.Minute},
    		EnableControllerAttachDetach: true,
    		MakeIPTablesUtilChains: true,
    		IPTablesMasqueradeBit: 14,
    		IPTablesDropBit: 15,
    		FailSwapOn: true,
    		ContainerLogMaxSize: "10Mi",
    		ContainerLogMaxFiles: 5,
    		ConfigMapAndSecretChangeDetectionStrategy: "Watch",
    		EnforceNodeAllocatable: []string{"pods"},
    	}
    
  • 创建cobra.Command,定义了Use、Long、DisableFlagParsing、Run这四个字段。

  • kubeletFlags选项添加到flagset

  • kubeletConfigurationFlag选项添加到flagset

  • globalFlag添加到flagset,globalFlag包含klog、verflag、Cadvisor、logflush相关选项

  • 设置help选项

  • 自定义command的helpfunc和usageFunc

看到这里会发现kubelet并没有使用cobra的命令解析和定义、helpfunc和usageFunc,这是为什么呢?

是因为globalFlag里面定义的选项flag,都是定义在pflag.Command下面–通过import模块自动注册。而cobra默认的helpfunc和usageFunc和excute会执行mergePersistentFlags()–会引入pflag.Command的flag,由于历史原因有一些不需要的选项也注册在pflag.Command, 它们也会被引用进来,所以才会新建一个flagset并自己解析命令,保持干净。

代码注释中也提到了,只是不是很详细

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
	// keep cleanFlagSet separate, so Cobra doesn't pollute it with the global flags
	kubeletFlags.AddFlags(cleanFlagSet)
	options.AddKubeletConfigFlags(cleanFlagSet, kubeletConfig)
	options.AddGlobalFlags(cleanFlagSet)
	cleanFlagSet.BoolP("help", "h", false, fmt.Sprintf("help for %s", cmd.Name()))

	// ugly, but necessary, because Cobra's default UsageFunc and HelpFunc pollute the flagset with global flags
	const usageFmt = "Usage:\n  %s\n\nFlags:\n%s"
	cmd.SetUsageFunc(func(cmd *cobra.Command) error {
		fmt.Fprintf(cmd.OutOrStderr(), usageFmt, cmd.UseLine(), cleanFlagSet.FlagUsagesWrapped(2))
		return nil
	})
	cmd.SetHelpFunc(func(cmd *cobra.Command, args []string) {
		fmt.Fprintf(cmd.OutOrStdout(), "%s\n\n"+usageFmt, cmd.Long, cmd.UseLine(), cleanFlagSet.FlagUsagesWrapped(2))
	})


// AddGlobalFlags explicitly registers flags that libraries (glog, verflag, etc.) register
// against the global flagsets from "flag" and "github.com/spf13/pflag".
// We do this in order to prevent unwanted flags from leaking into the Kubelet's flagset.
func AddGlobalFlags(fs *pflag.FlagSet) {

kubeletFlags 包含kubelet运行中不能改变的选项、或者不能安全改变的选项、不能再不同的实例共享的。

KubeletConfiguration定义了不同实例能够共享的选项。

// KubeletFlags contains configuration flags for the Kubelet.

// A configuration field should go in KubeletFlags instead of KubeletConfiguration if any of these are true:

// - its value will never, or cannot safely be changed during the lifetime of a node, or

// - its value cannot be safely shared between nodes at the same time (e.g. a hostname);

// KubeletConfiguration is intended to be shared between nodes.

categories分类var变量flagdefault默认值description
ContainerRuntimeOptionsContainerRuntime–container-runtimedockerThe container runtime to use. Possible values: ‘docker’, ‘remote’.
ContainerRuntimeOptionsRuntimeCgroups–runtime-cgroups"”Optional absolute name of cgroups to create and run the runtime in.
ContainerRuntimeOptionsRedirectContainerStreaming–redirect-container-streamingfalseEnables container streaming redirect. If false, kubelet will proxy container streaming data between apiserver and container runtime; if true, kubelet will return an http redirect to apiserver, and apiserver will access container runtime directly. The proxy approach is more secure, but introduces some overhead. The redirect approach is more performant, but less secure because the connection between apiserver and container runtime may not be authenticated.
ContainerRuntimeOptions dockerExperimentalDockershim–experimental-dockershimfalseEnable dockershim only mode. In this mode, kubelet will only start dockershim without any other functionalities. This flag only serves test purpose, please do not use it unless you are conscious of what you are doing. [default=false]
ContainerRuntimeOptions dockerDockershimRootDirectory–experimental-dockershim-root-directory/var/lib/dockershimPath to the dockershim root directory.
ContainerRuntimeOptions dockerPodSandboxImage–pod-infra-container-imagek8s.gcr.io/pause:3.2The image whose network/ipc namespaces containers in each pod will use.
ContainerRuntimeOptions dockerDockerEndpoint–docker-endpointunix:///var/run/docker.sockUse this for the docker endpoint to communicate with.
ContainerRuntimeOptions dockerImagePullProgressDeadline.Duration–image-pull-progress-deadline1 * time.MinuteIf no pulling progress is made before this deadline, the image pulling will be cancelled.
ContainerRuntimeOptions dockerNetworkPluginName–network-plugin""<Warning: Alpha feature> The name of the network plugin to be invoked for various events in kubelet/pod lifecycle.
ContainerRuntimeOptions dockerCNIConfDir–cni-conf-dir/opt/cni/conf.d“<Warning: Alpha feature> The full path of the directory in which to search for CNI config files.
ContainerRuntimeOptions dockerCNIBinDir–cni-bin-dir/opt/cni/bin<Warning: Alpha feature> A comma-separated list of full paths of directories in which to search for CNI plugin binaries.
ContainerRuntimeOptions dockerCNICacheDir–cni-cache-dir/var/lib/cni/cache<Warning: Alpha feature> The full path of the directory in which CNI should store cache files.
ContainerRuntimeOptions dockerNetworkPluginMTU–network-plugin-mtu0<Warning: Alpha feature> The MTU to be passed to the network plugin, to override the default. Set to 0 to use the default 1460 MTU.
mainKubeletConfigFile–config"”The Kubelet will load its initial configuration from this file. The path may be absolute or relative; relative paths start at the Kubelet’s current working directory. Omit this flag to use the built-in default configuration values. Command-line flags override configuration from this file.
mainKubeConfig–kubeconfig""Path to a kubeconfig file, specifying how to connect to the API server. Providing –kubeconfig enables API server mode, omitting –kubeconfig enables standalone mode.
mainBootstrapKubeconfig–bootstrap-kubeconfig""Path to a kubeconfig file that will be used to get client certificate for kubelet. If the file specified by –kubeconfig does not exist, the bootstrap kubeconfig is used to request a client certificate from the API server. On success, a kubeconfig file referencing the generated client certificate and key is written to the path specified by –kubeconfig. The client certificate and key file will be stored in the directory pointed by –cert-dir.
mainReallyCrashForTesting–really-crash-for-testingfalseIf true, when panics occur crash. Intended for testing.
mainChaosChance–chaos-chance0If > 0.0, introduce random client errors and latency. Intended for testing.
mainRunOnce–runoncefalseIf true, exit after spawning pods from static pod files or remote urls. Exclusive with –enable-server
mainEnableServer–enable-servertrueEnable the Kubelet’s server
mainHostnameOverride–hostname-override""If non-empty, will use this string as identification instead of the actual hostname. If –cloud-provider is set, the cloud provider determines the name of the node (consult cloud provider documentation to determine if and how the hostname is used).
mainNodeIP–node-ip""IP address of the node. If set, kubelet will use this IP address for the node. If unset, kubelet will use the node’s default IPv4 address, if any, or its default IPv6 address if it has no IPv4 addresses. You can pass :: to make it prefer the default IPv6 address rather than the default IPv4 address.
mainProviderID–provider-id""Unique identifier for identifying the node in a machine database, i.e cloudprovider
mainCertDirectory–cert-dir/var/lib/kubelet/pkiThe directory where the TLS certs are located. If –tls-cert-file and –tls-private-key-file are provided, this flag will be ignored.
mainCloudProvider–cloud-provider""The path to the cloud provider configuration file. Empty string for no configuration file.
mainCloudConfigFile–cloud-config""The path to the cloud provider configuration file. Empty string for no configuration file.
mainRootDirectory–root-dir/var/lib/kubeletDirectory path for managing kubelet files (volume mounts,etc).
mainDynamicConfigDir–dynamic-config-dir""The Kubelet will use this directory for checkpointing downloaded configurations and tracking configuration health. The Kubelet will create this directory if it does not already exist. The path may be absolute or relative; relative paths start at the Kubelet’s current working directory. Providing this flag enables dynamic Kubelet configuration. The DynamicKubeletConfig feature gate must be enabled to pass this flag; this gate currently defaults to true because the feature is beta.
mainRegisterNode–register-nodetrueRegister the node with the apiserver. If –kubeconfig is not provided, this flag is irrelevant, as the Kubelet won’t have an apiserver to register with.
experimentalExperimentalMounterPath–experimental-mounter-path""[Experimental] Path of mounter binary. Leave empty to use the default mount.
experimentalExperimentalKernelMemcgNotification–experimental-kernel-memcg-notificationfalseIf enabled, the kubelet will integrate with the kernel memcg notification to determine if memory eviction thresholds are crossed rather than polling.
experimentalRemoteRuntimeEndpoint–container-runtime-endpointunix:///var/run/dockershim.sock“[Experimental] The endpoint of remote runtime service. Currently unix socket endpoint is supported on Linux, while npipe and tcp endpoints are supported on windows. Examples:‘unix:///var/run/dockershim.sock’, ’npipe:////./pipe/dockershim’
experimentalRemoteImageEndpoint–image-service-endpoint"”[Experimental] The endpoint of remote image service. If not specified, it will be the same with container-runtime-endpoint by default. Currently unix socket endpoint is supported on Linux, while npipe and tcp endpoints are supported on windows. Examples:‘unix:///var/run/dockershim.sock’, ’npipe:////./pipe/dockershim’
experimentalExperimentalCheckNodeCapabilitiesBeforeMount–experimental-check-node-capabilities-before-mountfalse[Experimental] if set true, the kubelet will check the underlying node for required components (binaries, etc.) before performing the mount
experimentalExperimentalNodeAllocatableIgnoreEvictionThreshold–experimental-allocatable-ignore-evictionfalseWhen set to ’true’, Hard Eviction Thresholds will be ignored while calculating Node Allocatable. See https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/ for more details. [default=false]
experimentalNodeLabels–node-labels""<Warning: Alpha feature> Labels to add when registering the node in the cluster. Labels must be key=value pairs separated by ‘,’. Labels in the ‘kubernetes.io’ namespace must begin with an allowed prefix (kubelet.kubernetes.io, node.kubernetes.io) or be in the specifically allowed set (beta.kubernetes.io/arch, beta.kubernetes.io/instance-type, beta.kubernetes.io/os, failure-domain.beta.kubernetes.io/region, failure-domain.beta.kubernetes.io/zone, kubernetes.io/arch, kubernetes.io/hostname, kubernetes.io/os, node.kubernetes.io/instance-type, topology.kubernetes.io/region, topology.kubernetes.io/zone)
experimentalVolumePluginDir–volume-plugin-dir/usr/libexec/kubernetes/kubelet-plugins/volume/exec/The full path of the directory in which to search for additional third party volume plugins
experimentalLockFilePath–lock-file""<Warning: Alpha feature> The path to file for kubelet to use as a lock file.
experimentalExitOnLockContention–exit-on-lock-contentionfalseWhether kubelet should exit upon lock-file contention.
experimentalSeccompProfileRoot–seccomp-profile-root/var/lib/kubelet/seccomp<Warning: Alpha feature> Directory path for seccomp profiles.
experimentalBootstrapCheckpointPath–bootstrap-checkpoint-path""<Warning: Alpha feature> Path to the directory where the checkpoints are stored
experimentalNodeStatusMaxImages–node-status-max-images50<Warning: Alpha feature> The maximum number of images to report in Node.Status.Images. If -1 is specified, no cap will be applied.
deprecatedBootstrapKubeconfig–experimental-bootstrap-kubeconfig""Use –bootstrap-kubeconfig
deprecatedMinimumGCAge.Duration–minimum-container-ttl-duration0Minimum age for a finished container before it is garbage collected. Examples: ‘300ms’, ’10s’ or ‘2h45m’. Use –eviction-hard or –eviction-soft instead. Will be removed in a future version.
deprecatedMaxPerPodContainerCount–maximum-dead-containers-per-container1Maximum number of old instances to retain per container. Each container takes up some disk space. Use –eviction-hard or –eviction-soft instead. Will be removed in a future version.
deprecatedMaxContainerCount–maximum-dead-containers-1Maximum number of old instances of containers to retain globally. Each container takes up some disk space. To disable, set to a negative number. Use –eviction-hard or –eviction-soft instead. Will be removed in a future version.
deprecatedMasterServiceNamespace–master-service-namespacedefaultThe namespace from which the kubernetes master services should be injected into pods. This flag will be removed in a future version.
deprecatedRegisterSchedulable–register-schedulabletrueRegister the node as schedulable. Won’t have any effect if register-node is false. This flag will be removed in a future version.
deprecatedNonMasqueradeCIDR–non-masquerade-cidr10.0.0.0/8Traffic to IPs outside this range will use IP masquerade. Set to ‘0.0.0.0/0’ to never masquerade. This flag will be removed in a future version.
deprecatedKeepTerminatedPodVolumes–keep-terminated-pod-volumesfalseKeep terminated pod volumes mounted to the node after the pod terminates. Can be useful for debugging volume related issues. This flag will be removed in a future version.
deprecatedEnableCAdvisorJSONEndpoints–enable-cadvisor-json-endpointsfalseEnable cAdvisor json /spec and /stats/* endpoints. This flag will be removed in a future version.

kubeletConfigurationFlag分为两类, 一类deprecated command line是可以在命令行(即将被废弃)和配置文件和dynamic kubelet config中配置的,另一类configuration是不能配置在命令行的,只能在配置文件中和dynamic kubelet config中。

categories分类var变量flagdefault默认值description
deprecated command lineFailSwapOn–fail-swap-ontrueMakes the Kubelet fail to start if swap is enabled on the node.
deprecated command lineStaticPodPath–pod-manifest-path""Path to the directory containing static pod files to run, or the path to a single static pod file. Files starting with dots will be ignored.
deprecated command lineSyncFrequency.Duration–sync-frequency1 * time.MinuteMax period between synchronizing running containers and config
deprecated command lineFileCheckFrequency.Duration–file-check-frequency20 * time.SecondDuration between checking config files for new data
deprecated command lineHTTPCheckFrequency.Duration–http-check-frequency20 * time.SecondDuration between checking http for new data
deprecated command lineStaticPodURL–manifest-url""URL for accessing additional Pod specifications to run
deprecated command lineStaticPodURLHeader–manifest-url-header""Comma-separated list of HTTP headers to use when accessing the url provided to –manifest-url. Multiple headers with the same name will be added in the same order provided. This flag can be repeatedly invoked. For example: --manifest-url-header 'a:hello,b:again,c:world' --manifest-url-header 'b:beautiful'
deprecated command lineAddress–address0.0.0.0The IP address for the Kubelet to serve on (set to 0.0.0.0 for all IPv4 interfaces and :: for all IPv6 interfaces)
deprecated command linePort–port10250The port for the Kubelet to serve on.
deprecated command lineReadOnlyPort–read-only-port0The read-only port for the Kubelet to serve on with no authentication/authorization (set to 0 to disable)
deprecated command lineAuthentication.Anonymous.Enabled–anonymous-authtrueEnables anonymous requests to the Kubelet server. Requests that are not rejected by another authentication method are treated as anonymous requests. Anonymous requests have a username of system:anonymous, and a group name of system:unauthenticated.
deprecated command lineAuthentication.Webhook.Enabled–authentication-token-webhookfalseUse the TokenReview API to determine authentication for bearer tokens.
deprecated command lineAuthentication.Webhook.CacheTTL.Duration–authentication-token-webhook-cache-ttl2 * time.MinuteThe duration to cache responses from the webhook token authenticator.
deprecated command lineAuthentication.X509.ClientCAFile–client-ca-file""If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate.
deprecated command lineAuthorization.Mode–authorization-modeAlwaysAllowAuthorization mode for Kubelet server. Valid options are AlwaysAllow or Webhook. Webhook mode uses the SubjectAccessReview API to determine authorization.
deprecated command lineAuthorization.Webhook.CacheAuthorizedTTL.Duration–authorization-webhook-cache-authorized-ttl5 * time.MinuteThe duration to cache ‘authorized’ responses from the webhook authorizer.
deprecated command lineAuthorization.Webhook.CacheUnauthorizedTTL.Duration–authorization-webhook-cache-unauthorized-ttl30 * time.SecondThe duration to cache ‘unauthorized’ responses from the webhook authorizer.
deprecated command lineTLSCertFile–tls-cert-file""File containing x509 Certificate used for serving HTTPS (with intermediate certs, if any, concatenated after server cert). If –tls-cert-file and –tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory passed to –cert-dir.
deprecated command lineTLSPrivateKeyFile–tls-private-key-file""File containing x509 private key matching –tls-cert-file.
deprecated command lineServerTLSBootstrap–rotate-server-certificatesfalseAuto-request and rotate the kubelet serving certificates by requesting new certificates from the kube-apiserver when the certificate expiration approaches. Requires the RotateKubeletServerCertificate feature gate to be enabled, and approval of the submitted CertificateSigningRequest objects.
deprecated command lineTLSCipherSuites–tls-cipher-suites""Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used.
Preferred values: TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384.
Insecure values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_RC4_128_SHA.
deprecated command lineTLSMinVersion–tls-min-version""Minimum TLS version supported. Possible values: VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13
deprecated command lineRotateCertificates–rotate-certificatesfalse<Warning: Beta feature> Auto rotate the kubelet client certificates by requesting new certificates from the kube-apiserver when the certificate expiration approaches.
deprecated command lineRegistryPullQPS–registry-qps5If > 0, limit registry pull QPS to this value. If 0, unlimited.
deprecated command lineRegistryBurst–registry-burst10Maximum size of a bursty pulls, temporarily allows pulls to burst to this number, while still not exceeding registry-qps. Only used if –registry-qps > 0
deprecated command lineEventRecordQPS–event-qps5If > 0, limit event creations per second to this value. If 0, unlimited.
deprecated command lineEventBurst–event-burst10Maximum size of a bursty event records, temporarily allows event records to burst to this number, while still not exceeding event-qps. Only used if –event-qps > 0
deprecated command lineEnableDebuggingHandlers–enable-debugging-handlerstrueEnables server endpoints for log collection and local running of containers and commands
deprecated command lineEnableContentionProfiling–contention-profilingfalseEnable lock contention profiling, if profiling is enabled
deprecated command lineHealthzPort–healthz-port10248The port of the localhost healthz endpoint (set to 0 to disable)
deprecated command lineHealthzBindAddress–healthz-bind-address127.0.0.1The IP address for the healthz server to serve on (set to 0.0.0.0 for all IPv4 interfaces and :: for all IPv6 interfaces)
deprecated command lineOOMScoreAdj–oom-score-adj-999The oom-score-adj value for kubelet process. Values must be within the range [-1000, 1000]
deprecated command lineClusterDomain–cluster-domain""Domain for this cluster. If set, kubelet will configure all containers to search this domain in addition to the host’s search domains
deprecated command lineClusterDNS–cluster-dns""Comma-separated list of DNS server IP address. This value is used for containers DNS server in case of Pods with "dnsPolicy=ClusterFirst". Note: all DNS servers appearing in the list MUST serve the same set of records otherwise name resolution within the cluster may not work correctly. There is no guarantee as to which DNS server may be contacted for name resolution.
deprecated command lineStreamingConnectionIdleTimeout.Duration–streaming-connection-idle-timeout4 * time.HourMaximum time a streaming connection can be idle before the connection is automatically closed. 0 indicates no timeout. Example: ‘5m’
deprecated command lineNodeStatusUpdateFrequency.Duration–node-status-update-frequency10 * time.SecondSpecifies how often kubelet posts node status to master. Note: be cautious when changing the constant, it must work with nodeMonitorGracePeriod in nodecontroller.
deprecated command lineImageMinimumGCAge.Duration–minimum-image-ttl-duration2 * time.MinuteMinimum age for an unused image before it is garbage collected. Examples: ‘300ms’, ’10s’ or ‘2h45m’.
deprecated command lineImageGCHighThresholdPercent–image-gc-high-threshold85The percent of disk usage after which image garbage collection is always run. Values must be within the range [0, 100], To disable image garbage collection, set to 100.
deprecated command lineImageGCLowThresholdPercent–image-gc-low-threshold80The percent of disk usage before which image garbage collection is never run. Lowest disk usage to garbage collect to. Values must be within the range [0, 100] and should not be larger than that of –image-gc-high-threshold.
deprecated command lineVolumeStatsAggPeriod.Duration–volume-stats-agg-period1 * time.Minutepecifies interval for kubelet to calculate and cache the volume disk usage for all pods and volumes. To disable volume calculations, set to 0.
deprecated command lineFeatureGates–feature-gates""A set of key=value pairs that describe feature gates for alpha/experimental features.
deprecated command lineKubeletCgroups–kubelet-cgroups""Optional absolute name of cgroups to create and run the Kubelet in.
deprecated command lineSystemCgroups–system-cgroups""Optional absolute name of cgroups in which to place all non-kernel processes that are not already inside a cgroup under /. Empty for no container. Rolling back the flag requires a reboot.
deprecated command lineCgroupsPerQOS–cgroups-per-qostrueEnable creation of QoS cgroup hierarchy, if true top level QoS and pod cgroups are created.
deprecated command lineCgroupDriver–cgroup-drivercgroupfsDriver that the kubelet uses to manipulate cgroups on the host. Possible values: ‘cgroupfs’, ‘systemd’
deprecated command lineCgroupRoot–cgroup-root""Optional root cgroup to use for pods. This is handled by the container runtime on a best effort basis. Default: ‘’, which means use the container runtime default.
deprecated command lineCPUManagerPolicy–cpu-manager-policynoneCPU Manager policy to use. Possible values: ’none’, ‘static’. Default: ’none’
deprecated command lineCPUManagerReconcilePeriod.Duration–cpu-manager-reconcile-period10 * time.Second<Warning: Alpha feature> CPU Manager reconciliation period. Examples: ’10s’, or ‘1m’. If not supplied, defaults to NodeStatusUpdateFrequency
deprecated command lineQOSReserved–qos-reserved""<Warning: Alpha feature> A set of ResourceName=Percentage (e.g. memory=50%) pairs that describe how pod resource requests are reserved at the QoS level. Currently only memory is supported. Requires the QOSReserved feature gate to be enabled.
deprecated command lineTopologyManagerPolicy–topology-manager-policy“none”Topology Manager policy to use. Possible values: ’none’, ‘best-effort’, ‘restricted’, ‘single-numa-node’.
deprecated command lineRuntimeRequestTimeout.Duration–runtime-request-timeout2 * time.MinuteTimeout of all runtime requests except long running request - pull, logs, exec and attach. When timeout exceeded, kubelet will cancel the request, throw out an error and retry later.
deprecated command lineHairpinMode–hairpin-modepromiscuous-bridgeHow should the kubelet setup hairpin NAT. This allows endpoints of a Service to loadbalance back to themselves if they should try to access their own Service. Valid values are "promiscuous-bridge", "hairpin-veth" and "none".
deprecated command lineMaxPods–max-pods110Number of Pods that can run on this Kubelet.
deprecated command linePodCIDR–pod-cidr""The CIDR to use for pod IP addresses, only used in standalone mode. In cluster mode, this is obtained from the master. For IPv6, the maximum number of IP’s allocated is 65536
deprecated command linePodPidsLimit–pod-max-pids-1Set the maximum number of processes per pod. If -1, the kubelet defaults to the node allocatable pid capacity.
deprecated command lineResolverConfig–resolv-conf/etc/resolv.confResolver configuration file used as the basis for the container DNS resolution configuration.
deprecated command lineCPUCFSQuota–cpu-cfs-quotatrueEnable CPU CFS quota enforcement for containers that specify CPU limits
deprecated command lineCPUCFSQuotaPeriod–cpu-cfs-quota-period100 * time.MillisecondSets CPU CFS quota period value, cpu.cfs_period_us, defaults to Linux Kernel default
deprecated command lineEnableControllerAttachDetach–enable-controller-attach-detachtrueEnables the Attach/Detach controller to manage attachment/detachment of volumes scheduled to this node, and disables kubelet from executing any attach/detach operations
deprecated command lineMakeIPTablesUtilChains–make-iptables-util-chainstrueIf true, kubelet will ensure iptables utility rules are present on host.
deprecated command lineIPTablesMasqueradeBit–iptables-masquerade-bit14The bit of the fwmark space to mark packets for SNAT. Must be within the range [0, 31]. Please match this parameter with corresponding parameter in kube-proxy.
deprecated command lineIPTablesDropBit–iptables-drop-bit15The bit of the fwmark space to mark packets for dropping. Must be within the range [0, 31].
deprecated command lineContainerLogMaxSize–container-log-max-size10Mi<Warning: Beta feature> Set the maximum size (e.g. 10Mi) of container log file before it is rotated. This flag can only be used with –container-runtime=remote.
deprecated command lineContainerLogMaxFiles–container-log-max-files5<Warning: Beta feature> Set the maximum number of container log files that can be present for a container. The number must be >= 2. This flag can only be used with –container-runtime=remote.
deprecated command lineAllowedUnsafeSysctls–allowed-unsafe-sysctls""Comma-separated whitelist of unsafe sysctls or unsafe sysctl patterns (ending in *). Use these at your own risk.
deprecated command lineMaxOpenFiles–max-open-files1000000Number of files that can be opened by Kubelet process.
deprecated command lineContentType–kube-api-content-typeapplication/vnd.kubernetes.protobufContent type of requests sent to apiserver.
deprecated command lineKubeAPIQPS–kube-api-qps5QPS to use while talking with kubernetes apiserver
deprecated command lineKubeAPIBurst–kube-api-burst10Burst to use while talking with kubernetes apiserver
deprecated command lineSerializeImagePulls–serialize-image-pullstruePull images one at a time. We recommend not changing the default value on nodes that run docker daemon with version < 1.9 or an Aufs storage backend. Issue #10959 has more details.
deprecated command lineEvictionHard–eviction-hardmemory.available<100Mi,nodefs.available<10%,nodefs.inodesFree<5%,imagefs.available<15%A set of eviction thresholds (e.g. memory.available<1Gi) that if met would trigger a pod eviction.
deprecated command lineEvictionSoft–eviction-soft""A set of eviction thresholds (e.g. memory.available<1.5Gi) that if met over a corresponding grace period would trigger a pod eviction.
deprecated command lineEvictionSoftGracePeriod–eviction-soft-grace-period""A set of eviction grace periods (e.g. memory.available=1m30s) that correspond to how long a soft eviction threshold must hold before triggering a pod eviction.
deprecated command lineEvictionPressureTransitionPeriod.Duration–eviction-pressure-transition-period5 * time.MinuteDuration for which the kubelet has to wait before transitioning out of an eviction pressure condition.
deprecated command lineEvictionMaxPodGracePeriod–eviction-max-pod-grace-period0Maximum allowed grace period (in seconds) to use when terminating pods in response to a soft eviction threshold being met. If negative, defer to pod specified value.
deprecated command lineEvictionMinimumReclaim–eviction-minimum-reclaim0A set of minimum reclaims (e.g. imagefs.available=2Gi) that describes the minimum amount of resource the kubelet will reclaim when performing a pod eviction if that resource is under pressure.
deprecated command linePodsPerCore–pods-per-core0Number of Pods per core that can run on this Kubelet. The total number of Pods on this Kubelet cannot exceed max-pods, so max-pods will be used if this calculation results in a larger number of Pods allowed on the Kubelet. A value of 0 disables this limit.
deprecated command lineProtectKernelDefaults–protect-kernel-defaultsfalseDefault kubelet behaviour for kernel tuning. If set, kubelet errors if any of kernel tunables is different than kubelet defaults.
deprecated command lineReservedSystemCPUs–reserved-cpus""A comma-separated list of CPUs or CPU ranges that are reserved for system and kubernetes usage. This specific list will supersede cpu counts in –system-reserved and –kube-reserved.
deprecated command lineSystemReserved–system-reserved""A set of ResourceName=ResourceQuantity (e.g. cpu=200m,memory=500Mi,ephemeral-storage=1Gi) pairs that describe resources reserved for non-kubernetes components. Currently only cpu and memory are supported. See http://kubernetes.io/docs/user-guide/compute-resources for more detail. [default=none]
deprecated command lineKubeReserved–kube-reserved""A set of ResourceName=ResourceQuantity (e.g. cpu=200m,memory=500Mi,ephemeral-storage=1Gi) pairs that describe resources reserved for kubernetes system components. Currently cpu, memory and local ephemeral storage for root file system are supported. See http://kubernetes.io/docs/user-guide/compute-resources for more detail. [default=none]
deprecated command lineEnforceNodeAllocatable–enforce-node-allocatablepodsA comma separated list of levels of node allocatable enforcement to be enforced by kubelet. Acceptable options are ’none’, ‘pods’, ‘system-reserved’, and ‘kube-reserved’. If the latter two options are specified, ‘–system-reserved-cgroup’ and ‘–kube-reserved-cgroup’ must also be set, respectively. If ’none’ is specified, no additional options should be set. See https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/ for more details.
deprecated command lineSystemReservedCgroup–system-reserved-cgroup""Absolute name of the top level cgroup that is used to manage non-kubernetes components for which compute resources were reserved via ‘–system-reserved’ flag. Ex. ‘/system-reserved’. [default=’’]
deprecated command lineKubeReservedCgroup–kube-reserved-cgroup""Absolute name of the top level cgroup that is used to manage kubernetes components for which compute resources were reserved via ‘–kube-reserved’ flag. Ex. ‘/kube-reserved’. [default=’’]
configurationNodeStatusReportFrequency5 * time.MinuteThe frequency that kubelet posts node status to master if node status does not change. Kubelet will ignore this frequency and post node status immediately if any change is detected. It is only used when node lease feature is enabled. nodeStatusReportFrequency’s default value is 1m. But if nodeStatusUpdateFrequency is set explicitly, nodeStatusReportFrequency’s default value will be set to nodeStatusUpdateFrequency for backward compatibility.
configurationNodeLeaseDurationSeconds40 * time.SecondThe duration the Kubelet will set on its corresponding Lease, when the NodeLease feature is enabled. This feature provides an indicator of node health by having the Kubelet create and periodically renew a lease, named after the node, in the kube-node-lease namespace. If the lease expires, the node can be considered unhealthy. The lease is currently renewed every 10s, per KEP-0009. In the future, the lease renewal interval may be set based on the lease duration. Requires the NodeLease feature gate to be enabled.
configurationConfigMapAndSecretChangeDetectionStrategyWatchA mode in which config map and secret managers are running.
configurationShowHiddenMetricsForVersion""The previous version for which you want to show hidden metrics. Only the previous minor version is meaningful, other values will not be allowed. The format is ., e.g.: ‘1.16’. The purpose of this format is make sure you have the opportunity to notice if the next release hides additional metrics, rather than being surprised when they are permanently removed in the release after that.

globalFlags包含了klog选项、cadvisor选项、CredentialProvider选项、logflush选项。

klog选项定义通过klog.InitFlags

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
// InitFlags is for explicitly initializing the flags.
func InitFlags(flagset *flag.FlagSet) {
	if flagset == nil {
		flagset = flag.CommandLine
	}

	flagset.StringVar(&logging.logDir, "log_dir", logging.logDir, "If non-empty, write log files in this directory")
	flagset.StringVar(&logging.logFile, "log_file", logging.logFile, "If non-empty, use this log file")
	flagset.Uint64Var(&logging.logFileMaxSizeMB, "log_file_max_size", logging.logFileMaxSizeMB,
		"Defines the maximum size a log file can grow to. Unit is megabytes. "+
			"If the value is 0, the maximum file size is unlimited.")
	flagset.BoolVar(&logging.toStderr, "logtostderr", logging.toStderr, "log to standard error instead of files")
	flagset.BoolVar(&logging.alsoToStderr, "alsologtostderr", logging.alsoToStderr, "log to standard error as well as files")
	flagset.Var(&logging.verbosity, "v", "number for the log level verbosity")
	flagset.BoolVar(&logging.skipHeaders, "add_dir_header", logging.addDirHeader, "If true, adds the file directory to the header") //这里应该是&logging.addDirHeader,在下一个klog版本v2.0.0修复了,所有的1.18版本都会有这个问题
	flagset.BoolVar(&logging.skipHeaders, "skip_headers", logging.skipHeaders, "If true, avoid header prefixes in the log messages")
	flagset.BoolVar(&logging.skipLogHeaders, "skip_log_headers", logging.skipLogHeaders, "If true, avoid headers when opening log files")
	flagset.Var(&logging.stderrThreshold, "stderrthreshold", "logs at or above this threshold go to stderr")
	flagset.Var(&logging.vmodule, "vmodule", "comma-separated list of pattern=N settings for file-filtered logging")
	flagset.Var(&logging.traceLocation, "log_backtrace_at", "when logging hits line file:N, emit a stack trace")
}

cadvisor定义过程

cadvisor flag通过匿名import来导入的,代码保存在vendor/github.com/google/cadvisor下

1
2
3
4
5
6
7
8
// ensure libs have a chance to globally register their flags
	_ "github.com/google/cadvisor/container/common"
	_ "github.com/google/cadvisor/container/containerd"
	_ "github.com/google/cadvisor/container/docker"
	_ "github.com/google/cadvisor/container/raw"
	_ "github.com/google/cadvisor/machine"
	_ "github.com/google/cadvisor/manager"
	_ "github.com/google/cadvisor/storage"

github.com/google/cadvisor/container/docker为例,它定义了下面这些选项

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
var ArgDockerEndpoint = flag.String("docker", "unix:///var/run/docker.sock", "docker endpoint")
var ArgDockerTLS = flag.Bool("docker-tls", false, "use TLS to connect to docker")
var ArgDockerCert = flag.String("docker-tls-cert", "cert.pem", "path to client certificate")
var ArgDockerKey = flag.String("docker-tls-key", "key.pem", "path to private key")
var ArgDockerCA = flag.String("docker-tls-ca", "ca.pem", "path to trusted CA")

// The namespace under which Docker aliases are unique.
const DockerNamespace = "docker"

// Regexp that identifies docker cgroups, containers started with
// --cgroup-parent have another prefix than 'docker'
var dockerCgroupRegexp = regexp.MustCompile(`([a-z0-9]{64})`)

var dockerEnvWhitelist = flag.String("docker_env_metadata_whitelist", "", "a comma-separated list of environment variable keys that needs to be collected for docker containers")

var (
	// Basepath to all container specific information that libcontainer stores.
	dockerRootDir string

	dockerRootDirFlag = flag.String("docker_root", "/var/lib/docker", "DEPRECATED: docker root is read from docker info (this is a fallback, default: /var/lib/docker)")

	dockerRootDirOnce sync.Once

	// flag that controls globally disabling thin_ls pending future enhancements.
	// in production, it has been found that thin_ls makes excessive use of iops.
	// in an iops restricted environment, usage of thin_ls must be controlled via blkio.
	// pending that enhancement, disable its usage.
	disableThinLs = true
)

cadvisor选项中除了--docker-roothousekeeping-interval不是deprecated,其他选项都是deprecated。其中update_machine_info_interval这个flag未注册到kubelet中,根据commit记录,update_machine_info_interval是在升级cadvisor版本到0.33.0时候新添加进来的,可能因为这些都是deprecated,所以没有注册到kubelet中。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
// These flags were also implicit from cadvisor, but are actually used by something in the core repo:
	// TODO(mtaufen): This one is stil used by our salt, but for heaven's sake it's even deprecated in cadvisor
	register(global, local, "docker_root")
	// e2e node tests rely on this
	register(global, local, "housekeeping_interval")

	// These flags were implicit from cadvisor, and are mistakes that should be registered deprecated:
	const deprecated = "This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed."

	registerDeprecated(global, local, "application_metrics_count_limit", deprecated)
	registerDeprecated(global, local, "boot_id_file", deprecated)
	registerDeprecated(global, local, "container_hints", deprecated)
	registerDeprecated(global, local, "containerd", deprecated)
	registerDeprecated(global, local, "docker", deprecated)
	registerDeprecated(global, local, "docker_env_metadata_whitelist", deprecated)
	registerDeprecated(global, local, "docker_only", deprecated)
	registerDeprecated(global, local, "docker-tls", deprecated)
	registerDeprecated(global, local, "docker-tls-ca", deprecated)
	registerDeprecated(global, local, "docker-tls-cert", deprecated)
	registerDeprecated(global, local, "docker-tls-key", deprecated)
	registerDeprecated(global, local, "enable_load_reader", deprecated)
	registerDeprecated(global, local, "event_storage_age_limit", deprecated)
	registerDeprecated(global, local, "event_storage_event_limit", deprecated)
	registerDeprecated(global, local, "global_housekeeping_interval", deprecated)
	registerDeprecated(global, local, "log_cadvisor_usage", deprecated)
	registerDeprecated(global, local, "machine_id_file", deprecated)
	registerDeprecated(global, local, "storage_driver_user", deprecated)
	registerDeprecated(global, local, "storage_driver_password", deprecated)
	registerDeprecated(global, local, "storage_driver_host", deprecated)
	registerDeprecated(global, local, "storage_driver_db", deprecated)
	registerDeprecated(global, local, "storage_driver_table", deprecated)
	registerDeprecated(global, local, "storage_driver_secure", deprecated)
	registerDeprecated(global, local, "storage_driver_buffer_duration", deprecated)

CredentialProviderFlags

在cmd\kubelet\app\options\globalflags.go里匿名import

1
2
	// ensure libs have a chance to globally register their flags
	_ "k8s.io/kubernetes/pkg/credentialprovider/azure"

在pkg\credentialprovider\azure\azure_credentials.go定义

1
2
var flagConfigFile = pflag.String("azure-container-registry-config", "",
	"Path to the file containing Azure container registry configuration information.")

version

--version选项定义在staging\src\k8s.io\component-base\version\verflag\verflag.go,它接受三个值false (默认值)、true(不加参数默认为true)、raw(显示json格式的详细版本信息)。

后面文章专门介绍kubernetes的版本相关的发布规则、version的相关hack技术

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
func VersionVar(p *versionValue, name string, value versionValue, usage string) {
	*p = value
	flag.Var(p, name, usage)
	// "--version" will be treated as "--version=true"
	flag.Lookup(name).NoOptDefVal = "true"
}

func Version(name string, value versionValue, usage string) *versionValue {
	p := new(versionValue)
	VersionVar(p, name, value, usage)
	return p
}

const versionFlagName = "version"

var (
	versionFlag = Version(versionFlagName, VersionFalse, "Print version information and quit")
)

log选项

定义在staging\src\k8s.io\component-base\logs\logs.go

1
2
3
const logFlushFreqFlagName = "log-flush-frequency"

var logFlushFreq = pflag.Duration(logFlushFreqFlagName, 5*time.Second, "Maximum number of seconds between log flushes")
categories分类var变量flagdefault默认值description
kloglogDir–log-dir""If non-empty, write log files in this directory
kloglogFile–log-file""If non-empty, use this log file
kloglogFileMaxSizeMB–log-file-max-size1800Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited.
klogtoStderr–logtostderrtruelog to standard error instead of files
klogalsoToStderr–alsologtostderrfalselog to standard error as well as files
klogverbosity-v, –v0number for the log level verbosity
klogaddDirHeader–add-dir-headerfalseIf true, adds the file directory to the header
klogskipHeaders–skip-headersfalseIf true, avoid header prefixes in the log messages
klogskipLogHeaders–skip-log-headersfalseIf true, avoid headers when opening log files
klogstderrThreshold–stderrthreshold2logs at or above this threshold go to stderr
klogvmodule–vmodule""comma-separated list of pattern=N settings for file-filtered logging
klogtraceLocation–log-backtrace-at""when logging hits line file:N, emit a stack trace
cadvisordockerRootDirFlag–docker-root/var/lib/dockerDEPRECATED: docker root is read from docker info (this is a fallback, default: /var/lib/docker)
cadvisorArgDockerEndpoint–dockerunix:///var/run/docker.sockdocker endpoint
cadvisorArgDockerTLS–docker-tlsfalseuse TLS to connect to docker
cadvisorArgDockerCert–docker-tls-certcert.pempath to client certificate
cadvisorArgDockerKey–docker-tls-keykey.pempath to private key
cadvisorArgDockerCA–docker-tls-caca.pempath to trusted CA
cadvisordockerEnvWhitelist–docker-env-metadata-whitelist""a comma-separated list of environment variable keys that needs to be collected for docker containers
cadvisorArgContainerHints–container-hints/etc/cadvisor/container_hints.jsonlocation of the container hints file
cadvisorArgContainerdEndpoint–containerd/run/containerd/containerd.sockcontainerd endpoint
cadvisorArgContainerdNamespace–containerd-namespacek8s.iocontainerd namespace
cadvisordockerOnly–docker-onlyfalseOnly report docker containers in addition to root stats
cadvisordisableRootCgroupStats–disable-root-cgroup-statsfalseDisable collecting root Cgroup stats
cadvisormachineIdFilePath–machine-id-file/etc/machine-id,/var/lib/dbus/machine-idComma-separated list of files to check for machine-id. Use the first one that exists.
cadvisorbootIdFilePath–boot-id-file/proc/sys/kernel/random/boot_idComma-separated list of files to check for boot-id. Use the first one that exists.
cadvisorenableLoadReader–enable-load-readerfalseWhether to enable cpu load reader
cadvisorHousekeepingInterval–housekeeping-interval1 * time.SecondInterval between container housekeepings
cadvisorglobalHousekeepingInterval–global-housekeeping-interval1 * time.MinuteInterval between global housekeepings
cadvisorlogCadvisorUsage–log-cadvisor-usagefalseWhether to log the usage of the cAdvisor container
cadvisoreventStorageAgeLimit–event-storage-age-limitdefault=24hMax length of time for which to store events (per type). Value is a comma separated list of key values, where the keys are event types (e.g.: creation, oom) or "default" and the value is a duration. Default is applied to all non-specified event types
cadvisoreventStorageEventLimit–event-storage-event-limitdefault=100000Max number of events to store (per type). Value is a comma separated list of key values, where the keys are event types (e.g.: creation, oom) or "default" and the value is an integer. Default is applied to all non-specified event types
cadvisorapplicationMetricsCountLimit–application-metrics-count-limit100Max number of application metrics to store (per container)
cadvisorArgDbUsername–storage-driver-userrootdatabase username
cadvisorArgDbPassword–storage-driver-passwordrootdatabase password
cadvisorArgDbHost–storage-driver-hostlocalhost:8086database host:port
cadvisorArgDbName–storage-driver-dbcadvisordatabase name
cadvisorArgDbTable–storage-driver-tablestatstable name
cadvisorArgDbIsSecure–storage-driver-securefalseuse secure connection with database
cadvisorArgDbBufferDuration–storage-driver-buffer-duration60 * time.SecondWrites in the storage driver will be buffered for this duration, and committed to the non memory backends as a single transaction
CredentialProviderflagConfigFile–azure-container-registry-config""Path to the file containing Azure container registry configuration information.
versionversionFlag–versionfalsePrint version information and quit
loglogFlushFreq–log-flush-frequency5 * time.SecondMaximum number of seconds between log flushes

特殊的选项,就是显示命令的帮助信息,使用-h或–help。

1
cleanFlagSet.BoolP("help", "h", false, fmt.Sprintf("help for %s", cmd.Name()))

相关内容