First of all, I would like to thank the karmada community for providing the tickets to KubeCon, and I met with core contributors and maintainers of karmada such as zhen chang, hongcai Ren, and Wei jiang.

In previous years, attending the technology conference left me with no deep impression and no harvest. This time I forced myself to record it, deepen my impression and summarize my gains.

I am interested in colocation, so the sharing I listened to is basically related to this.

updated: all of the kubeCon china 2023 videos is released. youtube video list, WeChat subscription account

slides: https://kccncosschn2023.sched.com/?iframe=no

istioCon china 2023 slides: https://istioconchina2023.sched.com/ https://github.com/cloudnativeto/academy/tree/master/istiocon-china-2023

DNS servers are typically stable components of infrastructure and are rarely changed. However, if the IP address of a DNS server needs to be updated, here’s how to change the DNS configuration of Kubernetes nodes.

1 Standard Procedure

To change the node’s DNS configuration, follow these steps:

1. Replace the DNS server IP address directly in the /etc/resolv.conf file on node.

2. The above step updates only the /etc/resolv.conf file on the node. It does not update the /etc/resolv.conf inside the Node Local DNS and CoreDNS pods.

1 Karmada: What is it?

Karmada is a Kubernetes multi-cluster management system. It allows resources to be distributed across multiple clusters while maintaining the original way of using the API server. It offers features such as cross-cloud multi-cluster management, multi-policy multi-cluster scheduling, cross-cluster fault tolerance for applications, a global unified resource view, multi-cluster service discovery, and Federated Horizontal Pod Autoscaling (FederatedHPA) capabilities. Its design philosophy inherits from Cluster Federation v2 and is currently an open-source CNCF sandbox project. For more details, visit the official website at https://karmada.io/.

1 Background

Use kubebuilder to develop a vpa-related operator. This operator will watch all vpa creation, deletion and update in the cluster. controller-runtime provides predict to filter out unnecessary events, and use predicate.GenerationChangedPredicate to filter out vpa update status. However, it was found that the status update of vpa (recommended value updated by vpa-recommender) also triggered Reconcile.

Encountered Strange Phenomenon: Spark-generated job pods are consistently scheduled on the same node, meaning that pods from different jobs are all being scheduled to the same node. This results in an uneven distribution of pods, even though the nodes have no taints, and their resource availability is similar. The jobs do not have any nodeSelector, nodeAffinity, nodeName, or PodTopologySpread.

Introduction to VPA

VPA, short for Vertical Pod Autoscaler, is an open-source implementation based on the Google paper Autopilot: Workload Autoscaling at Google Scale. It recommends container resource requests based on historical monitoring data from the containers within pods. In other words, VPA scales by directly modifying the resource requests (and limits, if configured in VPA resources) within the pod.

Key Benefits:

1. Increases node resource utilization.
2. Suitable for long-running, homogeneous applications.

Limitations:

Kubernetes provides a method to modify the configuration of the /etc/resolv.conf file for pods using the spec.dnsConfig and spec.dnsPolicy fields. You can find specific information on this in the Customizing DNS Service documentation. However, this approach leads to the recreation of pods.

In our specific business scenario, we need pods to use local DNS instead of the centralized CoreDNS, even for pods created before the change in cluster DNS configuration. We need to update the nameserver for these existing pods to point to the local DNS server. However, we cannot actively delete pods or restart containers. This practice is not considered ideal in container usage, but it aligns with our company culture, as the business application doesn’t support graceful termination.