Kubernetes在非云环境中的配置和运行（系列教程第六篇）：主节点和工作节点

摘要：

作者： Marcos Vallim

正文：

本文将逐一介绍Kubernetes主节点和工作节点的各个组件，包括控制器管理（Controller Manager）、API服务器、etcd、调度器（Scheduler）、Kubelet等。强烈建议读者扩展阅读参考链接内容，以更好地理解每个组件的工作机制，以及它们在Kubernetes集群中的作用，

在前一篇文章中，我们介绍了Kubernetes，概述了其主要组件。这些文章构成了Kubernetes系列教程。希望读者有兴趣深入了解Kubernetes在非云环境中的安装和配置。

想要先睹为快的读者，不必等待所有文章更新，可直接克隆该项目的GitHub代码库。代码库中的文档正在持续改进中，并完全可用。代码库地址为：mvallim/kubernetes-under-the-hood

本教程适用于那些规划安装Kubernetes集群并希望理解各组件工作机制的读者。

主节点（Master）

主节点负责编排容器相关的所有活动，这些容器运行在工作节点（Worker）上。在各项任务中，主节点调度和部署集群应用，并采集工作节点和Pods的信息。
Masters are responsible for orchestrating all activities related to the containers that run on the worker nodes. It is responsible for scheduling and deploying clustered applications and collecting information about worker nodes and Pods, among many other activities.

主节点的配置方法 Some approaches for configuring Master nodes

使用etcd节点的堆叠控制平面（Stacked control plane），

此配置方法中，服务作为容器运行，由kubeadm自动配置。In this approach, the services run as containers and are automatically set up by kubeadm.

堆叠高可用集群的拓扑如下图所示。其中，etcd提供分布式数据存储集群，并堆叠在由运行控制平面组件的kubeadm管理节点所构成的集群之上。
A stacked HA cluster is a topology (see the image below) where the distributed data storage cluster provided by etcd is stacked on top of the cluster formed by the nodes managed by kubeadm that run control plane components.

每个控制平面节点均运行API服务器（api-server）、调度器（scheduler）和控制器管理（controller-manager）进程。其中，API服务器进程通过负载均衡器提供给工作节点使用（在此，使用的负载均衡器是HA Proxy），并创建本地etcd成员。该本地成员只与运行在同一节点上的API服务器进程通信。类似地，调度器和控制器管理进程也采用同样机制。

这种拓扑实现了控制平面和etc成员在运行节点上的耦合，相比于外部etcd节点而言更易于建立、管理和复制。
This topology couples the control planes and etcd members on the same node where they run. It is simpler to set up than a cluster with external etcd nodes, and simpler to manage for replication.

但是，堆叠集群存在耦合失败的风险。如果一个节点宕机，那么会丢失etcd成员和控制平面进程。这时需要增加冗余度，通过添加更多的控制平面降低此类风险。
However, a stacked cluster runs into the risk of failed coupling. If one node goes down, both an etcd member and a control plane instance are lost, and redundancy is compromised. You can mitigate this risk by adding more control plane nodes.

由此建议对于高可用集群，至少运行三个堆叠控制平面节点。
You should therefore run a minimum of three stacked control plane nodes for an HA cluster.

kubeadm高可用拓扑：堆叠etcd

参考链接：https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/ha-topology/

使用外部etcd节点的堆叠控制平面

在此配置方法中，服务作为容器运行，由kubeadm部分配置。In this approach, the services run as containers and are partially configured by kubeadm.

使用外部etcd节点的高可用集群拓扑如下图所示。其中，由etcd提供的分布式数据存储集群，独立于由运行控制平面组件的节点所组成的集群。
An HA cluster with external etcd nodes is a topology (see the image below) where the distributed data storage cluster provided by etcd is external to the cluster formed by the nodes that run control plane components.

类似于堆叠etcd拓扑，在外部etcd拓扑中，每个控制平面节点均运行API服务器（api-server）、调度器（scheduler）和控制器管理（controller-manager）进程。其中，API服务器进程通过负载均衡器提供给工作节点使用。但是，etcd成员运行在独立的机器上，每个etcd主机与每个控制平面节点的API服务器通信。
Like in the stacked etcd topology, each control plane node in an external etcd topology runs an instance of the api-server, scheduler, and controller-manager. And the api-server is exposed to worker nodes using a load balancer. However, etcd members run on separate hosts, and each etcd host communicates with the api-server of each control plane node.

这种拓扑解耦了控制平面和etc成员。这样的设置相对与堆叠高可用拓扑而言，控制平面或etc成员的宕机对集群冗余性的影响甚微。
This topology decouples the control plane and etcd member. It therefore provides an HA setup where losing a control plane instance or an etcd member has less impact and does not affect the cluster redundancy as much as the stacked HA topology.

但是，该拓扑相比于堆叠高可用拓扑所需的主机数量增加了一倍。最小需要三台控制平面节点主机、三台etcd节点主机。
However, this topology requires twice the number of hosts as the stacked HA topology. A minimum of three hosts for control plane nodes and three hosts for etcd nodes are required for an HA cluster with this topology.

kubeadm高可用拓扑：外部etcd

参考链接：https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/ha-topology/

使用外部etcd节点的控制平面服务， Control plane services and external etcd nodes

在此配置方法中，服务作为独立进程运行，不使用kubeadm而采用手工配置。该方法更为灵活，但是在建立集群中需要人工做更多工作。In this approach, the services run as standalone processes and should be manually configured, without using kubeadm. It provides more flexibility but also demands more work to be done by the person setting up the cluster.

使用外部etcd节点的高可用集群控制平面的拓扑如下图所示。其中，其中，由etcd提供的分布式数据存储集群，独立于由运行控制平面组件的节点所组成的集群。
An HA cluster Control Plane with external etcd nodes is a topology (see the image below) where the distributed data storage cluster provided by etcd is external to the cluster formed by the nodes that run control plane components.

类似于使用外部etcd节点的堆叠控制平面拓扑，在该拓扑中，每个控制平面节点均运行API服务器（api-server）、调度器（scheduler）和控制器管理（controller-manager）进程。其中，API服务器进程通过负载均衡器提供给工作节点使用。etcd成员运行在独立主机上，每个etcd主机与与每个控制平面节点的API服务器通信。
Like in the stacked control plane and external etcd nodes topology, each control plane node in an external etcd topology runs an instance of the api-server, scheduler, and controller-manager. And the api-server is exposed to worker nodes using a load balancer. etcd members run on separate hosts, and each etcd host communicates with all api-server of the control plane nodes.

该拓扑在同一节点上以独立服务方式运行API服务器、调度器和控制器管理进程，而etcd采用单独节点运行。它提供的高可用设置相比于堆叠高可用拓扑而言，控制平面进程和etcd成员的影响甚微，并不影响集群冗余度。
This topology runs api-server, controller-manager and scheduler as standalone services in the same node, while etcd is ran on its own node. It therefore provides an HA setup where losing a control plane instance or an etcd member has less impact and does not affect the cluster redundancy as much as in the stacked HA topology.

但是，该拓扑相比于堆叠高可用拓扑所需的主机数量增加了一倍。最小需要三台控制平面节点主机、三台etcd节点主机。并且，各服务必须逐个手工安装和配置。
However, this topology requires twice the number of hosts as the stacked HA topology. A minimum of three hosts for control plane nodes and three hosts for etcd nodes are required for an HA cluster with this topology. Also, you must install and configure the services one-by-one.

使用外部etcd服务的Kubernetes控制平面

本教程使用的方法

推荐使用etcd节点的堆叠控制平面拓扑配置。因为此配置所需的手工配置最少，使用的服务实例也最少，

各组件概述

• kubeadm：该工具提供kubeadm Init和join命令，是创建Kubernetes集群最佳实践的一种“捷径”。kubeadm执行建立并运行最小可用集群所需的操作。在设计上考虑的是引导机器，而非配置机器。 同样，安装各种优良的附加组件（例如Kubernetes仪表板、监视解决方案和特定于云的附加组件）也不在kubeadm的考虑范围内。使用kubeadm可作为所有部署的基础，更轻松地创建一致的集群。A tool built to provide kubeadm init and kubeadm join as best-practice “fast paths” for creating Kubernetes clusters. kubeadm performs the actions necessary to get a minimum viable cluster up and running. By design, it cares only about bootstrapping, not about provisioning machines. Likewise, installing various nice-to-have addons, like the Kubernetes Dashboard, monitoring solutions, and cloud-specific addons, is not in scope. Using kubeadm as the basis of all deployments will make it easier to create conformant clusters.
• kubelet：运行在工作节点上的服务。kubelet读取Pod的Manifests文件，确保文件所定义的容器已启动并保持运行。: A service ran on the worker nodes that reads Pod manifests and ensures that their defined containers have been started and are running.
• etcd：一种一致性高可用键值存储，用于Kubernets所有集群节点的支撑存储。如果一个Kubernetes集群使用etcd作为支持存储，需确定对数据具有备份计划。Consistent and highly-available key-value store used as Kubernetes’ backing store for all cluster data. If your Kubernetes cluster uses etcd as its backing store, make sure you have a back up plan for those data.
• containerd：一种容器运行时，侧重于简单性、稳定性和可移植性。containerd作为驻留进程运行在Linux和Windows上，负责获取并存储容器镜像、执行容器、提供网络访问等功能。此系列文章中，使用了Docker。: A container runtime with an emphasis on simplicity, robustness and portability. containerd is a container runtime that runs as a daemon on Linux or Windows. containerd takes care of fetching and storing container images, executing containers, providing network access, and more. (In our solution we’ll use Docker for this purpose)
• API服务器（api-server）：主节点运行组件，提供Kubernetes API。它是Kubernetes控制平面的前端，设计上课水平扩展。也就是说，它通过部署更多实例实现扩展。: Component on the master that exposes the Kubernetes API. It is the front-end for the Kubernetes control plane. It is designed to scale horizontally — that is, it scales by deploying more instances.
• 控制器管理（controller-manager）：主节点上的组件，运行控制器。逻辑上看，每个控制器是一个独立进程。但为了降低复杂性，所有控制器编译为同一二进制程序，以一个进程运行。: Component on the master that runs controllers. Logically, each controller is a separate process but, to reduce complexity, they are all compiled into a single binary and run in a single process.
• 调度器（scheduler）：主节点组件，监控新创建并未分配工作节点的Pod，并选择运行Pod的工作节点。调度中考虑因素包括所需的单独和合计资源、硬件/软件/策略上的限制、亲和性和反亲和性（Affinity/Anti-Affinity）规范、数据本地性、工作负载间干扰和截至时刻等。: Component on the master that watches newly created Pods that have no worker node assigned, and selects a worker node for them to run into. Factors taken into account for scheduling decisions include individual and collective resource requirements, hardware/software/policy constraints, affinity and anti-affinity specifications, data locality, inter-workload interference and deadlines.

Pod创建流程（图片来自heptio.com）

• kube-proxy：运行在集群每个工作节点上的网络代理。kube-proxy负责转发请求，支持在一组后台函数间的TCP/UDP流和轮询调度TCP/UDP转发。: It is a network proxy that runs on each worker node in the cluster. kube-proxy is responsible for request forwarding. kube-proxy allows TCP and UDP stream forwarding or round-robin TCP and UDP forwarding across a set of backend functions.

• DNS集群扩展：Kubernets DNS在集群上调度DNS Pod和服务，并配置kubelets单个容器使用解析DNS名的DNS服务IP。 cluster add-on: Kubernetes DNS schedules a DNS Pod and Service on the cluster, and configures the kubelets to tell individual containers to use the DNS Service’s IP to resolve DNS names.

  集群中定义的每个服务（包括DNS服务器本身）将分配一个DNS名。默认情况下，客户Pod的DNS搜索包括Pod自身的命名空间，以及集群的默认域名。下面的例子给出了很好的解释：
  Every Service defined in the cluster (including the DNS server itself) is assigned a DNS name. By default, a client Pod’s DNS search list will include the Pod’s own namespace and the cluster’s default domain. This is best illustrated by example:

  给定在Kubernetes命名空间“bar”中的服务“foo”。在该命名空间中运行的Pod，可以通过基本的DNS查询“foo”，查找到该服务。在命名空间“quux”中运行的Pod，可以通过DNS查询“foo.bar”，查找到该服务。
  Assume a Service named foo in the Kubernetes namespace bar. A Pod running in namespace bar can look up this service by simply doing a DNS query for foo. A Pod running in namespace quux can look up this service by doing a DNS query for foo.bar.

• cni插件：是一类网络插件，符合appc/CNI规范。它支持连接到运行在不同节点上的Pod，提供集成不同类型网络解决方案（例如Overlay网络、完全第三层网络等）的灵活性。: This plugin is a type of Network plugin that adheres to the appc/CNI specification. This is what enables connecting Pods running on different nodes and flexibility to integrate different kind of network solutions (overlays, pure L3, etc).

  • 更多Kubernetes和CNI信息，参考官方文档“Network plugins”。

参考链接：
* https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/
* https://kubernetes.io/docs/reference/glossary/?fundamental=true
* https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#introduction

工作节点（Worker）

工作节点是由Kubernetes有效管理运行容器所在的机器（即节点，可以是物理的，也可以是虚拟机）。为了使工作节点被Kubernetes管理，节点必须安装Kubelet代理。通过Kubelet代理实现所有与主节点的通信，由此可执行所有集群操作。
Workers are the machines (nodes, which can be physical or VMs) where the containers managed by Kubernetes effectively run. In order for worker nodes to be managed by Kubernetes, they must have Kubelet agents installed on them. It is through this agent that all communication with the master happens and, as a consequence, the cluster operations are performed.

工作节点的配置方法 Some approaches for configuring Worker nodes

堆叠工作节点 Stacked worker nodes

在这种配置方法中，服务作为容器运行，由kubeadm自动配置。In this approach, the services run as containers and are automatically set up by kubeadm.

堆叠工作节点的拓扑如下图所示。其中，每个节点均运行kubelet、kube-proxy、cni-plugins和containerd进程。
A stacked worker is a topology (see image above) where each node runs an instance of kubelet, kube-proxy, cni-plugins, and containerd.

这种拓扑结构的工作节点易于配置。只需要安装kubeadm、kubelet和containerd。kube-proxy和cni插件等其它组件将在工作节点加入集群时初始化，即在运行kubeadm join命令后。
It is simpler to configure a worker in this topology. It is necessary to have only kubeadm, kubelet and containerd installed. The other components (kube-proxy and cni-plugins) will be initialized when the worker joins the cluster when the kubeadm join command is executed.

该配置方法中，kube-proxy和cni-plugins作为容器运行。
This approach runs kube-proxy and cni-plugins as containers.

工作节点服务

在这种配置方法中，服务作为独立进程运行，需要手工配置，无需使用kubeadm。更为灵活，但是需要建立集群者做更多的工作。In this approach, the services run as standalone processes and should be manually configured, without using kubeadm. It provides more flexibility but also demands more work to be done by the person setting up the cluster.

工作节点服务的拓扑如下图所示。其中，每个节点均运行kubelet、kube-proxy、cni-plugins和containerd进程。每个服务需依次安装和配置。
A worker service is a topology (see image above) where each node runs an instance of kubelet, kube-proxy, cni-plugins, and containerd. Also you must install and configure the services one-by-one.

该配置方法中，kube-proxy和cni-plugins作为独立服务运行。
This approach runs kube-proxy and cni-plugins as standalone services.

本教程使用的方法

使用堆叠工作节点，因为这需要的配置工作更少。

组件

kubelet、kube-proxy、cni-plugins和containerd等组件在主节点和工作节点上工作机制相同，具体定义如上所示。
.

小结

希望读者喜欢该文，以及系列教程中的其它文章。下一篇文章将深入介绍Kubernetes架构，解释每个组件是如何应用的。

希望读者喜欢该系列教程所有文章。在下一篇文章中，我们将深入etcd的技术细节，介绍其组成及相互交互。

稍安勿躁，该系列教程的理论部分即将在两到三篇文章内结束。下面将给出Kubernetes集群的实际操作。

欢迎在文章下面给出反馈和评论，这对于改进系列教程非常重要。

建议读者关注[作者的Medium账号](Marcos Vallim)，一览系列教程全貌，并第一时间掌握新文章的发布。

原文链接： Kubernetes Journey — Up and running out of the cloud — Master and Worker