kubeadm快速搭建Kubernetes集群

博主：田晓勇
发布时间：2020 年 09 月 29 日
392次浏览
暂无评论
10469字数
分类： Kubernetes
百度已收录

环境准备

机器类型	操作系统	主机地址	部署规划
VM	CentOS Linux release 7.8.2003 (Core)	10.25.78.50	Master
VM	CentOS Linux release 7.8.2003 (Core)	10.25.78.100	Worker

部署步骤

注意： Master执行步骤1-10，Worker执行步骤1-8以及11。

步骤一：关闭防火墙

[root@localhost ~]# systemctl stop firewalld
[root@localhost ~]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
   Active: inactive (dead) since Tue 2020-09-29 12:46:30 CST; 25s ago
     Docs: man:firewalld(1)
  Process: 16368 ExecStart=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS (code=exited, status=0/SUCCESS)
 Main PID: 16368 (code=exited, status=0/SUCCESS)

...

步骤二：关闭selinux

[root@localhost ~]# getenforce
Enforcing
[root@localhost ~]# /usr/sbin/sestatus -v
SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             targeted
Current mode:                   enforcing
Mode from config file:          enforcing
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Max kernel policy version:      28

Process contexts:
Current context:                unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
Init context:                   system_u:system_r:init_t:s0
/usr/sbin/sshd                  unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

File contexts:
Controlling terminal:           unconfined_u:object_r:user_devpts_t:s0
/etc/passwd                     system_u:object_r:passwd_file_t:s0
/etc/shadow                     system_u:object_r:shadow_t:s0
/bin/bash                       system_u:object_r:shell_exec_t:s0
/bin/login                      system_u:object_r:login_exec_t:s0
/bin/sh                         system_u:object_r:bin_t:s0 -> system_u:object_r:shell_exec_t:s0
/sbin/agetty                    system_u:object_r:getty_exec_t:s0
/sbin/init                      system_u:object_r:bin_t:s0 -> system_u:object_r:init_exec_t:s0
/usr/sbin/sshd                  system_u:object_r:sshd_exec_t:s0

关闭 swap

此功能必须关闭！

临时关闭

[root@localhost ~]# swapoff -a
[root@localhost ~]# free
              total        used        free      shared  buff/cache   available
Mem:        8010580      332616     4431364       49956     3246600     7322028
Swap:             0           0           0

可以看到，执行free指令，显示swap这一项的各项指标都为0，代表已经关闭成功！

永久关闭（推荐）

[root@localhost ~]# vim /etc/fstab 
#
# /etc/fstab
# Created by anaconda on Mon Sep 28 04:19:44 2020
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/centos-root /                       xfs     defaults        0 0
UUID=500a3a94-5192-4583-950e-9f0cd7af56d4 /boot                   xfs     defaults        0 0
# 将下面这一行注释掉
# /dev/mapper/centos-swap swap                    swap    defaults        0 0

步骤四：添加域名映射

（一）修改`hosts`文件

[root@localhost ~]# cat >> /etc/hosts << EOF
> 10.25.78.50  k8s-master
> 10.25.78.100 k8s-worker-1
> EOF
[root@localhost ~]# cat /etc/hosts
# 如果是工作节点，将"k8s-master"修改为"k8s-worker-1"
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4 k8s-master
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

10.25.78.50  k8s-master
10.25.78.100 k8s-worker-1

（二）修改`hostname`

Master节点：

[root@localhost ~]# hostnamectl set-hostname k8s-master
[root@localhost ~]# hostname
k8s-master

Worker节点：

[root@localhost ~]# hostnamectl set-hostname k8s-worker-1
[root@localhost ~]# hostname
k8s-worker-1

步骤五：修改iptables配置

需要将桥接的IPV4流量传递到iptables 的链。

[root@localhost ~]# cat > /etc/sysctl.d/k8s.conf << EOF
> net.bridge.bridge-nf-call-ip6tables = 1
> net.bridge.bridge-nf-call-iptables = 1
> EOF
[root@localhost ~]# cat /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
[root@localhost ~]# sysctl --system
* Applying /usr/lib/sysctl.d/00-system.conf ...
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0
* Applying /usr/lib/sysctl.d/10-default-yama-scope.conf ...
...

步骤六：安装Docker

（一）执行安装

# step 1: 安装必要的一些系统工具
[root@localhost ~]# sudo yum install -y yum-utils device-mapper-persistent-data lvm2
# Step 2: 添加软件源信息
[root@localhost ~]# sudo yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
# Step 3: 更新并安装Docker-CE
[root@localhost ~]# sudo yum makecache fast
[root@localhost ~]# sudo yum -y install docker-ce
# Step 4: 开启Docker服务
[root@localhost ~]# systemctl enable docker && systemctl start docker
[root@localhost ~]# docker version
Client: Docker Engine - Community
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 17:03:45 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46d9d
  Built:            Wed Sep 16 17:02:21 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.3.7
  GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

（二）修改配置

此时docker已经完成安装，但是K8s中有很多镜像都在国外，为了从国内镜像托管站点活得镜像加速支持，建议修改Docker的配置文件，增加Registry Mirror参数，将镜像配置写入到配置参数中去。

新建Docker配置文件

[root@localhost ~]# vim /etc/docker/daemon.json

添加对应的配置信息

[root@localhost ~]# cat /etc/docker/daemon.json 
{
    "registry-mirrors": ["http://f1361db2.m.daocloud.io","https://fmyskxvt.mirror.aliyuncs.com","https://dockerhub.mirrors.nwafu.edu.cn","https://registry.docker-cn.com"],
    "exec-opts": ["native.cgroupdriver=systemd"]
}

重启Docker

[root@localhost ~]# systemctl daemon-relaod
[root@localhost ~]# systemctl restart docker

步骤七：添加阿里云yum源

[root@localhost ~]# cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

步骤八：安装 Kubernetes

由于官网未开放同步方式, 可能会有索引gpg检查失败的情况, 这时请用 yum install -y --nogpgcheck kubelet kubeadm kubectl 安装

[root@localhost ~]# yum install -y kubelet kubeadm kubectl
[root@localhost ~]# systemctl enable kubelet

步骤九：部署Master节点

此时kubeadm已经进入GA阶段，其控制面初始化和加入节点步骤都支持大量的可定制内容，因此kubeadm还提供了配置文件功能用于负载定制。同时，kubeadm将配置文件以ConfigMap的形式保存与集群中，便于后续的查询和升级工作。kubeadm config子命令提供了对这一组功能的支持：

指令	描述
kubeadm config upload from-file	由配置文件上传到集群中生成ConfigMap。
kubeadm config upload from-flags	由配置参数生成ConfigMap。
kubeadm config view	查看当前集群中的配置值。
kubeadm config print init-defaults	输出 `kubeadm init`默认参数文件的内容。
kubeadm config print join-defaults	输出 `kubeadm join`默认参数文件的内容。
kubeadm config migrate	在新旧版本之间进行配置转换。
kubeadm config images list	列出所需的镜像列表。
kubnadm config images pull	拉去镜像到本地。

（一）生成初始化配置文件

[root@localhost ~]# kubeadm config print init-defaults
[root@localhost ~]# ll
total 8
-rw-------. 1 root root 1285 Sep 28 04:22 anaconda-ks.cfg
-rw-r--r--. 1 root root  863 Sep 29 02:26 init-default.yaml

（二）修改初始化配置文件

[root@localhost ~]# cat init-default.yaml 
apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
# 将此处修复为当前vm的IP地址
  advertiseAddress: 10.25.78.50
  bindPort: 6443
nodeRegistration:
  criSocket: /var/run/dockershim.sock
  name: k8s-master
  taints:
  - effect: NoSchedule
    key: node-role.kubernetes.io/master
---
apiServer:
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:
  type: CoreDNS
etcd:
  local:
    dataDir: /var/lib/etcd
# 将此处修改为国内阿里云镜像仓库的地址
imageRepository: registry.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: v1.19.0
networking:
  dnsDomain: cluster.local
  serviceSubnet: 10.96.0.0/12
scheduler: {}

（三）执行初始化Master节点

建议至少2 cpu ,2G，非硬性要求，1cpu，1G也可以搭建起集群。但是：1个cpu的话初始化master的时候会报the number of available CPUs 1 is less than the required 2，部署插件或者pod时可能会报warning：FailedScheduling：Insufficient cpu, Insufficient memory

[root@localhost ~]# kubeadm init --config=init-default.yaml 
W0929 02:26:52.411295   19019 configset.go:348] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
[init] Using Kubernetes version: v1.19.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [k8s-master kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 10.25.78.50]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [k8s-master localhost] and IPs [10.25.78.50 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [k8s-master localhost] and IPs [10.25.78.50 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 13.501901 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.19" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node k8s-master as control-plane by adding the label "node-role.kubernetes.io/master=''"
[mark-control-plane] Marking the node k8s-master as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: abcdef.0123456789abcdef
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 10.25.78.50:6443 --token abcdef.0123456789abcdef \
    --discovery-token-ca-cert-hash sha256:7e6bded86e1fd37f7578942a0889f536e41a5eb22ff0b59753c42ad486adc933

这里需要注意kubeadm init命令执行完成后的最后几行提示信息，其中包含了加入节点的指令（kubeadm join ...）以及所需的Token。

（四）执行后续操作

这里的操做都是执行完kubeadm init系统提示我们要做的动作，如上所示。

[root@localhost ~]# mkdir -p $HOME/.kube
[root@localhost ~]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@localhost ~]# sudo chown $(id -u):$(id -g) $HOME/.kube/config

（五）验证安装

此时已经在Master上安装了Kubernetes，但是集群内部还没有可以使用的工作Node，而且缺乏对容器网络的配置，此时可以验证当前的系统层面的Pod和ConfigMap。

[root@localhost ~]# kubectl get pod -n kube-system
NAME                                 READY   STATUS    RESTARTS   AGE
coredns-6d56c8448f-fnrhj             0/1     Pending   0          21m
coredns-6d56c8448f-rdp7k             0/1     Pending   0          21m
etcd-k8s-master                      1/1     Running   0          21m
kube-apiserver-k8s-master            1/1     Running   0          21m
kube-controller-manager-k8s-master   1/1     Running   0          21m
kube-proxy-zjfct                     1/1     Running   0          21m
kube-scheduler-k8s-master            1/1     Running   0          21m
[root@localhost ~]# kubectl get cm -n kube-system
NAME                                 DATA   AGE
coredns                              1      25m
extension-apiserver-authentication   6      25m
kube-proxy                           2      25m
kubeadm-config                       2      25m
kubelet-config-1.19                  1      25m

可以看到在ConfigMap中生成了名为kubeadm-config的ConfigMap对象。

步骤十：安装网络插件

执行kubectl get node命令，会发现Kubernetes提示Master为NotReady状态，这是因为还没有安装CNI网络插件。

[root@localhost ~]# kubectl get node
NAME         STATUS     ROLES    AGE   VERSION
k8s-master   NotReady   master   27m   v1.19.2

接下来，我们根据kubeadm的提示安装CNI网路插件即可。对与CNI网络插件，可以由很多选择，请参考https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#pod-network的详细说明。这里我们选择weave插件，执行下面一条命令即可一键完成安装。

[root@k8s-worker-1 ~]# kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubelet version | base64 | tr -d '\n')"

步骤十一：部署Worker节点

[socde type="green"]前提： 此时已经将步骤1-8执行完成。[/scode]

（一）生成默认`join-config`初始化配置文件

[root@k8s-worker-1 ~]# kubeadm config print join-defaults > join-default.yaml
[root@k8s-worker-1 ~]# ll
total 8
-rw-------. 1 root root 1285 Sep 28 04:22 anaconda-ks.cfg
-rw-r--r--. 1 root root  863 Sep 29 02:26 join-default.yaml

（二）根据实际情况修改配置文件

[root@k8s-worker-1 ~]# cat join-default.yaml 
apiVersion: kubeadm.k8s.io/v1beta2
caCertPath: /etc/kubernetes/pki/ca.crt
discovery:
  bootstrapToken:
  # 将此处修改为Master节点的IP地址。
    apiServerEndpoint: 10.25.78.50:6443
    token: abcdef.0123456789abcdef
    unsafeSkipCAVerification: true
  timeout: 5m0s
  tlsBootstrapToken: abcdef.0123456789abcdef
kind: JoinConfiguration
nodeRegistration:
  criSocket: /var/run/dockershim.sock
  name: k8s-worker-1
  taints: null

（三）执行`kubeadm join`命令

[root@k8s-worker-1 ~]# kubeadm join --config=join-default.yaml
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

出现以上结果说明已经成功将节点加入集群中。

步骤十二：验证Kubernetes集群是否安装完成

执行下面命令，验证Kubernetes集群相关的Pod是否都正常的创建并运行。

[root@localhost ~]# kubectl get pods --all-namespaces
NAMESPACE     NAME                                 READY   STATUS    RESTARTS   AGE
kube-system   coredns-6d56c8448f-fnrhj             1/1     Running   0          91m
kube-system   coredns-6d56c8448f-rdp7k             1/1     Running   0          91m
kube-system   etcd-k8s-master                      1/1     Running   0          19m
kube-system   kube-apiserver-k8s-master            1/1     Running   0          20m
kube-system   kube-controller-manager-k8s-master   1/1     Running   0          19m
kube-system   kube-proxy-qjpz9                     1/1     Running   0          16m
kube-system   kube-proxy-zjfct                     1/1     Running   0          91m
kube-system   kube-scheduler-k8s-master            1/1     Running   0          20m
kube-system   weave-net-4drlg                      2/2     Running   1          16m
kube-system   weave-net-p4ssv                      2/2     Running   0          24m

此时可以看到，所有的Pod都是运行良好的；但是如果出现状态错误的Pod，则可以执行kubectl --namespace=kube-system describe pod <Pod_name>来查看具体的错误原因，最为常见的错误就是镜像没有下载完成。
也可以查看下Kubernetes集群下当前节点的状态。

[root@localhost ~]# kubectl get nodes
NAME           STATUS   ROLES    AGE   VERSION
k8s-master     Ready    master   96m   v1.19.2
k8s-worker-1   Ready    <none>   22m   v1.19.2

至此，通过kubeadm工具就实现了快速搭建Kubernetes集群；如果安装失败则可以执行kubeadm reset进行重置，将主机恢复原状后再次进行安装。

问题解决

安装失败怎么办？

如果使用kubeadm init或者kubeadm join命令安装出现问题，解决问题后别忘记使用kubeadm reset来还原主机配置。

[root@localhost ~]# kubeadm reset
# 重置网络配置
[root@localhost ~]# rm -rf /etc/cni/net.d
# 重置iptables
[root@localhost ~]# iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X
[root@localhost ~]# sysctl net.bridge.bridge-nf-call-iptables=1
# 手动执行以下命令来清除对应的残余网卡信息
[root@localhost ~]# sudo ip link del cni0
[root@localhost ~]# sudo ip link del flannel.1
# 删除 $HOME/.kube/config，清除当前的Kubernetes配置
[root@localhost ~]# rm -rf $HOME/.kube/config

最后修改：2020 年 10 月 12 日 10 : 22 PM

如果觉得我的文章对你有用，请随意赞赏

发表评论取消回复

评论 *

私密评论

名称 *

邮箱 *

地址

乌拉拉
OωO好棒呀！！！
fank
ヾ(≧∇≦*)ゝ云服务器代理，有需要可以联系，qq 1...
你哥
你这么吊，我同意了吗？
吴承恩
啊啊啊

kubeadm快速搭建Kubernetes集群

田晓勇 • 2020 年 09 月 29 日

环境准备

机器类型	操作系统	主机地址	部署规划
VM	CentOS Linux release 7.8.2003 (Core)	10.25.78.50	Master
VM	CentOS Linux release 7.8.2003 (Core)	10.25.78.100	Worker

部署步骤

注意： Master执行步骤1-10，Worker执行步骤1-8以及11。

步骤一：关闭防火墙

[root@localhost ~]# systemctl stop firewalld
[root@localhost ~]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
   Active: inactive (dead) since Tue 2020-09-29 12:46:30 CST; 25s ago
     Docs: man:firewalld(1)
  Process: 16368 ExecStart=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS (code=exited, status=0/SUCCESS)
 Main PID: 16368 (code=exited, status=0/SUCCESS)

...

步骤二：关闭selinux

[root@localhost ~]# getenforce
Enforcing
[root@localhost ~]# /usr/sbin/sestatus -v
SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             targeted
Current mode:                   enforcing
Mode from config file:          enforcing
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Max kernel policy version:      28

Process contexts:
Current context:                unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
Init context:                   system_u:system_r:init_t:s0
/usr/sbin/sshd                  unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

File contexts:
Controlling terminal:           unconfined_u:object_r:user_devpts_t:s0
/etc/passwd                     system_u:object_r:passwd_file_t:s0
/etc/shadow                     system_u:object_r:shadow_t:s0
/bin/bash                       system_u:object_r:shell_exec_t:s0
/bin/login                      system_u:object_r:login_exec_t:s0
/bin/sh                         system_u:object_r:bin_t:s0 -> system_u:object_r:shell_exec_t:s0
/sbin/agetty                    system_u:object_r:getty_exec_t:s0
/sbin/init                      system_u:object_r:bin_t:s0 -> system_u:object_r:init_exec_t:s0
/usr/sbin/sshd                  system_u:object_r:sshd_exec_t:s0

关闭 swap

此功能必须关闭！

临时关闭

[root@localhost ~]# swapoff -a
[root@localhost ~]# free
              total        used        free      shared  buff/cache   available
Mem:        8010580      332616     4431364       49956     3246600     7322028
Swap:             0           0           0

可以看到，执行free指令，显示swap这一项的各项指标都为0，代表已经关闭成功！

永久关闭（推荐）

[root@localhost ~]# vim /etc/fstab 
#
# /etc/fstab
# Created by anaconda on Mon Sep 28 04:19:44 2020
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/centos-root /                       xfs     defaults        0 0
UUID=500a3a94-5192-4583-950e-9f0cd7af56d4 /boot                   xfs     defaults        0 0
# 将下面这一行注释掉
# /dev/mapper/centos-swap swap                    swap    defaults        0 0

步骤四：添加域名映射

（一）修改`hosts`文件

[root@localhost ~]# cat >> /etc/hosts << EOF
> 10.25.78.50  k8s-master
> 10.25.78.100 k8s-worker-1
> EOF
[root@localhost ~]# cat /etc/hosts
# 如果是工作节点，将"k8s-master"修改为"k8s-worker-1"
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4 k8s-master
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

10.25.78.50  k8s-master
10.25.78.100 k8s-worker-1

（二）修改`hostname`

Master节点：

[root@localhost ~]# hostnamectl set-hostname k8s-master
[root@localhost ~]# hostname
k8s-master

Worker节点：

[root@localhost ~]# hostnamectl set-hostname k8s-worker-1
[root@localhost ~]# hostname
k8s-worker-1

步骤五：修改iptables配置

需要将桥接的IPV4流量传递到iptables 的链。

[root@localhost ~]# cat > /etc/sysctl.d/k8s.conf << EOF
> net.bridge.bridge-nf-call-ip6tables = 1
> net.bridge.bridge-nf-call-iptables = 1
> EOF
[root@localhost ~]# cat /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
[root@localhost ~]# sysctl --system
* Applying /usr/lib/sysctl.d/00-system.conf ...
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0
* Applying /usr/lib/sysctl.d/10-default-yama-scope.conf ...
...

步骤六：安装Docker

（一）执行安装

# step 1: 安装必要的一些系统工具
[root@localhost ~]# sudo yum install -y yum-utils device-mapper-persistent-data lvm2
# Step 2: 添加软件源信息
[root@localhost ~]# sudo yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
# Step 3: 更新并安装Docker-CE
[root@localhost ~]# sudo yum makecache fast
[root@localhost ~]# sudo yum -y install docker-ce
# Step 4: 开启Docker服务
[root@localhost ~]# systemctl enable docker && systemctl start docker
[root@localhost ~]# docker version
Client: Docker Engine - Community
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 17:03:45 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46d9d
  Built:            Wed Sep 16 17:02:21 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.3.7
  GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

（二）修改配置

新建Docker配置文件

[root@localhost ~]# vim /etc/docker/daemon.json

添加对应的配置信息

[root@localhost ~]# cat /etc/docker/daemon.json 
{
    "registry-mirrors": ["http://f1361db2.m.daocloud.io","https://fmyskxvt.mirror.aliyuncs.com","https://dockerhub.mirrors.nwafu.edu.cn","https://registry.docker-cn.com"],
    "exec-opts": ["native.cgroupdriver=systemd"]
}

重启Docker

[root@localhost ~]# systemctl daemon-relaod
[root@localhost ~]# systemctl restart docker

步骤七：添加阿里云yum源

[root@localhost ~]# cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

步骤八：安装 Kubernetes

由于官网未开放同步方式, 可能会有索引gpg检查失败的情况, 这时请用 yum install -y --nogpgcheck kubelet kubeadm kubectl 安装

[root@localhost ~]# yum install -y kubelet kubeadm kubectl
[root@localhost ~]# systemctl enable kubelet

步骤九：部署Master节点

指令	描述
kubeadm config upload from-file	由配置文件上传到集群中生成ConfigMap。
kubeadm config upload from-flags	由配置参数生成ConfigMap。
kubeadm config view	查看当前集群中的配置值。
kubeadm config print init-defaults	输出 `kubeadm init`默认参数文件的内容。
kubeadm config print join-defaults	输出 `kubeadm join`默认参数文件的内容。
kubeadm config migrate	在新旧版本之间进行配置转换。
kubeadm config images list	列出所需的镜像列表。
kubnadm config images pull	拉去镜像到本地。

（一）生成初始化配置文件

[root@localhost ~]# kubeadm config print init-defaults
[root@localhost ~]# ll
total 8
-rw-------. 1 root root 1285 Sep 28 04:22 anaconda-ks.cfg
-rw-r--r--. 1 root root  863 Sep 29 02:26 init-default.yaml

（二）修改初始化配置文件

[root@localhost ~]# cat init-default.yaml 
apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
# 将此处修复为当前vm的IP地址
  advertiseAddress: 10.25.78.50
  bindPort: 6443
nodeRegistration:
  criSocket: /var/run/dockershim.sock
  name: k8s-master
  taints:
  - effect: NoSchedule
    key: node-role.kubernetes.io/master
---
apiServer:
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:
  type: CoreDNS
etcd:
  local:
    dataDir: /var/lib/etcd
# 将此处修改为国内阿里云镜像仓库的地址
imageRepository: registry.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: v1.19.0
networking:
  dnsDomain: cluster.local
  serviceSubnet: 10.96.0.0/12
scheduler: {}

（三）执行初始化Master节点

[root@localhost ~]# kubeadm init --config=init-default.yaml 
W0929 02:26:52.411295   19019 configset.go:348] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
[init] Using Kubernetes version: v1.19.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [k8s-master kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 10.25.78.50]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [k8s-master localhost] and IPs [10.25.78.50 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [k8s-master localhost] and IPs [10.25.78.50 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 13.501901 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.19" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node k8s-master as control-plane by adding the label "node-role.kubernetes.io/master=''"
[mark-control-plane] Marking the node k8s-master as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: abcdef.0123456789abcdef
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 10.25.78.50:6443 --token abcdef.0123456789abcdef \
    --discovery-token-ca-cert-hash sha256:7e6bded86e1fd37f7578942a0889f536e41a5eb22ff0b59753c42ad486adc933

这里需要注意kubeadm init命令执行完成后的最后几行提示信息，其中包含了加入节点的指令（kubeadm join ...）以及所需的Token。

（四）执行后续操作

这里的操做都是执行完kubeadm init系统提示我们要做的动作，如上所示。

[root@localhost ~]# mkdir -p $HOME/.kube
[root@localhost ~]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@localhost ~]# sudo chown $(id -u):$(id -g) $HOME/.kube/config

（五）验证安装

此时已经在Master上安装了Kubernetes，但是集群内部还没有可以使用的工作Node，而且缺乏对容器网络的配置，此时可以验证当前的系统层面的Pod和ConfigMap。

[root@localhost ~]# kubectl get pod -n kube-system
NAME                                 READY   STATUS    RESTARTS   AGE
coredns-6d56c8448f-fnrhj             0/1     Pending   0          21m
coredns-6d56c8448f-rdp7k             0/1     Pending   0          21m
etcd-k8s-master                      1/1     Running   0          21m
kube-apiserver-k8s-master            1/1     Running   0          21m
kube-controller-manager-k8s-master   1/1     Running   0          21m
kube-proxy-zjfct                     1/1     Running   0          21m
kube-scheduler-k8s-master            1/1     Running   0          21m
[root@localhost ~]# kubectl get cm -n kube-system
NAME                                 DATA   AGE
coredns                              1      25m
extension-apiserver-authentication   6      25m
kube-proxy                           2      25m
kubeadm-config                       2      25m
kubelet-config-1.19                  1      25m

可以看到在ConfigMap中生成了名为kubeadm-config的ConfigMap对象。

步骤十：安装网络插件

执行kubectl get node命令，会发现Kubernetes提示Master为NotReady状态，这是因为还没有安装CNI网络插件。

[root@localhost ~]# kubectl get node
NAME         STATUS     ROLES    AGE   VERSION
k8s-master   NotReady   master   27m   v1.19.2

[root@k8s-worker-1 ~]# kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubelet version | base64 | tr -d '\n')"

步骤十一：部署Worker节点

[socde type="green"]前提： 此时已经将步骤1-8执行完成。[/scode]

（一）生成默认`join-config`初始化配置文件

[root@k8s-worker-1 ~]# kubeadm config print join-defaults > join-default.yaml
[root@k8s-worker-1 ~]# ll
total 8
-rw-------. 1 root root 1285 Sep 28 04:22 anaconda-ks.cfg
-rw-r--r--. 1 root root  863 Sep 29 02:26 join-default.yaml

（二）根据实际情况修改配置文件

[root@k8s-worker-1 ~]# cat join-default.yaml 
apiVersion: kubeadm.k8s.io/v1beta2
caCertPath: /etc/kubernetes/pki/ca.crt
discovery:
  bootstrapToken:
  # 将此处修改为Master节点的IP地址。
    apiServerEndpoint: 10.25.78.50:6443
    token: abcdef.0123456789abcdef
    unsafeSkipCAVerification: true
  timeout: 5m0s
  tlsBootstrapToken: abcdef.0123456789abcdef
kind: JoinConfiguration
nodeRegistration:
  criSocket: /var/run/dockershim.sock
  name: k8s-worker-1
  taints: null

（三）执行`kubeadm join`命令

[root@k8s-worker-1 ~]# kubeadm join --config=join-default.yaml
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

出现以上结果说明已经成功将节点加入集群中。

步骤十二：验证Kubernetes集群是否安装完成

执行下面命令，验证Kubernetes集群相关的Pod是否都正常的创建并运行。

[root@localhost ~]# kubectl get pods --all-namespaces
NAMESPACE     NAME                                 READY   STATUS    RESTARTS   AGE
kube-system   coredns-6d56c8448f-fnrhj             1/1     Running   0          91m
kube-system   coredns-6d56c8448f-rdp7k             1/1     Running   0          91m
kube-system   etcd-k8s-master                      1/1     Running   0          19m
kube-system   kube-apiserver-k8s-master            1/1     Running   0          20m
kube-system   kube-controller-manager-k8s-master   1/1     Running   0          19m
kube-system   kube-proxy-qjpz9                     1/1     Running   0          16m
kube-system   kube-proxy-zjfct                     1/1     Running   0          91m
kube-system   kube-scheduler-k8s-master            1/1     Running   0          20m
kube-system   weave-net-4drlg                      2/2     Running   1          16m
kube-system   weave-net-p4ssv                      2/2     Running   0          24m

[root@localhost ~]# kubectl get nodes
NAME           STATUS   ROLES    AGE   VERSION
k8s-master     Ready    master   96m   v1.19.2
k8s-worker-1   Ready    <none>   22m   v1.19.2

至此，通过kubeadm工具就实现了快速搭建Kubernetes集群；如果安装失败则可以执行kubeadm reset进行重置，将主机恢复原状后再次进行安装。

问题解决

安装失败怎么办？

如果使用kubeadm init或者kubeadm join命令安装出现问题，解决问题后别忘记使用kubeadm reset来还原主机配置。

[root@localhost ~]# kubeadm reset
# 重置网络配置
[root@localhost ~]# rm -rf /etc/cni/net.d
# 重置iptables
[root@localhost ~]# iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X
[root@localhost ~]# sysctl net.bridge.bridge-nf-call-iptables=1
# 手动执行以下命令来清除对应的残余网卡信息
[root@localhost ~]# sudo ip link del cni0
[root@localhost ~]# sudo ip link del flannel.1
# 删除 $HOME/.kube/config，清除当前的Kubernetes配置
[root@localhost ~]# rm -rf $HOME/.kube/config

环境准备

部署步骤

步骤一：关闭防火墙

步骤二：关闭selinux

关闭 swap

临时关闭

永久关闭（推荐）

步骤四：添加域名映射

（一）修改hosts文件

（二）修改hostname

步骤五：修改iptables配置

步骤六：安装Docker

（一）执行安装

（二）修改配置

步骤七：添加阿里云yum源

步骤八：安装 Kubernetes

步骤九：部署Master节点

（一）生成初始化配置文件

（二）修改初始化配置文件

（三）执行初始化Master节点

（四）执行后续操作

（五）验证安装

步骤十：安装网络插件

步骤十一：部署Worker节点

（一）生成默认join-config初始化配置文件

（二）根据实际情况修改配置文件

（三）执行kubeadm join命令

步骤十二：验证Kubernetes集群是否安装完成

问题解决

安装失败怎么办？

发表评论 取消回复

kubeadm快速搭建Kubernetes集群

环境准备

部署步骤

步骤一：关闭防火墙

步骤二：关闭selinux

关闭 swap

临时关闭

永久关闭（推荐）

步骤四：添加域名映射

（一）修改hosts文件

（二）修改hostname

步骤五：修改iptables配置

步骤六：安装Docker

（一）执行安装

（二）修改配置

步骤七：添加阿里云yum源

步骤八：安装 Kubernetes

步骤九：部署Master节点

（一）生成初始化配置文件

（二）修改初始化配置文件

（三）执行初始化Master节点

（四）执行后续操作

（五）验证安装

步骤十：安装网络插件

步骤十一：部署Worker节点

（一）生成默认join-config初始化配置文件

（二）根据实际情况修改配置文件

（三）执行kubeadm join命令

步骤十二：验证Kubernetes集群是否安装完成

问题解决

安装失败怎么办？

（一）修改`hosts`文件

（二）修改`hostname`

（一）生成默认`join-config`初始化配置文件

（三）执行`kubeadm join`命令

发表评论取消回复

（一）修改`hosts`文件

（二）修改`hostname`

（一）生成默认`join-config`初始化配置文件

（三）执行`kubeadm join`命令