kubeadm安装k8s

发表于 更新于 分类于 阅读次数： 本文字数： 6.6k 阅读时长 ≈ 6 分钟

安装

安装前提

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
设置主机名
$ hostnamectl set-hostname k8s-master
$ hostnamectl set-hostname k8s-node1
 
关闭防火墙：
$ systemctl stop firewalld
$ systemctl disable firewalld
 
关闭selinux：
$ sed -i 's/enforcing/disabled/' /etc/selinux/config
$ setenforce 0
 
关闭swap：
$ swapoff -a  $ 临时
$ vim /etc/fstab  $ 永久
 
添加主机名与IP对应关系（记得设置主机名）：
cat <<EOF > /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.220.100 k8s-master
192.168.220.101 k8s-node1
192.168.220.102 k8s-node1
EOF
 
将桥接的IPv4流量传递到iptables的链：【有一些ipv4的流量不能走iptables链【linux内核的一个过滤器，每个流量都会经过他，然后再匹配是否可进入当前应用进程去处理】，导致流量丢失】：
$ cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
 
$ sysctl --system

安装Docker
$ wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo
$ yum -y install docker-ce-18.06.1.ce-3.el7
$ systemctl enable docker && systemctl start docker
$ docker --version
Docker version 18.06.1-ce, build e68fc7a

docker 加速镜像配置
$ mkdir /etc/docker/
cat > /etc/docker/daemon.json <<EOF
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m", "max-file":"3"
  },
  "storage-driver": "overlay2",
  "storage-opts": [
    "overlay2.override_kernel_check=true"
  ]
}
EOF
 
# Restart docker.
systemctl daemon-reload
systemctl restart docker
 
##max-file=3，意味着一个容器有三个日志，分别是id+.json、id+1.json、id+2.json
 
尝试过修改kubelet的cgroup dirver(文件位置：/lib/systemd/system/kubelet.service.d/10-kubeadm.conf)，但是每次启动kubeadm时会被覆盖掉，导致kubelet启动失败，只能修改docker的cgroup dirver设置；
文档中docker驱动是systemd
 
参考：https://kubernetes.io/docs/setup/production-environment/container-runtimes/

添加源

1
2
3
4
5
6
7
8
9
10
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

修改docker cgroup

docker的Cgroup Driver和kubelet的Cgroup Driver不一致导致的，此处选择修改docker的和kubelet一致

1
2
3
4
5
6
7
8
9
docker info | grep Cgroup
Cgroup Driver: cgroupfs
编辑文件/usr/lib/systemd/system/docker.service

ExecStart=/usr/bin/dockerd --exec-opt native.cgroupdriver=systemd
systemctl daemon-reload
systemctl restart docker
 docker info | grep Cgroup
Cgroup Driver: systemd
1
yum install kubelet-1.16.9 kubeadm-1.16.9 kubectl-1.16.9
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
cat <<EOF > kubeadm-config 
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
kubernetesVersion: v1.16.9
imageRepository: registry.aliyuncs.com/google_containers
#controlPlaneEndpoint: 192.168.220.100:6443 # haproxy地址及端口
networking:
  dnsDomain: cluster.local
  podSubnet: 10.128.0.0/23
  serviceSubnet: 10.192.0.0/22
EOF


#高版本
--control-plane-endpoint xxxx:6443 #指定apiserver的域名和端口 用域名加端口就不用在后面去改证书
--upload-certs
–service-cidr #集群内部虚拟网络，指定Cluster IP的网段
1
kubeadm init --config=/root/kubeadm-config --node-name=192.168.220.100

安装Pod网络插件（CNI）

1
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

或者

1
kubectl apply -f https://docs.projectcalico.org/v3.8/getting-started/kubernetes/installation/hosted/kubernetes-datastore/calico-networking/1.7/calico.yaml

检查证书到期时间

1
kubeadm alpha certs check-expiration

续订全部证书

1
kubeadm alpha certs renew all

kubectl自动补全

1
2
3
4
source <(kubectl completion bash)
echo "source <(kubectl completion bash)" >> ~/.bashrc
#或者
echo "source <(kubectl completion bash)" >> /etc/profile

添加污点

1
2
kubectl taint nodes node1 key=value:NoSchedule
kubectl taint node 192.168.88.9 node-role.kubernetes.io=master:NoSchedule

删除污点

1
2
kubectl taint nodes kube11 key:NoSchedule-
kubectl taint node 192.168.3.180 node-role.kubernetes.io/master:NoSchedule-

添加节点

1
kubeadm token create --print-join-command

获取admin token

1
2
3
kubectl create serviceaccount kubeapps-operator
kubectl create clusterrolebinding kubeapps-operator --clusterrole=cluster-admin --serviceaccount=default:kubeapps-operator
kubectl get secret $(kubectl get serviceaccount kubeapps-operator -o jsonpath='{.secrets[].name}') -o jsonpath='{.data.token}' | base64 --decode

安装kubeapps

1
2
helm repo add bitnami https://charts.bitnami.com/bitnami
helm install --name kubeapps --namespace kubeapps bitnami/kubeapps

添加app源

1
helm repo add apphub https://apphub.aliyuncs.com/

添加helm源

1
2
3
4
helm repo remove stable
helm repo add stable http://mirror.azure.cn/kubernetes/charts/
helm repo add incubator http://mirror.azure.cn/kubernetes/charts-incubator/
helm repo update

自动化脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
#/bin/bash

 
# 关闭防火墙：
systemctl stop firewalld
systemctl disable firewalld
 
# 关闭selinux：
sed -i 's/enforcing/disabled/' /etc/selinux/config
setenforce 0
 
# 关闭swap：
swapoff -a  $ 临时
# vim /etc/fstab  $ 永久
 
 
# 将桥接的IPv4流量传递到iptables的链：
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
 
sysctl --system

# 安装Docker
yum clean all
yum makecache
wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo
yum -y install docker-ce-18.06.1.ce-3.el7
systemctl enable docker && systemctl start docker
docker --version


# docker 镜像配置
mkdir /etc/docker/ -p
cat > /etc/docker/daemon.json <<EOF
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m", "max-file":"3"
  },
  "registry-mirrors": ["https://aa0v400l.mirror.aliyuncs.com"],
  "storage-driver": "overlay2",
  "storage-opts": [
    "overlay2.override_kernel_check=true"
  ]
}
EOF
 
# Restart docker.
systemctl daemon-reload
systemctl restart docker

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
yum makecache
yum install kubelet-1.16.9 kubeadm-1.16.9 kubectl-1.16.9 -y

cat <<EOF > kubeadm-config 
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
kubernetesVersion: v1.16.9
imageRepository: registry.aliyuncs.com/google_containers
networking:
  dnsDomain: cluster.local
  podSubnet: 10.128.0.0/23
  serviceSubnet: 10.192.0.0/22
EOF

kubeadm init --config=/root/kubeadm-config --node-name=172.12.1.10