环境准备
- 操作系统：ubuntu 16.04
- 虚拟机两台：Master(192.168.241.160)、Slave(192.168.241.161)
- 硬件配置：CPU≥2，内存≥2G
每个节点安装docker
curl -fsSL https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update
sudo apt-get install docker-ce
注意：如果提示curl: (1) Protocol "https" not supported，则需要重装curl：
git clone https://github.com/bagder/curl.git
sudo apt-get build-dep curl
cd curl
autoreconf -fi # 执行失败再执行一次
./configure
make
sudo make install
每个节点安装kubernetes
配置k8s更新源：
echo "deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list
sudo gpg --keyserver keyserver.ubuntu.com --recv-keys BA07F4FB #对安装包进行签名
sudo gpg --export --armor BA07F4FB | sudo apt-key add -
sudo apt-get update
关闭虚拟内存
sudo swapoff -a #暂时关闭
vi /etc/fstab #永久关闭，注释掉swap那一行，推荐永久关闭
安装最新版的k8s：
sudo apt-get install -y kubelet kubeadm kubectl kubernetes-cni
设置开机启动：
sudo systemctl enable kubelet && systemctl start kubelet
查看kubectl版本：
kubectl version
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.4", GitCommit:"d360454c9bc
d1634cf4cc52d1867af5491dc9c5f", GitTreeState:"clean", BuildDate:"2020-11-11T13:17:17Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"linux/amd64"}The connection to the server localhost:8080 was refused - did you specify the right host or port
?
配置k8s集群
配置网络，实现节点之间可以ping通，修改/etc/hosts文件：
192.168.241.160 master
192.168.241.161 slave
配置Master节点的k8s，并使用 kubeadm 拉取镜像
创建管理工具kubeadm对应配置文件
kubeadm config print init-defaults > kubeadm.conf
修改配置文件：
修改advertiseAddress和imageRepository、kubernetes-version
apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
token: abcdef.0123456789abcdef
ttl: 24h0m0s
usages:
- signing
- authentication
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.241.160
bindPort: 6443
nodeRegistration:
criSocket: /var/run/dockershim.sock
name: master
taints:
- effect: NoSchedule
key: node-role.kubernetes.io/master
---
apiServer:
timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: registry.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: v1.19.4
networking:
dnsDomain: cluster.local
podSubnet: 10.244.0.0/16
serviceSubnet: 10.96.0.0/12
scheduler: {}
可以使用命令查看需要下载的镜像：
> cd working
> kubeadm config images list --config kubeadm.conf
registry.aliyuncs.com/google_containers/kube-apiserver:v1.19.0
registry.aliyuncs.com/google_containers/kube-controller-manager:v1.19.0
registry.aliyuncs.com/google_containers/kube-scheduler:v1.19.0
registry.aliyuncs.com/google_containers/kube-proxy:v1.19.0
registry.aliyuncs.com/google_containers/pause:3.2
registry.aliyuncs.com/google_containers/etcd:3.4.13-0
registry.aliyuncs.com/google_containers/coredns:1.7.0
使用kubeadm init进行初始化操作
> sudo kubeadm init --config ./kubeadm.conf
sudo kubeadm reset -f命令用来清除文件，可以重新执行init命令。
初始化过程比较久，需要把最后的结果记下来：
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.241.160:6443 --token abcdef.0123456789abcdef \
--discovery-token-ca-cert-hash sha256:8cb3a7f37c1258dc5fccac3e82c716b72d066b313806b975e9b6f5
8b8b00e9b5
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
查看启动状况
> kubectl get nodes
NAME STATUS ROLES AGE VERSION
master Ready master 118m v1.19.4
> fwj@master:~$ kubectl get cs
Warning: v1 ComponentStatus is deprecated in v1.19+
NAME STATUS MESSAGE ERROR
scheduler Healthy ok
controller-manager Healthy ok
etcd-0 Healthy {"health":"true"}
现在只有一个master节点。
配置内部通信flannel网络(master和slave都要配)
wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
确保这个文件的配置是对的：
net-conf.json: |
{
"Network": "10.244.0.0/16",
"Backend": {
"Type": "vxlan"
}
}
应用当前配置文件
kubectl apply -f kube-flannel.yml
podsecuritypolicy.policy/psp.flannel.unprivileged created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds created
状态变为ready:
配置node节点
拷贝配置文件到每个node:
scp /etc/kubernetes/admin.conf fwj@slave:/home/fwj/
配置并加入节点：
mkdir -p $HOME/.kube
sudo cp -i $HOME/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
sudo kubeadm join 192.168.241.160:6443 --token abcdef.0123456789abcdef \
--discovery-token-ca-cert-hash sha256:8cb3a7f37c1258dc5fccac3e82c716b72d066b313806b975e9b6f58b8b00e9b5
查看node是否已经加入到k8s集群中(需要等一段时间才能ready):
> kubectl get nodes
NAME STATUS ROLES AGE VERSION
master Ready master 128m v1.19.4
slave Ready <none> 3m28s v1.19.4
通过yaml部署应用
编写配置文件mysql-rc.yaml：
apiVersion: v1
kind: ReplicationController
metadata:
name: mysql
spec:
replicas: 1 #Pod副本的期待数量
selector:
app: mysql #符合目标的Pod拥有此标签
template: #根据此模板创建Pod的副本（实例）
metadata:
labels:
app: mysql #Pod副本拥有的标签，对应RC的Selector
spec:
containers: #Pod内容器的定义部分
- name: mysql #容器的名称
image: hub.c.163.com/library/mysql #容器对应的Docker image
ports:
- containerPort: 3306 #容器应用监听的端口号
env: #注入容器内的环境变量
- name: MYSQL_ROOT_PASSWORD
value: "123456"
加载文件到集群中，等待几分钟等待docker下载完成。
> kubectl create -f mysql-rc.yaml
replicationcontroller/mysql created
查看pods：
> kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-gc4j8 1/1 Running 0 58s
查看容器：
> kubectl describe pod mysql
Name: mysql-gc4j8
Namespace: default
Priority: 0
Node: slave/192.168.241.161 # 可以看到部署节点
Start Time: Tue, 24 Nov 2020 17:53:50 +0800
Labels: app=mysql
Annotations: <none>
Status: Running
IP: 10.244.1.2
IPs:
IP: 10.244.1.2
Controlled By: ReplicationController/mysql
Containers:
mysql:
Container ID: docker://17008055bb0e95fda7c59eda46291298d2bff2138d7170d6306e0350816e1654
Image: hub.c.163.com/library/mysql
Image ID: docker-pullable://hub.c.163.com/library/mysql@sha256:b2bce1a792237ac5df78877
d583f34b09ab023a77130921a6bcce67ce2d24ff0 Port: 3306/TCP
Host Port: 0/TCP
State: Running
Started: Tue, 24 Nov 2020 17:54:48 +0800
Ready: True
Restart Count: 0
Environment:
MYSQL_ROOT_PASSWORD: 123456
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-hdb86 (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
default-token-hdb86:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-hdb86
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 3m default-scheduler Successfully assigned default/mysql-gc4j8 to slav
e Normal Pulling 2m59s kubelet Pulling image "hub.c.163.com/library/mysql"
Normal Pulled 2m3s kubelet Successfully pulled image "hub.c.163.com/library/
mysql" in 56.05480276s Normal Created 2m2s kubelet Created container mysql
Normal Started 2m2s kubelet Started container mysql
编写Tomcat配置文件mytomcat.yaml：
apiVersion: v1
kind: ReplicationController
metadata:
name: myweb
spec:
replicas: 3 #Pod副本的期待数量
selector:
app: myweb #符合目标的Pod拥有此标签
template: #根据此模板创建Pod的副本（实例）
metadata:
labels:
app: myweb #Pod副本拥有的标签，对应RC的Selector
spec:
containers: #Pod内容器的定义部分
- name: myweb #容器的名称
image: hub.c.163.com/library/tomcat:latest #容器对应的Docker image
ports:
- containerPort: 8080 #容器应用监听的端口号
env: #注入容器内的环境变量
- name: MYSQL_SERVICE_HOST
value: "mysql"
- name: MYSQL_SERVICE_PORT
value: "3306"
部署应用：
kubectl create -f mytomcat.yaml
注意：删除可以用：
kubectl delete -f mytomcat.yaml
编写服务配置文件myweb-svc.yaml：
apiVersion: v1
kind: Service
metadata:
name: myweb
spec:
type: NodePort
ports:
- port: 8080
nodePort: 30001
selector:
app: myweb
部署服务：
kubectl create -f myweb-svc.yaml
验证：
> kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-gc4j8 1/1 Running 0 3h36m
myweb-cx8f4 1/1 Running 0 9m37s
myweb-fn485 1/1 Running 0 9m37s
myweb-k8472 1/1 Running 0 9m37s
> kubectl get services
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 6h2m
myweb NodePort 10.108.222.188 <none> 8080:30001/TCP 42m
浏览器访问http://192.168.241.160:30001，可以看到Tomcat界面。
部署Dashboard
docker pull registry.aliyuncs.com/google_containers/kubernetes-dashboard:v2.0.4
docker tag registry.aliyuncs.com/google_containers/kubernetes-dashboard:v2.0.4 kubernetesui/dashboard:v2.0.4
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.4/aio/deploy/recommended.yaml
访问dashboard
方式一 设置NodePort：
暴露端口，修改type: ClusterIP->type: NodePort
kubectl -n kubernetes-dashboard edit service kubernetes-dashboard
查看开放的端口：
kubectl -n kubernetes-dashboard get service kubernetes-dashboard
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes-dashboard NodePort 10.99.234.105 <none> 11443:31211/TCP 13h
访问网址:https://192.168.241.160:31112即可打开面板。v1版本在Chrome会出现您的连接不是私密连接问题，v2版本则不会。
方式二 使用kubectl proxy：
使用kubectl proxy命令就可以使API server监听在本地的8001端口上 使用命令如下:
kubectl proxy --address='0.0.0.0' --accept-hosts='^*$' &
则在内网的任意节点浏览器中可以使用地址访问，只能本机访问
http://192.168.241.160:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/
创建service account并绑定默认cluster-admin管理员集群角色：
kubectl create serviceaccount dashboard-admin -n kubernetes-dashboard
kubectl create clusterrolebinding dashboard-admin --clusterrole=cluster-admin --serviceaccount=kubernetes-dashboard:dashboard-admin
kubectl -n kubernetes-dashboard describe secret $(kubectl -n kubernetes-dashboard get secret | grep dashboard-admin | awk '{print $1}')
token：
eyJhbGciOiJSUzI1NiIsImtpZCI6ImVzQVpUSThVdm1nV3RjZkplNTFOQ3ducUhZRm9kSEpOcDd5ZENDckRfVTAifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi11c2VyLXRva2VuLWh3OGhnIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiI3YzFjZmQ5MS1hYzE0LTQ5ZTYtODU2Mi1lYWU0NzNhNjE2ODIiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZXJuZXRlcy1kYXNoYm9hcmQ6YWRtaW4tdXNlciJ9.S4KoWTmOhlv1JL7NlhuRug5-pKQ6tEB6dDVVry6THUjfi46qmemnXvnQggCHwMS_csghvRYDBjxcEDYn6UojH56GILrOKbsxpkteg263VJ9Uc3iRtISEjvDYwZgOyZYhakA6Ex8F2MKAmrUJnsAMFNWhNt2FE0VKte4VsCec8iiYuwzIL5LtB-s8OfjIza0f0Dsqjz4n7iO06PsM8YxgLf1k1t5N2809UkFcWYeLqpznzXdgMn8Zus7VXk-tpJRl9MuS7lrsuI50gOLz2bPlkcsxfL4xZUHzD-LWyDtericgMDawRWMlvM9B301PsVexkjGd2Q4cuUCR4kBJpKKomQ
常见问题
1、kubectl get cs提示controller-manager Unhealthy Get “http://127.0.0.1:10252/healthz“: dial tcp 127.0.0.1:10252: connect: connection refused
解决：注释掉/etc/kubernetes/manifests下的kube-controller-manager.yaml和kube-scheduler.yaml的- – port=0。
2、通过谷歌浏览器访问dashboard的时候会有如下提示：您的连接不是私密连接
解决：
mkdir key && cd key
#生成证书
openssl genrsa -out dashboard.key 2048
openssl req -new -out dashboard.csr -key dashboard.key -subj '/CN=192.168.241.160'
openssl x509 -req -in dashboard.csr -signkey dashboard.key -out dashboard.crt
#删除原有的证书secret
kubectl delete secret kubernetes-dashboard-certs -n kube-system
#创建新的证书secret
kubectl create secret generic kubernetes-dashboard-certs --from-file=dashboard.key --from-file=dashboard.crt -n kube-system
#查看pod
kubectl get pod -n kube-system
#重启pod
kubectl delete pod <pod name> -n kube-system
3、token一直登陆不了
解决：token复制有问题，把里面的换行去掉。