k3s部署和管理k8s集群(demo)

一、部署
更多信息请参考: https://docs.rancher.cn/docs/k3s/installation/ha/_index

K3S简述：K3s (轻量级 Kubernetes)： 和 RKE 类似，也是经过认证的 Kubernetes 发行版。它比 RKE 更新，更易用且更轻量化，全部组件都在一个小于 100 MB 的二进制文件中。从 Rancher v2.4 开始，Rancher 可以安装在 K3s 集群上。

1、所有节点关闭防火墙

systemctl stop firewalld 
systemctl disable firewalld 
setenforce 0 sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config

2、db-lb节点安装docker，部署maraidb和外部nginx lb
1>安装docker、配置docker

yum -y install yum-utils device-mapper-persistent-data lvm2 yum-utils
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum -y install docker-ce

配置docker
mkdir -p /etc/docker
cat /etc/docker/daemon.json
{
  "registry-mirrors": [
    "https://docker.mirrors.ustc.edu.cn",
    "http://hub-mirror.c.163.com"
  ],
  "max-concurrent-downloads": 10,
  "log-driver": "json-file",
  "log-level": "warn",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
    },
  "data-root": "/var/lib/docker"
}

使配置生效
systemctl daemon-reload
systemctl restart docker
groupadd docker
useradd docker -g docker

2>安装mariadb和外部nginx lb
nginx.conf

worker_processes 4;
worker_rlimit_nofile 40000;

events {
    worker_connections 8192;
}

stream {
    upstream rancher_servers_http {
        least_conn;
        server 192.168.86.12:80 max_fails=3 fail_timeout=5s;
        server 192.168.86.13:80 max_fails=3 fail_timeout=5s;
    }
    server {
        listen 80;
        proxy_pass rancher_servers_http;
    }

    upstream rancher_servers_https {
        least_conn;
        server 192.168.86.12:443 max_fails=3 fail_timeout=5s;
        server 192.168.86.13:443 max_fails=3 fail_timeout=5s;
    }
    server {
        listen     443;
        proxy_pass rancher_servers_https;
    }

}

docker run -itd --restart=unless-stopped -p 80:80 -p 443:443 --name rancher-lb -v /root/nginx.conf:/etc/nginx/nginx.conf nginx:latest

#密码根据实际环境修改，也可以不用docker启动
docker run -itd --restart=always -p 3306:3306 --name rancherDB -v /var/lib/mysql  -e MYSQL_ROOT_PASSWORD=rancherDB mariadb

3、k3s所有节点执行（注意修改mysql连接地址和密码）

curl -sfL http://rancher-mirror.cnrancher.com/k3s/k3s-install.sh  | INSTALL_K3S_MIRROR=cn sh -s - server --datastore-endpoint="mysql://root:rancherDB@tcp(192.168.86.11:3306)/k3s"

4、执行完成后配置kubectl

mkdir ~/.kube
cat /etc/rancher/k3s/k3s.yaml >>~/.kube/config
kubectl get pod -o wide --all-namespaces
#等pod都起来再执行下一步

5、安装helm、rancher

 wget http://rancher-mirror.cnrancher.com/helm/v3.3.0/helm-v3.3.0-linux-amd64.tar.gz
 tar -xf helm-v3.3.0-linux-amd64.tar.gz
 mv helm /usr/bin/
 
helm repo add rancher-stable http://rancher-mirror.oss-cn-beijing.aliyuncs.com/server-charts/stable
helm repo add jetstack https://charts.jetstack.io

helm repo update


kubectl create namespace cattle-system
kubectl create namespace cert-manager

kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.15.0/cert-manager.crds.yaml
helm install cert-manager jetstack/cert-manager --namespace cert-manager --version v0.15.0

kubectl get pods --namespace cert-manager
#hostname是本地解析或者dns解析后访问入口，根据实际情况填写
helm install rancher rancher-stable/rancher --namespace cattle-system --set hostname=rancher.demo.com

pod都起来以后windos的hosts内lb的IP解析hostname，通过域名访问
192.168.86.11 rancher.demo.com

二、管理k8s集群
1、导入现有k8s集群




2、通过rancher自建集群


按照提示结合实际环境信息配置集群选项，然后下一步

FAQ
导入集群提示无法解析主机名，添加hosts解析后pod仍启动失败
解决办法：如果有内部dns使用内部dns没有就通过在在cattle-cluster-agent或cattle-node-agent中添加HostAliases解决
示例：

kubectl -n cattle-system patch  deployments cattle-cluster-agent --patch '{
    "spec": {
        "template": {
            "spec": {
                "hostAliases": [
                    {
                      "hostnames":
                      [
                        "rancher.demo.com"
                      ],
                      "ip": "192.168.86.11"
                    }
                ]
            }
        }
    }
}'

kubectl -n cattle-system patch  daemonsets cattle-node-agent --patch '{
 "spec": {
     "template": {
         "spec": {
             "hostAliases": [
                 {
                    "hostnames":
                      [
                        "rancher.demo.com"
                      ],
                    "ip": "192.168.86.11"
                 }
             ]
         }
     }
 }
}'