1.dashboard概述
- Dashboard是基于网页的Kubernetes用户界面。您可以使用Dashboard将容器应用部署到Kubernetes集群中，也可以对容器应用排错，还能管理集群资源。您可以使用Dashboard获取运行在集群中的应用的概览信息，也可以创建或者修改Kubernetes资源( 如Deployment，Job,DaemonSet等等)
- Dashboard同时展示了Kubernetes集群中的资源状态信息和所有报错信息
官网：
https://github.com/kubernetes/dashboard
metrics插件：
dashboard资源文件
https://github.com/kubernetes/dashboard0
2.部署metrics-server
1)开启apiserver聚合服务（中继路由）
# 添加参数 ~]# vim /etc/kubernetes/manifests/kube-apiserver.yaml #apiserver启动文件 # spec.containers.command 最下面手动添加如下一行(注意对对齐，否则所有查询会报错) - --enable-aggregator-routing=true ...... ~]# systemctl restart kubelet # 验证 ~]# kubectl -n kube-system get pod kube-apiserver-master -o yaml |grep enable-aggregator-routing
2）证书的申请与签发
linux的用户k8s的用户的用户区别
linux用户存储在password
k8s用户，证书的签发机构，签发证书可信（在k8s内只记录一小时，一小时后删除记录，但是只要k8s签发过的就可以在k8s内用），所以在k8s内有用户的概念，但是不能查找用户，只要用户带有签发的证书就可以登录k8s；所以node节点想要查询就要有证书
metrics使用证书，为kubelet签发证书
# 注：如果没有证书是无法查看节点状态的 # 在文件var/lib/kubelet/config.yaml的配置文件中添加 ~]# vim /var/lib/kubelet/config.yaml serverTLSBootstrap: true # 重启kubelet服务，等待几分钟就能看到了;包括master在内的所有节点都需要做 # 默认情况下是没有开启证书签发的（在master操作） ~]# kubectl get csr No resources found # 申请证书示例（在所有节点操作） ~]# vim /var/lib/kubelet/config.yaml # 在文件的最后一行添加申请证书；注：改错节点服务会起不来 serverTLSBootstrap: true # 重启kubelet（在所有节点操作） ~]# systemctl restart kubelet # 查看（master操作） ~]# kubectl get csr NAME AGE SIGNERNAME REQUESTOR REQUESTEDDURATION CONDITION csr-6fbtl 91s kubernetes.io/kubelet-serving system:node:node1 <none> Pending csr-bctqw 11m kubernetes.io/kubelet-serving system:node:master <none> Pending csr-bz6hx 2m14s kubernetes.io/kube-apiserver-client-kubelet system:node:vm-0-114-centos <none> Pending csr-gk7tx 62s kubernetes.io/kubelet-serving system:node:vm-0-142-centos <none> Pending # ----------------签发证书必须在 master 上执行 ------------------------- # 看到Pending的状态我们就可以签发证书了（每台都签发一下） ~]# kubectl get csr | awk '{print $1}'| grep -v NAME| xargs -i kubectl certificate approve {} ~]# kubectl get csr NAME AGE SIGNERNAME REQUESTOR REQUESTEDDURATION CONDITION csr-6fbtl 91s kubernetes.io/kubelet-serving system:node:node1 <none> Approved,Issued csr-bctqw 11m kubernetes.io/kubelet-serving system:node:master <none> Approved,Issued csr-bz6hx 2m14s kubernetes.io/kube-apiserver-client-kubelet system:node:vm-0-114-centos <none> Approved,Issued csr-gk7tx 62s kubernetes.io/kubelet-serving system:node:vm-0-142-centos <none> Approved,Issued # Approved,Issued则是签发成功 # 申请的多余证书可以使用 （kubectl delete certificatesigningrequests 证书名称） 删除；如果误删了证书删除也不需要再申请，因为已经有记录了 2.安装mertics插件 1）官网 GitHub - kubernetes-sigs/metrics-server: Scalable and efficient source of container resource metrics for Kubernetes built-in autoscaling pipelines. 2）下载镜像和资源文件，并导入私有仓库 rbac.yaml 授权控制器 pdb.yaml 中断控制器 deployment.yaml 主进程metrics（修改镜像地址） service.yaml 后端metrics主进程服务 apiservice.yaml 注册集群API ## 先导入镜像然后操作（这里我找了一些资料暂时没找到国内的镜像，所以如果要安装部署metrics的话要跨网） ~]# docker load -i metrisc-server.tar.gz ~]# docker tag gcr.io/k8s-staging-metrics-server/metrics-server:master 172.17.0.98:5000/metrics-server:master ~]# docker push 172.17.0.98:5000/metrics-server:master
3)安装控制器
## 第一步安装授权控制器 # 手动书写文件或官方下载 ~]# vim rbac.yaml --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:aggregated-metrics-reader labels: rbac.authorization.k8s.io/aggregate-to-view: "true" rbac.authorization.k8s.io/aggregate-to-edit: "true" rbac.authorization.k8s.io/aggregate-to-admin: "true" rules: - apiGroups: ["metrics.k8s.io"] resources: ["pods", "nodes"] verbs: ["get", "list", "watch"] --- apiVersion: v1 kind: ServiceAccount metadata: name: metrics-server namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: metrics-server-auth-reader namespace: kube-system roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: extension-apiserver-authentication-reader subjects: - kind: ServiceAccount name: metrics-server namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: metrics-server:system:auth-delegator roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:auth-delegator subjects: - kind: ServiceAccount name: metrics-server namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:metrics-server rules: - apiGroups: - "" resources: - pods - nodes - nodes/stats - namespaces - configmaps verbs: - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:metrics-server roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:metrics-server subjects: - kind: ServiceAccount name: metrics-server namespace: kube-system # 安装授权控制器 ~]# kubectl apply -f rbac.yaml clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created serviceaccount/metrics-server created rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created clusterrole.rbac.authorization.k8s.io/system:metrics-server created clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created ## 第二步安装中断控制器 # 官方下载文件或手动书写 ~]# vim pdb.yaml --- apiVersion: policy/v1beta1 kind: PodDisruptionBudget metadata: name: metrics-server namespace: kube-system labels: k8s-app: metrics-server spec: minAvailable: 100% selector: matchLabels: k8s-app: metrics-server # 安装中断控制器 ~]# kubectl apply -f pdb.yaml Warning: policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget poddisruptionbudget.policy/metrics-server created ## 第三安装主进程 # 官方下载文件或手动书写 ~]# vim deployment.yaml --- apiVersion: apps/v1 kind: Deployment metadata: name: metrics-server namespace: kube-system labels: k8s-app: metrics-server spec: selector: matchLabels: k8s-app: metrics-server strategy: rollingUpdate: maxUnavailable: 0 template: metadata: name: metrics-server labels: k8s-app: metrics-server spec: serviceAccountName: metrics-server volumes: # mount in tmp so we can safely use from-scratch images and/or read-only containers - name: tmp-dir emptyDir: {} containers: - name: metrics-server image: gcr.io/k8s-staging-metrics-server/metrics-server:master imagePullPolicy: IfNotPresent args: - --cert-dir=/tmp - --secure-port=4443 - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname - --kubelet-use-node-status-port # - --kubelet-insecure-tls ports: - name: main-port containerPort: 4443 protocol: TCP readinessProbe: httpGet: path: /healthz port: main-port scheme: HTTPS securityContext: readOnlyRootFilesystem: true runAsNonRoot: true runAsUser: 1000 volumeMounts: - name: tmp-dir mountPath: /tmp nodeSelector: kubernetes.io/os: linux # 安装主进程 ~]# kubectl apply -f deployment.yaml deployment.apps/metrics-server created ## 第四安装service服务 # 官方下载文件或手动书写 ~]# vim service.yaml --- apiVersion: v1 kind: Service metadata: name: metrics-server namespace: kube-system labels: kubernetes.io/name: "Metrics-server" kubernetes.io/cluster-service: "true" spec: selector: k8s-app: metrics-server ports: - port: 443 protocol: TCP targetPort: main-port # 装service服务 ~]# kubectl apply -f service.yaml service/metrics-server created ## 最后安装集群api # 官方下载文件或手动书写 ~]# vim apiservice.yaml --- apiVersion: apiregistration.k8s.io/v1beta1 kind: APIService metadata: name: v1beta1.metrics.k8s.io spec: service: name: metrics-server namespace: kube-system group: metrics.k8s.io version: v1beta1 insecureSkipTLSVerify: true groupPriorityMinimum: 100 versionPriority: 100 # 安装集群api ~]# kubectl apply -f apiservice.yaml apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created #-------------------------------- 查询验证 ---------------------------------------------- ~]# kubectl -n kube-system get pod NAME READY STATUS RESTARTS AGE metrics-server-78dfb54777-4dcjl 1/1 Running 0 116s ~]# kubectl -n kube-system get apiservices | grep metrics NAME SERVICE AVAILABLE AGE v1beta1.metrics.k8s.io kube-system/metrics-server True 2m20s # 如果安装成功了就可以查看节点资源了 ~]# kubectl top node NAME CPU(cores) CPU% MEMORY(bytes) MEMORY% master 95m 4% 840Mi 48% node-0001 24m 1% 266Mi 15% node-0002 24m 1% 270Mi 15% node-0003 26m 1% 280Mi 16% #CPU(cores)：毫核 1000分之95秒(把每个执行的周期分为1000，那么一个周期就是95毫秒) #CPU% CPU / 内核数：（两内核就是2000） # HPA就是根据资源占用情况来做的 ## 监控容器的示例 # 安装之前写的myapache创建一个容器 ~]# kubectl apply -f myapache.yaml deployment.apps/myapache created # 查看pod容器的资源 ~]# kubectl top pod error: metrics not available yet # 等待大约 30 秒 ~]# kubectl top pod NAME CPU(cores) MEMORY(bytes) myapache-7d689bf8f-lfr5h 1m 6Mi # 设置访问次数来实验pod ~]# curl http://10.244.2.17/info.html?id=5000000 <pre> Array ( [REMOTE_ADDR] => 10.244.0.0 [REQUEST_METHOD] => GET [HTTP_USER_AGENT] => curl/7.29.0 [REQUEST_URI] => /info.php?id=5000000 [id] => 5000000 ) php_host: myapache-7d689bf8f-lfr5h ~]# kubectl top pod NAME CPU(cores) MEMORY(bytes) myapache-7d689bf8f-w4rtt 1000m 8Mi
3.搭建dashboard
1）搭建并访问dashboard
# 书写资源文件或者官网下载（下载只要改镜像就可以了） ~]# docker pull kubernetesui/dashboard:v2.0.0 ~]# docker tag kubernetesui/dashboard:v2.0.0 172.17.0.98:5000/dashboard:v2.0.0 ~]# vim recommended.yaml # Copyright 2017 The Kubernetes Authors. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. apiVersion: v1 kind: Namespace metadata: name: kubernetes-dashboard --- apiVersion: v1 kind: ServiceAccount metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboard --- kind: Service apiVersion: v1 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboard spec: ports: - port: 443 targetPort: 8443 selector: k8s-app: kubernetes-dashboard --- apiVersion: v1 kind: Secret metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard-certs namespace: kubernetes-dashboard type: Opaque --- apiVersion: v1 kind: Secret metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard-csrf namespace: kubernetes-dashboard type: Opaque data: csrf: "" --- apiVersion: v1 kind: Secret metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard-key-holder namespace: kubernetes-dashboard type: Opaque --- kind: ConfigMap apiVersion: v1 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard-settings namespace: kubernetes-dashboard --- kind: Role apiVersion: rbac.authorization.k8s.io/v1 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboard rules: # Allow Dashboard to get, update and delete Dashboard exclusive secrets. - apiGroups: [""] resources: ["secrets"] resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs", "kubernetes-dashboard-csrf"] verbs: ["get", "update", "delete"] # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map. - apiGroups: [""] resources: ["configmaps"] resourceNames: ["kubernetes-dashboard-settings"] verbs: ["get", "update"] # Allow Dashboard to get metrics. - apiGroups: [""] resources: ["services"] resourceNames: ["heapster", "dashboard-metrics-scraper"] verbs: ["proxy"] - apiGroups: [""] resources: ["services/proxy"] resourceNames: ["heapster", "http:heapster:", "https:heapster:", "dashboard-metrics-scraper", "http:dashboard-metrics-scraper"] verbs: ["get"] --- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard rules: # Allow Metrics Scraper to get metrics from the Metrics server - apiGroups: ["metrics.k8s.io"] resources: ["pods", "nodes"] verbs: ["get", "list", "watch"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboard roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: kubernetes-dashboard subjects: - kind: ServiceAccount name: kubernetes-dashboard namespace: kubernetes-dashboard --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: kubernetes-dashboard roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: kubernetes-dashboard subjects: - kind: ServiceAccount name: kubernetes-dashboard namespace: kubernetes-dashboard --- kind: Deployment apiVersion: apps/v1 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboard spec: replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: k8s-app: kubernetes-dashboard template: metadata: labels: k8s-app: kubernetes-dashboard spec: containers: - name: kubernetes-dashboard image: 172.17.0.98：5000/dashboard:v2.0.0 # 改这为私有仓库名 imagePullPolicy: Always ports: - containerPort: 8443 protocol: TCP args: - --auto-generate-certificates - --namespace=kubernetes-dashboard # Uncomment the following line to manually specify Kubernetes API server Host # If not specified, Dashboard will attempt to auto discover the API server and connect # to it. Uncomment only if the default does not work. # - --apiserver-host=http://my-address:port volumeMounts: - name: kubernetes-dashboard-certs mountPath: /certs # Create on-disk volume to store exec logs - mountPath: /tmp name: tmp-volume livenessProbe: httpGet: scheme: HTTPS path: / port: 8443 initialDelaySeconds: 30 timeoutSeconds: 30 securityContext: allowPrivilegeEscalation: false readOnlyRootFilesystem: true runAsUser: 1001 runAsGroup: 2001 volumes: - name: kubernetes-dashboard-certs secret: secretName: kubernetes-dashboard-certs - name: tmp-volume emptyDir: {} serviceAccountName: kubernetes-dashboard nodeSelector: "kubernetes.io/os": linux # Comment the following tolerations if Dashboard must not be deployed on master tolerations: - key: node-role.kubernetes.io/master effect: NoSchedule --- kind: Service apiVersion: v1 metadata: labels: k8s-app: dashboard-metrics-scraper name: dashboard-metrics-scraper namespace: kubernetes-dashboard spec: ports: - port: 8000 targetPort: 8000 selector: k8s-app: dashboard-metrics-scraper --- kind: Deployment apiVersion: apps/v1 metadata: labels: k8s-app: dashboard-metrics-scraper name: dashboard-metrics-scraper namespace: kubernetes-dashboard spec: replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: k8s-app: dashboard-metrics-scraper template: metadata: labels: k8s-app: dashboard-metrics-scraper annotations: seccomp.security.alpha.kubernetes.io/pod: 'runtime/default' spec: containers: - name: dashboard-metrics-scraper image: 172.17.0.98/metrics-scraper:v1.0.4 # 改这 ports: - containerPort: 8000 protocol: TCP livenessProbe: httpGet: scheme: HTTP path: / port: 8000 initialDelaySeconds: 30 timeoutSeconds: 30 volumeMounts: - mountPath: /tmp name: tmp-volume securityContext: allowPrivilegeEscalation: false readOnlyRootFilesystem: true runAsUser: 1001 runAsGroup: 2001 serviceAccountName: kubernetes-dashboard nodeSelector: "kubernetes.io/os": linux # Comment the following tolerations if Dashboard must not be deployed on master tolerations: - key: node-role.kubernetes.io/master effect: NoSchedule volumes: - name: tmp-volume emptyDir: {}
# 创建容器 ~]# kubectl apply -f recommended.yaml ~]# kubectl -n kubernetes-dashboard get service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) dashboard-metrics-scraper ClusterIP 10.254.76.85 <none> 8000/TCP kubernetes-dashboard ClusterIP 10.254.211.125 <none> 443/TCP ~]# kubectl -n kubernetes-dashboard get pod NAME READY STATUS RESTARTS AGE dashboard-metrics-scraper-57bf85fcc9-vsz74 1/1 Running 0 52s kubernetes-dashboard-7b7f78bcf9-5k8vq 1/1 Running 0 52s
注：测试只能在集群内访问（ClusterIP），这时需要创建service实现外部访问
# 该资源文件可以当上面的（recommended.yaml）文件中复制 ~]# vim service.yaml --- kind: Service apiVersion: v1 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboard spec: ports: - port: 443 nodePort: 30443 # 新添加 targetPort: 8443 selector: k8s-app: kubernetes-dashboard type: NodePort # 新添加 ~]# kubectl apply -f service.yaml ~]# kubectl -n kubernetes-dashboard get service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE dashboard-metrics-scraper ClusterIP 10.254.66.25 <none> 8000/TCP 2m6s kubernetes-dashboard NodePort 10.254.165.155 <none> 443:30443/TCP 2m6s
公网访问只要在云上为 node 节点绑定弹性公网IP [ https://弹性公网IP:30443/ ]
登录
此时需要授权才能用
2）权限查询
role： 名称空间的系统权限
clusterrole：全局权限
系统权限
# 查询某一个名称空间内的所有系统权限 ~]# kubectl -n kube-system get role NAME CREATED AT extension-apiserver-authentication-reader 2022-01-22T08:33:50Z kube-proxy 2022-01-22T08:33:52Z kubeadm:kubelet-config-1.23 2022-01-22T08:33:51Z kubeadm:nodes-kubeadm-config 2022-01-22T08:33:51Z system::leader-locking-kube-controller-manager 2022-01-22T08:33:50Z system::leader-locking-kube-scheduler 2022-01-22T08:33:50Z system:controller:bootstrap-signer 2022-01-22T08:33:50Z system:controller:cloud-provider 2022-01-22T08:33:50Z system:controller:token-cleaner 2022-01-22T08:33:50Z # 查询名称空间内的系统权限的详细信息 ~]# kubectl -n kube-system describe role kube-proxy Name: kube-proxy Labels: <none> Annotations: <none> PolicyRule: Resources Non-Resource URLs Resource Names Verbs --------- ----------------- -------------- ----- configmaps [] [kube-proxy] [get]
全局权限
# 查看所有全局权限 ~]# kubectl -n kube-system get clusterrole # 指定查看某一个权限的详细信息 ~]# kubectl -n kube-system describe clusterrole admin
3）token认证登录
# 书写权限资源 ~]# vim admin-token.yaml --- apiVersion: v1 kind: ServiceAccount metadata: name: admin-user namespace: kubernetes-dashboard --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: admin-user roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: - kind: ServiceAccount name: admin-user namespace: kubernetes-dashboard ~]# kubectl apply -f admin-token.yaml ~]# kubectl -n kubernetes-dashboard get secrets NAME TYPE DATA AGE admin-user-token-bxjlz kubernetes.io/service-account-token 3 23s ~]# kubectl -n kubernetes-dashboard describe secrets admin-user-token-bxjlz Name: admin-user-token-bxjlz ... ... ca.crt: 1025 bytes namespace: 20 bytes token: ..... # 这里这个很长的字符串就是你要找的认证 token
把密码粘贴并点击登录
4）使用dashboard
# 创建一个错误的apache容器 ~]# vim myapache.yaml --- kind: Deployment apiVersion: apps/v1 metadata: name: myapache annotations: kubernetes.io/change-cause: httpd.v1 spec: selector: matchLabels: myapp: httpd replicas: 2 template: metadata: labels: myapp: httpd spec: containers: - name: webcluster image: 172.17.0.98:5000/myos:http # 故意把镜像改错 stdin: false tty: false ports: - protocol: TCP containerPort: 80 restartPolicy: Always ~]# kubectl apply -f myapache.yaml ~]# kubectl get pods NAME READY STATUS RESTARTS AGE myapache-7c4546fdd-b76d7 0/1 ImagePullBackOff 0 45s myapache-7c4546fdd-nkv4b 0/1 ContainerCreating 0 45s
登录查看
查看报错
修改错误（修改时修改资源控制器）
查看
修改副本数
版权声明：本文为kali_yao原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接和本声明。