问题复现

[root@k8s-master kubernetes]# kubectl get pod -n kube-system -owide
NAME                                       READY   STATUS    RESTARTS   AGE     IP                NODE         NOMINATED NODE   READINESS GATES
calico-kube-controllers-647d84984b-v4cn8   1/1     Running   0          42m     10.244.195.130    k8s-node48   <none>           <none>
calico-node-8pznz                          0/1     Running   0          42m     192.168.171.73    k8s-node73   <none>           <none>
calico-node-bcr94                          1/1     Running   0          42m     192.168.169.226   k8s-master   <none>           <none>
calico-node-cb8qq                          1/1     Running   0          42m     192.168.171.48    k8s-node48   <none>           <none>
coredns-6d8c4cb4d-rmnkl                    1/1     Running   0          3h19m   10.244.235.193    k8s-master   <none>           <none>
coredns-6d8c4cb4d-wzjbz                    1/1     Running   0          3h19m   10.244.235.194    k8s-master   <none>           <none>
etcd-k8s-master                            1/1     Running   3          3h19m   192.168.169.226   k8s-master   <none>           <none>
kube-apiserver-k8s-master                  1/1     Running   3          3h19m   192.168.169.226   k8s-master   <none>           <none>

查看日志

[root@k8s-master01 ~]# kubectl describe pod calico-node-8pznz -n kube-system
···
···
umber of node(s) with BGP peering established = 1
calico/node is not ready: BIRD is not ready: BGP not established with 192.168.169.226,192.168.171.48

排查原因

• 进入到出现故障 pod，查看router id值发现绑定地址异常，并非宿主机的IP地址。因此宿主机上通过route无法看到正确的路由

  [root@master01 ~]# kubectl exec -ti calico-node-8pznz -n kube-system -- bash
  [root@k8s-master1 /]# cat /etc/calico/confd/config/bird.cfg
  ...
  # Generated by confd
  include "bird_aggr.cfg";
  include "bird_ipam.cfg";
  
  router id 172.18.0.1;
  
  # Configure synchronization between routing tables and kernel.
  ...
  

解决方法

调整calicao 网络插件的网卡发现机制，修改IP_AUTODETECTION_METHOD对应的value值。官方提供的yaml文件中，ip识别策略（IPDETECTMETHOD）没有配置，即默认为first-found，这会导致一个网络异常的ip作为nodeIP被注册，从而影响node-to-node mesh。我们可以修改成can-reach或者interface的策略，尝试连接某一个Ready的node的IP，以此选择出正确的IP。

• 因此需要修改calico.yaml中配置，添加两行，具体如下：

  [root@master01 ~]# vi calico.yaml
  # Cluster type to identify the deployment type
  - name: CLUSTER_TYPE
    value: "k8s,bgp"
  #新增两行配置
  - name: IP_AUTODETECTION_METHOD
    value: "interface=eth.*, enp.*"
  # Auto-detect the BGP IP address.
  - name: IP
    value: "autodetect"
  # Enable IPIP
  - name: CALICO_IPV4POOL_IPIP
    value: "Always"
  

• 说明：通过ifconfig发现集群中网卡名称有多个，因此配置成interface=eth.*, enp.*
  
  
• 在master节点上重新应用calico.yaml网络方案

  [root@k8s-master kubernetes]# kubectl delete -f calico.yaml
  [root@k8s-master kubernetes]# kubectl apply -f calico.yaml
  [root@k8s-master kubernetes]# kubectl get pod -n kube-system 
  NAME                                       READY   STATUS    RESTARTS   AGE
  calico-kube-controllers-647d84984b-v4cn8   1/1     Running   0          61m
  calico-node-8pznz                          1/1     Running   0          61m
  calico-node-bcr94                          1/1     Running   0          61m
  calico-node-cb8qq                          1/1     Running   0          61m
  

参考文章：

https://www.cnblogs.com/Applogize/p/14381275.html
https://blog.csdn.net/u011635437/article/details/118036666