master 上查看 node 状态,可见 devops010012007114 状态为 NotReady

[app@devops010012007068 zhanglin]<20250120 14:26:35>$ k get nodes
NAME                 STATUS     ROLES    AGE      VERSION
devops010012007068   Ready      master   3y240d   v1.19.8+k3s1
devops010012007069   Ready      <none>   288d     v1.19.8+k3s1
devops010012007070   Ready      <none>   3y240d   v1.19.8+k3s1
devops010012007071   Ready      <none>   3y240d   v1.19.8+k3s1
devops010012007073   Ready      <none>   3y240d   v1.19.8+k3s1
devops010012007109   Ready      <none>   542d     v1.19.8+k3s1
devops010012007111   Ready      <none>   542d     v1.19.8+k3s1
devops010012007113   Ready      <none>   542d     v1.19.8+k3s1
devops010012007114   NotReady   <none>   207d     v1.19.8+k3s1
devops010012007116   Ready      <none>   542d     v1.19.8+k3s1

node devops010012007114 上查看 k3s-agent 状态,有报错:

[root@localhost app]# systemctl status k3s-agent -l
● k3s-agent.service - Lightweight Kubernetes
   Loaded: loaded (/etc/systemd/system/k3s-agent.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2025-01-20 12:20:21 CST; 2h 8min ago
     Docs: https://k3s.io
  Process: 9197 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
  Process: 9167 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
  Process: 9148 ExecStartPre=/bin/sh -xc ! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service (code=exited, status=0/SUCCESS)
 Main PID: 9209 (k3s-agent)
    Tasks: 10
   Memory: 122.8M
   CGroup: /system.slice/k3s-agent.service
           └─9209 /usr/local/bin/k3s agent

Jan 20 14:27:55 localhost.localdomain k3s[9209]: time="2025-01-20T14:27:55.207578649+08:00" level=error msg="Failed to retrieve agent config: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag"
Jan 20 14:28:00 localhost.localdomain k3s[9209]: time="2025-01-20T14:28:00.220559879+08:00" level=error msg="Failed to retrieve agent config: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag"
Jan 20 14:28:05 localhost.localdomain k3s[9209]: time="2025-01-20T14:28:05.233453150+08:00" level=error msg="Failed to retrieve agent config: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag"
Jan 20 14:28:10 localhost.localdomain k3s[9209]: time="2025-01-20T14:28:10.246519473+08:00" level=error msg="Failed to retrieve agent config: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag"
Jan 20 14:28:15 localhost.localdomain k3s[9209]: time="2025-01-20T14:28:15.259321936+08:00" level=error msg="Failed to retrieve agent config: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag"
Jan 20 14:28:20 localhost.localdomain k3s[9209]: time="2025-01-20T14:28:20.272576559+08:00" level=error msg="Failed to retrieve agent config: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag"
Jan 20 14:28:25 localhost.localdomain k3s[9209]: time="2025-01-20T14:28:25.285244932+08:00" level=error msg="Failed to retrieve agent config: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag"
Jan 20 14:28:30 localhost.localdomain k3s[9209]: time="2025-01-20T14:28:30.297600069+08:00" level=error msg="Failed to retrieve agent config: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag"
Jan 20 14:28:35 localhost.localdomain k3s[9209]: time="2025-01-20T14:28:35.313884545+08:00" level=error msg="Failed to retrieve agent config: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag"
Jan 20 14:28:40 localhost.localdomain k3s[9209]: time="2025-01-20T14:28:40.326131507+08:00" level=error msg="Failed to retrieve agent config: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag"

使用 journalctl 也可查看 log:

[root@localhost app]# journalctl -u k3s-agent -f
-- Logs begin at Mon 2025-01-20 12:20:12 CST. --
Jan 20 14:28:10 localhost.localdomain k3s[9209]: time="2025-01-20T14:28:10.246519473+08:00" level=error msg="Failed to retrieve agent config: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag"
Jan 20 14:28:15 localhost.localdomain k3s[9209]: time="2025-01-20T14:28:15.259321936+08:00" level=error msg="Failed to retrieve agent config: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag"
Jan 20 14:28:20 localhost.localdomain k3s[9209]: time="2025-01-20T14:28:20.272576559+08:00" level=error msg="Failed to retrieve agent config: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag"
Jan 20 14:28:25 localhost.localdomain k3s[9209]: time="2025-01-20T14:28:25.285244932+08:00" level=error msg="Failed to retrieve agent config: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag"
Jan 20 14:28:30 localhost.localdomain k3s[9209]: time="2025-01-20T14:28:30.297600069+08:00" level=error msg="Failed to retrieve agent config: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag"
Jan 20 14:28:35 localhost.localdomain k3s[9209]: time="2025-01-20T14:28:35.313884545+08:00" level=error msg="Failed to retrieve agent config: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag"
Jan 20 14:28:40 localhost.localdomain k3s[9209]: time="2025-01-20T14:28:40.326131507+08:00" level=error msg="Failed to retrieve agent config: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag"
Jan 20 14:28:45 localhost.localdomain k3s[9209]: time="2025-01-20T14:28:45.339057664+08:00" level=error msg="Failed to retrieve agent config: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag"
Jan 20 14:28:50 localhost.localdomain k3s[9209]: time="2025-01-20T14:28:50.352345609+08:00" level=error msg="Failed to retrieve agent config: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag"
Jan 20 14:28:55 localhost.localdomain k3s[9209]: time="2025-01-20T14:28:55.364271075+08:00" level=error msg="Failed to retrieve agent config: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag"
Jan 20 14:29:00 localhost.localdomain k3s[9209]: time="2025-01-20T14:29:00.377075556+08:00" level=error msg="Failed to retrieve agent config: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag"

其中的报错:

Failed to retrieve agent config: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag

曾把我引入歧途,我一直以为是密码错误,或者是缺乏 --with-node-id flag
但是注意到 or contents of '/etc/rancher/node/password' may not match server node-passwd entry,且发现 [root@localhost app]# hostname 是 localhost
联想到服务器曾重启,hostname 好像被恢复成了默认,似乎重置 hostname 可能有用。

于是尝试:

[root@localhost app]# hostname devops010012007114

重启后:

[root@devops010012007114 app]# 

此时注意到 hostname 已变为 devops010012007114

然后回到 master 上查看状态,果然恢复了:

[app@devops010012007068 zhanglin]<20250120 14:29:36>$ k get nodes
NAME                 STATUS   ROLES    AGE      VERSION
devops010012007068   Ready    master   3y240d   v1.19.8+k3s1
devops010012007069   Ready    <none>   288d     v1.19.8+k3s1
devops010012007070   Ready    <none>   3y240d   v1.19.8+k3s1
devops010012007071   Ready    <none>   3y240d   v1.19.8+k3s1
devops010012007073   Ready    <none>   3y240d   v1.19.8+k3s1
devops010012007109   Ready    <none>   542d     v1.19.8+k3s1
devops010012007111   Ready    <none>   542d     v1.19.8+k3s1
devops010012007113   Ready    <none>   542d     v1.19.8+k3s1
devops010012007114   Ready    <none>   207d     v1.19.8+k3s1
devops010012007116   Ready    <none>   542d     v1.19.8+k3s1
Comments
Write a Comment