What is this can you help me? Kubernetes Version: v1.30.4 +rke2r1, rke2 server --debug #7264

cengizhanR · 2024-11-11T17:14:28Z

cengizhanR
Nov 11, 2024

[r# rke2 server --debug
WARN[0000] Unknown flag --use-service-account-credentials found in config.yaml, skipping
WARN[0000] not running in CIS mode
INFO[0000] Applying Pod Security Admission Configuration
INFO[0000] Starting rke2 v1.30.4+rke2r1 (9517eea519b780e154dd791c555c698e84a0e5cd)
INFO[0000] Managed etcd cluster bootstrap already complete and initialized
WARN[0000] Cluster CA certificate is not trusted by the host CA bundle, but the token does not include a CA hash. Use the full token from the server's node-token file to enable Cluster CA validation.
INFO[0000] Reconciling bootstrap data between datastore and disk
DEBU[0000] Reconciling ClientCAKey at '/var/lib/rancher/rke2/server/tls/client-ca.key'
DEBU[0000] Reconciling ETCDPeerCAKey at '/var/lib/rancher/rke2/server/tls/etcd/peer-ca.key'
DEBU[0000] Reconciling ETCDServerCA at '/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt'
DEBU[0000] Reconciling ServerCA at '/var/lib/rancher/rke2/server/tls/server-ca.crt'
DEBU[0000] Reconciling ClientCA at '/var/lib/rancher/rke2/server/tls/client-ca.crt'
DEBU[0000] Reconciling ETCDServerCAKey at '/var/lib/rancher/rke2/server/tls/etcd/server-ca.key'
DEBU[0000] Reconciling EncryptionConfig at '/var/lib/rancher/rke2/server/cred/encryption-config.json'
DEBU[0000] Reconciling EncryptionHash at '/var/lib/rancher/rke2/server/cred/encryption-state.json'
DEBU[0000] Reconciling IPSECKey at '/var/lib/rancher/rke2/server/cred/ipsec.psk'
DEBU[0000] Reconciling ETCDPeerCA at '/var/lib/rancher/rke2/server/tls/etcd/peer-ca.crt'
DEBU[0000] Reconciling PasswdFile at '/var/lib/rancher/rke2/server/cred/passwd'
DEBU[0000] Reconciling ServerCAKey at '/var/lib/rancher/rke2/server/tls/server-ca.key'
DEBU[0000] Reconciling ServiceKey at '/var/lib/rancher/rke2/server/tls/service.key'
DEBU[0000] Reconciling RequestHeaderCA at '/var/lib/rancher/rke2/server/tls/request-header-ca.crt'
DEBU[0000] Reconciling RequestHeaderCAKey at '/var/lib/rancher/rke2/server/tls/request-header-ca.key'
INFO[0000] Successfully reconciled with datastore
INFO[0000] start
INFO[0000] schedule, now=2024-11-11T20:11:12+03:00, entry=1, next=2024-11-12T00:00:00+03:00
DEBU[0000] Getting S3 snapshot cluster ID and server token hash
INFO[0000] Starting etcd for existing cluster member
INFO[0000] Defragmenting etcd database
INFO[0001] Running kube-apiserver --advertise-address=10.225.83.142 --advertise-port=6443 --allow-privileged=true --anonymous-auth=false --api-audiences=https://kubernetes.default.svc.cluster.local,rke2 --audit-log-maxage=30 --audit-log-mode=blocking-strict --audit-log-path=/var/lib/rancher/rke2/server/logs/audit.log --audit-policy-file=/etc/rancher/rke2/audit-policy.yaml --authorization-mode=RBAC,Node --bind-address=0.0.0.0 --cert-dir=/var/lib/rancher/rke2/server/tls/temporary-certs --client-ca-file=/var/lib/rancher/rke2/server/tls/client-ca.crt --egress-selector-config-file=/var/lib/rancher/rke2/server/etc/egress-selector-config.yaml --enable-admission-plugins=NodeRestriction --enable-aggregator-routing=true --enable-bootstrap-token-auth=true --encryption-provider-config=/var/lib/rancher/rke2/server/cred/encryption-config.json --encryption-provider-config-automatic-reload=true --etcd-cafile=/var/lib/rancher/rke2/server/tls/etcd/server-ca.crt --etcd-certfile=/var/lib/rancher/rke2/server/tls/etcd/client.crt --etcd-keyfile=/var/lib/rancher/rke2/server/tls/etcd/client.key --etcd-servers=https://127.0.0.1:2379 --kubelet-certificate-authority=/var/lib/rancher/rke2/server/tls/server-ca.crt --kubelet-client-certificate=/var/lib/rancher/rke2/server/tls/client-kube-apiserver.crt --kubelet-client-key=/var/lib/rancher/rke2/server/tls/client-kube-apiserver.key --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --profiling=false --proxy-client-cert-file=/var/lib/rancher/rke2/server/tls/client-auth-proxy.crt --proxy-client-key-file=/var/lib/rancher/rke2/server/tls/client-auth-proxy.key --requestheader-allowed-names=system:auth-proxy --requestheader-client-ca-file=/var/lib/rancher/rke2/server/tls/request-header-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6443 --service-account-issuer=https://kubernetes.default.svc.cluster.local --service-account-key-file=/var/lib/rancher/rke2/server/tls/service.key --service-account-signing-key-file=/var/lib/rancher/rke2/server/tls/service.current.key --service-cluster-ip-range=10.43.0.0/16 --service-node-port-range=30000-32767 --storage-backend=etcd3 --tls-cert-file=/var/lib/rancher/rke2/server/tls/serving-kube-apiserver.crt --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 --tls-min-version=VersionTLS12 --tls-private-key-file=/var/lib/rancher/rke2/server/tls/serving-kube-apiserver.key
INFO[0001] Removed kube-apiserver static pod manifest
INFO[0001] Running kube-scheduler --authentication-kubeconfig=/var/lib/rancher/rke2/server/cred/scheduler.kubeconfig --authorization-kubeconfig=/var/lib/rancher/rke2/server/cred/scheduler.kubeconfig --bind-address=127.0.0.1 --kubeconfig=/var/lib/rancher/rke2/server/cred/scheduler.kubeconfig --profiling=false --secure-port=10259 --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 --tls-min-version=VersionTLS12
INFO[0001] Running kube-controller-manager --allocate-node-cidrs=true --authentication-kubeconfig=/var/lib/rancher/rke2/server/cred/controller.kubeconfig --authorization-kubeconfig=/var/lib/rancher/rke2/server/cred/controller.kubeconfig --bind-address=127.0.0.1 --cluster-cidr=10.42.0.0/16 --cluster-signing-kube-apiserver-client-cert-file=/var/lib/rancher/rke2/server/tls/client-ca.nochain.crt --cluster-signing-kube-apiserver-client-key-file=/var/lib/rancher/rke2/server/tls/client-ca.key --cluster-signing-kubelet-client-cert-file=/var/lib/rancher/rke2/server/tls/client-ca.nochain.crt --cluster-signing-kubelet-client-key-file=/var/lib/rancher/rke2/server/tls/client-ca.key --cluster-signing-kubelet-serving-cert-file=/var/lib/rancher/rke2/server/tls/server-ca.nochain.crt --cluster-signing-kubelet-serving-key-file=/var/lib/rancher/rke2/server/tls/server-ca.key --cluster-signing-legacy-unknown-cert-file=/var/lib/rancher/rke2/server/tls/server-ca.nochain.crt --cluster-signing-legacy-unknown-key-file=/var/lib/rancher/rke2/server/tls/server-ca.key --configure-cloud-routes=false --controllers=*,tokencleaner,-service,-route,-cloud-node-lifecycle --kubeconfig=/var/lib/rancher/rke2/server/cred/controller.kubeconfig --profiling=false --root-ca-file=/var/lib/rancher/rke2/server/tls/server-ca.crt --secure-port=10257 --service-account-private-key-file=/var/lib/rancher/rke2/server/tls/service.current.key --service-cluster-ip-range=10.43.0.0/16 --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 --tls-min-version=VersionTLS12 --use-service-account-credentials=true
INFO[0001] Running cloud-controller-manager --allocate-node-cidrs=true --authentication-kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig --authorization-kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig --bind-address=127.0.0.1 --cloud-config=/var/lib/rancher/rke2/server/etc/cloud-config.yaml --cloud-provider=rke2 --cluster-cidr=10.42.0.0/16 --configure-cloud-routes=false --controllers=*,-route,-service --feature-gates=CloudDualStackNodeIPs=true --kubeconfig=/var/lib/rancher/rke2/server/cred/cloud-controller.kubeconfig --leader-elect-resource-name=rke2-cloud-controller-manager --node-status-update-frequency=1m0s --profiling=false
INFO[0001] Server node token is available at /var/lib/rancher/rke2/server/token
INFO[0001] To join server node to cluster: rke2 server -s https://10.225.83.142:9345 -t ${SERVER_NODE_TOKEN}
INFO[0001] Agent node token is available at /var/lib/rancher/rke2/server/agent-token
INFO[0001] To join agent node to cluster: rke2 agent -s https://10.225.83.142:9345 -t ${AGENT_NODE_TOKEN}
INFO[0001] Wrote kubeconfig /etc/rancher/rke2/rke2.yaml
INFO[0001] Run: rke2 kubectl
DEBU[0001] Supervisor proxy using supervisor=https://127.0.0.1:9345 apiserver=https://127.0.0.1:9345 lb=false
INFO[0001] Cluster-Http-Server 2024/11/11 20:11:12 http: TLS handshake error from 127.0.0.1:11782: remote error: tls: bad certificate
DEBU[0001] Supervisor proxy apiserver port changed; apiserver=https://127.0.0.1:6443 lb=false
INFO[0001] Pod for etcd is synced
INFO[0001] etcd data store connection OK
INFO[0001] Saving cluster bootstrap data to datastore
INFO[0001] Waiting for API server to become available
DEBU[0001] Comparing /bootstrap/64783e740815 to /bootstrap/64783e740815
DEBU[0001] checking bootstrap key /bootstrap/64783e740815 against /bootstrap/64783e740815
WARN[0001] Bootstrap key already exists
INFO[0002] Password verified locally for node sblg-uisrv02
INFO[0002] certificate CN=sblg-uisrv02 signed by CN=rke2-server-ca@1725635472: notBefore=2024-09-06 15:11:12 +0000 UTC notAfter=2025-11-11 17:11:13 +0000 UTC
INFO[0002] Using private registry config file at /etc/rancher/rke2/registries.yaml
INFO[0002] Module overlay was already loaded
INFO[0002] Module nf_conntrack was already loaded
INFO[0002] Module br_netfilter was already loaded
INFO[0002] Module iptable_nat was already loaded
INFO[0002] Module iptable_filter was already loaded
DEBU[0002] getConntrackMax: using conntrack-min
INFO[0002] Runtime image index.docker.io/rancher/rke2-runtime:v1.30.4-rke2r1 bin and charts directories already exist; skipping extract
INFO[0002] Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-snapshot-validation-webhook.yaml to set cluster configuration values
INFO[0002] Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-traefik.yaml to set cluster configuration values
INFO[0002] Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-canal.yaml to set cluster configuration values
INFO[0002] No cluster configuration value changes necessary for manifest /var/lib/rancher/rke2/server/manifests/rke2-ingress-nginx-config.yaml
INFO[0002] Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-ingress-nginx.yaml to set cluster configuration values
INFO[0002] Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-snapshot-controller.yaml to set cluster configuration values
INFO[0002] Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-calico-crd.yaml to set cluster configuration values
INFO[0002] Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-flannel.yaml to set cluster configuration values
INFO[0002] Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-metrics-server.yaml to set cluster configuration values
INFO[0002] Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-traefik-crd.yaml to set cluster configuration values
INFO[0002] Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-snapshot-controller-crd.yaml to set cluster configuration values
INFO[0002] Updated manifest /var/lib/rancher/rke2/server/manifests/harvester-cloud-provider.yaml to set cluster configuration values
INFO[0002] Updated manifest /var/lib/rancher/rke2/server/manifests/harvester-csi-driver.yaml to set cluster configuration values
INFO[0002] Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-calico.yaml to set cluster configuration values
INFO[0002] Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-coredns.yaml to set cluster configuration values
INFO[0002] Updated manifest /var/lib/rancher/rke2/server/manifests/rancher-vsphere-cpi.yaml to set cluster configuration values
INFO[0002] Updated manifest /var/lib/rancher/rke2/server/manifests/rancher-vsphere-csi.yaml to set cluster configuration values
INFO[0002] Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-cilium.yaml to set cluster configuration values
INFO[0002] Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-multus.yaml to set cluster configuration values
DEBU[0002] Searching for crun container runtime
DEBU[0002] Error searching for crun in $PATH: exec: "crun": executable file not found in $PATH
DEBU[0002] Searching for nvidia-experimental container runtime
DEBU[0002] Error searching for nvidia-experimental in $PATH: exec: "nvidia-container-runtime-experimental": executable file not found in $PATH
DEBU[0002] Searching for nvidia container runtime
DEBU[0002] Error searching for nvidia in $PATH: exec: "nvidia-container-runtime": executable file not found in $PATH
DEBU[0002] Searching for wasmtime container runtime
DEBU[0002] Error searching for wasmtime in $PATH: exec: "containerd-shim-wasmtime-v1": executable file not found in $PATH
DEBU[0002] Searching for lunatic container runtime
DEBU[0002] Error searching for lunatic in $PATH: exec: "containerd-shim-lunatic-v1": executable file not found in $PATH
DEBU[0002] Searching for slight container runtime
DEBU[0002] Error searching for slight in $PATH: exec: "containerd-shim-slight-v1": executable file not found in $PATH
DEBU[0002] Searching for spin container runtime
DEBU[0002] Error searching for spin in $PATH: exec: "containerd-shim-spin-v2": executable file not found in $PATH
DEBU[0002] Searching for wws container runtime
DEBU[0002] Error searching for wws in $PATH: exec: "containerd-shim-wws-v1": executable file not found in $PATH
DEBU[0002] Searching for wasmedge container runtime
DEBU[0002] Error searching for wasmedge in $PATH: exec: "containerd-shim-wasmedge-v1": executable file not found in $PATH
DEBU[0002] Searching for wasmer container runtime
DEBU[0002] Error searching for wasmer in $PATH: exec: "containerd-shim-wasmer-v1": executable file not found in $PATH
INFO[0002] Logging containerd to /var/lib/rancher/rke2/agent/containerd/containerd.log
INFO[0002] Running containerd -c /var/lib/rancher/rke2/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /var/lib/rancher/rke2/agent/containerd
INFO[0002] containerd is now running
INFO[0002] Importing images from /var/lib/rancher/rke2/agent/images/rke2-images.linux-amd64.tar.zst
INFO[0004] Cluster-Http-Server 2024/11/11 20:11:16 http: TLS handshake error from 10.225.83.4:61792: EOF
Cluster-Http-Server 2024/11/11 20:11:16 http: TLS handshake error from 10.225.83.4:61792: EOFINFO[0014] Cluster-Http-Server 2024/11/11 20:11:26 http: TLS handshake error from 10.225.83.4:33022: EOF
INFO[0018] Cluster-Http-Server 2024/11/11 20:11:29 http: TLS handshake error from 10.225.83.5:50272: EOF
INFO[0019] Cluster-Http-Server 2024/11/11 20:11:31 http: TLS handshake error from 10.225.83.4:23200: EOF
INFO[0028] Cluster-Http-Server 2024/11/11 20:11:39 http: TLS handshake error from 10.225.83.5:19296: EOF
INFO[0029] Cluster-Http-Server 2024/11/11 20:11:41 http: TLS handshake error from 10.225.83.4:14816: EOF
INFO[0031] Waiting for API server to become available
INFO[0031] Waiting for API server to become available
W1111 20:11:43.253154 2088910 logging.go:59] [core] [Channel #22 SubChannel #23] grpc: addrConn.createTransport failed to connect to {Addr: "/run/k3s/containerd/containerd.sock", ServerName: "localhost", Attributes: {"<%!p(networktype.keyType=grpc.internal.transport.networktype)>": "unix" }, }. Err: connection error: desc = "error reading server preface: read unix @->/run/k3s/containerd/containerd.sock: read: connection reset by peer"
ERRO[0031] Error encountered while importing /var/lib/rancher/rke2/agent/images/rke2-images.linux-amd64.tar.zst: failed to import images from /var/lib/rancher/rke2/agent/images/rke2-images.linux-amd64.tar.zst: failed to ingest "blobs/sha256/8c255812117336386e9c17e86d537d16164ffeecbe242b41c70573b30c888b85": failed to copy: failed to send write: error reading from server: EOF: unavailable
DEBU[0031] Starting rke2-cert-monitor with monitoring period 30m0s
INFO[0031] Creating rke2-cert-monitor event broadcaster
INFO[0031] Connecting to proxy                           url="wss://127.0.0.1:9345/v1-rke2/connect"
DEBU[0031] Running rke2-cert-monitor certificate expiration check
DEBU[0031] Kubelet image credential provider bin directory check failed: stat /var/lib/rancher/credentialprovider/bin: no such file or directory
INFO[0031] Running kubelet --address=0.0.0.0 --alsologtostderr=false --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=cgroupfs --client-ca-file=/var/lib/rancher/rke2/agent/client-ca.crt --cloud-provider=external --cluster-dns=10.43.0.10 --cluster-domain=cluster.local --container-runtime-endpoint=unix:///run/k3s/containerd/containerd.sock --containerd=/run/k3s/containerd/containerd.sock --eviction-hard=imagefs.available<5%,nodefs.available<5% --eviction-minimum-reclaim=imagefs.available=10%,nodefs.available=10% --fail-swap-on=false --feature-gates=CloudDualStackNodeIPs=true --healthz-bind-address=127.0.0.1 --hostname-override=sblg-uisrv02 --kubeconfig=/var/lib/rancher/rke2/agent/kubelet.kubeconfig --kubelet-cgroups=/rke2 --log-file=/var/lib/rancher/rke2/agent/logs/kubelet.log --log-file-max-size=50 --logtostderr=false --node-ip=10.225.83.142 --node-labels= --pod-manifest-path=/var/lib/rancher/rke2/agent/pod-manifests --protect-kernel-defaults=true --read-only-port=0 --register-with-taints=CriticalAddonsOnly=true:NoExecute --resolv-conf=/etc/resolv.conf --serialize-image-pulls=false --stderrthreshold=FATAL --streaming-connection-idle-timeout=5m --tls-cert-file=/var/lib/rancher/rke2/agent/serving-kubelet.crt --tls-private-key-file=/var/lib/rancher/rke2/agent/serving-kubelet.key
DEBU[0031] Tunnel server handing HTTP/1.1 GET request for /v1-rke2/connect from 127.0.0.1:9424
INFO[0031] Handling backend connection request [sblg-uisrv02]
INFO[0031] Cluster-Http-Server 2024/11/11 20:11:43 http: TLS handshake error from 127.0.0.1:9438: remote error: tls: bad certificate
INFO[0031] Remotedialer connected to proxy               url="wss://127.0.0.1:9345/v1-rke2/connect"
ERRO[0031] Sending HTTP 503 response to 127.0.0.1:9460: runtime core not ready
INFO[0031] Running kube-proxy --cluster-cidr=10.42.0.0/16 --conntrack-max-per-core=0 --conntrack-tcp-timeout-close-wait=0s --conntrack-tcp-timeout-established=0s --healthz-bind-address=127.0.0.1 --hostname-override=sblg-uisrv02 --kubeconfig=/var/lib/rancher/rke2/agent/kubeproxy.kubeconfig --metrics-bind-address=0.0.0.0:10249 --proxy-mode=iptables
INFO[0033] Cluster-Http-Server 2024/11/11 20:11:44 http: TLS handshake error from 10.225.83.5:52218: EOF
INFO[0034] Cluster-Http-Server 2024/11/11 20:11:46 http: TLS handshake error from 10.225.83.4:14272: EOF
DEBU[0036] Wrote ping
INFO[0037] Cluster-Http-Server 2024/11/11 20:11:49 http: TLS handshake error from 10.225.83.6:64106: remote error: tls: bad certificate
INFO[0039] Cluster-Http-Server 2024/11/11 20:11:51 http: TLS handshake error from 10.225.83.4:40752: EOF
DEBU[0041] Wrote ping
INFO[0043] Cluster-Http-Server 2024/11/11 20:11:54 http: TLS handshake error from 10.225.83.5:64032: EOF
INFO[0044] Cluster-Http-Server 2024/11/11 20:11:56 http: TLS handshake error from 10.225.83.4:42322: EOF
DEBU[0046] Wrote ping
DEBU[0051] Wrote ping
INFO[0053] Cluster-Http-Server 2024/11/11 20:12:04 http: TLS handshake error from 10.225.83.5:58338: EOF
INFO[0054] Cluster-Http-Server 2024/11/11 20:12:06 http: TLS handshake error from 10.225.83.4:45412: EOF
DEBU[0056] Wrote ping
INFO[0061] Waiting for API server to become available
INFO[0061] Waiting for API server to become available
DEBU[0061] Wrote ping
INFO[0064] Cluster-Http-Server 2024/11/11 20:12:16 http: TLS handshake error from 10.225.83.4:51360: EOF
DEBU[0066] Wrote ping
DEBU[0071] Wrote ping
INFO[0073] Cluster-Http-Server 2024/11/11 20:12:24 http: TLS handshake error from 10.225.83.5:17184: EOF
INFO[0074] Cluster-Http-Server 2024/11/11 20:12:26 http: TLS handshake error from 10.225.83.4:3020: EOF
DEBU[0076] Wrote ping
INFO[0078] Cluster-Http-Server 2024/11/11 20:12:29 http: TLS handshake error from 10.225.83.5:18240: EOF
DEBU[0081] Wrote ping
DEBU[0086] Wrote ping
INFO[0091] Waiting for API server to become available
INFO[0091] Waiting for API server to become available
DEBU[0091] Wrote ping
INFO[0094] Cluster-Http-Server 2024/11/11 20:12:46 http: TLS handshake error from 10.225.83.4:42528: EOF
DEBU[0096] Wrote ping
DEBU[0101] Wrote ping
INFO[0103] Cluster-Http-Server 2024/11/11 20:12:54 http: TLS handshake error from 10.225.83.5:22464: EOF
DEBU[0106] Wrote ping
INFO[0108] Cluster-Http-Server 2024/11/11 20:12:59 http: TLS handshake error from 10.225.83.5:53440: EOF
DEBU[0111] Wrote ping
INFO[0113] Cluster-Http-Server 2024/11/11 20:13:04 http: TLS handshake error from 10.225.83.5:48626: EOF
DEBU[0116] Wrote ping
INFO[0121] Waiting for API server to become available
INFO[0121] Waiting for API server to become available
DEBU[0121] Wrote ping
INFO[0123] Cluster-Http-Server 2024/11/11 20:13:14 http: TLS handshake error from 10.225.83.5:42336: EOF
DEBU[0126] Wrote ping
INFO[0128] Cluster-Http-Server 2024/11/11 20:13:19 http: TLS handshake error from 10.225.83.5:49184: EOF
DEBU[0131] Wrote ping
INFO[0133] Cluster-Http-Server 2024/11/11 20:13:24 http: TLS handshake error from 10.225.83.5:2464: EOF
INFO[0134] Cluster-Http-Server 2024/11/11 20:13:26 http: TLS handshake error from 10.225.83.4:24992: EOF
DEBU[0136] Wrote ping
INFO[0138] Cluster-Http-Server 2024/11/11 20:13:29 http: TLS handshake error from 10.225.83.5:53946: EOF
INFO[0139] Cluster-Http-Server 2024/11/11 20:13:31 http: TLS handshake error from 10.225.83.4:44964: EOF
DEBU[0141] Wrote ping
INFO[0143] Cluster-Http-Server 2024/11/11 20:13:34 http: TLS handshake error from 10.225.83.5:39104: EOF
DEBU[0146] Wrote ping

cengizhanR · 2024-11-11T17:22:46Z

cengizhanR
Nov 11, 2024
Author

this is config yaml for rke2

 cat /etc/rancher/rke2/config.yaml
secrets-encryption: true
token: fuzzybunnyslippers
tls-san:
  - bdada.com
  - 190.16.5.55
node-ip: "10.10ç10.10"
etcd-expose-metrics: true
write-kubeconfig-mode: "0600"
use-service-account-credentials: true
node-taint:
- "CriticalAddonsOnly=true:NoExecute"
kube-controller-manager-arg:
- bind-address=127.0.0.1
- use-service-account-credentials=true
- tls-min-version=VersionTLS12
- tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
kube-scheduler-arg:
- tls-min-version=VersionTLS12
- tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
kube-proxy-arg:
- metrics-bind-address=0.0.0.0:10249
kube-apiserver-arg:
- tls-min-version=VersionTLS12
- tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
- authorization-mode=RBAC,Node
- anonymous-auth=false
- audit-policy-file=/etc/rancher/rke2/audit-policy.yaml
- audit-log-mode=blocking-strict
- audit-log-maxage=30
- audit-log-path=/var/lib/rancher/rke2/server/logs/audit.log
kubelet-arg:
- protect-kernel-defaults=true
- read-only-port=0
- authorization-mode=Webhook
- streaming-connection-idle-timeout=5m

0 replies

cengizhanR · 2024-11-11T17:48:32Z

cengizhanR
Nov 11, 2024
Author

this is F5 domain, bbdada.com when ı put F5 in front of my server and nodes ı started to get these errors. Or it can about CPU and Memory on servers?

0 replies

brandond · 2024-11-11T17:53:42Z

brandond
Nov 11, 2024
Maintainer

What is the problem? I don't see anything wrong here. What errors? All I see are info and debug messages.

5 replies

cengizhanR Nov 11, 2024
Author

it says handshake erors, these are normal thank you for your return @brandond ,

INFO[0121] Waiting for API server to become available
INFO[0121] Waiting for API server to become available
DEBU[0121] Wrote ping
INFO[0123] Cluster-Http-Server 2024/11/11 20:13:14 http: TLS handshake error from 10.225.83.5:42336: EOF
DEBU[0126] Wrote ping
INFO[0128] Cluster-Http-Server 2024/11/11 20:13:19 http: TLS handshake error from 10.225.83.5:49184: EOF
DEBU[0131] Wrote ping
INFO[0133] Cluster-Http-Server 2024/11/11 20:13:24 http: TLS handshake error from 10.225.83.5:2464: EOF
INFO[0134] Cluster-Http-Server 2024/11/11 20:13:26 http: TLS handshake error from 10.225.83.4:24992: EOF
DEBU[0136] Wrote ping
INFO[0138] Cluster-Http-Server 2024/11/11 20:13:29 http: TLS handshake error from 10.225.83.5:53946: EOF
INFO[0139] Cluster-Http-Server 2024/11/11 20:13:31 http: TLS handshake error from 10.225.83.4:44964: EOF
DEBU[0141] Wrote ping
INFO[0143] Cluster-Http-Server 2024/11/11 20:13:34 http: TLS handshake error from 10.225.83.5:39104: EOF
DEBU[0146] Wrote ping

brandond Nov 11, 2024
Maintainer

Your F5 load balancer health checks are not completing the TLS handshake, they are just connecting to the port and then closing the connection without sending any request. Because you have turned up the log level, you get a message when that happens.

If you don't like it, change the health check, or turn off debug.

cengizhanR Nov 11, 2024
Author

@brandond Does this cause problems? Do I need to turn off F5's heatch check controller? Thank you very much

brandond Nov 11, 2024
Maintainer

If it was a problem it wouldn't only show up in debug mode. Why did you enable debug logging if you don't want to see extra messages?

I would not recommend disabling health checks... but you should use a HTTPS health check that actually makes a request if you don't want to see these debug logs caused by the aborted connection.

cengizhanR Nov 11, 2024
Author

Thank you so much @brandond

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

What is this can you help me? Kubernetes Version: v1.30.4 +rke2r1, rke2 server --debug #7264

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 3 comments 5 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

What is this can you help me? Kubernetes Version: v1.30.4 +rke2r1, rke2 server --debug #7264

cengizhanR Nov 11, 2024

Replies: 3 comments · 5 replies

cengizhanR Nov 11, 2024 Author

cengizhanR Nov 11, 2024 Author

brandond Nov 11, 2024 Maintainer

cengizhanR Nov 11, 2024 Author

brandond Nov 11, 2024 Maintainer

cengizhanR Nov 11, 2024 Author

brandond Nov 11, 2024 Maintainer

cengizhanR Nov 11, 2024 Author

cengizhanR
Nov 11, 2024

Replies: 3 comments 5 replies

cengizhanR
Nov 11, 2024
Author

cengizhanR
Nov 11, 2024
Author

brandond
Nov 11, 2024
Maintainer

cengizhanR Nov 11, 2024
Author

brandond Nov 11, 2024
Maintainer

cengizhanR Nov 11, 2024
Author

brandond Nov 11, 2024
Maintainer

cengizhanR Nov 11, 2024
Author