Updating HA clusters fails #857

franpog859 · 2024-12-17T13:27:39Z

What happened?

I created a HA cluster, the k0smotron.io/v1beta1 Cluster spec.replicas equals 3. I tried to follow the docs to update the k0s, I changed the spec.version from v1.30.4-k0s.0 to v1.31.2-k0s.0. The impact is as follows:

# on the management cluster
$ kubectl -n infra get po -w
kmc-franek-test1-0                     2/3     Running     1 (53s ago)   3h37m
kmc-franek-test1-1                     3/3     Running     0             3h34m
kmc-franek-test1-2                     2/3     Running     1 (51s ago)   3m30s
kmc-franek-test1-etcd-0                1/1     Running     0             3h37m
kmc-franek-test1-etcd-1                1/1     Running     0             2m12s
kmc-franek-test1-etcd-2                0/1     Completed   4 (56s ago)   3m30s
...
kmc-franek-test1-etcd-2                0/1     CrashLoopBackOff   4 (43s ago)   4m8s

$ kubectl -n infra describe po kmc-franek-test1-2
...
  Warning  Unhealthy  7m57s (x4 over 8m27s)   kubelet            Liveness probe failed: command timed out: "k0s status" timed out after 1s
  Warning  Unhealthy  4m17s (x17 over 8m57s)  kubelet            Readiness probe failed: command timed out: "k0s status" timed out after 1s

$ kubectl -n infra describe po kmc-franek-test1-etcd-2
...
  Warning  Unhealthy  7m53s (x2 over 8m3s)    kubelet            Readiness probe failed: command timed out: "etcdctl endpoint health" timed out after 1s
  Warning  BackOff    3m58s (x20 over 7m34s)  kubelet            Back-off restarting failed container etcd in pod kmc-franek-test1-etcd-2_infra(123123123-1231-1231-1231-12312312313)


# on the k0s cluster
$ kubectl get po -A
Error from server: etcdserver: request timed out

$ kubectl version
Client Version: v1.32.0
Kustomize Version: v5.5.0
Server Version: v1.30.4+k0s

Involved Areas

Standalone

What did you expect to happen?

The cluster is upgraded and works fine. Maybe I use wrong documentation, could you point me to some working instructions?

Steps to reproduce

Create a cluster with spec.replicas=3, wait for it to start working, edit the cluster and bump the version, watch the cluster burn

k0smotron version

v1.1.2

k0s version

v1.30.4-k0s.0

Anything else we need to know?

No response

The text was updated successfully, but these errors were encountered:

franpog859 · 2024-12-17T14:31:14Z

I checked behavior with the newest k0smotron version v1.3.0-rc.0, it's similar, etcd pods in CrashLoopBackOff status, cluster is not updated

$ kubectl -n infra describe po kmc-fr1-etcd-1 | tail
  Normal   Created    79s (x4 over 2m2s)  kubelet            Created container dns-check
  Normal   Pulled     79s (x3 over 2m1s)  kubelet            Container image "k0sproject/k0s:v1.31.2-k0s.0" already present on machine
  Normal   Started    79s (x4 over 2m2s)  kubelet            Started container dns-check
  Normal   Pulled     79s                 kubelet            Container image "quay.io/k0sproject/etcd:v3.5.13" already present on machine
  Normal   Created    79s                 kubelet            Created container init
  Normal   Started    79s                 kubelet            Started container init
  Normal   Pulled     78s                 kubelet            Container image "quay.io/k0sproject/etcd:v3.5.13" already present on machine
  Normal   Created    78s                 kubelet            Created container etcd
  Normal   Started    78s                 kubelet            Started container etcd
  Warning  Unhealthy  77s                 kubelet            Readiness probe failed:

$ kubectl -n infra describe po kmc-fr1-2 | tail
  Normal   Created    3m45s                 kubelet            Created container controller
  Normal   Started    3m45s                 kubelet            Started container controller
  Normal   Pulled     3m45s                 kubelet            Container image "quay.io/k0sproject/prometheus:v2.44.0" already present on machine
  Normal   Created    3m45s                 kubelet            Created container monitoring-agent
  Normal   Started    3m45s                 kubelet            Started container monitoring-agent
  Normal   Pulled     3m45s                 kubelet            Container image "nginx:1.19.10" already present on machine
  Normal   Created    3m45s                 kubelet            Created container monitoring-proxy
  Normal   Started    3m45s                 kubelet            Started container monitoring-proxy
  Warning  Unhealthy  103s (x8 over 2m43s)  kubelet            Readiness probe failed: command timed out: "k0s status" timed out after 1s
  Warning  Unhealthy  103s (x4 over 2m13s)  kubelet            Liveness probe failed: command timed out: "k0s status" timed out after 1s

makhov · 2024-12-18T12:39:31Z

Yeah, thanks for reporting the issue! I prepared a fix, we'll try to merge and release new version asap

franpog859 · 2024-12-19T12:01:24Z

I checked the latest image with the fix present. The etcd pods indeed are not crashing anymore, the other ones do though 🤔 It does happen only when upgrading the cluster. May I ask you to reopen the issue if it turns out to be still k0smotron-related, @makhov ?

$ kubectl -n infra get po | grep fr2
kmc-fr2-0                    3/3     Running            0               2d1h
kmc-fr2-1                    3/3     Running            0               2d1h
kmc-fr2-2                    2/3     CrashLoopBackOff   8 (18s ago)     33m
kmc-fr2-etcd-0               1/1     Running            0               46m
kmc-fr2-etcd-1               1/1     Running            0               47m
kmc-fr2-etcd-2               1/1     Running            0               47m

$ kubectl -n infra describe po kmc-fr2-2 | tail
  Normal   Pulled     33m                   kubelet            Container image "nginx:1.19.10" already present on machine
  Normal   Pulled     33m                   kubelet            Container image "quay.io/k0sproject/prometheus:v2.44.0" already present on machine
  Normal   Created    33m                   kubelet            Created container monitoring-agent
  Normal   Started    33m                   kubelet            Started container monitoring-proxy
  Warning  Unhealthy  31m (x3 over 31m)     kubelet            Liveness probe failed: command timed out: "k0s status" timed out after 1s
  Normal   Started    31m (x2 over 33m)     kubelet            Started container controller
  Normal   Created    31m (x2 over 33m)     kubelet            Created container controller
  Normal   Pulled     31m (x2 over 33m)     kubelet            Container image "k0sproject/k0s:v1.31.2-k0s.0" already present on machine
  Warning  Unhealthy  8m36s (x50 over 32m)  kubelet            Readiness probe failed: command timed out: "k0s status" timed out after 1s
  Warning  BackOff    3m31s (x70 over 29m)  kubelet            Back-off restarting failed container controller in pod kmc-fr2-2_infra(1231231-123123-1231-1231-1231231231231)

$ kubectl -n infra logs kmc-fr2-2 | grep -C 5 error | head
Defaulted container "controller" out of: controller, monitoring-agent, monitoring-proxy, certs-init (init)
time="2024-12-19 11:37:16" level=info msg="I1219 11:37:16.469215      45 shared_informer.go:320] Caches are synced for *generic.policySource[*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicy,*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicyBinding,k8s.io/apiserver/pkg/admission/plugin/policy/validating.Validator]" component=kube-apiserver stream=stderr
time="2024-12-19 11:37:16" level=info msg="I1219 11:37:16.469237      45 policy_source.go:224] refreshing policies" component=kube-apiserver stream=stderr
time="2024-12-19 11:37:16" level=info msg="I1219 11:37:16.497085      45 handler_discovery.go:450] Starting ResourceDiscoveryManager" component=kube-apiserver stream=stderr
time="2024-12-19 11:37:16" level=info msg="I1219 11:37:16.499243      45 handler.go:286] Adding GroupVersion metrics.k8s.io v1beta1 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 11:37:16" level=info msg="E1219 11:37:16.499286      45 controller.go:146] \"Unhandled Error\" err=<" component=kube-apiserver stream=stderr
time="2024-12-19 11:37:16" level=info msg="\tError updating APIService \"v1beta1.metrics.k8s.io\" with err: failed to download v1beta1.metrics.k8s.io: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: error trying to reach service: No agent available" component=kube-apiserver stream=stderr
time="2024-12-19 11:37:16" level=info msg="\t, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]" component=kube-apiserver stream=stderr
time="2024-12-19 11:37:16" level=info msg=" > logger=\"UnhandledError\"" component=kube-apiserver stream=stderr
time="2024-12-19 11:37:16" level=info msg="E1219 11:37:16.498710      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=1237397658593747123" component=konnectivity stream=stderr
time="2024-12-19 11:37:16" level=info msg="E1219 11:37:16.498865      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=1239348103447653456" component=konnectivity stream=stderr

$ kubectl -n infra get po kmc-fr2-0 -oyaml | grep nodeName
  nodeName: 12.123.12.123
$ kubectl -n infra get po kmc-fr2-1 -oyaml | grep nodeName
  nodeName: 12.123.12.124
$ kubectl -n infra get po kmc-fr2-2 -oyaml | grep nodeName
  nodeName: 12.123.12.125

$ kubectl -n k0smotron get deploy k0smotron-controller-manager -oyaml | grep image
        image: gcr.io/kubebuilder/kube-rbac-proxy:v0.13.1
        imagePullPolicy: IfNotPresent
        image: quay.io/k0sproject/k0smotron:v1.3.0
        imagePullPolicy: IfNotPresent

$ kubectl version
Client Version: v1.32.0
Kustomize Version: v5.5.0
Server Version: v1.31.3+IKS

$ kubectl -n kube-system get apiservices.apiregistration.k8s.io | grep metrics
v1beta1.metrics.k8s.io                          kube-system/metrics-server    True        2d3h

makhov · 2024-12-19T14:38:51Z

Yeah, I'll check. Could you also send a full log of crashing pod?

franpog859 · 2024-12-19T15:00:28Z

Sure, the logs look like this. Then it just repeats itself

Toggle, not to clutter the issue

$ kubectl -n infra logs kmc-fr3-2 | head -500
Defaulted container "controller" out of: controller, monitoring-agent, monitoring-proxy, certs-init (init)
iptables-save v1.8.10 (nf_tables): Could not fetch rule set generation id: Permission denied (you must be root)
iptables-nft-save v1.8.10 (nf_tables): Could not fetch rule set generation id: Permission denied (you must be root)
time="2024-12-19 14:49:23" level=warning pre-flight-check=os/NET property="no kernel config found"
time="2024-12-19 14:49:23" level=info msg="using api address: 123.123.123.123"
time="2024-12-19 14:49:23" level=info msg="using listen port: 30443"
time="2024-12-19 14:49:23" level=info msg="using sans: [123.123.123.123 kmc-fr3-lb kmc-fr3-lb.infra kmc-fr3-lb.infra.svc 123.123.123.123 kmc-fr3-lb.infra.svc.cluster.local]"
time="2024-12-19 14:49:23" level=info msg="DNS address: 123.123.123.123"
time="2024-12-19 14:49:23" level=info msg="using storage backend etcd"
time="2024-12-19 14:49:23" level=info msg="initializing Etcd"
time="2024-12-19 14:49:23" level=info msg="initializing Konnectivity"
time="2024-12-19 14:49:23" level=info msg="initializing APIServer"
time="2024-12-19 14:49:23" level=info msg="initializing K0sControllersLeaseCounter"
time="2024-12-19 14:49:23" level=info msg="initializing LeasePool"
time="2024-12-19 14:49:23" level=info msg="initializing Manager"
time="2024-12-19 14:49:23" level=info msg="initializing K0SControlAPI"
time="2024-12-19 14:49:23" level=info msg="initializing CSRApprover"
time="2024-12-19 14:49:23" level=info msg="initializing Status"
time="2024-12-19 14:49:23" level=info msg="Staging '/var/lib/k0s/bin/konnectivity-server'"
time="2024-12-19 14:49:23" level=info msg="Staging '/var/lib/k0s/bin/etcd'"
time="2024-12-19 14:49:23" level=info msg="Listening address /run/k0s/status.sock" component=status
time="2024-12-19 14:49:23" level=info msg="Staging '/var/lib/k0s/bin/kube-apiserver'"
time="2024-12-19 14:49:24" level=info msg="starting Etcd"
time="2024-12-19 14:49:24" level=info msg="starting Konnectivity"
time="2024-12-19 14:49:24" level=info msg="Starting to supervise" component=konnectivity
time="2024-12-19 14:49:24" level=info msg="Started successfully, go nuts pid 36" component=konnectivity
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762083      36 options.go:148] ServerCert set to \"\"." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762131      36 options.go:149] ServerKey set to \"\"." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762135      36 options.go:150] ServerCACert set to \"\"." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762140      36 options.go:151] ClusterCert set to \"/var/lib/k0s/pki/server.crt\"." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762146      36 options.go:152] ClusterKey set to \"/var/lib/k0s/pki/server.key\"." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762151      36 options.go:153] ClusterCACert set to \"\"." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762155      36 options.go:154] Mode set to \"grpc\"." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762161      36 options.go:155] UDSName set to \"/run/k0s/konnectivity-server/konnectivity-server.sock\"." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762167      36 options.go:156] DeleteUDSFile set to true." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762174      36 options.go:157] Server port set to 0." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762181      36 options.go:158] Server bind address set to \"\"." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762187      36 options.go:159] Agent port set to 30132." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762193      36 options.go:160] Agent bind address set to \"\"." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762210      36 options.go:161] Admin port set to 8133." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762219      36 options.go:162] Admin bind address set to \"127.0.0.1\"." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762227      36 options.go:163] Health port set to 8092." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762234      36 options.go:164] Health bind address set to \"localhost\"." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762242      36 options.go:165] Keepalive time set to 1h0m0s." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762258      36 options.go:166] Frontend keepalive time set to 1h0m0s." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762267      36 options.go:167] EnableProfiling set to false." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762275      36 options.go:168] EnableContentionProfiling set to false." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762284      36 options.go:169] ServerID set to ac6ccb123123123132123." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762294      36 options.go:170] ServerCount set to 1." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762304      36 options.go:171] AgentNamespace set to \"kube-system\"." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762315      36 options.go:172] AgentServiceAccount set to \"konnectivity-agent\"." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762324      36 options.go:173] AuthenticationAudience set to \"system:konnectivity-server\"." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762336      36 options.go:174] KubeconfigPath set to \"/var/lib/k0s/pki/konnectivity.conf\"." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762348      36 options.go:175] KubeconfigQPS set to 0.000000." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762360      36 options.go:176] KubeconfigBurst set to 0." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762394      36 options.go:177] ProxyStrategies set to \"destHost,default\"." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.762402      36 options.go:178] CipherSuites set to [\"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256\" \"TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384\" \"TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256\" \"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256\" \"TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384\" \"TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256\"]." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.766983      36 server.go:130] Starting frontend server for client connections." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.768359      36 server.go:145] Starting agent server for tunnel connections." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.769027      36 server.go:152] Starting admin server for debug connections." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.769101      36 server.go:159] Starting health server for healthchecks." component=konnectivity stream=stderr
time="2024-12-19 14:49:24" level=info msg="starting APIServer"
time="2024-12-19 14:49:24" level=info msg="Starting kube-apiserver"
time="2024-12-19 14:49:24" level=info msg="Starting to supervise" component=kube-apiserver
time="2024-12-19 14:49:24" level=info msg="Started successfully, go nuts pid 46" component=kube-apiserver
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912020      46 flags.go:64] FLAG: --admission-control=\"[]\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912088      46 flags.go:64] FLAG: --admission-control-config-file=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912093      46 flags.go:64] FLAG: --advertise-address=\"123.123.123.123\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912118      46 flags.go:64] FLAG: --aggregator-reject-forwarding-redirect=\"true\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912123      46 flags.go:64] FLAG: --allow-metric-labels=\"[]\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912132      46 flags.go:64] FLAG: --allow-metric-labels-manifest=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912135      46 flags.go:64] FLAG: --allow-privileged=\"true\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912138      46 flags.go:64] FLAG: --anonymous-auth=\"false\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912140      46 flags.go:64] FLAG: --api-audiences=\"[https://kubernetes.default.svc,system:konnectivity-server]\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912147      46 flags.go:64] FLAG: --apiserver-count=\"1\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912150      46 flags.go:64] FLAG: --audit-log-batch-buffer-size=\"10000\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912153      46 flags.go:64] FLAG: --audit-log-batch-max-size=\"1\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912156      46 flags.go:64] FLAG: --audit-log-batch-max-wait=\"0s\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912159      46 flags.go:64] FLAG: --audit-log-batch-throttle-burst=\"0\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912162      46 flags.go:64] FLAG: --audit-log-batch-throttle-enable=\"false\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912164      46 flags.go:64] FLAG: --audit-log-batch-throttle-qps=\"0\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912168      46 flags.go:64] FLAG: --audit-log-compress=\"false\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912170      46 flags.go:64] FLAG: --audit-log-format=\"json\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912174      46 flags.go:64] FLAG: --audit-log-maxage=\"0\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912202      46 flags.go:64] FLAG: --audit-log-maxbackup=\"0\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912206      46 flags.go:64] FLAG: --audit-log-maxsize=\"0\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912208      46 flags.go:64] FLAG: --audit-log-mode=\"blocking\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912211      46 flags.go:64] FLAG: --audit-log-path=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912213      46 flags.go:64] FLAG: --audit-log-truncate-enabled=\"false\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912216      46 flags.go:64] FLAG: --audit-log-truncate-max-batch-size=\"10485760\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912224      46 flags.go:64] FLAG: --audit-log-truncate-max-event-size=\"102400\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912227      46 flags.go:64] FLAG: --audit-log-version=\"audit.k8s.io/v1\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912229      46 flags.go:64] FLAG: --audit-policy-file=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912232      46 flags.go:64] FLAG: --audit-webhook-batch-buffer-size=\"10000\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912234      46 flags.go:64] FLAG: --audit-webhook-batch-initial-backoff=\"10s\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912237      46 flags.go:64] FLAG: --audit-webhook-batch-max-size=\"400\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912240      46 flags.go:64] FLAG: --audit-webhook-batch-max-wait=\"30s\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912242      46 flags.go:64] FLAG: --audit-webhook-batch-throttle-burst=\"15\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912245      46 flags.go:64] FLAG: --audit-webhook-batch-throttle-enable=\"true\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912247      46 flags.go:64] FLAG: --audit-webhook-batch-throttle-qps=\"10\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912250      46 flags.go:64] FLAG: --audit-webhook-config-file=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912252      46 flags.go:64] FLAG: --audit-webhook-initial-backoff=\"10s\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912255      46 flags.go:64] FLAG: --audit-webhook-mode=\"batch\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912257      46 flags.go:64] FLAG: --audit-webhook-truncate-enabled=\"false\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912260      46 flags.go:64] FLAG: --audit-webhook-truncate-max-batch-size=\"10485760\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912262      46 flags.go:64] FLAG: --audit-webhook-truncate-max-event-size=\"102400\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912264      46 flags.go:64] FLAG: --audit-webhook-version=\"audit.k8s.io/v1\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912267      46 flags.go:64] FLAG: --authentication-config=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912269      46 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl=\"2m0s\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912272      46 flags.go:64] FLAG: --authentication-token-webhook-config-file=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912274      46 flags.go:64] FLAG: --authentication-token-webhook-version=\"v1beta1\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912277      46 flags.go:64] FLAG: --authorization-config=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912279      46 flags.go:64] FLAG: --authorization-mode=\"[Node,RBAC]\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912285      46 flags.go:64] FLAG: --authorization-policy-file=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912289      46 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl=\"5m0s\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912292      46 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl=\"30s\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912294      46 flags.go:64] FLAG: --authorization-webhook-config-file=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912297      46 flags.go:64] FLAG: --authorization-webhook-version=\"v1beta1\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912300      46 flags.go:64] FLAG: --bind-address=\"0.0.0.0\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912322      46 flags.go:64] FLAG: --cert-dir=\"/var/run/kubernetes\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912326      46 flags.go:64] FLAG: --client-ca-file=\"/var/lib/k0s/pki/ca.crt\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912328      46 flags.go:64] FLAG: --cloud-config=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912331      46 flags.go:64] FLAG: --cloud-provider=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912714      46 flags.go:64] FLAG: --contention-profiling=\"false\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912720      46 flags.go:64] FLAG: --cors-allowed-origins=\"[]\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912725      46 flags.go:64] FLAG: --debug-socket-path=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912728      46 flags.go:64] FLAG: --default-not-ready-toleration-seconds=\"300\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912732      46 flags.go:64] FLAG: --default-unreachable-toleration-seconds=\"300\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912735      46 flags.go:64] FLAG: --default-watch-cache-size=\"100\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912738      46 flags.go:64] FLAG: --delete-collection-workers=\"1\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912741      46 flags.go:64] FLAG: --disable-admission-plugins=\"[]\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912751      46 flags.go:64] FLAG: --disable-http2-serving=\"false\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912753      46 flags.go:64] FLAG: --disabled-metrics=\"[]\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912757      46 flags.go:64] FLAG: --egress-selector-config-file=\"/var/lib/k0s/konnectivity.conf\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912760      46 flags.go:64] FLAG: --emulated-version=\"[]\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912764      46 flags.go:64] FLAG: --enable-admission-plugins=\"[NodeRestriction]\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912770      46 flags.go:64] FLAG: --enable-aggregator-routing=\"false\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912773      46 flags.go:64] FLAG: --enable-bootstrap-token-auth=\"true\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912775      46 flags.go:64] FLAG: --enable-garbage-collector=\"true\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912778      46 flags.go:64] FLAG: --enable-logs-handler=\"false\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912780      46 flags.go:64] FLAG: --enable-priority-and-fairness=\"true\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912783      46 flags.go:64] FLAG: --encryption-provider-config=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912786      46 flags.go:64] FLAG: --encryption-provider-config-automatic-reload=\"false\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912788      46 flags.go:64] FLAG: --endpoint-reconciler-type=\"none\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912791      46 flags.go:64] FLAG: --etcd-cafile=\"/var/lib/k0s/pki/etcd-ca.crt\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912794      46 flags.go:64] FLAG: --etcd-certfile=\"/var/lib/k0s/pki/apiserver-etcd-client.crt\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912797      46 flags.go:64] FLAG: --etcd-compaction-interval=\"5m0s\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912800      46 flags.go:64] FLAG: --etcd-count-metric-poll-period=\"1m0s\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912802      46 flags.go:64] FLAG: --etcd-db-metric-poll-interval=\"30s\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912804      46 flags.go:64] FLAG: --etcd-healthcheck-timeout=\"2s\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912807      46 flags.go:64] FLAG: --etcd-keyfile=\"/var/lib/k0s/pki/apiserver-etcd-client.key\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912810      46 flags.go:64] FLAG: --etcd-prefix=\"fr3\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912812      46 flags.go:64] FLAG: --etcd-readycheck-timeout=\"2s\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912815      46 flags.go:64] FLAG: --etcd-servers=\"[https://kmc-fr3-etcd:2379]\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912819      46 flags.go:64] FLAG: --etcd-servers-overrides=\"[]\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912823      46 flags.go:64] FLAG: --event-ttl=\"1h0m0s\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912826      46 flags.go:64] FLAG: --external-hostname=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912828      46 flags.go:64] FLAG: --feature-gates=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912833      46 flags.go:64] FLAG: --goaway-chance=\"0\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912836      46 flags.go:64] FLAG: --help=\"false\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912839      46 flags.go:64] FLAG: --http2-max-streams-per-connection=\"0\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912842      46 flags.go:64] FLAG: --kubelet-certificate-authority=\"/var/lib/k0s/pki/ca.crt\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912847      46 flags.go:64] FLAG: --kubelet-client-certificate=\"/var/lib/k0s/pki/apiserver-kubelet-client.crt\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912850      46 flags.go:64] FLAG: --kubelet-client-key=\"/var/lib/k0s/pki/apiserver-kubelet-client.key\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912852      46 flags.go:64] FLAG: --kubelet-port=\"10250\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912855      46 flags.go:64] FLAG: --kubelet-preferred-address-types=\"[InternalIP,ExternalIP,Hostname]\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912860      46 flags.go:64] FLAG: --kubelet-read-only-port=\"10255\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912862      46 flags.go:64] FLAG: --kubelet-timeout=\"5s\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912865      46 flags.go:64] FLAG: --kubernetes-service-node-port=\"0\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912868      46 flags.go:64] FLAG: --lease-reuse-duration-seconds=\"60\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912871      46 flags.go:64] FLAG: --livez-grace-period=\"0s\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912873      46 flags.go:64] FLAG: --log-flush-frequency=\"5s\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912876      46 flags.go:64] FLAG: --log-json-info-buffer-size=\"0\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912881      46 flags.go:64] FLAG: --log-json-split-stream=\"false\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912896      46 flags.go:64] FLAG: --log-text-info-buffer-size=\"0\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912900      46 flags.go:64] FLAG: --log-text-split-stream=\"false\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912905      46 flags.go:64] FLAG: --logging-format=\"text\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912908      46 flags.go:64] FLAG: --max-connection-bytes-per-sec=\"0\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912912      46 flags.go:64] FLAG: --max-mutating-requests-inflight=\"200\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912916      46 flags.go:64] FLAG: --max-requests-inflight=\"400\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912920      46 flags.go:64] FLAG: --min-request-timeout=\"1800\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912923      46 flags.go:64] FLAG: --oidc-ca-file=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912927      46 flags.go:64] FLAG: --oidc-client-id=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912930      46 flags.go:64] FLAG: --oidc-groups-claim=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912934      46 flags.go:64] FLAG: --oidc-groups-prefix=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912937      46 flags.go:64] FLAG: --oidc-issuer-url=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912940      46 flags.go:64] FLAG: --oidc-required-claim=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912946      46 flags.go:64] FLAG: --oidc-signing-algs=\"[RS256]\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912951      46 flags.go:64] FLAG: --oidc-username-claim=\"sub\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912955      46 flags.go:64] FLAG: --oidc-username-prefix=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912958      46 flags.go:64] FLAG: --peer-advertise-ip=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912962      46 flags.go:64] FLAG: --peer-advertise-port=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912966      46 flags.go:64] FLAG: --peer-ca-file=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912969      46 flags.go:64] FLAG: --permit-address-sharing=\"false\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912973      46 flags.go:64] FLAG: --permit-port-sharing=\"false\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912976      46 flags.go:64] FLAG: --profiling=\"false\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912980      46 flags.go:64] FLAG: --proxy-client-cert-file=\"/var/lib/k0s/pki/front-proxy-client.crt\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912985      46 flags.go:64] FLAG: --proxy-client-key-file=\"/var/lib/k0s/pki/front-proxy-client.key\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912989      46 flags.go:64] FLAG: --request-timeout=\"1m0s\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.912999      46 flags.go:64] FLAG: --requestheader-allowed-names=\"[front-proxy-client]\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913015      46 flags.go:64] FLAG: --requestheader-client-ca-file=\"/var/lib/k0s/pki/front-proxy-ca.crt\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913020      46 flags.go:64] FLAG: --requestheader-extra-headers-prefix=\"[X-Remote-Extra-]\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913025      46 flags.go:64] FLAG: --requestheader-group-headers=\"[X-Remote-Group]\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913035      46 flags.go:64] FLAG: --requestheader-username-headers=\"[X-Remote-User]\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913043      46 flags.go:64] FLAG: --runtime-config=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913050      46 flags.go:64] FLAG: --secure-port=\"30443\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913054      46 flags.go:64] FLAG: --service-account-extend-token-expiration=\"true\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913058      46 flags.go:64] FLAG: --service-account-issuer=\"[https://kubernetes.default.svc]\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913079      46 flags.go:64] FLAG: --service-account-jwks-uri=\"https://kubernetes.default.svc/openid/v1/jwks\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913084      46 flags.go:64] FLAG: --service-account-key-file=\"[/var/lib/k0s/pki/sa.pub]\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913092      46 flags.go:64] FLAG: --service-account-lookup=\"true\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913096      46 flags.go:64] FLAG: --service-account-max-token-expiration=\"0s\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913099      46 flags.go:64] FLAG: --service-account-signing-key-file=\"/var/lib/k0s/pki/sa.key\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913103      46 flags.go:64] FLAG: --service-cluster-ip-range=\"123.123.0.0/12\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913106      46 flags.go:64] FLAG: --service-node-port-range=\"30000-32767\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913114      46 flags.go:64] FLAG: --show-hidden-metrics-for-version=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913118      46 flags.go:64] FLAG: --shutdown-delay-duration=\"0s\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913121      46 flags.go:64] FLAG: --shutdown-send-retry-after=\"false\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913125      46 flags.go:64] FLAG: --shutdown-watch-termination-grace-period=\"0s\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913129      46 flags.go:64] FLAG: --storage-backend=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913132      46 flags.go:64] FLAG: --storage-initialization-timeout=\"1m0s\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913139      46 flags.go:64] FLAG: --storage-media-type=\"application/vnd.kubernetes.protobuf\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913143      46 flags.go:64] FLAG: --strict-transport-security-directives=\"[]\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913229      46 flags.go:64] FLAG: --tls-cert-file=\"/var/lib/k0s/pki/server.crt\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913235      46 flags.go:64] FLAG: --tls-cipher-suites=\"[TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256]\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913250      46 flags.go:64] FLAG: --tls-min-version=\"VersionTLS12\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913256      46 flags.go:64] FLAG: --tls-private-key-file=\"/var/lib/k0s/pki/server.key\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913260      46 flags.go:64] FLAG: --tls-sni-cert-key=\"[]\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913266      46 flags.go:64] FLAG: --token-auth-file=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913269      46 flags.go:64] FLAG: --tracing-config-file=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913273      46 flags.go:64] FLAG: --v=\"1\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913279      46 flags.go:64] FLAG: --version=\"false\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913284      46 flags.go:64] FLAG: --vmodule=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913290      46 flags.go:64] FLAG: --watch-cache=\"true\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913294      46 flags.go:64] FLAG: --watch-cache-sizes=\"[]\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.913324      46 options.go:228] external host was not specified, using 123.123.123.123" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.925802      46 server.go:142] Version: v1.31.2+k0s" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:24" level=info msg="I1219 14:49:24.925840      46 server.go:144] \"Golang settings\" GOGC=\"\" GOMAXPROCS=\"\" GOTRACEBACK=\"\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:25" level=info msg="I1219 14:49:25.645241      46 shared_informer.go:313] Waiting for caches to sync for node_authorizer" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:25" level=info msg="I1219 14:49:25.653647      46 plugins.go:157] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:25" level=info msg="I1219 14:49:25.653677      46 plugins.go:160] Loaded 13 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,ClusterTrustBundleAttest,CertificateSubjectRestriction,ValidatingAdmissionPolicy,ValidatingAdmissionWebhook,ResourceQuota." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:25" level=info msg="I1219 14:49:25.653866      46 instance.go:232] Using reconciler: none" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:25" level=info msg="I1219 14:49:25.655245      46 shared_informer.go:313] Waiting for caches to sync for *generic.policySource[*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicy,*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicyBinding,k8s.io/apiserver/pkg/admission/plugin/policy/validating.Validator]" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:25" level=info msg="I1219 14:49:25.689752      46 handler.go:286] Adding GroupVersion apiextensions.k8s.io v1 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:25" level=info msg="W1219 14:49:25.689784      46 genericapiserver.go:765] Skipping API apiextensions.k8s.io/v1beta1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:25" level=info msg="I1219 14:49:25.765693      46 deleted_kinds.go:60] NewResourceExpirationEvaluator with currentVersion: 1.31." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:25" level=info msg="I1219 14:49:25.821613      46 cacher.go:463] cacher (resourcequotas): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:25" level=info msg="I1219 14:49:25.859448      46 cacher.go:463] cacher (configmaps): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:25" level=info msg="I1219 14:49:25.884335      46 cacher.go:463] cacher (namespaces): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:25" level=info msg="I1219 14:49:25.897817      46 cacher.go:463] cacher (secrets): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:25" level=info msg="I1219 14:49:25.911490      46 cacher.go:463] cacher (serviceaccounts): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:25" level=info msg="I1219 14:49:25.920963      46 cacher.go:463] cacher (podtemplates): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:25" level=info msg="I1219 14:49:25.937254      46 cacher.go:463] cacher (limitranges): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:25" level=info msg="I1219 14:49:25.975242      46 cacher.go:463] cacher (persistentvolumes): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:25" level=info msg="I1219 14:49:25.987446      46 cacher.go:463] cacher (persistentvolumeclaims): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.059992      46 cacher.go:463] cacher (customresourcedefinitions.apiextensions.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.085277      46 cacher.go:463] cacher (endpoints): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.093352      46 cacher.go:463] cacher (nodes): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.155823      46 cacher.go:463] cacher (pods): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.174602      46 cacher.go:463] cacher (services): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.183793      46 cacher.go:463] cacher (serviceaccounts): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.192945      46 apis.go:118] Enabling API group \"\"." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.204096      46 cacher.go:463] cacher (replicationcontrollers): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.232473      46 handler.go:286] Adding GroupVersion  v1 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.232664      46 apis.go:105] API group \"internal.apiserver.k8s.io\" is not enabled, skipping." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.232696      46 apis.go:118] Enabling API group \"authentication.k8s.io\"." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.232737      46 apis.go:118] Enabling API group \"authorization.k8s.io\"." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.246454      46 cacher.go:463] cacher (horizontalpodautoscalers.autoscaling): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.253011      46 apis.go:118] Enabling API group \"autoscaling\"." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.256497      46 cacher.go:463] cacher (horizontalpodautoscalers.autoscaling): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.272142      46 cacher.go:463] cacher (jobs.batch): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.279838      46 apis.go:118] Enabling API group \"batch\"." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.282700      46 cacher.go:463] cacher (cronjobs.batch): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.291164      46 apis.go:118] Enabling API group \"certificates.k8s.io\"." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.296602      46 cacher.go:463] cacher (certificatesigningrequests.certificates.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.302085      46 apis.go:118] Enabling API group \"coordination.k8s.io\"." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.305419      46 cacher.go:463] cacher (leases.coordination.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.316980      46 apis.go:118] Enabling API group \"discovery.k8s.io\"." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.323870      46 cacher.go:463] cacher (endpointslices.discovery.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.332585      46 cacher.go:463] cacher (networkpolicies.networking.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.379175      46 cacher.go:463] cacher (ingresses.networking.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.387419      46 apis.go:118] Enabling API group \"networking.k8s.io\"." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.390549      46 cacher.go:463] cacher (ingressclasses.networking.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.397218      46 apis.go:118] Enabling API group \"node.k8s.io\"." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.400610      46 cacher.go:463] cacher (runtimeclasses.node.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.409009      46 apis.go:118] Enabling API group \"policy\"." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.411024      46 cacher.go:463] cacher (poddisruptionbudgets.policy): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.430855      46 cacher.go:463] cacher (roles.rbac.authorization.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.475059      46 cacher.go:463] cacher (rolebindings.rbac.authorization.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.498590      46 apis.go:118] Enabling API group \"rbac.authorization.k8s.io\"." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.500222      46 cacher.go:463] cacher (clusterroles.rbac.authorization.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.506713      46 cacher.go:463] cacher (clusterrolebindings.rbac.authorization.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.514466      46 apis.go:118] Enabling API group \"scheduling.k8s.io\"." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.518845      46 cacher.go:463] cacher (priorityclasses.scheduling.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.529903      46 cacher.go:463] cacher (storageclasses.storage.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.547482      46 cacher.go:463] cacher (volumeattachments.storage.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.566395      46 cacher.go:463] cacher (csinodes.storage.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.583421      46 cacher.go:463] cacher (csidrivers.storage.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.601235      46 apis.go:118] Enabling API group \"storage.k8s.io\"." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.601261      46 apis.go:105] API group \"storagemigration.k8s.io\" is not enabled, skipping." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.603924      46 cacher.go:463] cacher (csistoragecapacities.storage.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.619106      46 cacher.go:463] cacher (flowschemas.flowcontrol.apiserver.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.630866      46 cacher.go:463] cacher (prioritylevelconfigurations.flowcontrol.apiserver.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.643959      46 cacher.go:463] cacher (flowschemas.flowcontrol.apiserver.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.662960      46 apis.go:118] Enabling API group \"flowcontrol.apiserver.k8s.io\"." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.668010      46 cacher.go:463] cacher (prioritylevelconfigurations.flowcontrol.apiserver.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.697887      46 cacher.go:463] cacher (statefulsets.apps): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.698324      46 cacher.go:463] cacher (deployments.apps): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.713393      46 cacher.go:463] cacher (daemonsets.apps): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.731790      46 apis.go:118] Enabling API group \"apps\"." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.734096      46 cacher.go:463] cacher (replicasets.apps): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.741237      46 cacher.go:463] cacher (controllerrevisions.apps): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.751610      46 cacher.go:463] cacher (validatingwebhookconfigurations.admissionregistration.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.763818      46 cacher.go:463] cacher (mutatingwebhookconfigurations.admissionregistration.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.775073      46 cacher.go:463] cacher (validatingadmissionpolicies.admissionregistration.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.788583      46 apis.go:118] Enabling API group \"admissionregistration.k8s.io\"." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.794151      46 cacher.go:463] cacher (validatingadmissionpolicybindings.admissionregistration.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.804648      46 apis.go:118] Enabling API group \"events.k8s.io\"." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.804671      46 apis.go:105] API group \"resource.k8s.io\" is not enabled, skipping." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.845454      46 handler.go:286] Adding GroupVersion authentication.k8s.io v1 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.845475      46 genericapiserver.go:765] Skipping API authentication.k8s.io/v1beta1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.845520      46 genericapiserver.go:765] Skipping API authentication.k8s.io/v1alpha1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.847095      46 handler.go:286] Adding GroupVersion authorization.k8s.io v1 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.847114      46 genericapiserver.go:765] Skipping API authorization.k8s.io/v1beta1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.848629      46 handler.go:286] Adding GroupVersion autoscaling v2 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.850551      46 handler.go:286] Adding GroupVersion autoscaling v1 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.850569      46 genericapiserver.go:765] Skipping API autoscaling/v2beta1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.850581      46 genericapiserver.go:765] Skipping API autoscaling/v2beta2 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.853686      46 handler.go:286] Adding GroupVersion batch v1 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.853708      46 genericapiserver.go:765] Skipping API batch/v1beta1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.855538      46 handler.go:286] Adding GroupVersion certificates.k8s.io v1 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.855552      46 genericapiserver.go:765] Skipping API certificates.k8s.io/v1beta1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.855557      46 genericapiserver.go:765] Skipping API certificates.k8s.io/v1alpha1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.856273      46 handler.go:286] Adding GroupVersion coordination.k8s.io v1 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.856287      46 genericapiserver.go:765] Skipping API coordination.k8s.io/v1beta1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.856294      46 genericapiserver.go:765] Skipping API coordination.k8s.io/v1alpha1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.856826      46 handler.go:286] Adding GroupVersion discovery.k8s.io v1 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.856844      46 genericapiserver.go:765] Skipping API discovery.k8s.io/v1beta1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.858246      46 handler.go:286] Adding GroupVersion networking.k8s.io v1 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.858262      46 genericapiserver.go:765] Skipping API networking.k8s.io/v1beta1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.858268      46 genericapiserver.go:765] Skipping API networking.k8s.io/v1alpha1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.858671      46 handler.go:286] Adding GroupVersion node.k8s.io v1 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.858685      46 genericapiserver.go:765] Skipping API node.k8s.io/v1beta1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.858690      46 genericapiserver.go:765] Skipping API node.k8s.io/v1alpha1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.859361      46 handler.go:286] Adding GroupVersion policy v1 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.859388      46 genericapiserver.go:765] Skipping API policy/v1beta1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.861020      46 handler.go:286] Adding GroupVersion rbac.authorization.k8s.io v1 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.861036      46 genericapiserver.go:765] Skipping API rbac.authorization.k8s.io/v1beta1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.861041      46 genericapiserver.go:765] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.861441      46 handler.go:286] Adding GroupVersion scheduling.k8s.io v1 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.861455      46 genericapiserver.go:765] Skipping API scheduling.k8s.io/v1beta1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.861459      46 genericapiserver.go:765] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.863260      46 handler.go:286] Adding GroupVersion storage.k8s.io v1 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.863279      46 genericapiserver.go:765] Skipping API storage.k8s.io/v1beta1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.863283      46 genericapiserver.go:765] Skipping API storage.k8s.io/v1alpha1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.864278      46 handler.go:286] Adding GroupVersion flowcontrol.apiserver.k8s.io v1 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.865162      46 handler.go:286] Adding GroupVersion flowcontrol.apiserver.k8s.io v1beta3 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.865177      46 genericapiserver.go:765] Skipping API flowcontrol.apiserver.k8s.io/v1beta2 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.865184      46 genericapiserver.go:765] Skipping API flowcontrol.apiserver.k8s.io/v1beta1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.868286      46 handler.go:286] Adding GroupVersion apps v1 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.868305      46 genericapiserver.go:765] Skipping API apps/v1beta2 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.868309      46 genericapiserver.go:765] Skipping API apps/v1beta1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.869879      46 handler.go:286] Adding GroupVersion admissionregistration.k8s.io v1 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.869893      46 genericapiserver.go:765] Skipping API admissionregistration.k8s.io/v1beta1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.869898      46 genericapiserver.go:765] Skipping API admissionregistration.k8s.io/v1alpha1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.870436      46 handler.go:286] Adding GroupVersion events.k8s.io v1 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.870450      46 genericapiserver.go:765] Skipping API events.k8s.io/v1beta1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.871399      46 deleted_kinds.go:60] NewResourceExpirationEvaluator with currentVersion: 1.31." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.889660      46 handler.go:286] Adding GroupVersion apiregistration.k8s.io v1 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="W1219 14:49:26.889680      46 genericapiserver.go:765] Skipping API apiregistration.k8s.io/v1beta1 because it has no resources." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:26" level=info msg="I1219 14:49:26.894445      46 cacher.go:463] cacher (apiservices.apiregistration.k8s.io): initialized" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.567621      46 dynamic_cafile_content.go:160] \"Starting controller\" name=\"request-header::/var/lib/k0s/pki/front-proxy-ca.crt\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.567643      46 dynamic_cafile_content.go:160] \"Starting controller\" name=\"client-ca-bundle::/var/lib/k0s/pki/ca.crt\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.568491      46 secure_serving.go:213] Serving securely on [::]:30443" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.568561      46 genericapiserver.go:683] [graceful-termination] waiting for shutdown to be initiated" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.568608      46 dynamic_serving_content.go:135] \"Starting controller\" name=\"serving-cert::/var/lib/k0s/pki/server.crt::/var/lib/k0s/pki/server.key\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.568727      46 customresource_discovery_controller.go:292] Starting DiscoveryController" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.568750      46 tlsconfig.go:243] \"Starting DynamicServingCertificateController\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.568774      46 dynamic_serving_content.go:135] \"Starting controller\" name=\"aggregator-proxy-cert::/var/lib/k0s/pki/front-proxy-client.crt::/var/lib/k0s/pki/front-proxy-client.key\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.568871      46 apf_controller.go:377] Starting API Priority and Fairness config controller" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.568921      46 system_namespaces_controller.go:66] Starting system namespaces controller" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.569338      46 gc_controller.go:78] Starting apiserver lease garbage collector" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.570001      46 cluster_authentication_trust_controller.go:443] Starting cluster_authentication_trust_controller controller" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.570010      46 shared_informer.go:313] Waiting for caches to sync for cluster_authentication_trust_controller" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.570109      46 controller.go:119] Starting legacy_token_tracking_controller" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.570116      46 shared_informer.go:313] Waiting for caches to sync for configmaps" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.570170      46 controller.go:142] Starting OpenAPI controller" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.570203      46 controller.go:90] Starting OpenAPI V3 controller" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.570221      46 naming_controller.go:294] Starting NamingConditionController" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.570238      46 establishing_controller.go:81] Starting EstablishingController" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.570256      46 nonstructuralschema_controller.go:195] Starting NonStructuralSchemaConditionController" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.570269      46 apiapproval_controller.go:189] Starting KubernetesAPIApprovalPolicyConformantConditionController" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.570282      46 crd_finalizer.go:269] Starting CRDFinalizer" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.577979      46 dynamic_cafile_content.go:160] \"Starting controller\" name=\"client-ca-bundle::/var/lib/k0s/pki/ca.crt\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.578065      46 dynamic_cafile_content.go:160] \"Starting controller\" name=\"request-header::/var/lib/k0s/pki/front-proxy-ca.crt\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.578809      46 local_available_controller.go:156] Starting LocalAvailability controller" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.578819      46 cache.go:32] Waiting for caches to sync for LocalAvailability controller" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.578840      46 remote_available_controller.go:411] Starting RemoteAvailability controller" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.578845      46 cache.go:32] Waiting for caches to sync for RemoteAvailability controller" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.578865      46 apiservice_controller.go:100] Starting APIServiceRegistrationController" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.578869      46 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.578887      46 aggregator.go:169] waiting for initial CRD sync..." component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.579117      46 crdregistration_controller.go:114] Starting crd-autoregister controller" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.579124      46 shared_informer.go:313] Waiting for caches to sync for crd-autoregister" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.579215      46 controller.go:78] Starting OpenAPI AggregationController" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.579244      46 controller.go:80] Starting OpenAPI V3 AggregationController" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.656055      46 shared_informer.go:320] Caches are synced for *generic.policySource[*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicy,*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicyBinding,k8s.io/apiserver/pkg/admission/plugin/policy/validating.Validator]" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.656076      46 policy_source.go:224] refreshing policies" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.668935      46 apf_controller.go:382] Running API Priority and Fairness config worker" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.668954      46 apf_controller.go:385] Running API Priority and Fairness periodic rebalancing process" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.669514      46 controller.go:95] Found stale data, removed previous endpoints on kubernetes service, apiserver didn't exit successfully previously" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.670191      46 shared_informer.go:320] Caches are synced for configmaps" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.670199      46 shared_informer.go:320] Caches are synced for cluster_authentication_trust_controller" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.684273      46 cache.go:39] Caches are synced for LocalAvailability controller" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.684523      46 cache.go:39] Caches are synced for RemoteAvailability controller" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.685358      46 cache.go:39] Caches are synced for APIServiceRegistrationController controller" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.702976      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=4338190911497974454" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.705875      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=8268745603614721120" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.706064      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=563285439194438516" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.706073      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=9119316630857075395" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.706197      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=8785056763392181058" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.719760      46 controller.go:615] quota admission added evaluator for: leases.coordination.k8s.io" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.751973      46 shared_informer.go:320] Caches are synced for node_authorizer" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.752695      46 remote_available_controller.go:448] \"Unhandled Error\" err=\"v1beta1.metrics.k8s.io failed with: failing or missing response from https://123.123.123.123:443/apis/metrics.k8s.io/v1beta1: Get \\\"https://123.123.123.123:443/apis/metrics.k8s.io/v1beta1\\\": No agent available\" logger=\"UnhandledError\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.776992      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=2885487272180443729" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.785127      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=1239428343698797741" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.785519      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=4893725708088100864" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.785784      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=7812015050867375332" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.785959      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=1479530501443762347" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.791978      46 remote_available_controller.go:448] \"Unhandled Error\" err=\"v1beta1.metrics.k8s.io failed with: failing or missing response from https://123.123.123.123:443/apis/metrics.k8s.io/v1beta1: Get \\\"https://123.123.123.123:443/apis/metrics.k8s.io/v1beta1\\\": No agent available\" logger=\"UnhandledError\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.810004      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=5341311895114018911" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.810662      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=7788537446236271924" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.813988      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=4694643239194843789" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.815818      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=326963793105511329" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.820611      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=4094644801992451044" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.849738      46 remote_available_controller.go:448] \"Unhandled Error\" err=\"v1beta1.metrics.k8s.io failed with: failing or missing response from https://123.123.123.123:443/apis/metrics.k8s.io/v1beta1: Get \\\"https://123.123.123.123:443/apis/metrics.k8s.io/v1beta1\\\": No agent available\" logger=\"UnhandledError\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.856107      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=4180612167110407831" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.857007      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=800223481053404449" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.857750      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=8132311226499820926" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.857809      46 handler_discovery.go:450] Starting ResourceDiscoveryManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.859609      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=1796733967947162709" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.860025      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=4157558264311806998" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.861278      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=2444620300262207623" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="I1219 14:49:27.862478      46 handler.go:286] Adding GroupVersion metrics.k8s.io v1beta1 to ResourceManager" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.863458      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=4577596227564582133" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.863778      46 controller.go:146] \"Unhandled Error\" err=<" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="\tError updating APIService \"v1beta1.metrics.k8s.io\" with err: failed to download v1beta1.metrics.k8s.io: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: error trying to reach service: No agent available" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="\t, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg=" > logger=\"UnhandledError\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.865814      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=4321648861154923610" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.867390      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=8624976380386823261" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="W1219 14:49:27.875736      46 handler_proxy.go:99] no RequestInfo found in the context" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.876089      46 controller.go:146] \"Unhandled Error\" err=<" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="\tError updating APIService \"v1beta1.metrics.k8s.io\" with err: failed to download v1beta1.metrics.k8s.io: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="\t, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg=" > logger=\"UnhandledError\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.878320      46 remote_available_controller.go:448] \"Unhandled Error\" err=\"v1beta1.metrics.k8s.io failed with: failing or missing response from https://123.123.123.123:443/apis/metrics.k8s.io/v1beta1: Get \\\"https://123.123.123.123:443/apis/metrics.k8s.io/v1beta1\\\": No agent available\" logger=\"UnhandledError\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.882798      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=7523746904920695982" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.886214      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=8606434480158600905" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.886410      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=9093852836716231391" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.886688      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=7537047413429461346" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.886834      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=8958076786102950433" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.886969      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=660881177216357472" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.887485      46 remote_available_controller.go:448] \"Unhandled Error\" err=\"v1beta1.metrics.k8s.io failed with: failing or missing response from https://123.123.123.123:443/apis/metrics.k8s.io/v1beta1: Get \\\"https://123.123.123.123:443/apis/metrics.k8s.io/v1beta1\\\": No agent available\" logger=\"UnhandledError\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.888193      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=6838305674774118751" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.888461      46 controller.go:146] \"Unhandled Error\" err=<" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="\tError updating APIService \"v1beta1.metrics.k8s.io\" with err: failed to download v1beta1.metrics.k8s.io: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: error trying to reach service: No agent available" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="\t, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg=" > logger=\"UnhandledError\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.892900      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=8214738347937850179" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.892900      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=1409559527652406066" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.893007      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=5726696642937173245" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.893119      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=1205310518351752516" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.893177      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=5161420542815568981" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.893857      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=839711473956106355" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="W1219 14:49:27.921703      46 handler_proxy.go:99] no RequestInfo found in the context" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.921772      46 controller.go:146] \"Unhandled Error\" err=<" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="\tError updating APIService \"v1beta1.metrics.k8s.io\" with err: failed to download v1beta1.metrics.k8s.io: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="\t, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg=" > logger=\"UnhandledError\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.921835      46 remote_available_controller.go:448] \"Unhandled Error\" err=\"v1beta1.metrics.k8s.io failed with: failing or missing response from https://123.123.123.123:443/apis/metrics.k8s.io/v1beta1: Get \\\"https://123.123.123.123:443/apis/metrics.k8s.io/v1beta1\\\": No agent available\" logger=\"UnhandledError\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.926267      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=872153870479479411" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.927054      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=3771689690801517699" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.927222      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=6123129644955176827" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.929488      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=448006432760155709" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.931253      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=604323509426099733" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.931318      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=77500085895216582" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.931947      46 remote_available_controller.go:448] \"Unhandled Error\" err=\"v1beta1.metrics.k8s.io failed with: failing or missing response from https://123.123.123.123:443/apis/metrics.k8s.io/v1beta1: Get \\\"https://123.123.123.123:443/apis/metrics.k8s.io/v1beta1\\\": No agent available\" logger=\"UnhandledError\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.933568      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=6411311908553482091" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.935271      46 controller.go:146] \"Unhandled Error\" err=<" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="\tError updating APIService \"v1beta1.metrics.k8s.io\" with err: failed to download v1beta1.metrics.k8s.io: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: error trying to reach service: No agent available" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="\t, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg=" > logger=\"UnhandledError\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.935209      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=9080895250997425628" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.935288      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=9040809026789930104" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.935420      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=457333319365437642" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.935511      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=6827390666218976431" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.935735      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=3337686810772714110" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.938434      36 server.go:579] \"Failed to get a backend\" err=\"No agent available\" dialID=7088619259342354147" component=konnectivity stream=stderr
time="2024-12-19 14:49:27" level=info msg="E1219 14:49:27.944179      46 remote_available_controller.go:448] \"Unhandled Error\" err=\"v1beta1.metrics.k8s.io failed with: failing or missing response from https://123.123.123.123:443/apis/metrics.k8s.io/v1beta1: Get \\\"https://123.123.123.123:443/apis/metrics.k8s.io/v1beta1\\\": No agent available\" logger=\"UnhandledError\"" component=kube-apiserver stream=stderr
time="2024-12-19 14:49:27" level=info msg="W1219 14:49:27.944480      46 handler_proxy.go:99] no RequestInfo found in the context" component=kube-apiserver stream=stderr

franpog859 added the bug Something isn't working label Dec 17, 2024

makhov self-assigned this Dec 18, 2024

makhov mentioned this issue Dec 18, 2024

Update HA k0smotron cluster fix & test #858

Merged

makhov closed this as completed in #858 Dec 19, 2024

makhov reopened this Dec 19, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updating HA clusters fails #857

Updating HA clusters fails #857

franpog859 commented Dec 17, 2024 •

edited

Loading

franpog859 commented Dec 17, 2024 •

edited

Loading

makhov commented Dec 18, 2024

franpog859 commented Dec 19, 2024

makhov commented Dec 19, 2024

franpog859 commented Dec 19, 2024

Updating HA clusters fails #857

Updating HA clusters fails #857

Comments

franpog859 commented Dec 17, 2024 • edited Loading

What happened?

Involved Areas

What did you expect to happen?

Steps to reproduce

k0smotron version

k0s version

Anything else we need to know?

franpog859 commented Dec 17, 2024 • edited Loading

makhov commented Dec 18, 2024

franpog859 commented Dec 19, 2024

makhov commented Dec 19, 2024

franpog859 commented Dec 19, 2024

franpog859 commented Dec 17, 2024 •

edited

Loading

franpog859 commented Dec 17, 2024 •

edited

Loading