How do you renew the kubernetes expired certificate? Kubernetes uses various certificates for secure communication within the cluster. These certificates are essential for securing communication between various components, such as API servers, kubelets, and etcd.
Environment:
root@kmaster1:~# kubectl version --short
Client Version: v1.22.0
Server Version: v1.22.15
root@kmaster1:~#
Here are some of the important certificates used in Kubernetes:
- CA Certificate: The Certificate Authority (CA) certificate is used to sign and issue other certificates in the cluster. It is used to establish trust between various components and users in the Kubernetes cluster.
- API Server Certificate: The API server certificate is used by the Kubernetes API server to authenticate itself to the kubelets and other components in the cluster. It is crucial for securing the Kubernetes API server.
- Service Account Key Pair: Kubernetes generates a key pair for each service account to authenticate and authorize access to the API server. These key pairs are used by the API server to verify the identity of the services and pods that are accessing the API server.
- Kubelet Client Certificate: Each kubelet in the cluster is issued a client certificate that it uses to authenticate itself to the API server. This certificate is used to ensure secure communication between the kubelets and the API server.
- Kubelet Server Certificate: This certificate is used by the API server to authenticate itself to the kubelets. It ensures secure communication between the API server and the kubelets.
- Etcd Client and Server Certificates: Etcd, the key-value store used by Kubernetes for storing cluster data, also requires certificates for secure communication between the etcd nodes and other components in the cluster.
How to check the kubernetes certificate status?
kubeadm command helps to get the consolidated certificate status.
Example:
root@kmaster1:~# kubeadm certs check-expiration [check-expiration] Reading configuration from the cluster... [check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml' [check-expiration] Error reading configuration from the Cluster. Falling back to default configuration CERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED admin.conf Oct 15, 2023 17:02 UTC <invalid> no apiserver Oct 15, 2023 17:02 UTC <invalid> ca no apiserver-etcd-client Oct 15, 2023 17:02 UTC <invalid> etcd-ca no apiserver-kubelet-client Oct 15, 2023 17:02 UTC <invalid> ca no controller-manager.conf Oct 15, 2023 17:02 UTC <invalid> no etcd-healthcheck-client Oct 15, 2023 17:02 UTC <invalid> etcd-ca no etcd-peer Oct 15, 2023 17:02 UTC <invalid> etcd-ca no etcd-server Oct 15, 2023 17:02 UTC <invalid> etcd-ca no front-proxy-client Oct 15, 2023 17:02 UTC <invalid> front-proxy-ca no scheduler.conf Oct 15, 2023 17:02 UTC <invalid> no CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED ca Oct 12, 2032 17:02 UTC 8y no etcd-ca Oct 12, 2032 17:02 UTC 8y no front-proxy-ca Oct 12, 2032 17:02 UTC 8y no
How to renew all the expired certificates?
Use the kubeadm command to renew all the expired certificates. You need to invoke this command across all the master nodes to renew it. In a multi-master kubernetes environment, if you do not renew it on any master node might not be able to join the cluster.
root@kmaster1:~# kubeadm certs renew all [renew] Reading configuration from the cluster... [renew] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml' [renew] Error reading configuration from the Cluster. Falling back to default configuration certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed certificate for serving the Kubernetes API renewed certificate the apiserver uses to access etcd renewed certificate for the API server to connect to kubelet renewed certificate embedded in the kubeconfig file for the controller manager to use renewed certificate for liveness probes to healthcheck etcd renewed certificate for etcd nodes to communicate with each other renewed certificate for serving etcd renewed certificate for the front proxy client renewed certificate embedded in the kubeconfig file for the scheduler manager to use renewed Done renewing certificates. You must restart the kube-apiserver, kube-controller-manager, kube-scheduler and etcd, so that they can use the new certificates. root@kmaster1:~#
Check the certificate expiration
root@kmaster1:~# kubeadm certs check-expiration [check-expiration] Reading configuration from the cluster... [check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml' CERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED admin.conf Nov 01, 2024 11:46 UTC 364d no apiserver Nov 01, 2024 11:46 UTC 364d ca no apiserver-etcd-client Nov 01, 2024 11:46 UTC 364d etcd-ca no apiserver-kubelet-client Nov 01, 2024 11:46 UTC 364d ca no controller-manager.conf Nov 01, 2024 11:46 UTC 364d no etcd-healthcheck-client Nov 01, 2024 11:46 UTC 364d etcd-ca no etcd-peer Nov 01, 2024 11:46 UTC 364d etcd-ca no etcd-server Nov 01, 2024 11:46 UTC 364d etcd-ca no front-proxy-client Nov 01, 2024 11:46 UTC 364d front-proxy-ca no scheduler.conf Nov 01, 2024 11:46 UTC 364d no CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED ca Oct 12, 2032 17:02 UTC 8y no etcd-ca Oct 12, 2032 17:02 UTC 8y no front-proxy-ca Oct 12, 2032 17:02 UTC 8y no root@kmaster1:~#
Containerizing the kube components has become the standard practice in modern Kubernetes deployments. Since version 1.6, Kubernetes has made significant advancements and improvements, with subsequent versions bringing additional features, enhancements, and bug fixes to the container orchestration platform. so we can check the container status of the kube components to ensure all the required components are running post-cert renewal.
root@kmaster1:~# crictl ps CONTAINER CREATED STATE NAME POD ID POD 913c587ffca94 10 seconds ago Running calico-node 4d5f2d474f414 calico-node-sf4p6 c55db51eb460a 13 seconds ago Running speaker e05297000f1d7 speaker-bdb42 d4ace950e7661 14 seconds ago Running kube-proxy f025f563e8c47 kube-proxy-s98mm 1ffdd4d1acfde 38 seconds ago Running kube-apiserver adb8e14a168b1 kube-apiserver-kmaster2 78864fe54476e 4 minutes ago Running etcd ddbd6921ad883 etcd-kmaster2 4af5d035c11e1 10 hours ago Running kube-scheduler 5bf51b4ff24a4 kube-scheduler-kmaster2 3a348db904341 10 hours ago Running kube-controller-manager 895aab2008071 kube-controller-manager-kmaster2 root@kmaster1:~#
Managing these certificates is crucial for the security of the Kubernetes cluster. It involves generating and renewing certificates, distributing them to the appropriate components, and ensuring that they are kept secure and up to date. Kubernetes provides various tools and APIs for managing these certificates, and there are also third-party tools available for this purpose.