As we were seeing some debug options in kubernetes in last two articles, in this let’s speak about one of the important key issues in kubernetes related to troubleshooting an issue. Kubernetes has lack of built-in observability tool which seen as one significant downside of it. As we aware log and event metrics are essential in troubleshooting or managing the resources or services.
In Kubernetes, we have to use some third-party tools to solve this issue, in this section lets check about some opensource tools to watch kubernetes events.
Key Problem
As we know applications running on Kubernetes cluster are dynamic in nature. Which means pods, replicas, deployments in your cluster keeps going on and off over the period due to their ephemeral nature. It is important that we should check what happened and why the resources go on and off, there could be various reasons, few could be.
- to debug historic incidents
- to debug common tasks like:
- finding info and events related to Kubernetes resources (like pods, replicasets, deployments, etc) that have been deleted, like.
- finding info related to pods/replicasets that are replaced by newer pods/replicasets after a deployment update
- getting details of pods evicted from a lost node
- getting the nodes availability and any lost node details.
- knowing rollout details of older deployments
- discovering hosts where pods a from previous deployment were running
- retrieving timings of pod replacements and their health checks
- long term behavioral analysis of your workloads running on your Kubernetes cluster
- and so on…
Simple word, we should have the all the information about the events happening in your Kubernetes cluster.
About Kubernetes Events
Kubernetes events show what is happening in a cluster when there is a state change or error from other resources in the system. It offers you information regarding changes, such as why the system cannot pull the docker image or why some pods were evicted from the cluster. Events are resource types created automatically by all core components and extensions in a cluster through the API Server.
Even though it provides by default, it has various limitations,
- Kubernetes Events can generally only be accessed using kubectl
- The default retention period of kubernetes events is 1 hour.
- The retention period can be increased using –event-ttl flag of kube-apiserver. But doing so can cause issues with the cluster’s key-value store.
- There is no way to visualize these events.
Continue reading on Sloop — Kubernetes Events History Visualization — FoxuTech
