How to Get Logs of Deployment from Kubernetes

1. Overview

Logs are crucial for monitoring and debugging applications, providing insights into how they perform and where issues might arise. In Kubernetes, logs are especially valuable, as they help track the behavior of applications running in complex, distributed environments.

In this tutorial, we’ll explore ways to retrieve logs from a Kubernetes deployment, offering techniques for effective troubleshooting and performance monitoring.

2. Understanding Kubernetes Logs

Kubernetes generates logs at the pod level, as each pod hosts one or more containers that generate runtime logs. A deployment manages a group of identical pods, and when we want to get logs from a deployment, we actually fetch the logs from its underlying pods.

The Kubernetes command-line tool, kubectl, is the primary way to interact with our Kubernetes cluster, including fetching logs. Let’s go step-by-step through the process of accessing these logs.

3. Accessing Deployment Logs

Kubernetes provides several commands to retrieve logs, each serving different scenarios depending on whether we want logs from individual pods, specific containers, or the entire deployment.

The first step in fetching logs from a deployment is to list the pods managed by it. To do this, we can use the following command to get the pod names for a specific deployment:

kubectl get pods --selector app=ops_deployment

NAME                               READY   STATUS    RESTARTS   AGE
ops_deployment-6b79f7f5d8-bz2kd     1/1     Running   0          15m
ops_deployment-6b79f7f5d8-j9t4x     1/1     Running   0          15m
ops_deployment-6b79f7f5d8-ptb7c     1/1     Running   0          15m

Each of these pods has its own logs, which we can retrieve individually or as a group.

3.1. Fetching Logs from a Specific Pod

To view logs from a specific pod, we use the kubectl logs command followed by the pod’s name:

kubectl logs <pod_name>

For instance, let’s look at logs from one of the three pods of our ops_deployment:

kubectl logs ops_deployment-6b79f7f5d8-bz2kd

2024-11-02T10:25:24.842Z INFO  Starting application...
2024-11-02T10:25:25.123Z INFO  Application started successfully
2024-11-02T10:26:01.987Z WARN  Connection timeout on service call
2024-11-02T10:26:05.543Z ERROR Service failed with status 500

This gives us logs generated by ops_deployment-6b79f7f5d8-bz2kd pod, allowing us to troubleshoot issues or monitor its activity.

Alternatively, we can loop through each pod in our ops_deployment using a simple for loop. This approach streamlines the process of fetching logs from each pod in a multi-pod deployment:

for pod in $(kubectl get pods -l app=ops_deployment -o name); do
    echo "===== Logs from ${pod} ====="
    kubectl logs $pod
    echo ""
done

3.2. Retrieving Logs from All Pods in a Deployment

To streamline log retrieval and avoid fetching logs from each pod individually, we can gather logs from all pods in a deployment with the following command:

kubectl logs -l app=ops_deployment --all-containers

2024-11-02T10:28:45.543Z INFO Starting container in pod ops_deployment-6b79f7f5d8-bz2kd
2024-11-02T10:28:46.643Z INFO Starting container in pod ops_deployment-6b79f7f5d8-j9t4x
2024-11-02T10:28:47.743Z INFO Starting container in pod ops_deployment-6b79f7f5d8-ptb7c

This is especially useful when our deployment manages several pods.

3.3. Fetching Logs for a Specific Container in a Pod

If our pod contains multiple containers, such as in a sidecar pattern, we can retrieve logs from a specific container. In Kubernetes, a sidecar is an additional container that runs alongside the main application container within the same pod. This design allows the sidecar to perform supportive tasks – like logging, monitoring, or proxying – without impacting the main container’s responsibilities. To fetch logs from a particular container in a multi-container pod, we specify the container name in the kubectl logs command:

kubectl logs ops_deployment-6b79f7f5d8-bz2kd -c app-container

2024-11-02T10:30:16.231Z INFO  App container started successfully.

This allows us to view logs from a specific container specified in the command. It’s particularly useful when we need logs from one container in a multi-container pod scenario.

3.4. Retrieving Previous Logs

If a pod has restarted or crashed, Kubernetes lets us access logs from the previous instance by using the previous flag. This option is helpful for troubleshooting, as it retrieves logs from the last terminated container, providing insights into what might have caused the failure:

kubectl logs <pod_name> --previous

This is particularly useful for identifying issues that caused a pod to crash.

3.5. Streaming Live Logs from a Pod

To monitor logs in real-time, Kubernetes lets us stream logs as they are generated by using the -f flag:

kubectl logs <pod_name> -f

This command continuously displays new log entries, making it useful for actively tracking application behavior and debugging issues as they occur.

3.6. Using Centralized Log Management Tools

For production clusters with extensive logging needs, we can make use of a centralized solution like Elasticsearch, Fluentd, and Kibana (EFK) stack or Prometheus and Grafana for logging purposes. These tools aggregate logs across nodes, offering a searchable, comprehensive view of all Kubernetes activity.

Let’s look into how these three tools come together in the stack.

Elasticsearch plays a key role in our logging pipeline, acting as both the storage and search engine. It efficiently indexes logs, making them easily searchable and accessible. With its ability to handle large volumes of data, Elasticsearch enables us to perform fast queries and analyze even the most complex log datasets. This allows us to quickly locate specific logs, troubleshoot issues, and monitor application behavior in real-time, ensuring a seamless and responsive logging experience.

Fluentd is our go-to log collector and forwarder. It seamlessly gathers logs from various sources across our environment, processes them, and sends them to Elasticsearch for storage. With its extensive plugin ecosystem, Fluentd lets us filter, format, and enrich log data to meet our specific needs. This flexibility makes it a perfect fit for managing logs efficiently, especially in a dynamic Kubernetes cluster with diverse log sources.

Kibana provides a user-friendly way to explore and visualize the log data stored in Elasticsearch. It lets us create dashboards, charts, and graphs to monitor trends and monitor system health. With its powerful visualization tools, we can quickly spot patterns and detect anomalies, making it easier to stay on top of monitoring and take a proactive approach to troubleshooting.

These tools simplify log management by providing us with an efficient solution for large-scale log collection and analysis.

4. Best Practices for Managing Kubernetes Logs

Retrieving logs is essential for troubleshooting, but following best practices helps ensure efficient log management in our Kubernetes clusters:

• Set Log Retention Policies: To save storage and avoid clutter, we can set retention policies based on our operational and compliance needs, ensuring older logs are deleted after a set period
• Monitor Pod Resource Usage: High volumes of logs can lead to performance bottlenecks. We should regularly monitor our pods’ CPU and memory usage to ensure they’re not being affected by excessive logging
• Limit Log Verbosity: We should ensure that our applications only log what’s necessary. Too much verbosity can make it difficult to pinpoint issues and can clutter our log files
• Centralize Logs: Using centralized logging solutions like ELK (Elasticsearch, Logstash, Kibana) or Fluentd helps us aggregate logs from across the cluster, simplifying monitoring and making it easier to pinpoint specific issues

5. Conclusion

In this article, we explored various approaches to retrieving logs from a Kubernetes deployment, ranging from basic pod-level logs to real-time streaming. We also highlighted best practices to streamline and enhance log management.

By leveraging these commands and tools, we can effectively monitor our deployments, swiftly identify issues, and take proactive measures to maintain a healthy and reliable environment.