Kubernetes Ingress Configuration for External Services

1. Overview

Configuring Kubernetes Ingress for external services helps manage traffic routing from outside the cluster to internal services or external endpoints. It supports IP- and hostname-based proxying, enabling flexible and secure communication using methods like TLS termination. Managing dynamic IPs and optimizing ingress traffic improves scalability and performance.

Implementing best practices such as static IP management and proper egress traffic handling in cloud environments like AKS can help build a resilient Kubernetes architecture.

In this tutorial, we’ll explore different methods for proxying traffic and securing external services in Kubernetes. These methods are essential for configuring Kubernetes Ingress and managing the traffic.

2. Proxying to External Services to Configure Kubernetes Ingress

There are two primary methods for proxying external services via Kubernetes Ingress. Let’s explore them now.

2.1. Proxying via IP Address for Kubernetes Ingress

Kubernetes requires a combination of a Service without a selector and an Endpoints resource to route traffic to an external service using an IP address.

Service defines the port for incoming traffic but doesn’t connect to internal pods since the target is external:

apiVersion: v1
kind: Service
metadata:
name: external-service
spec:
ports:
  - protocol: TCP
port: 80

Endpoints map the external IP address to the Service, ensuring traffic is routed to the right place:

apiVersion: v1
kind: Endpoints
metadata:
name: external-service
subsets:
  - addresses:
  - ip: 1.2.3.4 # External IP
ports:
  - port: 80

After setting up the Service and Endpoints, we can create an Ingress resource to define traffic routing from an external domain:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: external-ingress
spec:
rules:
  - host: external.example.com
http:
paths:
  - path: /
backend:
service:
name: external-service
port:
number: 80

This configuration proxies requests from external.example.com to the external service using the specified IP address.

2.2. Proxying via Hostname

For services identified by a hostname (such as a remote API), Kubernetes uses an ExternalName Service, which simplifies the setup by mapping a Service to an external hostname:

apiVersion: v1
kind: Service
metadata:
name: external-service
spec:
type: ExternalName
externalName: my-external-backend.example.com

The Ingress resource directs traffic as follows:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: external-ingress
spec:
rules:
  - host: external.example.com
http:
paths:
  - path: /
backend:
service:
name: external-service
port:
number: 80

With this setup, requests to external.example.com are proxied to my-external-backend.example.com.

2.3. Handling Dynamic IPs in Ingress

Managing dynamic IPs presents unique challenges. If the external service’s IP frequently changes, the ExternalName Service is more practical since it resolves the hostname on each request, accommodating any IP changes automatically. In contrast, if we use an IP-based setup, it must require manually updating the Endpoints resource whenever the IP changes.

For highly dynamic environments, we should consider using DNS-based external services where IP addresses are resolved dynamically, avoiding the need to modify configurations manually.

2.4. Ensuring TLS Termination for External Services

For secure communication, we have to configure TLS termination in the Ingress controller. By default, even if the incoming traffic uses HTTPS, the proxying to external services may still happen over HTTP. To ensure end-to-end encryption, we can use the following annotation in the Ingress resource:

annotations:
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"

We then set the servicePort to 443 to maintain a secure connection to the external service:

backend:
service:
name: external-service
port:
number: 443

For ExternalName Services, TLS termination at the external service’s hostname ensures that the secure channel is maintained throughout the routing process.

3. Best Practices for Kubernetes Ingress with External Services

Here are the brief details about the best practices for Kubernetes Ingress with external services:

3.1. Static IPs for Stability

Using static IPs for Load Balancers and external services is essential when working in cloud environments such as Azure or AWS. We must ensure the Ingress resource routes traffic through a fixed IP for consistent routing.

3.2. Use Web Application Firewalls (WAF)

For added security, we can deploy a WAF before the Ingress controller to scan traffic for potential vulnerabilities. Azure Kubernetes Service (AKS) integrates easily with WAF solutions like Azure Application Gateway, which adds a layer of security to external traffic.

3.3. Ingress Controller Selection

We can choose an Ingress controller that fits the specific needs. While NGINX is the most common, options like Traefik or HAProxy may offer additional features tailored to different environments or workloads.

3.4. TLS Everywhere

We must ensure secure traffic by enabling TLS on both the client-side (Ingress controller) and the server-side (external service). Hence, it is essential to configure HTTPS for backends when dealing with sensitive data or external services.

3.5. EndpointSlices

For large-scale environments with multiple external endpoints, we can consider using EndpointSlices instead of traditional Endpoints. This newer API scales better and performs better, especially in larger clusters.

4. Handling Egress Traffic in AKS for Kubernetes Ingress

When configuring Kubernetes Ingress for external services in Azure Kubernetes Service (AKS), handling egress traffic efficiently and securely is crucial. Here are the key strategies:

4.1. User-Defined Routes (UDR)

UDRs direct outbound traffic from the AKS cluster through specific IPs or NAT gateways. It makes sure that egress traffic can be routed through secure channels or specific IP addresses required by external services for whitelisting.

4.2. NAT Gateway for Static Egress IP

Using a NAT Gateway helps ensure that all outbound traffic from the AKS cluster uses a single, static public IP. This is particularly important for services that require static IP whitelisting, ensuring consistency in external communication.

4.3. Network Security Groups (NSGs)

Apply NSGs to restrict which AKS pods can access external services. NSGs help limit egress traffic to specific IP ranges or services, reducing exposure to unwanted external services.

4.4. Azure Firewall

To provide an extra layer of security and ensure the safety of the deployments, we can use Azure Firewall to monitor and control outbound traffic. It offers centralized management of egress rules and enhances security with threat detection and logging features.

By combining UDR, NAT Gateway, NSGs, and Azure Firewall, AKS clusters can securely and efficiently manage egress traffic, particularly when interacting with external services through Kubernetes Ingress.

5. Conclusion

In this article, we’ve explored the different methods for configuring Kubernetes Ingress for external services, handling TLS termination, and optimizing egress traffic management for AKS clusters.