Load Balancers Basics-Advanced

Q1: What is a load balancer, and why is it used in distributed systems?

Ans: A load balancer acts as a reverse proxy that sits in front of your backend servers and routes incoming client requests across them. In distributed systems, it is used to prevent any single server from becoming a bottleneck, maximize resource utilization, optimize response latency, and ensure high availability by abstracting the underlying server infrastructure from the client.

Q2: Explain the difference between Layer 4 and Layer 7 load balancing.

Ans: The differences are as follow: 1. Layer 4 Load Balancing – operates at the transport layer (TCP/UDP). It routes traffic based on network details like IP address and port number without inspecting the application data package, making it incredibly fast and efficient. 2. Layer 7 Load Balancing – operates at the application layer (HTTP/HTTPS). It inspects the actual payload content, enabling smart routing based on HTTP headers, cookies, URL paths, or query parameters, though it requires more CPU overhead for SSL/TLS termination and deep packet inspection.

Q3: How does a load balancer improve fault tolerance and scalability?

Ans: It improves these characteristics via two main mechanisms: 1. Fault Tolerance: By continuously running health checks against backend instances, the load balancer automatically detects node failures and dynamically diverts traffic away from unhealthy servers, preventing application downtime. 2. Scalability: It allows you to horizontally scale your application by seamlessly adding or removing compute nodes to the backend pool as traffic fluctuates, without requiring any configuration changes or downtime on the client side.

Q4: What are the common load balancing algorithms, and when would you use each?

Ans: The common strategies include: 1. Round Robin: Requests are distributed sequentially. Best used when all backend servers have identical hardware specs and the requests require similar processing power. 2. Least Connections: Directs traffic to the server with the fewest active sessions. Ideal for long-lived connections (like databases or WebSockets) where request duration varies widely. 3. IP Hash: Combines the client's IP address to determine which server receives the request. Used when you need to natively map a specific user to the same backend server without tracking session state on the proxy.

Q5: What is session persistence (sticky sessions), and how is it implemented?

Ans: Session persistence is a mechanism that ensures all subsequent requests from a specific client during a session are continuously routed to the exact same backend server. It is typically implemented at Layer 7 using HTTP Cookies (either generated by the load balancer itself or reading an application-defined cookie) or at Layer 4 by mapping the client's source IP address.

Q6: How does AWS Application Load Balancer (ALB) differ from Network Load Balancer (NLB)?

Ans: The key differences are: 1. ALB: Operates at Layer 7, handling HTTP/HTTPS protocols. It offers advanced features like path-based routing, host-based routing, and native AWS WAF integration. 2. NLB: Operates at Layer 4, handling TCP, UDP, and TLS traffic. It is designed for ultra-high throughput, can handle millions of requests per second with ultra-low latency, and provides static or Elastic IP support per Availability Zone.

Q7: What is the role of Target Groups in AWS ALB?

Ans: Target Groups act as a logical grouping mechanism used to route requests to one or more registered destinations (such as EC2 instances, ECS tasks, Lambda functions, or specific IP addresses). The ALB evaluates its listener rules, identifies the matching rule, and forwards the incoming request to the specific Target Group associated with that rule, which also handles health checking for those targets.

Q8: How does AWS ALB handle SSL/TLS termination?

Ans: AWS ALB handles SSL/TLS termination by offloading the cryptographic decryption process from your backend servers. You bind an SSL/TLS certificate (typically provisioned through AWS Certificate Manager) directly to an HTTPS listener on the ALB. The ALB decrypts the incoming secure traffic and forwards it to your target groups over unencrypted HTTP (or encrypted HTTPS if required), freeing up backend CPU cycles.

Q9: Can you configure the load balancing algorithm in AWS ELB/ALB? Why or why not?

Ans: Yes, you can configure the routing algorithm at the Target Group level for an AWS ALB. By default, ALB uses the Round Robin algorithm, but it can be changed to Least Outstanding Requests (LOR). This allows you to optimize routing for workloads where requests vary significantly in processing time, though configurations are limited to these AWS-supported options.

Q10: What is cross-zone load balancing in AWS, and what are its trade-offs?

Ans: Cross-zone load balancing allows each load balancer node to distribute traffic evenly across all registered targets in all enabled Availability Zones (AZs) rather than just its own zone. Trade-offs: It ensures completely uniform traffic distribution even if your AZs have an unequal number of instances. However, it can introduce slight inter-AZ network latency and potentially increase data transfer costs across zones.

Q11: How does Kubernetes Service of type LoadBalancer work with cloud providers?

Ans: When you provision a Kubernetes Service with spec.type: LoadBalancer, the cloud-controller-manager built into the cloud provider (like AWS, GCP, or Azure) intercepts the event. It automatically provisions a native managed infrastructure load balancer (e.g., an AWS Classic or Network Load Balancer) and configures it to forward traffic to the NodePorts assigned across your cluster nodes.

Q12: What’s the difference between Kubernetes Ingress and Service LoadBalancer?

Ans: The operational differences are: 1. Service LoadBalancer: Exposes a single specific service to the external web by provisioning a distinct cloud infrastructure load balancer, which can become expensive and hard to manage at scale. 2. Ingress: Acts as an API Gateway and single entry point that manages external access to multiple cluster services. It operates at Layer 7, providing path-based/host-based routing, SSL termination, and name-based virtual hosting under a single load balancer IP/DNS.

Q13: How do you implement pod-level load balancing in Kubernetes?

Ans: Pod-level load balancing bypasses traditional kube-proxy iptables or NodePort hopping by routing traffic directly from the external load balancer to individual pod IPs. This is implemented using container network interfaces (CNIs) supporting native routing or by using an Ingress controller (like NGINX Ingress or AWS Load Balancer Controller using the IP target type) that resolves endpoints directly to pod IPs via the endpoints API.

Q14: What role does kube-proxy play in load balancing?

Ans: Kube-proxy runs on every node in a Kubernetes cluster and maintains the network rules that map a Service's virtual IP (ClusterIP) to its underlying Pod endpoints. It continually watches the API server and manipulates host network tools like iptables or IPVS to intercept traffic aimed at a service IP and distribute it randomly or via round-robin across the backend pods.

Q15: How would you troubleshoot uneven traffic distribution across pods?

Ans: The troubleshooting checklist involves: 1. Check HTTP Keep-Alive / Long-Lived Connections: Persistent connections (like gRPC or WebSockets) lock onto a single pod, causing imbalances if using Layer 4 routing. Fix by using an L7 Ingress controller with connection-level rebalancing. 2. Review kube-proxy Mode: Verify if kube-proxy is running in iptables mode (which uses random statistical probability routing) and consider switching to IPVS mode for true least-connection/round-robin distribution. 3. Inspect Readiness Probes: Ensure that pods aren't toggling in and out of a 'NotReady' status, which temporarily drops them from the endpoints list.

Q16: How do health checks influence load balancer routing decisions?

Ans: Health checks serve as the gatekeeper for the backend pool. The load balancer continuously probes a designated endpoint at set intervals. If a backend node fails a specific number of consecutive health checks (falling below the unhealthy threshold), the load balancer flags it as inactive and immediately stops routing traffic to it, restoring traffic only after it passes consecutive checks above the healthy threshold.

Q17: What’s the difference between DNS-based load balancing and reverse proxy load balancing?

Ans: The structural differences are: 1. DNS-Based Load Balancing: Resolves a domain name to multiple IP addresses (e.g., via Round-Robin DNS or Geolocation routing). The client connects directly to the resolved IP. It is fast but suffers from slow failover times due to client-side DNS caching. 2. Reverse Proxy Load Balancing: The client connects to a dedicated proxy device (the load balancer), which evaluates the request and proxies it to the backend. It offers instant failover, deep request inspection, and routing flexibility, but introduces a single point of entry and network hop.

Q18: How do you secure a load balancer against DDoS attacks?

Ans: You can mitigate DDoS vulnerabilities by employing these security layers: 1. Integration with Cloud Armor/Shield: Attach managed anti-DDoS scrubbing services like AWS Shield Advanced or Cloudflare to absorb volumetric Layer 3/4 attacks. 2. Web Application Firewall (WAF): Bind a WAF to filter out malicious Layer 7 request patterns, enforce rate-limiting constraints per client IP, and drop requests that violate protocols. 3. Infrastructure Hardening: Configure security groups to accept connections only on required ports (80/443) and close public access to the backend infrastructure entirely.

Q19: What metrics would you monitor to evaluate load balancer performance?

Ans: You should monitor the following key performance indicators (KPIs): 1. Request Count & Throughput: Total volume of active connections and data processed per second. 2. Latency / Target Response Time: The duration it takes for backend servers to respond to the load balancer. 3. HTTP Error Rates: Spikes in 5xx errors (indicating backend failures) or 4xx errors (indicating client or configuration issues). 4. Unhealthy Host Count: The number of backend instances currently failing their health checks.

Q20: Describe a real-world scenario where misconfigured load balancing caused downtime. How did you fix it?

Ans: A common production disaster occurs when a high-traffic application utilizes an infrastructure load balancer with unadjusted health check timeout values paired with heavy backend processing spikes. Scenario: Under a sudden load burst, backend API instances took 6 seconds to respond to a deep health check endpoint, while the load balancer's timeout was set to 5 seconds. The load balancer falsely marked every single backend node as 'Unhealthy' and cut off all traffic, resulting in total downtime and 502/503 errors. The Fix: 1. Immediately decoupled the health check target from intensive logic to a lightweight, dedicated static endpoint (e.g., /healthz). 2. Adjusted the health check configurations by increasing the timeout buffer and lowering the consecutive failure threshold to prevent aggressive cascading evictions.

Q21: What are the main types of AWS Load Balancers and when would you use each?

Ans: AWS offers Application Load Balancer (ALB), Network Load Balancer (NLB), and Gateway Load Balancer (GLB). ALB operates at Layer 7, ideal for HTTP/HTTPS traffic with features like path-based routing and WAF integration. NLB operates at Layer 4, designed for ultra-low latency and millions of requests per second, suitable for TCP/UDP workloads. GLB is used for deploying and scaling third-party virtual appliances such as firewalls or intrusion detection systems.

Q22: When should you choose an Application Load Balancer (ALB) over a Network Load Balancer (NLB)?

Ans: Choose ALB when you need advanced Layer 7 features like host/path-based routing, SSL termination, or WebSocket support. Use NLB when you require static IPs, extreme performance, or need to handle non-HTTP protocols such as TCP/UDP.

Q23: What production issues commonly occur with ALB and what are their root causes?

Ans: Common issues include misconfigured listener rules causing incorrect routing, SSL certificate expiration leading to failed HTTPS connections, and WAF rules blocking legitimate traffic. Root causes often stem from misconfiguration, expired certificates, or overly restrictive security policies.

Q24: What production issues commonly occur with NLB and what are their root causes?

Ans: Issues include backend targets failing health checks due to incorrect port configuration, sudden traffic spikes overwhelming targets, or misconfigured security groups blocking traffic. Root causes are usually misaligned health check settings, insufficient backend scaling, or restrictive network ACLs.

Q25: What is a Gateway Load Balancer (GLB) and when is it used?

Ans: GLB is used to deploy, scale, and manage third-party virtual appliances like firewalls or intrusion detection systems. It operates transparently at Layer 3, routing traffic through these appliances. It is chosen when deep packet inspection or advanced security filtering is required in production.

Q26: How do health checks influence load balancer routing decisions?

Ans: Load balancers only route traffic to healthy targets. Misconfigured health checks (wrong path, port, or timeout) can mark healthy targets as unhealthy, causing traffic imbalance or outages. Correct configuration and monitoring are critical in production.

Q27: What are common causes of 4XX errors in AWS Load Balancers?

Ans: 4XX errors (like 400, 403, 404) usually indicate client-side issues. Causes include malformed requests, unauthorized access due to missing/invalid credentials, or incorrect routing rules leading to 'Not Found'. Root causes are often misconfigured authentication, missing headers, or incorrect listener rules.

Q28: What are common causes of 5XX errors in AWS Load Balancers?

Ans: 5XX errors (like 502, 503, 504) indicate server-side issues. Causes include backend targets failing health checks, timeouts due to slow application responses, or misconfigured integration responses. Root causes are often application crashes, insufficient scaling, or misaligned timeout settings.

Q29: How would you troubleshoot a 502 Bad Gateway error from an ALB?

Ans: Check CloudWatch logs for backend errors, validate that the target group is healthy, and confirm that the application is returning valid responses. Root causes are often invalid response formats, backend crashes, or mismatched SSL/TLS configurations.

Q30: How would you troubleshoot a 504 Gateway Timeout error from an ALB?

Ans: Investigate backend latency by checking application logs and CloudWatch metrics. Ensure timeout settings in ALB are aligned with backend response times. Root causes are usually slow database queries, unoptimized application code, or insufficient compute resources.