Key metrics in AWS ELB monitoring

Amazon Web Services Elastic Load Balancer (ELB) is a scalable and highly available load balancing solution with high throughput. It operates across OSI layers 3, 4, and 7 with its three variations: Application Load Balancer (ALB), Network Load Balancer (NLB), and Gateway Load Balancer (GWLB).

This article will take a detailed look at how the three variations operate, their configurations, and the key metrics you should monitor when using them.

Three flavors of AWS ELB

Let’s first review the different types of Elastic Load Balancer.

ALB operates at the 7th OSI layer, the application layer. It's by far the most popular ELB use case and commonly implemented to distribute HTTP traffic across multiple servers. Its predecessor is now known as Classic Load Balancer and may be used by legacy applications.

NLB operates at OSI layer 4, the transport layer. NLB distributes TCP, UDP, and TLS traffic across servers.

GWLB operates at level 3, the IP layer. It is primarily used with virtual appliances using the GENEVE protocol.

How does ELB function?

All three variations of ELB function in a similar manner and have three major components: listeners, target group(s), and the load balancer. We will discuss these further below.

Fig. 1: Components of ELB (Source: AWS documentation)

Listeners

The first configuration you need to specify is the listener. For ALB, the protocols supported are HTTP and HTTPS with ports in the range 1 - 65535. NLB supports the TCP, UDP, TLS, and TPC_UDP protocols. The TCP_UDP protocol is for accepting both TCP and UDP traffic on the same port. You should note that for HTTPS and TLS, you need to set up an SSL certificate on the load balancer to facilitate SSL termination at the load balancer level. GWLB listens to all IP traffic on all ports; it does not permit any selective port configuration.

Once the listener protocol and port have been configured, you must configure the listener rule(s). These allow you to specify where and how to route incoming requests. Requests are usually forwarded to one or more target groups.

ALB supports a rich set of rule actions and conditions besides the default forwarding; these include fixed HTTP responses and even authentication via OIDC providers or Amazon Cognito.

Target Groups

Target groups allow you to combine one or more targets together. Targets will be an IP address, an EC2 instance, or when working with ALB, a Lambda function. For NLB and GWLB, you can have an EC2 instance, an IP address, or ALB as the target type.

An important component of configuring a target is setting up the health check, which is how the load balancer periodically checks a target's state of health. Some important settings when configuring health checks are:

• Health check protocol: ALB, NLB, and GWLB all support the HTTP and HTTPS protocol for health checks. GWLB additionally supports the TCP protocol, which is its default configuration.
• Healthy threshold count: The number of successive positive responses needed before an unhealthy target is considered healthy.
• Unhealthy threshold count: The number of successive failed responses needed before a healthy target is considered unhealthy.
• Health check timeout seconds: The time in seconds after which no response is considered a failed response.
• Health check interval seconds: The time between health check requests for a target.

Load Balancer Configuration

Along with the listener and target group configuration, you must configure a load balancer with subnets and security groups.

A minimum of two subnets in different availability zones is recommended and at least eight free IP addresses to allow the load balancer to scale and achieve high availability. The security groups ensure that only the intended traffic is permitted to pass through the load balancer.

Monitoring for AWS ELB

AWS provides the following ways to monitor and troubleshoot your load balancer and registered targets:

• CloudWatch metrics
• Access logs
• Request tracing
• CloudTrail logs

In this post, we'll focus on key CloudWatch and access log metrics. A comprehensive list of metrics is available in the official documentation.

CloudWatch metrics

Using the AWS CloudWatch service, you can monitor your load balancer in real time. AWS ELB frequently pushes statistics to CloudWatch that are recorded as time-series data. These data points are referred to as metrics and can be aggregated and viewed via a dashboard. CloudWatch additionally allows you to configure alarms, which send notifications when a metric level has been breached.

Access log metrics

Web servers like Apache or Nginx maintain an access log for every request that reaches the server. AWS ELB also provides an access log, but it is turned off by default. Once it's enabled, requests are recorded to the log and stored in AWS S3. It's worth knowing that storing these access logs is an additional expense billed per the S3 service pricing. These access logs record useful metrics, which can be used to monitor and troubleshoot your load balancer.

Now, let’s look at the key metrics in these two categories. Although there are common metrics across load balancer variations, ALB in particular has a specific set of metrics one should look out for.

Application Load Balancer key metrics

Listed below are some of the metrics to look out for when working with Application Load Balancer or ALB.

CloudWatch Metrics

• ActiveConnectionCount: Displays the total number of active TCP connections from clients to the load balancer and from the load balancer to the targets; best aggregated via the sum, with a sustained high number indicating high traffic or long-running requests to your servers; health check metrics require monitoring to ensure targets can keep up
• HTTP_ELB_5XX_Count: Number of HTTP 5XX errors returned from the load balancer, not from targets
• RejectedConnectionCount: Count of dropped connections after the load balancer hits its capacity.
• Healthy/Unhealthy Host Count: Number of healthy or unhealthy targets, i.e., successful or failed health checks on the targets
• LambdaInternalError: Indicates either an internal load balancer error or an error from a Lambda function when using these as a target; the error_reason field in the access log shows the exact error
• ConsumedLCUs: Reports the cost in terms of units consumed; helps you avoid any surprises in your monthly AWS bill if monitored

Access Log Metrics

• request_processing_time: Total time in seconds from when the load balancer receives the request until it forwards it to the target; a “-1” means a possible failure to forward the request or a timeout from the target
• error_reason: Error reason text when a LambdaInternalError occurs

Network Load Balancer key metrics

Listed below are some of the metrics to look out for while working with NLB.

CloudWatch Metrics

• PortAllocationErrorCount: Port allocation errors raised on breaching the capacity of 55,000 unique connections per target
• Healthy/UnHealthyHostCount: Same as for ALB
• ConsumedLCUs: Same as for ALB

Gateway Load Balancer key metrics

Listed below are some of the metrics to look out for while working with Gateway Load Balancer or GLB.

CloudWatch Metrics

• Healthy/UnHealthyHostCount: Same as for ALB
• ConsumedLCUs: Same as for ALB

Summary

To summarize, we looked at the various load balancing options provided by AWS, i.e., ALB, NLB, and GWLB. Listener rules specify which port and protocol to listen on for incoming requests and the destination target group(s) to subsequently forward the requests. Target groups are configured with the destination of a request and health checks, which determine unhealthy target group instances that need to be replaced.

Using the key metrics discussed above, you will be able to monitor the health of your load balancer and attached targets, pinpoint issues that may arise, and keep an eye on operating costs.