Use the Amazon CloudWatch Contributor Insights service for general NGINX log analysis

Customers build, deploy, and maintain millions of web applications on AWS, and many deploy these applications using NGINX. The NGINX application server offers configurability, scalability, and the ability to handle millions of concurrent requests. Web application performance is critical in modern infrastructure and enterprise applications. Customers use CloudWatch to monitor response times and availability and ensure SLAs. Engineering teams also track several data points and metrics, such as server performance when traffic changes and engineering teams know that poor performance will degrade the customer experience. This leads to lower revenue performance, impacting compliance or risking operational downtime.

It is worth knowing that customers can use Contributor Insights rules, a feature of Amazon CloudWatch, to analyze web application logs. Customers can assess patterns in structured log events when streamed into CloudWatch logs, including any custom logs sent by applications in the cloud or from local servers, such as NGINX web server application logs. CloudWatch Contributor Insights enables product and engineering teams to view, investigate, and resolve issues that occur in web applications. Once configured, Contributor Insights runs continuously without manual intervention, helping operators isolate, diagnose, understand, and resolve problems that arise during an operational event.

This article will show you how to monitor and perform NGINX log analysis using CloudWatch Contributor Insights. NGINX logs provide insight into web applications, such as errors, response times, and performance. In addition to CloudWatch Contributor Insights, customers can activate CloudWatch Metrics and CloudWatch Alarms for comprehensive system monitoring.

Prerequisites

For this tutorial, the following prerequisites must be met:

1. Enable the NGINX server to run on the AWS account. In this example, the NodeJS web application and API will be installed, configured, and deployed using the included CloudFormation template.
2. The NGINX web application server runs in an AWS account with JSON web logs enabled (see NGINX configuration below).

Overview of the solution

The figure above shows how the NGINX Web Application Server application logs are sent to CloudWatch via the pre-installed CloudWatch agent in the instances. All Amazon Linux 2 AMIs include the CloudWatch agent. Contributor Insights rules analyze these logs, displaying the report on the CloudWatch dashboard.

Overview

In the AWS management console, navigate to CloudWatch to create a log group.

Step 1: Select Create log group.

In CloudWatch, navigate to Contributor Insights. On the Contributor Insights home page, select Create a Rule.

1. Select the NGINX log group by name from the drop-down list.
2. Select Custom rule for Rule type.
3. In the Contribution - section, enter the unique keys "remote_addr" and "status" to extract the remote request address and status of the request. These will be visualised on the CloudWatch dashboard.
4. Enter a name for the rule.
5. Select Create.
   
   

After creating a rule can take up to 5 minutes for the reported data to appear. At this point, the CloudFormation stack will be deployed, creating an EC2 instance running NGINX and a Node Express web application with two API endpoints to simulate the web application and sample API.

Step 2: Deploy the following CFT template:

https://nginx-app-server-cw-00.s3.us-west-2.amazonaws.com/Nginx-Web-App-Server-Express-App-Monitor-CloudWatch.yaml

This will take 5-10 minutes to complete. You can view the status of your deployment.

Once the stack is deployed, check the output data sheet for the NGINX URL:

Step 3: Deployment takes 5-10 minutes. When complete, click the site URL to display the following monitoring application running with two API endpoints failing and OK. The application runs and logs NGINX network logs in JSON format. These logs are streamed to CloudWatch in the access_json_log group for analysis.

When prompted with the following page, click continue to HTTP Site:

Step 4: View the status of the NGINX web application server, then click /admin/health/x1 and /return-status/200 to open in new tabs (open multiple times to simulate API access):

Step 5: Now that the sample web application and API are up and running, create a CloudWatch dashboard to display the 5XX, 4XX, 3XX response codes from the application. You will then create CloudWatch metrics for the 5XX 4XX 3XX response codes to use for CloudWatch alerts.

Search for log messages with response code 3XX.

Search for log messages with response code 4XX.

Search for log messages with response code 5XX.

Step 6: Now, create a metrics filter for visualization on the CloudWatch dashboard and the CloudWatch alert.

Similarly, create a new metric filter for status codes 2XX, 4XX, 5XX.

Step 7: Once you have created these metrics, the aggregated data will help with Contributor Insights reporting. Now add the Contributor Insight rule to the CloudWatch dashboard. Select Add to dashboard to add to an existing dashboard or create a new dashboard. Here you will add CI rules and a previously created Alert to show a single pane of glass for 5XX.

Summary

Through this article, the authors have explained how to stream NGINX logs from an EC2 instance to CloudWatch, analyze logs using filters, create charts to visualize NGINX status codes, and create alerts for status codes, all in a single window.

Monitoring and observability of web/API applications are critical to businesses and their operations. Product and engineering teams can view, investigate and troubleshoot issues occurring in web applications when web server logs are streamed to CloudWatch, using the extensive capabilities of CloudWatch Contributor Insights, CloudWatch Logs, CloudWatch Metrics and CloudWatch Alarms.