Creating Multi-Edge Data Architectures on AWS Wavelength and MongoDB

With the advent of high-speed 5G networks, enterprises have sought to deliver low-latency experiences in the Industrial Internet of Things IoT (IIoT), media and entertainment, automotive and more.
Data is at the heart of these low-latency experiences - whether it's a ground robot, a consumer smartphone, or a driverless car. The seamless exchange of data from the 5G manufacturer to the consumer, such as machine learning (ML) and even generative artificial intelligence (AI) technology, is critical to powering these new experiences.

Ensuring that the required data is available to consumers with the lowest possible latency in any location (for example, via multiple AWS wavelength zones), while providing a single view of the entire data set, is no easy task.

From 2020. MongoDB and Amazon Web Services (AWS) have been working to define benchmarks for low-latency distributed database patterns in AWS hybrid edge technologies. Specifically, this article builds on the earlier demonstration of MongoDB Realm on AWS Wavelength by enhancing the existing architecture with three significant new features:

• Replication from edge to cloud: Replication from edge to cloud in a multi-edge environment is enabled via Atlas Device Sync.
• Dynamic edge discovery: The Atlas Functions implementation directs mobile clients to the optimal MongoDB Realm instance running at the edge.
• Containerised application: a native implementation of Amazon Elastic Kubernetes Service (Amazon EKS) hosts the MongoDB Realm database directly on the carrier network.

In this article, the authors will also demonstrate how to use the AWS Load Balancer controller to deploy the Application Load Balancer (ALB) in front of the database.

MongoDB is a competency partner of AWS Data and Analytics and a vendor of AWS Marketplace. MongoDB, a data platform company for developers, enables innovators to unleash the power of software and data.

AWS wavelength use cases for MongoDB

From innovative manufacturing and predictive maintenance to agriculture, there is no shortage of low-latency applications that benefit from mobile edge processing.

Extending existing workloads in MongoDB to the edge has several key benefits:

• Security: Currently, responses from database queries must pass - encrypted or otherwise - over the public Internet to return to the mobile client. To protect highly sensitive workloads from the vast majority of malicious actors, mobile edge processing offers the possibility that all mobile traffic never leaves the radio access network (RAN) to the public Internet.
• Performance and reliability: minimising incremental hops to the application endpoint, especially at times of high congestion, can significantly reduce the application's latency budget. What's more, for high-volume, transaction-based applications, a mere 10 ms savings combined with more than one million transactions can result in more than two hours of latency savings.

AWS Wavelength Reference Architecture

To design this solution, we will need to extend the AWS region to the selected AWS wavelength zone, where we will deploy a containerized Realm application to communicate back to the MongoDB Atlas in the parent region.

Amazon VPC

For each region where a MongoDB Atlas cluster or AWS Wavelength zone application has been deployed, you must have a virtual private cloud (VPC).

In scenarios where you have deployed many Amazon VPCs or are using multi-operator wavelength zones (Verizon, Vodafone, Bell Canada), configure VPC peering and ensure that bi-directional peering is configured.

A full list of regions with available wavelength zones can be found on the AWS Wavelength Locations page.

MongoDB Atlas

Using MongoDB Atlas, a fully managed database, you can create an instance of a MongoDB Atlas cluster in any region of your choice. Supported Atlas regions can be found in the MongoDB Atlas documentation.

MongoDB Realm

In the previous article, the authors implemented self-managed MongoDB instances running on Amazon Elastic Compute Cloud (Amazon EC2) instances and managed by MongoDB Cloud Manager. However, synchronizing data between these zones becomes more complex in a multi-wavelength zone environment. This complexity can be resolved by introducing MongoDB Realm and Atlas Device Sync, which automate data synchronization between Realm-based endpoints and Atlas.

MongoDB Atlas Functions

To determine the optimal edge endpoint, you can use edge discovery APIs, such as Verizon's 5G Edge Discovery Service, in a thin-client model to move north-south service discovery to the application rather than the device.

Amazon EKS

The ability to orchestrate applications from a single, centralized control plane reduces the overhead associated with configuring multiple independent clusters or auto-scaling groups. For an edge computing database, you will use Amazon EKS to create a highly scalable environment to run multi-cluster workloads.

Within an EKS cluster, you will deploy containers containing an application, such as an MQTT broker, and a MongoDB Realm database.

To learn more about Amazon EKS on AWS Wavelength, see the AWS blog article on deploying geographically distributed EKS clusters on AWS Wavelength.

Overview of the solution

In this section, the authors will demonstrate step-by-step how to deploy each relevant component, starting with the edge discovery service registry, the underlying infrastructure on AWS, and then the MongoDB cluster and container application.

Step 1: Create Edge Discovery Automation

To start, you need a way to ensure that your Edge Discovery service mesh is always up-to-date with the latest endpoints targeting operators. Therefore, prior to deploying any edge infrastructure, you need to set up an automated workflow that dynamically registers the AWS wavelength endpoint. This can be done via Amazon EventBridge and AWS Lambda.

Step 2: Create an Atlas cluster

Next, create a master region database in AWS. Using MongoDB Atlas, MongoDB will maintain high availability of the cluster in the selected region.

Step 3: Create an Amazon EKS cluster

To deploy an AWS Wavelength application, you must first create a resource instance of the parent region. For example, an Amazon EKS cluster must be deployed with two subnets in the parent region (for example, us-east-1a, us-east-1b).

Step 4: Create a Realm application

For your edge application, you can use the MongoDB Atlas App Services to create an IoT application that includes the application code for your static website and CSS files. For your sample application, you can follow the directions in the MongoDB blog article to take advantage of low-latency innovation with MongoDB Atlas, Realm and AWS Wavelength.

In parallel, for dynamic application logic, create a container image that writes to the Realm database instance in the container itself; for example, you can visit this MQTT proxy to Realm on GitHub. This design separates the latency-tolerant experience, such as the application user interface, from the latency-critical IoT messaging data plane.

Step 5: Configure the Realm application

Next, you need to configure the MongoDB Realm application using the application code. To do this, push out the remote configuration (from the pre-developed application).

You must then configure the Atlas function, a serverless event-driven function that calls the Edge Discovery service to route to the optimal MongoDB Realm endpoint.

In this function, you extract the calling IP address so that you can call the Edge Discovery Service (EDS) API on behalf of the mobile client. You then authenticate the Edge Discovery Service and send a query to the optimal serviceEndpoint records, which returns the optimal Carrier IP address corresponding to the nearest wavelength zone (mecresponse.data.serviceEndpoints[0].serviceEndpoint.IPv4Address).

Note, however, that the EDS, appKey, and secretKey should be treated as confidential and out of code, for example, using AWS Secrets Manager.

exports = async function({ query, headers, body}, response) {
    const ipAddr = headers["X-Forwarded-For"][0];
    console.log("Using IP: " + ipAddr);  
    try {
      var tokenresponse = await axios.request(
        { 
          url: "https://5gedge.verizon.com/api/ts/v1/oauth2/token",
          method: 'post',
          headers: {"Content-Type": "application/x-www-form-urlencoded"},
          data: qs.stringify({"grant_type" : "client_credentials"}),
          auth: {"username": appKey, "password": secretKey}
        }
      );
    } catch(ex) {
      console.log(ex);
    } 
    try {
      var mecresponse = await axios.request(
        { 
          url: "https://5gedge.verizon.com/api/mec/eds/serviceendpoints?serviceEndpointsIds="+serviceEndpointsId+"&UEIdentityType=IPAddress&UEIdentity="+ipAddr,
          method: 'get',
          headers: {"Content-Type": "application/x-www-form-urlencoded", "Authorization": "Bearer "+tokenresponse.data.access_token},
          data: qs.stringify({"grant_type" : "client_credentials"})
        }
      );
      
    return mecresponse.data.serviceEndpoints[0].serviceEndpoint.IPv4Address; 
    } catch(ex) {
      console.log(ex);
    }
};

Step 6: Run Self-Managed Nodes

To complete the deployment of your EKS cluster, deploy self-managed workflow nodes in selected AWS wavelength zones. To learn more about configuring self-managed nodes in Wavelength, refer to the AWS documentation that describes how to use exctl to deploy to Wavelength zones.

Step 7: Deploy Realm applications

Next, create a deployment manifest, realm.yaml, which contains the Deployment and ClusterIP service objects.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: realm-database
  labels:
    app: realm-iot
spec:
  replicas: 1
  selector:
    matchLabels:
      app: realm-iot
  template:
    metadata:
      labels:
        app: realm-iot
    spec:
      containers:
      - name: realm-app-demo
        image: <your-realm-app-image>
        ports:
        - containerPort: 80    
      nodeSelector:
        failure-domain.beta.kubernetes.io/zone: <your-wavelength-zone-ID>

---
apiVersion: v1
kind: Service
metadata:
  name: realm-service
spec:
  selector:
    app: realm-iot
  ports:
    - port: 80
      targetPort: 80

When creating the input object, you can then configure the AWS Load Balancer controller to provide the application load balancer module.

Applications

To learn more about MongoDB and AWS Wavelength, you can review the hands-on workshop with AWS re:Invent or browse use cases, reference architectures, and official documents on MongoDB in edge networks:

• Realising the Benefits of 5G with MongoDB and Verizon 5G Edge: This paper explains how the speed, capacity, reliability, and responsiveness of Verizon's network combined with the sophistication of MongoDB's next-generation developer data platform can transform various industries.
• Delivering Data Faster with Verizon 5G Edge and MongoDB: learn the basics of mobile edge computing, edge discovery API workflows, and the MongoDB Atlas architecture.
• Computing in the Real World: learn about mobile edge computing (MEC) industry applications in industrial IoT, smart manufacturing, automotive and more.

You can also learn more about MongoDB in the AWS Marketplace.