Design, implementation, and care of LINK Mobility infrastructure

LINK Mobility is the leader in mobile communication and CPaaS solutions in Europe regarding the number of SMS messages sent.  In June 2020, the Group had 33,000    customers in Europe and sent 9.5 billion messages in 2019. , the LINK Mobility group belongs to the company LINK  Mobility Poland  Sp. z o.o., the owner of the SMS API brand. SMS API is a platform for mass sending SMS, MMS, and VMS messages through the customer panel or a simple and easy to integrate SMS API interface enabling integration with your own IT systems m.in e-commerce or CRM. Due to the desire to implement Kubernetes technology, the need to flexibly scale the infrastructure, and the global range of services provided by the Client, the AWS cloud was the first and only choice for LINK Mobility. In addition, the Company planned to use the Terraform tool to duplicate, with slight modifications, the infrastructure implemented in Poland for branches around the world.

AWS account configuration

The Customer has prepared the AWS account. It was up to us to identify and use the appropriate roles and chains of trust for the relevant accounts of the organization. We have configured access roles of the parties involved, service accounts used by automation, implemented the preservation of information about access and use of AWS API(AWS CloudTrail) and elements used by IaC tools (AWS  S3, Amazon Dynamo DB ) to maintain and protect the state of the infrastructure.

Infrastructure design

We started our work by designing the infrastructure in the developer version, and the whole was defined and implemented using the  Terraform tool. We have completed the production infrastructure in three AZ zones, which, combined with the Multi-AZ option for RDS,  allowed us to obtain full HA in the event of a failure of any zone.

VPC Network

Due to the complete separation of the production environment, we have created independent VPC networks for application components and services. We have extended the virtual network to include a private area (private subnets), i.e. without direct access from the Internet, from which outbound traffic comes through NAT Gateways. We have equipped the production network (private and public) with three sub-grids, each in a  differentvData  Center (AZ).  In addition, in the private area, we have prepared a VPN site-to-site tunnel,  used for encrypted access to the private site of the cloud,  directly from the corporate office area (and only from this area).

Kubernetes platform

We have implemented the Kubernetes cluster using Amazon Elastic Kubernetes Service  (EKS), which, in addition to the advantages characteristic of the Kubernetes platform, provides cost optimization of the resources used (cluster autoscaling), guarantees support for fast and frequent deployment of new applications, supports many environments consisting of predefined micro-service devices, also controls work and self-repair of additional functionalities.

In addition, to provide the planned functionalities of the cluster, we have also installed:

• Ingress-controller for handling ingress with cost optimization in the form of a single NLB load balancer for all supported ingress
• Cluster autoscaler for automatic scaling of machine groups with the rescheduling  of pods
• External-DNS entry updater to automatically handle host addresses defined in ingresses.

We have implemented all of the above tools using and under the control of the Helm tool.

Data layer

Considering primarily the requirements related to compliance with the practices implemented by the Client in software functioning and maintenance, we have launched databases in the form of Kubernetes pods,  using EBS disks through PresidentVolumeClaim. It is a form consistent with the microservice architecture pattern. A given microservice has its database, and the team responsible for the microserage is also accountable for its maintenance and configuration. The advantage of this type of infrastructure is the separation of hardware resources of the relevant databases belonging to the corresponding micro-servers.

Due to the critical role of the MySQL database, we decided to implement it in both environments in the form of the Amazon RDS service, which allowed us to unify the procedures related to backup, restore and database support by applications.

RabbitMQ message broker

The implementation of the data rail, as in the database case, was carried out by the requirements related to compliance with current practices. Thus, we launched the RabbitMQq instance in the form of Kubernetes pods, using (if they are running in "disc" mode) EBS disks via

PersistentVolumeClaim. It is a form consistent with the microservices architecture. A given micro-server group has its data bus, and individual teams are also responsible for its maintenance and configuration. The advantage of such an architecture is the separation at the level of using hardware resources of the corresponding data buses belonging to the related groups of microservices. The costs of such a bus are components of the costs of the  Kubernetes cluster, and Velero carries out the protection of disk data.

Object storage and other managed services

As storage of objects (static files, copies, archival logs, infrastructure status files) we used the AWS S3 service. In addition, we also used: 

Amazon KMS – for data encryption in AWS services in integration with native authentication and authorization methods in AWS.

System Manager Parameter Store – to store dynamic parameters available to applications using native for AWS IAM and KMS, including encrypted secrets, initialization parameters, etc.

Amazon Route 53 – as a DNS service, allowing you to maintain and manage DNS zones, both public and private, for a given VPC network.

AWS Certificate Manager – a DNS service that allows you to create and manage SSL certificates that will be used to provide trusted SSL traffic for services issued on the load balancer.

Monitoring and alerting

Monitoring of EKS clusters together with the rest of AWS services was  carried out using the DataDog service, which has both native integration with Kubernetes and with the AWS platform, thanks to which one desktop (single pane of glass) is enough for monitoring, which significantly shortens the response time to failures and incident analysis. As part of the PoC project (for the development environment), we implemented the DataDog agent in the Kubernetes cluster to assess the usefulness, functionality, and costs of such a solution. The implementation of altering  (actions triggered by exceeding a certain level by the indicated metrics) was based on the developed and flexible functionality of the DataDog platform.

Application deployment

The application on the Kubernetes platform was installed using the Helm package manager, which is currently most often used to manage applications in the Kubernetes environment. Helm offers functionalities that allow it to be used in an automated way (pipelines), allows you to  place

new versions of the application, roll back to previous versions, or delete them in a standardized way, using the definition of applications or their entire set in the form of so-called Charts, i.e., the definition of all components necessary in the cluster to operate Application.

Summary

For LINK Mobility, we have implemented a flexible, scalable (vertically and horizontally) infrastructure compliant with the  DevOps methodology and the Infrastructure as Code concept. We have minimized the expenditure necessary to duplicate the environment in other regions. The entire environment is based on the  Kubernetes platform and IaC (Terraform) tools. We have also maintained an optimal level of costs while maintaining good practices in the field of m.in the security rules of cloud systems.