Design and implementation of a k8s environment based on Amazon EKS
The customer whose case we are describing is a Norwegian company offering secure gateways and unreliable and secure IoT devices based on modern open standards that support security, smart home, and healthcare solutions. With the client's products, users can use and develop existing IoT capabilities as the client's tools evolve. If required, the company also supports its customers with software modifications and offers radio modules to adapt the customer's products to the IoT world.
Beginning of cooperation
At the start of the collaboration, the Customer's infrastructure was built in the AWS cloud on Kubernetes clusters in kOps. Up to that point, DevOps, with whom the cooperation had ended, had been responsible for developing and maintaining the infrastructure on the client's side. The situation was problematic in that no one could seamlessly take over the infrastructure and take care of deployment coordination, fine-tuning CiCD, or bringing started deployments within the infrastructure to completion.
The order was, therefore, to deal with a pervasive infrastructure that we had to discover ourselves piece by piece. Our first task was to gain in-depth knowledge and understanding of the existing environment, to determine what was necessary for the application to function correctly, what the dependencies were between the various elements of the infrastructure, and to break down a deployment made up of dozens of templates, code, and configurations.
Migration and optimization
The entire learning process was just a prelude that prepared us to migrate the client's infrastructure to a new environment based on Amazon EKS. By fully understanding and organizing the old infrastructure, it was easier for us to deploy the various environments on the new cluster. The process required us to rewrite all helms from scratch. We also prepared a generic pipeline to make deployment work with 100% efficiency in both the development and production environments.
Once we knew the environment down to the smallest detail, we started planning the migration of resources from the old k8s environment to the new one based on Amazon EKS. The entire operation took place ‘next to’ the running production, whose work could not be disrupted in any way. A huge challenge was to keep the extensive network of dependencies between the tools on the old cluster and the services already running on the new one intact. At the same time, we consulted with the client on new solutions necessary for developing the client's applications.
One of the most important optimisations that we implemented at the very beginning of our cooperation was improving the notification system for customers using the Customer's application, which was to alert them in emergency situations such as flooding or fire.
A group of machines with external IP was whitelisted in-house, which physically handled the sending of notifications and the eventual notification of the relevant emergency services. When instances with a ‘hooked up’ IP terminated, their IP had to be manually ‘hooked up’ again. If this had not been done, the alarm might not have worked. We corrected this by creating groups of nodes that, via the nategateway, leave the production cluster with a fixed IP number. The machines that connect via the Gateway are located underneath it. In this way, we have removed a considerable limitation in the application's functionality and guaranteed 24/7 notification operation.
Infrastructure outline
As mentioned, the client's environment consists of development and production clusters. There is AWS Route53 and ACM on the infrastructure front, followed by AWS load balancers, which are managed from within the cluster. Each cluster has its own VPC, while data management uses RDS, PostgreSQL, and MySQL.
The tools that make up the application layers:
- Amazon VPC - the layer where network organization is defined
- AWS Route53 - supports domains used by applications
- Amazon EKS - to support Kubernetes clusters
- AWS Certificate Manager - to support SSL certificates
- Amazon SES
- Amazon EFS
- Amazon RDS
- AWS Cognito
- AWS S3
SUMMARY
We started working with the client at a tough time for them, without the opportunity to ‘learn’ the infrastructure from people who had dealt with it before. We learned all the applied solutions ourselves, sorted out the infrastructure, and improved the most essential elements. We migrated the infrastructure prepared this way to the new k8s environment based on Amazon EKS, while implementing best practices and solutions to guarantee stable operation of the client's applications. We rewrote the extensive CICD pipeline, enabling regular and secure deployment. We are currently developing the environment with, among other things, the new Terraforma, providing consultancy services and 24/7 administrative support for the client's resources.
Read also:
44-200 Rybnik
Poland
REGON: 240692928
National Court Register (KRS): 275333
