Building Kubernetes-based infrastructure and taking care of resources

For our client, a modern cloud native company that stores all its resources in the cloud, and one of the leaders in the renewable energy market, we have provided consulting, design, construction and care of infrastructure in Amazon Web Services.

Start of cooperation and infrastructure

The client, when entering into cooperation with us, already had an infrastructure based on Kubernetes clusters on AWS. Initially, his intention was to entrust us only to take care of the existing environment. However, after conducting a standard review of the Client's infrastructure, we proposed a number of changes aimed at optimizing the speed of operation, reducing costs or implementing the IaC approach. Given the Client's very professional approach to the implementation and use of new technologies, together we decided to build the entire infrastructure from scratch, following the best practices recommended by AWS.

Hosters' insights into the possibilities of optimizing our infrastructure proved to be extremely accurate, which made us convinced of the need to implement them from the very beginning. Thanks to this decision, we now have an extremely modern environment, based on Kubernetes technology, developed in the IaC approach. Deciding to implement the solutions proposed by Hosters, we gained an environment that is secure, trouble-free and perfectly suited to our needs!

Methodology

The entire infrastructure was designed and implemented in the IAC methodology using Terraform and Helm tools. To manage the infrastructure code, we prepared a separate container.

Terraform is used to manage the entire infrastructure within AWS. It contains a description of all elements of this environment (AWS IAM, AWS VPC, Amazon EKS, Amazon ECR, Amazon RDS, AWS EC2). The build code, is common to the production, test and development environments, while the variables defined for both environments, were defined to separate files.

Helm - we used to implement the main tools within the Kubernetes cluster (Traefik, dashboard, Filebeat, Prometheus, RBAC). For each environment (test and production) we prepared separate files defining the service. We also prepared scripts to deploy the services within Kubernetes cluster.

We created three accounts (Prod, Dev and Test) for the entire project. Users and groups within AWS IAM are established manually (excluding infrastructure code), while roles and policies are created in infrastructure code.

All microservices were run within separate (for each environment) Kubernetes clusters. To manage traffic within the clusters, we used Traefik. In accordance with best practices, we configured separate services for each cluster, responsible for internal and external traffic. Within each cluster, we also implemented the Prometheus stack, through which metrics are collected from all microservices. To display the collected data, we used Grafana. Importantly, access to the above services, for security reasons, is not possible from the Internet.

Infrastructure elements

The entry point to the client's infrastructure, responsible for managing DNS zones within domains, is CloudFlare, which is responsible not only for external traffic, but also internal traffic. As part of the ElasticCloud service, we have deployed ElasticSearch and Kibana, which are jointly responsible for collecting application logs

Other tools

As part of the project, we also used:

AWS Lambda - for image scaling and report generation
AWS S3 - for storing the state of the infrastructure managed by Terraform and the files used by the application
Amazon API Gateway - one of the services that, through communication with Lambda, is responsible for generating reports
Amazon CloudWatch - a tool used for monitoring services running on the infrastructure - among other metrics for the RDS service

Summary

We designed and deployed an infrastructure in AWS environment, based on Kubernetes clusters, in an IAC-compliant approach. By using a number of AWS tools, we were able to optimize the costs associated with maintaining the infrastructure, and the result of our work is a stable and secure infrastructure that has not experienced any failures since its inception.

Read also:

• Multi-stage migration of Capsim infrastructure to AWS
• Implementation of cloud infrastructure for Magic Commerce, based on microservices, containers and IaC approach
• Implementation and maintenance of cloud infrastructure in AWS for Displate
• Migrating and building infrastructure in AWS for SimpleMining.net
• Design, build and launch infrastructure on AWS for Roler