AWS EKS with Karpenter

Karpenter is only available for non-production clusters. If you have created a production cluster, this option will not be visible.

Karpenter automatically launches just the right compute resources to handle your cluster's applications. It is designed to let you take full advantage of the cloud with fast and simple compute provisioning for Kubernetes clusters. You can our blog post for more information."

Creating a AWS EKS Cluster with Karpenter

1. Click on AWS as hosting mode and then Qovery Managed option:

   

   In the Create Cluster window enter:

   • Cluster name: enter the name of your choice for your cluster.
   • Description: enter a description to identify better your cluster.
   • Production cluster: select this option if your cluster will be used for production. Note: Karpenter is currently only available for non-production clusters.
   • Region: select the geographical area in which you want your cluster to be hosted.
   • Credentials: select one of the existing cloud provider credentials or add a new one by clicking on New Credentials. In the New credentials window, add the credentials that you have generated on your cloud provider console (Procedure for AWS account). Added credentials can be used later to create and manage additional cluster.

   To confirm, click Next.

2. In the Set Resources window, select:

   • Karpenter: Toggle the switch to enable Karpenter on your AWS EKS cluster
   • Instance types scopes: By editing it, you can apply different filters to the node architectures, categories, families, and sizes. On the right, you can view all the instance types that match the applied filters. This means Karpenter will be able to spawn nodes on any of the listed instance types.
     • Architectures: by default both AMD64 and ARM64 architectures are selected.
     • Default build architecture: by default AMD64. If you build your application with the Qovery CI, your application will be built using this architecture by default.
     • Families: by default all families are selected.
     • Sizes: by default all sizes are selected.
   • Spot instances: In order to reduce even more your costs, you can also enable the spot instances on your clusters. Spot instances cost up to 90% less compared to On-Demand prices. But keep in mind that spot instances can be terminated by the cloud provider at any time. Check this documentation for more information. Even if this flag is enabled, the statefulsets won't run on spot instances.
   • Disk size: select the size of the disks to be attached to your cluster instances (to locally store container images etc..).
   
   
   Instance type selection from your Qovery Console has direct consequences on your cloud provider’s bill. While Qovery allows you to switch to a different instance type whenever you want, it is your sole responsibility to keep an eye on your infrastructure costs, especially when you want to upsize.

   Please be aware that changing the instance type or disk size might cause a downtime for your service.

   For more information on the instance types provided by each cloud provider and their associated pricing, see What are the different instance types available when creating a cluster?

   Also, before downsizing, you need to ensure that your applications will still have enough resources to run correctly.

   
   

   To confirm, click Next.

3. In the Features step, select the features you want to enable on your cluster.

   If you want to manage the network layer of your cluster by yourself, you can switch VPC mode to Deploy on my existing VPC to use your own VPC instead of the one provided by Qovery.

   These options can only be configured during cluster creation and cannot be modified later.

   • VPC managed by Qovery
   • Use your existing VPC

   Static IP

   By default, when your cluster is created, its worker nodes are allocated public IP addresses, which are used for external communication. For improved security and control, the Static IP feature allows you to ensure that outbound traffic from your cluster uses specific IP addresses.

   Here is what will be deployed on your cluster:

   • Nat Gateways
   • Elastic IPs
   • Private subnets

   Once set up, here is the procedure to find your static IP addresses on AWS:

   • On your AWS account, select the VPC service.
   • On the left menu, you’ll find Elastic IP addresses. Once on it, in the Allocated IPv4 address column, you’ll have your public IPs.

   If you work in a sensitive business area such as financial technology, enabling the Static IP feature can help fulfil the security requirements of some of the external services you use, therefore making it easier for you to get whitelisted by them.

   This feature has been activated by default. Since February 1, 2024, AWS charge public IPv4 Addresses. Disabling it may cost you more, depending on the number of nodes in your cluster. Check this link for more information.

   Custom VPC Subnet

   Virtual Private Cloud (VPC) peering allows you to set up a connection between your Qovery VPC and another VPC on your AWS account. This way, you can access resources stored on your AWS VPC directly from your Qovery applications.

   A VPC can only be used if it has at least one range of IP addresses called a subnet. When you create a cluster, Qovery automatically picks a default subnet for it. However, to perform VPC peering, you may want to define which specific VPC subnet you want to use, so that you can avoid any conflicting settings. To do so, you can enable the Custom VPC Subnet feature on your cluster. For more information on how to set up VPC peering, see our dedicated tutorial.

4. In the Ready to install your cluster window, check that the services needed to install your cluster are correct.

   You can now press the Create and Install button.

   Your cluster is now displayed in your organization settings, featuring the Installing... status (orange status). Once your cluster is properly installed, its status turns to green and you will be able to deploy your applications on it.

   You can follow the execution of the action via the cluster status and/or by accessing the Cluster Logs

How was it? Did this tutorial work?  Yes  No

Migrating from AWS with auto-scaler to AWS with Karpenter

A SQS queue will be created. Before deploying your cluster, update the IAM permissions of the Qovery user, make sure to use the latest version here to add the permission on SQS.

You can easily activate Karpenter on your non-production existing cluster by following this process:

1. Open your Qovery Console.

2. On the left menu bar, click on the Cluster page.

3. To access your cluster settings, click on the wheel button:

   

4. Access to Resources section and switch on the toggle Activate Karpenter

5. Update your cluster by selecting the action Update from the drop-down menu.

6. Once the update is complete, your cluster will be running on Karpenter. By default, only the instance types selected when you created your AWS cluster with the auto-scaler will be configured. You can add additional instance types by editing the instance types in the resources section.

How was it? Did this tutorial work?  Yes  No

Managing your Cluster Settings

To manage the settings of an existing cluster:

1. Open your Qovery Console.

2. On the left menu bar, click on the Cluster page.

3. To access your cluster settings, click on the wheel button:

   

How was it? Did this tutorial work?  Yes  No

Below you can find a description of each section

General

The General tab allows you to define high-level information on your cluster:

Item	Description
Cluster Name	To edit the name of your cluster.
Description	To enter or edit the description of your cluster.
Production Cluster	To enter or edit the production flag of your cluster.

Credentials

Here you can manage here the cloud provider credentials associated with your cluster.

If you need to change the credentials:

• generate a new set of credentials on your cloud provider(Procedure for AWS account)
• create the new credential on the Qovery by opening the drop-down and selecting "New Credentials"

Once created and associated, you need to updating your cluster to apply the change.

Resources

Qovery allows you to modify the resources allocated for your cluster:

• The list of the instance types
• The spot instances activation
• The Node disk size (GB) field, enter the disk capacity you want to allocate to your worker node(s) (meaning how much data, in gigabytes, you want each worker node to be able to hold).

Instance type selection from your Qovery Console has direct consequences on your cloud provider’s bill. While Qovery allows you to switch to a different instance type whenever you want, it is your sole responsibility to keep an eye on your infrastructure costs, especially when you want to upsize.

For more information on the instance types provided by each cloud provider and their associated pricing, see What are the different instance types available when creating a cluster?

Image registry

In this tab, you will see that a container registry already exist (called registry-{$UIID}). This is your cloud provider container registry used by Qovery to manage the deployment of your applications by mirroring the docker images.

The credentials configured on this registry are the one used to create the cluster. But you can still update them if you prefer to manage them separately (dedicated pair of creds just to access the registry).

Check this link for more information.

Features

The Features tab in your cluster settings allows you to check if the Static IP, Custom VPC subnet, Deploy on existing VPC features are enabled on your cluster. The enabled features cannot be changed after the creation of the cluster.

Network

The Network tab in your cluster settings allows you to update your Qovery VPC route table so that you can perform VPC peering. For step-by-step guidelines on how to set up VPC peering, see our dedicated tutorial.