Clusters

This section brings you answers to all the questions our users usually ask about clusters:

What is a cluster?

At Qovery, when we refer to cluster, we mean Kubernetes cluster. A Kubernetes cluster is a collection of node machines that allows you to run containerized applications. It is usually made up of:

  • Pods: think of a pod as one instance of your application. Pods are the smallest deployable objects in Kubernetes, and they are hosted by worker nodes.
  • Worker Nodes: worker nodes essentially run your applications and workloads. When you create a cluster from your Qovery Console, it generates the set up of worker nodes (also called “instances”, “EC2 instances” for AWS users, or “droplets” for DigitalOcean users). Qovery allows you to define worker nodes settings, so that you end up deploying the right type of instances on your infrastructure based on your CPU, memory, storage and network performance needs.
  • a Control Plane (or Master Node): the control plane manages the worker nodes. Since we deploy managed Kubernetes services, the control plane is handled exclusively by your cloud provider, and left untouched by Qovery.

Application

For more information on Kubernetes clusters, see the Kubernetes documentation.

Why do I need a cluster?

Qovery is built on top of Kubernetes, which means we need Kubernetes clusters to be able to deploy and run your applications.

Thanks to clusters, you can easily deploy several (and many) instances of the same application, so that if one fails, the others can instantly take over. Also, clusters can auto-scale, meaning that the number of worker nodes in a cluster can automatically go up or down as traffic fluctuates on your application(s), thus ensuring high availability and performance. Clusters are also extremely useful to isolate your production environment from your staging environment.

In short, through the use of clusters, Kubernetes provides you with a resilient, flexible and powerful infrastructure, fit for production environment needs and requirements. And with the help of Qovery, setting up and maintaining your Kubernetes clusters has never been easier.

Qovery allows you to create and manage two types of clusters:

Managed K8S BETA - Single EC2 (K3s)
DescriptionA multi-node Kubernetes cluster managed by your cloud provider (EKS, Kapsule etc..)K3s Cluster running on a single EC2 instance (single-node) Available only on AWS and still in BETA
UsageHosting professional applications in production (resilient, scalable and powerful infrastructure). Scalable staging / preview / dev environmentsHobby projects, trying out Qovery, ephemeral environments deployment
Cloud provider costStarting from 200$/month, based on the chosen instance typestarting from 20$/month, based on the chosen instance type


What are the different instance types available when creating a cluster?

The range of instance types available at cluster creation depends on your cloud provider:

What is the default cluster?

The default cluster is the first cluster you installed in your organization.

When you create a new environment and leave the mode and cluster parameters set to the value Automatic, your environment is deployed to:

  • the cluster defined in one of your project rules,
  • or to the default cluster if no project rule applies.

For more information on deployment rules, see Project.

How does Qovery handle cluster updates and upgrades?

As far as cluster updates and upgrades to a newer version of Kubernetes are concerned, our Qovery engineering team handles everything in due time, so you don’t even need to think about it!

Usually, we work on a given upgrade for one month of intensive testing on our end in order to make sure everything will be smooth for you. Once we are pretty confident our stack is stable, we move on with the following steps which last approximately 3 weeks:

  1. Notify users about new version coming in approximatively 1 month before
  2. Upgrade clusters for a handful of beta-tester customers (1 week)
  3. Upgrade all non-production flagged clusters (1-2 week(s))
  4. Upgrade all clusters

If, somehow the planning or timeframe for the upgrade is clashing with your business needs, you will be able to contact us so we can arrange the best timeframe for you.

What do you do when a vulnerability is found?

Security is our main concern. When a vulnerability is found, here are the actions that we take:

  1. We quickly identify how significant is the impact of the vulnerability.
  2. We look at how we can solve or mitigate the vulnerability.
  3. We transparently communicate with our customers about the vulnerability to help them take the right actions.

Managing your Clusters with Qovery

From the Qovery Console, you can manage the settings of the clusters you want to run on your infrastructure. The clusters are then created (or updated) by the cloud provider that hosts them.

Creating a Cluster

To create a cluster:

  1. Open your Qovery Console.

  2. On the left menu bar, click on the Cluster page:

    Cluster Access

  3. Click Add Cluster:

    Add Cluster Button

  4. In the Create Cluster window enter:

    • Cluster name: enter the name of your choice for your cluster.
    • Description: enter a description to identify better your cluster.
    • Production cluster: select this option if your cluster will be used for production.
    • Cloud provider: select your cloud provider.
    • Region: select the geographical area in which you want your cluster to be hosted.
    • Credentials: select one of the existing cloud provider credentials or add a new one by clicking on New Credentials. In the New credentials window, add the credentials that you have generated on your cloud provider console (Procedure for AWS account, Procedure for Scaleway account, Procedure for GCP account). Added credentials can be used later to create and manage additional cluster.

    To confirm, click Next.

  5. In the Set Resources window, select:

    • Cluster: select the cluster type to use. Please refer to this section for more information.
    • Instance type: select the type of worker nodes you want to deploy to your cluster:
    • Disk size: select the size of the disks to be attached to your cluster instances (to locally store container images etc..). Setting available only on AWS.
    • Node auto-scaling: define the minimum and the maximum number of worker nodes that your cluster can run. The lowest number is the number of worker nodes running on your infrastructure at any time, while the highest number is the maximum number of worker nodes that can automatically be deployed as traffic grows. Please note that a minimum of 3 worker nodes is required to deploy your EKS cluster.


    To confirm, click Next.

  6. (Only for AWS K8S Clusters) In the Features window, select the features you want to enable on your cluster.

  7. (Only for Single EC2 K3S Clusters) In the Set SSH Key window:

    The SSH key enables you (or Qovery on your behalf) to freely manage your cluster. For information on how to generate an SSH key, see Generating an SSH Key for Your Cluster.

    You can add an SSH key to your cluster settings later, however it is recommended to do it at cluster creation to avoid downtime.

  8. In the Ready to install your cluster window, check that the services needed to install your cluster are correct.

    You can now press the Create and Install button.

    Your cluster is now displayed in your organization settings, featuring the Installing... status (orange status). Once your cluster is properly installed, its status turns to green and you will be able to deploy your applications on it.

Managing your Cluster Settings

To manage the settings of an existing cluster:

  1. Open your Qovery Console.

  2. On the left menu bar, click on the Cluster page:

    Cluster Access

  3. To access your cluster settings, click on the wheel button:

    Display Cluster Settings

Below you can find a description of each section

General

The General tab allows you to define high-level information on your cluster:

ItemDescription
Cluster NameTo edit the name of your cluster.
DescriptionTo enter or edit the description of your cluster.
Production ClusterTo enter or edit the production flag of your cluster.

Credentials

Here you can manage here the cloud provider credentials associated with your cluster.

If you need to change the credentials:

In the dedicated fields, enter the credentials you created on your cloud provider account:

Account ProviderField Labels
AWSAccess Key and Secret Access Key
ScalewayScaleway Access Key, Scaleway Secret Key, Scaleway Project ID and Scaleway Organization ID
GCPGCP JSON key

Once created and associated, you need to updating your cluster to apply the change.

Resources

Qovery allows you to modify the resources allocated for your cluster:

  • In the Instance type dropdown menu, select the type of worker node(s) you want to deploy to your cluster.
  • (AWS users only) In the Node disk size (GB) field, enter the disk capacity you want to allocate to your worker node(s) (meaning how much data, in gigabytes, you want each worker node to be able to hold).
  • (EKS users only) On the Nodes auto-scaling, define the range of worker nodes you want to deploy to your cluster.

Features

The Features tab in your cluster settings allows you to check if the Static IP, Custom VPC subnet, Deploy on existing VPC features are enabled on your cluster. The enabled features cannot be changed after the creation of the cluster.

Static IP

The Static IP feature is currently only available to clusters deployed on AWS with a VPC managed by Qovery and can only be enabled at cluster creation.

By default, when your cluster is created, its worker nodes are allocated public IP addresses, which are used for external communication. For improved security and control, the Static IP feature allows you to ensure that outbound traffic from your cluster uses specific IP addresses.

Here is what will be deployed on AWS:

  • Nat Gateways
  • Elastic IPs
  • Private subnets

Once set up, here is the procedure to find your static IP addresses:

  • On your AWS account, select the VPC service.
  • On the left menu, you’ll find Elastic IP addresses. Once on it, in the Allocated IPv4 address column, you’ll have your public IPs.
Custom VPC Subnet

The VPC feature is currently only available to clusters deployed on AWS with a VPC managed by Qovery and can only be enabled at cluster creation.

Virtual Private Cloud (VPC) peering allows you to set up a connection between your Qovery VPC and another VPC on your AWS account. This way, you can access resources stored on your AWS VPC directly from your Qovery applications.

A VPC can only be used if it has at least one range of IP addresses called a subnet. When you create a cluster, Qovery automatically picks a default subnet for it. However, to perform VPC peering, you may want to define which specific VPC subnet you want to use, so that you can avoid any conflicting settings. To do so, you can enable the Custom VPC Subnet feature on your cluster. For more information on how to set up VPC peering, see our dedicated tutorial.

Use existing VPC

The Deploy on existing VPC feature is currently only available to clusters deployed on AWS when you select Deploy on my existing VPC VPC mode and can only be enabled at cluster creation.

You have to specify the VPC id (1) and ensure that in your VPC settings you have enabled the DNS hostnames (2):

Existing VPC AWS DNS Hostnmaes

Then you have to specify the different subnets ids:

EKS:

The EKS subnets are mandatory, you have to specify at least one subnet id per zone (1) and ensure you have enabled the auto-assign public IPv4 address setting on your subnets (2).

Existing VPC AWS DNS Hostnmaes

Managed databases:

This section is exclusively for enabling managed databases (container databases will be enabled by default).

Depending on the managed databases you want to you use (MongoDB, RDS:MySQL/PostgreSQL and Redis), specify at least one subnet id per zone.

Network

The Network tab in your cluster settings allows you to update your Qovery VPC route table so that you can perform VPC peering. For step-by-step guidelines on how to set up VPC peering, see our dedicated tutorial.

Performing Actions on your Clusters

Qovery allows you to update, stop, restart or delete your clusters at organization level.

ActionDescription
Updating a clusterTo redeploy your cluster after a change has been made to it.
Stopping a clusterTo temporarily stop your cluster. Some services you have subscribed to via your cloud provider may still be active and incur costs when your cluster is stopped. For more information, see Stopping a cluster.
Restarting a clusterTo restart your cluster after it has been temporarily stopped.
Deleting a clusterTo delete your cluster. This is final and needs to be done properly to ensure all the services deployed by Qovery on your cloud provider's account are disabled, with no leftover cloud-related costs. For more information, see Deleting a cluster.

To access these actions:

  1. Open your Qovery Console.

  2. On the left menu bar, click on the Cluster page:

    Cluster Access

  3. To view your cluster actions, click Play button:

    Cluster Actions Menu

    A dropdown menu unfolds, featuring all the actions available on your cluster.

You can follow the execution of the action via the cluster status and/or by accessing the Cluster Logs

Updating a Cluster

If you made a change on your cluster, you need to run an update on your cluster to propagate remotely the new configuration.

To update your cluster, select the action Update from the drop-down menu.

A confirmation pop-up window opens before triggering the action.

Once confirmed, the status of your cluster turns Updating... (orange status).

Once the update is complete, the status dot next to your cluster turns green.

Stopping a Cluster

Qovery allows you to temporarily stop your cluster instead of deleting it.

To temporarily stop a cluster, select the Stop action from the drop-down menu. A confirmation pop-up window opens before triggering the action.

Once confirmed, the status of your cluster turns to Pausing... (orange status).

Once the stop is complete, the status dot next to your cluster turns to grey, and the status of your cluster turns to Paused (gray status).

Restarting a Cluster

You can restart a cluster after it has been temporarily stopped.

To restart your cluster, select the action Resume from the drop-down menu.

A confirmation pop-up window opens before triggering the action.

Once confirmed, the status of your cluster turns to Updating... (orange status).

Once your cluster has restarted, the status dot next to your cluster turns to green.

Deleting a Cluster

To delete a cluster, open the ... section and press Delete Cluster.

3 options can be chosen to delete a cluster:

1) Default This is the default behaviour, this option shall be chosen every time you want to delete properly a cluster from the Qovery console AND your cloud provider account.

This operation will delete:

  • Cloud provider: any resource created by Qovery on your cloud provider account to run this cluster will be deleted, including any application running on it.
  • Qovery organization: the configuration of this cluster and any linked environment.

2) Delete Cluster on cloud provider and Qovery configuration

This option shall be chosen when the cluster delete operation with the Default option fails since you have manually modified/deleted the RDS instances created by Qovery on your cloud provider account.

This operation will delete:

  • Cloud provider: any resource created by Qovery on your cloud provider account to run this cluster will be deleted, including any application running on it.
  • Qovery organization: the configuration of this cluster and any linked environment.

3) Delete Qovery config only

This option shall be chosen when you have already deleted any Qovery resource on your cloud account and you want to delete the cluster object from your Qovery console.

This operation will delete:

  • Cloud provider: nothing will be removed from your cloud account. You will have to manually delete any resource created by Qovery directly from your cloud provider console.
  • Qovery organization: the configuration of this cluster and any linked environment.

Once confirmed, the cluster status turns to Deleting... (red status) and once the deletion is complete, the cluster is removed from your organization settings.

Audit logs

To get the cluster filtered audit logs, open the ... section and press See audit logs.

You will be redirected to the audit logs section. A filter on the dedicated cluster will be applied. You only see the audit logs regarding cluster operations.

Get your cluster id

To get your Qovery cluster id, open the ... section and press Copy identifier.

The cluster id in Qovery will be in your clipboard.

Get your cluster kubeconfig file

If you need to get your kubeconfig file, open the ... section and press Get Kubeconfig.

Then the kubeconfig yaml file will be automatically downloaded.

Logs

Qovery allows you to access the logs of your cluster in order to follow its installation or investigate any issue happening on it.

To access the logs you need to open the cluster, click the log button

Cluster Logs

A new window is opened, displaying the logs of the cluster.

Cluster Logs

The tab system on the right allows you to access the cluster information and, if an error occurs, the detail of the error.

Cluster Logs

Generating an SSH Key for Your Cluster

To allow Qovery or yourself to connect remotely to your K3s instance and manage it, you need to generate an SSH key and add it to your cluster settings. To do so:

  1. On your computer, open a terminal.

  2. Run ssh-keygen -t, followed by the key type and an optional comment.

    For example, you can enter ssh-keygen -t rsa -b 2048 -C "<comment>".

  3. Press Enter.

    You should get an output similar to:

    {
    Generating public/private ed25519 key pair.
    Enter file in which to save the key (/home/user/.ssh/id_ed25519):
    }
  4. Accept the suggested filename and directory, unless you want to save your SSH key in a specific directory where you store other keys.

  5. Enter a passphrase:

    {
    Enter passphrase (empty for no passphrase):
    Enter same passphrase again:
    }

    A confirmation is displayed, including information about where your files are stored.

  6. Access the public key and copy its value
    {
    cat /home/user/.ssh/id_ed25519.pub | pbcopy
    }

    Note: Replace the .pub key path with the one where is located the key you have previously generated

You can add the generated public SSH key at cluster creation (see Creating a Cluster), or later from your cluster settings.

To do so:

Use custom domain and wildcard TLS for the whole cluster (beta)

By default, Qovery provides a domain (ex bool.sh) on every deployed cluster. It is used to provide a DNS and TLS certificate to every application requiring external access on a cluster.

You can customize the domain for every application. However, when it comes to having more than 100 custom domains with the same domain you will hit Let's Encrypt quotas.

To overcome this issue, you can use a wildcard TLS certificate for the whole cluster. It will allow you to have as many DNS records for a single domain as you want on the same cluster with a single TLS certificate.

At the moment, Qovery only supports wildcard TLS certificates with Cloudflare. To use it, you need to have a Cloudflare account and a domain name managed by Cloudflare. If you don't have one, you can create a free account and transfer your domain to Cloudflare.

Once you have a Cloudflare account and a domain name managed by Cloudflare, you need to create a Cloudflare API token. Go into your Cloudflare account, click on your profile picture, then My Profile. In the API Tokens section, click on Create Token. In the Create Custom Token section, select the following permissions:

  • API token a descriptive name: Qovery domain your domain name
  • Permissions:
    • Zone - DNS - Edit
    • Zone - Zone - Read
  • Zone Resources:
    • Include - Specific zone - your domain name

To finish, click on Continue to Summary and Create Token. Save the token somewhere safe, you will need it later.

Prepare the Token, the Cloudflare account email and the domain to be set on your cluster. Now contact Qovery and request to use your domain.

Cleaning up a Cluster from your AWS Account

To clean up a Qovery cluster from your cloud provider account, go to AWS Console>Services>Management & Governance>Resource Groups & Tag Editor> Create Resource Group:

AWS Console Cluster Cleanup

StepDescription
1In the Group type area, select Tag based.
2In the Tags field of the Grouping criteria area, enter ClusterId.
3Click Add.
4Click Preview Resources.
All your Qovery clusters are now displayed in the Group resources table, and you can delete them by hand.