Clusters
Managing your Clusters with Qovery
From the Qovery Console, you can manage the settings of the clusters you want to run on your infrastructure. The clusters are then created (or updated) by the cloud provider that hosts them.
Creating a Cluster
To create a cluster:
Open your Qovery Console.
On the left menu bar, click on the Cluster page:
Click
Add Cluster
:In the
Create Cluster
window enter:Cluster name
: enter the name of your choice for your cluster.Description
: enter a description to identify better your cluster.Production cluster
: select this option if your cluster will be used for production.Cloud provider
: select your cloud provider.Region
: select the geographical area in which you want your cluster to be hosted.Credentials
: select one of the existing cloud provider credentials or add a new one by clicking onNew Credentials
. In the New credentials window, add the credentials that you have generated on your cloud provider console (Procedure for AWS account, Procedure for Scaleway account, Procedure for GCP account). Added credentials can be used later to create and manage additional cluster.
To confirm, click
Next
.In the
Set Resources
window, select:Cluster
: select the cluster type to use. Please refer to this section for more information.Disk size
: select the size of the disks to be attached to your cluster instances (to locally store container images etc..). Setting available only on AWS.Instance type
: select the type of worker nodes you want to deploy to your cluster:Node auto-scaling
: define the minimum and the maximum number of worker nodes that your cluster can run. The lowest number is the number of worker nodes running on your infrastructure at any time, while the highest number is the maximum number of worker nodes that can automatically be deployed as traffic grows. Please note that a minimum of 3 worker nodes is required to deploy your EKS cluster.
For AWS EKS clusters, you have the possibility to enable
Karpenter
autoscaler to improve the efficiency and cost of running workloads on your cluster. You can check the official documentation for more information.Today, only new non-production clusters are supported. It means you won't be able to enable it on your already existing cluster. It will be supported soon.
By activating Karpenter, you have to set:
Disk size
: select the size of the disks to be attached to your cluster instances (to locally store container images etc..).Default node architecture
: If you build your application with the Qovery CI, your application will be built using this architecture by default.Spot instances
: In order to reduce even more your costs, you can also enable the spot instances on your clusters. Spot instances cost up to 90% less compared to On-Demand prices. But keep in mind that spot instances can be terminated by the cloud provider at any time. Check this documentation for more information. Even if this flag is enabled, the statefulsets won't run on spot instances.
To confirm, click
Next
.(Only for AWS K8S Clusters) In the
Features
window, select the features you want to enable on your cluster.(Only for Single EC2 K3S Clusters) In the
Set SSH Key
window:The SSH key enables you (or Qovery on your behalf) to freely manage your cluster. For information on how to generate an SSH key, see Generating an SSH Key for Your Cluster.
You can add an SSH key to your cluster settings later, however it is recommended to do it at cluster creation to avoid downtime.
In the
Ready to install your cluster
window, check that the services needed to install your cluster are correct.You can now press the
Create and Install
button.Your cluster is now displayed in your organization settings, featuring the
Installing...
status (orange status). Once your cluster is properly installed, its status turns to green and you will be able to deploy your applications on it.
Managing your Cluster Settings
To manage the settings of an existing cluster:
Open your Qovery Console.
On the left menu bar, click on the Cluster page:
To access your cluster settings, click on the wheel button:
Below you can find a description of each section
General
The General
tab allows you to define high-level information on your cluster:
Item | Description |
---|---|
Cluster Name | To edit the name of your cluster. |
Description | To enter or edit the description of your cluster. |
Production Cluster | To enter or edit the production flag of your cluster. |
Credentials
Here you can manage here the cloud provider credentials associated with your cluster.
If you need to change the credentials:
- generate a new set of credentials on your cloud provider(Procedure for AWS account, Procedure for Scaleway account, Procedure for GCP account)
- create the new credential on the Qovery by opening the drop-down and selecting "New Credentials"
In the dedicated fields, enter the credentials you created on your cloud provider account:
Account Provider | Field Labels |
---|---|
AWS | Access Key and Secret Access Key |
Scaleway | Scaleway Access Key , Scaleway Secret Key , Scaleway Project ID and Scaleway Organization ID |
GCP | GCP JSON key |
Once created and associated, you need to updating your cluster to apply the change.
Resources
Qovery allows you to modify the resources allocated for your cluster:
- In the
Instance type
dropdown menu, select the type of worker node(s) you want to deploy to your cluster. - (AWS users only) In the
Node disk size (GB)
field, enter the disk capacity you want to allocate to your worker node(s) (meaning how much data, in gigabytes, you want each worker node to be able to hold). - (EKS users only) On the
Nodes auto-scaling
, define the range of worker nodes you want to deploy to your cluster.
Image registry
In this tab, you will see that a container registry already exist (called registry-{$UIID}
).
This is your cloud provider container registry used by Qovery to manage the deployment of your applications by mirroring the docker images.
The credentials configured on this registry are the one used to create the cluster. But you can still update them if you prefer to manage them separately (dedicated pair of creds just to access the registry).
Check this link for more information.
Features
The Features
tab in your cluster settings allows you to check if the Static IP, Custom VPC subnet, Deploy on existing VPC features are enabled on your cluster. The enabled features cannot be changed after the creation of the cluster.
Static IP
The Static IP feature is currently only available to clusters deployed on AWS and GCP with a VPC managed by Qovery and can only be enabled at cluster creation.
By default, when your cluster is created, its worker nodes are allocated public IP addresses, which are used for external communication. For improved security and control, the Static IP feature allows you to ensure that outbound traffic from your cluster uses specific IP addresses.
Here is what will be deployed on AWS
:
- Nat Gateways
- Elastic IPs
- Private subnets
Here is what will be deployed on GCP
:
- Cloud Nats
- Static IPs
- Routers
Once set up, here is the procedure to find your static IP addresses on AWS
:
- On your AWS account, select the VPC service.
- On the left menu, you’ll find Elastic IP addresses. Once on it, in the Allocated IPv4 address column, you’ll have your public IPs.
Once set up, here is the procedure to find your static IP addresses on GCP
:
- On your GCP account, select the IP addresses service.
- In the list you will find your static IP used by your cluster router.
Custom VPC Subnet
The VPC feature is currently only available to clusters deployed on AWS with a VPC managed by Qovery and can only be enabled at cluster creation.
Virtual Private Cloud (VPC) peering allows you to set up a connection between your Qovery VPC and another VPC on your AWS account. This way, you can access resources stored on your AWS VPC directly from your Qovery applications.
A VPC can only be used if it has at least one range of IP addresses called a subnet. When you create a cluster, Qovery automatically picks a default subnet for it. However, to perform VPC peering, you may want to define which specific VPC subnet you want to use, so that you can avoid any conflicting settings. To do so, you can enable the Custom VPC Subnet feature on your cluster. For more information on how to set up VPC peering, see our dedicated tutorial.
Use existing VPC
The Deploy on existing VPC feature is currently only available to clusters deployed on AWS
and GCP
when you select Deploy on my existing VPC
VPC mode and can only be enabled at cluster creation.
Use existing VPC - AWS:
You have to specify the VPC id
(1) and ensure that in your VPC settings you have enabled the DNS hostnames
(2):
Then you have to specify the different subnets ids:
EKS:
The EKS subnets are mandatory, you have to specify at least one subnet id per zone (1) and ensure you have enabled the auto-assign public IPv4 address setting on your subnets (2).
Managed databases:
This section is exclusively for enabling managed databases (container databases will be enabled by default).
Depending on the managed databases you want to you use (MongoDB, RDS:MySQL/PostgreSQL and Redis), specify at least one subnet id per zone.
Use existing VPC - GCP:
In GCP you have two VPC modes: Automatic
or Custom
.
If you are using an automatic or a custom VPC, you have to set:
- Your VPC Name
- External project id (optional): by default, the project id used is the one specified in the credentials file. But if your VPC is defined in another GCP project, you have to specify the Project id.
In addition if you are using a custom VPC, you have to set:
- Your Subnet range name (
https://console.cloud.google.com/networking/networks/details/<your-vpc>?project=<your-project>&pageTab=SUBNETS
)
Network
The Network
tab in your cluster settings allows you to update your Qovery VPC route table so that you can perform VPC peering. For step-by-step guidelines on how to set up VPC peering, see our dedicated tutorial.
Performing Actions on your Clusters
Qovery allows you to update, stop, restart or delete your clusters at organization level.
Action | Description |
---|---|
Updating a cluster | To redeploy your cluster after a change has been made to it. |
Stopping a cluster | To temporarily stop your cluster. Some services you have subscribed to via your cloud provider may still be active and incur costs when your cluster is stopped. For more information, see Stopping a cluster. |
Restarting a cluster | To restart your cluster after it has been temporarily stopped. |
Deleting a cluster | To delete your cluster. This is final and needs to be done properly to ensure all the services deployed by Qovery on your cloud provider's account are disabled, with no leftover cloud-related costs. For more information, see Deleting a cluster. |
To access these actions:
Open your Qovery Console.
On the left menu bar, click on the Cluster page:
To view your cluster actions, click
Play
button:A dropdown menu unfolds, featuring all the actions available on your cluster.
You can follow the execution of the action via the cluster status and/or by accessing the Cluster Logs
Updating a Cluster
If you made a change on your cluster, you need to run an update on your cluster to propagate remotely the new configuration.
To update your cluster, select the action Update
from the drop-down menu.
A confirmation pop-up window opens before triggering the action.
Once confirmed, the status of your cluster turns Updating...
(orange status).
Once the update is complete, the status dot next to your cluster turns green.
Stopping a Cluster
Qovery allows you to temporarily stop your cluster instead of deleting it.
To temporarily stop a cluster, select the Stop
action from the drop-down menu.
A confirmation pop-up window opens before triggering the action.
Once confirmed, the status of your cluster turns to Pausing...
(orange status).
Once the stop is complete, the status dot next to your cluster turns to grey, and the status of your cluster turns to Paused
(gray status).
Restarting a Cluster
You can restart a cluster after it has been temporarily stopped.
To restart your cluster, select the action Resume
from the drop-down menu.
A confirmation pop-up window opens before triggering the action.
Once confirmed, the status of your cluster turns to Updating...
(orange status).
Once your cluster has restarted, the status dot next to your cluster turns to green.
Deleting a Cluster
To delete a cluster, open the ...
section and press Delete Cluster
.
3 options can be chosen to delete a cluster:
1) Default This is the default behaviour, this option shall be chosen every time you want to delete properly a cluster from the Qovery console AND your cloud provider account.
This operation will delete:
- Cloud provider: any resource created by Qovery on your cloud provider account to run this cluster will be deleted, including any application running on it.
- Qovery organization: the configuration of this cluster and any linked environment.
2) Delete Cluster on cloud provider and Qovery configuration
This option shall be chosen when the cluster delete operation with the Default
option fails since you have manually modified/deleted the RDS instances created by Qovery on your cloud provider account.
This operation will delete:
- Cloud provider: any resource created by Qovery on your cloud provider account to run this cluster will be deleted, including any application running on it.
- Qovery organization: the configuration of this cluster and any linked environment.
3) Delete Qovery config only
This option shall be chosen when you have already deleted any Qovery resource on your cloud account and you want to delete the cluster object from your Qovery console.
This operation will delete:
- Cloud provider: nothing will be removed from your cloud account. You will have to manually delete any resource created by Qovery directly from your cloud provider console.
- Qovery organization: the configuration of this cluster and any linked environment.
Once confirmed, the cluster status turns to Deleting...
(red status) and once the deletion is complete, the cluster is removed from your organization settings.
Audit logs
To get the cluster filtered audit logs, open the ...
section and press See audit logs
.
You will be redirected to the audit logs section. A filter on the dedicated cluster will be applied. You only see the audit logs regarding cluster operations.
Get your cluster id
To get your Qovery cluster id, open the ...
section and press Copy identifier
.
The cluster id in Qovery will be in your clipboard.
Get your cluster kubeconfig file
If you need to get your kubeconfig file, open the ...
section and press Get Kubeconfig
.
Then the kubeconfig yaml file will be automatically downloaded.
Logs
Qovery allows you to access the logs of your cluster in order to follow its installation or investigate any issue happening on it.
To access the logs you need to open the cluster, click the log button
A new window is opened, displaying the logs of the cluster.
The tab system on the right allows you to access the cluster information and, if an error occurs, the detail of the error.
Generating an SSH Key for Your Cluster
To allow Qovery or yourself to connect remotely to your K3s instance and manage it, you need to generate an SSH key and add it to your cluster settings. To do so:
On your computer, open a terminal.
Run
ssh-keygen -t
, followed by the key type and an optional comment.For example, you can enter
ssh-keygen -t rsa -b 2048 -C "<comment>"
.Press
Enter
.You should get an output similar to:
{Generating public/private ed25519 key pair.Enter file in which to save the key (/home/user/.ssh/id_ed25519):}Accept the suggested filename and directory, unless you want to save your SSH key in a specific directory where you store other keys.
Enter a passphrase:
{Enter passphrase (empty for no passphrase):Enter same passphrase again:}A confirmation is displayed, including information about where your files are stored.
- Access the public key and copy its value{cat /home/user/.ssh/id_ed25519.pub | pbcopy}
Note: Replace the .pub key path with the one where is located the key you have previously generated
You can add the generated public SSH key at cluster creation (see Creating a Cluster), or later from your cluster settings.
To do so:
- on your Qovery Console, access your cluster settings.
- In the
Remote Access
tab, enter your SSH key and clickSave
. - Launch the Update Cluster action to propagate the new key.
Use custom domain and wildcard TLS for the whole cluster (beta)
By default, Qovery provides a domain (ex bool.sh
) on every deployed cluster. It is used to provide a DNS and TLS certificate to every application requiring external access on a cluster.
You can customize the domain for every application. However, when it comes to having more than 100 custom domains with the same domain you will hit Let's Encrypt quotas.
To overcome this issue, you can use a wildcard TLS certificate for the whole cluster. It will allow you to have as many DNS records for a single domain as you want on the same cluster with a single TLS certificate.
At the moment, Qovery only supports wildcard TLS certificates with Cloudflare. To use it, you need to have a Cloudflare account and a domain name managed by Cloudflare. If you don't have one, you can create a free account and transfer your domain to Cloudflare.
Once you have a Cloudflare account and a domain name managed by Cloudflare, you need to create a Cloudflare API token. Go into your Cloudflare account, click on your profile picture, then My Profile
. In the API Tokens
section, click on Create Token
. In the Create Custom Token
section, select the following permissions:
- API token a descriptive name: Qovery domain
your domain name
- Permissions:
- Zone - DNS - Edit
- Zone - Zone - Read
- Zone Resources:
- Include - Specific zone -
your domain name
- Include - Specific zone -
To finish, click on Continue to Summary
and Create Token
. Save the token somewhere safe, you will need it later.
Prepare the Token, the Cloudflare account email and the domain to be set on your cluster. Now contact Qovery and request to use your domain.
Cleaning up a Cluster from your AWS Account
To clean up a Qovery cluster from your cloud provider account, go to AWS Console
>Services
>Management & Governance
>Resource Groups & Tag Editor
> Create Resource Group
:
Step | Description |
---|---|
1 | In the Group type area, select Tag based . |
2 | In the Tags field of the Grouping criteria area, enter ClusterId . |
3 | Click Add . |
4 | Click Preview Resources . All your Qovery clusters are now displayed in the Group resources table, and you can delete them by hand. |
FAQ
What is a cluster?
Please refer to the basic concepts section
Why do I need a cluster?
Qovery is built on top of Kubernetes, which means we need Kubernetes clusters to be able to deploy and run your applications.
Thanks to clusters, you can easily deploy several (and many) instances of the same application, so that if one fails, the others can instantly take over. Also, clusters can auto-scale, meaning that the number of worker nodes in a cluster can automatically go up or down as traffic fluctuates on your application(s), thus ensuring high availability and performance. Clusters are also extremely useful to isolate your production environment from your staging environment.
In short, through the use of clusters, Kubernetes provides you with a resilient, flexible and powerful infrastructure, fit for production environment needs and requirements. And with the help of Qovery, setting up and maintaining your Kubernetes clusters has never been easier.
Qovery allows you to create and manage two types of clusters:
Managed K8S | BETA - Single EC2 (K3s) | |
---|---|---|
Description | A multi-node Kubernetes cluster managed by your cloud provider (EKS, Kapsule etc..) | K3s Cluster running on a single EC2 instance (single-node) Available only on AWS and still in BETA |
Usage | Hosting professional applications in production (resilient, scalable and powerful infrastructure). Scalable staging / preview / dev environments | Hobby projects, trying out Qovery, ephemeral environments deployment |
Cloud provider cost | Starting from 200$/month, based on the chosen instance type | starting from 20$/month, based on the chosen instance type |
What are the different instance types available when creating a cluster?
The range of instance types available at cluster creation depends on your cloud provider:
- AWS offers over 400 instance types. You can view their details on the official AWS website, as well as their pricing.
- Scaleway also offers a wide range of instance types, whose details and pricing you can view on the official Scaleway website.
- GCP clusters are deployed in auto-pilot mode so you will have access by default to every instance type available
How does Qovery handle Kubernetes version upgrades?
As far as cluster upgrades to a newer version of Kubernetes are concerned, our Qovery engineering team handles everything in due time, so you don’t even need to think about it!
Usually, we work on a given upgrade for one month of intensive testing on our end in order to make sure everything will be smooth for you. Once we are pretty confident our stack is stable, we move on with the following steps which last approximately 3 weeks:
- Notify users about new version coming in approximatively 1 month before
- Upgrade clusters for a handful of beta-tester customers (1 week)
- Make available the new version for all clusters (new or existing)
- Upgrade all non-production flagged clusters (1 week)
- Upgrade all clusters the production clusters (1 week)
If, somehow the planning or timeframe for the upgrade is clashing with your business needs, you will be able to trigger the upgrade of your cluster manually via the "Upgrade to XX.XX" action available from the Play
menu of your cluster. This action will be available on your cluster once we will make the new version available globally (step 3), you will notice that the Play
button of your cluster will be highlighted in yellow.
What do you do when a vulnerability is found?
Security is our main concern. When a vulnerability is found, here are the actions that we take:
- We quickly identify how significant is the impact of the vulnerability.
- We look at how we can solve or mitigate the vulnerability.
- We transparently communicate with our customers about the vulnerability to help them take the right actions.