Configuration

Qovery BYOK is a set of Kubernetes components that you can configure to fit your needs. It is used to connect your Kubernetes cluster to Qovery control plane.

Components

How Qovery works with Self Managed Kubernetes cluster

They are two types of components:

Qovery components:

Qovery Control Plane: the Qovery Control Plane is the brain of Qovery. It is responsible for managing your applications and providing the API to interact with Qovery.
Qovery Cluster Agent (mandatory): the Qovery Cluster Agent is responsible for securely forwarding logs and metrics from your Kubernetes cluster to Qovery control plane.
Qovery Shell Agent (mandatory): the Qovery Shell Agent is responsible for giving you a secure remote shell access to your Kubernetes pods if you need it. E.g. when using qovery shell command.
Qovery Engine (optional): the Qovery Engine is responsible for managing your applications deployment on your Kubernetes cluster. It can be used Qovery side or is installed on your Kubernetes cluster.

Third-party components:

NGINX Ingress Controller (optional)
External DNS (optional)
Loki (optional)
Promtail (optional)
Cert Manager (optional)
...

Within the values.yaml file of the helm chart you can choose what you want to install and manage, and you will have a description of what services are used, and responsible for. You can disable them if you don't want to use them. And you can even install other components if you want to.

Configuration

At the top of the values file you will find a list of component that you can activate or not.

services:
  qovery:
    qovery-cluster-agent:
      enabled: true
    qovery-shell-agent:
      enabled: true
    qovery-engine:
      enabled: false
  ingress:
    ingress-nginx:
      enabled: true
...

Here's the complete list and the documentation explaining how to configure it.

Qovery

This is the configuration of Qovery itself and it is used by all Qovery components. This configuration is automatically generated when creating the cluster on the Qovery console (see [docs.getting-started.install-qovery.kubernetes.quickstart#install-qovery])

Do not share the jwtToken! Keep it in a safe place. It is used to authenticate the cluster.

Key	Required	Description
`qovery.clusterId`	Yes	The cluster ID. It is used to identify your cluster.
`qovery.clusterShortId`	Yes	The short cluster ID. It is used to identify your cluster.
`qovery.organizationId`	Yes	The organization ID. It is used to identify your organization.
`qovery.jwtToken`	Yes	The JWT token. It is used to authenticate your cluster.
`qovery.domain`	Yes	The domain name used by Qovery.
`qovery.domainWildcard`	Yes	The wildcard domain name used by Qovery.
`qovery.qoveryDnsUrl`	Yes	Qovery DNS url in case you want to use Qovery provided DNS
`qovery.lokiUrl`	No	Local Loki URL (required if Loki is set)
`qovery.promtailLokiUrl`	No	Promtail Loki URL (required if Promtail and Loki are set)
`qovery.acmeEmailAddr`	No	Email address used for `Let's Encrypt` TLS requests
`qovery.externalDnsPrefix`	No	ExernalDNS TXT record prefix (required if ExternalDNS is set)
`qovery.architectures`	No	Set cluster architectures (comma separated)

Qovery Cluster Agent


Required	Yes
If deployed	The cluster agent is responsible for securely forwarding logs and metrics from your Kubernetes cluster to Qovery control plane
If missing	The cluster will not report to Qovery control plane Kubernetes information, so the Qovery console will report unknown satus values

qovery-cluster-agent:
  fullnameOverride: qovery-cluster-agent

Qovery Shell Agent


Required	Yes
If deployed	Used to give a remote shell access to you Kubernetes pods (if user is allowed from Qovery RBAC) with the Qovery CLI
If missing	No remote connection will be possible, and Qovery support will not be able to help you to diagnose issues

qovery-shell-agent:
  fullnameOverride: qovery-shell-agent

Ingress


Required	No (but strongly recommended)
If deployed	Web services can be privately or publicly exposed
If missing	No web services will be exposed

Qovery us will be exposed NGINX Ingress Controller by default to route traffic to your applications.

Nginx Ingress Controller

Demo
AWS
GCP
Scaleway

Here is the minimum override configuration to be used:

ingress-nginx:
  fullnameOverride: ingress-nginx
  controller:
    useComponentLabel: true
    admissionWebhooks:
      enabled: false
    # Ingress class used when an application/container with public access is set
    ingressClass: nginx-qovery
    extraArgs:
      # Default TLS certificate name and path
      default-ssl-certificate: "qovery/letsencrypt-acme-qovery-cert"
    # Allows customization of the source of the IP address or FQDN to report in the ingress status field
    publishService:
      enabled: true

Other Ingress Controllers

Qovery supports other Ingress Controllers. Please contact us if you want to use another one. We will be happy to help you.

DNS


Required	No (but strongly recommended)
If deployed	Used to easily reach your applications with DNS records, even on private network
If missing	You will have easy access with dns names to your services, you'll have to use IPs

Qovery uses External DNS to automatically configure DNS records for your applications.

If you don't want or can't add your own DNS provider, Qovery proposes it's own managed sub-domain DNS provider for free. You'll then be able to later add your custom DNS record (no matter the provider) to point to your Qovery DNS sub-domain.

External DNS

Demo & QoveryDNS
Cloudflare

Here is one example with Qovery DNS provider:

external-dns:
  fullnameOverride: external-dns
  # set pdns for Qovery DNS managed (or you can use any supported provider by external-dns)
  provider: pdns
  # will use the domain name given by Qovery during the cluster setup phease
  domainFilters: [*domain]
  # an owner ID is set to avoid conflicts in case of multiple Qovery clusters
  txtOwnerId: *shortClusterId
  # a prefix to help Qovery to debug in case of issues
  txtPrefix: *externalDnsPrefix
  # set the Qovery DNS provider configuration
  pdns:
    apiUrl: *qoveryDnsUrl
    apiKey: *jwtToken
    apiPort: 443

Logging


Required	No (but strongly recommended)
If deployed	Retrieve and store application's log history
If missing	You'll have live logs, but you will miss log history for debugging purpose

Qovery uses Loki to store your logs in a S3 compatible bucket and Promtail to collect your logs.

Loki

Demo
AWS S3

Here is a configuration in Memory (no persistence) for Loki:

loki:
  fullnameOverride: loki
  loki:
    # no auth is set for internal cluster usage
    auth_enabled: false
    ingester:
      lifecycler:
        ring:
          kvstore:
            # we store it in memory for the demo, you'll lose history once Loki restarts
            store: inmemory
          replication_factor: 1
    schema_config:
      configs:
        - from: 2020-05-15
          store: boltdb-shipper
          object_store: filesystem
          schema: v11
          index:
            prefix: index_
            period: 24h
  monitoring:
    # all the monitoring part is disabled to reduce resource footprint for the demo usage
    dashboards:
      enabled: false
    rules:
      enabled: false
    serviceMonitor:
      enabled: false
      metricsInstance:
        enabled: false
    selfMonitoring:
      enabled: false
      grafanaAgent:
        installOperator: false
    grafanaAgent:
      enabled: false
    lokiCanary:
      enabled: false
  test:
    enabled: false
  gateway:
    enabled: false
  # we use a single binary to reduce resource footprint for the demo usage
  singleBinary:
    replicas: 1
    persistence:
      enabled: false
    extraVolumes:
      - name: data
        emptyDir: {}
      - name: storage
        emptyDir: {}
    extraVolumeMounts:
      - name: data
        mountPath: /data
      - name: storage
        mountPath: /var/loki

Promtail

A configuration example compatible with all providers:

promtail:
  fullnameOverride: promtail
  # promtail requires to be spawned in kube-system namespace
  namespace: kube-system
  priorityClassName: system-node-critical
  config:
    clients:
      # forward logs to Loki
      - url: *promtailLokiUrl
    snippets:
      extraRelabelConfigs:
        - action: labelmap
          # required to be able to watch logs from Qovery console interface
          regex: __meta_kubernetes_pod_label_(qovery_com_service_id|qovery_com_service_type|qovery_com_environment_id)

Certificates


Required	No (but strongly recommended)
If deployed	Cert-manager helps you to get TLS certificates through Let's Encrypt
If missing	Without it, you will not be able to automatically get TLS certificates

Qovery uses Cert Manager to automatically get TLS certificates for your applications.

Cert Manager

Here is the minimal setup for all cloud providers:

cert-manager:
  fullnameOverride: cert-manager
  # CRD are required
  installCRDs: true
  replicaCount: 1
  startupapicheck:
    jobAnnotations:
      helm.sh/hook: post-install,post-upgrade
    rbac:
      annotations:
        helm.sh/hook: post-install,post-upgrade
    serviceAccount:
      annotations:
        helm.sh/hook: post-install,post-upgrade

Qovery Cert Manager Webhook


Required	No (but if you're using Qovery DNS Provider)
If deployed	Required to get Let's Encrypt TLS if Qovery DNS Provider is used
If missing	Without it, you will not be able to automatically get TLS certificates with Qovery DNS Provider

Qovery DNS

A configuration example compatible with all providers:

qovery-cert-manager-webhook:
  fullnameOverride: qovery-cert-manager-webhook
  certManager:
    # set the same namespace than cert-manager
    namespace: qovery
    serviceAccountName: cert-manager
  secret:
    apiUrl: *qoveryDnsUrl
    apiKey: *jwtToken

Cert Manager Configs


Required	No
If deployed	This is an helper to deploy cert-manager config. But you can manually set it
If missing	Installing Cert-manager is not enough, you have to configure it to get TLS working

Demo
Qovery DNS
Cloudflare

This is the configuration of Cert Manager itself. It is used by all Cert Manager components.

cert-manager-configs:
  fullnameOverride: cert-manager-configs
  # set pdns to use Qovery DNS provider
  externalDnsProvider: pdns
  managedDns: [*domain]
  acme:
    letsEncrypt:
      emailReport: *acmeEmailAddr
      # As it's a demo cluster, we use the staging environment to avoid rate limit issues
      acmeUrl: https://acme-staging-v02.api.letsencrypt.org/directory
  provider:
    # set the provider of your choice or use the Qovery DNS provider
    pdns:
      apiPort: 443
      apiUrl: *qoveryDnsUrl
      apiKey: *jwtToken

Qovery uses Metrics Server to collect metrics from your Kubernetes cluster and scale your applications automatically based on custom metrics.

Observability

Metrics Server


Required	No (but strongly recommended)
If deployed	Mandatory if you want to retrive pod metrics for the Qovery agent and if you want to be able to use the horizontal pod scaling
If missing	No HPA and no application metrics in the QOveyr console

Demo
AWS
GCP
Scaleway

metrics-server:
  fullnameOverride: metrics-server
  defaultArgs:
    - --cert-dir=/tmp
    - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
    - --kubelet-use-node-status-port
    - --metric-resolution=15s
    - --kubelet-insecure-tls
  apiService:
    create: false

#Components

#Configuration

#Qovery

#Qovery Cluster Agent

#Qovery Shell Agent

#Ingress

#Nginx Ingress Controller

#Other Ingress Controllers

#DNS

#External DNS

#Logging

#Loki

#Promtail

#Certificates

#Cert Manager

#Qovery Cert Manager Webhook

#Cert Manager Configs

#Observability

#Metrics Server

Components

Configuration

Qovery

Qovery Cluster Agent

Qovery Shell Agent

Ingress

Nginx Ingress Controller

Other Ingress Controllers

DNS

External DNS

Logging

Loki

Promtail

Certificates

Cert Manager

Qovery Cert Manager Webhook

Cert Manager Configs

Observability

Metrics Server