tutorial

Kubernetes observability and monitoring with Datadog

How to integrate Datadog with Kubernetes on Qovery.
type: tutorialtechnology: qovery

While Qovery will soon provide basic metrics on apps resources usage, you might need a more advanced view on what happens on your infrastructure. There are many solutions on the market, one of them being Datadog. Datadog is one of the leading platforms for monitoring and observability, and it's pretty easy to integrate it with Qovery.

Goal

In this tutorial, we will install the Datadog agent on a Qovery cluster to gather metrics about infrastructure and applications.

  1. Create a Datadog Account

    The first step is to create an account on Datadog: https://www.datadoghq.com.

    You will be prompted to enter some information. An important decision you have to make is which site to use. It will determine where your data get stored.

    Warning: You can't migrate your data between regions, so chose carefully. More information here: https://docs.datadoghq.com/fr/getting_started/site/

    Datadog - Console

    After signup, you will be redirected to a wizard. Fill-in the information until you get to step 3.

  2. Prepare the agent Helm chart values

    When you get to step 3, Agent Setup, select Kubernetes in the list.

    Datadog - Console

    This screen shows you all the information you need to install the agent.

    Create a datadog-values.yaml file with the following content:

    datadog:
      clusterName: qovery-dx-cluster
    

      # datadog.site -- The site of the Datadog intake to send Agent data to
      ## Set to 'datadoghq.eu' to send data to the EU site.
      site: datadoghq.eu
    

      # Export custom Kubernetes labels as tags
      podLabelsAsTags:
        "qovery.com/*": "%%label%%"
    

      logs:
        enabled: true
        containerCollectAll: true
    

      # Set to false if you are not using APM.
      apm:
        enabled: true
      
      containerExcludeLogs: "kube_namespace:kube-system kube_namespace:qovery kube_namespace:cert-manager kube_namespace:nginx-ingress kube_namespace:logging kube_namespace:prometheus"
    

    # You can remove this part if you are not using APM.
    # Note that it it will be enabled for all your applications.
    clusterAgent:
      admissionController:
        enabled: true
        mutateUnlabelled: true
    • Replace <CLUSTER NAME> with a meaningful name to identify your cluster.
    • Set the site value corresponding to the one you selected upon account creation (you can find the information under 1 > With Helm V3 > --set datadog.site=<YOUR SITE VALUE>)

    There are many other values you can set. For advanced usage, check: https://github.com/Datadog/helm-charts/blob/main/charts/datadog/values.yaml

  3. Install Datadog agent in your cluster

    The Datadog agent is a program that will collect data from your cluster and forward it to Datadog. We'll install it uning Helm.

    First add the datadog Helm repository and update your local list:

    helm repo add datadog https://helm.datadoghq.com
    helm repo update

    Then we will install the agent in the datadog namespace:

    helm install datadog-agent \ 
      -n datadog --create-namespace \
      -f <PATH TO datadog-values.yaml> \
      --set datadog.apiKey='<API KEY>' \
      datadog/datadog
    • Edit the path to the datadog-values.yaml file you created
    • Replace <API KEY> with your actual API KEY. You can find it under 1 > With Helm V3 > --set datadog.apiKey=<API KEY>

  4. Check the agent is running properly

    Wait for a minute then run the following command:

    kubectl get pods -n datadog

    If the installation was successful, you should see an output similar to this one: 

    NAME                                                READY   STATUS    RESTARTS   AGE
    datadog-agent-2xhsv                                 3/3     Running   0          1m
    datadog-agent-cluster-agent-7f8bddd44-pwjnl         1/1     Running   0          1m
    datadog-agent-kube-state-metrics-577fcf6778-kc2gk   1/1     Running   0          1m
    datadog-agent-qfsl2                                 3/3     Running   0          1m
    datadog-agent-s5r5r                                 3/3     Running   0          1m

  5. Finish Setup

    Once Datadog receives your data, you should be able to click Next on the wizard. You might need to refresh the page in some cases. It can take a couple minutes before your data is ready.

    You will then arrive on Step 4

    Datadog - Console

    You can skip this part if you're not interested in monitoring your cloud account.

    Finally, restart your applications if you are using APM.

Conclusion

You now have Datadog agent running on your Qovery cluster. You can check their Getting Started guide to familiarize yourself with the product: https://docs.datadoghq.com/fr/getting_started.