Qovery provides by default an easy way to get access to your logs through the Console or the CLI. For statistics, debugging or security reasons, you may want to access all logs and perform a full-text search inside them.

Qovery implementation is based on Loki for performance and cost-effective reasons. However, Loki is not a full-text search engine. It is a log aggregation system. It is not designed to be queried directly.

Why Qovery does not provides current Loki access?

1. As mentioned Loki is not a full-text search and results may not reflect what you are looking for.
2. Loki is configured to answer usage from Qovery Console and CLI. Using it directly may impact Qovery Console and CLI performances or worst, lose logs and make it irresponsive.

Serveral solutions exists, with and without 3rd parties. We will cover here a solution without a third party. But if you're interrested, you can take a look at Datadog integration.

Note: in this tutorial, we are using Fluent-bit with proposed solutions above. However, if none of those solutions suits your needs, feel free to look at supported solution on the official website.

AWS permissions for Cloudwatch

We will create a dedicated service account (note: STS account can be used, but for simplicity reasons, we will use a dedicated service account).

On IAM create a policy with the following permissions, and name this policy fluent-bit-write-policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "CloudWatchLogs",
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutRetentionPolicy",
                "logs:PutLogEvents"
            ],
            "Resource": "arn:aws:logs:*:*:*"
        }
    ]
}

You can enforce this policy by cluster if you need, by updating the Resource content. But we want to keep it simple in this tutorial, so we will apply it to all clusters (so you can reuse the same service account if you want for other clusters).

Once done, let's create a user and attach the policy to it:

Finish the user creation and keep credentials for the coming section.

Helm

We will use AWS fluent-bit Helm Chart to setup logs streaming and deploy it with Qovery.

Add the AWS EKS helm repository

Add the AWS EKS helm repository in your Qovery settings by following this documentation

• Repository name: eks
• Kind: HTTPS
• Repository URL: https://aws.github.io/eks-charts

Create and deploy the helm chart within Qovery

Create a helm service in the Qovery environment of your choice (preferrably within a dedicated Tooling project) by following this documentation and these values:

• General:
  • Application name: fluent-bit
  • Source:
    • Helm source: Helm repository
    • Repository: eks (the name given during the AWS EKS helm repository added in the previous step)
    • Chart name: aws-for-fluent-bit
    • Version: 0.1.21 (this is the version we used for this setup, update it based on the chosen version)
    • Allow cluster-wide resources ✔️
• Values
  • Values override as file:
  • File source: Raw YAML
  • Raw YAML:

priorityClassName: system-node-critical

cloudWatch:
  enabled: true
  region: "<enter your cluster region>"
  logGroupName: "/aws/eks/fluentbit-<enter your cluster region>/logs"
  logRetentionDays: 7

env:
  - name: "AWS_ACCESS_KEY_ID"
    value: qovery.env.AWS_ACCESS_KEY
  - name: "AWS_SECRET_ACCESS_KEY"
    value: qovery.env.AWS_SECRET_ACCESS_KEY

firehose:
  enabled: false

kinesis:
  enabled: false

elasticsearch:
  enabled: false

You can take a look at additional configuration options on the AWS provided chart

Now get to the last step and just Create the service on Qovery.

Store the AWS Secrets as Qovery secrets

In the previous step we have assigned the macro qovery.env.qovery.env.AWS_ACCESS_KEY and qovery.env.AWS_SECRET_ACCESS_KEY to the AWS secrets. In this step we will create these secrets within the Qovery console.

• Open the service overview of the created Datadog service
• Enter the Variables section
• Add a new Variable with:
  • Variable = AWS_SECRET_ACCESS_KEY
  • Value = <your_SECRET_ACCESS_KEY>
  • Scope = Service (so that it is accessible only to this service)
  • Secret variable ✔️
• Add a new Variable with:
  • Variable = AWS_ACCESS_KEY
  • Value = <your_ACCESS_KEY>
  • Scope = Service (so that it is accessible only to this service)
  • Secret variable ✔️

If you need more information on how to manage your environment variables, have a look at this documentation

Deploy your chart

Open the Play button and trigger the deployment of your chart.

Cloudwatch usage

You can now use Cloudwatch to look at your logs. Connect to Cloudwatch, go into the Logs insight section, then you can perform queries:

1. Select the fluent-bit group of logs
2. Create a query (syntax examples)
3. Run your query
4. See the result and expand to filter on other elements