tutorial

Use AWS IAM roles with Qovery

Give AWS IAM permissions to your application/container/job with Qovery
type: tutorialtechnology: qovery

AWS IAM (Identity & Access Management) service allows AWS services to interact with each other by using roles. Those roles can easily be used to give permissions to your Qovery application, container or job.

It is a secure way to give your application permissions without having to manage credentials. More than that, it rotates the token automatically.

This tutorial will show you how to add AWS IAM roles to your Qovery application, container or job.

Application requiring S3 permissions

In this first step, we will create a simple application that needs AWS permissions to access s3 buckets.

Create an application

We are going to will create a simple container, but you can use an existing one if you want (or an application or job).

Here is a simple Debian container example:

debian app

Set only 1 instance and 128MB of memory is enough for this example. Then continue until you have the Create button, there is nothing more to setup.

Get Kubernetes namespace name

Then in this container (or any application in this environment) Variables, search for the variable called QOVERY_KUBERNETES_NAMESPACE_NAME and copy its value somewhere.

debian app

It is the Kubernetes namespace name where the container is located.

Configure OIDC provider

Get your Cluster OIDC provider URL

On your AWS console, go to your EKS cluster and Overview section. Copy the OpenID Connect provider URL:

EKS OIDC

Create an Identity provider

On your AWS console, go to IAM service, then Identity providers section, and Add provider button:

  1. Select the OpenID Connect provider type
  2. Paste the OpenID Connect provider URL previously copied to Provider URL
  3. Click on Get thumbprint button, once done the button will change to Edit URL
  4. Add sts.amazonaws.com as Audience
  5. Click on Add provider button

OIDC Connect

Configure AWS IAM roles

Create a role

Now we can create a role. In the IAM service, go to Roles section, and click on Create role button.

You have to select the Trusted entity type. For this tutorial, we are going to use the Web identity type.

Set the Identity provider to the one you just created, and the Audience to sts.amazonaws.com. Then click on the Next button.

Role create step 1

Role permissions

Select the policy of your choice. For this example, the policy AmazonS3ReadOnlyAccess will be used to list S3 buckets. Then click on the Next button.

To finish, set the role name and description of your choice and click on Create role button.

Configure trusted entities

Qovery environment scoped role

Once created, select your freshly created role, go to the Trust relationships tab, and click on Edit trust policy button.

role trusted default

Update the policy line regarding the OIDC condition from:

"oidc.eks.eu-west-3.amazonaws.com/id/xxxxxxx:aud": "sts.amazonaws.com"

to:

"oidc.eks.eu-west-3.amazonaws.com/id/xxxxxxx:sub": "system:serviceaccount:kubernetes_namespace:service_account_name"

Replace:

  • kubernetes_namespace: with the namespace name, corresponding to the Qovery environment (previously copied in step 1)
  • service_account_name: define a service account name which will be re-use later (ex: my-s3-role)

Once done, click on the Update policy button.

Last element to copy and save somewhere: is the role ARN.

In the end, you should have something like:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::yyyyyyy:oidc-provider/oidc.eks.us-east-2.amazonaws.com/id/xxxxxxx"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringEquals": {
                    "oidc.eks.eu-west-3.amazonaws.com/id/xxxxxxx:sub": "system:serviceaccount:kubernetes_namespace:service_account_name"
                }
            }
        }
    ]
}

Cluster scoped role

If you want to be able to keep the Role and permissions with the "On-demand environment" and "Clone" features, then you have to scope the role "cluster side" instead of the "Kubernetes namespace" side.

To do so, update the Condition with StringLike instead of StringEquals, and use a wildcard instead of the namespace name:

"Condition": {
    "StringLike": {
        "oidc.eks.eu-west-3.amazonaws.com/id/xxxxxxx:sub": "system:serviceaccount:z*:service_account_name"
    }
}

Replace:

  • service_account_name: define a service account name which will be re-use later (ex: my-s3-role)
  • z*: the wildcard to use to match all namespaces deployed with Qovery

Create a service account

This step will help you on deploying a service account on your Kubernetes cluster. In case you want to do it manually on the cluster with kubectl, you just have to push a service account like:

apiVersion: v1
kind: ServiceAccount
metadata:
 name: $SERVICE_ACCOUNT_NAME
 namespace: $QOVERY_KUBERNETES_NAMESPACE_NAME
 annotations:
   eks.amazonaws.com/role-arn: $AWS_ROLE_ARN

Kubernetes authentication

On AWS, there are several ways to authenticate to Kubernetes. To make it simple, we are going to use a dedicated IAM user, but you can select the best method for your need.

From your AWS Console, create an IAM user account, get Access key ID and Secret access key and save them somewhere.

Qovery helps IAM users to get quick access to the Kubernetes cluster. Simply add this user to the Admins group.

Create a Lifecycle job

In the same environment than your application, create a Lifecycle job which will be used to deploy a service account on the Kubernetes cluster:

Lifecycle creation

Here a container qoveryrd/create-sa:1.0 available on DockerHub made by Qovery is used, but you can fork this repository and update to your needs if you prefer.

Click on the Continue button and select the Start event because we want to deploy the service account at the environment start and Delete to delete it if we decide to remove it. Set parameters as well with the according action:

Lifecycle creation

Then click on the Continue button, set the resources (128Mb is enough) and click on the Continue button.

Then add the following environment variables to the job scope:

  • KUBERNETES_VERSION: the version of your Kubernetes cluster which will be used to download kubectl (ex: 1.23.0)
  • SERVICE_ACCOUNT_NAME: the name of the service account in Kubernetes (the same name you have declared for the role in the Trusted entities policy section)
  • AWS_ROLE_ARN: the AWS ARN role you have just created
  • AWS_ACCESS_KEY_ID: the AWS access key ID of the IAM user you have created (if you decided to use this authentication method)
  • AWS_SECRET_ACCESS_KEY: the AWS secret access key of the IAM user you have created (if you decided to use this authentication method)

Lifecycle creation

Then Create the Lifecycle job. Go into the Variables tab and create a Variable Alias on QOVERY_CLOUD_PROVIDER_REGION, name it AWS_DEFAULT_REGION and scope it to the job.

Lifecycle creation

You can now run your job by clicking on the Deploy now button. You should see the following output in your job logs:

-> Ensuring required environment variables are present
-> Downloading kubectl version 1.23.0
-> Generated service account:
apiVersion: v1
kind: ServiceAccount
metadata:
 name: my-s3-role
 namespace: xxxxxx
 annotations:
   eks.amazonaws.com/role-arn: arn:aws:iam::xxxxxx:role/my-s3-role
-> Getting kubeconfig
Added new context arn:aws:eks:region:id:cluster/cluster-name to /root/.kube/config
-> Deploying service account
serviceaccount/aws-permissions created

Set application service account

Set service account

The final step is to set this service account (pointing to the AWS role) to your application. Go into your application Advanced settings and set the Service account to the one you have just created:

Lifecycle creation

Deploy your application with the Deploy now button.

At this stage, the job should have been executed and the service account should be deployed on your Kubernetes cluster, and the Debian container, running.

Validate access

To validate the AWS role has correctly been deployed, we can connect to the pod, and see if we have the AWS token. We will use the Qovery CLI to connect to our pod:

$ qovery shell
Qovery: Select organization
Organization:
✔ Qovery
Qovery: Select project
Project:
✔ AWS roles tutorial
Qovery: Select environment
Environment:
✔ aws-role
Qovery: Select service
Services:
✔ debian

Now we are connected to the pod, we can check the AWS token:

$ env | grep AWS
AWS_DEFAULT_REGION=us-east-2
AWS_REGION=us-east-2
AWS_ROLE_ARN=arn:aws:iam::xxxxxx:role/my-s3-role
AWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/token
AWS_STS_REGIONAL_ENDPOINTS=regional

Token is here! Let's install the AWS CLI and validate the role access. We should be able to list S3 buckets:

$ apt-get update && apt-get -y install awscli
$ aws s3 ls
2022-09-23 06:56:38 aws-cloudtrail-logs-qovery
...

It works! We have access to S3 buckets using the AWS role.

Conclusion

The first setup phase can be time-consuming. However, once done, applying roles to your applications is very easy and fast. You can now use roles to access any AWS service!