Workload Identity in AKS with Terraform

Introduction

Most cloud providers offer a method to assign identities to the Pods running in Kubernetes, linking them to roles within the respective cloud environments.

For Amazon Web Services (AWS), this is achieved through IAM Roles for Service Accounts (IRSA), while Google Cloud Platform (GCP) uses Workload Identity. Azure, on the other hand, adopts a naming convention closer to GCP’s, known as Azure Workload Identity.

While there’s a decent tutorial available for setting this up manually, I discovered that finding the right Terraform resources can be a bit challenging. Therefore, I wanted to share my recent experience and the resources I found, particularly for those who prefer Terraform.

The AKS Cluster

The most crucial addition to a traditional AKS Terraform block is enabling the OIDC issuer and workload identity. These settings are essential as they allow the cluster to:

1. Properly issue tokens for workloads, such as your Pods.
2. Mutate your Pods to inject those tokens using a mutating webhook.

Instead of showcasing the Terraform code I utilized, I’ll contribute to the examples available in the Terraform Registry.

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~>3.0"
    }
  }
}

resource "azurerm_kubernetes_cluster" "example" {
  name                = "aks1"
  location            = azurerm_resource_group.example.location
  resource_group_name = azurerm_resource_group.example.name
  dns_prefix          = "aks1"
  default_node_pool {
    name       = "default"
    node_count = 1
    vm_size    = "Standard_D2_v2"
  }
  identity {
    type = "SystemAssigned"
  }
  tags = {
    Environment = "Production"
  }
  
  // These two are required!
  oidc_issuer_enabled       = true
  workload_identity_enabled = true
}

Depending on your terraform setup, you’ll need one of the outputs from your cluster. If you’re using modules you’ll need an outputs.tf file with the following

output "kubernetes_oidc_issuer_url" {
  value = azurerm_kubernetes_cluster.example.oidc_issuer_url
}

Once our cluster is up, we’ll be able to see azure-wi-webhook in the kube-system namespace, which will handle injecting the tokens.

Next, we’ll create our Managed Identities and permit them to be assumed within the cluster.

Managed Identities

These identities are created using the azurerm_user_assigned_identity resource, which enables us to create an identity and assign it the necessary permissions.

Let’s create an identity and grant it write access to a storage account. Additionally, we’ll allow the AKS to issue a token to this identity. We accomplish this using the azurerm_federated_identity_credential resource.

// main.tf
terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~>3.0"
    }
  }
}

locals {
  identity_name = "example_worker"
  resource_group_name = "example_resource_group"
  location = "West Europe"
  namespace = "my-namespace"
  service_account_name = "test-service-account"
}  

resource "azurerm_user_assigned_identity" "example_worker" {
  name                = local.identity_name
  resource_group_name = local.resource_group_name
  location            = local.location
}

// Allow our identity to be assumed by a Pod in the cluster
resource "azurerm_federated_identity_credential" "example_worker" {
  name                = local.identity_name
  resource_group_name = local.resource_group_name
  audience            = ["api://AzureADTokenExchange"]
  issuer              = var.kubernetes_oidc_issuer_url // Use the output from above or if in the same file
//issuer              = azurerm_kubernetes_cluster.example.oidc_issuer_url
  parent_id           = azurerm_user_assigned_identity.example_worker.id
  subject             = "system:serviceaccount:${local.namespace}:${local.service_account_name}"
}

data "azurerm_storage_account" "example" {
  name                = "example_storage_account"
  resource_group_name = local.resource_group_name
}

// Give our managed identity some permissions
resource "azurerm_role_assignment" "example_worker_blob_contributor" {
  scope = data.azurerm_storage_account.example.id

  // using azure defined role
  role_definition_name = "Storage Blob Data Contributor"

  principal_id = azurerm_user_assigned_identity.example_worker.principal_id
}

Alright, that’s quite a bit of setup! However, before delving into the Kubernetes aspect, let’s focus on what we’ll need for our Service Account and Deployment/Pod.

Firstly, we’ll require the client_id for the user-managed identity we just created. We can create another output specifically for this if we plan to install components outside of Terraform or in a separate module.

output "managed_user_client_id" {
  value = azurerm_user_assigned_identity.example_worker.client_id
}

Once you execute this Terraform configuration, you should observe “Managed Identities” appearing within the designated storage account with Blob Contributor permissions granted

Service Account and Deployment

The ServiceAccount we create that maps to this managed identity has to match up with the same Namespace and name as we had above, otherwise, we won’t properly map any tokens.

namespace = "my-namespace"
service_account_name = "test-service-account"

The appropriate ServiceAccount needs one additional value, that client_id from before, mapped to the appropriate annotation. A relevant ServiceAccount would look like so

apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    azure.workload.identity/client-id: azure-mananged-identity-client-id
  name: test-service-account
  namespace: my-namespace

Where azure-mananged-identity-client-id ( you can get this in managed identity) is a made-up client_id, but should map to the one you created with Terraform.

The last piece we need is our Deployment. It needs to

1. Use the ServiceAccount we just created
2. Also, have a special podLabel we have to add, azure.workload.identity/use: "true” , otherwise, the webhook won’t do anything (despite using the ServiceAccount… anyway…)

An example deployment might look like (if we were just going to deploy busybox )

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ubuntu
  namespace: my-namespace
spec:  
  replicas: 1
  selector:
    matchLabels:
      app: ubuntu
  template:
    metadata:
      labels:
        app: ubuntu
        azure.workload.identity/use: "true"
    spec:
      # The SA we created above
      serviceAccountName: test-service-account
      containers:
      - name: ubuntu-container
        image: ubuntu:20.04
        command: ["/bin/bash"]
        args: ["-c", "sleep infinity"]

If everything is properly configured, you’ll notice the relevant environment variables for Azure Identity Authentication, such as a client ID and a token file, displayed accordingly.

$ kubectl exec ubuntu  -- env

Thanks for reading this far!! We appreciate your comments and feedback.

About The Author
Suraj Solanki
Senior DevOps Engineer
LinkedIn: https://www.linkedin.com/in/suraj-solanki