Managing Azure Resources using Spacelift

In this post I want to show how easy it is to create a Spacelift Stack that manages Azure resources. To keep things simple, I’m going to assume that you already know what Spacelift can do, and the basics of creating and running Stacks, but if not check out our terraform-starter repo that walks you through the process of setting up an account, and getting started.

Overview

In order to allow Spacelift to manage Azure resources, we need to do the following:

• Create a Service Principal in Azure with the correct ARM permissions to manage our resources.
• Create a Stack in Spacelift using the Azure provider.
• Give our Stack the credentials for our Service Principal.

All of the information you need to configure the Azure provider can be found here. This post just explains how to combine that information with Spacelift.

Creating our Service Principal

To create our Service Principal, we need to use the az ad sp create-for-rbac command (replacing <subscription-id> with your actual subscription ID):

az ad sp create-for-rbac --name spacelift-sp --role="Contributor" --scopes="/subscriptions/<subscription-id>"

This will output something like the following:

Changing "spacelift-sp" to a valid URI of "http://spacelift-sp", which is the required format used for service principal names
Creating 'Contributor' role assignment under scope '/subscriptions/<subscription-id>'
The output includes credentials that you must protect. Be sure that you do not include these credentials in your code or check the credentials into your source control. For more information, see https://aka.ms/azadsp-cli
{
  "appId": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa",
  "displayName": "spacelift-sp",
  "name": "http://spacelift-sp",
  "password": "A4xFo3z-Hv9BMOFlmHfXP2ESXugHMdaike",
  "tenant": "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb"
}

The command creates a new Service Principal called spacelift-sp, and grants it the Contributor role on your subscription. It also outputs the appId, password and tenant for the Service Principal. Make a note of these because you’ll need them later.

If you would rather assign permissions yourself later, you can run the following command to just create a Service Principal with no permissions:

az ad sp create-for-rbac --name spacelift-sp --skip-assignment

You should now be able to view your Service Principal in the Azure Active Directory -> App registrations section of the Azure Portal:

Creating a Stack

Now that we’ve got a service principal, we need to create a Stack to actually create some Azure resources. To keep things simple all we’re going to do is add some terraform definitions that specify the version of the Azure Provider we want to use, and create an Azure Resource Group:

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "=2.46.0"
    }
  }
}

# Configure the Microsoft Azure Provider
provider "azurerm" {
  features {}
}

resource "azurerm_resource_group" "test-group" {
    name = "test-group"
    location = "West Europe"
}

If you commit that change and try to run your Stack just now, you should see something similar to the following failure:

Here’s the full error message:

Error building AzureRM Client: obtain subscription() from Azure CLI: Error parsing json result from the Azure CLI: Error launching Azure CLI: exec: “az”: executable file not found in $PATH

At first glance this error is slightly confusing, but what it means is that because we didn’t provide any credentials to the Azure provider it attempted to fallback to using the az CLI for authentication. This works fine locally, but won’t work on Spacelift.

To solve this, we need to provide our stack with our credentials, which is explained in the next section.

Passing Credentials to the Stack

If you take a look at the Terraform documentation, you’ll find that there are a number of different ways that you can authenticate Terraform with Azure. What we’re going to do here is authenticate using a Service Principal with a Client Secret, and provide that configuration via environment variables.

To do this we need to set the following environment variables, using the information returned by the Azure CLI when we created our Service Principal:

• ARM_CLIENT_ID - the appId from the response.
• ARM_CLIENT_SECRET - the password from the response.
• ARM_SUBSCRIPTION_ID - your subscription ID.
• ARM_TENANT_ID - the tenant from the response.

You can set these environment variables directly on your Stack, but what we’re going to do here is create a Spacelift Context and attach it to our Stack. This allows us to share the same credentials with multiple Stacks if we want.

Add a new Context via the Contexts screen in Spacelift:

Once your context has been created, edit it and enter your environment variables, making sure to mark the ARM_CLIENT_SECRET as a secret rather than plaintext:

Now that we’ve got a Context containing our credentials, you can attach it to your Stack. To do this, click on the Manage link next to your Stack, go to Contexts, choose your Context, and click Attach:

Now that your Context is attached, you should be able to see the it in the Environment section of your Stack:

If you try to run your Azure Stack again you should get a successful plan:

And applying should now succeed:

If you take a look in the Azure Portal, you should now be able to see the Resource Group that was created:

Credential Expiry

Before finishing up, I wanted to just point out a few things about the Service Principal credential expiry. Although this isn’t really related to Spacelift, it’s something that you need to be aware of.

When you create your Service Principal via the Azure CLI, it automatically creates a Client Secret for you with an expiry of 1 year:

What this means is that if you do absolutely nothing, your Spacelift Stacks will stop working after the credentials expire a year later. To solve this, all you need to do is generate a new Client Secret, and update the ARM_CLIENT_SECRET environment variable in your Context.

Wrapping Up

Hopefully in this post I’ve shown that although Spacelift doesn’t have an official Azure integration (yet!), it’s simple to manage Azure resources using Spacelift. Once you’ve performed a little bit of initial setup, you should be able to rapidly get new Stacks up and running creating Azure resources.