Terraform Azure DevOps YAML Pipeline 0_o Part I

If you happen to be using terraform to manage your Azure infrastructure, and your are looking to start using the newer (at the time of writing) Azure DevOps YAML pipelines to create continuous integration/continuous deployment pipeline(s), then you are in luck!  

This post assumes some knowledge of terraform and Azure DevOps.  The same ideas can be extended to your favorite build system.

Terraform is a powerful Infrastructure as Code (IaC) automation tool from Hashicorp.  It provides a powerful state system that allows you to work and version your infrastructure collaboratively across a team or teams. Terraform is also a pleasure to work with compared to ARM templates in my opinion.  The main thing I personally like about it is the confidence it provides in actually creating and modifying your infrastructure.

YAML build pipelines are a declarative way to define your build definitions.  What makes YAML builds handy for use in DevOps is that they live in the code base, next to the code, rather than live completely in Azure DevOps.  This means it’s shared _code_ that can now be versioned right along side the rest of your code. .

Setting it up


  • Azure DevOps Repo
  • Azure DevOps service connection to azure or a service principal in azure
  • Azure storage account for remote terraform state
  • Yaml build pipelines

The basic idea is as follows:

  • Create Yaml pipeline to execute terraform plan on trigger (branch commit, timer, etc)
    • Your build trigger can either be defined in the yaml or on the build definition itself.
    • Add our plan output to the build summary so that we can review
  • Upon successful plan, we store the .tfplan as an artifact (more on this later)
  • Create release via the build
  • Execute our .tfplan we stored as an artifact during the build

With automation of infrastructure comes a certain amount of risk, in the early stages of automating an infrastructure pipeline, to exercise caution, I find it handy to turn on approvals for the release.  This way, you can get the release created from the artifact(s), and validate that all is well, then run your approvals and validate your infrastructure.

To Start, we will need an Azure DevOps service connection to your target azure subscription, or a service principal with the proper rights to your target Azure subscription.   Have your Azure DevOps admin create one for you if you do not already have one.

In an Azure DevOps repo, create a file called azure-pipelines.yml. Add the following YAML snippet, modifying the azureSubscription to identify the service connection and the name to whatever you like. We then call a script called plan.sh:

name: terraform-yaml-pipline example build
    - master
    - develop    
  vmImage: 'Ubuntu 16.04'

- task: AzureCLI@1
    azureSubscription: 'kevin-cloud'
    scriptPath: plan.sh
    addSpnToEnvironment: true
    ARM_STORAGE_ACCOUNT_NAME: "kevindemostorage"
    ARM_STORAGE_CONTAINER: "terraform-state"
    ARM_STORAGE_KEY: "dev.tfstate"

Pay special attention to this line in the AzCli task:

Here we are using the ADO AzureCli task to execute plan.sh,; a script we will create to house terraform commands, and set variables.

#!/usr/bin/env bash
export ARM_SUBSCRIPTION_ID=$(az account show --query="id" -o tsv)
export ARM_CLIENT_ID="${servicePrincipalId}"
export ARM_CLIENT_SECRET="${servicePrincipalKey}"
export ARM_TENANT_ID=$(az account show --query="tenantId" -o tsv)

export ARM_ACCESS_KEY=$(az storage account keys list -n ${ARM_STORAGE_ACCOUNT_NAME} --query="[0].value" -o tsv)

terraform init -backend-config="storage_account_name=$ARM_STORAGE_ACCOUNT_NAME" \
    -backend-config="container_name=$ARM_STORAGE_CONTAINER" \

terraform plan -out=demo.tfplan
addSpnToEnvironment: true

If you want to use a service principal other than the service connection’s, then we need to wire it up here. The four environment variables required for terraform (ARM_SUBSCRIPTION_ID, ARM_CLIENT_ID, ARM_CLIENT_SECRET, ARM_TENANT_ID) are documented here.

For us to collaborate with others, terraform needs to have a shared state.  In this example we use an storage account. You can create this storage account however you want (preferably using terraform 😉 ) But to keep this post shorter, let’s just assume we have a storage account. We set these as env vars on the azure cli task, and use them in the script for terraform init, and terraform plan.

An important bit to note is here:

export ARM_ACCESS_KEY=$(az storage account keys list -n ${ARM_STORAGE_ACCOUNT_NAME} --query="[0].value" -o tsv)

We set ARM_ACCESS_KEY using the az cli. This is required for the terraform azure remote state backend which we will define later. By leveraging the already authenticated cli (from the AzCli task) to set the env variable, we have one less environment variable/password for us to worry about passing around and needing set. This is also nice if the storage account keys get rotated every so often…your build won’t break!

In another file, main.tf, we will configure our remote state backend, specify that we will be using the azurerm provider, and create a simple resource, a basic azure resource group.:

terraform {
  backend "azurerm" {}

provider "azurerm" {
  version = "~>1.21"

resource "azurerm_resource_group" "rg_demo" {
    name = "rg-kevin-vsts"

We specify azurerm for our backend, and we specify azurerm as a provider so that we can use azurerm_resource_group create an azure resource.

Commit these files and push it to your Azure DevOps repo.

yaml pipeline

After committing these two files, we notice that AzDO has created a build for us. In my case, it created the build name with the convention of “<reponame> CI” (space between reponame and CI). We can also edit the YAML from here. If you have an existing build that you are wiring to a yaml pipeline, go ahead and set the build up now to use the yaml file in your repo.

Hopefully, if all went well, we go to the build summary and we should see something like this:

build summary

If we click on the AzureCLI summary, we will see something like this:

terraform plan output

Now that we have our demo.tfplan, we want to publish this artifact so that we can use it in a release. We will learn how to do the release half of this pipeline in part II where we tie it together!