Migrating Pulumi State from Pulumi Cloud to Azure Blob Storage

Moving your Pulumi state from the cloud service to Azure Blob Storage gives you more control, lower costs, and tighter CI/CD integration. Here's everything you need to know about the migration process.

Why Migrate?

• Cost: Azure Blob Storage is significantly cheaper than Pulumi Cloud for state storage
• Data Sovereignty: Keep your infrastructure state in your own Azure subscription
• CI/CD Integration: Easier integration with Azure DevOps pipelines
• No External Dependencies: Eliminate the need for Pulumi Cloud access tokens

Prerequisites

Before starting, ensure you have:

• Azure CLI installed and logged in (az login)
• Pulumi CLI installed
• jq for JSON manipulation
• Access to both Pulumi Cloud and Azure subscription
• PULUMI_ACCESS_TOKEN for Pulumi Cloud access

Migration Overview

The migration involves three critical steps:

1. Setup Azure Blob Storage backend
2. Migrate the state file
3. Re-encrypt configuration secrets

⚠️ IMPORTANT: The migration script only migrates the state file (resources). Config secrets are encrypted with Pulumi Cloud's key and must be re-set with your new passphrase.

Step 1: Setup Azure Blob Storage

Create the required Azure resources for storing Pulumi state:

./setup-pulumi-azure-backend.sh

This creates:

• Resource Group (default: rg-pulumi-state)
• Storage Account (globally unique name)
• Blob Container (default: pulumi-state)

Save these values—you'll need them for the next steps:

• Storage Account name
• Storage Account key
• Container name

Step 2: Migrate Pulumi State

2.1 Set Environment Variables

First, retrieve your Azure Storage Account key:

# Login to Azure CLI
az login
 
# Get the storage account key
AZURE_STORAGE_KEY=$(az storage account keys list \
  --account-name <your-storage-account> \
  --resource-group <your-resource-group> \
  --query '[0].value' -o tsv)
 
# Verify the key was retrieved
echo "Storage key retrieved: ${AZURE_STORAGE_KEY:0:10}..."

Then set all required environment variables:

export AZURE_STORAGE_ACCOUNT=<your-storage-account>
export AZURE_STORAGE_KEY="$AZURE_STORAGE_KEY"
export AZURE_BLOB_CONTAINER=pulumi-state
export PULUMI_CONFIG_PASSPHRASE=<choose-a-strong-passphrase>
export PULUMI_ACCESS_TOKEN=<your-pulumi-cloud-token>

🔑 Remember the passphrase—you'll need it for all future Pulumi operations!

2.2 Run Migration Script

./migrate-pulumi-to-azure-blob.sh

When prompted, enter the full stack name from Pulumi Cloud:

Stack name: <organization>/<project>/<stack>
# Example: myorg/my-infrastructure/dev

The script will:

1. Export state from Pulumi Cloud (with decrypted secrets)
2. Create a new stack in Azure Blob Storage with passphrase encryption
3. Import the state with re-encrypted secrets
4. Save credentials to .env.pulumi

2.3 Verify Migration

After migration, verify the state was imported correctly:

# Load credentials and login
set -a && source .env.pulumi && set +a && pulumi login "azblob://$AZURE_BLOB_CONTAINER"
 
# Select stack and verify
pulumi stack select dev
pulumi stack --show-urns

Step 3: Re-encrypt Config (CRITICAL)

⚠️ This step is required! The Pulumi.{stack}.yaml file contains secrets encrypted with Pulumi Cloud's encryption. These cannot be decrypted with the new passphrase.

3.1 Check Current State

Run pulumi preview to see if there are unexpected changes:

pulumi preview

Safe updates (normal after migration):

• [diff: ~osProfile] on VirtualMachine—password re-encrypted with new passphrase
• The actual password hasn't changed, only the encryption
• Running pulumi up will update the state without modifying the VM

Unsafe updates (need to fix config):

• Resources marked for replacement, especially with [diff: ~resourceGroupName]
• This means config values don't match deployed resources
• Do NOT proceed—fix the config first

3.2 View Existing Config Values

Check what values exist in the old config:

# View backed-up config from the migration
cat .pulumi-backup-*/Pulumi.dev.yaml

Note down all values, especially:

• resourceGroupName (must match what's deployed in Azure!)
• location
• Non-secret configuration values
• Azure subscription/tenant IDs

3.3 Delete and Recreate Config

# Delete old config file
rm Pulumi.dev.yaml
 
# Set non-secret values
pulumi config set azure-native:subscriptionId "<subscription-id>"
pulumi config set azure-native:tenantId "<tenant-id>"
pulumi config set location "<your-location>"
pulumi config set resourceGroupName "<exact-resource-group-name>"  # Must match Azure!
 
# Set secret values
pulumi config set --secret <secret-key-1> "<secret-value-1>"
pulumi config set --secret <secret-key-2> "<secret-value-2>"

🎯 CRITICAL: The resourceGroupName must exactly match what's deployed in Azure. Check the Azure Portal or run az group list to find the correct name.

3.4 Verify No Unexpected Changes

pulumi preview

Expected output:

Resources:
    8 unchanged

Step 4: Update CI/CD Pipeline

4.1 Add Pipeline Variables

In Azure DevOps, add these variables:

4.2 Configure Service Principal Access

The pipeline uses service principal authentication for better security:

# Get your service principal's Object ID
SP_OBJECT_ID="<service-principal-object-id>"
 
# Get your storage account resource ID
STORAGE_ACCOUNT_ID=$(az storage account show \
  --name <your-storage-account> \
  --resource-group <your-resource-group> \
  --query id -o tsv)
 
# Assign "Storage Blob Data Contributor" role
az role assignment create \
  --role "Storage Blob Data Contributor" \
  --assignee-object-id "$SP_OBJECT_ID" \
  --assignee-principal-type ServicePrincipal \
  --scope "$STORAGE_ACCOUNT_ID"

4.3 Remove Old Variables

Remove these variables (no longer needed):

• PULUMI_ACCESS_TOKEN - Pulumi Cloud token
• PULUMI_STORAGE_KEY - Storage account key (replaced by service principal)

4.4 Pipeline Configuration

The pipeline uses the service principal from the AzureCLI task:

env:
  AZURE_STORAGE_ACCOUNT: $(pulumiStorageAccount)
  AZURE_CLIENT_ID: $(servicePrincipalId)
  AZURE_CLIENT_SECRET: $(servicePrincipalKey)
  AZURE_TENANT_ID: $(tenantId)
  PULUMI_CONFIG_PASSPHRASE: $(PULUMI_CONFIG_PASSPHRASE)

Login with:

pulumi login "azblob://$(pulumiBlobContainer)"

Local Development Workflow

Option 1: Using .env.pulumi (Storage Key)

# Load environment variables and login
set -a && source .env.pulumi && set +a && pulumi login "azblob://$AZURE_BLOB_CONTAINER"
 
# Select your stack
pulumi stack select dev
 
# Run commands as normal
pulumi preview
pulumi up

Option 2: Manually Retrieve Storage Key

# Login to Azure CLI
az login
 
# Retrieve storage key
export AZURE_STORAGE_ACCOUNT=<your-storage-account-name>
export AZURE_STORAGE_KEY=$(az storage account keys list \
  --account-name $AZURE_STORAGE_ACCOUNT \
  --resource-group rg-pulumi-state \
  --query '[0].value' -o tsv)
 
# Set other variables
export AZURE_BLOB_CONTAINER=pulumi-state
export PULUMI_CONFIG_PASSPHRASE=<your-passphrase>
 
# Login and use Pulumi
pulumi login "azblob://$AZURE_BLOB_CONTAINER"

Option 3: Using Service Principal

# Set service principal credentials
export AZURE_STORAGE_ACCOUNT=<your-storage-account-name>
export AZURE_CLIENT_ID=<your-sp-client-id>
export AZURE_CLIENT_SECRET=<your-sp-secret>
export AZURE_TENANT_ID=<your-tenant-id>
export PULUMI_CONFIG_PASSPHRASE=<your-passphrase>
 
# Login and use Pulumi
pulumi login "azblob://pulumi-state"

Troubleshooting

"7 to replace" - Resources Being Recreated

Cause: Config values don't match deployed resources (usually resourceGroupName).

Solution:

1. Run pulumi preview --diff to see exactly what's different
2. Check the actual resource names in Azure Portal
3. Update config to match: pulumi config set resourceGroupName "<correct-name>"

"unable to open bucket azblob://..." Error

Cause: Missing environment variables.

Solution:

set -a && source .env.pulumi && set +a
# Or export variables manually

"could not find access token for https://api.pulumi.com"

Cause: State file still references Pulumi Cloud secrets manager.

Solution: Re-run the migration script or manually update the state's secrets_providers section.

Secrets Cannot Be Decrypted

Cause: Trying to use old config file with new passphrase.

Solution: Delete the old Pulumi.{stack}.yaml and re-set all config values.

Authentication Methods

CI/CD Pipeline

• Uses Service Principal authentication (RBAC-based)
• More secure with better audit trails
• Requires "Storage Blob Data Contributor" role
• No storage keys needed in pipeline variables

Migration Script

• Uses Storage Account Key authentication
• Simpler for one-time migration operations
• Key retrieved automatically via Azure CLI
• Saved to .env.pulumi for local development

Local Development

• Can use either method
• .env.pulumi contains storage key by default
• Switch to service principal by setting AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, AZURE_TENANT_ID

Security Best Practices

• ✅ Never commit .env.pulumi—it contains secrets (add to .gitignore)
• ✅ Store passphrase securely—required for all operations
• ✅ Use service principal for CI/CD—better security and audit trails
• ✅ Rotate storage keys periodically if using key-based auth
• ✅ Use separate storage accounts for different environments

Clean Up (Optional)

After verifying everything works, delete the stack from Pulumi Cloud:

# Login to Pulumi Cloud
pulumi login
 
# Select and delete the old stack
pulumi stack select <org>/<project>/<stack>
pulumi stack rm <org>/<project>/<stack> --yes

Conclusion

Migrating from Pulumi Cloud to Azure Blob Storage gives you more control over your infrastructure state while reducing costs. The key is understanding that config secrets need to be re-encrypted after migration. Follow the steps carefully, especially the config re-encryption, and you'll have a smooth transition to self-hosted state management.

Remember to keep your passphrase safe and document your authentication method choice for your team!