Define and enforce tagging standards in Azure

In the world of cloud infrastructure management, consistent and accurate resource tagging is crucial. It's nice to be able to detect when tags deviate from your standards, but even better to correct those deviations automatically. Enter the Azure Tags mod for Flowpipe, a new tool that enables you to automate tag management using a declarative ruleset. Let's take a look.

Add mandatory tags

Here's a Steampipe query that reveals incorrectly-tagged Azure storage accounts.

select name, tags from azure_storage_account where name ~ 'flowpipe'
+------------------+------------------+
| name             | tags             |
+------------------+------------------+
| flowpipestorage1 | {"cc":"default"} |
| flowpipestorage2 | {}               |
+------------------+------------------+

First, we'd like to apply the mandatory tag env:dev to all of them. To do that, we'll use the detect_and_correct_storage_accounts_with_incorrect_tags pipeline in the Azure Tags mod.

Per the README, the setup entails:

• Cloning the mod

• Visiting its directory and installing dependencies (flowpipe mod install)

• Starting Steampipe as a service (steampipe service start)

• Using credential_import to give Flowpipe access to the credentials in Steampipe azure.spc

• Configuring tag rules

We'll use these definitions in add-notify.fpvars.

approvers = []
base_tag_rules = {
  add = {
    env = "dev"
  }
}

With approvers = [], the mod will report tag violations to the default notifier, the terminal, but take no action. The only rule is to add our desired mandatory tag.

Now we'll run the pipeline to detect and correct tags on storage accounts, and point it at that package of variables.

$ flowpipe pipeline run detect_and_correct_storage_accounts_with_incorrect_tags --var-file=add-notify.fpvars

Flowpipe reports the changes it would make.

Detected flowpipestorage1 (288007e6-582c-478d-97cf-c27eea910d65/eastus) with incorrect tags. Existing tags: cc=default New tags: cc=default, env=dev

Detected flowpipestorage2 (288007e6-582c-478d-97cf-c27eea910d65/eastus) with incorrect tags. New tags: env=dev

To make the changes, we'll use add-wizard.fpvars, with the same tag rules but with approvers = ["default"]. That activates an input step that will (in this case) prompt to Skip or Apply the proposed change.

Flowpipe pauses on the first detected violation.

$ flowpipe pipeline run detect_and_correct_storage_accounts_with_incorrect_tags --var-file=add-wizard.fpvars
┃ Detected flowpipestorage1 (288007e6-582c-478d-97cf-c27eea910d65/eastus) with incorrect tags. Tags that will be added or updated: env=dev.
┃ > Skip
┃   Apply

We'll down-arrow and choose Apply. Flowpipe reports:

Applied changes to tags on flowpipestorage1.

Because we specified the default approver, this interaction happens in the terminal. You could shift the interaction to Slack, MSTeams, or email by naming — in approvers — one or more notifiers configured with alternate integrations.

We'll choose Apply for the second storage account and recheck the situation.

+--------------------+------------------------------------------+
| name               | tags                                     |
+--------------------+------------------------------------------+
| flowpipestorage1   | {"cc":"default","env":"dev"}             |
| flowpipestorage2   | {"env":"dev"}                            |
+--------------------+------------------------------------------+

That's a good start, but we now want to correct cc, that's a bogus tag that some people use for cost_center. Let's fix that.

Update tag keys

We'll use this ruleset in update-fix.fpvars.

approvers = []

base_tag_rules = {
  update_keys = {
    cost_center = ["~*:^cc|cost_centre$"]
  }
}

incorrect_tags_default_action = "apply"

With approvers = [] the rules run without pausing for input.

The update_keys rule uses pattern matching to converge a set of bogus tags (cc or cost_centre) onto cost_center.

The default incorrect_tags_default_action is notify, which would be a dry run, but we want to apply the fix so we set it to apply.

We run the pipeline again, with this new package of variables.

$ flowpipe pipeline run detect_and_correct_storage_accounts_with_incorrect_tags --var-file=update-fix.fpvars

Flowpipe reports:

Applied changes to tags on flowpipestorage1.

And here's the new situation.

+--------------------+---------------------------------------+
| name               | tags                                  |
+--------------------+---------------------------------------+
| flowpipestorage1   | {"cost_center":"default","env":"dev"} |
| flowpipestorage2   | {"env":"dev"}                         |
+--------------------+---------------------------------------+

Tag operations

We've seen add and update_keys, here's the full set of operations documented in the README.

• Add. Ensure resources have mandatory tags.

• Remove. Remove specified tags.

• Remove Except. Inverse of remove: remove all but specified tags.

• Update Keys. Rewrite tag keys.

• Update Values. Rewrite tag values.

To match tags you can use literal strings, or Postgres regular expressions.

Automate the fixes

It would be tedious to apply these kinds of changes with one-at-a-time approval. Let's automate with a new ruleset in continuous-compliance.fpvars.

approvers = []

base_tag_rules = {
  update_keys = {
    cost_center = ["~*:^cc|cost_centre$"]
  }
  add = {
    env = "dev"
  }
}

storage_accounts_with_incorrect_tags_trigger_enabled   = true
storage_accounts_with_incorrect_tags_trigger_schedule  = "daily"
incorrect_tags_default_action                          = "apply"

Note that you can combine operations like add and update_keys to build up a complex ruleset.

Here we're also using config variables to active the pipeline's query trigger, define its schedule, and define the default action (apply). With these settings, Flowpipe will run the pipeline daily and apply the changes we've seen to new storage accounts.

To run the pipeline on a schedule, use Flowpipe in server mode.

flowpipe server --var-file=continuous-compliance.fpvars

Nothing will happen the first time because we've already fixed the storage accounts. But if we create flowpipestorage3 with no tags, it will — within a day — automagically be tagged with env:dev.

Continuous compliance for tags

The Azure Tags mod for Flowpipe enables flexible and powerful management of Azure resource tags at scale. With declarative rulesets and automated workflows, you can ensure consistent tagging across your Azure infrastructure with or without manual intervention. Whether you're looking to standardize existing tags, enforce new tagging policies, or maintain ongoing compliance, this mod provides the tools you need to streamline your tagging discipline. Give it a try and please let us know how it goes.

See it in action