Workflow for DevOps: Approvals and inputs to engage your team

Workflows can gather — and automatically act on — all kinds of data, but sometimes you need a human in the loop. Here's a query that finds an IAM access key that's active beyond what our 90-day policy allows.

select
  access_key_id,
  user_name,
  account_id,
  _ctx ->> 'connection_name' as conn_name
from
  aws_iam_access_key
where
  create_date < now() - interval '90 days'
  and status = 'Active'

It's nice to be able to find this non-compliant resource, but you also want to bring it into compliance, and that often requires somebody's input. Now, with the input step, you can create a pipeline that will pause, ask for human input, and then continue based on that input.

Require approval to deactivate an expired access key

This sample workflow kicks off with a trigger that uses that same query.

trigger "query" "list_expired_iam_access_keys" {
  title       = "List Expired IAM Access Keys"
  description = "List IAM access keys older than 90 days."
  database    = var.database
  schedule    = "* * * * *" # Every minute
  primary_key = "access_key_id"

  sql = <<-EOQ
    select
      access_key_id,
      user_name,
      account_id,
      _ctx ->> 'connection_name' as conn_name
    from
      aws_iam_access_key
    where
      create_date < now() - interval '90 days'
      and status = 'Active';
  EOQ

  capture "insert" {
    pipeline = pipeline.deactivate_expired_iam_access_keys_with_approval
    args = {
      access_keys = self.inserted_rows
    }
  }
}

The capture block relays one or more expired keys to a pipeline that uses for_each to loop over one or more keys, calling a pipeline for each key. Here the input step in the per-key pipeline.

step "input" "prompt_deactivate_expired_iam_access_key" {
  notifier = notifier[param.notifier]
  subject  = "Request to deactivate expired IAM access key ${param.access_key.access_key_id} for user ${param.access_key.user_name} [${param.acce
ss_key.account_id}]"
  prompt   = "Do you want to deactivate IAM access key ${param.access_key.access_key_id} belonging to ${param.access_key.user_name} [${param.acce
ss_key.account_id}]?"
  type     = "button"

  option "Deactivate" {
    label = "Deactivate"
    value = "deactivate"
    style = "ok"
  }

  option "Active" {
    label = "Keep expired key active"
    value = "active"
    style = "alert"
  }
}

Flowpipe's input step is generic. It defines what the choices are, and how they're represented — in this case, as buttons — but the input step says nothing about how they're rendered. That's just a configuration detail.

Alternative renderings

Here's how the choices are rendered in email.

And here's how they look in Slack.

Buttons aren't the only input type, you can also use text, select, and multiselect. Another sample, which finds S3 buckets missing a required tag, then prompts with a hardcoded list of choices, uses select.

We'll circle back to the configuration that produces these effects, but first let's walk through the rest of this workflow.

Acting on the input

This step, which runs only if the person chose the Deactivate option, calls a pipeline in the AWS mod for Flowpipe to deactivate the key.

step "pipeline" "deactivate_iam_access_key" {
  if       = step.input.prompt_deactivate_expired_iam_access_key.value == "deactivate"
  pipeline = aws.pipeline.update_iam_access_key
  args = {
    cred          = param.access_key.conn_name
    user_name     = param.access_key.user_name
    access_key_id = param.access_key.access_key_id
    status        = "Inactive"
  }
}

The next step sends a confirmation message.

step "message" "notify_iam_access_key_deactivated" {
  if         = step.input.prompt_deactivate_expired_iam_access_key.value == "deactivate"
  depends_on = [step.pipeline.deactivate_iam_access_key]

  notifier = notifier[param.notifier]
  text     = "Deactivated IAM access key ${param.access_key.access_key_id} for user ${param.access_key.user_name} [${param.access_key.account_id}]"
}

The trigger's var.database could refer to any supported database — including Steamipe, Postgres, and SQLite — but since we're using the AWS plugin we'll set that variable to postgres://steampipe@localhost:9193/steampipe and start Steampipe as a service.

steampipe service start

Then we'll start the Flowpipe server, with a base URL matching a webhook we've set up in Slack.

flowpipe server --base-url=https://9f6b-157-131-95-205.ngrok-free.app --verbose

Here we can see that the query trigger found a stale key and kicked off an interaction with Slack.

The pipeline's input step will now wait for a response. When a person presses either button, control returns to the pipeline. If Deactivate was the choice, the two steps we've seen will run: perform the deactivation, and send a confirmation message. Otherwise the workflow sends a message confirming that no action was taken.

Configure an integration with a notifier for Slack

Along with the database variable, this sample mod defines a notifier variable which we'll call expired_access_key. It's referenced from a config file we've named integrations.fpc which includes an integration that defines how Flowpipe talks to an external system, in this case Slack. It also includes a notifier that binds to the integration and provides details specific to it — in this case, the channel (security) where the interaction will occur, and to which the confirmation message will be sent.

integration "slack" "default" {
  token = env("SLACK_TOKEN")
}

notifier "expired_access_key" {

  notify {
    integration = integration.slack.default
    channel     = "security"
  }

}

None of this configuration is hardwired in the sample mod. To perform the same interaction in email instead of Slack, you'd just write a different set of definitions.

integration "email" "default" {
  smtp_tls      = "required"
  smtps_port    = 587
  smtp_host     = "smtp.gmail.com"
  smtp_username = "judell@example.com"
  smtp_password = env("MY_EMAIL_PASSWORD")
  from = "judell@example.com"
}

notifier "expired_access_key" {

  notify {
    integration = integration.email.default
    to = ["security@example.com"]
  }

}

You can even include several notify blocks in a notifier in order to send a message to multiple channels!

See it in action

Build your own human-in-the-loop workflows

The integration and notifier blocks work with input and message steps to define how Flowpipe communicates and interacts with various external systems, so your pipelines can focus on what should happen, not how. It all adds up to a powerful and flexible toolkit for workflows that interact with people. Try building some interactive workflows, and let us know how it goes!