Control AWS costs with 29 workflows for FinOps

Many DevOps challenges follow similar patterns. So we're creating a set of off-the-shelf Flowpipe mods to address these common scenarios head-on, starting with AWS Thrifty. For example, suppose you want to detect and release unattached elastic IP addresses. Here are three basic patterns for doing that.

• Wizard (with approval): Run Flowpipe on the command line, detect unattached EIPs, act on the choices Skip or Release.

• Scheduled (with approval): Run on a schedule, detect, act on the choices Skip or Release.

• Scheduled (no approval): Run on a schedule, detect and automatically Release.

You'll write no code to do any of those things, instead you'll configure mod variables such as:

• The database (Steampipe or other) that feeds the pipeline with the output of a SQL query.

• A notifier that binds the mod to a communication channel — an integration — that might be email, Slack, or MS Teams.

• One more approvers, which are notifiers that obtain action/approval decisions.

• Whether to run on a trigger.

• The default action, e.g. notify or release.

Run as a wizard, with Slack-based approval

To run this pipeline using Slack to receive notifications and take actions, install the mod, start Steampipe (steampipe service start), configure a Slack integration and notifier, and start the Flowpipe server.

Now run this command.

flowpipe pipeline run detect_and_correct_vpc_eips_unattached \
  --arg approvers='["slack"]' \
  --arg host=local

In this case the pipeline detects one unattached EIP, and waits for a decision to be made in Slack channel.

If we choose Release, the pipeline calls a utility pipeline in the AWS library mod to release the EIP, and the Slack message changes to confirm the action.

If there are more than one unattached EIPs, these Slack interactions will occur serially. To batch them, you can switch the max_concurrency from the default, which is 1, to your desired batch size.

Here max_concurrency is 3; we've chosen to release the first two but take no action on the third.

Detect on a schedule, respond with human input

To use this pattern, activate a trigger to run the same query for unattached EIPs, but on a schedule instead of interactively. Because this requires setting a few variables, we'll package them into a file named for this scenario: scheduled-interactive.fpvars.

notifier="slack"
approvers=["slack"]
vpc_eips_unattached_trigger_enabled=true
vpc_eips_unattached_trigger_schedule="* * * * *"

And then run the server, pointing to the .fpvars file and the server's public endpoint.

flowpipe server --var-file scheduled-interactive.fpvars --base-url=https://feline-just-gorilla.ngrok-free.app/

Note that if the trigger's query finds an unattached EIP that you choose to Skip, it won't be detected again because Flowpipe's query trigger remembers query results it has already seen. But if you change your mind and want to release an EIP that you skipped, you can use the wizard mode to do that.

Detect on a schedule, respond automatically

To use this pattern, we'll use a slightly different package of variables: scheduled-automatic.fpvars.

notifier="slack"
approvers=[]
vpc_eips_unattached_default_action="release" 
vpc_eips_unattached_trigger_enabled=true
vpc_eips_unattached_trigger_schedule="* * * * *"

When approvers is the empty list, Flowpipe automically use the default action. Normally that's notify but here we override to release. Now, unattached EIPs detected on each run of the pipeline will be automatically released. It's as if there's a robot clicking the Release button for each detected EIP.

Configuration-driven workflow

A detect-and-corect mod like this one doesn't require you to write any SQL or HCL. It's a tool that you configure and then use to perform a range of tasks you'd like to automate, with or without approval. Give flowpipe-mod-aws-thrifty a try, and please let us know how it goes.

See it in action