Many DevOps challenges follow similar patterns. So we're creating a set of off-the-shelf Flowpipe mods to address these common scenarios head-on, starting with AWS Thrifty. For example, suppose you want to detect and release unattached elastic IP addresses. Here are three basic patterns for doing that.
Wizard (with approval): Run Flowpipe on the command line, detect unattached EIPs, act on the choices
Skip
orRelease
.Scheduled (with approval): Run on a schedule, detect, act on the choices
Skip
orRelease
.Scheduled (no approval): Run on a schedule, detect and automatically
Release
.
You'll write no code to do any of those things, instead you'll configure mod variables such as:
The database (Steampipe or other) that feeds the pipeline with the output of a SQL query.
A notifier that binds the mod to a communication channel — an integration — that might be email, Slack, or MS Teams.
One more
approvers
, which are notifiers that obtain action/approval decisions.Whether to run on a trigger.
The default action, e.g.
notify
orrelease
.
Run as a wizard, with Slack-based approval
To run this pipeline using Slack to receive notifications and take actions, install the mod, start Steampipe (steampipe service start
), configure a Slack integration and notifier, and start the Flowpipe server.
Now run this command.
flowpipe pipeline run detect_and_correct_vpc_eips_unattached \--arg approvers='["slack"]' \--arg host=local
In this case the pipeline detects one unattached EIP, and waits for a decision to be made in Slack channel.
If we choose Release
, the pipeline calls a utility pipeline in the AWS library mod to release the EIP, and the Slack message changes to confirm the action.
If there are more than one unattached EIPs, these Slack interactions will occur serially. To batch them, you can switch the max_concurrency
from the default, which is 1, to your desired batch size.
Here max_concurrency
is 3; we've chosen to release the first two but take no action on the third.
Detect on a schedule, respond with human input
To use this pattern, activate a trigger to run the same query for unattached EIPs, but on a schedule instead of interactively. Because this requires setting a few variables, we'll package them into a file named for this scenario: scheduled-interactive.fpvars
.
notifier="slack"approvers=["slack"]vpc_eips_unattached_trigger_enabled=truevpc_eips_unattached_trigger_schedule="* * * * *"
And then run the server, pointing to the .fpvars
file and the server's public endpoint.
flowpipe server --var-file scheduled-interactive.fpvars --base-url=https://feline-just-gorilla.ngrok-free.app/
Note that if the trigger's query finds an unattached EIP that you choose to Skip
, it won't be detected again because Flowpipe's query trigger remembers query results it has already seen. But if you change your mind and want to release an EIP that you skipped, you can use the wizard mode to do that.
Detect on a schedule, respond automatically
To use this pattern, we'll use a slightly different package of variables: scheduled-automatic.fpvars
.
notifier="slack"approvers=[]vpc_eips_unattached_default_action="release"vpc_eips_unattached_trigger_enabled=truevpc_eips_unattached_trigger_schedule="* * * * *"
When approvers
is the empty list, Flowpipe automically use the default action. Normally that's notify
but here we override to release
. Now, unattached EIPs detected on each run of the pipeline will be automatically released. It's as if there's a robot clicking the Release
button for each detected EIP.
Configuration-driven workflow
A detect-and-corect mod like this one doesn't require you to write any SQL or HCL. It's a tool that you configure and then use to perform a range of tasks you'd like to automate, with or without approval. Give flowpipe-mod-aws-thrifty
a try, and please let us know how it goes.