Changelog

Subscribe to Flowpipe changelog via RSS or join #changelog on our Slack community to stay updated on everything we ship.

Changelog image

Flowpipe CLI v0.5.0 - Mod `pull` strategy, `variable` list and show commands

cli

What's new?

  • Add support for installing mods from a branch or from the local file system. (#849).

    To install from a branch:

    flowpipe mod install github.com/turbot/flowpipe-mod-aws-thrifty#main

    To reference a mod in the local file system:

    flowpipe mod install ../mods/local_mod_folder
  • Add --pull flag to mod command to control the mod update strategy. (#849). Possible update strategies are:

    • full - check branch and tags for both latest and accuracy
    • latest - update everything to latest, but only branches - not tags - are commit checked (which is the same as latest)
    • development - update branches and broken constraints to latest, leave satisfied constraints unchanged
    • minimal - only update broken constraints, do not check branches for new commits
  • Variable list and show commands. (#373)

Bug fixes

  • Pipeline references declared in subsequent files are correctly identified and processed.
  • Preserves pipeline params ordering as specified in the pipeline definition. (#408)

Flowpipe AWS Mod v0.3.0 – Added delete_dynamodb_table pipeline

mod

What's new?

  • Added the following new pipeline:
    • delete_dynamodb_table

Flowpipe AWS Thrifty Mod v0.2.0 – Added detect and correct pipeline for DynamoDB tables with stale data

mod

What's new?

  • Added Detect and Correct pipeline for DynamoDB tables with stale data. (#34)

Flowpipe CLI v0.4.6 - `locals` loading bug fix

cli

Bug fixes

  • Load locals in order of dependency. (#399).

Flowpipe AWS Thrifty Mod v0.1.0 – Initial mod release

mod

What's new?

  • Added 30 new 'detect and correct' pipelines to identify unused and underutilized AWS resources, as well as deprecated resource configurations. These pipelines also suggest potential remediation actions to optimize costs. For usage information and a full list of pipelines, please see AWS Thrifty Mod.

Flowpipe CLI v0.4.5 - Stalled pipeline execution and trigger's common attributes fixes

cli

Bug fixes

  • Pipeline execution no longer stalls when concurrency limit is applied and if clause returns false. (#836).
  • Trigger's common attributes (title, description, tags, documentation) allow functions and expresions. (#394).

Flowpipe CLI v0.4.4 - Query step args attribute, file watcher, duplicate step names and invalid notifier reference fixes

cli

Bug fixes

  • Param can be used in query step's args attribute. (#830).
  • File watcher now correctly detect changes in the loop block. (#808).
  • Duplicate step names are now detected and reported as an error. (#820).
  • Better error message for invalid notifier reference. (#826).

Flowpipe CLI v0.4.3 - Lazy create `flowpipe.db`, `max_concurrency` and HTTP integration output fixes

cli

Bug fixes

  • Lazy create flowpipe.db. (#808).
  • Respect max_concurrency in pipeline and input steps. (#815).
  • Misleading error message for invalid step dependencies. (#816).
  • HTTP integration address is shown correctly at the beginning of each input step loop. (#818).

Flowpipe CLI v0.4.2 - `loop`, `max_concurrency` and error handling fixes

cli

Bug fixes

  • loop block now works in container, function, message and input steps.
  • Use HCL expressions in max_concurrency step argument. (#800).
  • throw, retry and error block now works for input step.

Flowpipe CLI v0.4.1 - Erroneous error message, `max_concurrency`, `try()` function, and URL bug fixes

cli

Bug fixes

  • Input step respects the max_concurrency argument. (#798).
  • Erroneous error message detecting a missing credential where there isn't one.
  • HCL try() function should be evaluated at runtime rather than parse time.
  • Integration and input step URLs should use the provided custom host & port. (#792).
  • Shows filename and line number for invalid step references.

Flowpipe CLI v0.4.0 - Microsoft Teams integration and CLI argument bug fix

cli

What's new?

Bug fixes

  • Function step output attribute should be called response not result. (#789).
  • Pipeline execution should not fail when a string argument is passed with double quotes. (#791).

Flowpipe CLI v0.3.2 - Multi-select preselected Slack option and mod reload fixes

cli

Bug fixes

  • Multiselect Inputs with preselected Options now correctly pre-populate in Slack.
  • Change detection in throw and output block in pipeline steps works correctly with ternary operators and will not trigger mod reload for white space changes.

Flowpipe CLI v0.3.1 - Multi-select option and mod reload fixes

cli

Bug fixes

  • Multi-select option in input step now works. (#776).
  • Input step white space changes will not trigger mod reload. (#297).

Flowpipe CLI v0.3.0 – Human workflow, Slack and email messaging, Import Steampipe credentials, Concurrency controls.

cli

What's new?

  • Workflow - message step for easy notifications. Documentation.
  • Workflow - input step for buttons, text and other data. Documentation.
  • Workflow - simple, reusable integration and notifier configuration for HTTP, Slack and Email. Documentation
  • Import Steampipe connections as Flowpipe credentials. Documentation.
  • Manage concurrency of pipelines and steps.
  • New credential types: alicloud and mastodon.
  • Shorter hash for HTTP triggers for simpler URLs.
  • DuckDB support in query step & trigger.
  • Step metadata, like started_at and finished_at added under a flowpipe attribute.
  • Moved flowpipe.db into the mod-level .flowpipe directory.
  • connection_string in query step and trigger renamed to database.

Deprecation

Bug fixes

  • log_level workspace setting is now respected (#618).
  • Default listen flag should be network, not localhost (#694)
  • Trigger attributes are now validated (#225).
  • Pipeline output attributes are now validated (#239).
  • Pipeline param default value data type is now validated against the specified type (#262).
  • Removed titles when merging multiple error messages (#263).
  • Runtime resolution of pipeline reference and credentials are now working correctly. (#732).
  • Scheduled triggers are now re-scheduled when mod files have changed.
  • File watcher reliability improvements.

Flowpipe Zendesk Mod v0.1.2 – Fixed metadata param type in create_ticket pipeline

mod

Bug fixes

  • Updated metadata param type in create_ticket pipeline to be consistent with similar param types.

Flowpipe Vault Mod v0.1.1 – Fixed secret param type in create_secret pipeline

mod

Bug fixes

  • Fixed secret param type in create_secret pipeline.

Flowpipe Samples v0.4.1 - Fixed broken links in new sample READMEs

mod

Bug fixes

  • Fixed broken links to credential import docs in various sample READMEs.

Flowpipe Samples v0.4.0 - Added new samples highlighting query trigger, input and message steps

mod

What's new?

  • Added the following new sample mods: (#108)
    • add_s3_bucket_cost_center_tags
    • aws_iam_access_key_events_notifier_with_multiple_pipelines
    • aws_iam_access_key_events_notifier_with_single_pipeline
    • deactivate_expired_aws_iam_access_keys_using_queries
    • deactivate_expired_aws_iam_access_keys_with_approval
    • notify_new_aws_iam_access_keys

Enhancements

  • Updated all AWS, Azure, PagerDuty, Slack, Zendesk library mod dependency versions in several sample mods. (#108)

Flowpipe Zendesk Mod v0.1.1 – Removed duplicate ticket_id param from update_ticket_comment pipeline

mod

Bug fixes

  • Removed duplicate ticket_id param from update_ticket_comment pipeline.

Flowpipe PagerDuty Mod v0.1.1 – Fixed invalid type declaration for input parameter in create_user pipeline

mod

Bug fixes

  • Fixed invalid type for license param in create_user pipeline. (#6)

Flowpipe Azure Mod v0.1.1 – Fixed several param types in Compute VM test pipelines

mod

Bug fixes

  • Fixed mismatched types for generate_ssh_keys param in various Compute VM test pipelines.

Flowpipe Slack Mod v0.2.1 – Fixed the type mismatch of the input parameter in the get_channel_history pipeline

mod

Bug fixes

  • Fixed the type mismatch of the input parameter in the get_channel_history pipeline. (#20)

Flowpipe CLI v0.2.3 – Podman support, respect Docker env vars, complete all step retries before failing

cli

Bug fixes

  • Only trigger pipeline failure after a step has completed all retries (#630).
  • DOCKER_HOST, DOCKER_API_VERSION, DOCKER_CERT_PATH, DOCKER_TLS_VERIFY environment variables are now correctly passed to the Docker client (#651).
  • Do not set memory_swappiness when using Podman (#652).

Flowpipe GitHub Mod v0.2.0 – Added create_branch, delete_branch and get_branch pipelines

mod

What's new?

  • Added create_branch, delete_branch and get_branch pipelines. (#10)

Flowpipe Slack Mod v0.2.0 – Added pipeline get_channel_id

mod

What's new?

  • Added pipeline get_channel_id. (#17).

Bug fixes

  • Fixed the input parameters of the test_post_message pipeline. (#17)

Flowpipe CLI v0.2.2 – Container and function steps build error and complex data type handling

cli

Bug fixes

  • Build error no longer suppressed in container and function steps (#625).
  • Handles complex data types in step output (#626).

Flowpipe GCP Mod v0.2.0 – Fix commands in add_labels_to_compute_disk and add_labels_to_compute_instance pipelines

mod

Bug fixes

  • Fix the commands in add_labels_to_compute_disk and add_labels_to_compute_instance pipelines. (#7)

Flowpipe CLI v0.2.1 – Handle null values in query trigger

cli

Bug fixes

  • Map MySQL query results to correct types (#604).
  • Handle null values in query trigger results (#611).
  • Convert binary data in query results to a string.
  • Docker containers now clear the cache to get correct parameters (#561).
  • Improved error message when Flowpipe CLI port is already in use (#603).

Flowpipe CLI v0.2.0 – Database query trigger, GET & POST methods for HTTP trigger, set a timeout for steps

cli

What's new?

  • Query trigger type to watch & event on database changes. Documentation.
  • HTTP trigger can now handle both GET and POST methods. Documentation.
  • Query steps & triggers now support Postgres, MySQL, SQLite, and Postgres.
  • Define container step using a source argument for inline image definitions.
  • Add a timeout to pipeline steps.
  • Enable or disable triggers using the enabled attribute.
  • Improved and expanded output for flowpipe server.
  • Improved and standardized output for CLI list and show commands.
  • Expanded intervals available in schedule and query triggers (e.g. 5m, 10m, etc).
  • New credential types: BitBucket, Datadog, Freshdesk, JumpCloud, ServiceNow, Turbot Guardrails.
  • Automatic check & notify for new CLI versions.

Bug fixes

  • Implemented a more descriptive error message for server startup failures.
  • Fixed Step Arguments unable to be referenced in the Pipeline definition.
  • Added missing execution_mode argument to HTTP Trigger (#533).
  • Fixed args arguments unable to be updated in the Pipeline Step loop block (#559).
  • Fixed an issue in the bootstrap process for identifying the config path.

Flowpipe Samples v0.3.0 - Added query_and_stop_aws_ec2_instances_by_tag sample mod and fixed bugs in README

mod

What's new?

  • Added the query_and_stop_aws_ec2_instances_by_tag sample mod that can be used with Flowpipe. (#99)

Bug fixes

  • Fixed the README docs to use --arg instead of --pipeline-arg as the argument flag. (#102)
  • Fixed the link for installing mod dependencies in all the README docs. (#98)

Flowpipe IP2Locationio Mod v0.2 – Updated credential section of README to use api_key instead of token

mod

Bug fixes

  • Updated the credential section of README to use api_key instead of token. (#7)

Flowpipe CLI v0.1.1 – Fixed inaccurate SQL query string validation

cli

Bug fixes

  • Removed inaccurate SQL Query string validation to check for arguments. (#516)

Flowpipe Samples v0.1.0 - 35 new mods

mod

35 new, ready-to-use Flowpipe sample mods are now available! These mods serve as practical examples, showcasing the patterns and applications of various library mods. Every mod comes with specific instructions for installation and use, enabling fast and easy setup.

A full list of sample mods can be found in the Flowpipe Hub and the source code is available at turbot/flowpipe-samples.

Flowpipe CLI v0.1 – Initial release

cli

Introducing Flowpipe, a cloud scripting engine. Automation and workflow to connect your clouds to the people, systems and data that matter. Pipelines for DevOps written in HCL.

Initial support for:

  • Pipeline execution
  • Steps: container, email, function, http, pipeline, query, sleep, transform
  • Triggers: schedule, http
  • Credential management
  • Mod composition

Learn more at:

Flowpipe Library Mods - 28 new mods

mod