Write Pipelines
Basics
A pipeline is composed of one or more steps:
pipeline "my_job" {step "http" "my_http_step" {url = "https://example.com/my/webhook"}}
Each step has a type and a name, and the arguments are dependent on the type. The step types are sometimes referred to as "primitives". The previous example included an http
step, but there are others - query
, function
, container
, etc. You can even run other pipelines using a pipeline
step.
You can run a pipeline manually from the command line:
flowpipe pipeline run my_job
You can also define a trigger that will start the pipeline in response to an event. For example, you can run it on a schedule with an interval
trigger:
trigger "schedule" "my_hourly_trigger" {schedule = "hourly"pipeline = pipeline.my_job}pipeline "my_job" {step "http" "my_http_step" {url = "https://example.com/my/webhook"}}
Triggers, like steps, require 2 labels - a type and a name. There are multiple trigger types - http
, schedule
, query
, etc.