Managing Workspaces
A Flowpipe workspace
is a "profile" that allows you to define a unified environment that the Flowpipe client can interact with.
Flowpipe workspaces allow you to define multiple named configurations:
workspace "default" {log_level = "warn"output = "pretty"memory_max_mb = 1024}workspace "local_server" {host = "local"listen = "network"port = 7103input = falsememory_max_mb = 2048base_url = "https://84c5df474.ngrok-free.dev"}workspace "remote_server" {host = "https://flowpipe.mydomain.com:7103"}
and easily switch between them using the --workspace
argument or FLOWPIPE_WORKSPACE
environment variable:
flowpipe server start --workspace local_serverflowpipe pipeline run my_pipeline --workspace local_serverflowpipe pipeline list --workspace remote_serverflowpipe pipeline run remote_pipeline --workspace remote_server
Defining Workspaces
Workspace configurations can be defined in any .fpc
file in the configuration file search path but by convention they are defined in a file named workspaces.fpc
. This file may contain multiple workspace
definitions that can then be referenced by name.
Any unset arguments will assume the default values - you don't need to set them all:
workspace "default" {output = "pretty"update_check = truetelemetry = falseport = 7103}
You can use base=
to inherit settings from another profile:
workspace "dev" {base = workspace.defaultlog_level = "warn"}workspace "prod" {base = workspace.defaultlog_level = "info"input = falseupdate_check = falsehost = local}
Using Workspaces
The workspace named default
is special; if a workspace named default
exists,
--workspace
is not specified in the command, and FLOWPIPE_WORKSPACE
is not set, then Flowpipe uses the default
workspace:
flowpipe pipeline run my_pipeline
It is common to create the default
workspace in ~/.flowpipe/config/workspaces.fpc
. You can also create a "directory local" default for a specific directory. Because the default configuration file search path includes the current directory / mod location at a higher precedence than ~/.flowpipe/config
, your "directory local" default will take precedence when you run Flowpipe from that directory.
You can pass any workspace to --workspace
to use its values:
flowpipe pipeline run my_pipeline --workspace=dev
Or do the same with the FLOWPIPE_WORKSPACE
environment variable:
FLOWPIPE_WORKSPACE=dev flowpipe pipeline run my_pipeline
If you specify the --workspace
argument and the FLOWPIPE_WORKSPACE
environment variable, the --workspace
argument wins:
# prod will be used as the effective workspaceexport FLOWPIPE_WORKSPACE=devflowpipe pipeline run my_pipeline --workspace=prod
If you specify the --workspace
argument and more specific arguments, any more specific arguments will override the workspace values:
# will use "local" as the host, but dev workspace for any OTHER optionsflowpipe pipeline run my_pipeline --workspace=dev --host=local
Environment variable values override default
workspace settings when the default
workspace is implicitly used:
# will use 7777 as the port, but get the rest of the values from the default workspaceexport FLOWPIPE_PORT=7777flowpipe server
If the default workspace is explicitly passed to the --workspace
argument, its values will override any individual environment variables:
# will NOT use 7777 as port - will use ALL of the values from default workspace so the port is 7103export FLOWPIPE_PORT=7777flowpipe server --workspace=default
The same is true of any named workspace:
# will NOT use 7777 as port - will use ALL of the values from prod workspace so the port is 7103export FLOWPIPE_PORT=7777flowpipe server --workspace=prod