Command Line Reference
This page describes how SDK-based taps and targets can be invoked via the command line interface, or “CLI”.
Common CLI Options
The below options apply to taps and targets alike.
Prints information about the tap or target, including a full list of supported CLI options.
Prints the version of the tap or target along with the SDK version and then exits.
Prints important information about the tap or target, including the list of supported CLI commands, the
--version metadata, and list of supported capabilities.
Note: By default, the format of
--about is plain text. You can invoke
--about in combination with the
--format option described below to have the output printed in different formats.
--format=json is specified, the
--about information will be printed as
json in order to easily process the metadata in automated workflows.
--format=markdown is specified, the
--about information will be printed as Markdown, optimized for copy-pasting into the maintainer’s
README.md file. Among other helpful guidance, this automatically creates a markdown table of all settings, their descriptions, and their default values.
The SDK supports one or more
--config inputs when run from the CLI.
If one of the supplied inputs is
--config=ENVaccording to the user’s preference), the environment variable parsing rules will be applied to ingest config values from environment variables.
One or more files can also be sent to
--config. If multiple files are sent, they will be processed in sequential order. If one or more files conflict for a given setting, the latter provided files will override earlier provided files.
This behavior allows to you easily inject environment overrides by adding
--config=path/to/overrides.jsonat the end of the CLI command text.
--config=ENVis set and one or more files conflict with an environment variable setting, the environment variable setting will always have precedence, regardless of ordering.
One benefit of this approach is that credentials and other secrets can be stored completely separately from general settings: either by having two distinct
config.jsonfiles or by using environment variables for secure settings and
config.jsonfiles for the rest.
--config=ENV is specified, the SDK will automatically capture and pass along any
values from environment variables, and from a
.env file if present within the current
working directory, which match the exact name of a setting, along with a prefix determined
by the plugin name.
For example: For a sample plugin named
tap-my-exampleand settings named “username” and “access_key”, the SDK will automatically scrape the settings from environment variables
TAP_MY_EXAMPLE_ACCESS_KEY, if they exist.
Tap-Specific CLI Options
Used to specify the path to a state file. The state file is used for resuming incremental progress on subsequent executions of the tap.
Used to specify the path to a catalog file. The state file is used for resuming incremental progress on subsequent executions of the tap.
Input Catalog Stream Selection
The SDK automatically applies selection logic as described by the Singer Spec.
Selection rules are applied at three levels:
Streams are filtered out if they are deselected or ommitted in the input catalog.
RECORD messages are filtered based upon selection rules in the input catalog.
SCHEMA messages are filtered based upon selection rules in the input catalog.
When invoked on its own, the
--test flag will cause the tap to emit one record per stream and then exit.
When invoked with the
schema option, such as
--test schema, the tap will only emit
SCHEMA messages downstream and will skip emitting any
RECORD messages. This option is helpful if you want to the target to pre-create all target tables without inserting any records.
Runs the tap in discovery mode and then exits without syncing any data.
Target-Specific CLI Options
Specifies that the target should read messages from a file instead of reading its data from
This option is helpful in testing scenarios where you want to invoke the target directly without repeatedly re-invoking the tap.