singer_sdk.Target
- class singer_sdk.Target
Bases:
PluginBase
,SingerReader
Abstract base class for targets.
The Target class manages config information and is responsible for processing the incoming Singer data stream and orchestrating any needed target Sink objects. As messages are received from the tap, the Target class will automatically create any needed target Sink objects and send records along to the appropriate Sink object for that record.
- __init__(config: dict | PurePath | str | list[PurePath | str] | None = None, parse_env_config: bool = False, validate_config: bool = True) None
Initialize the target.
- Parameters
config – Target configuration. Can be a dictionary, a single path to a configuration file, or a list of paths to multiple configuration files.
parse_env_config – Whether to look for configuration values in environment variables.
validate_config – True to require validation of config settings.
- add_sink(stream_name: str, schema: dict, key_properties: list[str] | None = None) Sink
Create a sink and register it.
This method is internal to the SDK and should not need to be overridden.
- Parameters
stream_name – Name of the stream.
schema – Schema of the stream.
key_properties – Primary key of the stream.
- Returns
A new sink for the stream.
- classmethod append_builtin_config(config_jsonschema: dict) None
Appends built-in config to config_jsonschema if not already set.
To customize or disable this behavior, developers may either override this class method or override the capabilities property to disabled any unwanted built-in capabilities.
For all except very advanced use cases, we recommend leaving these implementations “as-is”, since this provides the most choice to users and is the most “future proof” in terms of taking advantage of built-in capabilities which may be added in the future.
- Parameters
config_jsonschema – [description]
- cli = <Command cli>
- property config: Mapping[str, Any]
Get config.
- Returns
A frozen (read-only) config dictionary map.
- drain_all(is_endofpipe: bool = False) None
Drains all sinks, starting with those cleared due to changed schema.
This method is internal to the SDK and should not need to be overridden.
- Parameters
is_endofpipe – This is passed by the
_process_endofpipe()
which is called after the target instance has finished listening to the stdin
- drain_one(sink: Sink) None
Drain a specific sink.
This method is internal to the SDK and should not need to be overridden.
- Parameters
sink – Sink to be drained.
- get_sink(stream_name: str, *, record: dict | None = None, schema: dict | None = None, key_properties: list[str] | None = None) Sink
Return a sink for the given stream name.
A new sink will be created if schema is provided and if either schema or key_properties has changed. If so, the old sink becomes archived and held until the next drain_all() operation.
Developers only need to override this method if they want to provide a different sink depending on the values within the record object. Otherwise, please see default_sink_class property and/or the get_sink_class() method.
Raises
singer_sdk.exceptions.RecordsWithoutSchemaException
if sink does not exist and schema is not sent.- Parameters
stream_name – Name of the stream.
record – Record being processed.
schema – Stream schema.
key_properties – Primary key of the stream.
- Returns
The sink used for this target.
- get_sink_class(stream_name: str) type[Sink]
Get sink for a stream.
Developers can override this method to return a custom Sink type depending on the value of stream_name. Optional when default_sink_class is set.
- Parameters
stream_name – Name of the stream.
- Raises
ValueError – If no
singer_sdk.sinks.Sink
class is defined.- Returns
The sink class to be used with the stream.
- listen(file_input: IO[str] | None = None) None
Read from input until all messages are processed.
- Parameters
file_input – Readable stream of messages. Defaults to standard in.
This method is internal to the SDK and should not need to be overridden.
- property max_parallelism: int
Get max parallel sinks.
The default is 8 if not overridden.
- Returns
Max number of sinks that can be drained in parallel.
- classmethod print_about(format: Optional[str] = None) None
Print capabilities and other tap metadata.
- Parameters
format – Render option for the plugin information.
- classmethod print_version(print_fn: ~typing.Callable[[~typing.Any], None] = <built-in function print>) None
Print help text for the tap.
- Parameters
print_fn – A function to use to display the plugin version. Defaults to print.
- sink_exists(stream_name: str) bool
Check sink for a stream.
This method is internal to the SDK and should not need to be overridden.
- Parameters
stream_name – Name of the stream
- Returns
True if a sink has been initialized.
- property state: dict
Get state.
- Raises
NotImplementedError – If the derived plugin doesn’t override this method.