singer_sdk.Target

class singer_sdk.Target

Abstract base class for targets.

The Target class manages config information and is responsible for processing the incoming Singer data stream and orchestrating any needed target Sink objects. As messages are received from the tap, the Target class will automatically create any needed target Sink objects and send records along to the appropriate Sink object for that record.

__init__(config: Optional[Union[dict, PurePath, str, List[Union[PurePath, str]]]] = None, parse_env_config: bool = False, validate_config: bool = True) → None

Initialize the target.

Parameters

  • config – Target configuration. Can be a dictionary, a single path to a configuration file, or a list of paths to multiple configuration files.

  • parse_env_config – Whether to look for configuration values in environment variables.

  • validate_config – True to require validation of config settings.

add_sink(stream_name: str, schema: dict, key_properties: Optional[List[str]] = None) → Sink

Create a sink and register it.

This method is internal to the SDK and should not need to be overridden.

Parameters

  • stream_name – Name of the stream.

  • schema – Schema of the stream.

  • key_properties – Primary key of the stream.

Returns

A new sink for the stream.

cli = <Command cli>
drain_all(is_endofpipe: bool = False) → None

Drains all sinks, starting with those cleared due to changed schema.

This method is internal to the SDK and should not need to be overridden.

Parameters

is_endofpipe – This is passed by the _process_endofpipe() which is called after the target instance has finished listening to the stdin

drain_one(sink: Sink) → None

Drain a specific sink.

This method is internal to the SDK and should not need to be overridden.

Parameters

sink – Sink to be drained.

get_sink(stream_name: str, *, record: Optional[dict] = None, schema: Optional[dict] = None, key_properties: Optional[List[str]] = None) → Sink

Return a sink for the given stream name.

A new sink will be created if schema is provided and if either schema or key_properties has changed. If so, the old sink becomes archived and held until the next drain_all() operation.

Developers only need to override this method if they want to provide a different sink depending on the values within the record object. Otherwise, please see default_sink_class property and/or the get_sink_class() method.

Raises singer_sdk.exceptions.RecordsWithoutSchemaException if sink does not exist and schema is not sent.

Parameters

  • stream_name – Name of the stream.

  • record – Record being processed.

  • schema – Stream schema.

  • key_properties – Primary key of the stream.

Returns

The sink used for this target.

get_sink_class(stream_name: str) → Type[Sink]

Get sink for a stream.

Developers can override this method to return a custom Sink type depending on the value of stream_name. Optional when default_sink_class is set.

Parameters

stream_name – Name of the stream.

Raises

ValueError – If no singer_sdk.sinks.Sink class is defined.

Returns

The sink class to be used with the stream.

property max_parallelism: int

Get max parallel sinks.

The default is 8 if not overridden.

Returns

Max number of sinks that can be drained in parallel.

sink_exists(stream_name: str) → bool

Check sink for a stream.

This method is internal to the SDK and should not need to be overridden.

Parameters

stream_name – Name of the stream

Returns

True if a sink has been initialized.