SDK Implementation Details - Record Metadata
The SDK can automatically generate
_sdc_ (“Singer Data Capture”) metadata properties when
performing data loads in SDK-based targets.
add_record_metadata is defined as
a config option by the developer, and if the user sets
their own configuration, the following columns will be automatically added to each record:
_sdc_extracted_at- Timestamp indicating when the record was extracted the record from the source.
_sdc_received_at- Timestamp indicating when the record was received by the target for loading.
_sdc_batched_at- Timestamp indicating when the record’s batch was initiated.
_sdc_deleted_at- Passed from a Singer tap if DELETE events are able to be tracked. In general, this is populated when the tap is synced LOG_BASED replication. If not sent from the tap, this field will be null.
_sdc_sequence- The epoch (milliseconds) that indicates the order in which the record was queued for loading.
_sdc_table_version- Indicates the version of the table. This column is used to determine when to issue TRUNCATE commands during loading, where applicable.