Type Alias PbStreamSourceInfo

pub type PbStreamSourceInfo = StreamSourceInfo;

Alias for StreamSourceInfo.

Aliased Type

struct PbStreamSourceInfo {

    pub row_format: i32,
    pub row_schema_location: String,
    pub use_schema_registry: bool,
    pub proto_message_name: String,
    pub csv_delimiter: i32,
    pub csv_has_header: bool,
    pub format: i32,
    pub row_encode: i32,
    pub name_strategy: i32,
    pub key_message_name: Option<String>,
    pub external_table: Option<ExternalTableDesc>,
    pub cdc_source_job: bool,
    pub is_distributed: bool,
    pub format_encode_options: BTreeMap<String, String>,
    pub format_encode_secret_refs: BTreeMap<String, SecretRef>,
}

Fields

row_format: i32

deprecated

row_schema_location: String

use_schema_registry: bool

This means use confluent schema registry and is false for aws glue schema registry. Eventually we will deprecate it and rely on enum SchemaLocation derived from format_encode_options below.

• schema.location false
• schema.registry true
• aws.glue.schema_arn false

proto_message_name: String

csv_delimiter: i32

csv_has_header: bool

format: i32

row_encode: i32

name_strategy: i32

key_message_name: Option<String>

external_table: Option<ExternalTableDesc>

cdc_source_job: bool

This field should now be called is_shared. Not renamed for backwards compatibility.

Whether the stream source is a shared source (it has a streaming job). This is related with RFC: Reusable Source Executor.

Currently, the following sources can be shared:

• Direct CDC sources (mysql & postgresql & sqlserver)
• MQ sources (Kafka)

is_distributed: bool

Only used when cdc_source_job is true. If false, requires_singleton will be set in the stream plan.

• Direct CDC sources: false
• MQ sources (Kafka): true

format_encode_options: BTreeMap<String, String>

Options specified by user in the FORMAT ENCODE clause.

format_encode_secret_refs: BTreeMap<String, SecretRef>

Handle the source relies on any sceret. The key is the propertity name and the value is the secret id and type. For format and encode options.