Amazon S3 Sink properties reference
Amazon S3 Sink connector properties reference.
The following table collects connector properties that are specific for the Amazon S3 Sink Connector. For properties common to all sink connectors, see the upstream Apache Kafka documentation.
| Property Name | Description | Type | Default Value | Accepted Values | Recommended Value |
|---|---|---|---|---|---|
| aws.s3.bucket | The target S3 bucket name. | String | none | Any valid S3 bucket name. | |
| aws.s3.service_endpoint | The target S3 host and port. | String | none | Any valid S3 endpoint. | |
| aws.access_key_id | The AWS secret key ID to authenticate. | String | none | Any valid secret key issued by AWS. | |
| aws.secret_access_key | The AWS secret access key to authenticate. | String | none | Any valid access key issued by AWS. | |
| value.converter | Value conversion class. | String | none | com.cloudera.dim.kafka.connect.converts.AvroConverter | |
| value.converter.passthrough.enabled | Configures whether the AvroConverter translates an Avro record into Kafka Connect Data or transparently passes the Avro encoded bytes as payload. | Boolean | true | true, false | True if input and output are both Avro. |
| value.converter.schema.registry.url | The URL to the Schema Registry server. | String | none | ||
| output.storage | The S3 storage implementation class. | String | none | com.cloudera.dim.kafka.connect.s3.S3PartitionStorage | |
| output.writer | The output file writer which determines the type of file to be written.
The value of this property should be the FQCN of a class that implements the
PartitionWriter interface. |
String | none |
|
com.cloudera.dim.kafka.connect.partition.writers.avro.AvroPartitionWriter |
| output.avro.passthrough.enabled |
Configures Whether the output writer expects an Avro encoded Kafka
Connect data record. Must match the configuration
of |
Boolean | none | true, false | True if input and output are both Avro. |
