Cloudera Docs

HTTP Sink properties reference

Review the following reference for a comprehensive list of the connector properties that are specific to the HTTP Sink connector.

The properties listed in this reference must be added to the connector configuration with the following prefix:
parameter.[***CONNECTOR NAME***] Parameters:

In addition to the properties listed here, this connector also accepts certain properties of the Kafka Connect framework as well as the properties of the NiFi Stateless Sink connector. When creating a new connector using the SMM UI, all valid properties are presented in the default configuration template. You can view the configuration template to get a full list of valid properties. In addition, for more information regarding the accepted properties not listed here, you can review the Apache Kafka documentation and the Stateless NiFi Sink properties reference.

Basic Authentication Password

Description
The password to be used for authentication when connecting to the URL specified in the Remote URL property. If an authentication method other than Basic Authentication is used, this property must be completely removed from the configuration JSON.
Default Value
Accepted Values
Required
false

Basic Authentication Username

Description
The username used for authentication when connecting to the URL specified in the Remote URL property. Cannot include control characters (0-31), ':', or DEL (127). If an authentication method other than Basic Authentication is used, this property must be completely removed from the configuration JSON.
Default Value
Accepted Values
Required
false

Content-Type

Description
The Content-Type to specify in the HTTP header of the POST request sent by this connector.
Default Value
application/octet-stream
Accepted Values
Required
true

Date Format

Description
Specifies the format to use when reading date fields from JSON. If Forward Raw Data is set to false, the format defined here also applies to the date fields in the output JSON message.
Default Value
yyyy-MM-dd
Accepted Values
Required
true

Forward Raw Data

Description
Specifies whether messages from Kafka should be forwarded as is or converted to JSON. If set to false the Kafka Message Data Format parameter must be specified.
Default Value
true
Accepted Values
true, false
Required
true

Kafka Message Data Format

Description
Specifies the format of the messages the connector receives from Kafka. If the Forward Raw Data property is set to true then this property is ignored. However, even in a case like this, this property must be assigned a valid value.
Default Value
AVRO
Accepted Values
AVRO, JSON
Required
true

Kerberos Keytab for Schema Registry

Description
The fully-qualified filename of the Kerberos keytab associated with the principal for accessing Schema Registry.
Default Value
The location of the default keytab which is empty and can only be used for unsecure connections.
Accepted Values
Required
true

Kerberos Principal for Schema Registry

Description
The Kerberos principal used for authenticating to Schema Registry.
Default Value
default
Accepted Values
Required
true

Keystore Filename for Secure HTTP

Description
The fully-qualified filename of a keystore. This keystore is used to establish a secure connection with the HTTP server using mutual TLS.

If the HTTP server does not require client certificate authentication, this property must be completely removed from the configuration JSON.

Default Value
Accepted Values
Required
false

Keystore Key Password for Secure HTTP

Description
The password used to access the key stored in the keystore file configured in the Keystore Filename for Secure HTTP property.

If the HTTP server does not require client certificate authentication, this property must be completely removed from the configuration JSON.

Default Value
Accepted Values
Required
false

Keystore Password for Secure HTTP

Description
The password used to access the contents of the keystore configured in the Keystore Filename for Secure HTTP property.

If the HTTP server does not require client certificate authentication, this property must be completely removed from the configuration JSON.

Default Value
Accepted Values
Required
false

Keystore Type for Secure HTTP

Description
The type of the keystore configured in the Keystore Filename for Secure HTTP property.

If the HTTP server does not require client certificate authentication, this property must be completely removed from the configuration JSON.

Default Value
Accepted Values
BCFKS, PKCS12, JKS
Required
false

Output Grouping for JSON

Description
Specifies how JSON objects are grouped in the connector output.

If set to output-array, the output will consist of an array of JSON objects.

f set to output-oneline, each line of the output data becomes one JSON object. That is, each JSON object occupies one line in the output.

This property is only used if forwarding raw data is disabled (Forward Raw Data is set to false). This is because when forwarding raw data is disabled, the data coming from Kafka gets converted to JSON. However, this property must still be set even if forwarding raw data is enabled.

Default Value
output-oneline
Accepted Values

output-array, output-oneline

Required
true

Remote URL

Description
The remote URL to connect to. Must include scheme, host, port, and path.
Default Value
https://localhost:22000/contentListener
Accepted Values
Required
true

Schema Access Strategy

Description
Specifies the strategy used for determining the schema of the Kafka record. The value you set here depends on the data format set in Kafka Message Data Format.
  • If set to Schema Registry, the schema is read from Schema Registry. This setting can be used with both Avro and JSON formats.
  • If set to Infer Schema, the schema is inferred based on the input file. This setting can only be used if Kafka Message Data Format is JSON.
  • If set to Embedded Schema, the schema embedded in the input is used. This setting can only be used if Kafka Message Data Format is Avro.
  • If set to HWX Content-Encoded Schema Reference, the schema is read from Schema Registry. This setting can only be used if Kafka Message Data Format is Avro. In this case the Avro messages are expected to have a reference to the schema in Schema Registry encoded within the message content.
This property is not used if record processing is disabled (Forward Raw Data is set to true).
Default Value
Schema Registry
Accepted Values
Schema Registry, Infer Schema, Embedded Schema, HWX Content-Encoded Schema Reference
Required
true

Schema Branch

Description
The name of the branch to use when looking up the schema in Schema Registry. Schema Branch and Schema Version cannot be specified at the same time. If one is specified, the other needs to be removed from the configuration. If Schema Registry is not used, this property must be completely removed from the configuration.
Default Value
Accepted Values
Required
false

Schema Name

Description
The schema name to look up in Schema Registry.

If the Schema Access Strategy property is set to Schema Registry, this property must contain a valid schema name.

If Schema Registry is not used, this property must be completely removed from the configuration JSON.

Default Value
Accepted Values
Required
false

Schema Registry URL

Description
The URL of the Schema Registry server. If Schema Registry is not used, use the default value.
Default Value
http://localhost:7788/api/v1
Accepted Values
Required
true

Schema Version

Description
The version of the schema to look up in Schema Registry. If Schema Registry is used and a schema version is not specified, the latest version of the schema is retrieved. Schema Branch and Schema Version cannot be specified at the same time. If one is specified, the other needs to be removed from the configuration. If Schema Registry is not used, this property must be completely removed from the configuration.
Default Value
Accepted Values
Required
false

Time Format

Description
Specifies the format to use when reading time fields from JSON. If Forward Raw Data is set to false, the format defined here also applies to the time fields in the output JSON message.
Default Value
HH:mm:ss
Accepted Values
Required
true

Timestamp Format

Description
Specifies the format to use when reading timestamp fields from JSON. If Forward Raw Data is set to false, the format defined here also applies to the timestamp fields in the output JSON message.
Default Value
HH:mm:ss.SSS
Accepted Values
Required
true

Truststore Filename for Schema Registry

Description
The fully-qualified filename of a truststore. This truststore is used to establish a secure connection with Schema Registry using TLS.
Default Value
The location of the default truststore which is empty and can only be used for unsecure connection.
Accepted Values
Required
true

Truststore Filename for Secure HTTP

Description
The fully-qualified filename of a truststore. This truststore is used to establish a secure connection with the HTTP server using TLS
Default Value
The location of the default truststore which is empty and can only be used for unsecure connections.
Accepted Values
Required
true

Truststore Password for Schema Registry

Description
The password used to access the contents of the truststore configured in the Truststore Filename for Schema Registry property.
Default Value
password
Accepted Values
Required
true

Truststore Password for Secure HTTP

Description
The password used to access the contents of the truststore configured in the Truststore Filename for Secure HTTP property.
Default Value
password
Accepted Values
Required
true

Truststore Type for Schema Registry

Description
The type of the truststore configured in the Truststore Filename for Schema Registry property.
Default Value
JKS
Accepted Values
BCFKS, PKCS12, JKS
Required
true

Truststore Type for Secure HTTP

Description
The type of the truststore configured in the Truststore Filename for Secure HTTP property.
Default Value
JKS
Accepted Values
BCFKS, PKCS12, JKS
Required
true