RecordSchemaCacheService
Implementation: VolatileSchemaCache
Specifies a Schema Cache to add the Record Schema to so that Record Readers can quickly lookup the schema. | Schema Protocol Version | schema-protocol-version | 1 | | The protocol version to be used for Schema Write Strategies that require a protocol version, such as Hortonworks Schema Registry strategies. Valid protocol versions for Hortonworks Schema Registry are integer values 1, 2, or 3. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
This Property is only considered if the [Schema Write Strategy] Property is set to one of the following values: [HWX Schema Reference Attributes], [HWX Content-Encoded Schema Reference] |
Schema Access Strategy | schema-access-strategy | Inherit Record Schema | - Inherit Record Schema
![The schema used to write records will be the same schema that was given to the Record when the Record was created. The schema used to write records will be the same schema that was given to the Record when the Record was created.](../../../../../html/images/iconInfo.png) - Use 'Schema Name' Property
![The name of the Schema to use is specified by the 'Schema Name' Property. The value of this property is used to lookup the Schema in the configured Schema Registry service. The name of the Schema to use is specified by the 'Schema Name' Property. The value of this property is used to lookup the Schema in the configured Schema Registry service.](../../../../../html/images/iconInfo.png) - Use 'Schema Text' Property
![The text of the Schema itself is specified by the 'Schema Text' Property. The value of this property must be a valid Avro Schema. If Expression Language is used, the value of the 'Schema Text' property must be valid after substituting the expressions. The text of the Schema itself is specified by the 'Schema Text' Property. The value of this property must be a valid Avro Schema. If Expression Language is used, the value of the 'Schema Text' property must be valid after substituting the expressions.](../../../../../html/images/iconInfo.png)
| Specifies how to obtain the schema that is to be used for interpreting the data. |
Schema Registry | schema-registry | | Controller Service API: SchemaRegistry Implementations: ClouderaSchemaRegistry HortonworksSchemaRegistry ConfluentSchemaRegistry AvroSchemaRegistry AmazonGlueSchemaRegistry | Specifies the Controller Service to use for the Schema Registry
This Property is only considered if the [Schema Access Strategy] Property is set to one of the following values: [Use 'Schema Name' Property] |
Schema Name | schema-name | ${schema.name} | | Specifies the name of the schema to lookup in the Schema Registry property Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
This Property is only considered if the [Schema Access Strategy] Property has a value of "Use 'Schema Name' Property". |
Schema Version | schema-version | | | Specifies the version of the schema to lookup in the Schema Registry. If not specified then the latest version of the schema will be retrieved. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
This Property is only considered if the [Schema Access Strategy] Property has a value of "Use 'Schema Name' Property". |
Schema Branch | schema-branch | | | Specifies the name of the branch to use when looking up the schema in the Schema Registry property. If the chosen Schema Registry does not support branching, this value will be ignored. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
This Property is only considered if the [Schema Access Strategy] Property has a value of "Use 'Schema Name' Property". |
Schema Text | schema-text | ${avro.schema} | | The text of an Avro-formatted Schema Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
This Property is only considered if the [Schema Access Strategy] Property has a value of "Use 'Schema Text' Property". |
Date Format | Date Format | | | Specifies the format to use when reading/writing Date fields. If not specified, Date fields will be assumed to be number of milliseconds since epoch (Midnight, Jan 1, 1970 GMT). If specified, the value must match the Java Simple Date Format (for example, MM/dd/yyyy for a two-digit month, followed by a two-digit day, followed by a four-digit year, all separated by '/' characters, as in 01/01/2017). |
Time Format | Time Format | | | Specifies the format to use when reading/writing Time fields. If not specified, Time fields will be assumed to be number of milliseconds since epoch (Midnight, Jan 1, 1970 GMT). If specified, the value must match the Java Simple Date Format (for example, HH:mm:ss for a two-digit hour in 24-hour format, followed by a two-digit minute, followed by a two-digit second, all separated by ':' characters, as in 18:04:15). |
Timestamp Format | Timestamp Format | | | Specifies the format to use when reading/writing Timestamp fields. If not specified, Timestamp fields will be assumed to be number of milliseconds since epoch (Midnight, Jan 1, 1970 GMT). If specified, the value must match the Java Simple Date Format (for example, MM/dd/yyyy HH:mm:ss for a two-digit month, followed by a two-digit day, followed by a four-digit year, all separated by '/' characters; and then followed by a two-digit hour in 24-hour format, followed by a two-digit minute, followed by a two-digit second, all separated by ':' characters, as in 01/01/2017 18:04:15). |
CSV Format | CSV Format | Custom Format | - Custom Format
![The format of the CSV is configured by using the properties of this Controller Service, such as Value Separator The format of the CSV is configured by using the properties of this Controller Service, such as Value Separator](../../../../../html/images/iconInfo.png) - RFC 4180
![CSV data follows the RFC 4180 Specification defined at https://tools.ietf.org/html/rfc4180 CSV data follows the RFC 4180 Specification defined at https://tools.ietf.org/html/rfc4180](../../../../../html/images/iconInfo.png) - Microsoft Excel
![CSV data follows the format used by Microsoft Excel CSV data follows the format used by Microsoft Excel](../../../../../html/images/iconInfo.png) - Tab-Delimited
![CSV data is Tab-Delimited instead of Comma Delimited CSV data is Tab-Delimited instead of Comma Delimited](../../../../../html/images/iconInfo.png) - MySQL Format
![CSV data follows the format used by MySQL CSV data follows the format used by MySQL](../../../../../html/images/iconInfo.png) - Informix Unload
![The format used by Informix when issuing the UNLOAD TO file_name command The format used by Informix when issuing the UNLOAD TO file_name command](../../../../../html/images/iconInfo.png) - Informix Unload Escape Disabled
![The format used by Informix when issuing the UNLOAD TO file_name command with escaping disabled The format used by Informix when issuing the UNLOAD TO file_name command with escaping disabled](../../../../../html/images/iconInfo.png)
| Specifies which "format" the CSV data is in, or specifies if custom formatting should be used. |
Value Separator | Value Separator | , | | The character that is used to separate values/fields in a CSV Record. If the property has been specified via Expression Language but the expression gets evaluated to an invalid Value Separator at runtime, then it will be skipped and the default Value Separator will be used. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
This Property is only considered if the [CSV Format] Property has a value of "Custom Format". |
Include Header Line | Include Header Line | true | | Specifies whether or not the CSV column names should be written out as the first line. |
Quote Character | Quote Character | " | | The character that is used to quote values so that escape characters do not have to be used. If the property has been specified via Expression Language but the expression gets evaluated to an invalid Quote Character at runtime, then it will be skipped and the default Quote Character will be used. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
This Property is only considered if the [CSV Format] Property has a value of "Custom Format". |
Escape Character | Escape Character | \ | | The character that is used to escape characters that would otherwise have a specific meaning to the CSV Parser. If the property has been specified via Expression Language but the expression gets evaluated to an invalid Escape Character at runtime, then it will be skipped and the default Escape Character will be used. Setting it to an empty string means no escape character should be used. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
This Property is only considered if the [CSV Format] Property has a value of "Custom Format". |
Comment Marker | Comment Marker | | | The character that is used to denote the start of a comment. Any line that begins with this comment will be ignored. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
This Property is only considered if the [CSV Format] Property has a value of "Custom Format". |
Null String | Null String | | | Specifies a String that, if present as a value in the CSV, should be considered a null field instead of using the literal value.
This Property is only considered if the [CSV Format] Property has a value of "Custom Format". |
Trim Fields | Trim Fields | true | | Whether or not white space should be removed from the beginning and end of fields
This Property is only considered if the [CSV Format] Property has a value of "Custom Format". |
Quote Mode | Quote Mode | Quote Minimal | - Quote All Values
![All values will be quoted using the configured quote character. All values will be quoted using the configured quote character.](../../../../../html/images/iconInfo.png) - Quote Minimal
![Values will be quoted only if they are contain special characters such as newline characters or field separators. Values will be quoted only if they are contain special characters such as newline characters or field separators.](../../../../../html/images/iconInfo.png) - Quote Non-Numeric Values
![Values will be quoted unless the value is a number. Values will be quoted unless the value is a number.](../../../../../html/images/iconInfo.png) - Do Not Quote Values
![Values will not be quoted. Instead, all special characters will be escaped using the configured escape character. Values will not be quoted. Instead, all special characters will be escaped using the configured escape character.](../../../../../html/images/iconInfo.png)
| Specifies how fields should be quoted when they are written
This Property is only considered if the [CSV Format] Property has a value of "Custom Format". |
Record Separator | Record Separator | \n | | Specifies the characters to use in order to separate CSV Records
This Property is only considered if the [CSV Format] Property has a value of "Custom Format". |
Include Trailing Delimiter | Include Trailing Delimiter | false | | If true, a trailing delimiter will be added to each CSV Record that is written. If false, the trailing delimiter will be omitted.
This Property is only considered if the [CSV Format] Property has a value of "Custom Format". |
Character Set | csvutils-character-set | UTF-8 | | The Character Encoding that is used to encode/decode the CSV file |
State management:
This component does not store state.Restricted:
This component is not restricted.System Resource Considerations:
None specified.