Updates a DeltaLake table with information of new added files.
deltalake, deltatable, cloud, storage, parquet
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
| Display Name | API Name | Default Value | Allowable Values | Description |
|---|---|---|---|---|
| Hadoop Configuration Resources | hadoop-config-resources | A file, or a comma separated list of files, which contain the Hadoop configuration (core-site.xml, etc.) for the Hadoop filesystem where files are being added. This property expects a comma-separated list of file resources. Supports Expression Language: true (will be evaluated using variable registry only) | ||
| Storage Location | storage-location | Use this property to set the storage location in use. Example: 's3a://myBucket/myDirectory'. In case the property is not specified, the processor will use the value set in Hadoop Configuration Resources. Supports Expression Language: true (will be evaluated using variable registry only) | ||
| DeltaLake Table Path | deltalake-table-path | Path (relative to the root of the target Hadoop file system) where the DeltaLake table is located. Supports Expression Language: true (will be evaluated using variable registry only) | ||
| Data File URL | file-url | ${hadoop.file.url} | The URL of the file that was added and with which to update the DeltaLake table. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry) | |
| Kerberos User Service | kerberos-user-service | Controller Service API: KerberosUserService Implementations: KerberosTicketCacheUserService KerberosKeytabUserService KerberosPasswordUserService | Specifies the Kerberos User Controller Service that should be used for authenticating with Kerberos. | |
| Number of Commit Retries | number-of-commit-retries | 10 | Number of times to retry a commit before failing. | |
| Commit Wait Time | commit-wait-time | 100 ms | Time to wait before retrying a commit. |
| Name | Description |
|---|---|
| success | DeltaLake table successfully updated |
| failure | DeltaLake table update failed |