PutCDPObjectStore

Description:

Write FlowFile data to an object store.

Tags:

hadoop, HCFS, HDFS, put, copy, filesystem, CDP, GCP, GCS, Google, S3, AWS, ADLS, Azure

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

Display Name	API Name	Default Value	Allowable Values	Description
Storage Location	cdp-storage-location			Use this property to set the storage location in use. Example: 's3a://myBucket/myDirectory'. In case the property is not specified the processor will use the value set in /etc/hadoop/conf/core-site.xml Supports Expression Language: true (will be evaluated using variable registry only)
Directory	Directory			The path (absolute or relative to the Storage Location) to the parent directory to which files should be written into. The directory will be created if it doesn't exist. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Conflict Resolution Strategy	Conflict Resolution Strategy	fail	replace ignore fail	Indicates what should happen when a file with the same name already exists in the output directory
Kerberos Credentials Service	kerberos-credentials-service		Controller Service API: KerberosCredentialsService Implementation: KeytabCredentialsService	Specifies the Kerberos Credentials Controller Service that should be used for authenticating with Kerberos
CDP Username	Kerberos Principal			CDP User name. Recommendation is to create a dedicated Machine User in the CDP User Management UI. Supports Expression Language: true (will be evaluated using variable registry only)
CDP Password	Kerberos Password			Workload password associated to your CDP User. You can set it in the CDP User Management UI. If you don't want to use a workload password, you can use the Kerberos Credentials controller service property. Sensitive Property: true
Writing Strategy	writing-strategy	Simple write	Write and rename Simple write	Defines the approach for writing the FlowFile data.

Dynamic Properties:

Supports Sensitive Dynamic Properties: No

Dynamic Properties allow the user to specify both the name and value of a property.

Name	Value	Description
A Hadoop client configuration name	The value to set it to	Sets and if already set, overwrites the Hadoop client configuration with the given name. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)

Relationships:

Name	Description
success	Files that have been successfully written to the object store are transferred to this relationship
failure	Files that could not be written to the object store for some reason are transferred to this relationship

Reads Attributes:

Name	Description
filename	The name of the file written to the object store comes from the value of this attribute.

Writes Attributes:

Name	Description
filename	The name of the file written to HDFS is stored in this attribute.
absolute.hdfs.path	The absolute path to the file on HDFS is stored in this attribute.
hadoop.file.url	The hadoop url for the file is stored in this attribute.
target.dir.created	The result(true/false) indicates if the folder is created by the processor.
filename	The name of the file written to object store is stored in this attribute.

State management:

This component does not store state.

Restricted:

Required Permission	Explanation
write distributed filesystem	Provides operator the ability to delete any file that NiFi has access to in HDFS or the local filesystem.

Input requirement:

This component requires an incoming relationship.

System Resource Considerations:

None specified.