PutHDFS 2.4.0.4.3.3.0-40

Bundle
org.apache.nifi | nifi-hadoop-nar
Description
Write FlowFile data to Hadoop Distributed File System (HDFS)
Tags
HCFS, HDFS, copy, filesystem, hadoop, put
Input Requirement
REQUIRED
Supports Sensitive Dynamic Properties
false
  • Additional Details for PutHDFS 2.4.0.4.3.3.0-40

    PutHDFS

    SSL Configuration:

    Hadoop provides the ability to configure keystore and/or truststore properties. If you want to use SSL-secured file system like swebhdfs, you can use the Hadoop configurations instead of using SSL Context Service.

    1. create ‘ssl-client.xml’ to configure the truststores.

    ssl-client.xml Properties:

    Property Default Value Explanation
    ssl.client.truststore.type jks Truststore file type
    ssl.client.truststore.location NONE Truststore file location
    ssl.client.truststore.password NONE Truststore file password
    ssl.client.truststore.reload.interval 10000 Truststore reload interval, in milliseconds

    ssl-client.xml Example:

    <configuration>
    <property>
        <name>ssl.client.truststore.type</name>
        <value>jks</value>
    </property>
    <property>
        <name>ssl.client.truststore.location</name>
        <value>/path/to/truststore.jks</value>
    </property>
    <property>
        <name>ssl.client.truststore.password</name>
        <value>clientfoo</value>
    </property>
    <property>
        <name>ssl.client.truststore.reload.interval</name>
        <value>10000</value>
    </property>
</configuration>
    1. put ‘ssl-client.xml’ to the location looked up in the classpath, like under NiFi conriguration directory.
    2. set the name of ‘ssl-client.xml’ to hadoop.ssl.client.conf in the ‘core-site.xml’ which HDFS processors use.
    <configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>swebhdfs://{namenode.hostname:port}</value>
    </property>
    <property>
        <name>hadoop.ssl.client.conf</name>
        <value>ssl-client.xml</value>
    </property>
</configuration>
Properties
Restrictions
Required Permission Explanation
write distributed filesystem Provides operator the ability to delete any file that NiFi has access to in HDFS or the local filesystem.
Relationships
Name Description
failure Files that could not be written to HDFS for some reason are transferred to this relationship
success Files that have been successfully written to HDFS are transferred to this relationship
Reads Attributes
Name Description
filename The name of the file written to HDFS comes from the value of this attribute.
Writes Attributes
Name Description
filename The name of the file written to HDFS is stored in this attribute.
absolute.hdfs.path The absolute path to the file on HDFS is stored in this attribute.
hadoop.file.url The hadoop url for the file is stored in this attribute.
target.dir.created The result(true/false) indicates if the folder is created by the processor.
See Also