Storm-HDFS: Trident APIs

The Trident API implements a StateFactory class with an API that resembles the methods from the storm-code API, as shown in the following code sample:

...
Fields hdfsFields = new Fields("field1", "field2");
 
FileNameFormat fileNameFormat = new DefaultFileNameFormat()
 .withPrefix("trident")
 .withExtension(".txt")
 .withPath("/trident");
 
RecordFormat recordFormat = new DelimitedRecordFormat()
 .withFields(hdfsFields);
 
FileRotationPolicy rotationPolicy = new FileSizeRotationPolicy(5.0f, FileSizeRotationPolicy.Units.MB);
 
HdfsState.Options options = new HdfsState.HdfsFileOptions()
 .withFileNameFormat(fileNameFormat)
 .withRecordFormat(recordFormat)
 .withRotationPolicy(rotationPolicy)
 .withFsUrl("hdfs://localhost:8020");
 
StateFactory factory = new HdfsStateFactory().withOptions(options);
 
TridentState state = stream.partitionPersist(factory, hdfsFields, new HdfsUpdater(), new Fields());

See the javadoc for the Trident API, included with the storm-hdfs connector, for more information.

Limitations

Directory and file names changes are limited to a prepackaged file name format based on a timestamp.