ReplaceText

Description:

Updates the content of a FlowFile by searching for some textual value in the FlowFile content (via Regular Expression/regex, or literal value) and replacing the section of the content that matches with some alternate value. It can also be used to append or prepend text to the contents of a FlowFile.

Tags:

Text, Regular Expression, Update, Change, Replace, Modify, Regex

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

Display NameAPI NameDefault ValueAllowable ValuesDescription
Replacement StrategyReplacement StrategyRegex Replace
  • Prepend Insert the Replacement Value at the beginning of the FlowFile or the beginning of each line (depending on the Evaluation Mode). For "Line-by-Line" Evaluation Mode, the value will be prepended to each line. Similarly, for "First-Line", "Last-Line", "Except-Last-Line" and "Except-First-Line" Evaluation Modes,the value will be prepended to header alone, footer alone, all lines except header and all lines except footer respectively. For "Entire Text" evaluation mode,the value will be prepended to the entire text.
  • Append Insert the Replacement Value at the end of the FlowFile or the end of each line (depending on the Evaluation Mode). For "Line-by-Line" Evaluation Mode, the value will be appended to each line. Similarly, for "First-Line", "Last-Line", "Except-Last-Line" and "Except-First-Line" Evaluation Modes,the value will be appended to header alone, footer alone, all lines except header and all lines except footer respectively. For "Entire Text" evaluation mode,the value will be appended to the entire text.
  • Surround Prepends text before the start of the FlowFile (or the start of each line, depending on the configuration of the Evaluation Mode property) as well as appending text to the end of the FlowFile (or the end of each line, depending on the configuration of the Evaluation Mode property)
  • Regex Replace Interpret the Search Value as a Regular Expression and replace all matches with the Replacement Value. The Replacement Value may reference Capturing Groups used in the Search Value by using a dollar-sign followed by the Capturing Group number, such as $1 or $2. If the Search Value is set to .* then everything is replaced without even evaluating the Regular Expression.
  • Literal Replace Search for all instances of the Search Value and replace the matches with the Replacement Value.
  • Always Replace Always replaces the entire line or the entire contents of the FlowFile (depending on the value of the <Evaluation Mode> property) and does not bother searching for any value. When this strategy is chosen, the <Search Value> property is ignored.
  • Substitute Variables Substitute variable references (specified in ${var} form) using FlowFile attributes for looking up the replacement value by variable name. When this strategy is chosen, both the <Search Value> and <Replacement Value> properties are ignored.
The strategy for how and what to replace within the FlowFile's text content.
Search ValueRegular Expression(?s)(^.*$)The Search Value to search for in the FlowFile content. Only used for 'Literal Replace' and 'Regex Replace' matching strategies
Supports Expression Language: true (will be evaluated using flow file attributes and Environment variables)

This Property is only considered if the [Replacement Strategy] Property is set to one of the following values: [Regex Replace], [Literal Replace]
Replacement ValueReplacement Value$1The value to insert using the 'Replacement Strategy'. Using "Regex Replace" back-references to Regular Expression capturing groups are supported, but back-references that reference capturing groups that do not exist in the regular expression will be treated as literal value. Back References may also be referenced using the Expression Language, as '$1', '$2', etc. The single-tick marks MUST be included, as these variables are not "Standard" attribute names (attribute names must be quoted unless they contain only numbers, letters, and _).
Supports Expression Language: true (will be evaluated using flow file attributes and Environment variables)

This Property is only considered if the [Replacement Strategy] Property is set to one of the following values: [Prepend], [Regex Replace], [Always Replace], [Append], [Literal Replace]
Text to PrependText to PrependThe text to prepend to the start of the FlowFile, or each line, depending on teh configured value of the Evaluation Mode property
Supports Expression Language: true (will be evaluated using flow file attributes and Environment variables)

This Property is only considered if the [Replacement Strategy] Property has a value of "Surround".
Text to AppendText to AppendThe text to append to the end of the FlowFile, or each line, depending on teh configured value of the Evaluation Mode property
Supports Expression Language: true (will be evaluated using flow file attributes and Environment variables)

This Property is only considered if the [Replacement Strategy] Property has a value of "Surround".
Character SetCharacter SetUTF-8The Character Set in which the file is encoded
Maximum Buffer SizeMaximum Buffer Size1 MBSpecifies the maximum amount of data to buffer (per file or per line, depending on the Evaluation Mode) in order to apply the replacement. If 'Entire Text' (in Evaluation Mode) is selected and the FlowFile is larger than this value, the FlowFile will be routed to 'failure'. In 'Line-by-Line' Mode, if a single line is larger than this value, the FlowFile will be routed to 'failure'. A default value of 1 MB is provided, primarily for 'Entire Text' mode. In 'Line-by-Line' Mode, a value such as 8 KB or 16 KB is suggested. This value is ignored if the <Replacement Strategy> property is set to one of: Append, Prepend, Always Replace
Evaluation ModeEvaluation ModeLine-by-Line
  • Line-by-Line
  • Entire text
Run the 'Replacement Strategy' against each line separately (Line-by-Line) or buffer the entire file into memory (Entire Text) and run against that.
Line-by-Line Evaluation ModeLine-by-Line Evaluation ModeAll
  • All
  • First-Line
  • Last-Line
  • Except-First-Line
  • Except-Last-Line
Run the 'Replacement Strategy' against each line separately (Line-by-Line) for all lines in the FlowFile, First Line (Header) alone, Last Line (Footer) alone, Except the First Line (Header) or Except the Last Line (Footer).

Relationships:

NameDescription
successFlowFiles that have been successfully processed are routed to this relationship. This includes both FlowFiles that had text replaced and those that did not.
failureFlowFiles that could not be updated are routed to this relationship

Reads Attributes:

None specified.

Writes Attributes:

None specified.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship.

Example Use Cases:

Use Case:

Append text to the end of every line in a FlowFile

Keywords:

raw text, append, line

Configuration:

"Evaluation Mode" = "Line-by-Line"

"Replacement Strategy" = "Append"

"Replacement Value" is set to whatever text should be appended to the line.

For example, to insert the text <fin> at the end of every line, we would set "Replacement Value" to <fin>.

We can also use Expression Language. So to insert the filename at the end of every line, we set "Replacement Value" to ${filename}



Use Case:

Prepend text to the beginning of every line in a FlowFile

Keywords:

raw text, prepend, line

Configuration:

"Evaluation Mode" = "Line-by-Line"

"Replacement Strategy" = "Prepend"

"Replacement Value" is set to whatever text should be prepended to the line.

For example, to insert the text <start> at the beginning of every line, we would set "Replacement Value" to <start>.

We can also use Expression Language. So to insert the filename at the beginning of every line, we set "Replacement Value" to ${filename}



Use Case:

Replace every occurrence of a literal string in the FlowFile with a different value

Keywords:

replace, string, text, literal

Configuration:

"Evaluation Mode" = "Line-by-Line"

"Replacement Strategy" = "Literal Replace"

"Search Value" is set to whatever text is in the FlowFile that needs to be replaced.

"Replacement Value" is set to the text that should replace the current text.

For example, to replace the word "spider" with "arachnid" we set "Search Value" to spider and set "Replacement Value" to arachnid.



Use Case:

Transform every occurrence of a literal string in a FlowFile

Keywords:

replace, transform, raw text

Configuration:

"Evaluation Mode" = "Line-by-Line"

"Replacement Strategy" = "Regex Replace"

"Search Value" is set to a regular expression that matches the text that should be transformed in a capturing group.

"Replacement Value" is set to a NiFi Expression Language expression that references $1 (in quotes to escape the reference name).

For example, if we wanted to lowercase any occurrence of WOLF, TIGER, or LION, we would use a "Search Value" of (WOLF|TIGER|LION) and a "Replacement Value" of ${'$1':toLower()}.

If we want to replace any identifier with a hash of that identifier, we might use a "Search Value" of identifier: (.*) and a "Replacement Value" of identifier: ${'$1':hash('sha256')}



Use Case:

Completely replace the contents of a FlowFile to a specific text

Keywords:

replace, raw text

Configuration:

"Evaluation Mode" = "Entire text"

"Replacement Strategy" = "Always Replace"

"Replacement Value" is set to the new text that should be written to the FlowFile. This text might include NiFi Expression Language to reference one or more attributes.



System Resource Considerations:

ResourceDescription
MEMORYAn instance of this component can cause high usage of this system resource. Multiple instances or high concurrency settings may result a degradation of performance.