PublishGCPubSub

Description:

Publishes the content of the incoming flowfile to the configured Google Cloud PubSub topic. The processor supports dynamic properties. If any dynamic properties are present, they will be sent along with the message in the form of 'attributes'.

Tags:

google, google-cloud, gcp, message, pubsub, publish

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

Display Name	API Name	Default Value	Allowable Values	Description
Project ID	gcp-project-id			Google Cloud Project ID Supports Expression Language: true (will be evaluated using variable registry only)
Proxy Configuration Service	proxy-configuration-service		Controller Service API: ProxyConfigurationService Implementation: StandardProxyConfigurationService	Specifies the Proxy Configuration Controller Service to proxy network requests. If set, it supersedes proxy settings configured per component. Supported proxies: HTTP + AuthN
GCP Credentials Provider Service	GCP Credentials Provider Service		Controller Service API: GCPCredentialsService Implementation: GCPCredentialsControllerService	The Controller Service used to obtain Google Cloud Platform credentials.
Input Batch Size	Input Batch Size	100		Maximum number of FlowFiles processed for each Processor invocation
Message Derivation Strategy	Message Derivation Strategy	FlowFile Oriented	FlowFile Oriented Record Oriented	The strategy used to publish the incoming FlowFile to the Google Cloud PubSub endpoint.
Record Reader	Record Reader		Controller Service API: RecordReaderFactory Implementations: EBCDICRecordReader XMLReader AvroReader CiscoEmblemSyslogMessageReader ReaderLookup JsonPathReader JASN1Reader GrokReader JsonTreeReader WindowsEventLogReader YamlTreeReader IPFIXReader CEFReader SyslogReader ExcelReader ParquetReader ScriptedReader CSVReader Syslog5424Reader	The Record Reader to use for incoming FlowFiles This Property is only considered if the [Message Derivation Strategy] Property has a value of "Record Oriented".
Record Writer	Record Writer		Controller Service API: RecordSetWriterFactory Implementations: RecordSetWriterLookup ScriptedRecordSetWriter JsonRecordSetWriter AvroRecordSetWriter CSVRecordSetWriter ParquetRecordSetWriter XMLRecordSetWriter FreeFormTextRecordSetWriter	The Record Writer to use in order to serialize the data before sending to GCPubSub endpoint This Property is only considered if the [Message Derivation Strategy] Property has a value of "Record Oriented".
Maximum Message Size	Maximum Message Size	1 MB		The maximum size of a Google PubSub message in bytes. Defaults to 1 MB (1048576 bytes) This Property is only considered if the [Message Derivation Strategy] Property has a value of "FlowFile Oriented".
Topic Name	gcp-pubsub-topic			Name of the Google Cloud PubSub Topic Supports Expression Language: true (will be evaluated using variable registry only)
Batch Size Threshold	gcp-pubsub-publish-batch-size	15		Indicates the number of messages the cloud service should bundle together in a batch. If not set and left empty, only one message will be used in a batch
Batch Bytes Threshold	gcp-batch-bytes	3 MB		Publish request gets triggered based on this Batch Bytes Threshold property and the Batch Size Threshold property, whichever condition is met first. Supports Expression Language: true (will be evaluated using variable registry only)
Batch Delay Threshold	gcp-pubsub-publish-batch-delay	100 millis		Indicates the delay threshold to use for batching. After this amount of time has elapsed (counting from the first element added), the elements will be wrapped up in a batch and sent. This value should not be set too high, usually on the order of milliseconds. Otherwise, calls might appear to never complete.
API Endpoint	api-endpoint	pubsub.googleapis.com:443		Override the gRPC endpoint in the form of [host:port] Supports Expression Language: true (will be evaluated using variable registry only)

Dynamic Properties:

Supports Sensitive Dynamic Properties: No

Dynamic Properties allow the user to specify both the name and value of a property.

Name	Value	Description
Attribute name	Value to be set to the attribute	Attributes to be set for the outgoing Google Cloud PubSub message Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)

Relationships:

Name	Description
retry	FlowFiles are routed to this relationship if the Google Cloud Pub/Sub operation fails but attempting the operation again may succeed.
success	FlowFiles are routed to this relationship after a successful Google Cloud Pub/Sub operation.
failure	FlowFiles are routed to this relationship if the Google Cloud Pub/Sub operation fails.

Reads Attributes:

None specified.

Writes Attributes:

Name	Description
gcp.pubsub.messageId	ID of the pubsub message published to the configured Google Cloud PubSub topic
gcp.pubsub.count.records	Count of pubsub messages published to the configured Google Cloud PubSub topic
gcp.pubsub.topic	Name of the Google Cloud PubSub topic the message was published to

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship.

System Resource Considerations:

Resource	Description
MEMORY	The entirety of the FlowFile's content will be read into memory to be sent as a PubSub message.