Receives messages from Microsoft Azure Event Hubs with checkpointing to ensure consistent event processing. Checkpoint tracking avoids consuming a message multiple times and enables reliable resumption of processing in the event of intermittent network failures. Checkpoint tracking requires external storage and provides the preferred approach to consuming messages from Azure Event Hubs. In clustered environment, ConsumeAzureEventHub processor instances form a consumer group and the messages are distributed among the cluster nodes (each message is processed on one cluster node only).
| Display Name | API Name | Default Value | Allowable Values | Description |
|---|
| Event Hub Namespace | event-hub-namespace | | | The namespace that the Azure Event Hubs is assigned to. This is generally equal to <Event Hub Names>-ns.
Supports Expression Language: true (will be evaluated using variable registry only) |
| Event Hub Name | event-hub-name | | | The name of the event hub to pull messages from.
Supports Expression Language: true (will be evaluated using variable registry only) |
| Service Bus Endpoint | Service Bus Endpoint | Azure | - Azure
 - Azure China
- Azure Germany
- Azure US Government
| To support namespaces not in the default windows.net domain. |
| Transport Type | Transport Type | AMQP | - AMQP
- AMQP_WEB_SOCKETS
| Advanced Message Queuing Protocol Transport Type for communication with Azure Event Hubs |
| Shared Access Policy Name | event-hub-shared-access-policy-name | | | The name of the shared access policy. This policy must have Listen claims.
Supports Expression Language: true (will be evaluated using variable registry only) |
| Shared Access Policy Key | event-hub-shared-access-policy-primary-key | | | The key of the shared access policy. Either the primary or the secondary key can be used.
Sensitive Property: true |
| Use Azure Managed Identity | use-managed-identity | false | | Choose whether or not to use the managed identity of Azure VM/VMSS |
| Consumer Group | event-hub-consumer-group | $Default | | The name of the consumer group to use.
Supports Expression Language: true (will be evaluated using variable registry only) |
| Consumer Hostname | event-hub-consumer-hostname | | | DEPRECATED: This property is no longer used.
Supports Expression Language: true (will be evaluated using variable registry only) |
| Record Reader | record-reader | | Controller Service API:
RecordReaderFactory
Implementations: Syslog5424Reader
CEFReader
ReaderLookup
CiscoEmblemSyslogMessageReader
CSVReader
GrokReader
SyslogReader
JsonTreeReader
JsonPathReader
XMLReader
AvroReader
JASN1Reader
ExcelReader
ParquetReader
EBCDICRecordReader
WindowsEventLogReader
IPFIXReader
ScriptedReader | The Record Reader to use for reading received messages. The event hub name can be referred by Expression Language '${eventhub.name}' to access a schema. |
| Record Writer | record-writer | | Controller Service API:
RecordSetWriterFactory
Implementations: JsonRecordSetWriter
ParquetRecordSetWriter
CSVRecordSetWriter
ScriptedRecordSetWriter
XMLRecordSetWriter
FreeFormTextRecordSetWriter
AvroRecordSetWriter
RecordSetWriterLookup | The Record Writer to use for serializing Records to an output FlowFile. The event hub name can be referred by Expression Language '${eventhub.name}' to access a schema. If not specified, each message will create a FlowFile. |
| Initial Offset | event-hub-initial-offset | End of stream | - Start of stream
- End of stream
| Specify where to start receiving messages if offset is not stored in Azure Storage. |
| Prefetch Count | event-hub-prefetch-count | 300 | | No Description Provided.
Supports Expression Language: true (will be evaluated using variable registry only) |
| Batch Size | event-hub-batch-size | 10 | | The number of messages to process within a NiFi session. This parameter affects throughput and consistency. NiFi commits its session and Event Hubs checkpoints after processing this number of messages. If NiFi session is committed, but fails to create an Event Hubs checkpoint, then it is possible that the same messages will be received again. The higher number, the higher throughput, but possibly less consistent.
Supports Expression Language: true (will be evaluated using variable registry only) |
| Message Receive Timeout | event-hub-message-receive-timeout | 1 min | | The amount of time this consumer should wait to receive the Prefetch Count before returning.
Supports Expression Language: true (will be evaluated using variable registry only) |
| Storage Account Name | storage-account-name | | | Name of the Azure Storage account to store event hub consumer group state.
Supports Expression Language: true (will be evaluated using variable registry only) |
| Storage Account Key | storage-account-key | | | The Azure Storage account key to store event hub consumer group state.
Sensitive Property: true
Supports Expression Language: true (will be evaluated using variable registry only) |
| Storage SAS Token | storage-sas-token | | | The Azure Storage SAS token to store Event Hub consumer group state. Always starts with a ? character.
Sensitive Property: true
Supports Expression Language: true (will be evaluated using variable registry only) |
| Storage Container Name | storage-container-name | | | Name of the Azure Storage container to store the event hub consumer group state. If not specified, event hub name is used.
Supports Expression Language: true (will be evaluated using variable registry only) |