Site to Site protocol sequence

Configuring these properties correctly would require some understandings on Site-to-Site protocol sequence.

  1. A client initiates Site-to-Site protocol by sending a HTTP(S) request to the specified remote URL to get remote cluster Site-to-Site information. Specifically, to '/nifi-api/site-to-site'. This request is called SiteToSiteDetail.

  2. A remote NiFi node responds with its input and output ports, and TCP port numbers for RAW and TCP transport protocols.

  3. The client sends another request to get remote peers using the TCP port number returned at #2. From this request, raw socket communication is used for RAW transport protocol, while HTTP keeps using HTTP(S). This request is called Peers.

  4. A remote NiFi node responds with list of available remote peers containing hostname, port, secure and workload such as the number of queued FlowFiles. From this point, further communication is done between the client and the remote NiFi node.

  5. The client decides which peer to transfer data from/to, based on workload information.

  6. The client sends a request to create a transaction to a remote NiFi node.

  7. The remote NiFi node accepts the transaction.

  8. Data is sent to the target peer. Multiple Data packets can be sent in batch manner.

  9. When there is no more data to send, or reached to batch limit, the transaction is confirmed on both end by calculating CRC32 hash of sent data.

  10. The transaction is committed on both end.