Tutorial: Invoking an HTTP endpoint with curl

This tutorial walks you through invoking an HTTP Inbound Connection Endpoint with curl using the ListenHTTP filter to Kafka ReadyFlow from the ReadyFlow Gallery.

  1. Deploy the ListenHTTP filter to Kafka Ready Flow.
    1. Navigate to the ReadyFlow Gallery, locate the ListenHTTP filter to Kafka ReadyFlow and click View Added Flow Definition.
    2. Click Deploy and select your target environment to start the Deployment Wizard for the latest version of this ReadyFlow.
    3. Specify a deployment name, for example, Inbound Connections curl and click Next.
    4. Select Allow NiFi to receive data checkbox to configure an endpoint host.
    5. Accept the automatically created endpoint hostname and automatically discovered port by clicking Next.
    6. Optional: This ReadyFlow performs schema validation for incoming events using Cloudera’s Schema Registry before sending the events to a Kafka topic. If you have a Streams Messaging cluster available, fill in the Kafka and Schema Registry connection properties.

      If you only want to validate inbound connection endpoint connectivity, enter dummy values for the empty parameters, set the Input and Output format to JSON while keeping the Listening Port set to 7001.

    7. At Sizing & Scaling select the Extra Small NiFi Node Size and click Next.
    8. Add a KPI on the ListenHTTP processor to monitor how many bytes it is receiving, by clicking Add new KPI.

      Make the following settings:
      KPI Scope
      Processor Name
      Metric to Track
      Bytes Received
    9. Click Next.
    10. Review the information provided and click Deploy.

      Soon after the flow deployment has started, the client certificate and private key required for sending data to the NiFi flow become available for the flow deployment that is being created.

  2. Collect the information required to configure your load balancer.
    1. Once the deployment has been created successfully, select it in the Dashboard and click Manage Deployment.

    2. In the Deployment Settings section, navigate to the NiFi Configuration tab to find information about the associated inbound connection endpoint.

    3. Copy the endpoint hostname and port and download the certificate and private key.

  3. Create the curl request to validate connectivity to the HTTP inbound connection endpoint.
    Using the endpoint hostname, port, client certificate and private key you can now construct a curl command to call the endpoint and validate connectivity:

    curl -v -X POST https://[***ENDPOINT HOSTNAME***]:7001/contentListener --key [***/PATH/TO/***]client-private-key-encoded --cert [***/PATH/TO/***]client-certificate-encoded

    You receive an HTTP 200 response code in a similar message, indicating that your client was able to securely connect to the inbound connection endpoint:

    *   Trying
    * Connected to [***ENDPOINT HOSTNAME***] ( port 7001 (#0)
    * ALPN, offering h2
    * ALPN, offering http/1.1
    * successfully set certificate verify locations:
    *  CAfile: /Users/mkohs/letsencrypt-stg-root-x1.pem
    *  CApath: none
    * (304) (OUT), TLS handshake, Client hello (1):
    * (304) (IN), TLS handshake, Server hello (2):
    * TLSv1.2 (IN), TLS handshake, Certificate (11):
    * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
    * TLSv1.2 (IN), TLS handshake, Request CERT (13):
    * TLSv1.2 (IN), TLS handshake, Server finished (14):
    * TLSv1.2 (OUT), TLS handshake, Certificate (11):
    * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
    * TLSv1.2 (OUT), TLS handshake, CERT verify (15):
    * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
    * TLSv1.2 (OUT), TLS handshake, Finished (20):
    * TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
    * TLSv1.2 (IN), TLS handshake, Finished (20):
    * SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
    * ALPN, server did not agree to a protocol
    * Server certificate:
    *  subject: CN=dfx.dtefgqis.xcu2-8y8x.dev.cldr.work
    *  start date: May 11 21:02:20 2022 GMT
    *  expire date: Aug  9 21:02:19 2022 GMT
    *  subjectAltName: host "[***ENDPOINT HOSTNAME***]" matched cert's "[***ENDPOINT HOSTNAME***]"
    *  issuer: C=US; O=(STAGING) Let's Encrypt; CN=(STAGING) Artificial Apricot R3
    *  SSL certificate verify ok.
    > POST /contentListener HTTP/1.1
    > Host: [***ENDPOINT HOSTNAME***]:7001
    > User-Agent: curl/7.79.1
    > Accept: */*
    * Mark bundle as not supporting multiuse
    < HTTP/1.1 200 OK
    < Date: Thu, 12 May 2022 00:45:57 GMT
    < Content-Type: text/plain
    < Content-Length: 0
    < Server: Jetty(9.4.46.v20220331)
    * Connection #0 to host [***ENDPOINT HOSTNAME***] left intact
  4. Use curl to post data to the HTTP inbound connection endpoint.

    If you want to post events to the NiFi deployment you can add header and content definition to the curl request.

    This example sends JSON data following a simple schema to the endpoint:

    curl -v -X POST [***ENDPOINT HOSTNAME***]:7001/contentListener \
     --key [***/PATH/TO/***]client-private-key-encoded \
     --cert [***/PATH/TO/***]client-certificate-encoded \
     -H 'Content-Type: application/json' \
     -d '{"created_at":6453,"id":6453,"text":"This is my test event","timestamp_ms":34534,"id_store":12}' 
    The NiFi deployment tries to validate the schema against the schema name and Schema Registry that you provided when deploying the ReadyFlow. If you provided dummy values, you receive a response indicating that the flow was unable to look up the schema.