Prerequisites to Adding a New Telemetry Data Source

Part of customizing your Cloudera Cybersecurity Platform (CCP) configuration is adding a new telemetry data source. Before CCP can process the information from your new telemetry data source, you must use one of the telemetry data collectors to ingest the information into the telemetry ingest buffer. Information moves from the data ingest buffer into the Apache Metron real-time processing security engine, where it is parsed, enriched, triaged, and indexed. Finally, certain telemetry events can initiate alerts that can be assessed in the Metron dashboard.

Before you add a new telemetry data source, you must ensure that your system set up meets the Cloudera Cybersecurity Platform (CCP) requirements.
  • Ensure that the new sensor is installed and set up.

  • Ensure that Apache NiFi or another telemetry data collection tool can feed the telemetry data source events into an Apache Kafka topic.

  • Determine your requirements.

    For example, you might decide that you need to meet the following requirements:

    • Proxy events from the data source logs must be ingested in real-time.

    • Proxy logs must be parsed into a standardized JSON structure suitable for analysis by Metron.

    • In real-time, new data source proxy events must be enriched so that the domain names contain the IP information.

    • In real-time, the IP within the proxy event must be checked against for threat intelligence feeds.

    • If there is a threat intelligence hit, an alert must be raised.

    • The SOC analyst must be able to view new telemetry events and alerts from the new data source.

  • Set CCP values.

    When you install CCP, you set up several hosts. Note the locations of these hosts, their port numbers, and the Metron version for future use:


    The host on which a Kafka broker is installed.


    The host on which an Apache ZooKeeper server is installed.


    The host on which your sensor probes are installed. If you do not have any sensors installed, choose the host on which an Apache Storm supervisor is running.


    The host on which you install Apache NiFi.


    The host in your inventory hosts file that you put in the "enrichment" group.


    The host on which Amazon Elasticsearch or Apache Solr is running. This is the host in your inventory hosts file that you put in the "search" group. Pick one of the search hosts.


    The port of the search host where indexing is configured. (For example, 9300)


    The host on which your Metron UI web application is running. This is the host in your inventory hosts file that you put in the "web" group.


    The release of the Metron binaries you are working with. (For example, CCP-2.0.0)