Parsing a New Data Source to HCP
Parsers transform raw data (textual or raw bytes) into JSON messages suitable for downstream enrichment and indexing by HCP. There is one parser for each data source and the information is piped to the Enrichment/Threat Intelligence topology.
You can transform the field output in the JSON messages into information and formats to make the output more useful. For example, you can change the timestamp field output from GMT to your timezone.
You must make two decisions before you parse a new data source:
Type of parser you will use for your data source
For more information about which parser to use, see Parsers.
HCP supports two types of parsers: Java and general purpose:
General Purpose - HCP supports two general purpose parsers: Grok and CSV. These parsers are ideal for structured or semi structured logs that are well understood and telemetries with lower volumes of traffic.
A Java parser is appropriate for a telemetry type that is complex to parse, with high volumes of traffic.
How you will parse the new data source
HCP enables you to parse a new data source and transform data fields using the HCP Management module or the command line interface. Both methods are described in the following sections: