CiscoEmblemSyslogMessageReader 2.4.0.4.11.0.0-352

Bundle
com.cloudera | nifi-cdf-syslog-nar
Description
A Cisco EMBLEM extension of Syslog Message Reader. Supports standard EMBLEM fields.
Tags
cisco, emblem, reader, record, syslog
Input Requirement
Supports Sensitive Dynamic Properties
false
  • Additional Details for CiscoEmblemSyslogMessageReader 2.4.0.4.11.0.0-352

    CiscoEmblemSyslogMessageReader

    RFC 3164 BSD syslog

    The CiscoEmblemSyslogMessageReader supports reading messages formatted according to the BSD syslog protocol defined in RFC 3164. The RFC standard defines the following parts of a log:

    • PRIORITY
    • HEADER
    • MESSAGE

    The PRIORITY element is optional and is not included in some messages. The HEADER element is required and consists of a TIMESTAMP and HOSTNAME. The MESSAGE element is required and contains an unformatted log string.

    RFC 3164 Timestamps

    The TIMESTAMP portion of the HEADER element consists of a month, day, hour, minute, and second, defined according to the system producing the log. The day component of the TIMESTAMP consists of one or two digits, with a space character preceding a single digit day. The following TIMESTAMP is an example of a single digit day:

    Jan  1 12:30:45

    The following TIMESTAMP is an example of a day with two digits:

    Jan 15 10:15:30

    The RFC 3164 format does not include the year or timezone of the log, requiring parsing systems to infer those details during processing.

    RFC 5424 Timestamps

    The newer syslog standard defined in RFC 5424 requires a TIMESTAMP to be expressed using the ISO 8601 format that includes the year and timezone offset. The following TIMESTAMP is an example of the ISO 8601 format with the timezone indicating Coordinated Universal Time:

    1970-01-01T10:15:30Z

    Cisco EMBLEM syslog

    The Cisco EMBLEM syslog format follows the RFC 3164 syslog standard. Some versions of Cisco products support defining the TIMESTAMP portion using the RFC 5424 standard. The EMBLEM format prefixes the MESSAGE portion of each log with the following standard parts:

    • FACILITY
    • LEVEL
    • MESSAGE NUMBER

    The FACILITY portion provides the product that generated the message, such as ASA.

    The LEVEL portion provides the severity level of the message from 1 to 7 defined as follows:

    Level Description
    1 ALERT
    2 CRITICAL
    3 ERROR
    4 WARNING
    5 NOTIFICATION
    6 INFORMATIONAL
    7 DEBUGGING

    The MESSAGE NUMBER portion provides the identifier associated with the particular message format. Cisco maintains documentation describing the message pattern and variable elements for each message number. The Cisco ASA Series Syslog Messages documentation describes messages for Cisco ASA products.

    The Cisco EMBLEM prefix begins with a percent character followed by the standard elements and ended with a colon character. The following prefix provides an example of a FACILITY of ASA, a LEVEL of 1 and a MESSAGE NUMBER of 101001:

    %ASA-1-101001:

    The following provides an example of a Cisco EMBLEM syslog message using the RFC 3164 TIMESTAMP:

    Jan 15 10:15:30 FIREWALL-1 %ASA-1-101001: (Primary) Failover cable OK.

    Cisco EMBLEM Parsing

    The CiscoEmblemSyslogMessageReader uses multiple sets of regular expression patterns to parse messages. Each Record that the Reader parses contains a set of standard fields as well as additional fields based on the regular expression pattern associated with the Cisco EMBLEM Message Number.

    LOG Records

    For log strings that do not match the EMBLEM format pattern, the Reader produces a simplified Record with the following fields:

    • log
    • format

    When the Reader cannot parse logs according to the Cisco EMBLEM format, the log field contains the value of LOG.

    EMBLEM Records

    The first step of the parsing process transforms the log string into a record with standard syslog fields. For log strings that match the expected Cisco EMBLEM format, the Reader produces a Record with the following fields:

    • log
    • format
    • timestamp
    • hostname
    • message
    • facility
    • level
    • messageNumber

    The log field contains the original log string and the format field contains the value of EMBLEM.

    The timestamp field contains a parsed date and time. For values formatted according to RFC 3164, the Reader uses the current system year and the current system timezone. This approach requires that the processing system has the same clock settings as the system generating syslog messages. Timestamps formatted according to RFC 5424 do not require default values for year and timezone values.

    PARSED Records

    After parsing the original log string and matching the standard EMBLEM pattern, the Reader attempts to find a second regular expression pattern based on the MESSAGE NUMBER found. The Reader includes a default set of regular expressions for a subset of Cisco Syslog Message Numbers. When the Reader does not find a second regular expression for the Message Number, the Reader does not perform any further processing. When the Reader finds a second regular expression, the Reader attempts to parse the message portion of the log. When the Reader matches the second regular expression, the Reader sets the format field value to PARSED and also sets one or more fields defined in the regular expression.

    Regular Expression Field Patterns

    The CiscoEmblemSyslogMessageReader uses regular expression patterns with named-capturing groups to parse EMBLEM messages into one or more custom record fields. The Java Pattern documentation defines a set of supported characters for named-capturing groups limited to letters and numbers.

    The Reader can be configured with a file containing comma-separated values where the first column contains the Cisco Message Number and the second column contains the regular expression pattern to be matched. Patterns defined in the file will override the default patterns defined in the Reader.

    Example Field Pattern Parsing

    The following pattern provides an example of what the Reader uses for parsing messages with a message number of 106001:

    (?<direction>Inbound) (?<protocol>TCP) connection (?<outcome>denied) from (?<sourceAddress>[^/]+)/(?<sourcePort>\d{1,5}) to (?<destinationAddress>[^/]+)/(?<destinationPort>\d{1,5}) flags (?<protocolFlags>[A-Z\s]+?) on interface (?<sourceInterface>[\S]+)

    The pattern is capable of parsing the following log:

    Dec 31 06:15:30 FIREWALL-1 %ASA-2-106001: Inbound TCP connection denied from 10.0.0.1/25000 to 192.168.1.1/443 flags URG SYN RST on interface OUTSIDE

    After the Reader matches the first Cisco EMBLEM syslog pattern, the Reader uses the message number pattern to parse the following message portion of the log:

    Inbound TCP connection denied from 10.0.0.1/25000 to 192.168.1.1/443 flags URG SYN RST on interface OUTSIDE

    Based on the message number pattern, the Reader parses the message into the following fields:

    Field Value
    direction Inbound
    protocol TCP
    outcome denied
    sourceAddress 10.0.0.1
    sourcePort 25000
    destinationAddress 192.168.1.1
    destinationPort 443
    protocolFlags URG SYN RST
    sourceInterface OUTSIDE
Properties