Enabling Topology Event Logging

The topology event inspector lets you view tuples as they flow through different stages of a Storm topology. This tool is useful for inspecting tuples emitted from a spout or a bolt in the topology pipeline while the topology is running; you do not need to stop or redeploy the topology to use the event inspector. The normal flow of tuples from spouts to bolts is not affected by turning on event logging.

Configuring Topology Event Logging

Event logging sends events (tuples) from each component to an internal eventlogger bolt.

Event logging is disabled by default, due to a slight performance degradation associated with eventlogger tasks.

To enable event logging, set the topology.eventlogger.executors property to a non-zero value when submitting your topology. You can set the property globally in the storm.yaml file, or use the command line. For more information about topology.eventlogger.executors and other property settings, see Configuring Apache Storm for Production Environments.

Enabling Event Logging

To log events for an entire topology, click the "Debug" button under "Topology actions" in the topology view. This setting logs tuples from all spouts and bolts in a topology at the specified sampling percentage.

To log events at a specific spout or bolt level, navigate to the corresponding component page and click "Debug" under component actions:

Viewing Event Logs

Prerequisite: The Storm "logviewer" process should be running so that you can view the logged tuples. If it is not already running, start the log viewer by running the following command from the storm installation directory:

bin/storm logviewer

To view tuples:

From the Storm UI, navigate to the specific spout or bolt component page.
Click on the "events" link in the Debug column of the component summary. This will open a view similar to the following:
Each line in the event log contains an entry corresponding to a tuple emitted from a specific spout or bolt, presented in a comma-separated format:
```
Timestamp, Component name, Component task-id, MessageId (incase of anchoring), List of emitted values
```
Navigate between different pages to view logged events.

Accessing Event Logs on a Secure Cluster

If you want to view logs in secure mode, ensure that your browser is configured to use authentication with all supervisor nodes running logviewer. This process is similar to the process used to configure access to the Storm UI on a secure cluster (described in Configuring the Storm UI).

Add domains to the white list by setting network.negotiate-auth.trusted-uris to a comma-separated list containing one or more domain names and URLs. For example, the following steps configure Firefox to use authentication with two nodes:

Go to the about:config configuration page.
Search for the network.negotiate-auth.trusted-uris configuration option.
Double-click on the option.
An "Enter string value" dialog box opens.
In this box, enter the values http://node1.example.com, http://node2.example.com.
Click OK to finish.
Close and relaunch the browser.

If your cluster is not managed by Ambari, refer to the UI/Logviewer section of Apache Storm security documentation for additional configuration guidelines.

Disabling Event Logs

To disable event logging for a component or for a topology, click "Stop Debug" under the Component actions or Topology actions page (respectively) in the Storm UI.

The following example disables topology "test-topology":

Extending Event Logging

The Storm eventlogger bolt uses the IEventLogger interface to log events. The default implementation is aFileBasedEventLogger, which logs events to a log file at logs/workers-artifacts/<topology-id>/<worker-port>/events.log.

To extend event logging functionality (for example, to build a search index or log events to a database), add an alternate implementation of the IEventLogger interface.

/**
      * EventLogger interface for logging the event info to a sink like log file or db
      * for inspecting the events via UI for debugging.
      */public interface IEventLogger {
          void prepare(Map stormConf, TopologyContext context);
          /**
          * Invoked when the {@link EventLoggerBolt} receives a tuple from the spouts or bolts that
          * have event logging enabled.
          *
          * @param e the event
          */
          void log(EventInfo e);
          /**
          * Invoked when the event logger bolt is cleaned up
          */
          void close();
          }

See JIRA STORM-954 for more details.