KafkaSpout Limitations
Limitations
The current version of the Kafka spout contains the following limitations:
Does not support Kafka 0.7x brokers.
Storm developers must include
${STORM_HOME}/lib/*
in the CLASSPATH environment variable from the command line when runningkafka-topology
in local mode. Otherwise, developers will likely receive ajava.lang.NoClassDefFoundError
exception:java -cp "/usr/lib/storm/contrib/storm-kafka-example-0.9.1.2.1.1.0-320-jar-with-dependencies.jar: /usr/lib/storm/lib/*" org.apache.storm.kafka.TestKafkaTopology <zookeeper_host>
Secure Hadoop clusters must comment out the following statement from
${STORM_HOME}/bin/kafka-server-start.sh
:EXTRA_ARGS="-name kafkaServer -loggc"
Core-storm API Constructor
public SpoutConfig(BrokerHosts hosts, String topic, String zkRoot, String id)
Table 5.5. SpoutConfig Parameters
SpoutConfig Parameter
Description
hosts
Any implementation of the
BrokerHosts
interface, currently eitherZkHosts
orStaticHosts
.topic
Name of the Kafka topic.
zkroot
Root directory in ZooKeeper where all topics and partition information is stored. By default, this is
/brokers
.id
Unique identifier for this spout.
Trident API Constructors
public TridentKafkaConfig(BrokerHosts hosts, String topic) public TridentKafkaConfig(BrokerHosts hosts, String topic, String id)
Table 5.6. TridentKafkaConfig Parameters
TridentKafkaConfig
Description
hosts
Any implementation of the
BrokerHosts
interface, currently eitherZkHosts
orStaticHosts
.topic
Name of the Kafka topic.
clientid
Unique identifier for this spout.