Kafka(1)Install on Redhat and Try Java Client

tanzdorttn

Kafka(1)Install on Redhat and Try Java Client

1. Install and Try Kafka
>wget http://people.apache.org/~nehanarkhede/kafka-0.7.0-incubating/kafka-0.7.0-incubating-src.tar.gz
>tar zxvf kafka-0.7.0-incubating-src.tar.gz
>mv kafka-0.7.0-incubating-src /opt/tools/kafka-0.7.0
>cd /opt/tools/kafka-0.7.0
>./sbt update
>./sbt package

2. Start the Server and try messages
Start the zookeeper first
>bin/zookeeper-server-start.sh config/zookeeper.properties

>bin/kafka-server-start.sh config/server.properties

We can send the message from producer and consumer
Producer, we type in this window
>bin/kafka-console-producer.sh --zookeeper localhost:2181 --topic test

Consumer, we got message which we typed.
>bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning

3. Try to Run JAVA Client
First problem, how to find the jar package for kafka. This is a problem. There is not easy way like maven or pom to do that.

Kafka is not build via maven, there is a build tool named sbt. Simple Build Tool. I have no time to learn this right now. So I just use
this command to download and build all the related jar packages.
>./sbt release-zip

There is example using java, it is under this directory /opt/tools/kafka-0.7.0/examples/bin. How do I know where the jar packages live in classpath? Check this in the file java-producer-consumer-demo.sh and java-simple-consumer-demo.sh.

I download the jar package from there directories:
$base_dir/project/boot/scala-2.8.0/lib/*.jar
$base_dir/core/lib_managed/scala_2.8.0/compile/*.jar
$base_dir/core/lib/*.jar
$base_dir/core/target/scala_2.8.0/*.jar
$base_dir/examples/target/scala_2.8.0/*.jar

Update these related jar package to my local maven repository.

The 2 demos can run well on redhat server, localhost.

The POM.xml will be as follow:
<dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.10</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>kafka</groupId>
      <artifactId>kafka</artifactId>
      <version>0.7.0</version>
    </dependency>
    <dependency>
        <groupId>org.scala-lang</groupId>
        <artifactId>scala-library</artifactId>
        <version>2.8.0.RC7</version>
    </dependency>
<dependency>
  <groupId>log4j</groupId>
  <artifactId>log4j</artifactId>
  <version>1.2.16</version>
</dependency>
<dependency>
  <groupId>org.I0Itec</groupId>
  <artifactId>zkclient</artifactId>
  <version>20110412</version>
</dependency>
<dependency>
  <groupId>org.apache.zookeeper</groupId>
  <artifactId>zookeeper</artifactId>
  <version>3.3.3</version>
</dependency>

references:
http://www.54chen.com/java-ee/linkedin-kafka-usage.html
http://incubator.apache.org/kafka/
http://incubator.apache.org/kafka/quickstart.html