windows下kafka源码阅读环境搭建
[*]工具准备：jdk1.8,scala-2.11.11,gradle-3.1,zookeeper-3.4.5,kafka-0.10.0.1-src.tgz, kafka_2.11-0.10.0.1.tgz
[*]安装jdk
[*]安装scala
[*]搭建zookeeper
[*]kafka源码构建
解压kafka-0.10.0.1-src.tgz，命令行进行kafka-0.10.0.1-src，执行gradle idea
https://s5.运维网.com/oss/201711/05/e7b90ed54a50810c84f71b7b03bc4dbc.png-wh_500x0-wm_3-wmp_4-s_3555752997.png
注：
1)gradle idea，用国外的库构建的下载速度很慢，在进入项目里面，找到build.gradle文件的修改库为阿里镜像库
buildscript {
repositories {
maven{ url 'http://maven.aliyun.com/nexus/content/groups/public/'}
}
apply from: file('gradle/buildscript.gradle'), to: buildscript
dependencies {
// For Apache Rat plugin to ignore non-Git files
classpath "org.ajoberstar:grgit:1.5.0"
classpath 'com.github.ben-manes:gradle-versions-plugin:0.12.0'
classpath 'org.scoverage:gradle-scoverage:2.0.1'
}
}
allprojects {
apply plugin: 'idea'
apply plugin: 'eclipse'
apply plugin: "jacoco"
allprojects {
repositories {
maven{ url 'http://maven.aliyun.com/nexus/content/groups/public/'}
}
}
2)执行gradle idea时，报错
* What went wrong:
A problem occurred evaluating root project 'kafka-0.10.0.1-src'.
> Failed to apply plugin
> No such property: useAnt for class: org.gradle.api.tasks.scala.ScalaCompileOptions
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
解决方法：打开kafka-0.10.0.1-src目录下build.gradle文件
添加如下行
view plain copy
ScalaCompileOptions.metaClass.daemonServer = true
ScalaCompileOptions.metaClass.fork = true
ScalaCompileOptions.metaClass.useAnt = false
ScalaCompileOptions.metaClass.useCompileDaemon = false
7.用intellij idea打开kafka-0.10.0.1-src
https://s4.运维网.com/oss/201711/05/2e54b08ee3d7b2412c280f36f31cd0bd.png-wh_500x0-wm_3-wmp_4-s_2374849679.png
8.复制log4j.properties文件放到core模块的src/main/scala路径下，这样才能正常的显示日志
https://s4.运维网.com/oss/201711/05/2ed43d098b7cebdf2faa21876ad0dac4.png-wh_500x0-wm_3-wmp_4-s_3605728480.png
9.增加scala-sdk
https://s1.运维网.com/oss/201711/05/0c3b177e184622b505534f8879c596fc.png-wh_500x0-wm_3-wmp_4-s_88991643.png
10.配置启动参数
https://s4.运维网.com/oss/201711/05/12e18887649e11e1f9db69d044850dc6.png-wh_500x0-wm_3-wmp_4-s_1139839183.png
11.启动项目
https://s3.运维网.com/oss/201711/05/ce3716f9bb285fd7a644b66b2441c228.png-wh_500x0-wm_3-wmp_4-s_475031007.png
12.验证
解压kafka_2.11-0.10.0.1.tgz文件，命令行进入bin/windows
1）创建topic
C:\webserver\kafka_2.11-0.10.0.1\bin\windows>kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --
partitions 1 --topic testMsg
Created topic "testMsg".
控制台日志如下：
https://s5.运维网.com/oss/201711/05/38fb278207d888d2f482d0fced205b5d.png-wh_500x0-wm_3-wmp_4-s_3427900052.png
2）执行生产者命令，产生消息：
https://s5.运维网.com/oss/201711/05/d95d2efbc0fb9428444026348165dd22.png-wh_500x0-wm_3-wmp_4-s_3822918000.png
3）执行消费者命令，消费消息：
https://s1.运维网.com/oss/201711/05/c81d90a057bfd0c7c3221db7726f6f21.png-wh_500x0-wm_3-wmp_4-s_2851271549.png
问题：
ERROR Processor got uncaught exception. (kafka.network.Processor)
java.nio.BufferUnderflowException
解决方案：
这是客户端的jar包和集群的版本号在小版本不一致导致，保持版本号一致即可。
scala-2.11.11,kafka-0.10.0.1-src.tgz, kafka_2.11-0.10.0.1.tgz
页:
[1]