Git+Gerrit+Jenkins 持续集成系统安装与升级
Git版本机环境迁移与升级Git版本机环境迁移与升级说明技术组
文档修订记录
日期修订内容修订人版本备注2017-12-20首次编写陈超允V1.0初始化
[*]准备工作
[*]备份gerrit目录与数据库
#登陆203
#备份gerrit目录,代码库在GERRIT_HOME/Review/git下
# cd /home/
# tar -zcvf gerrit.tar.gz gerrit/
# mkdir -p /home/dev0/
# mv gerrit.tar.gz /home/dev0/#备份数据库# mysqldump –ugerrituser -p gerritdb>/home/dev0/reviewdb.sql
# ls
gerrit.tar.gzreviewdb.sql
[*]
[*]备份 Jenkins
#确定JENKINS_HOME=/home/.jenkins
#由于以往jenkins 开发构建时,保留所有构建记录,不能立即打包
#两种做法,一是删除以往构建,二是打包时跳过历史构建目录,这里选择删除以往构建
# cd /home/.jenkins/jobs/
# rm -rf ./*/builds/* ./*/workspace/* ./*/workspace/.git*
#修改jenkins项目构建配置为
#修改完成后对JENKINS_HOME打包# tar -zcvf jenkins-home.tar.gz .jenkins/# mv jenkins-home.tar.gz dev0/# cd dev0/# lsgerrit.tar.gzjenkins-home.tar.gzreviewdb.sql#传送到云服务器# scp * ccy001:/u01/soft/
[*]
[*]备份私服库
#登陆192.168.1.111
# pwd
/u01/mw/mvn
# ls
nexusnexus-2.11.2-03sonatype-work
# tar -zcvf nexus-2.11.2-03.tar.gz nexus-2.11.2-03/
# tar -zcvf sonatype-work.tar.gz sonatype-work/
#传送
# scp nexus-2.11.2-03.tar.gz sonatype-work.tar.gz ccy001:/u01/soft
[*]安装步骤
[*]JDK
# mkdir -p /u01/jdk/
# cp /u01/soft/jdk1.8.0_131.tar.gz /u01/jdk/
# cd /u01/jdk/
# ls
jdk1.8.0_131.tar.gz
# tar -zxvf jdk1.8.0_131.tar.gz
# rm -rf jdk1.8.0_131.tar.gz
#配置环境变量,添加如下内容
# vim /etc/profile
export JAVA_TOOL_OPTIONS=-Dfile.encoding=UTF8
export JAVA_HOME=/u01/jdk/jdk1.8.0_131
export JAVA_BIN=$JAVA_HOME/bin
export PATH=$PATH:$JAVA_HOME/bin
export MAVEN_HOME=/u01/mvn/apache-maven-3.3.9
export GIT_HOME=/usr/local/git/bin
export GIT_CORE=/usr/local/git/libexec/git-core
export>
export JAVA_HOME JAVA_BIN PATH> export PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:$JAVA_HOME/bin:/root/bin
export PATH=$PATH:$GIT_HOME:$MAVEN_HOME/bin:$GIT_CORE
# source /etc/profile
# java -version
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
java version "1.8.0_131"
#创建软链接
# ln -s /u01/jdk/jdk1.8.0_131/bin/java /usr/bin/java
[*]
[*]Maven 安装
# mkdir -p /u01/mvn
# cp /u01/soft/apache-maven-3.3.9.tar.gz /u01/mvn/
# cd /u01/mvn/
# ls
apache-maven-3.3.9.tar.gz
# tar -zxvf apache-maven-3.3.9.tar.gz
#修改setting 为云环境上私服地址,替换192.168.1.111为dev0
# cd /u01/mvn/apache-maven-3.3.9/conf/
# vim settings.xml
#vim 执行 :g/192.168.1.111/s//dev0/g
# mvn -v
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
Apache Maven 3.3.9…
#创建软链接
# ln -s /u01/mvn/apache-maven-3.3.9/bin/mvn /usr/bin/mvn
[*]
[*]Git 安装
# unzip v2.3.0.zip
# cd git-2.3.0/
#安装git依赖包
# yum -y install curl-devel expat-devel gettext-devel openssl-devel zlib-devel gcc perl-ExtUtils-MakeMaker
# make prefix=/usr/local/git all
# make prefix=/usr/local/git install
# git --version
git version 2.3.0
#创建软链接
# ln -s /usr/local/git/bin/git /usr/bin/git
[*]
[*]Nginx 安装
# cd /u01/soft/
# wget http://nginx.org/download/nginx-1.12.2.tar.gz
# tar -zxvf nginx-1.12.2.tar.gz
# cd nginx-1.12.2
#安装依赖包
#yum -y install openssl openssl-devel
#yum -y install pcre* zlib*
#yum install -y gcc gcc-c++
#yum install GeoIP
#./configure --with-stream
#make && make install
#./configure --with-http_stub_status_module --with-stream --with-http_geoip_module --with-http_ssl_module --with-http_realip_module
#make && make install
#安装完的目录/usr/local/nginx
#创建启停重启脚本
# cd /usr/local/nginx/
# cat startNginx.sh
cd /usr/local/nginx
/usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf
# cat stopNginx.sh
pkill -9 nginx
# cat restartNginx.sh
cd /usr/local/nginx/sbin/
./nginx -s>
[*]
[*]
[*]设为系统服务自启
# cat /etc/init.d/nginx
#!/bin/sh
#
# Startup script for the nginx Continuous Integration server
# (via Jakarta Tomcat Java Servlets and JSP server)
#
# chkconfig: - 85 15
# description: Jakarta nginx Java Servlets and JSP server
# processname: nginx
# pidfile: /u01/app/nginx/.pid
export JAVA_TOOL_OPTIONS=-Dfile.encoding=UTF8
EXCUTE_USER=root
APP_NAME=nginx
LOCKFILE=/var/lock/$APP_NAME
APP_HOME=/usr/local/nginx
case "$1" in
start)
echo -n "Starting $APP_NAME: "
su -p -s /bin/sh $EXCUTE_USER -c "$APP_HOME/start*.sh"
RETVAL=$?
echo
[ $RETVAL = 0 ] && touch $LOCKFILE
;;
stop)
echo -n "Shutting down $APP_NAME: "
su -p -s /bin/sh $EXCUTE_USER -c "$APP_HOME/stop*.sh"
RETVAL=$?
echo
[ $RETVAL = 0 ] && rm -f $LOCKFILE
;;
restart)
$0 stop
$0 start
;;
condrestart)
[ -e $LOCKFILE ] && $0 restart
;;
status)
su -p -s /bin/sh $EXCUTE_USER -c "ps -ef |grep $APP_NAME"
;;
*)
echo "Usage: $0 {start|stop|restart|status}"
exit 1
esac
exit 0
#设置服务
#chmod a+x /etc/init.d/nginx
#chkconfig nginx on
# chkconfig --list |grep nginx
nginx 0:关闭 1:关闭 2:启用 3:启用 4:启用 5:启用 6:关闭
#service nginx status
#nginx 配置后续再做
[*]
[*]GitWeb安装
# yum -y install gitweb
#yum -yinstall perl-Time-HiRes
#yum -y install perl-CPAN
#yum -y install perl-CGI
#配置gitweb 展示的那些代码库的根目录
# vim /etc/gitweb.conf
$projectroot = "/u01/gerrit/Review/git"
#创建软链接
# mkdir -p /usr/lib/cgi-bin/
# ln -s /usr/local/git/share/gitweb/gitweb.cgi /usr/lib/cgi-bin/gitweb.cgi
[*]
[*]MySQL 安装
# mkdir -p /u01/mysql
# cd /u01/mysql/
# cp /u01/soft/mysql-5.7.17-linux-glibc2.5-x86_64.tar.gz ./
# tar -zxvf mysql-5.7.17-linux-glibc2.5-x86_64.tar.gz
# rm -rf mysql-5.7.17-linux-glibc2.5-x86_64.tar.gz
# cd mysql-5.7.17-linux-glibc2.5-x86_64/
# mkdir /u01/mysql/logs /u01/mysql/data
#先将配置好的my.conf 文件传至服务器
#直接用root用户装,修改user=root
# vim /etc/my.cnf
user=root
# mysql-5.7.17-linux-glibc2.5-x86_64/bin/mysqld --initialize --user=root --basedir=/u01/mysql --datadir=/u01/mysql/data--innodb_undo_tablespaces=3 --explicit_defaults_for_timestamp
#此处需要注意记录生成的临时密码,如:root@localhost: jh7n;CSoXbgh
# mysql-5.7.17-linux-glibc2.5-x86_64/bin/mysql_ssl_rsa_setup --datadir=/u01/mysql/data/
#查看默认密码
#cat /u01/mysql/logs/mysql-error.log
# cp support-files/mysql.server /etc/init.d/mysql
# chkconfig --add mysql
# chkconfig mysql on
# chkconfig --list | grep mysql
mysql 0:off 1:off 2:on 3:on 4:on 5:on 6:off
# vim /etc/init.d/mysql
basedir=/u01/mysql/mysql-5.7.17-linux-glibc2.5-x86_64
datadir=/u01/mysql/data
#配置环境变量
# vim /etc/profile
export MYSQL_HOME=/u01/mysql/mysql-5.7.17-linux-glibc2.5-x86_64/bin
#创建软链接
# ln -s /u01/mysql/mysql-5.7.17-linux-glibc2.5-x86_64/bin/mysqld /usr/bin/mysqld
#登陆mysql
# mysql --user=root -p
mysql> SET PASSWORD=PASSWORD('password');
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
# 添加远程访问权限
mysql> use mysql;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> update user set host = '%' where user = 'root';
Query OK, 1 row affected (0.01 sec)
#重启
# service mysql restart
[*]
[*]
[*]导入gerrit数据库脚本
mysql>CREATE DATABASE IF NOT EXISTS reviewdb DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
mysql> use reviewdb;
mysql> source /u01/soft/dev0/reviewdb.sql
[*]
[*]
[*]Mysql反向代理配置
#配置mysql 反向代理流
# pwd
/usr/local/nginx
# vim conf/nginx.conf
#gerrit-mysql
stream {
server {
listen 3300;
proxy_pass dev0:3306;
proxy_buffer_size 512k;
allow all;
}
}
#重启nginx
# sh restartNginx.sh
#用navicat 远程连接测试是否能连上
[*]
[*]Gerrit 安装
#解压从203备份的gerrit tar包
# cp soft/dev0/gerrit.tar.gz ./
# tar -zxvf gerrit.tar.gz
#修改配置
# cd /u01/gerrit/Review/
# cat etc/gerrit.config
basePath = git
canonicalWebUrl = http://39.108.0.229/gerrit
type = mysql
hostname = localhost
database = reviewdb
username = root
type = LUCENE
type = HTTP
smtpServer = smtp.qq.com
smtpServerPort = 465
smtpEncryption = ssl
smtpUser = 474849750@qq.com
smtpPass = ybvnzfujouqocaaa
sslVerify = false
from=CodeReview<474849750@QQ.COM>
cgi = /usr/local/git/share/gitweb/gitweb.cgi
user = root
javaHome = /u01/jdk/jdk1.8.0_131/jre
listenAddress = *:29418
listenUrl = http://*:8080/gerrit
directory = cache
#重启失败,重新更新下索引
# cd Review/bin/
# java -jar gerrit.war reindex -d /u01/gerrit/Review
#安装gerrit为系统服务,开机自启
# vim /etc/init.d/gerrit
#!/bin/bash
#chkconfig:2345 20 90
#description:jenkins
#processname:jenkins
export JAVA_HOME=/u01/jdk/jdk1.8.0_131
case $1 in
start) su root /u01/gerrit/start-gerrit.sh;;
stop) su root/u01/gerrit/stop-gerrit.sh;;
restart) su root /u01/gerrit/restart-gerrit.sh;;
status)su -p -s /bin/sh root -c "ps -ef |grep gerrit";;
*) echo "require start|stop|restart|status" ;;
esac
# chmod a+x /etc/init.d/gerrit
# chkconfig gerrit on
# chkconfig --list |grep gerrit
gerrit 0:off 1:off 2:on 3:on 4:on 5:on 6:off
[*]
[*]
[*]配置gerrit nginx反向代理
#上传http 认证文件
# pwd
/usr/local/nginx/gerrit
# cp /u01/soft/dev0/gerrit2.passwd ./
#配置gerritserver
# vim conf/nginx.conf
location /gerrit{
#root /home/gerrit/Review;
proxy_pass http://dev0:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_headerX-Forwarded-For$proxy_add_x_forwarded_for;
auth_basic "Gerrit Code Review";
auth_basic_user_file /usr/local/nginx/gerrit/gerrit2.passwd;
}
#另外还需配置gerrit 端口 29418 的代理
#stream tcp
stream {
#gerrit-mysql
server {
listen 3300;
proxy_pass dev0:3306;
proxy_buffer_size 512k;
allow all;
}
#gerrit-ssh
server{
listen 29418;
proxy_buffer_size 512k;
proxy_pass dev0:29418;
allow all;
}
}
#最后配置腾讯云增加入站规则TCP:29418
[*]
[*]
[*]配置gitweb 权限
在单独项目的ACL里为项目的用户组增加了对refs/meta/config的read权限
[*]
[*]Jenkins 安装
#解压jenkins tar包
#修改启停脚本,改变jenkins 家目录
# cat start-jenkins.sh
#!/bin/bash
export JAVA_TOOL_OPTIONS=-Dfile.encoding=UTF8
export JENKINS_HOME=/u01/jenkins/.jenkins
cd /u01/jenkins
nohup /u01/jdk/jdk1.8.0_131/bin/java -Dfile.encoding=utf-8 -Dhudson.util.ProcessTree.disable=true -jar jenkins.war --httpPort=8081 >> system_out.log 2>&1 &
echo $!> /u01/jenkins/.pid
echo "===jenkins started!"
#解压 jenkins 配置 tar包
# tar -zxvf jenkins-home.tar.gz
#跟上述做法相同,将jenkins 作为系统服务
# vim /etc/init.d/Jenkins#!/bin/sh## Startup script for the nginx Continuous Integration server## chkconfig: - 85 15# description: Jakarta nginx Java Servlets and JSP server# processname: tomcat# pidfile: /var/run/nginx.pidJENKINS_USER=rootLOCKFILE=/var/lock/jenkinsAPP_NAME=jenkinsJENKINS_HOME=/u01/jenkinscase "$1" instart) echo -n "Starting $APP_NAME: " su -p -s /bin/sh $JENKINS_USER -c "$JENKINS_HOME/start-jenkins.sh" RETVAL=$? echo [ $RETVAL = 0 ] && touch $LOCKFILE ;;stop) echo -n "Shutting down $APP_NAME: " su -p -s /bin/sh $JENKINS_USER -c "$JENKINS_HOME/stop-jenkins.sh" RETVAL=$? echo [ $RETVAL = 0 ] && rm -f $LOCKFILE ;;restart) $0 stop $0 start ;;condrestart) [ -e $LOCKFILE ] && $0 restart ;;status) su -p -s /bin/sh $JENKINS_USER -c "ps -ef |grep $APP_NAME" ;;*) echo "Usage: $0 {start|stop|restart|status}" exit 1esacexit 0# chmod a+x /etc/init.d/jenkins# chkconfig jenkins on# chkconfig --list |grep Jenkins#创建公钥# ssh-keygen#启动jenkins
[*]
[*]
[*]Jenkins nginx反向代理配置
# cat conf/nginx.conf
#端口8081
server {
listen 80;
server_namelocalhost;
location /{
client_max_body_size 30m;
client_body_buffer_size 128k;
proxy_buffer_size 26400k;
proxy_buffers 32 25000k;
proxy_busy_buffers_size 33280k;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://dev0:8081;
access_log logs/jenkins_access.log;
}
location /gerrit-trigger{
client_max_body_size 30m;
client_body_buffer_size 128k;
proxy_buffer_size 26400k;
proxy_buffers 32 25000k;
proxy_busy_buffers_size 33280k;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://dev0:8081/gerrit-trigger;
access_log logs/jenkins_access.log;
}
}
#登陆jenkins 更改jenkins地址
#端口8081后续经测试,不能正常跳转。改为80后ok
[*]
[*]
[*]Jenkins 配置修改
[*]系统设置àgerrit triggeràgerritServerà Gerrit Connection Setting
因为gerrit与jenkins装在同一台机子上,HostName:127.0.0.1; Frontend URL:http://127.0.0.1:8080/
[*]配置公网ip(pelican)从节点,用于发布上线包脚本
[*]pelican配置dev0免密登陆认证
[*]节点配置
[*]job上绑定slave-pelican
[*]
[*]
[*]jenkins job源码管理修改
(1)改为本地10.0.0.16,修改所有构建项目地址
Repository URL:ssh://jenkins@10.0.0.16:29418/Weidi_ccbc
(2)用jekins用户登陆gerrit,添加jenkins所在机器(dev0)的ssh-key
[*]Gerrit 升级说明
原203机子gerrit-2.8.1 最高升级至gerrit-2.11.10,再高的版本则导致eclipse gerrit插件不能使用,gerrit-web也不能使用 gitweb。
[*]
[*]Gerrit升级步骤
[*]上传gerrit-2.9.war、gerrit-2.10.war、gerrit-2.11.10.war 至dev0
# scp gerrit-2.10.war gerrit-2.9.war gerrit-2.11.10.war dev0:/u01/soft/
[*]依次安装 2.9-->2.10-->2.11
#java -jar gerrit-2.9.warinit -d /u01/gerrit/Review/
#java -jar gerrit-2.10.war init -d /u01/gerrit/Review/
#java -jar gerrit-2.11.10.war init -d /u01/gerrit/Review/
#安装提示
#Location of Git repositories : /home/review_site/git/
#Database server type : mysql
#Database name : gerritdb
#Database username : gerritusr
#Authentication method : http
#Behind reverse proxy ? y#Canonical URL : http://39.108.0.229/gerrit/
[*]
[*]升级日志
[*]
增加commit>
[*]增加 GitWeb 操作界面
[*]可在gerrit 的一次commit 上,在浏览器上增加或修改文件
[*]进入代码review 界面,去除换行符”\r”
[*]Review 时,可选择一段代码块,按下 c ,添加draft,选中代码块高亮
[*]Eclipse 配置修改
[*]
[*]Maven 地址修改
Nexus 地址: http://39.108.0.229:8081/nexus/WindowsàpreferresàMavenàUser settingà
点击 open fileà将原来的192.168.1.111 换成39.108.0.229保存完,点击 Update settingsàReindexàApply
[*]
[*]
[*] Gerrit 配置修改
WindowàShow ViewàOtheràMylynàTeam repositoryà
#原来192.168.1.203的可以右键àdisconnected右键ànew repositoryàTask repositoryàGerrit Code ReviewàServer: http://39.108.0.229/gerritFinish 即可.#删除原有项目ImportàProject from gitàgerritàtihu-gerritàrefresh选择要导入项目。#此步选择cancel重新选择 importàExisting Maven Projectà找到刚刚clone 下来的项目导入即可
[*]安装问题记录
[*]Gerrit提交信息乱码
#查看gerrit 数据库编码
mysql> show variables like '%character%';
| character_set_server | latin1 |
首先保证服务器的系统编码是支持中文的
# cat /etc/sysconfig/i18n
LANG="zh_CN.UTF-8"
#永久性修改mysql
# vim /etc/my.cnf
.......
......
character_set_server = utf8 【添加这一行内容】
#重启
# service mysql restart
[*]
[*]Gerrit 发送邮件异常
# vim Review/etc/gerrit.config
smtpServer = smtp.qq.com
smtpServerPort = 465
smtpEncryption = ssl
smtpUser = 474849750@qq.com
smtpPass = ybvnzfujouqocaaa
sslVerify = false
from=CodeReview474849750@qq.com
[*]
[*]Gerrit 做review时,The page you requested was not found, or you do not have permission to view
原因:gerrit nginx配置代理时,去掉反斜杠 /
[*]
[*]User jenkins has no capability to connect to Gerrit event stream
解决方案:
管理员登录gerrità ProjectsàListàAll-Projectsà ProjectsàAccess
à Global CapabilitiesàStream Events 点击 Non-Interactive Usersà
à添加 jenkins 用户到 ‘Non-Interactive Users’ 组
[*]
[*]GitWeb "Can't locate Time/HiRes.pm in @INC"问题
yum -yinstall perl-Time-HiRes
yum -y install perl-CPAN
yum -y install perl-CGI
[*]
[*]Gerrit的git unpack error问题
#解决的办法就是到reviewdb数据库里,先从patch_sets表中找出change_id是 对应commitid的记录,确定无误后
我们要做的是将其关闭,并把状态置为Abandon。(友情提醒,涉及重要业务,注意备份数据库)
mysql> update changes set open='N',status='A' where change_id=1974;
[*]
[*]jenkins build的时候拿的不是新提交的代码
源码管理à 选择git,Refspec:refs/changes/*:refs/changes/*Branch Specifier (blank for 'any'):$GERRIT_REFSPEC
[*]
[*]jenkins 配置slave的时候命令找不到
#把要用的命令创建一个快捷方式到/usr/bin,如
ln -s /usr/local/git/bin/git /usr/bin/git
ln -s /u01/mvn/apache-maven-3.3.9/bin/mvn /usr/bin/mvn
ln -s /u01/jdk/jdk1.8.0_131/bin/java /usr/share/java
ln -s /u01/mvn/apache-maven-3.3.9/bin/mvn /usr/bin/mvn
ln -s /usr/local/git/bin/git /usr/bin/git
[*]
[*]com.jcraft.jsch.JSchException: Auth fail,User: jenkins KeyFile: /root/.ssh/id_rsa
原因是新安装的gerrit jenkins用户没有配置主机的ssh key,登陆jenkins用户到gerrit,添加SSH Public key 就好了
[*]
[*]jenkins 打包构建时,杀掉jenkins脚本里执行的进程,解决办法
解决办法:
[*]启动 Jenkins 的时候加上 -Dhudson.util.ProcessTree.disable=true,也就是 java -Dhudson.util.ProcessTree.disable=true -jar jenkins.war
[*]临时改变BUILD_ID值,使得jenkins不会找到并结束掉run.sh启动的后台进程,
OLD_BUILD_ID=$BUILD_ID
echo $OLD_BUILD_ID
BUILD_ID=dontKillMe
./run.sh restart
#改回原来的BUILD_ID值
BUILD_ID=$OLD_BUILD_ID
echo $BUILD_ID
[*]
[*]git 克隆项目出现no matching key exchange method found. Their offer: diffie-hellman-group1-sha1
解决办法:在window上用git bash执行
vim ~/.ssh/config
#插入Host * KexAlgorithms +diffie-hellman-group1-sha1珠海数通天下科技有限公司
页:
[1]