金庸武功之“北冥神功”
一。拓扑环境：IP:192.168.93.202docker registry服务器
IP:192.168.93.201docker client服务器
关闭selinux及防火墙
修改/etc/hosts文件
192.168.93.202 docker.shengjing.com
hostnamectl set-hostname docker.shengjing.com(永久修改主机名）
安装依赖的软件包：
yum install gcc make pcre-devel pcre openssl-devel httpd-tools zlib-devel -y
生成根密钥：
先把
/etc/pki/CA/cacert.pem
/etc/pki/CA/index.txt
/etc/pki/CA/index.txt.attr
/etc/pki/CA/index.txt.old
/etc/pki/CA/serial
/etc/pki/CA/serial.old
删除掉!
cd /etc/pki/CA
(umask 077; openssl genrsa -out /etc/pki/CA/private/cakey.pem 4096)
# (umask 077; openssl genrsa -out /etc/pki/CA/private/cakey.pem 4096)
Generating RSA private key, 2048 bit long modulus
..................................................................................................................................+++
.......................................................+++
e is 65537 (0x10001)
生成根证书
openssl req -new -x509 -key /etc/pki/CA/private/cakey.pem -out /etc/pki/CA/cacert.pem -days 365
Generating RSA private key, 2048 bit long modulus
..................................................................................................................................+++
.......................................................+++
e is 65537 (0x10001)
# openssl req -new -x509 -key private/cakey.pem -out cacert.pem
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) :CN
State or Province Name (full name) []:beijing
Locality Name (eg, city) :beijing
Organization Name (eg, company) :sjwl
Organizational Unit Name (eg, section) []:sjwl
Common Name (eg, your name or your server's hostname) []:docker.shengjing.com
Email Address []:mengxl@shengjing360.com
会提示输入一些内容，因为是私有的，所以可以随便输入，最好记住能与后面保持一致,特别是"Common Name”。必须要和hostname显示的一致。mengxl@shengjing360.com要记住！
上面的自签证书cacert.pem应该生成在/etc/pki/CA下
# ls
cacert.pemcertscrlnewcertsprivate
为nginx web服务器生成ssl密钥
mkdir /usr/local/nginx/ssl
cd /usr/local/nginx/ssl
#(umask077; opensslgenrsa -out/usr/local/nginx/ssl/nginx.key2048)
Generating RSA private key, 2048 bit long modulus
.....................................................+++
....................................................................................................................+++
e is 65537 (0x10001)
查看nginx服务器的密钥
# ls
nginx.key
为nginx生成证书签署请求
执行 openssl req -new -key /usr/local/nginx/ssl/nginx.key -out /usr/local/nginx/ssl/nginx.csr
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) :CN
State or Province Name (full name) []:beijing
Locality Name (eg, city) :beijing
Organization Name (eg, company) :sjwl
Organizational Unit Name (eg, section) []:sjwl
Common Name (eg, your name or your server's hostname) []:docker.shengjing.com
Email Address []:mengxl@shengjing360.com
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
同样会提示输入一些内容，Commone Name一定要是你要授予证书的服务器域名或主机名，challenge password不填。
私有CA根据请求来签发证书
# touch /etc/pki/CA/index.txt
# touch /etc/pki/CA/serial
# echo 00 > /etc/pki/CA/serial
执行
# openssl ca-in/usr/local/nginx/ssl/nginx.csr-out/etc/pki/CA/certs/nginx.crt-days365
Using configuration from /etc/pki/tls/openssl.cnf
Check that the request matches the signature
Signature ok
Certificate Details:
Serial Number: 0 (0x0)
Validity
Not Before: Nov7 16:28:22 2016 GMT
Not After : Nov7 16:28:22 2017 GMT
Subject:
countryName = CN
stateOrProvinceName = beijing
organizationName = sjwl
organizationalUnitName = sjwl
commonName = docker.shengjing.com
emailAddress = mengxl@shengjing360.com
X509v3 extensions:
X509v3 Basic Constraints:
CA:FALSE
Netscape Comment:
OpenSSL Generated Certificate
X509v3 Subject Key Identifier:
04:91:10:DD:9E:37:81:66:5E:66:E4:CE:EB:02:E0:D3:27:FC:F7:7B
X509v3 Authority Key Identifier:
keyid:C8:F9:00:19:C0:61:7E:71:B8:16:FD:08:43:AD:82:F7:9E:BC:20:91
Certificate is to be certified until Nov7 16:28:22 2017 GMT (365 days)
Sign the certificate? :y
1 out of 1 certificate requests certified, commit? y
Write out database with 1 new entries
Data Base Updated
同样会提示输入一些内容，选择y就可以了!
查看nginx的证书
# cd /etc/pki/CA/certs/
# ls
nginx.crt
cp/etc/pki/CA/certs/nginx.crt /usr/local/nginx/ssl/
查看证书中的信息：
~]# opensslx509-in /etc/pki/CA/certs/nginx.crt-noout-serial-subject
安装,配置,运行nginx
(1) 添加组和用户
# groupadd www -g 58
# useradd -u 58 -g www www
cd /usr/local/src
wget http://nginx.org/download/nginx-1.11.2.tar.gz
tar -zxvf nginx-1.11.2.tar.gz
cdnginx-1.11.2
# ./configure --user=www --group=www --prefix=/usr/local/nginx--with-http_ssl_module --with-http_stub_status_module--with-pcre--with-http_addition_module --with-http_realip_module--with-http_flv_module
# make && make install
vi /usr/local/nginx/conf/nginx.conf
userwww;
worker_processes4;
#error_loglogs/error.log;
#error_loglogs/error.lognotice;
#error_loglogs/error.loginfo;
pid logs/nginx.pid;
events {
worker_connections4096;
}
http {
include mime.types;
default_typeapplication/octet-stream;
#log_formatmain'$remote_addr - $remote_user [$time_local] "$request" '
# '$status $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';
#access_loglogs/access.logmain;
sendfile on;
#tcp_nopush on;
#keepalive_timeout0;
keepalive_timeout65;
#gzipon;
upstream registry {
server 192.168.93.202:5000;
}
server {
listen 80;
server_namelocalhost;
#charset koi8-r;
#access_loglogs/host.access.logmain;
location / {
root html;
indexindex.html index.htm;
}
#error_page404 /404.html;
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504/50x.html;
location = /50x.html {
root html;
}
}
server {
listen 443 ;
server_namedocker.shengjing.com;
add_header Docker-Distribution-Api-Version registry/2.0 always;
ssl on;
ssl_certificate /usr/local/nginx/ssl/nginx.crt;
ssl_certificate_key/usr/local/nginx/ssl/nginx.key;
ssl_session_cache shared:SSL:1m;
ssl_session_timeout5m;
# ssl_ciphersHIGH:!aNULL:!MD5;
# ssl_prefer_server_cipherson;
location / {
auth_basic "registry";
auth_basic_user_file /usr/local/nginx/conf/.htpasswd; #htpasswd的位置
root html;
indexindex.html index.htm;
#proxy_pass http://registry;
#proxy_set_headerHost $http_host;
proxy_set_headerX-Real-IP $remote_addr;
proxy_set_headerAuthorization"";
client_body_buffer_size 128k;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffer_size 8k;
proxy_buffers 4 32k;
proxy_busy_buffers_size 64k;#如果系统很忙的时候可以申请更大的proxy_buffers 官方推荐*2
proxy_temp_file_write_size64k;#proxy缓存临时文件的大小
}
location /_ping {
auth_basic off;
proxy_pass http://registry;
}
location /v1/_ping {
auth_basic off;
proxy_pass http://registry;
}
}
}
验证配置：
/usr/local/nginx/sbin/nginx -t
启动nginx:
# /usr/local/nginx/sbin/nginx
# ss -ntpl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:22 *:* users:(("sshd",pid=841,fd=3))
LISTEN 0 100 127.0.0.1:25 *:* users:(("master",pid=1408,fd=13))
LISTEN 0 128 *:443 *:* users:(("nginx",pid=4796,fd=6),("nginx",pid=4795,fd=6),("nginx",pid=4794,fd=6),("nginx",pid=4793,fd=6),("nginx",pid=4792,fd=6))
LISTEN 0 128 :::22 :::* users:(("sshd",pid=841,fd=4))
LISTEN 0 100 ::1:25 :::* users:(("master",pid=1408,fd=14))
# ps -ef | grep -i "nginx"
root 4792 10 12:07 ? 00:00:00 nginx: master process /usr/local/nginx/sbin/nginx
www 479347920 12:07 ? 00:00:00 nginx: worker process
www 479447920 12:07 ? 00:00:00 nginx: worker process
www 479547920 12:07 ? 00:00:00 nginx: worker process
www 479647920 12:07 ? 00:00:00 nginx: worker process
root 480419940 12:08 pts/0 00:00:00 grep --color=auto -i nginx
生成htpasswd
# htpasswd -cb /usr/local/nginx/conf/.htpasswd admin admin
浏览器中：https://docker.shengjing.com
输入用户：admin
密码： admin
出现nginx测试页面
当把这条#proxy_pass http://registry;的注释#去掉
页面就报502，因为后端代理docker registry还没有启动
至此证明https认证完成
###############################################################################
搭建docker registry V2
# systemctl stop docker
# vim /etc/sysconfig/docker
修改后：
OPTIONS='--selinux-enabled --insecure-registry dockertest.shengjing.com'
添加--insecure-registry docker.shengjing.com 红色的是私有仓库的域名，更前面生成密钥时候的域名对应
配置密钥文件
mkdir -p /etc/docker/certs.d/hc.docker.io
cp /etc/pki/CA/cacert.pem/etc/docker/certs.d//hc.docker.io/ca-certificates.crt
cat /etc/pki/CA/cacert.pem>> /etc/pki/tls/certs/ca-bundle.crt
#systemctl restart docker
#vim config.yml
version: 0.1
log:
level: debug
formatter: text
fields:
service: registry
environment: staging
storage:
delete:
enabled: true
cache:
layerinfo: inmemory
filesystem:
rootdirectory: /var/lib/registry
http:
addr: :5000
secret: admin
#mkdir data
docker run -d -p 5000:5000 --restart=always --name registry -v `pwd`/config.yml:/etc/docker/registry/config.yml -v `pwd`/data:/var/lib/registry registry:2
这里注意如果运行时报如下错误：
# docker run -d -p 5000:5000 --restart=always --name registry -v `pwd`/config.yml:/etc/docker/registry/config.yml -v `pwd`/data:/var/lib/registry registry:2
Unable to find image 'registry:2' locally
Trying to pull repository docker.io/library/registry ...
Get https://registry-1.docker.io/v2/library/registry/manifests/2: net/http: TLS handshake timeout
docker: Get https://registry-1.docker.io/v2/library/registry/manifests/2: net/http: TLS handshake timeout.
See '/usr/bin/docker-current run --help'.
没别的原因，就是网络问题，因为docker hub在国外，墙很厚啊，呵呵！多试几次就行了
# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.io/registry 2 c9bd19d022f6 2 weeks ago 33.27 MB
验证
# curl https://admin:admin@dockertest.xxxx.com/v2/
结果返回{}
登录
docker login https://dockertest.xxxx.com
Username: admin
Password: #admin
Email: abcd@qq.com
WARNING: login credentials saved in /root/.docker/config.json
Login Succeeded
验证push镜像到私有仓库
在登录的前题下，本机验证
# docker pull busybox
docker pull busybox
Using default tag: latest
Trying to pull repository docker.io/library/busybox ...
latest: Pulling from docker.io/library/busybox
56bec22e3559: Pull complete
Digest: sha256:29f5d56d12684887bdfa50dcd29fc31eea4aaf4ad3bec43daf19026a7ce69912
Status: Downloaded newer image for docker.io/busybox:latest
# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.io/registry 2 c9bd19d022f6 2 weeks ago 33.27 MB
docker.io/busybox latest e02e811dd08f 4 weeks ago 1.093 MB
# docker tag e02e811dd08f docker.shengjing.com/busybox:v1
# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.io/registry 2 c9bd19d022f6 2 weeks ago 33.27 MB
docker.io/busybox latest e02e811dd08f 4 weeks ago 1.093 MB
docker.shengjing.com/busybox v1 e02e811dd08f 4 weeks ago 1.093 MB
# docker push docker.shengjing.com/busybox:v1
The push refers to a repository
e88b3f82283b: Pushed
v1: digest: sha256:9393222c6789842b16bcf7306b6eb4b486d81a48d3b8b8f206589b5d1d5a6101 size: 505
# curl https://admin:admin@docker.shengjing.com/v2/_catalog
{"repositories":["busybox"]}
在client端测试
配置密钥
#scp /etc/pki/CA/cacert.pem root@192.168.93.201:/root/
在client上（192.168.93.201）
vi /etc/sysconfig/docker
OPTIONS='--selinux-enabled --insecure-registry docker.shengjing.com'
mkdir -pv /etc/docker/certs.d/docker.shengjing.com
cp /root/cacert.pem/etc/docker/certs.d/docker.shengjing.com/
cat /etc/docker/certs.d/docker.shengjing.com/cacert.pem > /etc/pki/tls/certs/ca-bundle.crt
vi /etc/hosts
192.168.93.202 docker.shengjing.com
重启下服务器
第一次我登陆时报错，如下：
# docker login https://docker.shengjing.com
Username: admin
Password:
Email:
Error response from daemon: invalid registry endpoint https://docker.shengjing.com/v0/: unable to ping registry endpoint https://docker.shengjing.com/v0/
v2 ping attempt failed with error: Get https://docker.shengjing.com/v2/: x509: certificate signed by unknown authority
v1 ping attempt failed with error: Get https://docker.shengjing.com/v1/_ping: x509: certificate signed by unknown authority. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add `--insecure-registry docker.shengjing.com` to the daemon's arguments. In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag; simply place the CA certificate at /etc/docker/certs.d/docker.shengjing.com/ca.crt
怎么折腾都不行，看错误说是证书问题，最后放大招，另一台client测试，没问题，出问题的服务器恢复快照后也没问题，奇了葩了。
正常应该是这样：
# docker login https://docker.shengjing.com
Username (admin): admin
Password:
WARNING: login credentials saved in /root/.docker/config.json
Login Succeeded
# curl https://admin:admin@docker.shengjing.com/v2/
{}#
到此，私有基于证书的ngix + registryv2 结束，累死老子了，最郁闷的快搭建完了才发现有个更牛逼的开源企业级私有registry，还是提供界面的中文的，哈哈，各位关注我接下来的博客吧
docker harbor
参考博客：http://www.mamicode.com/info-detail-1181975.html
页:
[1]