docker registry v2 搭建

lihu129c

          距离 Docker Registry 升级到V2已经有一年多，最近在尝试搭建v2的环境。虽说v2的存储结构变的更加合理以及安全，但是就目前的版本而言，感觉真是不是很好用！具体真正方便使用还有好长一段距离，先不管这个，我们先介绍搭建过程.
    搭建过程很曲折，看了很多的资料才成功，希望这篇记录能帮到各位。

注：本博客部分内容是复制的imkh（简书作者），来自 <http://www.jianshu.com/p/f2705a5da6a2> 。 如果侵权，请及时告知！






        环境：Fedora：23

[root@library ~]# cat /etc/redhat-release
Fedora release 23 (Twenty Three)
[root@library ~]# uname -a
Linux library 4.5.7-202.fc23.x86_64 #1 SMP Tue Jun 28 18:22:51 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Server 两台：
172.29.26.233 guiqiu-virtualbox
172.29.26.231 library

在 172.29.26.233 上架设registry，然后在 172.29.26.231上进行测试


需要安装的rpm：
yum install -y docker-compose
yum install -y docker
yum install -y httpd-tools

配置server的/etc/hosts ,为后面的ssl 做准备：
[root@guiqiu-virtualbox~]# cat /etc/hosts
127.0.0.1                localhost.localdomain
localhost
::1                localhost6.localdomain6 localhost6
172.29.26.233 guiqiu-virtualbox registry registry.com.cn
172.29.26.231 library  library.com.cn
[root@guiqiu-virtualbox~]#
[root@library~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
172.29.26.233 registry registry.com.cn guiqiu-virtualbox
172.29.26.231 library  library.com.cn
[root@library ~]#
在后面产生私有证书的时候，registry的comman name 我们使用registry.com.cn


创建一个工作目录，例如/data/progrmas/docker，并在该目录下创建docker-compose.yml文件，将以下docker-compose.yml内容复制粘贴到你的docker-compose.yml文件中。
内容大致意思为，基于“nginx”
image运行nginx容器，暴露容器443端口到host 443端口。并挂载当前目录下的nginx/目录为容器的/etc/nginx/config.d目录。
nginx link到registry容器。基于registry:2
image创建registry容器，将容器5000端口暴露到host
5000端口，使用环境变量指明使用/data为根目录，并将当前目录下data/文件夹挂载到容器的/data目录,其中registry使用外部的config.yml,也同样是通过docker volume 的功能挂载到registry内部中。

频繁使用docker volume挂载的功能是为，将配置，存储，container进行分离，方便以后升级迁移。
下面是配置文件的详细内容，请将其复制到自己的目录下即可

$ mkdir /data/programs/docker -p
$ cd /data/programs/docker
$ mkdir data && mkdir nginx && mkdir registry


[root@guiqiu-virtualbox~]# cat /data/programs/docker/docker-compose.yml
nginx:
  image: "nginx:1.9"
  ports:
    - 443:443
  links:
    - registry:registry
  volumes:
    - ./nginx/:/etc/nginx/conf.d
registry:
  image: registry:2
  ports:
    - registry.com.cn:5000:5000
  environment:
    REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY:
/data
  volumes:
    - ./data:/data
    - ./registry:/etc/docker/registry
[root@guiqiu-virtualbox~]#

[root@guiqiu-virtualbox~]# cat /data/programs/docker/nginx/registry.conf
upstream
docker-registry {
  server registry:5000;
}
server {
  listen 443;
  server_name registry.com.cn;
  # SSL
  #ssl on;  
  #ssl_certificate /etc/nginx/conf.d/domain.crt;  
  #ssl_certificate_key /etc/nginx/conf.d/domain.key;  
  # disable any limits to avoid HTTP 413 for
large image uploads
  client_max_body_size 0;
  # required to avoid HTTP 411: see Issue #1486
(https://github.com/docker/docker/issues/1486)
  chunked_transfer_encoding on;
  location /v2/ {
    # Do not allow connections from docker 1.5
and earlier
    # docker pre-1.6.0 did not properly set the
user agent on ping, catch "Go *" user agents
    if ($http_user_agent ~
"^(docker\/1\.(3|4|5(?!\.[0-9]-dev))|Go ).*$" ) {
      return 404;
    }
    # To add basic authentication to v2 use
auth_basic setting plus add_header
    # auth_basic "registry.localhost";  
    # auth_basic_user_file /etc/nginx/conf.d/registry.password;  
    # add_header 'Docker-Distribution-Api-Version' 'registry/2.0'
always;  
    proxy_pass    http://docker-registry;
    proxy_set_header  Host   $http_host;   # required for docker client's sake
    proxy_set_header  X-Real-IP $remote_addr; # pass on real client's IP
    proxy_set_header  X-Forwarded-For   $proxy_add_x_forwarded_for;
    proxy_set_header  X-Forwarded-Proto $scheme;
    proxy_read_timeout 900;
  }
}


[root@guiqiu-virtualbox~]# cat /data/programs/docker/registry/config.yml
version: 0.1
log:
  level: debug
  fields:
    service: registry
    environment: development
  hooks:
    - type: mail
      disabled: true
      levels:
        - panic
      options:
        smtp:
          addr: mail.example.com:25
          username: mailuser
          password: password
          insecure: true
        from: sender@example.com
        to:
          - errors@example.com
storage:
    delete:
      enabled: true
    cache:
        blobdescriptor: redis
    filesystem:
        rootdirectory: /var/lib/registry
    delete:
      enable: true
    maintenance:
        uploadpurging:
            enabled: false
http:
    addr: :5000
    debug:
        addr: localhost:5001
redis:
  addr: localhost:6379
  pool:
    maxidle: 16
    maxactive: 64
    idletimeout: 300s
  dialtimeout: 10ms
  readtimeout: 10ms
  writetimeout: 10ms
notifications:
    endpoints:
        - name: local-5003
          url: http://localhost:5003/callback
          headers:
             Authorization: [Bearer <an
example token>]
          timeout: 1s
          threshold: 10
          backoff: 1s
          disabled: true
        - name: local-8083
          url: http://localhost:8083/callback
          timeout: 1s
          threshold: 10
          backoff: 1s
          disabled: true



配置文件创建完成后，回到工作目录执行docker-compose
up运行registry和nginx容器。

[root@guiqiu-virtualboxdocker]# docker-compose up
Starting
docker_registry_1
Starting
docker_nginx_1
Attaching to
docker_registry_1, docker_nginx_1
registry_1  |
time="2016-07-12T08:13:48.797050417Z" level=info msg="debug
server listening localhost:5001"
registry_1  |
time="2016-07-12T08:13:48.797678653Z" level=warning msg="No HTTP
secret provided - generated random secret. This may cause problems with uploads
if multiple re are behind a load-balancer. To provide a shared secret, fill in
http.secret in the configuration file or set the REGISTRY_HTTP_SECRET
environment variable." environment=dev go.version=go1.6.2
instance.id=ea722eaa-5160-488e-ae86-a8de9108fe09 service=registry
version=v2.4.1
registry_1  |
time="2016-07-12T08:13:48.797702548Z" level=info msg="endpoint
local-5003 disabled, skipping" environment=development go.version=go1.6.2
instance.id=ea722eaa-5-ae86-a8de9108fe09 service=registry version=v2.4.1
registry_1  |
time="2016-07-12T08:13:48.797714443Z" level=info msg="endpoint
local-8083 disabled, skipping" environment=development go.version=go1.6.2
instance.id=ea722eaa-5-ae86-a8de9108fe09 service=registry version=v2.4.1
registry_1  |
time="2016-07-12T08:13:48.822365764Z" level=info msg="using
redis blob descriptor cache" environment=development go.version=go1.6.2
instance.id=ea722eaa-5160-4-a8de9108fe09 service=registry version=v2.4.1
registry_1  |
time="2016-07-12T08:13:48.822449552Z" level=info msg="listening
on [::]:5000" environment=development go.version=go1.6.2
instance.id=ea722eaa-5160-488e-ae86-a809 service=registry version=v2.4.1

执行docker-compose
up后。注意是否有容器启动失败的消息，如果容器启动失败的消息，需要检查网络，是否能从dockerhub上pull
image（需代理，或使用使用国内镜像，使用国内镜像需更改docker-compose.yml文件中image项）。也由可能粘贴配置文件错误，需仔细检查。
启动后也可以使用docker
ps命令查看是否两个容器都正常运行

[root@guiqiu-virtualbox ~]# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                          NAMES
450c0a858f83        nginx:1.9           "nginx -g 'daemon off"   20 hours ago        Up 20 hours         80/tcp, 0.0.0.0:443->443/tcp   docker_nginx_1
13a8a7603a54        registry:2          "/bin/registry serve "   20 hours ago        Up 20 hours         0.0.0.0:5000->5000/tcp         docker_registry_1
确定docker容器都正常运行后，用curl 命令验证功能是否正常运行。使得registry.com.cn:5000和registry.com.cn:443访问registry都应该返回{}。

#都是返回{}
[root@guiqiu-virtualbox~]# curl http://registry.com.cn:5000/v2/


停止docker-composer ,进行下面的编辑：
添加用户名和密码
在/data/programs/docker/nginx目录下执行下面命令创建用户名和密码对，如果要创建多个用户名和密码对，则不是使用“-c“选项。
$ htpasswd -c
registry.password docker
然后修改Registry.conf文件，取消下面三行的注释。
auth_basic
"registry.localhost";
auth_basic_user_file /etc/nginx/conf.d/registry.password;  
add_header 'Docker-Distribution-Api-Version' 'registry/2.0' always;
再次执行docker-compose
up运行registry，这时使用localhost:5000端口访问得到的结果为”{}”,但是使用localhost:443访问将得到”401
Authorisation Required“的提示。加入用户名和密码验证才能得到与直接访问registry 5000端口相同的结果。

[root@guiqiu-virtualbox~]# curl http://registry.com.cn:5000/v2/
{}[root@guiqiu-virtualbox~]# curl http://registry.com.cn:443/v2/
<html>
<head><title>400
The plain HTTP request was sent to HTTPS port</title></head>
<body
bgcolor="white">
<center><h1>400
Bad Request</h1></center>
<center>The
plain HTTP request was sent to HTTPS port</center>
<hr><center>nginx/1.9.15</center>
</body>
</html>
[root@guiqiu-virtualbox~]#
加上密码再试下
[root@guiqiu-virtualbox~]#  curl http://docker:214040@registry.com.cn:443/v2/
<html>
<head><title>400
The plain HTTP request was sent to HTTPS port</title></head>
<body
bgcolor="white">
<center><h1>400
Bad Request</h1></center>
<center>The
plain HTTP request was sent to HTTPS port</center>
<hr><center>nginx/1.9.15</center>
</body>
</html>
[root@guiqiu-virtualbox~]#
加入SSL验证

如果你有经过认证机构认证的证书，则直接使用将证书放入nginx目录下即可。如果没有，则使用openssl创建自己的证书。
进行/data/programs/docker/nginx目录

    生成一个新的root key

$ openssl genrsa -out devdockerCA.key 2048
    生成根证书（一路回车即可）

$ openssl req -x509 -new -nodes -key devdockerCA.key -days 10000 -out devdockerCA.crt
    为server创建一个key。（这个key将被nginx配置文件registry.con中ssl_certificate_key域引用）

$openssl genrsa -out domain.key 2048
    制作证书签名请求。注意在执行下面命令时，命令会提示输入一些信息，”Common
Name”一项一定要输入你的域名（官方说IP也行，但是也有IP不能加密的说法），其他项随便输入什么都可以。不要输入任何challenge密码，直接回车即可。

$ openssl req -new -key domain.key -out dev-docker-registry.com.csr
You are about to be asked to enter information that will be incorporated into your certificate request.What you are about to enter is what is called a Distinguished Name or a DN.There are quite a few fields but you can leave some blank For some fields there will be adefault value, If you enter '.', the  field will be left blank.
Country Name (2 letter code) [XX]:
State or Province Name (full name) []:
Locality Name (eg,city) [Default City]:
Organization Name (eg, company) [Default Company Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:registry.com.cn
Email Address []:
Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []:
An optional company name []:
    签署认证请求

$ openssl x509 -req -in dev-docker-registry.com.csr -CA devdockerCA.crt -CAkey devdockerCA.key -CAcreateserial -out domain.crt -days 10000
    配置nginx使用证书

修改registry.conf配置文件，取消如下三行的注释

ssl on;
ssl_certificate  /etc/nginx/conf.d/domain.crt;
ssl_certificate_key /etc/nginx/conf.d/domain.key;
    运行Registry

执行docker-compose up
-d在后台运行Registry，并使用curl验证结果。这时使用localhost:5000端口仍然可以直接访问Registry，但是如果使用443端口通过nginx代理访问，因为已经加了SSL认证，所以使用http将返回“400
bad request”

$ curl http://localhost:5000/v2/
{}
$ curl http://localhost:443/v2/
<html>
<head><title>400
The plain HTTP request was sent to HTTPS port</title></head>
<body
bgcolor="white">
<center><h1>400
Bad Request</h1></center>
<center>The
plain HTTP request was sent to HTTPS port</center>
<hr><center>nginx/1.9.9</center>
</body>
</html>
应该使用https协议

$ curl https://localhost:443/v2/
curl: (60) Peer
certificate cannot be authenticated with known CA certificates
More details here: http://curl.haxx.se/docs/sslcerts.html
curl performs SSL
certificate verification by default, using a "bundle"
of Certificate Authority (CA) public keys (CA
certs). If the default
bundle file isn't adequate, you can specify an
alternate file
using the --cacert option.
If this HTTPS server
uses a certificate signed by a CA represented in
the bundle, the certificate verification
probably failed due to a
problem with the certificate (it might be
expired, or the name might
not match the domain name in the URL).
If you'd like to turn
off curl's verification of the certificate, use
the -k (or --insecure) option.
由于是使用的未经任何认证机构认证的证书，并且还没有在本地应用自己生成的证书。所以此时会提示使用的是未经认证的证书，可以使用“-k"选项不进行验证。


$ curl -k https://localhost:443/v2/
<html>
<head><title>401
Authorization Required</title></head>
<body
bgcolor="white">
<center><h1>401
Authorization Required</h1></center>
<hr><center>nginx/1.9.9</center>
</body>
</html>
客户端使用Registry

添加证书

Centos 6/7 添加证书具体步骤如下

    安装ca-certificates包


$ yum install ca-certificates
    使能动态CA配置功能


   $ update-ca-trust force-enable
    将key拷贝到/etc/pki/ca-trust/source/anchors/


$ cp devdockerCA.crt /etc/pki/ca-trust/source/anchors/
    使新拷贝的证书生效


   $ update-ca-trust extract
    证书拷贝后，需要重启docker以保证docker能使用新的证书


  $ service docker restart
Docker pull/push
image测试

制作要push到registry的镜像

#查看本地已有镜像
$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
registry            2                   cd57aad0bd45        3 days ago          224.5 MB
nginx               1.9                 813e3731b203        3 weeks ago         133.9 MB#为本地镜像打标签
$ docker tag
registry:2 docker-registry.com/registry:2
$ docker tag
nginx:1.9 docker-registry.com/nginx:1.9
$ docker images
REPOSITORY                     TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
registry                       2                   cd57aad0bd45        3 days ago          224.5 MB
docker-registry.com/registry   2                   cd57aad0bd45        3 days ago          224.5 MB
nginx                          1.9                 813e3731b203        3 weeks ago         133.9 MB
docker-registry.com/nginx      1.9                 813e3731b203        3 weeks ago         133.9 MBpush测试

#不登陆直接push镜像到registry，会提示失败
[root@PRO-REGISTRY-220
~]# docker push docker-registry.com/registry:2
The push refers to a
repository [docker-registry.com/registry] (len: 1)
cd57aad0bd45: Image
push failed
cd57aad0bd45:
Buffering to Disk
Please login prior to
push:
Username:
Error response from
daemon: no successful auth challenge for https://docker-registry.com/v2/ -
errors: [basic auth attempt to https://docker-registry.com/v2/realm "registry.localhost" failed with status: 401 Unauthorized]
#登陆后，再试
$docker login https://docker-registry.com
Username: docker
Password:
Email:
WARNING: login
credentials saved in /root/.docker/config.json
Login Succeeded
#可以push 镜像到registry
$ docker push
docker-registry.com/registry:2
The push refers to a
repository [docker-registry.com/registry] (len: 1)
cd57aad0bd45: Image
already exists
b3c39a7768ea: Image
successfully pushed
4725a48b84d4: Image
successfully pushed
7b4078296418: Image
successfully pushed
7bd663e30ad0: Image
successfully pushed
28864e830e4d: Image
successfully pushed
7bd2d56d8449: Image
successfully pushed
af88597ec24b: Image
successfully pushed
b2ae0a712b39: Image
successfully pushed
02e5bca4149b: Image
successfully pushed
895b070402bd: Image
successfully pushed
Digest:
sha256:92835b3e54c05b90e416a309d37ca02669eb5e78e14a0f5ccf44b90d4c21ed4c
搜索镜像

curl https://docker:123456@docker-registry.com/v2/_catalog
{"repositories":["registry"]}
curl https://docker:123456@docker-registry.com/v2/nginx/tags/list
{"name":"registry","tags":["2"]}

pull测试

$ docker logout https://docker-registry.com
Remove login
credentials for https://docker-registry.com
#不登陆registry直接pull镜像也会失败
$ docker pull
docker-registry.com/registry:2
Pulling repository
docker-registry.com/registry
Error: image
registry:2 not found
#登陆后再测试
$ docker login https://docker-registry.com
Username: docker
Password:
Email:
WARNING: login
credentials saved in /root/.docker/config.json
Login Succeeded
#登陆后可以pull
$ docker pull
docker-registry.com/registry:2
1.9: Pulling from
dev-docker-registry.com/registry
6d1ae97ee388: Already
exists
8b9a99209d5c: Already
exists
3244b9987276: Already
exists
50e5c9c52d5d: Already
exists
146400830f31: Already
exists
b412cc1cde63: Already
exists
7fe375038652: Already
exists
c43f11a030f9: Already
exists
152297b50994: Already
exists
01e808fa2993: Already
exists
813e3731b203: Already
exists
Digest:
sha256:af688d675460d336259d60824cd3992e3d820a90b4f31015ef49dc234a00adc3
Status: Downloaded
newer image for docker-registry.com/registry:2