diff --git "a/k8s\346\234\215\345\212\241/3-\351\203\250\347\275\262harbor\344\273\223\345\272\223.md" "b/k8s\346\234\215\345\212\241/3-\351\203\250\347\275\262harbor\344\273\223\345\272\223.md" new file mode 100644 index 0000000000000000000000000000000000000000..c99e75e770b2469b1ec84447e2c4822fc6814b78 --- /dev/null +++ "b/k8s\346\234\215\345\212\241/3-\351\203\250\347\275\262harbor\344\273\223\345\272\223.md" @@ -0,0 +1,427 @@ +# 必读!!! + +docker文章那边已经介绍了docker了,这里描述的比较粗略,可以理解成这篇文章只是作为一个复习来使用的,具体学习请看docker部分的harbor仓库内容 + +# 这儿为什么要安装部署harbor仓库呢? + +因为在企业中,我们有很多镜像都是基于企业定制的,然后我们定制好的镜像应该存在哪里呢?没错,就是存在harbor仓库中,k8s说白了就是管理docker的容器,而容器是基于镜像创建的,所以我们这儿需要部署harbor仓库,方便我们后续的学习 + +# harbor仓库概述 + +harbor类似于git的一种管理软件,harbor是存储容器镜像的。 + +官方的介绍如下: + +``` +What is Harbor? + +Harbor is an open source registry that secures artifacts with policies and role-based access control, ensures images are scanned and free from vulnerabilities, and signs images as trusted. Harbor, a CNCF Graduated project, delivers compliance, performance, and interoperability to help you consistently and securely manage artifacts across cloud native compute platforms like Kubernetes and Docker. + +翻译如下: +Harbor是一个开源注册表,它通过策略和基于角色的访问控制来保护工件,确保图像被扫描且没有漏洞,并将图像标记为可信。Harbor是CNCF毕业的项目,提供合规性、性能和互操作性,帮助您在Kubernetes和Docker等云原生计算平台上一致、安全地管理工件。 +``` + +说白了,harbor仓库就是一个存储镜像的私有仓库 + +### harbor来源: + +官网:https://goharbor.io/ + +github地址:https://github.com/goharbor/harbor/releases + +# 部署前环境准备 + +Ubuntu系统 22.04 + +| 主机名 | IP | 资源 | +| -------- | --------- | ---- | +| harbor50 | 10.0.0.50 | 2c4g | + +**需要安装docker、docker-compose**,相关资料自行查看本仓库中docker相关内容 + +# 安装部署harbor: + +## 单机版: + +```shell +mkdir -p /data +cd /data +#下载相关资源 +wget https://github.com/goharbor/harbor/releases/download/v2.10.3/harbor-offline-installer-v2.10.3.tgz + +#解压 +tar -xvf harbor-offline-installer-v2.10.3.tgz + + +#解压后的目录 +[root@harbor /data]# tree +. +├── harbor +│   ├── common.sh +│   ├── harbor.v2.10.3.tar.gz +│   ├── harbor.yml.tmpl +│   ├── install.sh +│   ├── LICENSE +│   └── prepare +└── harbor-offline-installer-v2.10.3.tgz + +1 directory, 7 files +``` + +**修改harbor.yml.tmpl文件** + +```shell +[root@harbor /data/harbor]# cp harbor.yml.tmpl harbor.yml +[root@harbor /data/harbor]# vim harbor.yml +#修改下列内容即可 +hostname: harbor.huangsir.com #访问harbor仓库的域名 +http: + port: 80 #http的端口 +harbor_admin_password: root #harbor仓库的密码 +data_volume: /data/harbor #持久化存储卷 + +``` + +**启动prepare脚本进行前置检查** + +```shell +[root@harbor /data/harbor]# ./prepare +... +#最后出现Successfully即可 +``` + +**启动install.sh进行安装** + +```shell +[root@harbor /data/harbor]# ./install.sh +``` + +**检查** + +```shell +[root@harbor /data/harbor]# docker-compose ps +WARN[0000] /data/harbor/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion +NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS +harbor-core goharbor/harbor-core:v2.10.3 "/harbor/entrypoint.…" core 13 minutes ago Up 13 minutes (healthy) +harbor-db goharbor/harbor-db:v2.10.3 "/docker-entrypoint.…" postgresql 13 minutes ago Up 13 minutes (healthy) +harbor-jobservice goharbor/harbor-jobservice:v2.10.3 "/harbor/entrypoint.…" jobservice 13 minutes ago Up 13 minutes (healthy) +harbor-log goharbor/harbor-log:v2.10.3 "/bin/sh -c /usr/loc…" log 13 minutes ago Up 13 minutes (healthy) 127.0.0.1:1514->10514/tcp +harbor-portal goharbor/harbor-portal:v2.10.3 "nginx -g 'daemon of…" portal 13 minutes ago Up 13 minutes (healthy) +nginx goharbor/nginx-photon:v2.10.3 "nginx -g 'daemon of…" proxy 13 minutes ago Up 13 minutes (healthy) 0.0.0.0:80->8080/tcp, :::80->8080/tcp +redis goharbor/redis-photon:v2.10.3 "redis-server /etc/r…" redis 13 minutes ago Up 13 minutes (healthy) +registry goharbor/registry-photon:v2.10.3 "/home/harbor/entryp…" registry 13 minutes ago Up 13 minutes (healthy) +registryctl goharbor/harbor-registryctl:v2.10.3 "/home/harbor/start.…" registryctl 13 minutes ago Up 13 minutes (healthy) + +``` + +### 访问测试 + +需要在本地的hosts文件添加解析 + +windows系统在这个路径下`C:\Windows\System32\drivers\etc` + +添加以下内容 + +``` +10.0.0.50 harbor.huangsir.com +``` + +打开浏览器访问测试 + +http://harbor.huangsir.com + +输入用户名:admin,密码root + +![image-20240914165030695](./images/image-20240914165030695.png) + +### 配置https + +配置https很简单,如果你有相对应的https证书直接修改相对应的配置即可,我这里先基于自建证书进行配置,因为购买的域名证书太简单了,并且需要花钱呐... + +如果是基于购买的域名证书则可以直接执行第六步即可 + +**1、生成ca.key证书和ca.crt证书** + +```shell +#步骤一:创建相关的目录 +[root@harbor /data/harbor]# mkdir -pv /usr/local/harbor/certs/{ca,harbor-server,docker-client} +mkdir: created directory '/usr/local/harbor' +mkdir: created directory '/usr/local/harbor/certs' +mkdir: created directory '/usr/local/harbor/certs/ca' +mkdir: created directory '/usr/local/harbor/certs/harbor-server' +mkdir: created directory '/usr/local/harbor/certs/docker-client' + + #进入目录,创建ca证书 +[root@harbor /data/harbor]# cd /usr/local/harbor/certs/ +[root@harbor /usr/local/harbor/certs]# openssl genrsa -out ca/ca.key 4096 + +#生成ca.srt证书 +[root@harbor /usr/local/harbor/certs]# openssl req -x509 -new -nodes -sha512 -days 3650 \ + -subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=huangsir.com" \ + -key ca/ca.key \ + -out ca/ca.crt + + #查看当前目录 +[root@harbor /usr/local/harbor/certs]# tree ca +ca +├── ca.crt +└── ca.key + +0 directories, 2 files + +``` + +**2、基于自建ca创建harbor证书** + +```shell +#创建证书 +[root@harbor /usr/local/harbor/certs]# openssl genrsa -out harbor-server/harbor.huangsir.com.key 4096 + +#查看目录是否生成成功 +[root@harbor /usr/local/harbor/certs]# tree harbor-server/ +harbor-server/ +└── harbor.huangsir.com.key + +0 directories, 1 file +[root@harbor /usr/local/harbor/certs]# + +``` + +**3、生成请求证书文件** + +```shell +#生成证书 +[root@harbor /usr/local/harbor/certs]# openssl req -sha512 -new \ + -subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=harbor.huangsir.com" \ + -key harbor-server/harbor.huangsir.com.key \ + -out harbor-server/harbor.huangsir.com.csr + +#验证 +[root@harbor /usr/local/harbor/certs]# tree harbor-server/ +harbor-server/ +├── harbor.huangsir.com.csr +└── harbor.huangsir.com.key + +0 directories, 2 files +[root@harbor /usr/local/harbor/certs]# + +``` + +**4、生成 x509 v3 的扩展文件用于认证** + +```shell +#生成文件 +root@harbor /usr/local/harbor/certs]# cat > harbor-server/v3.ext <<-EOF +authorityKeyIdentifier=keyid,issuer +basicConstraints=CA:FALSE +keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment +extendedKeyUsage = serverAuth +subjectAltName = @alt_names + +[alt_names] +DNS.1=harbor.huangsir.com +EOF + +#验证 +[root@harbor /usr/local/harbor/certs]# tree harbor-server/ +harbor-server/ +├── harbor.huangsir.com.csr +├── harbor.huangsir.com.key +└── v3.ext + +0 directories, 3 files +[root@harbor /usr/local/harbor/certs]# + +``` + +**5、基于 x509 v3 的扩展文件认证签发harbor server证书** + +```shell +#生成证书 +[root@harbor /usr/local/harbor/certs]# openssl x509 -req -sha512 -days 3650 \ + -extfile harbor-server/v3.ext \ + -CA ca/ca.crt -CAkey ca/ca.key -CAcreateserial \ + -in harbor-server/harbor.huangsir.com.csr \ + -out harbor-server/harbor.huangsir.com.crt + +#复制上面的 +Certificate request self-signature ok +subject=C = CN, ST = Beijing, L = Beijing, O = example, OU = Personal, CN = harbor.huangsir.com + +#查看目录 +[root@harbor /usr/local/harbor/certs]# tree harbor-server/ +harbor-server/ +├── harbor.huangsir.com.crt +├── harbor.huangsir.com.csr +├── harbor.huangsir.com.key +└── v3.ext + +0 directories, 4 files +[root@harbor /usr/local/harbor/certs]# + +``` + +整体目录结构如下 + +```shell +[root@harbor /usr/local/harbor/certs]# tree +. +├── ca +│   ├── ca.crt +│   └── ca.key +├── docker-client +└── harbor-server + ├── harbor.huangsir.com.crt + ├── harbor.huangsir.com.csr + ├── harbor.huangsir.com.key + └── v3.ext + +3 directories, 6 files +[root@harbor /usr/local/harbor/certs]# + +``` + +**6、修改harbor配置文件,配置https请求**, + +注意:如果是购买的域名证书,直接配置即可,上面的操作可以忽略 + +自己购买的证书,则将证书上传到服务器指定目录中,相对应的私钥和公钥指定对应的路径即可 + +vim /data/harbor/harbor.yml + +```shell + #主要修改下面的内容 + 13 https: + 14 # https port for harbor, default is 443 + 15 port: 443 + 16 # The path of cert and key files for nginx + 17 certificate: /usr/local/harbor/certs/harbor-server/harbor.huangsir.com.crt + 18 private_key: /usr/local/harbor/certs/harbor-server/harbor.huangsir.com.key + 19 # enable strong ssl ciphers (default: false) + 20 # strong_ssl_ciphers: false + +``` + +**7、重启重新安装 + +```shell +#停止容器 +[root@harbor /data/harbor]# docker-compose stop +... + +#删除容器 +[root@harbor /data/harbor]# docker-compose rm +... + +#检查 +[root@harbor /data/harbor]# docker-compose ps +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES + +#重新安装 +[root@harbor /data/harbor]# ./install.sh + +``` + +**8、访问测试** + +因为是自建证书,所以这儿是不安全的,浏览器是不认可的,所以这儿会显示不安全,但是这样已经成功了, + +如果是购买的域名证书,则可以直接访问,没有任何问题 + +![image-20240914171747255](./images/image-20240914171747255.png) + +# **k8s集群节点访问harbor仓库配置**(购买域名证书可忽略) + +K8S集群访问harbor仓库需要配置,否则是访问不了的,有几种方式可以配置,可以看下述 + +## 方式一: + +**步骤一:所有机器添加hosts解析(所有节点操作)** + +```shell +cat >> /etc/hosts <10514/tcp +harbor-portal goharbor/harbor-portal:v2.10.3 "nginx -g 'daemon of…" portal 13 minutes ago Up 13 minutes (healthy) +nginx goharbor/nginx-photon:v2.10.3 "nginx -g 'daemon of…" proxy 13 minutes ago Up 13 minutes (healthy) 0.0.0.0:80->8080/tcp, :::80->8080/tcp +redis goharbor/redis-photon:v2.10.3 "redis-server /etc/r…" redis 13 minutes ago Up 13 minutes (healthy) +registry goharbor/registry-photon:v2.10.3 "/home/harbor/entryp…" registry 13 minutes ago Up 13 minutes (healthy) +registryctl goharbor/harbor-registryctl:v2.10.3 "/home/harbor/start.…" registryctl 13 minutes ago Up 13 minutes (healthy) + +``` + +### 访问测试 + +需要在本地的hosts文件添加解析 + +windows系统在这个路径下`C:\Windows\System32\drivers\etc` + +添加以下内容 + +``` +10.0.0.50 harbor.huangsir.com +``` + +打开浏览器访问测试 + +http://harbor.huangsir.com + +输入用户名:admin,密码root + +![image-20240914165030695](./images/image-20240914165030695.png) + +### 配置https + +配置https很简单,如果你有相对应的https证书直接修改相对应的配置即可,我这里先基于自建证书进行配置,因为购买的域名证书太简单了,并且需要花钱呐... + +如果是基于购买的域名证书则可以直接执行第六步即可 + +**1、生成ca.key证书和ca.crt证书** + +```shell +#步骤一:创建相关的目录 +[root@harbor /data/harbor]# mkdir -pv /usr/local/harbor/certs/{ca,harbor-server,docker-client} +mkdir: created directory '/usr/local/harbor' +mkdir: created directory '/usr/local/harbor/certs' +mkdir: created directory '/usr/local/harbor/certs/ca' +mkdir: created directory '/usr/local/harbor/certs/harbor-server' +mkdir: created directory '/usr/local/harbor/certs/docker-client' + + #进入目录,创建ca证书 +[root@harbor /data/harbor]# cd /usr/local/harbor/certs/ +[root@harbor /usr/local/harbor/certs]# openssl genrsa -out ca/ca.key 4096 + +#生成ca.srt证书 +[root@harbor /usr/local/harbor/certs]# openssl req -x509 -new -nodes -sha512 -days 3650 \ + -subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=huangsir.com" \ + -key ca/ca.key \ + -out ca/ca.crt + + #查看当前目录 +[root@harbor /usr/local/harbor/certs]# tree ca +ca +├── ca.crt +└── ca.key + +0 directories, 2 files + +``` + +**2、基于自建ca创建harbor证书** + +```shell +#创建证书 +[root@harbor /usr/local/harbor/certs]# openssl genrsa -out harbor-server/harbor.huangsir.com.key 4096 + +#查看目录是否生成成功 +[root@harbor /usr/local/harbor/certs]# tree harbor-server/ +harbor-server/ +└── harbor.huangsir.com.key + +0 directories, 1 file +[root@harbor /usr/local/harbor/certs]# + +``` + +**3、生成请求证书文件** + +```shell +#生成证书 +[root@harbor /usr/local/harbor/certs]# openssl req -sha512 -new \ + -subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=harbor.huangsir.com" \ + -key harbor-server/harbor.huangsir.com.key \ + -out harbor-server/harbor.huangsir.com.csr + +#验证 +[root@harbor /usr/local/harbor/certs]# tree harbor-server/ +harbor-server/ +├── harbor.huangsir.com.csr +└── harbor.huangsir.com.key + +0 directories, 2 files +[root@harbor /usr/local/harbor/certs]# + +``` + +**4、生成 x509 v3 的扩展文件用于认证** + +```shell +#生成文件 +root@harbor /usr/local/harbor/certs]# cat > harbor-server/v3.ext <<-EOF +authorityKeyIdentifier=keyid,issuer +basicConstraints=CA:FALSE +keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment +extendedKeyUsage = serverAuth +subjectAltName = @alt_names + +[alt_names] +DNS.1=harbor.huangsir.com +EOF + +#验证 +[root@harbor /usr/local/harbor/certs]# tree harbor-server/ +harbor-server/ +├── harbor.huangsir.com.csr +├── harbor.huangsir.com.key +└── v3.ext + +0 directories, 3 files +[root@harbor /usr/local/harbor/certs]# + +``` + +**5、基于 x509 v3 的扩展文件认证签发harbor server证书** + +```shell +#生成证书 +[root@harbor /usr/local/harbor/certs]# openssl x509 -req -sha512 -days 3650 \ + -extfile harbor-server/v3.ext \ + -CA ca/ca.crt -CAkey ca/ca.key -CAcreateserial \ + -in harbor-server/harbor.huangsir.com.csr \ + -out harbor-server/harbor.huangsir.com.crt + +#复制上面的 +Certificate request self-signature ok +subject=C = CN, ST = Beijing, L = Beijing, O = example, OU = Personal, CN = harbor.huangsir.com + +#查看目录 +[root@harbor /usr/local/harbor/certs]# tree harbor-server/ +harbor-server/ +├── harbor.huangsir.com.crt +├── harbor.huangsir.com.csr +├── harbor.huangsir.com.key +└── v3.ext + +0 directories, 4 files +[root@harbor /usr/local/harbor/certs]# + +``` + +整体目录结构如下 + +```shell +[root@harbor /usr/local/harbor/certs]# tree +. +├── ca +│   ├── ca.crt +│   └── ca.key +├── docker-client +└── harbor-server + ├── harbor.huangsir.com.crt + ├── harbor.huangsir.com.csr + ├── harbor.huangsir.com.key + └── v3.ext + +3 directories, 6 files +[root@harbor /usr/local/harbor/certs]# + +``` + +**6、修改harbor配置文件,配置https请求**, + +注意:如果是购买的域名证书,直接配置即可,上面的操作可以忽略 + +自己购买的证书,则将证书上传到服务器指定目录中,相对应的私钥和公钥指定对应的路径即可 + +vim /data/harbor/harbor.yml + +```shell + #主要修改下面的内容 + 13 https: + 14 # https port for harbor, default is 443 + 15 port: 443 + 16 # The path of cert and key files for nginx + 17 certificate: /usr/local/harbor/certs/harbor-server/harbor.huangsir.com.crt + 18 private_key: /usr/local/harbor/certs/harbor-server/harbor.huangsir.com.key + 19 # enable strong ssl ciphers (default: false) + 20 # strong_ssl_ciphers: false + +``` + +**7、重启重新安装 + +```shell +#停止容器 +[root@harbor /data/harbor]# docker-compose stop +... + +#删除容器 +[root@harbor /data/harbor]# docker-compose rm +... + +#检查 +[root@harbor /data/harbor]# docker-compose ps +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES + +#重新安装 +[root@harbor /data/harbor]# ./install.sh + +``` + +**8、访问测试** + +因为是自建证书,所以这儿是不安全的,浏览器是不认可的,所以这儿会显示不安全,但是这样已经成功了, + +如果是购买的域名证书,则可以直接访问,没有任何问题 + +![image-20240914171747255](./images/image-20240914171747255.png) + +# **k8s集群节点访问harbor仓库配置**(购买域名证书可忽略) + +K8S集群访问harbor仓库需要配置,否则是访问不了的,有几种方式可以配置,可以看下述 + +## 方式一: + +**步骤一:所有机器添加hosts解析(所有节点操作)** + +```shell +cat >> /etc/hosts <