How to setup harbor private secure registry?
Introduction
What is Harbor? It’s an open source project backed by VMWare which delivers cloud native registry, Harbor can store, sign and scan containers images for vulnerabilities. It supports AD, LDAP authentification and can be built with multiple plugins/addons enabled. Why Harbor registry? Well at the time of writing this article - that is most feature rich open source registry available right now. Best part about Harbor - it’s completely free and can be installed on any system within minutes.
For more details, please visit Harbor GitHub: https://github.com/goharbor/harbor and home site: https://goharbor.io/
Quick tutorial
I will cover in few quick steps how to deploy it on premises. Please note following instructions covers Harbor deployment on CentOS v7.5 with docker runtime engine pre-installed (docker engine can be installed with yum command if it wasn’t yet installed).
To install required packages:
yum install -y docker docker-client docker-common docker-compose
To check for installed packages:
yum list installed | grep docker
docker.x86_64 2:1.13.1-88.git07f3374.el7.centos
docker-client.x86_64 2:1.13.1-88.git07f3374.el7.centos
docker-common.x86_64 2:1.13.1-88.git07f3374.el7.centos
docker-compose.noarch 1.18.0-2.el7 @epelrepo
Make sure docker engine is started and running:
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
Also worth to mention - if you are behind corporate proxy you might be required configuring proxy for Docker runtime else you won’t be able to pull docker public images.
You may need to switch SELinux to permissive mode on the registry node before continuing with this setup as it might cause issues serving and exposing Harbor services + using system local mounts.
getenforce && setenforce permissive
Also double check if firewalld or iptables active and if active what ports currently allowed (TCP: 80 and 443 ports would be required as minimum for external access)
[root@registry01 ~]# systemctl is-active firewalld
active
[root@registry01 ~]# systemctl is-active firewalld && firewall-cmd --list-all
active
public (active)
target: default
icmp-block-inversion: no
interfaces: eth0
sources:
services: dhcpv6-client ssh
ports: 443/tcp 80/tcp
protocols:
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
Before moving forward we need to prepare mount path where all registry related files, database and configs will be stored. In this example I will use /data (For prod environments I would recommend using mounted NFS storage).
mkdir /data && chmod 755 /data
Next we want to secure our registry with self signed certificate at least or generate CSR (Server Sign Request) and sign it by your CA authority.
To generate self signed key and cert:
openssl req -sha256 -x509 -days 3650 -nodes -newkey rsa:4096 -keyout registry01.example.com.key -out registry01.example.com.crt
Or to generate key and CSR (following step if you requesting certificate signed by your company CA authority):
/usr/bin/openssl genrsa -rand /dev/urandom -out server.key 4096
/usr/bin/openssl req -new -key server.key -out server.csr
Don’t forget you will need server.key and signed by CA server.crt file if you chosen second option.
Now time to setup Harbor..
First download Harbor offline installer (correct URL can be obtained at: https://github.com/goharbor/harbor/releases) and extract downloaded archive.
wget https://storage.googleapis.com/harbor-releases/release-1.7.0/harbor-offline-installer-v1.7.1.tgz
tar -xvf harbor-offline-installer-v1.7.1.tgz
Once archive extracted, navigate to harbor directory and list files. Main file you are interested in is harbor.cfg.
Following options must be filled correctly, else harbor will fail to deploy:
hostname = registry01.example.com
ui_url_protocol = https
ssl_cert = /data/cert/server.crt
ssl_cert_key = /data/cert/server.key
# Proxy values if your harbor registry sits behind proxy
http_proxy = http://proxy:80
https_proxy = http://proxy:80
no_proxy = 127.0.0.1,localhost,ui,registry,core,registry01.example.com
# Mail Server details if you planning to use mailing service for notifications and etc.
email_server = mailserver.example.com
email_server_port = 25
email_username =
email_password =
email_from = admin <registry01@example.com>
email_ssl = false
email_insecure = false
# Harbor admin password: main password to access Harbor UI as admin user
harbor_admin_password = _ComplexSecret_
# Other params you might be interested - AD related:
##By default the auth mode is db_auth, i.e. the credentials are stored in a local database.
#Set it to ldap_auth if you want to verify a user's credentials against an LDAP server.
#auth_mode = db_auth
auth_mode = ldap_auth
#The url for an ldap endpoint.
ldap_url = ldaps://ldaps-vip.example.com
#A user's DN who has the permission to search the LDAP/AD server.
#If your LDAP/AD server does not support anonymous search, you should configure this DN and ldap_search_pwd.
#ldap_searchdn = uid=searchuser,ou=people,dc=mydomain,dc=com
ldap_searchdn = CN=BindUser,OU=ServiceAccounts,OU=Users,OU=IT,DC=example,DC=com
#the password of the ldap_searchdn
ldap_search_pwd = _BIND_USER_PASSWORD_
#The base DN from which to look up a user in LDAP/AD
ldap_basedn = DC=example,DC=com
#Search filter for LDAP/AD, make sure the syntax of the filter is correct.
ldap_filter = (objectClass=person)
# The attribute used in a search to match a user, it could be uid, cn, email, sAMAccountName or other attributes depending on your LDAP/AD
ldap_uid = sAMAccountName
#the scope to search for users, 0-LDAP_SCOPE_BASE, 1-LDAP_SCOPE_ONELEVEL, 2-LDAP_SCOPE_SUBTREE
ldap_scope = 2
#Timeout (in seconds) when connecting to an LDAP Server. The default value (and most reasonable) is 5 seconds.
ldap_timeout = 5
#Verify certificate from LDAP server
ldap_verify_cert = false
#The base dn from which to lookup a group in LDAP/AD
ldap_group_basedn = OU=Groups,OU=CDL,DC=example,DC=com
#filter to search LDAP/AD group
ldap_group_filter = objectclass=group
#The attribute used to name a LDAP/AD group, it could be cn, name
ldap_group_gid = distinguishedName
#The scope to search for ldap groups. 0-LDAP_SCOPE_BASE, 1-LDAP_SCOPE_ONELEVEL, 2-LDAP_SCOPE_SUBTREE
ldap_group_scope = 2
#Turn on or off the self-registration feature
self_registration = on
#The expiration time (in minute) of token created by token service, default is 30 minutes
token_expiration = 30
#The flag to control what users have permission to create projects
#The default value "everyone" allows everyone to creates a project.
#Set to "adminonly" so that only admin user can create project.
project_creation_restriction = everyone
Once harbor.cfg filled correctly with your infrastructure/organization details - you can start deployment of Harbor.
Harbor can be setup with following command (to be executed inside extracted harbor directory):
./install.sh --with-clair --with-chartmuseum --with-notary
[Step 0]: checking installation environment ...
Note: docker version: 1.13.1
Note: docker-compose version: 1.10.1
[Step 1]: loading Harbor images ...
Loaded image: goharbor/registry-photon:v2.6.2-v1.7.1
Loaded image: goharbor/harbor-migrator:v1.7.1
Loaded image: goharbor/harbor-adminserver:v1.7.1
Loaded image: goharbor/harbor-core:v1.7.1
Loaded image: goharbor/harbor-log:v1.7.1
Loaded image: goharbor/harbor-jobservice:v1.7.1
Loaded image: goharbor/notary-server-photon:v0.6.1-v1.7.1
Loaded image: goharbor/clair-photon:v2.0.7-v1.7.1
Loaded image: goharbor/harbor-portal:v1.7.1
Loaded image: goharbor/harbor-db:v1.7.1
Loaded image: goharbor/redis-photon:v1.7.1
Loaded image: goharbor/nginx-photon:v1.7.1
Loaded image: goharbor/harbor-registryctl:v1.7.1
Loaded image: goharbor/notary-signer-photon:v0.6.1-v1.7.1
Loaded image: goharbor/chartmuseum-photon:v0.7.1-v1.7.1
[Step 2]: preparing environment ...
Clearing the configuration file: ./common/config/adminserver/env
Clearing the configuration file: ./common/config/core/env
Clearing the configuration file: ./common/config/core/app.conf
Clearing the configuration file: ./common/config/core/private_key.pem
Clearing the configuration file: ./common/config/db/env
Clearing the configuration file: ./common/config/jobservice/env
Clearing the configuration file: ./common/config/jobservice/config.yml
Clearing the configuration file: ./common/config/registry/config.yml
Clearing the configuration file: ./common/config/registry/root.crt
Clearing the configuration file: ./common/config/registryctl/env
Clearing the configuration file: ./common/config/registryctl/config.yml
Clearing the configuration file: ./common/config/nginx/conf.d/notary.upstream.conf
Clearing the configuration file: ./common/config/nginx/conf.d/notary.server.conf
Clearing the configuration file: ./common/config/nginx/cert/server.crt
Clearing the configuration file: ./common/config/nginx/cert/server.key
Clearing the configuration file: ./common/config/nginx/nginx.conf
Clearing the configuration file: ./common/config/log/logrotate.conf
Clearing the configuration file: ./common/config/notary/notary-signer.crt
Clearing the configuration file: ./common/config/notary/notary-signer.key
Clearing the configuration file: ./common/config/notary/notary-signer-ca.crt
Clearing the configuration file: ./common/config/notary/root.crt
Clearing the configuration file: ./common/config/notary/signer-config.postgres.json
Clearing the configuration file: ./common/config/notary/server-config.postgres.json
Clearing the configuration file: ./common/config/notary/signer_env
Clearing the configuration file: ./common/config/notary/server_env
Clearing the configuration file: ./common/config/clair/postgresql-init.d/README.md
Clearing the configuration file: ./common/config/clair/postgres_env
Clearing the configuration file: ./common/config/clair/config.yaml
Clearing the configuration file: ./common/config/clair/clair_env
Clearing the configuration file: ./common/config/chartserver/env
loaded secret from file: /data/secretkey
Generated configuration file: ./common/config/nginx/nginx.conf
Generated configuration file: ./common/config/adminserver/env
Generated configuration file: ./common/config/core/env
Generated configuration file: ./common/config/registry/config.yml
Generated configuration file: ./common/config/db/env
Generated configuration file: ./common/config/jobservice/env
Generated configuration file: ./common/config/jobservice/config.yml
Generated configuration file: ./common/config/log/logrotate.conf
Generated configuration file: ./common/config/registryctl/env
Generated configuration file: ./common/config/core/app.conf
Generated certificate, key file: ./common/config/core/private_key.pem, cert file: ./common/config/registry/root.crt
Copying sql file for notary DB
Generated certificate, key file: ./cert_tmp/notary-signer-ca.key, cert file: ./cert_tmp/notary-signer-ca.crt
Generated certificate, key file: ./cert_tmp/notary-signer.key, cert file: ./cert_tmp/notary-signer.crt
Copying certs for notary signer
Copying notary signer configuration file
Generated configuration file: ./common/config/notary/signer-config.postgres.json
Generated configuration file: ./common/config/notary/server-config.postgres.json
Copying nginx configuration file for notary
Generated configuration file: ./common/config/nginx/conf.d/notary.server.conf
loaded secret from file: /data/defaultalias
Generated configuration file: ./common/config/notary/signer_env
Copying offline data file for clair DB
Generated configuration file: ./common/config/clair/postgres_env
Generated configuration file: ./common/config/clair/config.yaml
Generated configuration file: ./common/config/clair/clair_env
Generated configuration file: ./common/config/chartserver/env
The configuration files are ready, please use docker-compose to start the service.
[Step 3]: checking existing instance of Harbor ...
[Step 4]: starting Harbor ...
Creating harbor-log
Creating redis
Creating registryctl
Creating harbor-db
Creating harbor-adminserver
Creating registry
Creating chartmuseum
Creating notary-signer
Creating clair
Creating harbor-core
Creating notary-server
Creating harbor-portal
Creating harbor-jobservice
Creating nginx
✔ ----Harbor has been installed and started successfully.----
Now you should be able to visit the admin portal at https://registry01.example.com.
For more details, please visit https://github.com/goharbor/harbor .
If all works as expected you should be able to see Harbor services containers running on your node:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
847ed52c4fb7 goharbor/nginx-photon:v1.7.1 "nginx -g 'daemon ..." 8 seconds ago Up 7 seconds (health: starting) 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:4443->4443/tcp nginx
c5443aca6d2c goharbor/harbor-jobservice:v1.7.1 "/harbor/start.sh" 10 seconds ago Up 8 seconds harbor-jobservice
a45e7d1254b3 goharbor/harbor-portal:v1.7.1 "nginx -g 'daemon ..." 10 seconds ago Up 8 seconds (health: starting) 80/tcp harbor-portal
70b8808e2e23 goharbor/notary-server-photon:v0.6.1-v1.7.1 "/bin/server-start.sh" 10 seconds ago Up 8 seconds notary-server
45d7ad05e69b goharbor/harbor-core:v1.7.1 "/harbor/start.sh" 11 seconds ago Up 9 seconds (health: starting) harbor-core
a742a73342a0 goharbor/clair-photon:v2.0.7-v1.7.1 "/docker-entrypoin..." 12 seconds ago Up 9 seconds (health: starting) 6060-6061/tcp clair
76f96075eb7e goharbor/notary-signer-photon:v0.6.1-v1.7.1 "/bin/signer-start.sh" 12 seconds ago Up 10 seconds notary-signer
370ae81df409 goharbor/chartmuseum-photon:v0.7.1-v1.7.1 "/docker-entrypoin..." 13 seconds ago Up 10 seconds (health: starting) 9999/tcp chartmuseum
0ad65b98c05a goharbor/registry-photon:v2.6.2-v1.7.1 "/entrypoint.sh /e..." 14 seconds ago Up 11 seconds (health: starting) 5000/tcp registry
fd8d961a541a goharbor/harbor-adminserver:v1.7.1 "/harbor/start.sh" 14 seconds ago Up 11 seconds (health: starting) harbor-adminserver
5c1316900042 goharbor/harbor-db:v1.7.1 "/entrypoint.sh po..." 14 seconds ago Up 12 seconds (health: starting) 5432/tcp harbor-db
606977b92135 goharbor/harbor-registryctl:v1.7.1 "/harbor/start.sh" 14 seconds ago Up 12 seconds (health: starting) registryctl
5350ce213c63 goharbor/redis-photon:v1.7.1 "docker-entrypoint..." 14 seconds ago Up 12 seconds 6379/tcp redis
344065adfdd9 goharbor/harbor-log:v1.7.1 "/bin/sh -c /usr/l..." 15 seconds ago Up 13 seconds (health: starting) 127.0.0.1:1514->10514/tcp harbor-log
And testing Harbor registry URL locally:
curl -k https://registry01.example.com | head -n 10
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 693 100 693 0 0 7531 0 --:--:-- --:--:-- --:--:-- 7615
<!doctype html>
<html>
<head>
<meta charset="utf-8">
<title>Harbor</title>
<base href="/">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="icon" type="image/x-icon" href="favicon.ico?v=2">
<link rel="stylesheet" href="styles.0a97e5c7dae15939bfca.css"></head>
You can also access Harbor UI within your browser.
Congratulations! At this point you have your registry running! Now you can test if you can login to registry as AD or Admin user.
[root@another_docker_node ]# docker login registry01.example.com:443
Username: AD_USER
Password:
Login Succeeded
At this point you can pull and push images if your user has correct permissions. Permissions can be defined in Harbor UI.
I hope this helped some new to Harbor users.