I have gitlab-ce instance running in docker. When GitLab is started for the first time, it successfully requests and issues Let’s Encrypt SSL certificates.
I expect that ~30 days before the expiration date Let’s Encrypt SSL certificates will be automatically renewed by Gitlab, but this does not happen. However, if I manually restart the GitLab docker container when the certificate expiration date is less than 30 days, the certificate will be reissued when the container starts. Why is the certificate not reissued until I restart the container?
Now I have 20 days left on my certificate to expire. I’m ready to run any additional diagnostics to resolve this issue. However, keep in mind that if I restart GitLab, the certificate will automatically be reissued (I’ve been seeing this for months) and I’ll have to wait many days for the issue to reoccur again.
The problem is most likely not related to a specific version of GitLab, it has been reproduced many times on different versions, regardless of when I installed GitLab updates.
Tried to grep the logs: docker compose logs | grep -i -e acme -e challenge -e encrypt, nothing related to SSL certificate auto-renew found.
My docker-compose.yml is:
version: '3.6' volumes: config: data: logs: backups-config: backups-data: services: gitlab: hostname: 'git.mydomain.com' image: 'gitlab/gitlab-ce:17.7.3-ce.0' environment: GITLAB_OMNIBUS_CONFIG: | gitlab_rails['backup_keep_time'] = 2592000 gitlab_rails['custom_html_header_tags'] = '<meta name="robots" content="noindex" />' external_url 'https://git.mydomain.com' nginx['enable'] = true nginx['client_max_body_size'] = '256m' nginx['redirect_http_to_https'] = true nginx['hide_server_tokens'] = 'on' node_exporter['enable'] = false letsencrypt['enable'] = true letsencrypt['alt_names'] = ['gitlab.mydomain.com','mydomain.com'] letsencrypt['contact_emails'] = ['[email protected]'] letsencrypt['auto_renew'] = true ports: - '0.0.0.0:22:22' - '0.0.0.0:80:80' - '0.0.0.0:443:443' - '0.0.0.0:5050:5050' volumes: - 'config:/etc/gitlab' - 'data:/var/opt/gitlab' - 'logs:/var/log/gitlab' - 'backups-config:/etc/gitlab/config_backup' - 'backups-data:/var/opt/gitlab/backups' stop_grace_period: 90s restart: unless-stopped cpus: 6.00 shm_size: '256m' mem_limit: 14g Looking for some clues:
$ docker compose exec gitlab cat /var/opt/gitlab/crond/letsencrypt-renew 27 0 */4 * * root /opt/gitlab/bin/gitlab-ctl renew-le-certs $ docker compose exec gitlab ls -la /var/log/gitlab/lets-encrypt/ | tail -rw-r--r-- 1 root root 1735 Oct 1 00:27 renewal.1727742423.log -rw-r--r-- 1 root root 1737 Oct 9 00:27 renewal.1728433622.log -rw-r--r-- 1 root root 1737 Oct 21 00:27 renewal.1729470422.log -rw-r--r-- 1 root root 1737 Nov 29 00:27 renewal.1732840022.log -rw-r--r-- 1 root root 1538 Dec 13 00:27 renewal.1734049623.log -rw-r--r-- 1 root root 1538 Dec 21 00:27 renewal.1734740822.log -rw-r--r-- 1 root root 1537 Dec 25 00:27 renewal.1735086422.log -rw-r--r-- 1 root root 1538 Jan 5 00:27 renewal.1736036823.log -rw-r--r-- 1 root root 1538 Jan 13 00:27 renewal.1736728022.log -rw-r--r-- 1 root root 1537 Jan 25 00:27 renewal.1737764822.log $ docker compose exec gitlab cat /var/log/gitlab/lets-encrypt/renewal.1737764822.log # Logfile created on 2025-01-25 00:27:06 +0000 by logger.rb/v1.5.3 [2025-01-25T00:27:07+00:00] INFO: Started Cinc Zero at chefzero://localhost:1 with repository at /opt/gitlab/embedded (One version per cookbook) [2025-01-25T00:27:07+00:00] INFO: *** Cinc Client 18.3.0 *** [2025-01-25T00:27:07+00:00] INFO: Platform: x86_64-linux [2025-01-25T00:27:07+00:00] INFO: Cinc-client pid: 83491 [2025-01-25T00:27:08+00:00] INFO: Setting the run_list to ["recipe[gitlab::letsencrypt_renew]"] from CLI options [2025-01-25T00:27:08+00:00] INFO: Run List is [recipe[gitlab::letsencrypt_renew]] [2025-01-25T00:27:08+00:00] INFO: Run List expands to [gitlab::letsencrypt_renew] [2025-01-25T00:27:08+00:00] INFO: Starting Cinc Client Run for git.fxmaster.tech [2025-01-25T00:27:08+00:00] INFO: Running start handlers [2025-01-25T00:27:08+00:00] INFO: Start handlers complete. [2025-01-25T00:27:10+00:00] INFO: Loading cookbooks [[email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]] [2025-01-25T00:27:13+00:00] INFO: Generating default secrets [2025-01-25T00:27:14+00:00] INFO: Generating /etc/gitlab/gitlab-secrets.json file [2025-01-25T00:27:15+00:00] INFO: Cinc Client Run complete in 6.394142053 seconds [2025-01-25T00:27:15+00:00] INFO: Running report handlers [2025-01-25T00:27:15+00:00] INFO: Report handlers complete 