Automate SSL Cert infrastructure #4038

Per #4029 Our current paid SSL certs are valid until 2027, however, they will still need to be manually replaced everywhere, and will need to be funded in the future to ensure continuity of service.

We can eliminate the maintenance and cost of keeping our certificates current by switching over to an ACME based solution (https://acmeclients.com/), giving us free, and automated replacements.

In general we should be using certbot everywhere we can, and fall back on acme.sh if there are any environments where certbot is unsupported.

unofficial-builds is already using this, so the pattern is easily replicable for our other services.

Progress

Certificates are issued with certbot over HTTP-01 (certbot certonly --webroot), using the distro-packaged certbot and its timer. No Cloudflare API token is involved.

Done

• certbot Ansible role, wired into the ci, ci-release, www and rsync plays (ansible: add certbot role and serve origins from Let's Encrypt #4523)
• ci.nodejs.org, ci-release.nodejs.org, direct.nodejs.org and unencrypted.nodejs.org serve Let's Encrypt certificates, with renewal dry-runs passing
• Sectigo custom certificate removed from Cloudflare; the zone uses Universal SSL

Outstanding

• Merge ansible: add certbot role and serve origins from Let's Encrypt #4523
• Verify the first real renewals reload nginx (about 30 days before 2027-01-14)
• www: bring its nginx sites and /etc/nginx/conf.d/ssl-defaults.conf under Ansible (the nginx role's site layout does not match what is on the host)
• rsync: bring /etc/nginx/conf.d/ssl-defaults.conf under Ansible
• Watch for release agents that fail to reconnect to ci-release (ECDSA certificate, ISRG Root X2 chain); if one does, reissue with --key-type rsa

Docs

• Rewrite doc/update-certificates.md for certbot and remove the manual GoGetSSL steps
• Update doc/non-ansible-configuration-notes.md

Cleanup

• Remove the old wildcard files in /etc/nginx/ssl/ (nodejs.key, nodejs_chained.crt and their .old copies) from ci, ci-release, rsync and www
• Remove the star_nodejs_org key and certificate from the secrets repo
• Revoke the Cloudflare API token created for the abandoned DNS-01 approach, and remove cloudflare_api_token from secrets/build/infra/ansible_vars.yml
• Review the benchmarking and coverage vhosts on www, which still reference the old wildcard and appear unused
• Move the other certbot users on www (iojs.org, www.iojs.org, roadmap.iojs.org, and the expired dist.libuv.org) into Ansible, or retire them
• Review the certificate-check workflow now that certificates are short-lived
• Close Certificates expiring on 2026-10-23 #4482 once the checks are green
• Let the GoGetSSL subscription lapse (prepaid until 2027-01-11)

Related

• Decide whether origin.nodejs.org (a live Cloudflare load balancer, used by the release Worker as a fallback) is still needed
• Remove ansible.nodejs.org (Remove ansible.nodejs.org #4487), then consider SSL/TLS mode Full (strict) on the Cloudflare zone