LetEncrypt with Nginx on EL7

Use the acme-tiny client as it is explainable (small and audit-able). This is wrapped with a small bash script to automate issue/renewal of the certificates.

All renewals use a new private key. This requires a new certificate signing request (CSR). Given that a new key pair is generated this script will have visibility of the private key.

It is important to note that domain validation for the x.509 certificate uses the http web site by the same name. This requires a two step deployment of a web site where the http site is put in place and then once the certificate is issued the https site can be put in place.

Residuals

This process seems to be working well for a few sample domains. The following issues are not addressed:

the certificate limits allowed by letsencrypt stop this being used by more than a small sample of certificates per domain
this process doesn't cover xmpp certificates
this process doesn't cover smtp relates certificates
the nginx process is reloaded using local mechanisms which require sudo privileges
nginx v1.9.10 doesn't support multiple certificates (as of Jan 2016) so only request RSA certs

Process

Get the latest version of the acme_tiny script:

# wget -P /usr/local/bin https://raw.githubusercontent.com/diafygi/acme-tiny/master/acme_tiny.py
# chmod +x /usr/local/bin/acme_tiny.py

Create an 'acme' user for the purposes of managing the x.509 certificates. This user will generate keys, request certificates and reload the nginx server.

# adduser --comment "Lets Encrypt" -r --create-home -d /var/lib/acme acme
# mkdir -p /etc/acme
# chown acme:acme /etc/acme
# chmod o+rx /var/lib/acme

Add the nginx user to the acme group so that it can read keys and certificates created by the acme user.

# usermod -G acme -a nginx

Add the script below into the path as 'letencrypt'.

Web site configurations

Each web site provides a simple redirect on it's http configuration to redirect to the common site which provides the http challege responses (see below).

Challenge web site

Create a directory where the acme user will create challenge files. This must be readable by the nginx process so that the challenge can be provided to the letsencrypt server.

 $ mkdir /var/lib/acme/challenges

Create a web site to provide the acme challege request files. This requires a DNS entry. It is http only web site (no https web site and certificate as this potentially causes a chicken and egg situation if the certificate expires). It references the directory created in the previous step (Note: if the Let Encrypt client was used on another host this could be configured to reverse proxy the requests to that host).

#
# An http only web site for valdating http-01 requests from Let Encrypt
#
server {
    listen       [::]:80;
    server_name  validation.lucidsolutions.co.nz;

    access_log   /var/log/nginx/validation.lucidsolutions.co.nz.access.log  main;

    root         /var/lib/acme/challenges/;
}

Per site redirect

Each site that requires a http certificate needs to have a small configuration change.

As root create a nginx configuration file ('/etc/nginx/acme-challenge') that will be included by all https web sites to redirect the acme challenge requests to a single host. (Note: If your web infrastructure is distributed this will need to be done on all hosts).

# redirect acme challeges to a single site
location /.well-known/acme-challenge/ {
  rewrite ^/.well-known/acme-challenge(/.*) http://validation.lucidsolutions.co.nz$1? permanent;
}

For each web site include the 'acme-challenge' file in the http server:

server {
        listen               [::]:80;
        include              /etc/nginx/acme-challenge;
}

Certificate installation

Allow the acme user to reload the nginx configuration using sudo. This will allow the script running as the 'acme' user to reload nginx when a new certificate is issued.

# cat > /etc/sudoers.d/acme <<EOF
# Allow the acme user to reload nginx
acme  ALL=NOPASSWD: /bin/systemctl reload nginx
EOF

Certificates configuration

There is no magic for harvesting the list of certificates. The list is defined in the file '/etc/acme/domains' (or '/etc/acmedomains-staging' for the staging list). One certificate per line, with the first name listed as the main certificate command name (CN). e.g.

www.lucidsolutions.co.nz lucidsolutions.co.nz
acme.lucidsolutions.co.nz

Note: Due to limitations of letsencrypt choose the list wisely. If there are too many in here then requests will be rate limited.

Automatic updates

Run the renewal/issue script once a day to check if certicicate need to be renewed. Edit the cron tab for the 'acme' user ('$ crontab -e'):

MAILTO=user-acmerenewals@example.com
42 2 * * * /usr/local/bin/letsencrypt

Appendices

Script

#!/bin/bash

set -e

#
#  Predefined constants. Overwride with sysconfig settings or command line options.
#
KEY_SIZE=4096
DHPARAM_SIZE=2048
OPENSSL_CNF="$(openssl version -d | cut -d'"' -f2)/openssl.cnf"
MIN_CERT_LIFETIME="$( expr 30 \* 86400)"

#
# Allow parameters to be set via a configuration file
#
[ -f /etc/sysconfig/acme ] && . /etc/sysconfig/acme


mkdir -p /etc/acme/{keys,live,archive}

#
#  Generate a new key pair and request a certificate.
#
function requestCertificate()
{
  QNAME=$1
  ALTNAMES=$*
  [ -n "$VERBOSE" ] && echo "Requesting certificate for $*, key size ${KEY_SIZE}"

  #
  # Create an account key if it doesn't exist. This is a one off task
  # that shouldn't need to be repeated.
  #
  if [ ! -f /etc/acme/account.key ] ; then
    [ -n "$VERBOSE" ] && echo "Creating account key"
    umask u=rw,g=r,o=
    openssl genrsa $KEY_SIZE > /etc/acme/account.key
    umask 022
  fi

  #
  # Get a copy of the intermediate 'Lets Encrpty' certificate.
  #
  [ -f /etc/acme/lets-encrypt-x1-cross-signed.pem ] || \
    /bin/wget -O - https://letsencrypt.org/certs/lets-encrypt-x1-cross-signed.pem > /etc/acme/lets-encrypt-x1-cross-signed.pem


  #
  #  Create a new key and generate a certificate signing request
  #
  umask u=rw,g=r,o=
  openssl genrsa $KEY_SIZE > /var/tmp/${QNAME}.key
  # openssl ecparam -genkey -name secp384r1 > /var/tmp/${QNAME}-p384.pem
  # openssl ecparam -genkey -name prime256v1 > /var/tmp/${QNAME}-p256.pem
  umask 022


  openssl req -new \
        -key /var/tmp/${QNAME}.key \
        -sha256 \
        -reqexts SAN \
        -subj "/CN=${QNAME}" \
        -out /var/tmp/${QNAME}.csr \
        -config <(cat "${OPENSSL_CNF}" <( printf "[SAN]\nsubjectAltName=%s\n" "DNS:$( echo $* | sed -e 's/ /,DNS:/g' )") )

  #
  #  If a staging environment is asked for then add it to the parameters
  #
  if [ -n "${STAGING}" ] ; then
    [ -n "$VERBOSE" ] && echo "Using the Lets Encrypt staging API"
    ACME_API="--ca https://acme-staging.api.letsencrypt.org/"
  fi

  #
  #  Get a certificate issued
  #
  acme_tiny.py \
        --account-key /etc/acme/account.key \
        --csr /var/tmp/${QNAME}.csr \
        --acme-dir /var/lib/acme/challenges/ \
        ${ACME_API} \
        > /var/tmp/${QNAME}.crt

  SERIAL=$( openssl x509 -noout  -serial -in /var/tmp/${QNAME}.crt | cut -f2 -d= )
  [ -n "$VERBOSE" ] && echo "New certificate has serial ${SERIAL}"


  #
  #  Copy the key and certificate to an archive directory
  #
  /bin/cp /var/tmp/${QNAME}.key /etc/acme/keys/${QNAME}.key.${SERIAL}
  /bin/cp /var/tmp/${QNAME}.crt /etc/acme/archive/${QNAME}.crt.${SERIAL}

  if [ -n "${STAGING}" ] ; then
    DEPLOY_SUFFIX="-staging"
  fi

  #
  # Link the key into a known place and create the cert with chain. Both
  # files must be readable by nginx
  #
  umask u=rw,g=r,o=
  ln --force /etc/acme/keys/${QNAME}.key.${SERIAL} /etc/acme/live/${QNAME}${DEPLOY_SUFFIX}.key
  umask 022
  cat /etc/acme/archive/${QNAME}.crt.${SERIAL} /etc/acme/lets-encrypt-x1-cross-signed.pem > /etc/acme/live/${QNAME}.letsencrypt${DEPLOY_SUFFIX}.crt

  if  [ ! -f /etc/acme/live/${QNAME}.dh ] ||
        [[ $(date +%s -r /etc/acme/live/${QNAME}.dh ) -lt $(date +%s --date="90 day ago") ]] ; then
    [ -n "$VERBOSE" ] && echo "DH parameters old or not present, regenerating ${DHPARAM_SIZE} bit parameters"
    nice openssl dhparam -rand - ${DHPARAM_SIZE} > /etc/acme/live/${QNAME}.dh
  fi

  #
  # Given a new private key, certificate and DH parameters reload the nginx configuration
  sudo /bin/systemctl reload nginx
}

function isCertExpired()
{
   CERTIFICATE=$1
   MAX_SECONDS_REMAINING=$2

   if [ -f "${CERTIFICATE}" ] ; then
     ENDDATE=$( openssl x509 -noout -enddate -in "${CERTIFICATE}" | sed 's/notAfter=//' )
     if [ -n "${ENDDATE}" ] ; then
       LIFETIME=$(expr $( date -d "$ENDDATE" +"%s" ) - $(date +%s) )
       [ -n "$VERBOSE" ] && echo "Certificate has approx. $( expr ${LIFETIME} / 86400 ) days remaining"
       if [ ${LIFETIME} -lt ${MAX_SECONDS_REMAINING} ] ; then
         [ -n "$VERBOSE" ] && echo "Certificate about to expire and needs reissue"
         return 0
       else
         [ -n "$VERBOSE" ] && echo "Certificate still ok"
       fi
     else
         [ -n "$VERBOSE" ] && echo "Can't determine the end date for certificate ${CERTIFICATE}"
     fi
   else
      [ -n "$VERBOSE" ] && echo "Certificate ${CERTIFICATE} not found"
      return 0  # needs to be issued
   fi
   return  1 # don't reissue
}


function requestCertificateIfRequired()
{
  if [ -n "${STAGING}" ] ; then
    DEPLOY_SUFFIX="-staging"
  fi

  QNAME="$1"
  CERTIFICATE="/etc/acme/live/${QNAME}.letsencrypt${DEPLOY_SUFFIX}.crt"

  if [ -n "${FORCE}" ] ; then
     requestCertificate $*
  elif isCertExpired "${CERTIFICATE}" "${MIN_CERT_LIFETIME}" ; then
     requestCertificate $*
  else
    [ -n "$VERBOSE" ] && echo "Certificate for $QNAME not requested"
  fi
}

usage()
{
  echo "Usage: $0 [-v] [-N] [domainname ...]" 1>&2;
  exit 1;
}

while getopts vNhsf arg; do
  case $arg in
    v)
      VERBOSE=true
      ;;
    N)
      DRY_RUN=true
      ;;
    h)
      usage
      exit;
      ;;
    s)
      STAGING=true
      ;;
    f)
      FORCE=true
      ;;
    ?)
      usage
      exit;
      ;;
  esac
done

shift $(( OPTIND - 1 ));

if [ -n "$*" ] ; then
  requestCertificateIfRequired $*
else
  #
  # process the list of domains
  #
  if [ ! -n "${STAGING}" ] ; then
    DOMAIN_LIST=/etc/acme/domains
  else
    DOMAIN_LIST=/etc/acme/domains-staging
  fi

  [ -n "$VERBOSE" ] && echo "Processing domain list ${DOMAIN_LIST}"
  grep -o '^[^#]*' ${DOMAIN_LIST} | \
  while read -r line || [[ -n "$line" ]]; do
    if [ -n "$line" ] ; then
        [ -n "$VERBOSE" ] && echo "" && echo "Processing line '$line'"
       requestCertificateIfRequired $line
    fi
  done
fi

LetEncrpyt Challenge

This show the log from a successful issue of a certificate. The client first requests the challenge to verify it is present, then the LetsEncrypt server requests it.

2001:4428:225:5::3 - - [16/Jan/2016:20:17:55 +1300] "GET /.well-known/acme-challenge/ZyBrYC544E7pNnC-VEe5m3paMbt8DsbO2wPBkD-cVsw 
    HTTP/1.1" 200 87 "-" "Python-urllib/2.7" "2001:4428:225:7::2"
::ffff:66.133.109.36 - - [16/Jan/2016:20:17:57 +1300] "GET /.well-known/acme-challenge/ZyBrYC544E7pNnC-VEe5m3paMbt8DsbO2wPBkD-cVsw 
    HTTP/1.1" 200 87 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" "-"

Rate Limited

Traceback (most recent call last):
  File "/usr/local/bin/acme_tiny.py", line 201, in <module>
    main(sys.argv[1:])
  File "/usr/local/bin/acme_tiny.py", line 197, in main
    signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, log=LOGGER, CA=args.ca)
  File "/usr/local/bin/acme_tiny.py", line 164, in get_crt
    raise ValueError("Error signing certificate: {0} {1}".format(code, result))
ValueError: Error signing certificate: 429 {"type":"urn:acme:error:rateLimited","detail":"Error creating new cert ::
   Too many certificates already issued for: lucidsolutions.co.nz","status":429}

Sections

Personal tools

LetEncrypt with Nginx on EL7

Residuals

Process

Web site configurations

Challenge web site

Per site redirect

Certificate installation

Certificates configuration

Automatic updates

Links

Lets Encrypt

Misc

ACME

Clients

Appendices

Script

LetEncrpyt Challenge

Rate Limited

Document Actions