Securing Dokku with Let's Encrypt TLS Certificates
TL;DR: My letsencrypt plugin for dokku makes securing dokku webservers with HTTPS a breeze.
Update 2016-01-12: In the meanwhile, my plugin has become the official letsencrypt plugin for dokku. Additionally, some usage and implementation details have changed so I've updated the post to reflect the new status.
Update 2016-03-08: Updated the example to reflect another API change.
Note: While this post is specific to using Let's Encrypt with Dokku, the proxying technique described below might also be of interest to others deploying applications with a reverse proxy setup and can be easily adapted.
Introduction
Since the free certificate authority (CA) Let's Encrypt has now entered public beta, anyone can request SSL/TLS certificates for their web servers that will be accepted across a wide range of browsers. This is perfect for securing small to medium-sized projects which are a blast to run in dokku, the awesome open-source heroku-style git push deployment system based on Docker!
While dokku supports HTTPS through manually providing it with a certificate and private key, obtaining a Let's Encrypt Certificate has previously required temporarily shutting down the nginx webserver powering dokku before running the domain validation process (which needs to bind to ports 80 and/or 443), retrieving the certificates and finally re-starting nginx. For some web applications, such a several-second downtime might not be feasible and is also terribly unelegant, so here is a better way of doing things :-)
A Brief Summary of ACME Domain Validation
In order to prove that you have control over the web server you want to create a certificate for, you will have to go through a Domain Validation process where your certificate authority challenges you to make a resource available whose location and contents are chosen by the certificate authority. The completely automated validation protocol used by Let's Encrypt is called the Automated Certificate Management Environment, short ACME and is also a work-in-progress IETF draft.
Importantly for our purposes, one of the valid ACME domain validation challenges is the Simple Http challenge in which the ACME server challenges the identifying client to make a JWS[1] resource signed by the client available at the path http://mydomain.com/.well-known/acme-challenge/<token>, where <token> is a token chosen by the ACME server as part of the challenge. For validation to succeed, this location has to be reachable from the web on port 80. Normally this constraint is fulfilled by using one of the following authenticators (command line argument -a) in the letsencrypt client:
apache: reconfiguring an Apache webserver using the Apache Let's Encrypt Plugin (not useful for the nginx-based dokku),nginx: reconfiguring the nginx Webserver using the nginx Let's Encrypt Plugin (still considered unstable and not included in the letsencrypt client by default, unsure if how it reconfigures nginx is compatible with dokku)webroot: Placing the JWS resource into the webroot of the pre-configured running webserver bound to port 80 (not practical with dokku since there typically is no webroot)manual: Performing the validation manually (user interaction is involved, not practicable)standalone: Temporarily taking control of port 80 by using the built-in webserver (this is what other tutorials use, but it conflicts with our dokku-managed nginx already listening to port 80!)
Zero-Downtime ACME Domain Validation
From our understanding of the ACME protocol, we can see that we can keep our own application running on port 80/443 as long as ACME requests on port 80/443 are handled correctly. If we forward ACME requests to the letsencrypt standalone webserver while other requests will be forwarded to our app, both the standalone webserver and our app can happily coexist!
For every app on the nginx webserver managed by dokku, a nginx config file located in /home/dokku/myapp/nginx.conf will be generated that handles proxying the requests arriving at port 80 of the web server to the app running within the container:
server {
listen [::]:80;
listen 80;
server_name myapp.mydomain.com;
access_log /var/log/nginx/myapp-access.log;
error_log /var/log/nginx/myapp-error.log;
location / {
# gzip settings omitted for brevity
proxy_pass http://myapp;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection upgrade;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Request-Start $msec;
}
include /home/dokku/myapp/nginx.conf.d/*.conf;
}
upstream myapp {
server 172.17.0.4:5000;
}
Since the generated config conveniently contains an include statement to load additional config files from /home/dokku/myapp/nginx.conf.d/, we can place a temporary config file there to add a reverse proxy to 127.0.0.1:8888 where we will run the Let's Encrypt standalone webserver:
location /.well-known {
proxy_pass http://127.0.0.1:8888;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Request-Start $msec;
}
After reloading the nginx config, ACME requests will now be forwarded to port 8888 on localhost.
Finally, we can use the dockerized simp_le Let's Encrypt Client to bind the standalone webserver to port 8888 and begin the ACME validation and certificate download:
$ docker run --rm -it \
-p 8888:80
-v "$CERTDIR:/certs" \
m3adow/docker-letsencrypt-simp_le \
--email my@email.tld \
-f account_key.json \
-f key.pem -f cert.pem -f chain.pem -f fullchain.pem \
-d myapp.mydomain.com \
Once the command completes successfully, we will have a full-chain certificate fullchain.pem and private key key.pem in $CERTDIR that can be installed in dokku using the certs:add command.
Finally, we remove the temporary reverse proxy and reload the nginx configuration.
dokku-letsencrypt
I've implemented the above workflow in my letsencrypt plugin for dokku (now the official dokku plugin for let's encrypt) which lets you verify your domain and install the certificate in one dokku command. I would be happy to hear about your experiences and pull requests are welcome!
$ dokku config:set --no-restart myapp DOKKU_LETSENCRYPT_EMAIL=your@email.tld
-----> Setting config vars
DOKKU_LETSENCRYPT_EMAIL: your@email.tld
$ dokku letsencrypt myapp
=====> Let's Encrypt myapp...
-----> Updating letsencrypt docker image...
latest: Pulling from m3adow/letsencrypt-simp_le
Digest: sha256:20f2a619795c1a3252db6508f77d6d3648ad5b336e67caaf801126367dbdfa22
Status: Image is up to date for m3adow/letsencrypt-simp_le:latest
done
-----> Enabling ACME proxy for myapp...
-----> Getting letsencrypt certificate for myapp...
- Domain 'myapp.mydomain.com'
hash of all pertinent configuration settings is a131be342a0d7661817a4c23b1a767f5da5abbf3
[ removed various log messages for brevity ]
-----> Certificate retrieved successfully.
-----> Symlinking let's encrypt certificates
-----> Configuring SSL for myapp.mydomain.com...(using /var/lib/dokku/plugins/available/nginx-vhosts/templates/nginx.ssl.conf.template)
-----> Creating https nginx.conf
-----> Running nginx-pre-reload
Reloading nginx
-----> Disabling ACME proxy for myapp...
done