Lua error when renewing certs

(See the actual error at the end of this post)

Everything was working fine. This is my `/usr/local/openresty/nginx/conf/nginx.conf` file:

```bash
http {
# ...

# Initial setup tasks.
init_by_lua_block {
auto_ssl = (require "resty.auto-ssl").new()

auto_ssl:set("allow_domain", function(domain)
-- Do not generate a certificate for our domains
local regex = [=[^(\S*\.?mysite\.org)$]=]
if ngx.re.match(domain, regex, "ijo") then
return true
end

-- Custom domains
if string.find(domain, ".mysite.org") == nil then
local http = require("resty.http")
local httpc = http.new()

httpc:set_timeout(3000)

local uri = os.getenv("ALLOW_DOMAIN_API")..domain.."/"
local res, err = httpc:request_uri(uri, {
ssl_verify = true,
method = "GET"
})
-- Only allow domain if our endpoint responds with HTTP 200,
-- disallow if 404, other response code or error
return res and res.status == 200
end

return false
end)

auto_ssl:init()
}

init_worker_by_lua_block {
auto_ssl:init_worker()
}

include ../sites/enabled/*;
}
```

We have domains of our own as `*.mysite.org` and third party custom domains. The `allow_domain` function returns true straight away if the domain is ours and if it is not, it checks with our backend to determine whether it's allowed and a cert should be issued.

Then, our server configuration at `/usr/local/openresty/nginx/sites/enabled/mysite.conf/`:

```bash
upstream app_server {
server unix:/home/myapp/run/gunicorn.sock fail_timeout=0;
}

server {
listen 80 default_server;
listen [::]:80 default_server;

# Redirect everything to https
location / {
return 301 https://$host$request_uri;
}

# Endpoint used by auto-ssl for performing domain verification with Let's Encrypt
location /.well-known/acme-challenge/ {
content_by_lua_block {
auto_ssl:challenge_server()
}
}
}

server {
server_name mysite.org www.mysite.org;
return 301 https://myapp.mysite.org$request_uri;

listen 443 ssl http2;
listen [::]:443 ssl http2;

# auto-ssl block that handles on-the-fly SSL cert generation
ssl_certificate_by_lua_block {
auto_ssl:ssl_certificate()
}

# dummy certs required for the test to pass
ssl_certificate /etc/letsencrypt/live/myapp.mysite.org/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/myapp.mysite.org/privkey.pem; # managed by Certbot

include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
}

server {
server_name myapp.mysite.org proxy.mysite.org;

listen 443 ssl http2 default_server;
listen [::]:443 ssl http2 default_server;

keepalive_timeout 5;
client_max_body_size 4G;

access_log /home/myapp/logs/openresty/access.log;
error_log /home/myapp/logs/openresty/error.log;

location /static/ {
alias /home/myapp/myapp/static/;
expires 365d;
}

location /site_media/ {
alias /home/myapp/myapp/media/;
expires 365d;
}

# checks for static file, if not found proxy to app
location / {
try_files $uri @proxy_to_app;
}

location @proxy_to_app {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
proxy_redirect off;
proxy_pass http://app_server;
}

error_page 502 /502.html;
location = /502.html {
root /home/myapp/myapp/error_page/;
}

# auto-ssl block that handles on-the-fly SSL cert generation
ssl_certificate_by_lua_block {
auto_ssl:ssl_certificate()
}

# dummy certs
ssl_certificate /etc/letsencrypt/live/myapp.mysite.org/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/myapp.mysite.org/privkey.pem; # managed by Certbot

include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
}

# Internal server running on port 8999 for handling certificate tasks (by auto-ssl)
server {
listen 127.0.0.1:8999;

# Increase the body buffer size, to ensure the internal POSTs can always
# parse the full POST contents into memory
client_body_buffer_size 128k;
client_max_body_size 128k;

location / {
content_by_lua_block {
auto_ssl:hook_server()
}
}
}
```

The issue: certs weren't getting renewed for two domains. One of our own (`www.mysite.org`) and a custom one (`www.someone.org`). They were expired and just wouldn't get renewed. Restarting the server didn't help.

In the end, I had to delete the cert files `/etc/resty-auto-ssl/storage/file/www.mysite.org%3Alatest` and `/etc/resty-auto-ssl/storage/file/www.someone.org%3Alatest` and restart openresty to force new certs to be generated.

Why wasn't resty auto ssl renewing them and what can I do to prevent this from happening again in the future?

Edit:

This is the actual logged error:

```
2021/09/01 06:16:40 [error] 16675#16675: *3276968 lua entry thread aborted: runtime error: ...sty/luajit/share/lua/5.1/resty/auto-ssl/servers/hook.lua:43: assertion failed!
stack traceback:
coroutine 0:
[C]: in function 'assert'
...sty/luajit/share/lua/5.1/resty/auto-ssl/servers/hook.lua:43: in function 'server'
.../local/openresty/luajit/share/lua/5.1/resty/auto-ssl.lua:95: in function 'hook_server'
content_by_lua(scheduler.conf:112):2: in main chunk, client: 127.0.0.1, server: , request: "POST /deploy-cert HTTP/1.1", host: "127.0.0.1:8999"
```
I am also getting this one, but less commonly:

```
2021/08/31 00:41:15 [crit] 25738#25738: *2177881 SSL_do_handshake() failed (SSL: error:1414F178:SSL routines:tls1_set_server_sigalgs:no shared signature algorithms error:1417D0E2:SSL routines:tls_process_client_hello:clienthello tlsext) while SSL handshaking, client: 2001:4ca0:108:42::5, server: [::]:443
```

Start with the logged assertion at resty/auto-ssl/servers/hook.lua:43 and its caller in resty/auto-ssl.lua:95, then trace the /deploy-cert request handled by the port-8999 hook_server configuration. Compare that flow with the stored certificate paths and renewal errors for both domains. Done means identifying the renewal failure and documenting a reliable prevention or recovery path.

Written by the indexing model from the issue text.