I have a bash script that creates nginx virtual hosts and then requests a ssl via certbot let's encrypt

The problem I'm running into is the bash script works great for the first 3-5 domains but then it starts throwing an unauthorized error but another weird thing is the error at the end says 404 not 403 and the letsencrypt log shows 200 repsonse example(I've replaced some sensitive info for privacy) ALSO keep in mind in may be relavant in answer all these virtual hosts I want to put to the same root/directory so if that has a part in it or may let me know:

HTTP 200
Server: nginx
Date: Sun, 01 May 2022 14:30:03 GMT
Content-Type: application/json
Content-Length: 1036
Connection: keep-alive
Boulder-Requester: 494001960
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: asdfasdfasdfasdfasdf
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "identifier": {
    "type": "dns",
    "value": "www.1.1.1.1.1(hiddenfor privacy).com"
  },
  "status": "invalid",
  "expires": "2022-05-08T14:29:24Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "invalid",
      "error": {
        "type": "urn:ietf:params:acme:error:unauthorized",
        "detail": "1.1.1.1.1(hiddenfor privacy): Invalid response from http://www.example.com/.well-known/acme-challenge/asdfasdfdasfsdf: 404",
        "status": 403
      },
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/asdfasdf/KIm4sQ",
      "token": ";alskdjf;laskdjf;lka",
      "validationRecord": [
        {
          "url": "http://www.example.com/.well-known/acme-challenge/a;sldkfja;sldkfja;lsdkfj",
          "hostname": "www.example.com",
          "port": "80",
          "addressesResolved": [
            "1.1.1.1.1(hiddenfor privacy)"
          ],
          "addressUsed": "1.1.1.1.1(hiddenfor privacy)"
        }
      ],
      "validated": "2022-05-01T14:30:02Z"
    }
  ]
}

I also get this in the nginx error log

2022/05/01 14:36:50 [error] 148403#148403: *1255 open() "/var/www/example_lander/example_example/.well-known/acme-challenge/asdfasdf" failed (2: No such file or directory), client: 35.89.74.29, server: example.com, request: "GET /.well-known/acme-challenge/asdfasdfasdfasdfasdfHTTP/1.1", host: "www.example.com"

here's the bash script I use to create virtual host and request ssl

# process for creating ssl and domain settings
cd /etc/nginx/sites-available/
wait
sudo sh -c "echo 'server {
    server_name $1 www.$1;
    root /var/www/example_lander/example_example;

    index index.html index.htm index.php;

     location ^~ /.well-known {
                allow all;
        }

    location / {
        try_files \$uri \$uri/ =404;
    }


    location ~ \.php$ {
        include snippets/fastcgi-php.conf;
        fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
     }

    location ~ /\.ht {
        deny all;
    }
}' > $1"
wait
sudo sh -c "ln -s /etc/nginx/sites-available/$1 /etc/nginx/sites-enabled/"
wait
sudo sh -c "service nginx reload"
wait
sudo sh -c "certbot --nginx -d $1 -d www.$1 --redirect --agree-tos"
wait

so as you can see I've also added the well_known error that seems to happen a lot to other users with let's encrypt but still get stuck witht hese 404/403 /200 random stuff that happens after 3-5 successful times of the bash script running successfully

CodePudding user response：

So it ended up being that the CPU usage was high due to certbot so in AWS I enabled unlimited mode for my t3.medium to allow the certs to be issued since there is a lot and that seemed to solve the problem basically my theory is that certbot most likely couldn't issue the cert quickly enough by the time the challenge happened so it would fail challenges before deploying due to the high cpu usage of certbot

CodePudding user response：

Have you tried standalone challenge? So you don't have to create a server for each domain, as long as DNS record of the domain points to the server running the challenge should do the job.

Example:

#!/bin/bash

certbot certonly --standalone -d "$1" -d "www.$1" \
--non-interactive --agree-tos --email <your_email> # \
# --pre-hook="<stop_nginx_maybe>" \ # stop anything that is running on port 80
# --post-hook="<start_nginx_maybe>" # start anything that was running on port 80

# print certificate
cat /etc/letsencrypt/live/$1/{fullchain.pem,privkey.pem}

And then you can add an entry to crontab that runs a renew script which after renew runs some code for each certificate.