changes on custom acme server/change compression mime types

Signed-off-by: Zoey <[email protected]>
ZoeyVid · Oct 20, 2024 · 927d5ca · 927d5ca
1 parent 7c92e83
commit 927d5ca
Show file tree

Hide file tree

Showing 28 changed files with 323 additions and 514 deletions.
diff --git a/Dockerfile b/Dockerfile
@@ -57,6 +57,7 @@ RUN apk upgrade --no-cache -a && \
     mv crowdsec-nginx-bouncer-* crowdsec-nginx-bouncer && \
     sed -i "/lua_package_path/d" /src/crowdsec-nginx-bouncer/nginx/crowdsec_nginx.conf && \
     sed -i "s|/etc/crowdsec/bouncers/crowdsec-nginx-bouncer.conf|/data/etc/crowdsec/crowdsec.conf|g" /src/crowdsec-nginx-bouncer/nginx/crowdsec_nginx.conf && \
+    sed -i "s|crowdsec-nginx-bouncer|crowdsec-npmplus-bouncer|g" /src/crowdsec-nginx-bouncer/nginx/crowdsec_nginx.conf && \
     sed -i "s|API_KEY=.*|API_KEY=|g" /src/crowdsec-nginx-bouncer/lua-mod/config_example.conf && \
     sed -i "s|ENABLED=.*|ENABLED=false|g" /src/crowdsec-nginx-bouncer/lua-mod/config_example.conf && \
     sed -i "s|API_URL=.*|API_URL=http://127.0.0.1:8080|g" /src/crowdsec-nginx-bouncer/lua-mod/config_example.conf && \
@@ -70,18 +71,18 @@ RUN apk upgrade --no-cache -a && \
     sed -i "s|APPSEC_PROCESS_TIMEOUT=.*|APPSEC_PROCESS_TIMEOUT=10000|g" /src/crowdsec-nginx-bouncer/lua-mod/config_example.conf
 
 
-FROM zoeyvid/nginx-quic:347-python
+FROM zoeyvid/nginx-quic:349-python
 SHELL ["/bin/ash", "-eo", "pipefail", "-c"]
 COPY rootfs /
-COPY --from=zoeyvid/certbot-docker:58 /usr/local          /usr/local
+COPY --from=zoeyvid/certbot-docker:59 /usr/local          /usr/local
 COPY --from=zoeyvid/curl-quic:420     /usr/local/bin/curl /usr/local/bin/curl
 
 ARG CRS_VER=v4.7.0
 RUN apk upgrade --no-cache -a && \
     apk add --no-cache ca-certificates tzdata tini \
     nodejs \
     bash nano \
-    logrotate apache2-utils \
+    logrotate \
     lua5.1-lzlib lua5.1-socket \
     coreutils grep findutils jq shadow su-exec \
     luarocks5.1 lua5.1-dev lua5.1-sec build-base git yarn && \
@@ -123,7 +124,8 @@ ENV NODE_ENV=production \
 # until https://github.com/certbot/certbot/issues/9967 is closed
 ENV PYTHONWARNINGS=ignore
 
-ENV PUID=0 \
+ENV ACME_SERVER="https://acme-v02.api.letsencrypt.org/directory" \
+    PUID=0 \
     PGID=0 \
     NIBEP=48693 \
     GOAIWSP=48683 \

diff --git a/README.md b/README.md
@@ -20,7 +20,6 @@ running at home or otherwise, including free TLS, without having to know too muc
 **Note: ModSecurity overblocking (403 Error)? Please see `/opt/npm/etc/modsecurity`, if you also use CRS please see [here](https://coreruleset.org/docs/concepts/false_positives_tuning).** <br>
 **Note: Other Databases like MariaDB may work, but are unsupported.** <br>
 **Note: access.log/stream.log, logrotate and goaccess are NOT enabled by default bceuase of GDPR, you can enable them in the compose.yaml.** <br>
-**Note: if you remove a cert, which is still used by a host, NPM/NPMplus will crash.** <br>
 
 
 ## Project Goal
@@ -97,6 +96,28 @@ so that the barrier for entry here is low.
 - if you have a healthcheck defined in your compose yaml file, remove it - this fork defines its own healthcheck in the Dockerfile, so you don't need to have it in compose anymore
 - please report all migration issues you have
 
+# Quick Setup
+1. Install Docker and Docker Compose (or portainer)
+- [Docker Install documentation](https://docs.docker.com/engine)
+- [Docker Compose Install documentation](https://docs.docker.com/compose/install/linux)
+2. Create a compose.yaml file similar to [this](https://github.com/ZoeyVid/NPMplus/blob/develop/compose.yaml) (or use it as a portainer stack):
+3. Bring up your stack by running (or deploy your portainer stack)
+```bash
+docker compose up -d
+```
+4. Log in to the Admin UI
+When your docker container is running, connect to it on port `81` for the admin interface.
+Sometimes this can take a little bit because of the entropy of keys.
+You may need to open port 81 in your firewall.
+You may need to use another IP-Address.
+[https://127.0.0.1:81](https://127.0.0.1:81)
+Default Admin User:
+```
+Email:    [email protected]
+Password: iArhP1j7p1P6TA92FA2FMbbUGYqwcYzxC4AVEe12Wbi94FY9gNN62aKyF1shrvG4NycjjX9KfmDQiwkLZH1ZDR9xMjiG2QmoHXi
+```
+Immediately after logging in with this default user you will be asked to modify your details and change your password.
+
 # Crowdsec
 1. Install crowdsec using this compose file: https://github.com/ZoeyVid/NPMplus/blob/develop/compose.crowdsec.yaml and enable LOGROTATE
 2. open `/opt/crowdsec/conf/acquis.d/npmplus.yaml` and fill it with:
@@ -175,34 +196,6 @@ location / {
 }
 ```
 
-# custom acme server
-1. Open this file: `nano` `/opt/npm/ssl/certbot/config.ini`
-2. uncomment the server line and change it to your acme server
-3. maybe set eab keys
-4. create your cert using the npm web ui
-
-# Quick Setup
-1. Install Docker and Docker Compose (or portainer)
-- [Docker Install documentation](https://docs.docker.com/engine)
-- [Docker Compose Install documentation](https://docs.docker.com/compose/install/linux)
-2. Create a compose.yaml file similar to [this](https://github.com/ZoeyVid/NPMplus/blob/develop/compose.yaml) (or use it as a portainer stack):
-3. Bring up your stack by running (or deploy your portainer stack)
-```bash
-docker compose up -d
-```
-4. Log in to the Admin UI
-When your docker container is running, connect to it on port `81` for the admin interface.
-Sometimes this can take a little bit because of the entropy of keys.
-You may need to open port 81 in your firewall.
-You may need to use another IP-Address.
-[https://127.0.0.1:81](https://127.0.0.1:81)
-Default Admin User:
-```
-Email:    [email protected]
-Password: iArhP1j7p1P6TA92FA2FMbbUGYqwcYzxC4AVEe12Wbi94FY9gNN62aKyF1shrvG4NycjjX9KfmDQiwkLZH1ZDR9xMjiG2QmoHXi
-```
-Immediately after logging in with this default user you will be asked to modify your details and change your password.
-
 ### prerun scripts (EXPERT option) - if you don't know what this is, ignore it
 run order: entrypoint.sh (prerun scripts) => start.sh => launch.sh <br>
 if you need to run scripts before NPMplus launches put them under: `/opt/npm/etc/prerun/*.sh` (please add `#!/bin/sh` / `#!/bin/bash` to the top of the script) <br>
@@ -221,6 +214,4 @@ If you want to sponsor them, please see [here](https://github.com/NginxProxyMana
 ## Getting Support
 1. [Found a bug?](https://github.com/ZoeyVid/NPMplus/issues)
 2. [Discussions](https://github.com/ZoeyVid/NPMplus/discussions)
-<!---
-3. [Reddit](https://reddit.com/r/nginxproxymanager)
---->
+3. [Reddit](https://reddit.com/r/NPMplus)
diff --git a/backend/internal/access-list.js b/backend/internal/access-list.js
@@ -1,5 +1,6 @@
 const _ = require('lodash');
 const fs = require('fs');
+const md5 = require('apache-md5');
 const batchflow = require('batchflow');
 const logger = require('../logger').access;
 const error = require('../lib/error');
@@ -483,15 +484,17 @@ const internalAccessList = {
 							if (typeof item.password !== 'undefined' && item.password.length) {
 								logger.info('Adding: ' + item.username);
 
-								utils
-									.execFile('htpasswd', ['-b', htpasswd_file, item.username, item.password])
-									.then((/* result */) => {
-										next();
-									})
-									.catch((err) => {
-										logger.error(err);
-										next(err);
-									});
+								try {
+									const hashedPassword = md5(item.password);
+									fs.appendFileSync(htpasswd_file, `${item.username}:${hashedPassword}\n`, { encoding: 'utf8' });
+									next();
+								} catch (err) {
+									logger.error(err);
+									next(err);
+								}
+							} else {
+								// Proceed to next if no password
+								next();
 							}
 						})
 						.error((err) => {

diff --git a/backend/internal/certificate.js b/backend/internal/certificate.js
@@ -15,7 +15,8 @@ const dnsPlugins = require('../certbot-dns-plugins.json');
 const internalAuditLog = require('./audit-log');
 const internalNginx = require('./nginx');
 
-const certbotCommand = 'certbot --logs-dir /tmp/certbot-log --work-dir /tmp/certbot-work --config-dir /data/tls/certbot --agree-tos -non-interactive --config /etc/tls/certbot.ini';
+const certbotCommand = 'certbot';
+const certbotArgs = ['--logs-dir', '/tmp/certbot-log', '--work-dir', '/tmp/certbot-work', '--config-dir', '/data/tls/certbot', '--config', '/etc/tls/certbot.ini', '--agree-tos', '--non-interactive', '--no-eff-email', '--register-unsafely-without-email'];
 
 function omissions() {
 	return ['is_deleted', 'owner.is_deleted'];
@@ -40,10 +41,8 @@ const internalCertificate = {
 			internalCertificate.intervalProcessing = true;
 			logger.info('Renewing TLS certs close to expiry...');
 
-			const cmd = '${certbotCommand} renew --quiet';
-
 			return utils
-				.exec(cmd)
+				.execFile(certbotCommand, [...certbotArgs, 'renew', '--quiet', '--no-random-sleep-on-renew'])
 				.then((result) => {
 					if (result) {
 						logger.info('Renew Result: ' + result);
@@ -498,7 +497,6 @@ const internalCertificate = {
 	 * @param   {Object}   data
 	 * @param   {Array}    data.domain_names
 	 * @param   {String}   data.meta.letsencrypt_email
-	 * @param   {Boolean}  data.meta.letsencrypt_agree
 	 * @returns {Promise}
 	 */
 	createQuickCertificate: (access, data) => {
@@ -619,7 +617,7 @@ const internalCertificate = {
 				reject(new error.ValidationError('Result Validation Error: Validation timed out. This could be due to the key being passphrase-protected.'));
 			}, 10000);
 			utils
-				.exec('openssl pkey -in ' + filepath + ' -check -noout 2>&1 ')
+				.execFile('openssl', ['pkey', '-in', filepath, '-check', '-noout'])
 				.then((result) => {
 					clearTimeout(failTimeout);
 					if (!result.toLowerCase().includes('key is valid')) {
@@ -670,7 +668,7 @@ const internalCertificate = {
 		const certData = {};
 
 		return utils
-			.exec('openssl x509 -in ' + certificate_file + ' -subject -noout')
+			.execFile('openssl', ['x509', '-in', certificate_file, '-subject', '-noout'])
 			.then((result) => {
 				const regex = /(?:subject=)?[^=]+=\s*(\S+)/gim;
 				const match = regex.exec(result);
@@ -679,7 +677,7 @@ const internalCertificate = {
 				}
 			})
 			.then(() => {
-				return utils.exec('openssl x509 -in ' + certificate_file + ' -issuer -noout');
+				return utils.execFile('openssl', ['x509', '-in', certificate_file, '-issuer', '-noout']);
 			})
 
 			.then((result) => {
@@ -690,7 +688,7 @@ const internalCertificate = {
 				}
 			})
 			.then(() => {
-				return utils.exec('openssl x509 -in ' + certificate_file + ' -dates -noout');
+				return utils.execFile('openssl', ['x509', '-in', certificate_file, '-dates', '-noout']);
 			})
 			.then((result) => {
 				// notBefore=Jul 14 04:04:29 2018 GMT
@@ -763,11 +761,7 @@ const internalCertificate = {
 	requestCertbot: (certificate) => {
 		logger.info('Requesting Certbot certificates for Cert #' + certificate.id + ': ' + certificate.domain_names.join(', '));
 
-		let cmd = `${certbotCommand} certonly --cert-name "npm-${certificate.id}" --domains "${certificate.domain_names.join(',')}" --server "${process.env.ACME_SERVER}" --authenticator webroot --webroot-path "/tmp/acme-challenge"`;
-
-		logger.info('Command:', cmd);
-
-		return utils.exec(cmd).then((result) => {
+		return utils.execFile(certbotCommand, [...certbotArgs, 'certonly', '--cert-name', `npm-${certificate.id}`, '--domains', `${certificate.domain_names.join(',')}`, '--server', `${process.env.ACME_SERVER}`, '--authenticator', 'webroot', '--webroot-path', '/tmp/acme-challenge']).then((result) => {
 			logger.success(result);
 			return result;
 		});
@@ -792,12 +786,8 @@ const internalCertificate = {
 		fs.mkdirSync('/data/tls/certbot/credentials', { recursive: true });
 		fs.writeFileSync(credentialsLocation, certificate.meta.dns_provider_credentials, { mode: 0o600 });
 
-		let mainCmd = `${certbotCommand} certonly --cert-name "npm-${certificate.id}" --domains "${certificate.domain_names.join(',')}" --server "${process.env.ACME_SERVER}" --authenticator ${dnsPlugin.full_plugin_name} --${dnsPlugin.full_plugin_name}-credentials "${credentialsLocation}"`;
-
-		logger.info('Command:', mainCmd);
-
 		try {
-			const result = await utils.exec(mainCmd);
+			const result = await utils.execFile(certbotCommand, [...certbotArgs, 'certonly', '--cert-name', `npm-${certificate.id}`, '--domains', `${certificate.domain_names.join(',')}`, '--server', `${process.env.ACME_SERVER}`, '--authenticator', dnsPlugin.full_plugin_name, `--${dnsPlugin.full_plugin_name}-credentials`, credentialsLocation]);
 			logger.info(result);
 			return result;
 		} catch (err) {
@@ -857,18 +847,10 @@ const internalCertificate = {
 	renewCertbot: async (certificate) => {
 		logger.info(`Renewing Certbot certificates for Cert #${certificate.id}: ${certificate.domain_names.join(', ')}`);
 
-		const cmdr = `${certbotCommand} revoke --cert-path "/data/tls/certbot/live/npm-${certificate.id}/fullchain.pem" --cert-path "/data/tls/certbot/live/npm-${certificate.id}/privkey.pem" --no-delete-after-revoke`;
-
-		logger.info('Command:', cmdr);
-
-		const revokeResult = await utils.exec(cmdr);
+		const revokeResult = await utils.execFile(certbotCommand, [...certbotArgs, 'revoke', '--cert-name', `npm-${certificate.id}`, '--no-delete-after-revoke']);
 		logger.info(revokeResult);
 
-		const cmd = `${certbotCommand} renew --force-renewal --cert-name "npm-${certificate.id}"`;
-
-		logger.info('Command:', cmd);
-
-		const renewResult = await utils.exec(cmd);
+		const renewResult = await utils.execFile(certbotCommand, [...certbotArgs, 'renew', '--force-renewal', '--cert-name', `npm-${certificate.id}`, '--no-random-sleep-on-renew']);
 		logger.info(renewResult);
 
 		return renewResult;
@@ -887,18 +869,10 @@ const internalCertificate = {
 
 		logger.info(`Renewing Certbot certificates via ${dnsPlugin.name} for Cert #${certificate.id}: ${certificate.domain_names.join(', ')}`);
 
-		const cmdr = `${certbotCommand} revoke --cert-path "/data/tls/certbot/live/npm-${certificate.id}/fullchain.pem" --cert-path "/data/tls/certbot/live/npm-${certificate.id}/privkey.pem" --no-delete-after-revoke`;
-
-		logger.info('Command:', cmdr);
-
-		const revokeResult = await utils.exec(cmdr);
+		const revokeResult = await utils.execFile(certbotCommand, [...certbotArgs, 'revoke', '--cert-name', `npm-${certificate.id}`, '--no-delete-after-revoke']);
 		logger.info(revokeResult);
 
-		const cmd = `${certbotCommand} renew --force-renewal --cert-name "npm-${certificate.id}"`;
-
-		logger.info('Command:', cmd);
-
-		const renewResult = await utils.exec(cmd);
+		const renewResult = await utils.execFile(certbotCommand, [...certbotArgs, 'renew', '--force-renewal', '--cert-name', `npm-${certificate.id}`, '--no-random-sleep-on-renew']);
 		logger.info(renewResult);
 
 		return renewResult;
@@ -912,10 +886,8 @@ const internalCertificate = {
 	revokeCertbot: (certificate, throw_errors) => {
 		logger.info('Revoking Certbot certificates for Cert #' + certificate.id + ': ' + certificate.domain_names.join(', '));
 
-		const mainCmd = `${certbotCommand} revoke --cert-path "/data/tls/certbot/live/npm-${certificate.id}/fullchain.pem" --cert-path "/data/tls/certbot/live/npm-${certificate.id}/privkey.pem" --no-delete-after-revoke`;
-
 		return utils
-			.exec(mainCmd)
+			.execFile(certbotCommand, [...certbotArgs, 'revoke', '--cert-name', `npm-${certificate.id}`, '--no-delete-after-revoke'])
 			.then(async (result) => {
 				fs.rm('/data/tls/certbot/credentials/credentials-' + certificate.id, { force: true }, (err) => {
 					if (err) {
@@ -939,64 +911,6 @@ const internalCertificate = {
 			});
 	},
 
-	/**
-	 * @param {Object}  in_use_result
-	 * @param {Number}  in_use_result.total_count
-	 * @param {Array}   in_use_result.proxy_hosts
-	 * @param {Array}   in_use_result.redirection_hosts
-	 * @param {Array}   in_use_result.dead_hosts
-	 */
-	disableInUseHosts: (in_use_result) => {
-		if (in_use_result.total_count) {
-			const promises = [];
-
-			if (in_use_result.proxy_hosts.length) {
-				promises.push(internalNginx.bulkDeleteConfigs('proxy_host', in_use_result.proxy_hosts));
-			}
-
-			if (in_use_result.redirection_hosts.length) {
-				promises.push(internalNginx.bulkDeleteConfigs('redirection_host', in_use_result.redirection_hosts));
-			}
-
-			if (in_use_result.dead_hosts.length) {
-				promises.push(internalNginx.bulkDeleteConfigs('dead_host', in_use_result.dead_hosts));
-			}
-
-			return Promise.all(promises);
-		} else {
-			return Promise.resolve();
-		}
-	},
-
-	/**
-	 * @param {Object}  in_use_result
-	 * @param {Number}  in_use_result.total_count
-	 * @param {Array}   in_use_result.proxy_hosts
-	 * @param {Array}   in_use_result.redirection_hosts
-	 * @param {Array}   in_use_result.dead_hosts
-	 */
-	enableInUseHosts: (in_use_result) => {
-		if (in_use_result.total_count) {
-			const promises = [];
-
-			if (in_use_result.proxy_hosts.length) {
-				promises.push(internalNginx.bulkGenerateConfigs('proxy_host', in_use_result.proxy_hosts));
-			}
-
-			if (in_use_result.redirection_hosts.length) {
-				promises.push(internalNginx.bulkGenerateConfigs('redirection_host', in_use_result.redirection_hosts));
-			}
-
-			if (in_use_result.dead_hosts.length) {
-				promises.push(internalNginx.bulkGenerateConfigs('dead_host', in_use_result.dead_hosts));
-			}
-
-			return Promise.all(promises);
-		} else {
-			return Promise.resolve();
-		}
-	},
-
 	testHttpsChallenge: async (access, domains) => {
 		await access.can('certificates:list');