Questions about settings and security (Clouflare tunnel setup)

[Xiaoh-123]

Hi !
First of all I want to thank the people behind this project and in the community who have made my life way easier with Meshcentral.

Now to my issues. Being the IT guy for my family and friends, I've always dreamt of something like Meshcentral to help me centralize all the computers I had to support, while also having a tool to do so remotely, and this project is just perfect for that. However, having just two years of homelabing behind me, I'm no expert and I feel like I'm doing something bad (or at least perfectible) with my Meshcentral instance.
I used tteck's script to get Meshcentral (currently 1.1.27) running inside an LXC on Proxmox 8.2. Since I have a Cloudflare tunnel setup already for some other services, I jumped on the opportunity to use that as a way to expose Meshcentral.
First, I must mention that I struggled immensely to get Meshcentral to work, and it was only thanks the Github and Reddit that I managed to make it work. And while everything is working fine, I'm pretty sure that what I did to make it work is not good, especially from a security standpoint.

Here's my config.json file :

{
  "$schema": "https://raw.githubusercontent.com/Ylianst/MeshCentral/master/meshcentral-config-schema.json",
  "__comment1__": "This is a simple configuration file, all values and sections that start with underscore (_) are ignored. Edit a section and remove the _ in front of the name. Refer to the user's guide for details.",
  "__comment2__": "See node_modules/meshcentral/sample-config-advanced.json for a more advanced example.",
  "settings": {
    "cert": "meshcentral.mydomain.tld",
    "_tlsOffload": true,
    "trustedproxy": "CloudFlare",
    "ignoreAgentHashCheck": true,
    "_WANonly": true,
    "_LANonly": true,
    "_sessionKey": "MyReallySecretPassword1",
    "_port": 80,
    "_aliasPort": 443,
    "_redirPort": 800,
    "_redirAliasPort": 80
  },
  "domains": {
    "": {
      "certUrl": "http://meshcentral.mydomain.tld:443",
      "_title": "MyServer",
      "_title2": "Servername",
      "_minify": true,
      "_newAccounts": true,
      "_userNameIsEmail": true
    }
  },
  "_letsencrypt": {
    "__comment__": "Requires NodeJS 8.x or better, Go to https://letsdebug.net/ first before trying Let's Encrypt.",
    "email": "[email protected]",
    "names": "myserver.mydomain.com",
    "skipChallengeVerification": true,
    "production": false
  }
}

My setup looks like this:

• I own mydomain.tld with Cloudflare
• gateway: 10.0.0.1
• Proxmox host: 10.0.0.99
• Docker LXC that runs cloudflared: 10.0.0.101
• Meshcentral LXC: 10.0.0.104
• Tunnel target in Cloudflare management panel: https://10.0.0.104:443

What I think is wrong is that I have to use "ignoreAgentHashCheck": true and in the Cloudflare panel I have to turn on the No TLS verify option. Reverting any of these two settings breaks the communication between server and agents both ways.
Do you think that there's a better way to set things up so that I can have it work without compromising on settings that look bad (to me)? Or are these settings actually not a risk?
I'm also unsure of what to do with tlsoffload and trustedproxy, because the former seems to break everything if turned on with any IP or domain or just yes as a parameter, and the latter seems to do nothing, but I was under the impression that they mattered.

Thanks in advance for your help, please ask for more info if needed.

[si458]

Using ignoreAgentHashCheck is a bad idea, because it basically ignores any checks for the remote agents!
What u need to do if set _tlsOffload to tlsOffload and make sure it's true,
Remove the ignoreAgentHashCheck
Then tell cloudflare to access http://meship:443 NOT HTTPS!

[Xiaoh-123]

Thank you for your help!
Thanks to your advice I realized what prevented the settings you suggested to work when I tried them before,
Basically the issue came from my local network setup: I use an Nginx Proxy Manager instance to provide local certificates with a DNS challenge, and I use AdGuard Home to rewrite all DNS queries directed to *.mydomain.tld to NginxPM. The idea behind this somewhat convoluted setup was to be able to access all my services through FQDN addresses, whether I'm querying from LAN or WAN.
The issue I had was that the NginxPM proxy host config also needed to point to http://meshcentral.mydomain.tld:443 (and not HTTPS).
So now everything works perfectly if I want to access the Meshcentral GUI from WAN or LAN, and clients work like a charm.

Along is my new config.json if it may help someone else:

{
  "$schema": "https://raw.githubusercontent.com/Ylianst/MeshCentral/master/meshcentral-config-schema.json",
  "__comment1__": "This is a simple configuration file, all values and sections that start with underscore (_) are ignored. Edit a section and remove the _ in front of the name. Refer to the user's guide for details.",
  "__comment2__": "See node_modules/meshcentral/sample-config-advanced.json for a more advanced example.",
  "settings": {
    "cert": "meshcentral.mydomain.tld",
    "tlsOffload": true,
    "trustedproxy": "CloudFlare",
    "_WANonly": true,
    "_LANonly": true,
    "_sessionKey": "MyReallySecretPassword1",
    "_port": 80,
    "_aliasPort": 443,
    "_redirPort": 800,
    "_redirAliasPort": 80
  },
  "domains": {
    "": {
      "certUrl": "http://meshcentral.mydomain.tld:443",
      "_title": "MyServer",
      "_title2": "Servername",
      "_minify": true,
      "_newAccounts": true,
      "_userNameIsEmail": true
    }
  },
  "_letsencrypt": {
    "__comment__": "Requires NodeJS 8.x or better, Go to https://letsdebug.net/ first before trying Let's Encrypt.",
    "email": "[email protected]",
    "names": "myserver.mydomain.com",
    "skipChallengeVerification": true,
    "production": false
  }
}

[Xiaoh-123]

Apparently I was to quick to rejoice, a day after implementing the above config, all the agents went offline again.
Putting the "ignoreAgentHashCheck": true into the config is the only way I found to make this work.
I am currently at loss after a full day of trying things out.

Any help is welcome.

[si458]

you have certUrl set incorrectly, it should be the url you use to access your web ui,
so it should be something like https://meshcentral.mydomain.tld
AND
the url SHOULD be accessable from within your meshcentral LXC,
one way to test is curl -v https://meshcentral.mydomain.tld and check it can connect and return the login page

[Xiaoh-123]

As promised in another post, here's what was wrong with my setup, which is probably a bit unique, but could maybe be useful to someone else.

My general backend is as follows:
Proxmox is hosting several LXCs to manage my network : Adguard Home, Nginx Proxy Manager, Cloudflared and of course Meshcentral.

OPNsense (bare metal) provides LAN on 10.0.0.1, with the DNS for this interface set to 10.0.0.100 (which is Adguard Home).

Adguard Home does DNS sinkholing and has its upstream set to 10.0.0.1, back to OPNsense which then uses Unbound magic to send outbound DNS requests to Quad9 with DNS over TLS.

Nginx Proxy Manager (10.0.0.105) is actually just there to handle local SSL to get rid of the self-signed alerts when accessing services from LAN. A DNS challenge certificate from Let's Encrypt was generated for *.mydomain.tld + mydomain.tld, then proxy hosts were created with the same addresses as I would use from WAN to access local services (i.e. meshcentral.mydomain.tld), and pointing to the local IPs (i.e. for Meshcentral http://10.0.0.104:443, with the LE certificate used and SSL forced).

I added a "catch all" DNS rewrite in Adguard as such: *.mydomain.tld => 10.0.0.105, which meant that any local query for an FQDN would not go on the WAN side and instantly go through Nginx Proxy Manager to be delivered with SSL thanks to the LE certificate.

My CloudFlare tunnel was setup as is: meshcentral.mydomain.tld to http://10.0.0.104:443 (with additional config, http host header = meshcentral.mydomain.tld)

And with that, Meshcentral config.json was setup with :
"cert": "meshcentral.mydomain.tld"
"tlsOffload": true
"certUrl": "https://meshcentral.mydomain.tld"

Meshcentral webGUI was accessible from LAN and WAN, but adding clients didn't work unless the "ignoreagenthashcheck" was added and set to true in the config.

At this point maybe some of you already know what was wrong all along but for the rest of you there it is.
The DNS rewrite was forwarding the "certUrl" request not to CloudFlare, which is supposed to hold the certificate that clients will see when trying to connect to the FQDN, but the Nginx Proxy Manager and it's LE certificate. The mismatch of these two certs was causing the inability to install/see the clients unless the hash check was ignored.

So after finally finding out, I removed my catch all rule in Adguard and switch to single rules, not creating one for Meshcentral, and voila, it now works perfectly.

Here's my config.json at the moment:

{
    "$schema": "https://raw.githubusercontent.com/Ylianst/MeshCentral/master/meshcentral-config-schema.json",
    "__comment1__": "This is a simple configuration file, all values and sections that start with underscore (_) are ignored. Edit a section and remove the _ in front of the name. Refer to the user's guide for details.",
    "__comment2__": "See node_modules/meshcentral/sample-config-advanced.json for a more advanced example.",
    "settings": {
      "cert": "meshcentral.mydomain.tld",
      "tlsOffload": true,
      "WANonly": true,
    },
    "domains": {
      "": {
        "certUrl": "https://meshcentral.mydomain.tld",
        "allowedOrigin": true,
      }
    }
  }

I'm not sure if WANonly and allowedOrigin are still required so you may try with and without.

Many thanks to @si458 who helped me by pointing me in the curl direction, it was by using curl -vI https://meshcentral.mydomain.tld (lower v, upper i) that I noticed that some requests were going to Cloudflare (when prompted from the WAN side) while others were going through 10.0.0.105 (from the LAN side). I then checked the certificates in my browser to see that one was from Google (CF) and the other from LE (Nginx Proxy Manager), and from that realized that there was an issue.

[alekssmacuks]

Hi @Xiaoh-123 and @si458

I hope you are doing well. I just wanted to tell that I have managed to set it up. I am using Cloudflare Proxy and Cloudflare Tunnel and followed this manual:
[(https://docs.tacticalrmm.com/unsupported_proxies/)].

I was very close following what @si458 had advised. However, I did not remove rmm.mydomain.tld from /etc/hosts. Also had not made http headers modifications as was advised in the manual. I hope this will help someone to set up and troubleshoot.