Code signing

[PathfinderNetworks]

Has anyone here purchased a code signing certificate and have signed your agents with it? I started the process of buying a certificate today and will be getting my agents signed with it once it is verified and active.
I'm REALLY hoping it will help to prevent antivirus vendors (namely Avast) from seeing it as malware and removing it from my devices.
Avast released an update today that contains a bug that I had alerted them about a month or so ago. That bug means the policies I have in place to prevent Avast from acting on meshagent.exe and the service no longer are honored. So now almost all of my devices are dropping out of MeshCentral (which is a HUGE issue).

Just wondering if anyone else has signed their agents with a verified cert and if helped, at all, with antivirus vendors flagging it?

[silversword411]

Avast...the spyware antivirus company

https://antivirus-review.com/blog/avast-programs-spy-on-their-users

Let us know if it helps once you get it implemented

[PathfinderNetworks]

You get what you pay for. I don't use the free version. I use Avast CloudCare- which utilizes their Business Premium endpoint. Very different product. Everything that is free is 'free' for a reason.

[si458]

We have just renewed our certificate from comodosslstore.com, £175 for 3 years,
But it would be better to get the ev one but its more expensive at £655 for 3 years

We havent had any problems with norton only their new stupid data protector tool which blocks meshcentral from accessing the registry (grrr) but thankfully doesnt stop its functioning as it still works 100%

I also don't codesign with meshcentral as Norton still detects it as a virus
So I codesign the exes manually with ezsignit, then put the new exes inside the agents folder

[PathfinderNetworks]

That's similar to what I paid for my Sectigo cert (which is also Comodo from what I understand). $180 US for 3 years for the OV cert.
One question, why would there be a difference if you use MeshCentral's built in authenticode-js module to sign it with the cert as opposed to signing it manually with another app, such as ezsignit? Shouldn't the end result be the same- that the application is signed with the code sign cert? It should look exactly the same no matter which process signs it- or so I'd think?

[si458]

I have absolutely no idea what the difference is!
I've been trying to work it out for months since meshcentral codesigning was a thing to no luck!
I throught it was the missing timestamp servers at first but when @Ylianst added that support,
Sign with meshcentral and Norton finds it as a virus, sign it with ezsignit no virus detected?

[PathfinderNetworks]

That is very, very odd. I wonder if Ylian can comment on why it may be doing that? I don't know a lot about code signing (actually I know very little as this is my first go at it). I am more familiar with SSL certificates in general though. I wonder if the difference is with the SHA level? Like maybe authenticode-js is using a lower version of SHA (like SHA256) whereas EZSignit is using SHA384? That's really the only thing I could think of which might be different?

[Ylianst]

I am really not sure why there would be a difference between Authenticode-JS and CodeSign. Authenticode-JS makes use of SHA2-386 by default. In general with MeshCentral I try to use one higher than the industry accepted cryptography. There are on discussion on that being a good way to go in case there is a quantum computer that can start eating at the low end crypto, the larger keys will buy some time to switch. Anyway, back to code signing...

What Bryan and I found when testing code signing of the agent with anti-virus is that it's super important to also change the metadata of the agent. This is the strings you see when you right-click, select "properties" on the agent in Windows and go to the detail tab.

If you sign and change the Metadata, you have a agent that is a lot more individualized and less likely to trigger an anti-virus. This is why it's been high priority to edit that in Authenticode-JS. Changing these strings are equal or more complicate that code signing, it's really a huge pain, but you can add the following in the "domains" section of the config.json:

      "agentFileInfo": {
        "filedescription": "sample_filedescription",
        "fileversion": "0.1.2.3",
        "legalcopyright": "Apache 2.1 License",
        "originalfilename": "sample_originalfilename.exe",
        "productname": "sample_productname",
        "productversion": "0.1.2.3"
      }

In some cases, only changing this may fix the anti-virus issue, no need for a trusted cert. This said, I always worry that changing these strings would generate an invalid executable, so I have not turned this on by default. It does seem to work, but I would recommend to check that the "meshcentral-data/signedagents" can be run before deploying the agent.

In any case, changing the metadata and signing with a trusted cert is certainly the best. If using a trusted code signing cert, you should also add this line in the "settings" section of the config.json:

    "AgentSignLock": true

(or watch this video if using your own signing tool). This will make it so your signed agent can't be used to connect to any other server and so, protects the reputation of your code signing cert.

Lastly, you can use a different signing tool and that is fine, you can still use Authenticode-JS to change the file metadata, test that the executable runs and then sign it using the other tool.

Sometime in the future, I will improve Authenticode-JS to also change the executable icon. That will make is more on brand and help individualize the agent some more.

[si458]

It might be a bug?
What does the productversion show on the original exe located on the server in the agents folder?
I'm AFK at moment sorry

[PathfinderNetworks]

The original agents are also showing 0.0.0.0 for the product version. So that makes sense that my signed agents would have that same product version when I don't have the productversion setting configured. But I would think the productversion setting should allow that metadata to be populated instead of making it blank. Again, probably not a big deal but might be something for Ylian to take a look at.
So far my newly signed agents aren't being flagged as malware by Avast. But we shall see how long that lasts.

[Ylianst]

Interesting, I will need to investigate this. Feel free to file it as a bug. Product version should be set and show correctly.

[Ylianst]

I am seeing the same problem for "ProductVersion". I will see what is going on.

[Ylianst]

I check in an improvement to the FileVersion and ProductVersion last night in GitHub, will be in MeshCentral v1.0.64. FileVersion will now work correctly, but for ProductVersion with trial and error I noticed that if I put a alphabetical character into the string it will show in the Windows properties box. So, "1.2.3.4" will show blank, but "v1.2.3.4" will show. It does not matter where the alphabetic character is, so "1.2.3a.4" will also show up.

In any case, I fixed up the config.json schema and code. You can try placing a alphabetic letter in productversion now, it should work.

[PathfinderNetworks]

So I wanted to post an update. For about 2 weeks this signed agent was working well in Avast. Avast was happy with it and not flagging it. Until today. Today I started getting alerts that meshagent.exe was being detected on various devices as a Win64:Malware-gen infection and it was being removed once again.
So signing and branding the agent was only a very temporary fix.
I guess I'm going to have to give up on MeshCentral as a remote management tool and move all devices to something that antivirus vendors 'like'.
Extremely frustrating and a MASSIVE amount of work.

[silversword411]

You should contact Avast, and see how you can whitelist a code signing cert. I don't understand why AV vendors don't provide this ability. If you want to have software, code sign it. Add to your agents personal whitelist, and they should keep their hands off it.

A code signing whitelist is much better than a file/folder based whitelist.

[PathfinderNetworks]

I agree with you. But it just doesn't seem to work that way. I've been battling this with Avast for years. I have techs that know about it but they can never tell me why they are listing is a virus or whatever infection of the day they choose. In the past they would white-list it. Now they seem to no longer want to do that. I do agree, however, they should allow me (as one of their resellers) to request our code signing cert be whitelisted and that any files signed with that should be off-limits. Thing is, even if they did that, I'm sure it wouldn't always work as they make changes that end up breaking exclusions and other policy settings that are already in place. They'd do something to break the code signing white-list as well. And it's not just Avast. Seems to be most antivirus vendors have the same issues. They use some common database for signatures. When a file gets on that database then all the vendors that use that database will flag it.

[silversword411]

Here's Microsoft's own documentation
https://docs.microsoft.com/en-us/microsoft-365/security/defender-endpoint/manage-indicators?view=o365-worldwide

...of course it only applies to their paid AV, and not "Windows Security" aka Windows Defender built into all windows since XP. Because why would they put that into their free product :(

[si458]

If I signed it with meshcentral our Norton would complain about it being a virus none stop
But
If I signed it with ezsignit then copied the exe to the other agents folder, it wouldn't complain one bit about being a virus
AV is weird at times

[silversword411]

@si458 can you sign with both, and compare everything and list the differences (other than the cert I'm sure)?

Is it the trust chain, what algorithm is used etc?

[renueman]

Hello all,
I just wanted to provide some feedback on the code signing problem. I have been lurking on this subject for a while. I have a lot of clients that I setup with 'Avira' antivirus and installed the MS Agent. This is what I observe - I install MS Agent by uploading the Agent file to the computer and turn off Avira while doing it. Then I run the install and turn Avira back on. It flaggs the Agent install file I uploaded but leaves everything else alone. Today I did another install but forget to turn off Avira and it never flagged the Agent install file until after I ran it. Then it removed it from the desktop I uploaded it too. However, It has left the installed Agent alone. I have several Agent installs on other clients and they are all still running fine.

Seems to me (IMHO) that concerning avast we should be reporting the Agent as a false positive - probably already done though. They should fix it themselves. Otherwise drop Avast and go Avira ? just MHO here.

I would also like to add that I am willing to be a tester for any changes if need be.

[silversword411]

Seems to me (IMHO) that concerning avast we should be reporting the Agent as a false positive

Well yes, that's the least we can do.

What I'd like to do however is get better tooling, and information on the AV and make them implement proper whitelisting methods...rather than playing Whack-a-Mole on the black box that is AV false positive reporting.

All AV's should allow you to manage something like this: https://docs.microsoft.com/en-us/microsoft-365/security/defender-endpoint/manage-indicators?view=o365-worldwide

Microsoft calls it IoCs. All AV should be able to whitelist by:

1. certificate fingerprints
2. URLs
3. IPs in CIDR format
4. Folders
5. Files
6. process names

[PathfinderNetworks]

Shockingly, I was able to get Avast to remove my signed agent from the definition database. So now it's not being seen as a virus. But I'm sure it won't last as, every time I've got them to do this in the past, it ends up showing back up as an infection a few months or so later. Hopefully, now that it's signed and branded, maybe that won't happen.

[renueman]

I might have spoken too soon. I just installed on a win10 with Avira and now it shows offline but teamviewer shows its up. Ill login later tonite and check it. Im no fan of Avast because of all the popups. Hopefully your fix lasts this time.

[PathfinderNetworks]

Thankfully, with Avast CloudCare (which uses the Avast Business Pro client) I have full control over things like those popups. I have all of my endpoints set to 'silent'. Which means the end user sees ZERO notifications from Avast. Not even if they download a virus- it only alerts me. It makes it all pretty much invisible to them. I can also password protect the client and its settings so they can't modify them in any way. It's really a great program- except for when this sort of thing happens. This bug they have with the client ignoring the exclusions is pretty major though.