Getting AntiVirus software to trust a Windows Application

From vecteezy.com (Glass Shield Anti-Virus)

One of our customers tried installing our Windows Application on one of their computers and their antivirus software complained. They in turn complained to us, so we had to find a solution. I am documenting the solution here.

)Useful pre-reading: About these papers)

Possible Causes

Antivirus engines may find a certain file suspicious for many different reasons. The most obvious reason is the file containing a byte pattern which has been encountered in some known malware.  Unfortunately, this does not necessarily mean that the file is infected; it may simply be a coincidence. Furthermore, many other checks performed by antivirus engines tend to fall well within the spectrum of paranoia.  It appears that some particularly paranoid antivirus engines will flag an executable as malware simply because it has not been code-signed or strong-name-signed, or because it has undergone obfuscation. Our application checked both of those boxes.  Your mileage may vary.

How to Troubleshoot

The first problem that you are likely to run into when troubleshooting issues of this kind is how to tell whether your application is passing antivirus checks or not. Obviously, the antivirus suite at the company that I work for was not finding anything suspicious with our application, or else we would have certainly noticed; however, a couple of antivirus suites that our client was running were taking issue with it. I did not have access to the antivirus suites that our client had, and even of I could somehow get my hands on them, I would not be able to install them on my machine, because the antivirus suite which had already been installed on my machine by our IT department played god and did not allow me to touch it.

Luckily, there is a website that provides a solution to the problem. It is called virustotal.com, it aggregates a large number of antivirus engines, and it allows you to upload an executable and see what all these engines think of it. It is quite famous in the security community.

Unfortunately, checking an application with virustotal.com is not an exact science, for many reasons. 

For one thing, virustotal.com said that out of more than 60 antivirus engines that they are aggregating, only four of them took issue with our application. As far as I am concerned, this is a pretty good indication that our application is fine, and these four engines are just being paranoid, but try telling that to the customer.

Complicating matters even worse was the fact that the same antivirus engine on virustotal.com that was being used by the antivirus suite that we were using at the company I work for was in fact flagging our application as suspicious, whereas the antivirus engines on virustotal.com that belong to the corresponding suites that our customer was using did not take issue with it, so go figure.

Nonetheless, it stands to reason that if you manage to produce your application in such a way that none of the antivirus engines on virustotal.com take issue with it, then chances are that the antivirus suites of customers will also not take issue with it. Even if this turns out to not be true, you will at the very least be able to tell your customer to go to virustotal.com and see for themselves and then just trust you.

Another interesting thing about virustotal.com is that it does not seem to care very much about DLLs used by an application. There is of course a chance that I have not quite understood how the whole thing works yet, but for now it seems like I could pretty much run any malicious code I want out there, as long as I package that code in a DLL which is loaded by an otherwise clean and innocent executable which passes the antivirus checks. More about this as I understand more about how things work.

Strong-name Signing

The first approach that I tried in the direction of solving the problem involved what is known as "strong-name signing". The problem with this approach is that it requires that every single DLL that is loaded by the application must also be strong-name signed. From a security stand-point this is of course absolutely correct, but here is the problem: even though we can see to it that all DLLs produced by us are also strong-name signed, we are additionally using many DLLs produced by third parties, over which we have no control, and which are not strong-name signed. 

There appears to be some solution out there that will automatically strong-name sign DLLs used by an application, but it works like magic, and of course, as the case usually is with magic, it miserably fails with certain DLLs that use some loading strategy that is off the beaten path, for example with the MathNet Numerics library, and possibly with others.

I tried hard to get strong-name signing to work, but eventually I gave up.

Code Signing

The next approach that I tried was just plain "signing", or "code signing", as soon as I realized that it is not the same thing as "strong-name signing". 

Code signing did not completely solve the problem, but at least it took us halfway there. With code signing we were able to reduce the number of antivirus engines that take issue with our application from four to two. As it turns out, the remaining two would only be appeased if we disabled the code obfuscation step in our build. I would like to take the opportunity to point out that code obfuscation is a perfectly legitimate technique, and that antivirus engines that take issue with it are junk.

A secure and proper application of code-signing involves obtaining a signing certificate from a certification authority, at a cost of a few hundred Euros per year.  Of course, when you are researching or troubleshooting the matter you are likely to skip that expense, and self-sign your files, until you are sure that this is the way you want to go. This is absolutely not secure, because there is no chain of authority to guarantee the authenticity of your certificate. It is worth pointing out that the two antivirus engines on virustotal.com that were appeased when they saw a code-signed version of our application did not bother checking the authenticity of our certificate!  The self-signed certificate was enough.  So, these two antivirus engines are junk, too.

What follows is all the voodoo magic that is necessary to get signing, and specifically self-signing, to work:

The original source of information is this stackoverflow question and accepted answer:
https://stackoverflow.com/questions/84847/how-do-i-create-a-self-signed-certificate-for-code-signing-on-windows
(Ignore the claim that makecert is deprecated, and the suggestion to use some Powershell command instead. Makecert works just fine, and if you have VisualStudio then you already have it, which is not the case with the recommended powershell command.)

First, we need to create what is known as a "self-signed certificate authority (CA)", using the following command: 

makecert -r -pe -n "CN=My CA" -ss CA -sr CurrentUser -a sha512 -cy authority -sky signature -sv MyCA.pvk MyCA.cer

This will create MyCA.pvk and MyCA.cer in the current directory.

Although makecert is a command-line utility, it will pop up a graphical dialog asking us to create a password.

You can just hit [Enter] to use no password, or you can create a password of your choice, but be sure to copy it to the clipboard, because you are going to need it a lot in the following steps, when you are going to be seeing dialogs like this:

Then, we need to do what is known as "Importing the CA certificate", using the following command:

certutil -user -addstore Root MyCA.cer

This will pop up a big dialog saying that it does not know where this certificate came from, and it cannot verify its authenticity, (of course, since we just created the Certification Authority out of thin air,) and asking us whether we are sure that we want to proceed installing it.

Just answer "yes".

Then, we need to create what is known as a "Code-signing Certificate" and is for some reason referred to by the acronym "SPC", using the following command:

makecert -pe -n "CN=My SPC" -a sha512 -cy end -sky signature -ic MyCA.cer -iv MyCA.pvk -sv MySPC.pvk MySPC.cer

Then, we need to convert the generated certificate and key into a PFX file, using the following command:

pvk2pfx -pvk MySPC.pvk -spc MySPC.cer -pfx MySPC.pfx

The above steps only have to be carried out once. From the moment that we have that MySPC.pfx file, we can keep using it to sign any number of files, as often as we want, using the following command:

signtool sign /v /t http://timestamp.comodoca.com/authenticode /f MySPC.pfx myexecutable.exe

The /t parameter followed by a URL allows us to timestamp the signature. A timestamped signature is required for performing a full authenticity check, but as I have already explained, the antivirus engines at virustotal.com do not seem to perform even a rudimentary authenticity check, so the timestamp is probably unnecessary in our case. Nonetheless, I have included it for the sake of completeness. There are various timestamp services out there that you can use, and you can find a list of them in the original stackoverflow article, however the available services inevitably change over time. I tried the globalsign.com URL but it appeared dead, so then I tried the comodoca.com URL and it worked, so comodoca.com it shall be, at least for now.

Verifying the results

Using the Windows File Explorer to look at the properties of the signed executable, we see that there is now a new property tab called "Digital Signatures":

Viewing "Details" on the signature yields this property page:

And clicking "View Certificate" yields this dialog:

Of course, Windows is finding the digital signature to be "OK" only because I have installed this self-created and fictitious "My CA" Certification Authority.  On a customer's computer it will most probably not show "OK" unless you purchase a certificate from a valid Certification Authority, but for the purpose of passing the silly virus checks on virustotal.com, it is okay.

Conclusion

After self-signing my executable, (and disabling obfuscation,) I uploaded it to virustotal.com, and it passed the checks of all antivirus engines. 

There is a chance that the antivirus solution on our client will perform a complete authenticity check, which will fail due to self-signing, in which case we will have to purchase a real certificate.

There is also a chance that individual DLLs used by our application will not pass this test, which will necessitate signing each one of them, using the same command.