Tuesday, June 19, 2007

Was this for our own good?

Those of you already using Flight Simulator X will have noticed by now that there's a new concept of authorizing third party DLLs and GAUges before they are loaded into FSX.

The first time a user installs a third party addon, depending on how many of those DLLs the addon uses, a security warning dialog box pops up asking the user if they "want to run this software" with a "Run - Don't Run" button choice. If the software (read: DLL or GAU) is code-signed, there will be information on the software publisher and a "More Options" tab which leads to radio boxes allowing the user to "Always run software from 'Publisher'", "Never run..." or "Ask me every time". This is default behavior by the new Windows Cryptograpy API.

FSX adds another layer after the user has (presumably) selected "Run" which allows them to "designate this module as 'Trusted' software", thus making FSX automatically load this software any time it runs again in the future, without having to go through the security dialog pop up again.

All this is well and good - for normal users. Of course, if the software publisher produces an update, the entire process has to happen once more, since the new DLL versions will be detected as different and the user will need to allow them again.

For us, developers, this is a pain in the butt. We produce new builds of our DLLs and gauges many times daily, so we need to keep "allowing" them each time, because if we "always trust" our own code, we are doing our users a disservice in case something goes wrong when FSX first loads the gauge (and I've already been bitten by that happening - and it wasn't even my fault! - more on that perhaps in another blog entry).

If this pain in the butt wasn't enough, I spent the past two days trying to figure out why FSX suddenly stopped asking about verification and designating modules as 'trusted'. Instead, it refused to load any 3rd party addon.

Two entire days. Wasted. I even tried removing FSUIPC and reinstalling it, thinking that I might have done something to my certification process to cause this, so another addon should surely be OK. Nope. FSX never asked me for that either - it simply failed to load FSUIPC too! OK - go ahead and remove the [Trusted] sector from fsx.cfg? Nope. Still couldn't care less - FSX will not load anything. WHY?

In the meantime, I had noticed that, when running the Certification Manager, it would sometimes give me an error when viewing my code signing certificates about "Cryptographic Operation Failed due to a local security option setting". Hmm... Whah? Huh? I know they're bought and paid for - and checked by a valid Root Authority! What gives? This was surely a mystery... but it led me to goog- I mean search MSDN for any related issues...

I can now thank Steve Patrick (at Spat's Weblog) for providing what turned out to be a very simple solution to this issue. While apparently unrelated, it turns out that something had switched my

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\WinTrust\

Trust Providers\Software Publishing


registry key to 0x63c00. This value is Microsoft's WinTrust policy selection flags, as described here. The 0x40000 flag means Allow only items in personal trust database and apparently, this clashes with FSX's ability to allow manual authorization of unrecognized DLLs, telling it instead to never trust anything outside the user's personal trust.

Unsetting this flag (putting it back to its default 0x23c00 value) returned all operations to normal and Flight Simulator was once again able to ask me to designate my modules as 'Trusted' software.

I never found who/what changed that registry key... but at least now I can return to my normal schedule of love with Microsoft's code signing necessities!

This is what I do for a living... who said programmers don't have "the life"! ?

6 comments:

Anonymous said...

OK Dude,
now I see why it is taking so long. Sounds like installing the FSX bird is going to be fun too. Guess there is now way around that. The love-hate relationship we have with Microsoft. Still a fanboy, but sometimes you have to say, WTF?
Thanks for the peek (and insight)into the way-cool life of a programmer.

François said...

Dontchalovum !!??? I can think of a few annoyances more.... but we still need 'em. ;-)

F.

Moby Dick said...

Anyone spending so much time on flight simulators might as well join the Air Force and do it for real! Good luck with your hobby, it sounds complicated, hopefully it is a lot of fun too.

Anonymous said...
This comment has been removed by a blog administrator.
Anonymous said...
This comment has been removed by a blog administrator.
markerdave said...

do you use window 7 ?

if i edit the regfile (state REG_DWORD) in the wintrust folder i get a whole different value.