My 3 favorites:

- Virtual Windows XP AKA 'XP Mode' (not all SKU's)
This is basically a small Virtual PC within the Windows 7 OS that allows you to run the application in it's own contained environment, fully mimicking Windows XP while seemingly sitting on the desktop.
Should make compatability issues of legacy applications during migration a simple thing to resolve.  Think of it as RDP'ing to a second XP box under your control.

- In-box network tracing
Just run netsh trace start capture=yes to start it and netsh trace stop to stop it, allows you to even make it persistent so you can monitor network traffic during the initial startup phase.  No more fiddling around with hubs and promoscuous mode sniffers or port replication.

- Plug & Play smartcards and smartcard readers
Just plug your reader in and insert your smartcard, the P&P architecture should autodetect it and attempt to download drivers from Windows Update.  No need to pre-install the card software or reader drivers (well, as long as the drivers are published).

The ScPolicySvc service works by monitoring a specific registry key (See Deconstructing the Smartcard Removal Policy Service).

The VPN client (Connection Manager aka CM) on the other hand doesn’t use the Credential Provider architecture, it uses its own code for picking which certificate from the smartcard will be used for logon.

The VPN component not using CredUI or LogonUI has two side-effects:

• The Smartcard Removal Policy Service doesn’t monitor logons made with the VPN client as the registry key isn’t touched when the VPN logon occurs
  
• The user logging on doesn’t get to pick which smartcard certificate will be used for the VPN connection – the VPN components does a simple certificate selection and picks the smartcard logon certificate in the default container (usually the last certificate enrolled for).

How to Support Smart Card Logon for Remote Access VPN Connections
http://technet.microsoft.com/en-us/library/cc875840.aspx

Deconstructing the Smartcard Removal Policy Service:
http://blogs.technet.com/instan/archive/2010/03/08/deconstructing-the-smartcard-removal-service.aspx

Where Is “Logon Using Dial-Up Connections” in Windows Vista?
http://blogs.technet.com/grouppolicy/archive/2007/07/30/where-is-logon-using-dial-up-connections-in-windows-vista.aspx

Customers may get this issue from time to time on every Sharepoint WFE server except one whenever the antivirus applications on the servers successfully update their antivirus definitions.

This only happens when more than one load-balanced Sharepoint WFE is involved and configured to update at exactly the same time and the antivirus application is configured to update at the same time on all servers.

EventID:

Event ID 6398 from Source Windows SharePoint Services 3 on agent computer SharepointN2.contoso.com has triggered this Alert Description : The Execute method of job definition Microsoft.SharePoint.Administration.SPAntivirusJobDefinition (ID 1ce16aea-f6b5-43e6-9d6c-015a44627fce) threw an exception.

More information is included below.

An update conflict has occurred, and you must re-try this action. The object SPWebService Parent=SPFarm Name=SHAREPOINT_PRODUCTION_ConfigDB is being updated by CONTOSO\SQLAdmin, in the OWSTIMER process, on machine SharepointN1.  View the tracing log for more information about the conflict. Hostname=SharepointN2.contoso.com

Time Received: Tue Mar 23 17:09:21 GMT+0100 (CET) 2010

Microsoft.SharePoint.Administration.SPAntivirusJobDefinition

In the case where the WFE’s have 3rd party antivirus applications this is effectively a cosmetic issue, the antivirus updates by the antivirus application on the servers are not affected but the reporting of the updated status to the main Sharepoint config database is delayed by 5 minutes per server.

I.e. if 3 servers are involved and they all try to report the updated status of the antivirus definitions at the same time, one will succeed and two will fail and report the 6398 event. 
5 minutes later those two will retry and one will succeed and the other will log 6398. 
The last server will then retry 5 minutes later and succeed.

See also http://social.technet.microsoft.com/Forums/en-US/forefrontSharePoint/thread/7f83dcbb-a7aa-4946-92f9-d0cbafa8f9a0

Configuring Forefront Server Security polling interval:
http://technet.microsoft.com/en-us/exchange/dd385798.aspx

On Windows 7 clients, if a smartcard certificate contains a Subject Alternate Name (SAN) it will by default be used for implicit mapping against a user in AD and whatever has been imported to the AltSecurityIdentities attribute in X509 format (the UPN SAN is special as that looks directly at the UserPrincipalname attribute).

If you want to override the contents of the SAN and use explicit mapping based on things outside of the SAN (like Subject Name and Issuer), you need to disable it on both client and KDC using the UseSubjectAltName registry value.

This also requires Windows 7 on the client side and Windows Server 2008 R2 on the server side (domain controller side).

See http://technet.microsoft.com/en-us/library/ff520074(WS.10).aspx for details.

(see also Spat's entry on http://blogs.msdn.com/b/spatdsg/archive/2010/06/14/howto_3a00_-disable-upn-mapping-for-smartcard-logon.aspx which covers this in more details.)

Well, maybe not quite... but hopefully it helps explain the concept better.

SSL is not the trusted stamp of approval that it was maybe 10-15 years ago, business requirements and competition between CA vendors has moved it away from being a cumbersome, manual and lengthy process to the point where you can point and click your way online to an SSL web server certificate within a couple of minutes (as long as you have a valid credit card).

The downside of this is that there is only minimal verification about the entity or persons behind most SSL certificates - so just browsing an SSL-enabled site doesn't provide much security beyond the obvious encryption of data between the client and server.

To address this, the CA vendors agreed on a more restricted version of SSL certificates that require a more extensive verification of the entity before they are issued.  Browser or application vendors have in turn agreed to support this - this is where the 'Green is good' part of the browser bar kicks in when accessing a web site protected by an Extended Validation (EV) SSL certificate. 

But what if you have a public Root CA and you want to be able to issue EV SSL certificates from it rather than go through one of the CA providers?
In that case you need to get your CA's on the CTL for each browser that you want to 'Go Green' on - the following shows how IE 7+ does this.

Note that the issuing CA must also be a minimum of W2k8 R2 to be able to issue EV certificates - internally or externally.
(See http://blogs.technet.com/b/askds/archive/2009/08/14/extended-validation-support-for-websites-using-internal-certificates.aspx).

Further details:

Internet Explorer and Business Value of Extended Validation SSL certificates
http://www.microsoft.com/windows/products/winfamily/ie/ev/default.mspx

Extended Validation SSL Update
http://blogs.msdn.com/b/windowsvistasecurity/archive/2007/12/18/extended-validation-ssl-update.aspx

Internet Explorer 8 Security - Extended Validation Certificates | TechNet
http://technet.microsoft.com/en-us/ie/dd878127.aspx

PKI Enhancements in Windows 7 and Windows Server 2008 R2
http://technet.microsoft.com/en-us/magazine/2009.05.pki.aspx

Extended Validation support for websites using internal certificates
http://blogs.technet.com/b/askds/archive/2009/08/14/extended-validation-support-for-websites-using-internal-certificates.aspx

Extended Validation Certificate
http://en.wikipedia.org/wiki/Extended_Validation_Certificate

One of our customers reported the following:

We have been evaluating EFS on Windows 7 as part of our upgrade from Windows XP project and have discovered that if you share a folder and encrypt a file within it locally, the same user is able to decrypt it remotely without the workstation being trusted for delegation.

If the user logs off then the remote EFS decryption operation fails.  If the user logs on again and opens any file encrypted with the same EFS certficate the remote user can decrypt it again.

We're concerned that this is a security risk in Windows 7 and from our tests with Notepad and encrypted .txt files we don't see the same behaviour when the target machine doing the sharing and encrypting is Windows XP.

After researching this we determined the following:

• EFS loads a mini-profile on the target server and uses it for the decryption - if a user is already logged on to the target no profile loading is required.
• As long as the same user has a handle open to an encrypted file, remote EFS decryption is possible even if the "server" isn't trusted for delegation.This is because the handle caches the private key used for the decryption operation and using it from the cache doesn't require delegation while loading it from disk does.
• Additional caching options for private keys were introduced in Vista SP2 (per CSP instance caching) - they were however not turned on by default.
• The same caching options are turned on by default in Windows 7.
• The per-CSP instance caching behaviour can be turned on or off on Vista SP2 and Windows 7 by setting the DWORD values CachePrivateKeys and PrivateKeyLifetimeSeconds under HKLM\Software\Policies\Microsoft\Cryptography.

I.e. remote EFS decryption without the target machine being trusted for delegation is also possible in Windows XP - as long as an open handle to the EFS-encrypted file exists locally on the remote server and the same user is logged onto it.
Notepad however closes the handle to the EFS-encrypted file as soon as it has finished decrypting it - using Notepad to test this on an XP machine is therefore not likely to give reliable results.

If you're quick enough to open it on the other machine however you can still manage to hit the window where the key is still loaded by the handle that Notepad has to the file and in that case decryption succeeds.
For testing this on XP, something that keeps a file handle open is needed (a ZIP file and 7-Zip or WinZip for example).

In short; private key caching is present on XP, W2k3 and Vista/W2k8/W2k8R2/Win7 - it's the caching that enables the remote EFS-decryption without needing to be delegated *if* the same user is already logged on and has already decrypted the file. 
On Win7 the additional per-CSP instance caching makes this more visible because the improvements introduced in Vista have been turned on by default and this allows the CSP to keep the key cached even if the handle to the encrypted file has been closed.

Further details:

Private Key Caching Constants
http://msdn.microsoft.com/en-us/library/aa387403(VS.85).aspx

You are no longer prompted to enter your private key password every time that the private key is accessed after you upgrade your computer to Windows XP Service Pack 2
http://support.microsoft.com/kb/890062

Windows Prompts You for Your Password Multiple Times When You Use Outlook If Strong Private Key Protection Is Set to High
http://support.microsoft.com/kb/821574/

The "Request For Permission to Use a Key" dialog box appears whenever you try to send an e-mail message in Outlook 2007 after you configure Outlook 2007 to use a digital signature in Windows Vista
http://support.microsoft.com/kb/936029

Using Encrypting File System
http://technet.microsoft.com/en-us/library/bb457116.aspx

Last month we finally closed two bugs that I've been engaged in on and off for well over a year and released two related hotfixes in the February hotfix release batch.

In late 2009, our Professional Support team got the following case from one of our ISV Partners (an established provider of security products, among them their own CSP):

" We have discovered multiple issues with Windows Server 2008 R2 Remote Desktop Services where the symptoms are as follow:

• If you connect with two users at *exactly* the same time and have the Smartcard Removal Policy set to Disconnect, you can get into a scenario where if one user pulls their smartcard the other user also gets disconnected.
  
• If pre-authentication is disabled in an RDP file that is used to connect and you connect with two users at *exactly* the same time, you can get into a scenario where one user sees his own smartcard credential tile on the login screen as well as the smartcard credential tile of the other user that connected at the same time.
  
• Occasionally, if you connect with two users at *exactly* the same time, you can get into the scenario where clicking on your tile and logging on with your PIN reconnects you to the session of another user (AKA "Session Mixup").
  
• If you disconnect a user and reconnect twice, that user can get into an infinite disconnect loop where she is immediately disconnected the next time she attempts to reconnect. "

When I started working on it the first hurdle was to get a reliable repro - if we can't repro the issue we simply can't file a bug report as our developers don't have anything to work with in that case.

However, we did finally get to the point where we could sit down and start talking to the development team about a possible fix, among the things we determined was that:

• The two "unexpected disconnect" issues only occurred in relation to Reconnections - the initial connections weren't affected.
• The Base CSP wasn't affected by the "foreign credential tile" problem while the 3rd party CSP was.
• The "Session Mixup" issue was never seen during our tests on RDS servers with RDP clients
  (Note that the phrase "Session Mixup" is verbatim from the customer but is a very broad definition and not really accurate in technical terms in this case - "Incorrect Session Arbitration" would be a more accurate term for the reported symptoms).

So, to summarize:

• Foreign smartcard logon tile appears on logon screen
  • Resolved in the Base CSP by creating and setting FilterCSPCardCacheByTSSessionConnectTime to 1 – it is by default 0 or not present.
  • 3^rd party CSP’s may need to do similar filtering of card entries in their cache based on session connect time or they will most likely hit the same issue.
  • Using a 3^rd party mini-driver that leverages the filtering functionality of the Base CSP is an alternative solution.

• Disconnect other user issue
  • Resolved in W2k8 R2 hotfix http://support.microsoft.com/kb/2301288 (updates Winscard.dll to 6.1.7601.21624)

• Disconnect self loop upon 2^nd reconnect issue
  • Resolved in W2k8 R2 hotfix http://support.microsoft.com/kb/2424375 (updates Winscard.dll to 6.1.7601.21642)

• Session arbitration connecting users to wrong sessions when under stress conditions (AKA the “Incorrect Session Arbitration” issue)
• We never saw this in our repro attempts on W2k8 R2 RDP servers

Note that both 2301288 and 2424375 are updates to the same binary (Winscard.dll) so you need only install the more recent one (2424375).

 Footnote: The primary purpose of this blog entry is to pull together disparate information about what is essentially a very generic end-user symptom (users with unexpected problems during reconnections to terminal servers) and list past fixes from the Microsoft side that have been known to address specific customer-reported issues that have exhibited those symptoms.  As the symptoms are very general and there may be other components (Microsoft or third party) that exhibit similar symptoms, it should be noted that this does not mean that these particular fixes will be addressing any and all such general symptoms in either a Microsoft or a third party product, in either past, present or future OS releases.

The secondary purpose of this entry is to reinforce the awareness that in a complex system with multiple components involved such as terminal servers with smartcard redirection, you need to be aware that an issue can have multiple causes with similar symptoms being reported by the end-user and could need multiple fixes in different components.

When a generic symptom is encountered and you have two different components involved it is very easy for either party to point a finger at the other as being the cause for the issue but this ultimately doesn't help identify the issue nor does it help the end-user.
I.e. you must look at the whole picture and not just focus on one piece of it - the correct way to troubleshoot this is to analyze and debug the specific component or components from both sides.

Further details:

Available Updates for Remote Desktop Services (Terminal Services) in Windows Server 2008 R2
http://support.microsoft.com/kb/2601888

A smart card logon to a terminal session stops responding server that is running Windows Server 2008 and Windows Server 2008 R2
http://support.microsoft.com/kb/949538

A remote desktop session may be incorrectly disconnected when a smart card is removed in another remote desktop session in Windows Server 2008 R2
http://support.microsoft.com/kb/2424375

A Remote Desktop Services session is disconnected automatically if you apply the "Interactive logon: smart card removal behavior" Group Policy setting in Windows Server 2008 R2 or in Windows 7
http://support.microsoft.com/kb/2301288

My colleague Jan had the following case recently:

Customer verbatim:

We've created a custom web server certificate template that we want to use to enroll certificates from for our web servers.  We've also removed the original Web Server template from the list of templates our CA is allowed to issue.

However, when we now go to the IIS 7/7.5 MMC root, click Server Certificates and choose Create Domain Certificate and go through the Certificate Wizard we are not seeing any CA and the Select button is greyed out.

After investigating this closer we determined the following:

The Certificate Helper code in the IIS 7.x MMC is hard-coded to look for a certificate template with the name WebServer.  This means that if no CA in your environment is allowed to issue a certificate template with that name it will not find any CA to enroll for.
Running certutil –templateCAs WebServer should list out which CA’s are allowed to issue the WebServer template.

The workaround is to enroll for a server certificate using the Certificates MMC snap-in, this is also preferable from a security perspective as the Create Domain Certificate request option doesn't allow you to choose the key size of the certificate (it's hardcoded to 512 or 1024) while you can specify the key size within the custom request in the Certificates MMC.

We resolved the following case recently:

On our W2k8 R2 Domain Controllers, autoenrollment is not working even if all the permissions are correct and the CA’s are allowed to issue the correct templates.  The funny thing is that if we open the Certificates MMC snap-in, right-click the Certificates node, choose All Tasks/Automatically Enroll and Retrieve Certificates we are able to enroll for the certificates.
Our W2k8 R2 member servers have no problems autoenrolling.

My colleague Jan (yes, the same Jan as before – busy as always) determined the following:

On Vista+ (Windows 7 and Windows Server 2008 R2) the autoenrollment functionality has been moved into Scheduled Tasks.

If you open up the Task Scheduler and expand down as follows:

+Task Scheduler Library
+++Microsoft
++++Windows
+++++CertificateServicesClient

You’ll see 3 separate tasks related to autoenrollment:

If the task scheduler service is stopped or disabled it means that the autoenrollment triggers will never be hit.  This also means that the MMC-initiated autoenrollment is in fact kicking off one of these tasks which bypasses the task scheduler.

In our case, the customer was applying a Group Policy to the DC’s which among other things Disabled the Task Scheduler service.  Once that was removed, Autoenrollment started working for the DC’s again.

In short; parts of the OS rely on the Task Scheduler service, it is not advisable to disable it.

Description of the scheduled tasks in Windows Vista
http://support.microsoft.com/kb/939039

A colleague on our PKI Server alias got the following question from a partner:

Our newly installed Windows Server 2008 R2 CA server got the time settings on it accidentally reset back to the BIOS defaults (1/1/2011) when the batteries on the motherboard were temporarily removed.

When the CA server was restarted afterwards we noticed that the CA server certificate was no longer present in the User store of the computer account and the ADCS service was unable to start afterwards.

It turns out the default setting for the ADCS service in Windows Server 2008 R2 is to remove the public keys of any CA server certificate that has expired or is not yet valid.
Looking at the CA certificate of the affected server it was installed in late March which made the CA certificate fall under the 'not yet valid' category after the backwards time jump which consequently caused the ADCS service to remove it from the store when it started up.

Another colleague (with me being the fly sitting on a CA-related distribution list) located registry settings which can be used to reverse this behaviour.

I've put together a Word document with the details on how to set up a federation trust between Visma Proceedo acting as a Relying Partner (RP) and ADFS 2.0 acting as the Identity Provider (IDP).

The document can be downloaded on http://blogs.technet.com/cfs-file.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-00-63-78/3021.Federation-trust-with-Visma-Proceedo.docx.

Further details:
http://www.visma.se/

I was looking at the Windows Server 2008 R2 KDC architecture with my colleague Jan earlier today concerning an issue when using smart cards with 3rd party domain controller certificates.

Our customer that Jan was working with had requested and received a certificate for their DC from Verisign but the W2k8 R2 DC just plainly refused to consider it as a valid KDC certificate and as a result smartcard logons were not possible.

The first knee-jerk reaction was to recite the requirements from the venerable KB´s “Requirements for Domain Controller certificates from a Third-party CA” (http://support.microsoft.com/kb/291010) and “Guidelines for enabling smart card logon with third-party certification authorities” (http://support.microsoft.com/kb/281245).

Both are however quite old and neither has really been updated since Windows 2000. 

For example; the GUID part is really only needed if you want to also use the certificate for SMTP replication (See http://support.microsoft.com/kb/947057).

So, barring an updated KB here are some notes on what we found out:

In short; the KDC on Windows Server 2008 R2 will look for one of three conditions when parsing its certificate store for KDC certificates to use:

• the OID for KDC Authentication (1.3.6.1.5.2.3.5)
• the presence of the Template Name DomainController in the certificate (all flavors of MS CA’s stamp this on certificates if it is a part of the request file received)
• the OID for SmartcardLogon (1.3.6.1.4.1.311.20.2.2)

If one of these is present in the certificate, the KDC will consider it potentially usable as a DC certificate capable of servicing smartcard logons.  The last two are present in the W2k3 code while the first one was introduced with Windows Server 2008.

The following two OID's are typically also used for the EKU´s:

• The OID for Server Authentication(1.3.6.1.5.5.7.3.1)
• The OID for Client Authentication (1.3.6.1.5.5.7.3.2)

In addition to this, the direct issuer of the potential KDC certificate needs to be in the NTAuth store of the DC and all certificates in the chain except the Root CA need to pass revocation checking as well.

Finally, as per RFC5280, the CRL of the SubCA that signs the CRL of the Domain Controller certificate must also include the CERT_CRL_SIGN_KEY_USAGE flagin the Key Usage extension of the SubCA certificate - otherwise revocation checking for the DC cert will fail on a Windows Server 2008 R2 or Windows 7 client.

        Note: This is a change from the Windows Server 2003 an Windows XP behaviour which did not enforce the RFC5280 requirements concerning the key usage flag of the certificate used to sign the CRL.

        Without these you'll most likely see KDC Event ID 19 or KDC Event ID 29 being logged on the DC whenever the KDC service starts and every 10 hours afterwards when it tries to locate a valid domain controller certificate.

Note that the easiest and most manageable solution (i.e. autoenrollment and renewal) for an internal DC certificate would probably be to just use a Windows Enterprise Issuing CA and the Kerberos Authentication certificate template for the DC’s, but it should also be possible to add the required OID’s to the .req file if you’re sending it to a 3rd party CA.

Other events on the DC indicating you're hitting this condition are: KDC_ERR_PADATA_TYPE_NOSUPP and 0xc00000BB being logged on the DC during a smartcard logon from a client.

Further details:

Requirements for Domain Controller certificates from a Third-party CA
http://support.microsoft.com/kb/291010

Guidelines for enabling smart card logon with third-party certification authorities
http://support.microsoft.com/kb/281245

Kerberos Authentication Template
http://technet.microsoft.com/sv-se/library/cc730826(WS.10).aspx

Key Usage requirements for SubCA's
http://blogs.technet.com/b/instan/archive/2011/09/12/event-id-29-when-starting-kdc-service-on-windows-server-2008-r2-dc-s.aspx

Updated requirements for a Windows Server 2008 R2 domain controller certificate from a 3rd party CA
http://social.technet.microsoft.com/wiki/contents/articles/updated-requirements-for-a-windows-server-2008-r2-domain-controller-certificate-from-a-3rd-party-ca.aspx

Following up on a previous post about Credential Roaming (aka DIMS): http://blogs.technet.com/b/instan/archive/2009/05/26/considerations-for-implementing-credential-roaming.aspx

With a recent DCR to Windows 7 & W2k8 R2 (http://support.microsoft.com/kb/2520487) it is now possible to filter out specific types of credentials from the credentials that will roam to your AD database.

Post-hotfix default behaviour is to not roam unaffiliated keys, unused keys and smartcard certificates.

The caveats here are:

• This is a client-side fix for Windows 7 / Windows 2008 R2* - legacy clients will still roam the above keys.
• Implementing this fix on the client-side after the above keys have already been roamed to the AD Database will not remove them from the NTDS.dit file on the DC's (i.e. its a preventitive fix - not a reactive fix).

*this fix has been ported to W2k8/Vista since this post was written.

Further details:

AD DS database size increases significantly when the Credential Roaming feature is enabled in Windows 7 or in Windows Server 2008 R2
http://support.microsoft.com/kb/2520487

A customer put the following questions to one of my colleagues:

On a lot of our Windows 7 clients we've noticed they periodically try to download a CAB file from Windows Update, but as our workstations are required to access the Internet via Proxy and they aren't able to authenticate against it the download fails and we see Event ID 11 being logged on the clients.

We've downloaded the CAB file and extracted it and we see an error message within the CTL GUI that states "This certificate trust list is not valid.  The certificate that signed the list is not valid."

1. Is Microsoft aware of this issue and if so why isn’t the .stl fixed or re-signed?

2. *OR* if the CTL is OK but we have a problem in our environment how do we fix it?

3. Is the CTL update actually needed?

a. If so, why can’t it be distributed on WSUS as every other update?

b. If not, how do we stop computers (clients and servers) from attempting the download?

CAPI2 Event ID 11 is a very generic error message, I wrote http://blogs.technet.com/b/instan/archive/2010/08/12/capi2-event-id-11-errors-on-machines-that-don-t-have-access-to-the-internet.aspx concerning CAPI and Event ID 11 errors some months ago but this particular aspect we're looking at in this case is more related to the CTL GUI than to the actual CAPI2 part.

In short; there are two EKU's that can be used for signing CTL's (from http://support.microsoft.com/kb/287547) - both of them are valid from an RFC-perspective:

• szOID_ROOT_LIST_SIGNER                 
  1.3.6.1.4.1.311.10.3.9
  Signer of a CTL containing trusted roots
• szOID_KP_QUALIFIED_SUBORDINATION [also sometimes referred to as Microsoft Trust List Signing: msCTLSign]
  1.3.6.1.4.1.311.10.3.10
  Can sign cross-cert and subordinate CA requests with qualified  subordination (name constraints, policy mapping, etc.)
  
  
  The problem here is that the CTL GUI expects the EKU 1.3.6.1.4.1.311.10.3.10 to be used while the chain verification code recognizes both 1.3.6.1.4.1.311.10.3.9 and  1.3.6.1.4.1.311.10.3.10 as valid.

The net result of this is a cosmetic issue where the CTL GUI gives an error message that has no other functionality impact or relevance for the OS besides confusing the human observer.

So the net answer to the above questions is No, there is not an issue with the STL file in the cab - the error message is bogus and is caused by the CTL GUI not recognizing 1.3.6.1.4.1.311.10.3.9 as a valid EKU for trust list signing.
....I.e. its a non-issue that is not related to what is causing the actual Event ID 11 to be logged.

The other issue in this case with the error message when attempting to download the CTL is related to the Update Root CA feature of Windows Vista and Windows 7.  If the client is unable to download or extract the updated CTL it will log the error message.

The CAB file referenced contains a list of Root CA's that are part of the Microsoft Root CA program - whenever a Windows 7 or Vista client encounters an untrusted Root CA certificate they check with Windows Update if the hash in it is present in the CTL and if it matches they add it to the relevant Trusted Root store on the client.

NOTE! The fetching of the CAB from Windows Update is performed in the security context of the Computer account using WinHTTP - *not* the security context of the User account logged on to the machine.  The user being able authenticate to the proxy and log on to the website does not mean that the computer account is able to do the same - Kerberos authentication for the computer account would be required as the computer account will always be presented as Anonymous when accessing other systems over the network when NTLM is involved.

If the clients are going through a proxy that requires authentication then the alternatives are to either allow anonymous access to a subset of sites or to try and determine why the clients are unable to authenticate.  For Vista+ clients there were some changes to the way WinHTTP authenticates when retrieving PKI objects from the network which will effectively mean that by default the machine will not attempt to authenticate outside of its own domain.

The Update Root CA feature can be turned off by disabling the Update Root CA feature via Group Policy, the negative aspects of disabling it are that those clients will not become aware of new Root CA's in the program nor will they become aware of Root CA's that have been removed from the updated CTL (in case of a compromise of a Root CA certificate that is part of the program for example).

Concernng why WSUS cannot distribute the CTL - WSUS distributes security hotfixes... the updated CTL is not a security hotfix and therefore it cannot be distributed by WSUS.

Details:

Windows root certificate program members
http://support.microsoft.com/kb/931125

Windows Vista, Windows 7

Root certificates on Windows Vista and later are distributed via the automatic root update mechanism – that is, per root certificate. When a user visits a secure Web site (by using HTTPS SSL), reads a secure email (S/MIME), or downloads an ActiveX control that is signed (code signing) and encounters a new root certificate, the Windows certificate chain verification software checks Microsoft Update for the root certificate. If it finds it, it downloads the current Certificate Trust List (CTL) containing the list of all trusted root certificates in the Program, and verifies that the root certificate is listed there; it then downloads the specified root certificate to the system and installs it in the Windows Trusted Root Certification Authorities Store. If the root certificate is not found, the certificate chain is not completed, and the system returns an error. To the user, a successful root update is seamless. The user does not see any security dialog boxes or warnings. The download happens automatically. In addition, Windows Vista and later client SKUs support weekly pre-fetching from Microsoft Update to check for updated root certificate properties (for example, extended validation (EV), code signing or server authentication properties, which are certificate properties added to a root certificate).

For detailed technical information about how Windows updates root certificates in Windows Vista and in later versions, visit the following website:
http://technet.microsoft.com/en-us/library/cc749331(WS.10).aspx

Link to download the updated CTL manually for testing: 
http://www.download.windowsupdate.com/msdownload/update/v3/static/trustedr/en/authrootstl.cab

Object IDs associated with Microsoft cryptography
http://support.microsoft.com/kb/287547

Description of the changes to network retrieval of PKI objects in Windows Vista Service Pack 1 and in Windows Server 2008
http://support.microsoft.com/kb/946401

Automatic Root Certificates Update Configuration [using Group Policy]
http://technet.microsoft.com/en-us/library/cc734054(WS.10).aspx

When working with Microsoft technologies you'll inevitably come across references to Legacy API's, Legacy OS's, etc.

Have you ever wondered what that means in technical terms?

Well, in technical terms this is just a reference to indicate that a newer piece of whatever API component or OS we're discussing is available.

This introduces the scenario where one day an API is suddenly termed a Legacy API (this is typically at the launch of a new OS version which introduces new API's).
By itself it doesn't mean that the functionality of the API has changed or that it is less functional than the day before.

It does however mean that future development is less likely to take place in those specific components or API's as there are new components and API's available.
It also means that you may have scenarios where a third party component that was using current API's on a previous version of an OS will technically be using "legacy" API's if installed on the next version of the OS - even if it is using the exact same API in both cases.

A classic example of this is the Smartcard mini-driver architecture vs. the CSP architecture.  As Windows Vista introduced new smartcard API's and architecture when compared to W2k3, all references to CSP's that don't use that architecture are now typically in the terms "Legacy CSP". 
Both the mini-driver and the legacy CSP methods were still supported at the launch of Vista but this introduced scenarios where you could get an issue where a hotfix would only resolve an issue for a mini-driver but not for legacy CSP's (for example the fix in http://support.microsoft.com/kb/949538).

Another example are the public Terminal Server API's that were introduced with Windows Server 2008 R2 - in addition to the new public TS API's W2k8 R2 also contains the same TS API's that Windows 2008 did.
However, since there are newer TS API's available on W2k8 R2 these API's are commonly referred to as "Legacy TS API's".

The morale of the story is: Even if you don't change, the world around you does :)

Further reading:

Using and Understanding [Legacy TS] APIs for Terminal Server
http://technet.microsoft.com/en-us/library/cc751287.aspx

Remote Desktop Services Developer Resources
http://archive.msdn.microsoft.com/rdsdev 

Remote Desktop Services [API] Reference
http://msdn.microsoft.com/en-us/library/aa383494(VS.85).aspx

ICspInformation::LegacyCsp property       
http://msdn.microsoft.com/en-us/library/windows/desktop/aa376752(v=vs.85).aspx

Windows Vista Smart Card Infrastructure
http://msdn.microsoft.com/en-us/library/bb905527.aspx

This is a bogus error message that can be safely ignored - it's caused by the domain join code ending up in a function which it doesn't need to run anyway during a domain join operation using the GUI.

What's failing is the attempt to change the Primary DNS suffix of the machine after the domain join has succeeded - but the Primary DNS suffix is already correct at that point which is why this can be ignored.

The domain join itself is not affected at all and will have succeeded if you're seeing this specific error message with the empty quotation marks as that code is only run after a successful domain join.

As frustrating as it is, this is unfortunately one of those cosmetic issues which aren't likely to be fixed until the next OS version - primarily because it has zero impact and no effect beyond the ugly error message.

A workaround that removes the error message is to populate the Primary DNS suffix of the machine before attempting the domain join.

Footnote: This doesn't repro on Windows 8 or Windows Server 2012

See http://support.microsoft.com/kb/2018583 for further details.

For a DC to be able to service smartcard logons the DC must have a valid and suitable certificate present in the personal store of the computer account.
This is typically autoenrolled for whenever a Windows CA server has been installed into the AD environment.

The KDC service on W2k8 R2 monitors the personal certificate store of the computer account it is running on and gets notified when changes occur.
At that point a KDC certificate selection process gets kicked off and the Kerberos Distribution Center service parses the contents of the store for any suitable certificates.

• If the current DC certificate is still valid and passes trust and revocation checks - the current KDC cert will continue to be used.
• If it fails then a new KDC cert selection process is kicked off and a new KDC cert is picked from the local store if it contains any.  If more than one cert qualifies as a potential KDC cert then typically the most recent one will be picked.

The same KDC certificate selection process is invoked when the KDC service is (re)started and every 10 hours afterwards.

One thing to note from a troubleshooting perspective is that it is perfectly possible for the KDC to be unhappy with a DC certificate that is currently being used successfully for LDAPS for example.
I.e. beware of drawing any conclusions from testing if an LDAPS connection to the DC works - the KDC component does additional checks when considering if the cert is a suitable KDC cert candidate.

If the personal store of the computer account doesn't contain any KDC cert at all or if only the Public key part of the potential KDC certificate is present then that will of course cause the KDC cert selection to fail as there will be 0 candidates for a KDC cert in that case.

Related links:

Event ID 29 when starting KDC service on Windows Server 2008 R2 DC's
http://blogs.technet.com/b/instan/archive/2011/09/12/event-id-29-when-starting-kdc-service-on-windows-server-2008-r2-dc-s.aspx

Smartcard logon using certificates from a 3rd party on a domain controller and KDC eventID 29
http://blogs.technet.com/b/instan/archive/2011/05/17/smartcard-logon-using-certificates-from-a-3rd-party-on-a-domain-controller-and-kdc-event-id-29.aspx

CAPI2 Event ID 11 Retake
http://blogs.technet.com/b/instan/archive/2011/09/27/capi2-event-id-11-retake.aspx 

We want to implement a smartcard solution but we're not ready for an implementation internally. 
We considered implementing a standalone CA to avoid making changes to the Configuration partition but as it isn't able to issue smartcard certificates we're now considering a 3rd party solution instead.

A Standalone CA can actually issue smartcard logon certificates - but since it's not using certificate templates for that then you would need to manually format the request correctly before sending it to the CA for signing.  This is one of the advantages of using an Enterprise Issuing CA as that will use the appropriate smartcard logon certificate template to automatically format the request in the way that is specified in the template.

Additionally, regardless of if you implement a 3rd party CA solution or a Microsoft CA - you will most likely still need to have the same basic principles in place.

• The Root CA and Issuing CA's being used must be added to your configuration partition
• Your DC's must have Domain Controller certificates capable of servicing smartcard logons.
• Your smartcards must be capable of being used for logons (although this is extensively configurable in Vista+ machines)

I.e. you will typically have the same requirements with a 3rd party solution as with a Microsoft Enterprise CA infrastructure - both will most likely require additions to your AD Configuration partition (the Enterprise CA just automates this for you).

One possible quick alternative to the manual setup of the standalone CA is to promote the Standalone CA to a DC in its own forest and use it as an external Enterprise CA - Windows Server 2008 R2 added the possibility of cross-forest enrollment so you would in addition to the smartcard logon functionality also be leveraging the autoenrollment functionality of the OS.

A new hotfix for Cryptnet.dll on Windows Server 2008 R2 has been released which covers a scenario which could cause a Domain Controller (or any service doing frequent revocation checking of certificates, such as NPS or ISA Server) to get into a state where revocation checks started failing.
The revocation check failures on the DC would then in turn lead to smartcard logon failures for end-users

The catalyst for this was if an OCSP location was stamped on a certificate in the chain and this OCSP location became temporarily (or permanently) unavailable then revocation checks would start failing for that certificate - even if multiple CRL's stamped on the certificate where reachable and current.
The technical details are that under the right conditions the DC would be using a back-off value for the CRL and this backoff-value was being refreshed every time a similar request for the same CRL was received.  The back-off value is typically used to avoid overloading the server hosting the CRL so that during the back-off time no further attempts to download the CRL from that server would be attempted - in this case it meant that the back-off time was never expiring and this in turn caused the DC to never attempt to download the CRL when the error conditions are present.

If the DC was rebooted the problem would be resolved until the next time the OCSP location became unavailable.

However, the issue would reproduce only very sporadically due to being masked if a valid cached CRL was available locally.

The update is available in http://support.microsoft.com/kb/2666300 and is recommended for any DC's servicing smartcard logons where a certificate in the chain contains an OCSP path.

further details:
You cannot use a certificate-based logon method to log on to an NPS server that is running Windows Server 2008 R2
http://support.microsoft.com/kb/2666300