Thursday, December 20, 2012

SQL 2012: msiexec.exe running constantly after SP1 install

Heads up on a SQL 2012 SP1 bug; this happened to me on two pretty bare 2012 installs:

Symptoms: 


After installing SQL 2012 SP1, 2 msiexec processes spawn and run constantly. These generate significant CPU and disk activity.

Description: 


An error with the installer causes the .NET NGEN (Native Image Generator) to run in an indefinite loop. According to one post, it may even cause registry bloat to the point where the registry reaches the maximum size and you may need to re-install the OS on the machine. :-/ That said, I haven't verified that claim.

Fix:

None yet available

Update: Microsoft posted the official fix (the same as the early one below) here.

Workarounds:


  1. (Per Steve Philip) "Uninstalling the Management Tools (both Basic and Complete) feature seemed to resolve the problem for us on most of our servers. On those that had the Data Tools feature installed, we also had to remove that."
  2. (Per the Microsoft SQL Team.. supposedly) "Hello all, we have identified a temporary workaround for this issue. We will continue to work on a permanent fix and will communicate more information as it becomes available.If you are currently experiencing this issue, perform the following steps appropriate for you CPU architecture: 
    1. - Open an elevated cmd.exe 
    2. - If you are on a 64-bit architecture machine, run both of the following commands in the cmd.exe window:
    3.      - %windir%\Microsoft.NET\Framework\v4.0.30319\ngen.exe queue pause
    4.      - %windir%\Microsoft.NET\Framework64\v4.0.30319\ngen.exe queue pause
    5. - If you are on a 32-bit architecture machine, run only the following command in the cmd.exe window:
    6.      - %windir%\Microsoft.NET\Framework\v4.0.30319\ngen.exe queue pause
    7. After running these command(s) CPU consumption should return to normal and the 1004/1001 events should cease being published to the event log.
    8. More information on ngen.exe can be found here: http://msdn.microsoft.com/en-us/library/vstudio/6t9t5wcf(v=vs.100).aspx
    9. Note - after a machine reboot, the workaround will need to be re-applied"
  3. (Per "DizzyBadger") "Just stop and disable ALL Microsoft .Net Framework NGEN services, and make sure you kill any lingering msiexec.exe processes. If your software hive has already reached 2GB on the other hand, you are screwed. (Again, unverified) Then you have to reinstall the OS, at least if it is a production machine. You can murk around in the registry and delete the keys manually, but there is no knowing what else has been scrambeled due to the fact that nothing can be added to the registry hive past 2GB. The bloated keys are HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\v2.0.50727\NGENService &HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\.NETFramework\v2.0.50727\NGENService"
  4. (Update1; Per Paul D., see comments) Paul came up with a workaround to allow for the optimization tasks to still run; since we're still waiting on the official hotfix from MSFT you may want to have a look at that fix here.
There is another workaround listed on the site involving removing and re-installing the SQL management studio after patching, but this didn't work for me. 


Given the potential need to rebuild the OS it would make sense to check your production systems for this problem now.

Update 2: In the Microsoft connect article, MSFT has posted a fix, but gave the following warning about using it: "Please Note: The patch should only be applied if you're on SQL 2102 SP1. If you have already applied a Cumulative update or an "On Demand" fix on top of SP1, please wait for the associated hotfix update to be made available.


Thanks and regards
Shamik Ghosh
[SQL Server Team]
"

I can't speak to the success of this patch; I'll be waiting for the associated hotfix.

Thursday, December 13, 2012

Exchange 2010 Initial Database Setup

Here's a quick how-to on Exchange 2010 addressing a couple common problems:

Assumptions:

  • You have completed Exchange 2010 setup and not yet migrated mailboxes to it. 
  • You want to create one or more mailbox databases and delete the default. (The default names make it more difficult to manage) 
  • You have more than one domain controller (See point 2 below)
  • You are executing commands directly on the server in question. (You can do it remotely, but the syntax of the commands is different) 

Steps: 

  1. First things first, let's create your new database. Pick an appropriate name and open up the Exchange Management Shell (as admin)
  2. Most Exchange 2010 setups I've been through have been subject the error detailed here. Though the article talks about a multiple domain environment it may still appear in a single domain setup. It is due to the replication delay between your domain controllers. For this reason, I've found that keying your preferred server can alleviate this and a few other issues. Long story short, pick your best (local) domain controller, and execute:  Set-ADServerSettings -PreferredServer DC.FQDN.Here  . If you would like to see the DC configuration, you can use the command Get-ADServerSettings | Format-List . 
  3. Now make the DB by using the command: New-MailboxDatabase -server 'NETBIOSNAME' -Name 'DB_NAME' -EdbFilePath ':\DB_PATH\DB_NAME.edb' -LogFolderPath ':\LOG_FILE_PATH'  .  Where: EdbFilePath and LogFolderPath are where you want the DB and its log files. For best performance, put the DB and Logs on different drives. (Different physical disk, not partitions) Use this command to create additional databases if you like. (Archive, for example) 
  4. Mount the database with the following command: mount-database -Identity db_name
  5. Now we must move the System Mailboxes from the default DB per this article. Execute the command Get-Mailbox -Arbitration | New-MoveRequest -TargetDatabase "db_name"
  6. Now the discovery search mailbox; Get-Mailbox | where {$_.Name -like "DiscoverySear*"}| New-MoveRequest -TargetDatabase 'db_name'
  7. To check the move request status, execute Get-MoveRequest
  8. Once the move requests are completed, clear them all by executing Get-MoveRequest|Remove-MoveRequest and confirm. Note: This will clear all move requests, so if you have setup any moves other than those outlined in this article and don't want to clear them you'll need to be more granular. 
  9. Dismount the original db name with the following command: Dismount-Database 'OriginalDBName'  and confirming, I.E. Dismount-Database 'Mailbox Database 1781398675'
  10. Remove the old database with the command: Remove-MailboxDatabase 'OriginalDBName' and confirm. 
  11. You can now delete the DB folder and the log folder of the db you just deleted. The remove-mailboxdatabase command should have told you where the DB was located and by default the log folder is in the same location. 

You did it, time to celebrate. Now is when you should setup your DAGs, etc. and then move user mailboxes. Let me know if there are any questions. 

Wednesday, December 12, 2012

Off-Topic: Eac3to 3.24 with Arcsoft dtsdecoderdll.dll 1.1.0.0 on Windows 8

When I'm not doing my job one of my hobbies involves archiving all my movies to my NAS and make them available to my HTPC for my wife to watch at the touch of a button. After upgrading to Windows 8, however, I noted the encoding tool eac3to no longer worked correctly with Arcsoft dtsdecoderdll.dll V 1.1.0.0. (The only version that correctly decodes DTS-MA 6.1 tracks) It seems that the this older version of the DLL is hard-coded to use the Microsoft VC DLL MSVCP71.dll, which will not register correctly on Windows 8. (It uses a newer version of VC out of the box) To work around this, you need to run eac3to in Windows 7 compatibility mode and copy the dlls to the syswow64 directory. (No need to register since the app calls them directly) Here's a step by step:
  1. Copy msvcr71.dll and msvcp71.dll from a Windows 7 machine to the :\Windows\syswow64 directory
  2. Ensure your copy of ASAudioHD.ax is registered successfully. (This accompanies the dtsdecoderdll.dll) 
  3. Navigate to your eac3to folder and right click eac3to.exe-> properties.
  4. Click the "Compatibility" tab and click "Change settings for all users" (may as well)
  5. Under "Compatibility mode" check "Run this program in compatibility mode for" and select "Windows 7
  6. Click "OK"
As stated above, there is no need to register the msvcr71 and msvcp71 files because the app calls them directly. Now, you can test it by opening a command prompt, navigating to the eac3to directory, and typing:

eac3to.exe -test

You should see the following: 


If encoding a DTS-MA 6.1 to 5.1, make sure to use the command line -0,1,2,3,5,6,4 -down6 .

Enjoy!

Tuesday, December 4, 2012

Reccomended Network Adapter DNS Settings for a Domain Controller\DNS Server

DNS client settings for your domain controller

This comes up at nearly every client I've been to, and I have yet to see any comprehensive article from MS on the topic. There is always quite a bit of confusion surrounding what you should set the preferred DNS servers to in the network adapter of the DNS server itself. Here are some quick guidelines to save you some time:

Do:

  • Use another DNS server in the same domain as the primary, secondary, etc...
  • Use the loopback address, but not as the preferred server. Set it as the last server in the order. (Use the Advanced tab if you have more than two servers)
  • Clear the automatically added ::1 as the primary and only DNS server for the IPv6 stack unless you actively use IPv6. If so, then the same rule applies knowing that ::1 is the loopback address for IPv6.
    • Bonus, here's the command: netsh interface ipv6 delete dnsservers "Local Area Connection" ::1 where "Local Area Connection" is the name of the connection in question. It may say something to the effect of "No DNS Servers" after you execute the command, but that tingling just lets you know it's working like Denorex.

Example: 

My lab has two DNS servers in one domain: 192.168.1.30 and 192.168.1.110. Here is the IP config of the 1.30 machine:


Where there another DNS server in the domain that would be listed BEFORE 127.0.0.1. Note that the DNS servers list does not include ::1 as the preferred server since I removed that with the command listed above.


Do Not:

  • Use the server's own IP as the primary. Why? If something becomes wrong with the zone configuration or the replication of the records you could inadvertently isolate domain controllers from each other and give entirely inconsistent results to clients making troubleshooting the troublesome DC more difficult.
  • Use the loopback address as anything but the last entry.

Discussion: Some prefer to set the server's own DNS as preferred to reduce network traffic, and while I don't recommend that myself I will stress one final thing:
Whatever you decide to do with settings, be consistent. Lack of consistency inevitably costs time and money, usually when you don't have either.

Saturday, November 10, 2012

Setup Share Nothing Hyper-V Live Migration: Step-by-Step

Forward

Share nothing live migration in 2012 is pretty cool; it allows you to move a running VM from one Hyper-V host to the other with NO elements shared between the two prior to the move. For more information about the feature, see:

Microsoft: Live Migration Description
Aidan Finn: Live Migration Demo

How-To

Setup Constrained Delegation

First we need to setup constrained delegation so we don't run into permissions issues during this operation. For more information regarding constrained delegation, see Matthijs Seldam's description and this Technet description

Perform the following for each Hyper-V host in your environment: 
  1. Find the computer object in Active Directory Administrative Center (preferably) or AD Users and Computers
  2. Drill into the properties and locate the "Delegation" location/tab
  3. Select "Trust this computer for delegation to specified services only" and "Use Kerberos Only"
  4. (Repeat for EVERY Hyper-V host you intend to use other than the host you are editing) Click "Add" and type the machine name for the other Hyper-V host in question. Click "OK". 
  5. Select the following: (CTRL Click for multiple targets)
    1. Microsoft Virtual System Migration Service/COMPUTER
    2. Microsoft Virtual System Migration Service/COMPUTER.FQDN
    3. cifs/COMPUTER
    4. cifs/COMPUTER.FQDN
  6. (Optional if using file servers for ISOs or VHD storage) For each file server repeat step 4 and then 5 adding ONLY the cifs services
  7. Click "OK" to save & close the properties of the host. Ensure you repeat step 4&5 for each Hyper-V host. 

Setup Hyper-V

Important: repeat this process for each Hyper-V host.
  1. In the Hyper-V manager, open the properties of the host you want to edit. 
  2. Select "Live Migrations" on the left. 
  3. On the Live Migrations property page, Check "Enable incoming and outgoing live migrations". Under Authentication protocol, select "Use Kerberos", under Incoming Live Migrations select "Use these IP addresses for live migration" and then add each IP address on this host you would like to use for migrations. More IPs=more available bandwidth. 
  4. Click "OK" to accept changes and close the properties. Make sure to repeat these steps for each host.  

Move that thing!

  1. In Hyper-V manager, right click the guest you want to migrate and select "Move"
  2. Click "Next" on the Before you Begin page. 
  3. Keep "Move the virtual machine" selected and click "Next"
  4. Specify the name of the destination Hyper-V server and click "Next"
  5. Keep "Move the virtual machine's data to a single location" and click "Next"
  6. Select the destination folder on the target server and click "Next". 
  7. At this point the wizard will present you with any other decisions that need be made concerning the move; one common one is which HyperV Switch should be used on the target host
  8. Click "Finish"
It should migrate; (if not see below) the guest migration will trigger after the disk has copied over. Migration time will depend  on your disk read performance reading on the former host, writing perf on the target host, and network bandwidth. 

OH NO!


So let's say you're migrating between hosts with differing processors; they could even be the same brand. In my case, for example, I am migrating between a "Core 2 Quad" era Xeon and an "Ivy Bridge" Xeon. If the differences are substantial enough the machine won't move because the features exposed to the VM are missing on one of the platforms. Unfortunately you'll have to power down the guest to perform this fix, but it's a one time deal so next time the live migration will work. Lets do it: 

  1. Power down the guest. 
  2. Open the settings and expand the CPU specification
  3. Under "Compatibility" select "Migrate to a physical computer with a different processor version"
  4. Hit "OK" and start the guest back up. Live migration should now work. 

A Note on Performance: 

While the live migration performs well, the weakest link in your chain will be the speed things move at. Take this example: 

Read from: 4x 7200RPM SATA in RAID 10 
Write To: 6x 7200RPM SATA in RAID 5
LAN: 3x 1GB Full-Duplex w/Jumbo Frames


It does seem to automatically use SMB 3.0 Multi-Channel, but I am having a bit of problem scaling performance. While the screenshot above is held back by the source read speed, I didn't realize much more performance reading from the RAID 5 array, which has very good read speeds, to a different SSD-based RAID 10 array, which has crazy write speeds. I'll post any additional performance info I find. 

Tuesday, November 6, 2012

Installing .NET 3.5 on Windows10/8/Server 2012 With Powershell

Here's how to install .NET 3.5 (or another feature by subbing the feature name) on Windows 8/10 or Server 2012 from Powershell:
  • Launch Powershell as Admin
Install .NET 3.5 using default sources (Windows Update or defined in group policy)
  • 2012: Install-WindowsFeature –name NET-Framework-Core
  • Win8/10: Enable-WindowsOptionalFeature –Online –FeatureName NetFx3 –All
Install .NET 3.5 using a network SxS share: 
  • 2012: Install-WindowsFeature –name NET-Framework-Core -source \\server\share\source\sxs
  • Win8/10:Enable-WindowsOptionalFeature –Online –FeatureName NetFx3 –All -LimitAccess -Source \\server\share\source\sxs
    • On Windows 8, -all indicates that all parent features should be enabled as well and -LimitAccess prevents it from connecting to Windows Update. These options are not valid on Win 2012. 
Install .NET 3.5 using DVD-ROM 
  • 2012: Install-WindowsFeature –name NET-Framework-Core -source j:\sources\sxs    (See note below)
  • Win8:Enable-WindowsOptionalFeature –Online –FeatureName NetFx3 –All -LimitAccess -Source J:\sources\sxs
Make sure to run Windows Update after install if you're installing from a non-updated source!

Note: Depending on your configuration, on server 2012 you may need to use the install.wim on the install media and specify the index of the proper version of your install. To do so, execute:

Get-WindowsImage -ImagePath {OpticalDrive}:\sources\install.wim
This will display a list of images with associated indexes. Take note of the index of your install type.
Then, instead of the -source line above, use -source:wim:e:\sources\install.wim:X (where X is the number of the appropriate index. 

Update: Confirmed this works on Windows 10 as well!

Sunday, November 4, 2012

Server 2012 Fixes HyperV VM Import Process

When copying VHD files from one machine to another or re-creating a VM, Windows 2008r2 and 2008 required you to manually grant rights to the VM Specific SID after import to successfully start the VM. The import process wouldn't stamp the copied files with the proper machine ACLs resulting in a "General Access Denied" error. (fix here) Server 2012 seems to have fixed that problem. When importing a HyperV guest to 2012, the correct rights are automatically applied. This is a welcome time-saving fix and thus far I'm very impressed with the Hyper-V improvements in 2012. More to come on that... this is obviously small relative to the major functionality changes, but I'm happy they paid attention to the details as well.


Thursday, October 25, 2012

Windows 8 Media Center Free For a Limited Time

Though it doesn't seem to work yet, you can go here and get your key for Windows Media Center for Windows 8 Pro. I'm guessing it will go active tomorrow. I love WMC, and I'm guessing this will be Microsoft's last release of it; it seems they've chosen the Xbox as the go-forward media platform. Here's hoping some third parties can replicate the SmartGlass functionality for other devices with different software. :) Thanks for the heads up @GaborFari via @danielauger.

Thursday, October 18, 2012

WSUS 3.0 vs. 2012/Windows 8 Notes

As I posted earlier, server 2012 includes a new version of WSUS. There are a few gotchas associated with WSUS and Server 2012/Windows 8, especially as it pertains to using a previous version of WSUS. Here are some key points:

  • Windows 8 and Server 2012 "clients" will NOT work with WSUS 3.0 SP2 or any version that isn't shipped with 2012 unless this update is installed BEFORE any clients connect to it. 
  • If your Win8/2012 clients attempted to talk to an older WSUS server before patching or upgrading, you will need to perform the following before they will update again: 
    • Net stop wuauserv
    • rd /s %windir%\softwaredistribution\
    • Net start wuauserv
  • If your new WSUS 2012 server is downstream from an older WSUS server, it will have the same effect as if your clients were pulling directly from that older server. All WSUS servers between the clients and MSFT need to be newer or patched. 
  • According to Microsoft, updates canNOT be scanned by an intermediary... i.e. HTTPS inspection must be turned off on content from Windows Update. 
Client errors may manifest themselves as "error 0x80246003". According to the WSUS error table that corresponds to an unrecognized hash. I haven't completed my research yet but I'm guessing that the new endpoints will only honor update packages from MSFT using a new, stronger hash to raise security in the aftermath of the Flame malware. 

Sources:

Wednesday, October 17, 2012

WSUS on 2012 Using SQL Install Note

Here's a quick tip that will hopefully save you some time:

As you may have noticed, WSUS can now be installed as a role service in Windows 2012. I'm using it successfully now with an "external" SQL server. If you plan on doing the same, note that when you install the role it will inform you that the Windows Internal Database feature is a prerequisite even though you don't intend on using it.

As you're going through install, feel free to un-check the Windows Internal Database feature. You'll get a chance to specify the SQL target during the configuration phase of the install and all should work right off the bat.

Wednesday, October 10, 2012

Unicast Network Load Balancing... Reliably

Microsoft Network Load Balancing can be difficult to setup reliably and there are a myriad of better options out there. With that enthusiastic endorsement, I'm writing this guide to walk you through (at a very high level) how to setup unicast clusters reliably.

Assumptions

  • 2008r2 Servers (this should work down to 2k3 but there will be minor differences) 
  • Both hosts are connected to the same switch or vSwitch
  • You can have at least two NICs per host
  • 1 static IP address per NIC. (i.e. 2 per host) 
  • 1 IP for your clustered address
  • You really want to do this and don't have a better way to split network traffic 
Example Network: 

In this example, we use the following addresses. Substitute in yours where applicable. 
  • HostA: 192.168.1.5, (IP For Cluster NIC) 192.168.1.6 (IP for Other)
  • HostB: 192.168.1.7, (IP For Cluster NIC) 192.168.1.8 (IP for Other)
  • ClusterIP: 192.168.1.22

Steps

  • Ensure each host has at least two NICs with the appropriate IPs configured. You *can* setup a unicast NLB cluster with one per host, but trust me you don't want to. (unless your app can't handle a multi-homed server)
  • (Do for each host) Navigate to the advanced TCP/IP settings->DNS tab of the adapter that will participate in the cluster and UNcheck the box "Register this connection's addresses in DNS"
    • Note: This will ensure that when the machine is looked up by its dedicated individual IP as opposed to the IP used by the cluster adapter which will share a MAC address with the other host. 
  • Install the NLB feature. If you're not sure, here's some help: howto
  • Open the NLB Manager Administrative Tool and right click the root->New Cluster
  • Under Host, type the first host you wish to add, click connect, and then select the IP of the adapter that you removed from DNS a couple steps above and click "Next"
  • Set the priority and default state. Priority is this host's priority in the cluster where a lower number represents a higher priority, and default state represents the participation of this host in the cluster. If you're not sure, take the defaults of 1 and Started. Click "Next"
  • Click "Add" and put in your Cluster IP address. Click "Next"
  • On the next "Cluster Parameters" screen, put in the DNS alias you will use for the cluster IP under "Full Internet Name" and set "Cluster operation mode" to "Unicast". Click "Next".
Cluster Params
  • On the New Cluster: Port Rules you can accept the defaults unless you want to be explicit with your clustered ports. Click "Finish"
    • Quick note: MS NLB doesn't support automatic failover based on service status, I.E. if the host is responding to any network requests it is assumed to be up even though the service may have failed. That's why I chose to accept the default and host all services on the cluster IP and dedicate a NIC to the task. This has proven to be substantially more reliable. For security, ensure your firewall is active and configured correctly. 
  • Right click the newly created cluster and click "Add host to cluster" and input the second host. Follow the same steps that we did above for this host and exit the wizard. 
  • If you're using VMWare: perform the steps under "Configuring Unicast Mode" listed in this document. This disables the automatic MAC relocation on the vSwitches. 
  • If desired, do a static DNS registration for the "full internet name" of the cluster. 
That should do it. Remember that you will have to do fail-over manually in most circumstances. 

Sources/Additional Research: 


Tuesday, October 2, 2012

Microsoft Clarifies TMG/Forefront Death Watch

As I'm sure you're aware, Microsoft recently announced the cancellation of Threat Management Gateway as well as Forefront Protection for Exchange & Sharepoint. This is disappointing news to many and I'm trying my best to avoid reading between the lines on this one.

That said, I found that on the fifth page of the comments the team posted a clarification post, which I'll quote here for reference:

Microsoft Server and Cloud Platform Team
 
September 20th, 2012 2:32 PM

"We wanted to clarify some details around the discontinuation of Forefront TMG to help address many of the questions we’ve seen posted in the comments section.

First, it is important to note that while Forefront TMG is being discontinued, it will continue to be supported in mainstream support through April 14, 2015 and in extended support through April 14, 2020.  When and how a customer transitions to a replacement solution will depend on how the customer is using TMG today.  Customer use scenarios vary, but these general guidelines should help:

•  For customers using TMG for caching, secure web gateway (forward proxy), and firewall, we recommend that, prior to April 14, 2020, customers examine the many vendor solutions available in market today that offer comparable features to the TMG product.   Microsoft does not plan to transition this functionality to any other Microsoft products.

• For customers using TMG for reverse proxy, transitioning to Forefront UAG is an option.  Most web publishing scenarios that are supported by TMG can be published by UAG, though specific functionality may not be identical.   For customers who do not want to transition to Forefront UAG, customers should plan on transitioning to an alternative vendor solution prior to April 14, 2020.

•  For customers using Forefront TMG Web Protection Services, we recommend that customers examine the many vendor solutions available in market today that offer comparable reputation services by Dec. 31, 2015.  This product will no longer receive updates starting January 1, 2016.

We hope that these general guidelines provide additional clarity.  Please continue to contact your Microsoft account teams or partner managers with any questions about your specific scenarios.   

Regards,

The Forefront TMG Team
"

Tuesday, July 31, 2012

HowTo: Run (Signed) Powershell Scripts as Scheduled Tasks with Minimal Rights

This is meant to be a straightforward howto: on the topic. While this sort of thing is quite common, I often times see people missing a few critical details that end up resulting in alot of unnecessary troubleshooting time. I will also cover steps needed to run signed scripts.

Assumes:
  • Powershell 2.0
  • Windows Server 2008 or R2 (will work on others but steps will be slightly different) 
  • If signing script you know how to get a cert and basics about managing certs

Creating a signed script (optional) 

  1. Ensure you have a valid code signing cert installed in your user store and that you're logged on as that user. This is easiest if your organization has PKI with a code signing template enabled. This user should NOT be the service account user, but rather the author of the script. (you I'm guessing)
  2. After finishing your script, execute: 
    1. $cert=@(Get-Childitem cert:\CurrentUser\My -codesigning)[0]  (assuming you have only one code signing cert) 
    2. Set-AuthenticodeSignature .\myscriptname.ps1 $cert
Note: whenever you modify the script you'll need to re-sign it. 

Stage Server/Account 

  1. Create the service account. To be controlled correctly this should be a domain account with no additional privileges above that of a standard user. For additional security, set it so the user can only log on to the computer you intend to run the script on. Use a  good password.
  2. Give that user "Log on as a batch job" rights on the target server using the local security policy or group policy if applicable.
  3. Create the directory the script will be stored in. Give the service account READ access to that directory.
  4. Copy the script to the target directory
  5. Variable step if signing or not:
    1. If Signing: 
      1. If signing execute as admin in powershell on the server: Set-ExecutionPolicy AllSigned
      2. Log onto the server with the service account and run the script manually. This will prompt powershell to ask if you want to trust the publisher based on the cert. Select that you will always trust this publisher. (which is you!) If you can't log on locally or on a VM console you'll need to temporarily grant the service acct RDP access. 
    2. If not signing, execute as admin: Set-ExecutionPolicy Unrestricted

 Setup Scheduled Task

  1. Open Task Scheduler and navigate to the folder you would like the task to reside in. 
  2. In the right plane, right click->Create New Task...
  3. Give the Task a Name and Description. Don't skip the description, it'll save time having it there in the future. 
  4. Select "Run whether user is logged on or not" and ensure the "Do not store password" box is NOT checked. Select "Run with highest privileges" ONLY if your script requires it. (Powershell itself does not) 
  5. Click "Change User or Group" and enter the service account you created earlier. 
  6. Select the "Triggers" tab and set triggers as appropriate
  7. Select the "Actions" tab and click "New..."
  8. Action: "Start a program"; Program/script: "powershell.exe" (full path shouldn't be necessary) ; Add arguments: "-NoLogo -NonInteractive -WindowStyle Hidden D:\path\to\your\script.ps1"
  9. Change settings on the "Settings" tab if desired
  10. Click "OK" and enter the password for the Service Account
That's it! If everything was done correctly it should run without problem. Depending on what the script does you may need to grant additional rights to filesystems, etc, but that's up to you to figure out since you wrote the script. :) I hope this post saved you some time getting your script up and running. 

Monday, July 2, 2012

No muss and/or fuss SQL DB Mirroring

So I've never really known what "muss" is, but this doesn't have any of that. It's sans muss. (without muss)

I must say that the SQL DB Mirroring error messages (at least as of 2008R2) leave something to be desired. Because of this, I figured I'd write a straightforward HOWTO on the topic. I've performed these steps on SQL 2008R2, but I suspect it will work on other versions as well.

This guide assumes the following:
  • You have a DB on the Principal server that you wish to mirror
  • That DB is NOT on the mirror target server (drop/delete it if so) 
  • You do NOT intend on using a witness server.... using one however won't invalidate this guide, but we don't address the extra couple steps required to do so. 
  • Your database is using a "Full" recovery model. (not possible otherwise) 
  • You have sa access to both the Principal and Mirror server
Here are the steps, in order:

  1. Log onto the Principal server
  2. Perform a full backup to disk of the database in question. Ensure this is a new backup that doesn't include any previous backups.
  3. Perform a tranlog only backup on the same database. While this step shouldn't be required, I've found that restoring a tranlog backup after the full results in a successful mirror pairing substantially more often. 
  4. Copy the backup files to the Mirror server
  5. On the Mirror server, perform a full restore of the desired database:
    1. On the "General" page, ensure the "To database" is set to the exact same database name
    2. On the "Options" page, select "Leave the database non-operational..." "RESTORE WITH NORECOVERY"
  6. Right click the newly restored DB and select "Tasks"->"Restore"->"Transaction Log"
    1. On the "General" page, select the appropriate file
    2. On the "Options" page, select "Leave the database non-operational..." "RESTORE WITH NORECOVERY"
  7. On the Principal server, select "Tasks"->"Mirror"
  8. Click "Configure Security"
  9. Select "No" for "Include Witness Server" and click "Next>"
  10. Leave the ports default unless there is a reason to do otherwise. Make sure this port is properly opened on all firewalls (including the local firewall) between the to servers!
  11. Select the target server, set the settings per your requirements (default are fine) and click "Next>". Double check that all firewalls are open on and between both servers. 
  12. Enter the appropriate service account for this replication. While it is a lesser secure configuration, using the SQL instance domain service account (if you have one) works without issue. NOTE: If your SQL service instance is running as a local user, (including system) you must use certificate authentication which is NOT covered here.
  13. Click "Next","Finish" then "Start Mirroring". All should work at this point. Note it is normal for the state on the Mirror server to be "Synchronized/Restoring".

Troubleshooting:
Explicitly grant connection rights to the "Mirroring" endpoint (note that's a name not a concept) to a given service account:
GRANT CONNECT ON ENDPOINT::Mirroring TO "Domain\Username"


Note that if you re-create the mirror it does not delete the endpoint. To delete the endpoint on each server: (assuming you took the default name of "Mirroring")
DROP ENDPOINT MIRRORING



References:
http://msdn.microsoft.com/en-us/library/ms189047(SQL.105).aspx
http://msdn.microsoft.com/en-us/library/ms189127.aspx

Good Luck!

Wednesday, April 11, 2012

Creating a Two Tier PKI With Windows 2008r2


I've recently undertaken the task of setting up a Public Key Infrastructure (offline Root, online issuer) for my current engagement. I've been through this a few times and there is a lot of info out there, so rather than write a detailed step-by-step, I'm going to list major notes & "gotchas" while providing links to other blogs and tech articles.

Update note: This article was authored in April of 2012, when SHA2xx adoption by network devices was such that SHA1 was still predominant in enterprises. When instructed below to use SHA1 for the online CA, please do not. SHA256 should be considered the minimum as of 2016.

Why an offline root? 


An offline root is a much more secure option than having your online issuing enterprise CA serve as root because if the root CA were to be compromised it could invalidate your entire PKI. By keeping your root CA offline (unplugged or shut down) you make it nearly impossible to compromise that box. In that scenario, if one of your online issuing CAs is compromised you will only need to revoke the certificates issued from that level down. (or just that CA depending on circumstances) This will save you from having to re-build from scratch.

More info: Technet Wiki entry by Kurt Hudson, Ed Price and others

Prerequisites:  

  • Assumes there is no existing CA infrastructure. If you have one you will need to either A>Decommission or B> Transition. Follow the Decom link for more info there; the transition piece can have many variables so I won't cover that here.
  • Assumes you have licenses for Windows. As of 2008, you will need at least 1 Standard edition (for the offline root) and 1 Enterprise edition. (for full functionality of the issuing CA, you can use Standard with slightly reduced functionality)

Step 1: Setup Offline CA

Setup Server

  • Setup a new Windows 2008r2 Standard Edition server. If running on a virtualization platform err on the side of compatibility vs. performance when selecting virtual hardware. Make sure you record the admin password for your company since after getting the system setup it'll be quite a long time before you need to log on again.
  • DO NOT JOIN THIS SERVER TO YOUR DOMAIN then fix your caps lock key. (mine was stuck) 
  • Enable RDP if desired. (available under 3>Customize This Server in the "Initial Configuration Tasks") The rest of this guide assumes you're connected via RDP.
  • Name the server as desired, then activate and then fully update using Windows Update. Make sure the time & time zone are set correctly. 

Install Certificate Services

  • From Server Manager, select "Roles->Add Roles"
  • On the Server Roles page, select "Active Directory Certificate Services" and hit "Next"
  • Under "Role Services", select "Certification Authority" only & hit "Next". 
  • The "Setup Type" should default to "Standalone"; ensure the "CA Type" defaults to "Root CA".
  • On the "Private Key" step, select "Create a new private key" and then we have a couple choices to make under "Cryptography"
    • CSP: Generally stick with the default "RSA#Microsoft Software Key Storage Provider" unless you have need of a different CSP by way of other network hardware such as smart cards, etc. You should know if you need a different CSP. If you don't, you don't. :)
    • Hash Algorithm: This can be a bit tough; SHA256 (SHA2) is a much more secure hash than SHA1, but some systems may not be able to use it. (WinXP SP2, some Oracle services) You'll need to weigh the risk here. In my scenario I'm deploying this time around for a small/mid business with significant legacy equipment, so I'll favor compatibility over security. If you're unsure, follow my lead and select the "SHA1" algorithm. If you're certain you can get away with it, (not WinXP SP3 will have enroll issues) select "SHA256" or better.
    • Key character length: This is a similar discussion as the hash. If you plan on having a key duration longer than 10 years, (which I'll recommend in a moment) it is generally suggested (2 links there) to shoot for 4096 bit, but that may have compatibility and performance tradeoffs. One notorious offender has been Cisco, but their main IOS resolved that issue in ver 12.4T. That said, ensure all devices you will need certs on and select for 4096 if possible. If you are unsure on not capable of verifying, select 2048.
    • "Allow administrator interaction when the private key is accessed by the CA.": Check this if your CSP (above) requires it. The default Microsoft Software KSP does NOT require it.
  • For "CA Name": Set a good common name for the root CA. This should not reflect the machine name but rather the organization, i.e. "MyCompany Root CA". Keep it short and without special characters. Set the Distinguished name suffix to something that doesn't reveal security information about your internal structure. This does not need to match AD as this isn't an AD integrated system. It can be blank if desired. I'm using "OU=PKI,O=CompanyName,C=US" where US is the country code. (use yours as appropriate)
  • Validity Period: 20 years. A good rule of thumb is that top level CAs must be at least twice the duration of the CA directly below it. Do 10 if you like, but I see little reason for it. See here for further considerations.
  • Certificate Database: Default locations for the offline root CA. We'll change the locations for performance reasons on the issuing CA but that's not a concern here.
  • Accept the rest of the dialogs and continue. 

Step 2: Configure Offline CA

Change the Default Cert Request Action

First we'll ensure that certificate requests will NOT be automatically processed by the Root CA. Note: this appears to be correct by default in Win2k8r2, but due to its importance I'm leaving the step in.
  • Open the cert authority and drill to Certification Authority->CA Name
  • Right Click->Properties
  • Policy Module->Properties
  • Check the box that says "Set the certificate request status to pending."
  • Click "OK" twice to get back to the main CA MMC

Set the CRL Publication Interval

The CRL defines which certificates have been revoked. You need to strike a balance between administrative overhead and time for effect on revocation. Since this CA is offline each time the CRL needs to be published it will be a manual process. The only certificate revocation that will be published by this is for a subordinate CA. For my scenario I'll be using the CRL publishing interval of 6 months.
  • Expand your Root CA object and Right Click "Revoked Certificates"->Properties
  • Set "CRL publication interval" to the desired time frame. I'm recommending 6 months.
  • Ensure "Publish Delta CRLs" is not checked.
  • Click "OK"

Set Configuration DN, Domain DN and Certificate Validity Period Registry Keys

Note: You can skip the ca\DSxxxxxxDN steps if you NEVER plan on publishing CRLs or Certs via Active Directory, but its an easy step so I don't recommend it.  

When you use variables in the AIA and CDP paths for the LDAP publication (i.e. ) these registry keys are referenced to populate those fields with the correct information. 
  • Open a command prompt as administrator
  • Execute "certutil -setreg ca\DSConfigDN CN=Configuration, DNpath" (i.e. certutil -setreg ca\DSConfigDN CN=Configuration,DC=CompanyName,DC=local)
  • Execute "certutil -setreg ca\DSDomainDN DNpath"
    • DNpath should be the appropriate path. To get it correct you can use ADSIEdit to connect to the default naming context. 
  • Execute "certutil -setreg ca\ValidityPeriodUnits 10" 
  • Execute "certutil -setreg ca\ValidityPeriod "Years"
    • These last two commands set the default duration of certificates issued. Since we will only issue for intermediate CAs we'll set it for 10 years. If you set your RootCA for 10 rather than 20 earlier, follow your own lead and set this to 5 instead.

Set CDP and AIA Locations

The CDP and AIA locations specify the place clients can retrieve the CRL and certificate. Remember this CA is offline, so we need a maintainable future proof strategy to serve those up. For that reason, I'm recommending that the primary delivery method be http using a custom DNS name dedicated for the task. I'll then use our public web infrastructure (with split DNS) and rely on the SLA associated with that platform. I can then use host headers to parse out the PKI requests and have everything served up on port 80. I will be using the hostname pki.company.net. Substitute yours where appropriate. Alternatively it would be easy to use the custom DNS to point to your (not yet built) online issuing CA. Additionally we'll publish to LDAP as well. Note that the default paths will work for that since we set the registry locations, but I'll still be making some additional changes for added security. See here, here, and here for more info.

Note: After the changes below you may be prompted multiple times to restart the CA service. Don't do it then; we'll restart it all when we're done here.
  • Select the CA name at the root and Right Click->Properties
  • Select "Extensions" and ensure "CRL Distribution Point (CDP)" is selected
  • Select "C:\Windows..." and ensure only "Publish CRLs to this location" is checked
  • Select "ldap:///..." and click "Remove"  (IF you want added security...I'll explain why in a moment)
  • Select "http://..." and click "Remove"
  • Select "file://...." and click "Remove"
  • Click "Add..." and type the following in "Location:" ldap:///CN=<CATruncatedName><CRLNameSuffix>,CN=01,CN=CDP,CN=Public Key Services,CN=Services,<ConfigurationContainer><CDPObjectClass> and click "OK".
    • Important note: This is one place where I'm taking an extra measure of security; normally the CN=01 would be CN=(machinename), but I would prefer not to reveal the real name of the internal server in the CDP string. I'll be using a two digit incrementing integer for top level CAs and a three digit incrementing integer for CAs down the tree. Feel free to leave this to the default of machinename if you desire.
  • With the new ldap:/// entry selected, ensure "Include in all CRLs. Specifies where to publish in the Active Directory when publishing manually." and "Include in the CDP extension of issued certificates" are checked
  • Click "Apply" 
  • Click "Add..." and type the following in "Location:" http://pki.company.net/CertEnroll/<CaName><CRLNameSuffix><DeltaCRLAllowed>.crl   (substitute the correct DNS name for your company)
  • With the new http:// entry selected, ensure "Include in CRLs. Clients use this to find Delta CRL locations." and "Include in the CDP extension of issued certificates" are checked
  • Click "Apply"
  • Under "Select extension:" select "Authority Information Access (AIA)"
  • If you chose to obscure the servername as listed a few points above: Select "C:\windows..." and click "Remove"
  • Select "http://..." and click "Remove"
  • Select "file://...." and click "Remove"
  • If you chose to obscure the servername as listed a few points above: Click "Add..." and type the following in "Location:" C:\windows\system32\certsrv\CertEnroll\01_<CaName><CertificateName>.crt
  • Click "Apply"
  • Click "Add..." and type the following in "Location:" http://pki.company.net/CertEnroll/01_<CaName><CertificateName>.crt Substitute pki.company.net for you URL and if you did NOT opt to obscure the servername replace 001 with <ServerShortName>.
  • With the new http:// entry selected, ensure "Include in the AIA extension of issued certificates" is checked
  • Click "OK"
  • When prompted to restart certificate services, select yes.
Notes: The default LDAP location for AIA should be fine provided you did the registry entry stuff above. The default C: and http:// locations are fine if you don't want to obscure the server names. At this point, double check all your entries. If you proceed from here and find out there was a typo in any one of these entries later you will have to revoke the issuing CA cert to re-mediate the problem.

Step 3: Setup Online CA

In this part of the exercise we'll install all the online CA features on one box. Depending on your environment you may want to split some of the services across multiple machines.  For scalability info, see: here.

Note, you may want to take a quick moment at this time to review Appendix A below regarding the role of the CAPolicy.inf file. 

Setup Server

  • Setup a new Windows 2008r2 server. For full functionality on the issuing CA, use Enterprise edition. (Note 2012 has full functionality in standard and up) I recommend x64 for scalability reasons. If possible I'd recommend having a separate drive to house either the DB or the logs for performance reasons. (unless the drive is on the same subsystem) This server should be joined to the domain before you start installing certificate services.
  • Enable RDP if desired. (available under 3>Customize This Server in the "Initial Configuration Tasks") The rest of this guide assumes you're connected via RDP.
  • Name the server as desired, then activate and then fully update using Windows Update. Make sure the time & time zone are set correctly. 

Install Certificate Services

  • From Server Manager, select "Roles->Add Roles"
  • On the Server Roles page, select "Active Directory Certificate Services" and hit "Next"
Note: the next step will prompt you to add additional role services required by two of the main roles. When prompted, click "Add Required Role Services" to continue. 
  • Under "Role Services", select "Certification Authority", "Certification Authority Web Enrollment","Online Responder" (if you plan on using OCSP, if you don't know what it is don't bother yet), and "Certificate Enrollment Policy Web Service" then click "Next". 
  • Change "Setup Type" to "Enterprise" and click "Next".
  • Change "CA Type" to "Subordinate CA" and click "Next".
  • On the "Private Key" step, select "Create a new private key" and then we have a couple choices to make under "Cryptography"
    • Refer to the info above for details about CSP, Hash, and Key Character Length. For my main issuing CA, which this will be, I will be selecting options to ensure compatibility. I would advise (at the time of this writing) against using 4096 bit for the issuing CA for performance reasons. 
    • CSP: "RSA#Microsoft Software Key Storage Provider" (unless otherwise needed)
    • Hash: "SHA1" (Unless you're brave enough for something nicer :) ) 
    • Key Character Length: 2048
    • "Allow administrator interaction when the private key is accessed by the CA.": Check this if your CSP (above) requires it. The default Microsoft Software KSP does NOT require it.
  • For "CA Name": Set a good common name for the issuing CA. This should not reflect the machine name but rather the organization, i.e. "MyCompany Intermediate CA xx". (Where xx is an increment) Keep it short and without special characters. Set the Distinguished name suffix to the same as the root CA.
  • Certificate Request: Select "Save a certificate request to file and manually send it later to a parent CA:" and save it locally to send over later. 
  • Validity Period: 20 years. A good rule of thumb is that top level CAs must be at least twice the duration of the CA directly below it. Do 10 if you like, but I see little reason for it. See here for further considerations.
  • Certificate Database: Optimally the database location and database log location should be placed on a disk other than the OS disk. (different disk, not just performance) If performance is a very large consideration (i.e. very, very heavy load) you can split the DB from the DB log location as well. I'll be placing both on D:. Create folders and place the certificate database location to D:\pki\CertServ and set the Certificate database log location to D:\pki\CertLog
  • Authentication Type: Select "Windows Integrated Authentication"
    • Note there may be situations one may want to use other methods, i.e. allowing requests originating from the internet. 
  • Server Authentication Certificate: Unless you already have a certificate for this server, select "Choose and assign a certificate for SSL later"
  • Web Server (IIS)->Role Services: Accept Defaults. The appropriate role services should already be selected, only add or remove if you have unique IIS needs. 
  • Confirmation: Save the report if you like and then click "Install"
  • Results: You will get a warning about the installation being incomplete since the Sub CA certificate needs to be processed. Save the report if desired and click "Close"  

Publish Root CA CRL & CRT to AD

  • Fire up & login to the Root CA and then Open the Root CA Certificate Authority
  • Expand the CA and right click on "Revoked Certificates"->"All Tasks"->"Publish"
  • Keep "New CRL" selected and click "OK".
  • Navigate to "C:\Windows\System32\certsrv\CartEnroll" 
  • IF you changed the AIA location above to replace the machinename with 01 as I did, rename the .crt file so that the machinename portion of the file is replace with 01. I.E. servername_mycompany root ca.crt becomes 01_mycompany root ca.crt
  • Copy the .crt and .crl files to a domain controller  (You will need to map a drive)
  • Log onto the domain controller the files were copied to and open an elevated cmd prompt
  • Navigate to where the files reside and publish the RootCA cert: certutil -dspublish -f "01_My Company Root CA.crt" RootCA
    • Where: certutil is the tool, -dspublish is the action, -f creates a new container in AD, and RootCA specifies the cert is a Root CA cert. Important Note: For these certutil commands to run successfully you must be in the Enterprise Admin group.
  • Publish the CRL: certutil -dspublish -f  "My Company Root CA.crl"

Publish Root Cert to Clients via GPO

  • May as well do this while we're still on the DC: Open Group Policy Management & Right Click Default Domain Policy->Edit
  • Drill to Computer Configuration->Policies->Windows Settings->Security Settings->Trusted Root Certification Authorities
  • Right Click->Import->Next
  • Select the Root Certificate->Next->Keep default of "Place all certificates in the following store"->Next->Finish
  • Log off DC

Publish Root CA CRL & CRT to Web

This step depends entirely on what type of web infrastructure you plan on using, so I'll skip it with only the following advice: Make sure the CRL and the CRT are available from the CDP and AIA locations you advertised earlier. If using the split DNS approach test the retrieval from internal and external clients. Make sure you allow + signs on IIS7 or higher by allowing "double escaping" in Request Filtering.

Set Sub CA CDP and AIA

If you planned a good infrastructure for your Offline CA, you can leverage that here as well.
  • Log on to Sub CA
  • Select the CA name and Right Click->Properties
  • Hit "OK" on the Active Directory Certificate Services warning. 
  • Select "Extensions" and ensure "CRL Distribution Point (CDP)" is selected
  • Select "C:\Windows..." and ensure "Publish CRLs to this location" and "Publish Delta CRLs to this location" are checked
  • Select "ldap:///..." and click "Remove"  (IF you want added security...I'll explain why in a moment)
  • Select "http://..." and click "Remove"
  • Select "file://...." and click "Remove"
  • Click "Add..." and type the following in "Location:" ldap:///CN=<CATruncatedName><CRLNameSuffix>,CN=001,CN=CDP,CN=Public Key Services,CN=Services,<ConfigurationContainer><CDPObjectClass> and click "OK".
    • Note: This is another place where I'm taking an extra measure of security; normally the CN=001 would be CN=(machinename), but I would prefer not to reveal the real name of the internal server in the CDP string. Since this is a sub CA I'm using a three digit incrementing integer. Feel free to leave this to the default of machinename if you desire.
  • With the new ldap:/// entry selected, ensure "Publish CRLs to this location","Include in all CRLs. Specifies where to publish in the Active Directory when publishing manually.", "Include in CRLs. Clients use this to find Delta CRL locations","Include in the CDP extension of issued certificates", and "Publish Delta CRLs to this location" are checked
  • Click "Apply" 
  • Click "Add..." and type the following in "Location:" http://pki.company.net/CertEnroll/<CaName><CRLNameSuffix><DeltaCRLAllowed>.crl   (substitute the correct DNS name for your company)
  • With the new http:// entry selected, ensure "Include in CRLs. Clients use this to find Delta CRL locations." and "Include in the CDP extension of issued certificates" are checked
  • Click "Apply"
  • Under "Select extension:" select "Authority Information Access (AIA)"
  • If you chose to obscure the servername as listed a few points above: Select "C:\windows..." and click "Remove"
  • Select "http://..." and click "Remove"
  • Select "file://...." and click "Remove"
  • If you chose to obscure the servername as listed a few points above: Click "Add..." and type the following in "Location:"C:\windows\system32\certsrv\CertEnroll\001_<CaName><CertificateName>.crt
  • Click "Apply"
  • Click "Add..." and type the following in "Location:" http://pki.company.net/CertEnroll/001_<CaName><CertificateName>.crt Substitute pki.company.net for you URL and if you did NOT opt to obscure the servername replace 001 with <ServerShortName>.
  • With the new http:// entry selected, ensure "Include in the AIA extension of issued certificates" is checked
  • Click "OK"

Process and Install Sub CA Certificate

  • Log onto Sub CA
  • Copy the previously created certificate request from the root of C:\ to the Offline CA
  • Log onto the Offline CA and Open the Certificate Authority 
  • Right Click the Root of the CA->"All Tasks"->"Submit New Request"->Select the request you just copied over
  • Click "Pending Requests"->Right Click (should be ID2)->"All Tasks"->"Issue"
  • Click "Issued Certificates"->Right Click (should be ID2)->"Export Binary Data"
  • Keep "Binary Certificate" selected and choose "Save binary data" & click "OK"
  • Save as "cert.p7b" locally and then move it to the Sub CA
  • Back on the Sub CA: Right click the root of the CA->"All Tasks"->"Install CA Certificate" & Select the p7b from the Root CA
  • Start the CA with the "Play" button at the top. 
  • Make sure you delete the requests and generated certificates (*.req and *.p7b)  from the filesystem on all servers.
  • If you chose to obscure the servername as listed a few points above: You may need to manually add the CRL to AD for the first publish due to a bug; Open a CMD window as administrator and execute certutil -dspublish -f  C:\windows\system32\certsrv\certenroll\CRLFileName.crl

Publish Sub CA CRL & CRT to Web

If you used the built-in web server you don't have much to worry about here, but if you, like I did, utilized your external facing web farm to leverage split DNS and that SLA, you'll need to devise a way to publish the CRL (and delta) regularly to the web farm. Since that shouldn't be part of the same domain you may need to get creative. I'm thinking about creating a dedicated service account, giving it an EFS cert, and then storing the powershell script on the server with the username/password required in there. That way only the svc acct will be able to read the script. I won't outline that now though... perhaps in a future post.

Test AIA and CDP Points With an Issued Certificate

With the first issued cert from your Sub CA, drop to a command line as administrator and execute: certutil -verify -urlfetch certname.cer . This will display the result of checking the AIA and CDP of both the root and the Sub. If there are any problems go back and clear them up now before issuing any more certs. If there is an issue with the root you'll have to fix it and re-issue the Sub CA cert.

When you're done, take the issuing CA Offline.

Step 4: Template Configuration, etc.  

At this point you're actually done with the setup, but it's not all that functional yet. The next section is a start on getting the system to actively issue certificates. Before proceeding in these steps it is recommended that you mess around a bit to familiarize yourself with the product. The examples below are only a few of what could and should be many different certificate templates for a fully featured PKI. Please take time to understand certificate templates before calling it completed and firing up your infrastructure.

Important Note:  Throughout this next step I will make a couple references to enabling auto-enrollment security permissions on a given certificate template. While this is what we will ultimately want, I recommend that you first enable enroll only, test the template with a manual enrollment & use, and then circle back and enable auto-enrollment after you have verified proper functionality.

Make Desired Certificate Template Modifications

Before opening the floodgates for certificate auto-enrollment, you will want to modify certificate templates as desired. Cert templates control everything from cert duration to cert purpose. There are some that are installed by default, but I'd highly recommend familiarizing yourself with the concepts and making necessary changes before certificates are issued. Here are some great links to get you there:
At a minimum I would configure user and EFS templates for key archival. For more information regarding Key Recovery Agents, see "Identify a Key Recovery Agent" at Technet. As a demonstration, let's setup Key Archival and fix up the basicEFS template:

Key Recovery Agent Setup

First, we need to select an account for the KRA:
  • Create or select a user that will act as the Key Recovery Agent. Choose wisely and document accordingly; this acct will need be around for awhile. I created a separate account for just this purpose.
Now, let's modify the Key Recovery Agent template:
  • On the issuing CA, Right Click "Certificate Templates" and click Manage
  • Right click "Key Recovery Agent" and click "Duplicate"
  • Select Windows 2003. This will make the template V2... if we don't need V3 functionality don't create it. 
  • Under "General" change the Template Display Name to Key Recovery Agent for CompanyName; I like to append the company name to the default name so you know on what it is based and who it's for. Leave the Template Name as is. (it will auto change)
  • Under "Request Handling" ensure that "Allow private key to be exported" is checked. This will allow us to export the key and save it somewhere safe. 
  • Under "Superseded Templates" click "Add...", select "Key Recovery Agent" and click "OK".
  • Under "Security" add the account you chose earlier and give it "Read" and "Enroll" rights.
  • Click "OK"
  • Start the CA server (Temporarily to for these tasks) 
  • Back on the CA, under "Certificate Templates", right click in a blank space on the middle plane and select "New"->"Certificate Template To Issue"
  • Select your new EFS certificate template and click "OK"
  • Repeat the last two steps for your new Key Recovery Agent. 
  • Select "Basic EFS" and press delete; click "Yes" to confirm 
 Now let's get a key for the KRA. The easiest way I know of to do this is as follows:

  • Open an administrative Command Prompt
  • Execute "runas /user:domain\kraAccount mmc" then hit Enter and type the password. Substitute the domain and kraAccount with the right account info. 
  • Add the certificates snap-in (personal) to the MMC and click "OK"
  • Right click on "Certificates (Current User)"->"Personal"->"All Tasks"->"Request New Certificate"
  • Click Next
  • You should see "Active Directory Enrollment Policy" under "Configured by your administrator". Select it and click "Next". 
  • Select "Key Recovery Agent for MultiTech" and click "Enroll"
  • After that completes, go back to your Issuing CA and look under "Pending Requests"
  • Right click the pending cert and approve
  • Navigate to "Issued Certificates"
  • Right click the newly issued KRA cert and Right Click->"All Tasks"->"Export Binary Data"
  • Select "Save binary data to a file" and save the cert (as whatever.p7b) to somewhere you can get to with the client.
  • Back at the client, Right click "Personal"->"All Tasks"->"Import" 
  • When prompted, select your new cert and navigate through the wizard. 
  • After importing, select the newly imported cert (under personal->certificates) and Right Click->"All Tasks"->"Export"
  • Hit "Next", "Yes, Export the Private Key", "Next", "Personal Information Exchange...","Next", type a password, "Next", select a location, "Next","Finish"
  • Copy that key to a safe location that will be backed up. 
Enable the Recovery Agent on the Issuing CA
  • On the Issuing CA, right click the CA name and click "Properties"
  • Hit the "Recovery Agents" tab and select "Archive the key"
  • Click "Add" and hit "OK" when presented with the cert you just created.
  • Click "OK"
  • Restart the service when prompted.

Modify the Basic EFS Template 

  • On the issuing CA, Right Click "Certificate Templates" and click Manage
  • Right click "Basic EFS" and click "Duplicate"
  • Select Windows 2003. This will make the template V2... if we don't need V3 functionality don't create it. 
  • Under "General" change the Template Display Name to Basic EFS for CompanyName; I like to append the company name to the default name so you know on what it is based and who it's for. Leave the Template Name as is. (it will auto change) 
  • Check "Do not automatically re-enroll if a duplicate certificate exists in Active Directory". Publish in Active Directory should already be checked. This will ensure if a user already has pulled an EFS cert and stored it in AD that the old one will be used rather than grabbing a new one.
  • Under "Request Handling" check "Archive subject's encryption private key". This will store a copy of the private key on the CA so that you can use the key recovery agent to get it in the future if it is a problem.
  • Under "Superseded Templates" click "Add...", select "Basic EFS" and click "OK". This will force people who pulled an EFS template to get a new cert with this one.
  • Under "Security" check "Autoenroll" for "Domain Admins", "Enterprise Admins", and "Domain Users" and/or anyone you would want to use EFS. (Important, see the note below)
  • Click "OK"
  • Start the CA server (Temporarily to import cert) 
  • Back on the CA, under "Certificate Templates", right click in a blank space on the middle plane and select "New"->"Certificate Template To Issue"
  • Select your new EFS certificate template and click "OK"
  • Select "Basic EFS" and press delete; click "Yes" to confirm 
  • Stop your CA to prevent issuing more certs before you have completed your template tweaks
Note: Unless you fully understand EFS, don't go forward without revoking the permissions to "Domain Users" on the template. If you have users who have files already encrypted with local or other PKI keys you'll need a migration strategy. It may be worthwhile to initially set the template to "enroll" rather than "autoenroll" and then use "cipher.exe /K" to gen a new key, "cipher.exe /U" to update previous files, and then "cipher.exe /x" to export the key for other machines. (remember the private key is not stored in Active Directory) For more info, see: 

Zow, that was fun. So much so I'll do another one. The default DC cert is the Windows 2000 geared "Domain Controller". If you, like I, have all 2008 or higher DCs you can use the "Kerberos Authentication" template. If you have mixed 2003/2008 or just 2003 use the "Domain Controller Authentication" template.
  • On the issuing CA, Right Click "Certificate Templates" and click Manage
  • Right click "Kerberos Authentication" and click "Duplicate"
  • Select Windows 2003. This will make the template V2... if we don't need V3 functionality don't create it. 
  • Under "General" change the Template Display Name to Kerberos Authentication for CompanyName; Leave the Template Name as is.
  • Under "Superseded Templates" click "Add...", select "Domain Controller" and click "OK". Rinse and repeat for "Domain Controller Authentication" 
  • Click "OK"
  • Start the CA server (Temporarily to import cert) 
  • Back on the CA, under "Certificate Templates", right click in a blank space on the middle plane and select "New"->"Certificate Template To Issue"
  • Select your new Kerberos certificate template and click "OK"
  • Select "Domain Controller" and press delete; click "Yes" to confirm  
  • Select "Domain Controller Authentication" and press delete; click "Yes" to confirm  (Unless you have 2k3 DCs)
To get SSL encrypted LDAP working, follow the instructions in this article.

Let's make one for Terminal Services & Standard Machine ID/Encryption:

Note: Let me save you some time & research and just state that you cannot automatically add the NETBIOS name as a subject alternative name with templates. If you want so suppress the certificate warnings on RDP connections you'll need to use the FQDN.
  • On the issuing CA, Right Click "Certificate Templates" and click Manage
  • Right click "Computer" and click "Duplicate"
  • Select Windows 2003. This will make the template V2... if we don't need V3 functionality don't create it. 
  • Under "General" change the Template Display Name to ComputerForCompanyName; Leave the Template Name as is.
    • Note: Per several articles, Microsoft dictates that: "Important: You must set the certificate template’s attributes Template display name and Template name to the same value."  Because of this, I've omitted all spaces from the Computer certificate in this demonstration. 
  • Under "Superseded Templates" click "Add...", select "Computer" and click "OK".  
  • Under "Security" check "Enroll" and "Autoenroll" for any Machines you want to autoenroll. You will have to add the specific machines/groups if need be.
  • Click "OK"
  • Start the CA server (Temporarily to import cert) 
  • Back on the CA, under "Certificate Templates", right click in a blank space on the middle plane and select "New"->"Certificate Template To Issue"
  • Select your new Computer certificate template and click "OK"
  • Select "Computer" and press delete; click "Yes" to confirm

Edit GPO to Specify Certificate Template

At the desired level (be it Default Domain Policy or a policy that applies to servers only, etc.) edit the target GPO. This procedure will automatically set templates for Windows Vista and higher machines. For 2k3 & XP you'll need to open the certificates MMC and request a cert manually. Fortunately they will detect the issuing CA and process the request from template quickly.
  • Navigate to "Computer Configuration"->"Administrative Templates"->"Windows Components"->"Remote Desktop Services"->"Remote Desktop Session Host"->"Security"
  • Edit "Server Authentication Certificate Template" 
  • Enable the setting and type the template name (ComputerForCompanyName)
  • Click "OK" and close the GPO Object

Configure Default Domain Policy for Auto-enrollment

  • Open group policy management and edit the "Default Domain Policy"
  • Navigate to "Computer Configuration"->"Windows Settings"->"Security Settings"->"Public Key Policies" and modify "Certificate Services Client - Auto-Enrollment Properties"
  • Set "Configuration Model" to "Enabled" and check "Renew expired certificates..." and "Update certificates that use certificate template" and click "OK"
  • Navigate to "User Configuration"->"Windows Settings"->"Security Settings"->"Public Key Policies" and modify "Certificate Services Client - Auto-Enrollment Properties"
  • Set "Configuration Model" to "Enabled" and check "Renew expired certificates..." and "Update certificates that use certificate template". Enable "Expiration Notification" if you like and click "OK"
Now double check everything. If all elements look to be configured correctly feel free to start up the issuing CA and watch the requests come in.

Congrats! You have setup an enterprise class Certificate Authority, one of the most crucial security components of any enterprise infrastructure. You probably deserve a caffeinated beverage now.

Appendix A: When to use a CAPolicy.inf file  

Note: See Windows 2008r2 CAPolicy.inf syntax for implementation specifics

The %systemroot%\CAPolicy.inf file can be put into place on a target CA to further customize the creation of the certificate associated with that CA and SubCAs. If you do wish to customize the Root CA cert with any of the values listed below you'll need to do so prior to the creation of the CA cert; you can't add this stuff after the fact without re-issuing impacted certs.

In Windows 2003, it was necessary to always use a CAPolicy.inf file to ensure the root didn't have a CDP or AID associated with its cert, but that was fixed in Windows 2008. Due to this, it's no longer required to use the CAPolicy.inf file for a proper implementation, but there are some points where you may still want one. Here are the major ones:


PolicyStatementExtension: Use this to issue a legal notice, link to your use policy, or (if you need to) specify an OID. (You'll know if you do, generally applicable for external facing CAs issuing certs to third parties) Important note, if you plan on using certificate templates with issuance policies, you should take a look at this reported issue and follow the advice there. The easiest one of his steps to implement at PKI hierarchy creation time is #2. I'll copy his logic here in case the link goes bad:  (OID 2.5.29.32.0 is shared among all MS PKI deployments)

[Version]
Signature= “$Windows NT$”
[PolicyStatementExtension]
Policies = AllIssuancePolicy
Critical = FALSE
[AllIssuancePolicy]
OID = 2.5.29.32.0

CRLDistributionPoint: This is only applicable for the RootCA and shouldn't be used in most cases. Most platforms don't honor CRLs for a root CA anyhow. Doing so would question the concept of a trusted root.

AuthorityInformationAccess: Same as above; you would only have the AIA if you already have the cert, and if you already have the root cert you either do or don't trust it already.

EnhanceKeyUsageExtension: Use this to limit the usage of this CA cert (or otherwise) to specific uses. If omitted, the CA can be used to issue any type of cert. Obviously you'll leave this blank in most scenarios.

BasicConstraintsExtension: If you are relatively certain of your long term hierarchy you may want to use this on the root CA to limit how many SubCAs may fall under this root CA. If set to 1 for example it would make it impossible for a SubCA to issue a cert for a lower level SubCA. If you desire to implement this on a SubCA but not the root, configure the registry rather than using the CAPolicy.inf.

CrossCertificateDistributionPoints: Use to specify a CCDP location (URL) if your CA has been certified by a third party PKI. You may also specify the SyncDeltaTime (how often the URL is updated) here if desired. This value will only be used when you have negotiated a cross-PKI certification. Obviously you would know if you've done this. :)

RequestAttribues: Generally used on a SubCA; this allows you to change the default SubCA template that will be used. To use this value you will need to have duplicated the Sub CA template to a new name, apply the changes you desire, publish the template, and set the CertificateTemplate value to your duplicated template. The default template is version 1, so if you would like a version 2 or 3 certificate for your SubCA you'll need to use this field.

certsrv_server: Control the renewal settings for issuing CA certs. Usually not used because 1> The defaults are sufficient for most and 2> They can be changed "on the fly" (with services restart) in the registry on CAs for all certs issued thereafter. You may want to take a look at setting LoadDefautTemplates to 0 if you want to ensure your CA doesn't issue any certs until you publish your custom templates. You will see it recommended in some places to explicitly set the renewal validity here even if you do use the defaults, but I opt not to. Though it may be spelled out directly in this text file, I'm more a proponent of handing off the "Tyranny of the Default" (thanks Steve Gibson) to anyone down the line. Generally this will ease troubleshooting because assumptions of the default are true. I.E. only change it if the default values will not work for you.

Now to address the oft-asked question "Do I need a CAPolicy file?" directly: Can you get by without a CAPolicy.inf in Windows 2008r2? Yes. Should you? Only after you review and understand its role in the process can you safely ignore it. More often than not you will want to use it to tweak a setting or two.

More references for CAPolicy.inf:
Designing and Implementing a PKI Part 2
Windows Server 2008r2 CAPolicy.inf Syntax
MS Social: CAPolicy.inf questions
CAPolicy.inf syntax

Appendix B: Publish new CRLs from Offline CA

Every six months (in our implementation) you will need to publish the new certificate revocation list to keep everything running smoothly. To do so:
  • Fire up the offline Root CA & login as admin
  • Open & expand CA object in the CA management MMC
  • Right click "Revoked Certificates"->"All Tasks"->"Publish"
  • Navigate to "C:\Windows\System32\certsrv\CertEnroll"
  • The new CRL should be there; now you need to copy it to the appropriate path on the webserver and publish to AD. Copying to the webserver speaks for itself, AD info follows:
  • Copy the file to a domain joined computer & log on to that box with Domain Admin privs
  • Open a command prompt as administrator  
  • Execute the command:  certutil -dspublish "My Company Root CA.crl"
  • Done! Log off the PC and shut down the Offline CA for another six months.

Questions/Comments/Improvements please let me know. Thanks and good luck!