tag:blogger.com,1999:blog-54585896967908153362024-03-14T12:38:15.617-05:00itTobyBecause if I don't write it down I might forget it.Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.comBlogger74125tag:blogger.com,1999:blog-5458589696790815336.post-72302894630579101922017-06-23T21:02:00.000-05:002018-01-06T22:02:54.317-06:00Automating Service Principal Setup in Azure Active Directory <p>When automating tasks in Azure, you'll often need a service principal.
Setting these up using the UI feels like driving my car to the mailbox.
Let's automate this. <br>
<br>
</p>
<table class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"
cellspacing="0"
cellpadding="0"
align="center">
<tbody>
<tr>
<td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj85WfMuz8lax1ZFKO0w71mrqChZnlrMhmrz__esp4BOnmi3POI2z5heloZZRL23Gx9QBa3J97tKFGO1Xu9bgVC-pcbKVh4TFk3ypHHOT6cop7smSliK0ipZK2_2MwZwt9LqTpifMpS6bw/s1600/ed209CroppedMod.jpg"
imageanchor="1"
style="margin-left: 1em; margin-right: 1em;"><img
data-original-height="255"
data-original-width="692"
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj85WfMuz8lax1ZFKO0w71mrqChZnlrMhmrz__esp4BOnmi3POI2z5heloZZRL23Gx9QBa3J97tKFGO1Xu9bgVC-pcbKVh4TFk3ypHHOT6cop7smSliK0ipZK2_2MwZwt9LqTpifMpS6bw/s640/ed209CroppedMod.jpg"
height="235"
width="640"
border="0"></a></td>
</tr>
<tr>
<td class="tr-caption" style="text-align: center;">The final product
when we finish automating everything.</td>
</tr>
</tbody>
</table>
<br>
<h3>Terms Bingo</h3>
<br>
Before getting into the article let's get a couple basic terms out of the
way first:<br>
<ul>
<li><strong><a target="_blank" href="https://en.wikipedia.org/wiki/Active_Directory">Active
Directory</a> (AD)</strong>: Microsoft's on-premises solution for
managing users, computers, etc. introduced in Windows 2000. </li>
<li><strong><a target="_blank" href="https://docs.microsoft.com/en-us/azure/active-directory/active-directory-whatis">Azure
Active Directory</a> (AAD)</strong>: Microsoft's cloud solution for
managing users, applications, and more. Hosted on Microsoft's Azure
platform, this can integrate with on-premises Active Directory if
desired, but is not required. <strong><br>
</strong></li>
<li><strong>(AAD) Tenant</strong>: An instance of Azure Active Directory
for a customer is called a "tenant". Most customers will have one
tenant, but larger organizations may have multiple for varying reasons.
</li>
</ul>
<h3><strong>A Service Principal?</strong></h3>
<br>
In the Active Directory world, automated tasks are performed by <a target="_blank"
href="https://technet.microsoft.com/en-us/library/dn617203%28v=ws.11%29.aspx">service
accounts</a>, be they traditional dedicated accounts or (group) managed
service accounts. The key to securely performing these tasks is ensuring
that unique security principals are used for each task, ensuring they have
only the amount of access they need to perform the task in question, and
that they're monitored proactively. <br>
<br>
Azure Active Directory does not have the concept of service accounts, but
there is a functional equivalent: <a target="_blank" href="https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-application-objects">Service
Principals</a>. Service principals are comprised of: <br>
<ul>
<li><strong>Azure Active Directory Registered Application: </strong>Registering
an application in AAD is a way to then grant permissions (using a
Service Principal) to that application within Azure and/or Azure Active
Directory. This can be an application hosted in Azure, externally, or in
our case, an automation task of another nature. An AAD registered
application can also be used by other tenants (with a Service Principal
on their side), but as we're talking about automating our own tasks that
is outside the scope of this article. </li>
<li><strong>The Service Principal itself: </strong>The service principal
is an association with an AAD Application that allows for granting of
permissions within that tenant. In our case we'll be discussing AAD
registered apps and service principals in a 1:1 ratio. </li>
</ul>
<br>
<h3>Azure Entitlement Domains</h3>
<br>
To facilitate the <a target="_blank" href="https://en.wikipedia.org/wiki/Principle_of_least_privilege">principle
of least privilege</a>, we need to understand the <a target="_blank" href="https://docs.microsoft.com/en-us/azure/active-directory/role-based-access-control-configure">levels
of granularity</a> by which permissions are assigned in Azure. There are
three levels rights can be assigned at: <br>
<ul>
<li><strong>Subscription: </strong>One can grant a service principal,
user, or other object access at an entire subscription level. There are
very few cases where doing so would adhere to the principle of least
privilege, so don't do this unless you have no option. </li>
<li><strong>Resource Group: </strong>A resource group is a logical
grouping of Azure resources. Depending on how you split your resource
groups, this is likely a common place to assign privileges. </li>
<li><strong>Object: </strong>Privileges can be assigned on a per-object
basis as well, but managing security on a per-object basis is very
complex and usually only done when absolutely necessary from a security
perspective. </li>
</ul>
<br>
<table class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"
cellspacing="0"
cellpadding="0"
align="center">
<tbody>
<tr>
<td style="text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBM4mpJGnqq_4_bB8dMuWTFAYeNDbdN3mXeak7C95suvmZeS_ShCofIPEUXbmDW0kDqju2WueheetVTR7I8jUacFxaaAdV0CKAkfr87ZTqCVG0mQZ2ahvO2iOyDQ3X0DK7a0oOjNYM1do/s1600/PaulBlart_2.jpg"
imageanchor="1"
style="margin-left: 1em; margin-right: 1em;"><img
data-original-height="289"
data-original-width="667"
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBM4mpJGnqq_4_bB8dMuWTFAYeNDbdN3mXeak7C95suvmZeS_ShCofIPEUXbmDW0kDqju2WueheetVTR7I8jUacFxaaAdV0CKAkfr87ZTqCVG0mQZ2ahvO2iOyDQ3X0DK7a0oOjNYM1do/s400/PaulBlart_2.jpg"
height="172"
width="400"
border="0"></a></td>
</tr>
<tr>
<td class="tr-caption" style="text-align: center;">Sorry son, your
RBAC doesn't give you access to the keyvault.</td>
</tr>
</tbody>
</table>
<h3>Usage Examples</h3>
<br>
So when would we use a service principal? Here are a couple examples: <br>
<br>
<ul>
<li><strong>Webjobs that interact with other resources: </strong>If you
have <a target="_blank" href="https://docs.microsoft.com/en-us/azure/app-service-web/web-sites-create-web-jobs">webjobs</a>
that interact with other resources in your subscription you may choose
to use a service principal to access those resources. An excellent
example of this is the <a target="_blank" href="https://www.siteextensions.net/packages/letsencrypt/">Let's
Encrypt! web app extension</a>.<br>
</li>
<li><strong>Automated Tasks: </strong>Azure automation and/or external
jobs running against Azure can leverage service principals to
authenticate and perform their work. Code deployment platforms such as
OctopusDeploy and VSTS are perfect examples. </li>
</ul>
<br>
<h3>Let Us Do This</h3>
<h5>And by this I mean the point of the article.</h5>
<br>
To make quick work of this from an automation perspective, we'll make a
quick PowerShell function we can re-use in other scripts. The main
PowerShell cmdlets we'll be leveraging are: <br>
<br>
<ul>
<li> <a target="_blank" href="https://docs.microsoft.com/en-us/powershell/module/azurerm.resources/new-azurermadapplication">New-AzureRmADApplication</a></li>
<li><a target="_blank" href="https://docs.microsoft.com/en-us/powershell/module/azurerm.resources/new-azurermadserviceprincipal">New-AzureRMADServicePrincipal</a></li>
</ul>
<p>Just running these two commands is easy enough, but that's not all that
useful from an automation perspective where we want to automate multiple
operations that rely on the creation or existence of a service principal.
To that end, we'll make a function that can accept a desired service
principal name, check to see if it exists & create it if not. This
function will output an object with all the necessary information for
further use. We'll also generate a password if one isn't specified and
report status regarding if the service principal already existed or
not.We'll return everything in an object so our scripts can take
appropriate actions for all possible scenarios. </p>
<br>
<p><strong>Critical Note:</strong> The code below contains reference to a
function that is not included, so before copying/pasting please read at
least the "<strong>Note</strong>" sections in the code discussion below. </p>
<strong> Update 1/2018: </strong>AzureRM 5.0 cmdlets require a securestring
for New-AzureRmADApplication whereas it was not supported previously. <br>
<br>
<p><a target="_blank" href="https://www.youtube.com/watch?v=DqNG_SrSa4o">HERE
COMES ANOTHER SCRIPT!</a></p>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">################################################################################
# Register-AzureServicePrincipal
# Given the correct input, does one of the following:
# 1> checks for existence of application registration
# 2> checks to see if the app is registered as a service principal
# 3> if neither of those is true, creates the app and service principal
# > outputs an object with all details possible. If the app already exists the
# password will be null because we can't look it up.
# INPUT: servicePrincipalName, the displayname of the desired App/ServicePrincipal and the desired password.
# The password field is optional and if omitted a 30 character random password will be generated and returned.
# OUTPUT: an object containing the following NoteProperties
# > ClientID: the GUID representing the application ID
# > ServicePrincipalID: the GUID representing the Service Principal association
# > SPNNames: The service principal names of the SP
# > ServicePrincipalPassword: A securestring of the Application Password. NOTE: This will be NULL if the app is already registered in AD as we cannot retrieve it.
# > ServicePrincipalAlreadyExists: boolean to indicate if the sp already existed or not
# USAGE NOTES: Assumes already logged into Azure with proper permissions and that the desired subscription is selected.
Function Register-AzureServicePrincipal{
param(
# The name for the service principal. We won't make this mandatory to allow for manual entry mode with guidance. Obviously it needs to be specified for automation.
[string]$servicePrincipalName,
# the password if you choose to specify it, otherwise the script will generate one for you.
[string]$servicePrincipalPassword
)
# Set the regex for the input validation on the SPN
$SPNNamingStandard='^[--z]{5,40}$'
Write-Host "Provisioning AzureAD App/Service Principal"
Write-Warning "The account operating this script MUST have the role Subscription Admin or Owner in the desired subscription"
$ErrorActionPreference = "Stop" # Error handing is not yet sufficient; try/catch the stuff below!
if (!$servicePrincipalName){
do {
Write-Host "SPN naming standard is (in RegEx): $SPNNamingStandard"
$servicePrincipalName=Read-Host "Service Principal Name not specified on startup; Please enter desired name or type GUID and press enter for a guid based random name"
if ($servicePrincipalName -eq "GUID"){
$guid=([guid]::NewGuid()).toString()
$servicePrincipalName="SPN-$guid"
}
} until ($servicePrincipalName -match $SPNNamingStandard)
}
# handle command line specification of GUID
if ($servicePrincipalName -eq "GUID"){
$guid=([guid]::NewGuid()).toString()
$servicePrincipalName="SPN-$guid"
}
# set URL and IdentifierUris
$homePage = "http://" + $servicePrincipalName
$identifierUri = $homePage
Write-Host "Desired Service Principal Name is $servicePrincipalName `n"
# Now we need to determine if 1> the Application exists and 2> if it has been registered as a service principal. This will guide our execution through the end of the function.
$appExists=Get-AzureRmADApplication -DisplayNameStartWith $servicePrincipalName -ErrorAction SilentlyContinue
# check for SPN only if app exists. SPN can't exist without app so no reason to check if not.
if ($appExists){$spnExists=Get-AzureRmADServicePrincipal | Where-Object {$_.ApplicationId -eq $appExists.ApplicationId} -ErrorAction SilentlyContinue}
# we only need a password if the app hasn't been created yet.
if (!$appExists){
# Generate a password if needed
if (!$servicePrincipalPassword){
$servicePrincipalPassword=New-RandomPassword -passwordLength 40
}
# NOTE! We had a convertto-securestring here but as it turns out new-azurermadapplication doesn't take a securestring, only a string
# NOTEUPDATE! AzureRM 5.0 and higher requires a securestring (yay!) This has been updated but notes left here for reference.
$servicePrincipalPassword=ConvertTo-SecureString $servicePrincipalPassword -AsPlainText -Force
}
# we set this to NULL as a "valid" return as the appID already exists and we can't lookup the password from here
else {$servicePrincipalPassword=$null}
# Create the App if it wasn't already
if (!$appExists){
$azureADApplication=New-AzureRmADApplication -DisplayName $servicePrincipalName -HomePage $homePage -IdentifierUris $identifierUri -Password $servicePrincipalPassword
Write-Host "Azure AAD Application creation completed successfully"
}
# if it already exists we'll just redirect the variable
else{$azureADApplication=$appExists}
$appID=$azureADApplication.ApplicationId
# Create new SPN if needed
if (!$spnExists){
$spn=New-AzureRmADServicePrincipal -ApplicationId $appId
Write-Host "SPN creation completed successfully"
}
else{$spn=$spnExists}
$spnNames=$spn.ServicePrincipalNames
# Create object to store information.
$outputObject=New-Object -TypeName PSObject
$outputObject | Add-Member -MemberType NoteProperty -Name ServicePrincipalName -Value $servicePrincipalName
$outputObject | Add-Member -MemberType NoteProperty -Name ClientID -Value $appID
$outputObject | Add-Member -MemberType NoteProperty -Name ServicePrincipalID -Value $spn.Id
$outputObject | Add-Member -MemberType NoteProperty -Name SPNNames -Value $spnNames
$outputObject | Add-Member -MemberType NoteProperty -Name ServicePrincipalPassword -Value $servicePrincipalPassword
if ($appExists -and $spnexists){$outputObject | Add-Member -MemberType NoteProperty -Name ServicePrincipalAlreadyExists -Value $true}
else {$outputObject | Add-Member -MemberType NoteProperty -Name ServicePrincipalAlreadyExists -Value $false}
return $outputObject
}
################################################################################
</code></pre> <br>
<h4>Discussion</h4>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;"> param(
# The name for the service principal.
[string]$servicePrincipalName,
# the password if you choose to specify it, otherwise the script will generate one for you.
[string]$servicePrincipalPassword
)
</code></pre>
<p>This is where the input to the function is defined; as you'll see below
the password is optional, but the servicePrincipalName is mandatory. I
don't mark the parameter as mandatory in the parameter definition to
facilitate interactive use of this script. Feel free to change add <em>mandatory=$true
</em>if desired. </p>
<p>While it would be logical to use<em> [securestring]</em> for the
servicePrincipalPassword, the PowerShell cmdlet we're going to use
downstream only supports regular strings at this time. </p>
<p></p>
<h5>A quick note on interactive vs. non-interactive scripts</h5>
<p>While the goal of automation should be running tasks headless, thus fully
non-interactive, there are some scenarios where facilitating both
non-interactive and interactive running can be very useful. In some cases
when acting as a consultant I will allow for full non-interactive running
of automation scripts so the customer can walk through each option guided
once to understand the context of the options available to them. At the
end of script execution, I echo out what the equivalent command line would
be to the console if it were run entirely headless. This scenario does
require a bit more control flow logic, mainly using an additional
parameter to specify we're in a headless scenario and error out quickly
prior to execution when running in those scenarios. This also allows for
use of code snippets (mainly functions) on a day-to-day basis as well as
in dedicated automation framework. </p>
<br>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;"> # Set the regex for the input validation on the SPN
$SPNNamingStandard='^[--z]{5,40}$'
Write-Host "Provisioning AzureAD App/Service Principal"
Write-Warning "The account operating this script MUST have the role Subscription Admin or Owner in the desired subscription"
$ErrorActionPreference = "Stop" # Error handing is not yet sufficient, try/catch the stuff below!
if (!$servicePrincipalName){
do {
Write-Host ""
Write-Host "SPN naming standard is (in RegEx): $SPNNamingStandard"
$servicePrincipalName=Read-Host "Service Principal Name not specified on startup; Please enter desired name or type GUID and press enter for a guid based random name"
if ($servicePrincipalName -eq "GUID"){
$guid=([guid]::NewGuid()).toString()
$servicePrincipalName="SPN-$guid"
}
} until ($servicePrincipalName -match $SPNNamingStandard)
}
</code></pre>$SPNNamingStandard and the following logic is if you want to use a
RegEx to ensure interactive input is validated. Change this expression to
meet your needs or return the entire section if you don't want to facilitate
interactive running. Limited time bonus offer, <a target="_blank" href="https://regex101.com/">here's
a link</a> to my favorite regular expression evaluation site!<br>
<br>
<strong>Note</strong>: As the script warns, the credential used to create
the Application ID/Service Principal must have the role Subscription Admin
or Owner to perform the provisioning actions. <br>
<br>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;"> # handle command line specification of GUID
if ($servicePrincipalName -eq "GUID"){
$guid=([guid]::NewGuid()).toString()
$servicePrincipalName="SPN-$guid"
}
</code></pre>This little trick allows the specification of "GUID" (i.e. <em>Register-AzureServicePrincipal
-$servicePrincipalName GUID</em>) to tell our function to generate a GUID
for the name. At the end you see I'm pre-pending a "SPN-" to the GUID to
ensure the programmatically generated ApplicationIDs/SPNs stand out. <br>
<br>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;"> # set URL and IdentifierUris
$homePage = "http://" + $servicePrincipalName
$identifierUri = $homePage
Write-Host "Desired Service Principal Name is $servicePrincipalName `n"
</code></pre>
<p>Homepage and IdentifierURI settings for a service principal that we're
using in the aforementioned capacity don't matter, but they do need to be
set, so we base them on the SP name itself an move on. <br>
</p>
<strong> Update 1/2018: </strong>AzureRM 5.0 cmdlets require a securestring
for New-AzureRmADApplication whereas it was not supported previously. <br>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;"> # Now we need to determine if 1> the Application exisists and 2> if it has been registered as a service principal. This will guide our execution through the end of the function.
$appExists=Get-AzureRmADApplication -DisplayNameStartWith $servicePrincipalName -ErrorAction SilentlyContinue
# check for SPN only if app exists. SPN can't exist without app so no reason to check if not.
if ($appExists){$spnExists=Get-AzureRmADServicePrincipal | Where-Object {$_.ApplicationId -eq $appExists.ApplicationId} -ErrorAction SilentlyContinue}
# we only need a password if the app hasn't been created yet.
if (!$appExists){
# Generate a password if needed
if (!$servicePrincipalPassword){
$servicePrincipalPassword=New-RandomPassword -passwordLength 40
}
# NOTE! We had a convertto-securestring here but as it turns out new-azurermadapplication doesn't take a securestring, only a string
# NOTEUPDATE! AzureRM 5.0 and higher requires a securestring (yay!) This has been updated but notes left here for reference.
$servicePrincipalPassword=ConvertTo-SecureString $servicePrincipalPassword -AsPlainText -Force
}
# we set this to NULL as a "valid" return as the appID already exists and we can't lookup the password from here
else {$servicePrincipalPassword=$null}
</code></pre> This code allows us to insert this function into a workstream
regardless of if the Application ID and service principal already exist. If
they do, we get all the information we can but set the password to $null
since we can't look it up. As you'll see below we also add another
noteproperty to inform the caller explicitly that the AppID/service
principal already existed. <br>
<br>
<strong>Note</strong>: This script block contains reference to another
function that I have not provided, <em>New-RandomPassword</em>. You'll need
to provide your own function that generates a password and call it here or
specify the desired password when calling the function explicitly. Perhaps
I'll write another post in the future to cover generating a random password
in PowerShell. <br>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;"> # Create the App if it wasn't already
if (!$appExists){
$azureADApplication=New-AzureRmADApplication -DisplayName $servicePrincipalName -HomePage $homePage -IdentifierUris $identifierUri -Password $servicePrincipalPassword
Write-Host "Azure AAD Application creation completed successfully"
}
# if it already exists we'll just redirect the variable
else{$azureADApplication=$appExists}
$appID=$azureADApplication.ApplicationId
# Create new SPN if needed
if (!$spnExists){
$spn=New-AzureRmADServicePrincipal -ApplicationId $appId
Write-Host "SPN creation completed successfully"
}
else{$spn=$spnExists}
$spnNames=$spn.ServicePrincipalNames
</code></pre>Now we do the actual creation of the Application ID and service
principal if necessary. Notice that if they already existed we set the
downstream variables to relay to the user. This section could be improved
with additional error handling if desired. <br>
<br>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;"> # Create object to store information.
$outputObject=New-Object -TypeName PSObject
$outputObject | Add-Member -MemberType NoteProperty -Name ServicePrincipalName -Value $servicePrincipalName
$outputObject | Add-Member -MemberType NoteProperty -Name ClientID -Value $appID
$outputObject | Add-Member -MemberType NoteProperty -Name ServicePrincipalID -Value $spn.Id
$outputObject | Add-Member -MemberType NoteProperty -Name SPNNames -Value $spnNames
$outputObject | Add-Member -MemberType NoteProperty -Name ServicePrincipalPassword -Value $servicePrincipalPassword
if ($appExists -and $spnexists){$outputObject | Add-Member -MemberType NoteProperty -Name ServicePrincipalAlreadyExists -Value $true}
else {$outputObject | Add-Member -MemberType NoteProperty -Name ServicePrincipalAlreadyExists -Value $false}
return $outputObject
</code></pre>Now we'll create our output object. This gives us everything we
might want to know (and probably more) for our consuming processes. Here's
what our object looks like: <br>
<ul>
<li>$output.ServicePrincipalName=(Suprise!!!) The service principal name</li>
<li>$ouput.ClientID=The client ID (appID) is one of the critical pieces of
info for downstream applications. This is what you'll specify when
authenticating later (think of it as your user ID). </li>
<li>$output.ServicePrincipalID=The SP ID, though not used in any capacity
directly that I've seen yet short of programmatically referencing it
when deleting, etc. </li>
<li>$output.SPNNames=The reference names of the service principal. These
would be used by third party apps, but in most cases I'm addressing with
this article they'll go unused. </li>
<li>$output.ServicePrincipalPassword=Keep it secret! Keep it safe! This is
a clear text copy of the password associated with this service
principal. Obviously this is the other key piece of information you'll
need as a takeaway. The prudent next step would be to check this into a
Azure Keyvault or something similar, but that's for another article... </li>
<li>$output.ServicePrincipalAlreadyExists=<em>$true</em> or <em>$false</em>,
this is also critical for downstream processing. If $true, you'll know
that this is newly created and the password is contained in the object
meaning you'll need to scrape and store/use that accordingly. If $false
it means you can look up the ID by the name if needed, but you better
have the password stored somewhere else as we can't look it up now.
Either way you have two clear courses of action. While we could have
relied on the password being $null, I added this property to
definitively set it one way or the other to account for any unknown
circumstances due to upstream changes down the road. </li>
</ul>
<p><strong>Note</strong>: Make sure you both store the password for the
newly created service principal as you <strong>won't be able to retrieve
it</strong> in the future. Also, make sure your session or variables are
cleared after running as the password exists in clear text in
memory. </p>
<p><strong>Update/Note 2</strong>: By default, this password is only good <strong>for
one year</strong>, and will expire after that time making it impossible
to use the SPN. To manage the password on an existing object, you'll need
to use the <strong>Get/Remove/New-AzureAdApplicationPasswordCredential </strong>cmdlets
in the <strong>AzureAD</strong> module (not AzureRM).
<meta charset="utf-8">
</p>
<br>
<p></p>
<h3>Bringing it Home</h3>
<br>
Now that we've created the function, let's use it to create a principal and
give it access to a resource group: <br>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">$spnOutput=Register-AzureServicePrincipal -servicePrincipalName <myPrincipalName> -servicePrincipalPassword <myPassword>
New-AzureRmRoleAssignment -ObjectId ($spnOutput.ServicePrincipalID.toString()) -RoleDefinitionName Contributor -ResourceGroupName <myResourceGroup>
</code></pre>This will use our function to create a service principal and give
it contributor level access to <myResourceGroup>. If you have multiple
environments in your subscription you should create a principal for each and
restrict access to resource group(s) associated with each environment. I
also recommend splitting production into a separate subscription; the entire
reasons behind that are a story for another article...<br>
<br>
<h3>Did it Work? <br>
</h3>
<br>
If you would like to manually check your work, you can find it in the <a target="_blank"
href="https://portal.azure.com/">Azure
portal</a> by navigating to the hamburger menu -> "More Services" ->
"Azure Active Directory" -> "App Registrations". <br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhocv7rCNxZehK8FazFQQX8NLH0iKJ5Jh7O47u3aUPi0LXH6yWi3dQQGDAuDteCCJlMWuf6Zg2gzjgyikLh0wpTbsfnMnTYIVHTD4euL1efDI9hoo_C3-uM7rQB4Hwjwcs2YdH_MTC6Ews/s1600/Screen+Shot+2017-06-23+at+4.19.15+PM.png"
imageanchor="1"
style="margin-left: 1em; margin-right: 1em;"><img
data-original-height="740"
data-original-width="1248"
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhocv7rCNxZehK8FazFQQX8NLH0iKJ5Jh7O47u3aUPi0LXH6yWi3dQQGDAuDteCCJlMWuf6Zg2gzjgyikLh0wpTbsfnMnTYIVHTD4euL1efDI9hoo_C3-uM7rQB4Hwjwcs2YdH_MTC6Ews/s400/Screen+Shot+2017-06-23+at+4.19.15+PM.png"
height="235"
width="400"
border="0"></a></div>
<div class="separator" style="clear: both; text-align: center;"> </div>
<br>
<br>
There you should see your newly created app. <br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhdsixo90o6YQk7Bk2UB5Kz39lw747IZVmdvfFmi70ZHdAzepu5OOrjeoPfi_4tQhheCj5Rsvg-0Fc-u4nmtnSqYhyphenhyphen10gANtZzxpxhEANwAOCcQ78uN30PovyjfKCcIh0SZ_y849ilZbzY/s1600/Screen+Shot+2017-06-23+at+4.23.15+PM.png"
imageanchor="1"
style="margin-left: 1em; margin-right: 1em;"><img
data-original-height="786"
data-original-width="1470"
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhdsixo90o6YQk7Bk2UB5Kz39lw747IZVmdvfFmi70ZHdAzepu5OOrjeoPfi_4tQhheCj5Rsvg-0Fc-u4nmtnSqYhyphenhyphen10gANtZzxpxhEANwAOCcQ78uN30PovyjfKCcIh0SZ_y849ilZbzY/s400/Screen+Shot+2017-06-23+at+4.23.15+PM.png"
height="213"
width="400"
border="0"></a></div>
<br>
You can also check the role based access controls on your resource group or
whatever object you applied the permissions to. <br>
<br>
<table class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"
cellspacing="0"
cellpadding="0"
align="center">
<tbody>
<tr>
<td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDhEDphGAsB0zLf3P4qzMgc_6lDf2tbTDK44TckM3QS_sjrV3iJglBCFMWgwQk-05oQe5j_ZvgfXmGzjlIn7m8x7AQnayI67J7MEGEvjzcMZ7NuDWev5Fd0XsWq3g6hkzsvTnuoVK7YxQ/s1600/Screen+Shot+2017-06-23+at+4.35.06+PM.png"
imageanchor="1"
style="margin-left: auto; margin-right: auto;"><img
data-original-height="904"
data-original-width="1328"
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDhEDphGAsB0zLf3P4qzMgc_6lDf2tbTDK44TckM3QS_sjrV3iJglBCFMWgwQk-05oQe5j_ZvgfXmGzjlIn7m8x7AQnayI67J7MEGEvjzcMZ7NuDWev5Fd0XsWq3g6hkzsvTnuoVK7YxQ/s400/Screen+Shot+2017-06-23+at+4.35.06+PM.png"
height="271"
width="400"
border="0"></a></td>
</tr>
<tr>
<td class="tr-caption" style="text-align: center;">Only the chosen
shall access Nachos Deathstar.</td>
</tr>
</tbody>
</table>
<br>
<br>
<h3>In Conclusion</h3>
<br>
The Azure model of auth/auth management is sound, but adherence to long
standing security design principles requires a bit of effort. Hopefully this
article will assist you in doing so. Please leave any
comments/criticism/coffee donations below. <br>
<p></p>
<br>
<h3>References</h3>
<p><a target="_blank" href="https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-application-objects">Microsoft:
Application and service principal objects in Azure Active Directory
(Great article on the differences between an AppID vs a service
principal)</a></p>
<p><a target="_blank" href="https://docs.microsoft.com/en-us/azure/active-directory/role-based-access-control-configure">Microsoft:
Role based access in Azure</a></p>Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com3tag:blogger.com,1999:blog-5458589696790815336.post-18131810336504714172017-02-05T11:39:00.000-06:002017-02-05T11:39:00.053-06:00HyperV Live Migration Changes in Windows Server 2016<p> </p> <p>After upgrading my lab servers to Windows Server 2016, I had an “interesting” (ask a Minnesotan what that means) weekend troubleshooting Hyper-V Live Migration, finally finding that there has been a major change in the way virtual machine migration works, and a couple gotchas. In an effort to save others the same trouble, I’ll discuss them here. </p> <table class="tr-caption-container" style="text-align: center; margin-left: auto; margin-right: auto" cellspacing="0" cellpadding="0" align="center"> <tbody> <tr> <td style="text-align: center"><a style="margin-left: auto; margin-right: auto" href="https://www.flickr.com/photos/141305295@N05/28181996014/" imageanchor="1"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgyJGVldOWpLct0AYi0T56nVlI8f7bPjcVjiNsDYjvR_cHrkyb2-yQptydvzkid1sdO3bGApT1Lwn5tpbdR6HMGYP7QNkQmSOBZmXQdSfYizPsYn0rhoGvYoCzXAJXrU9B0VEl4AVuF6ms/s400/sothere.jpg" width="400" height="265"></a></td></tr> <tr> <td class="tr-caption" style="text-align: center">Image From <a href="https://www.flickr.com/photos/141305295@N05/28181996014/">Polarstein on Flickr</a></td></tr></tbody></table> <div class="separator" style="text-align: center; clear: both"></div><br> <h3>Kerberos Constrained Delegation, 0x8009030E, and You(r Network Service Account)</h3> <p>“<em>No credentials are available in the security package</em>”, Event ID 20306. Under previous circumstances, this would have indicated that you didn’t have constrained delegation set up correctly as outlined in numerous other articles on the internet, but due to an underlying change the correct configuration is now different. </p> <p>Previously, failover would be set up as outlined in articles such as <a href="https://blogs.technet.microsoft.com/matthts/2012/06/10/configuring-kerberos-constrained-delegation-for-hyper-v-management/">this</a>, with each HyperV host set up to allow constrained delegation over the Kerberos protocol only. </p> <p> <div class="separator" style="text-align: center; clear: both"><a style="margin-left: 1em; margin-right: 1em" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiuDle3OluLzIBLpm0BjmTLsBRpYo1u-z846fG2tuhM9bbdU4JKl-dkrl59WSHyqRI34Pc5bqlCMKfIpN0I5RiGz6fDoXvnZ3Yt0Vsq3XizO0aEK5qetUqGTDn9TyzlGC96AzlRDZude1g/s1600/2017-02-04+23_56_05-Active+Directory+Users+and+Computers.png" imageanchor="1"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiuDle3OluLzIBLpm0BjmTLsBRpYo1u-z846fG2tuhM9bbdU4JKl-dkrl59WSHyqRI34Pc5bqlCMKfIpN0I5RiGz6fDoXvnZ3Yt0Vsq3XizO0aEK5qetUqGTDn9TyzlGC96AzlRDZude1g/s400/2017-02-04+23_56_05-Active+Directory+Users+and+Computers.png" width="343" height="400"></a></div><br> <p>Starting in server 2016, the delegation must be set up to allow delegation over <strong>any </strong>protocol as displayed here: </p> <p> <div class="separator" style="text-align: center; clear: both"><a style="margin-left: 1em; margin-right: 1em" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVnrQwbhaVSkBQuG8dfeHBiqfjeC3b52VWfLOhC_5yse7BzuGHbFj97ae6XBggBo9i_UCF9k3Ni_LsgDBoXX-1x9mBdHCz78V-9F1hmbxGX-lUJTiM3sJ90nTmscoCIA2kbD2GTVNugu0/s1600/2017-02-04+23_56_32-KERMIT+Properties.png" imageanchor="1"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVnrQwbhaVSkBQuG8dfeHBiqfjeC3b52VWfLOhC_5yse7BzuGHbFj97ae6XBggBo9i_UCF9k3Ni_LsgDBoXX-1x9mBdHCz78V-9F1hmbxGX-lUJTiM3sJ90nTmscoCIA2kbD2GTVNugu0/s400/2017-02-04+23_56_32-KERMIT+Properties.png" width="343" height="400"></a></div><br> <p>The reason for this is that 2016 has changed the <a href="https://msdn.microsoft.com/en-us/library/cc136992(v=vs.85).aspx">WMI provider</a> used to a <a href="https://msdn.microsoft.com/en-us/library/hh850319(v=vs.85).aspx">new version</a>, which relies on <a href="https://msdn.microsoft.com/en-us/library/aa384426(v=vs.85).aspx">WinRM</a> to execute remote procedures rather than DCOM. WinRM, running as the Network Service, cannot access the Kerberos service ticket obtained to perform the action. By allowing any protocol, a “<a href="https://blogs.msdn.microsoft.com/winsdk/2015/08/28/logon-as-a-user-without-a-password/">S4U</a>” logon is sufficient to authenticate the request. While this setting is somewhat less secure, the point is made by the Team PM (published a few days ago, link below) that sensitive (privileged) accounts in any domain should have the “<a href="https://blogs.technet.microsoft.com/poshchap/2015/05/01/security-focus-analysing-account-is-sensitive-and-cannot-be-delegated-for-privileged-accounts/">Account is sensitive and cannot be delegated</a>” flag enabled to mitigate delegation risk.</p> <h3>NIC Teaming, 0x8007274C/0x80072741, and You(r Service Startup Problem)</h3> <p>This may impact 2012/R2 as well, though for some reason it only bit me on 2016. If using <a href="https://technet.microsoft.com/en-us/windows-server-docs/networking/technologies/nic-teaming/nic-teaming">NIC teaming</a> on your host for your failover network, the interface may not be available when the Virtual Machine Management Service (VMMS) attempts to start on bootup. This condition will result in the service not opening the port (6600) on the server, which makes it impossible to failover virtual machines. To fix this, change the service startup type from “<em>Automatic</em>” to “<em>Automatic(Delayed Start)</em>”. With PowerShell as our weapon of choice (hey nano server!) this is a two-step process: </p><pre style="font-size: 12px; overflow: auto; border-top: #cccccc 1px dashed; height: auto; font-family: arial; border-right: #cccccc 1px dashed; width: 99%; background: #f0f0f0; border-bottom: #cccccc 1px dashed; color: black; padding-bottom: 0px; text-align: left; padding-top: 0px; padding-left: 0px; border-left: #cccccc 1px dashed; line-height: 20px; padding-right: 0px"><code style="word-wrap: normal; color: black">Set-Service –Name vmms –StartupType "Automatic"
Set-ItemProperty -Path "Registry::HKLM\System\CurrentControlSet\Services\vmms" -Name "DelayedAutoStart" -Value 1 -Type DWORD
</code></pre>
<p>The service type should already be automatic, but we’ll re-assert that here to be sure. This will only delay service (and thus VM) startup by a small bit, but ensure that the adapter is available when it does. </p>
<h3>EventID 21024, Failed at Migration Source, and You(r Crazy, Still Unexplained Error)</h3>
<p>This is an odd one I can’t fully explain, but I’m including it in the hopes it may save others some time. On 2 of the 3 hosts, I had the following error preventing live migration after full 2016 setup: </p>
<p><em>Virtual machine migration operation for 'VMNAME' failed at migration source 'VMHOST'. (Virtual machine ID GUID-GOES-HERE)</em></p>
<p>This error message was not accompanied by any supporting information whatsoever. After numerous network captures and log combing, I found evidence of something slightly off with domain membership. In both cases the host was able to process group policy for the computer object, but never for any logged on users. This led me to attempt <strong>leaving and re-joining the domain</strong>, which in all cases remediated the problem. Note that when doing so you will need to delete the computer account prior to re-joining, then set up the constrained delegation as outlined above for each host again. </p>
<p>I wish I had more information about the root cause of this issue, but with it fixed I’m moving on. </p>
<h3>In Closing</h3>
<p>The upgrades to my lab didn’t go as smoothly as I would like, but I’m glad to have these issues out of the way to make for smoother efforts with production efforts. Hopefully this information will help you as well!</p>
<h3>Additional References</h3>
<p><a href="https://blogs.technet.microsoft.com/virtualization/2017/02/01/live-migration-via-constrained-delegation-with-kerberos-in-windows-server-2016/">Microsoft Virtualization Blog: Live Migration via Constrained Delegation with Kerberos in Windows Server 2016</a></p>
<p><a href="https://blogs.technet.microsoft.com/roplatforms/2012/10/16/shared-nothing-migration-fails-0x8007274c/">Microsoft GTCS Romania EPS: Shared Nothing Migration Fails</a></p>
<p><a href="https://blogs.msdn.microsoft.com/canberrapfe/2012/01/01/kerberos-troubleshooting/">Canberra PFE Team Blog: Kerberos Troubleshooting</a></p>
<p><a href="https://www.youtube.com/watch?v=SkgTxQm9DWM">Nyan Cat: 10 Hours 4k UHD For Endless Kerberos Packet Caps and Analysis!</a></p>Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com2tag:blogger.com,1999:blog-5458589696790815336.post-41698755587642633562016-10-27T14:14:00.000-05:002018-01-09T12:15:13.403-06:00Tunnel to the Cloud: Azure Site to Site IPsec Connection <p>Do you have multi-continent datacenters with gobs of bandwidth, IOps, and
processing capability? No? I can help get you part of the way there... a
network presence in one. <br>
<br>
</p>
<table class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"
cellspacing="0"
cellpadding="0"
align="center">
<tbody>
<tr>
<td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjaQyca3bcqGEO_Qbhk1Y8ndZpAuwj9Qvyrt7JAgbk_pSl48JzenxLkluXkb0igmPPHPh6xpddw0Wcuw2nIUmOTaXTxWP_CT2QX4DQC3e8fkNVTPjoLbWrT1MhDyiLRX0rA2thIJsegPgE/s1600/tunnel.jpg"
imageanchor="1"
style="margin-left: auto; margin-right: auto;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjaQyca3bcqGEO_Qbhk1Y8ndZpAuwj9Qvyrt7JAgbk_pSl48JzenxLkluXkb0igmPPHPh6xpddw0Wcuw2nIUmOTaXTxWP_CT2QX4DQC3e8fkNVTPjoLbWrT1MhDyiLRX0rA2thIJsegPgE/s640/tunnel.jpg"
height="248"
width="640"
border="0"></a></td>
</tr>
<tr>
<td class="tr-caption" style="text-align: center;">A tunnel.</td>
</tr>
</tbody>
</table>
<br>
<h3>A Site to Site Connection?</h3>
<br>
It's easier to think of this as an extension to your network into another
datacenter over the internet. Using <a target="_blank" href="https://en.wikipedia.org/wiki/IPsec">IPsec</a>
we can provide a relatively (comments at the end) secure, direct connection
between on on-premises datacenter and <a target="_blank" href="https://portal.azure.com">Azure</a>
hosted resources by encrypting the traffic that flows between the two.
What do I mean by: <br>
<ul>
<li><strong>Secure: </strong>IPsec tunnels all your traffic so it is
encrypted over the internet; in reality, this is really "more secure"
rather than definitively secure, as the effective security depends
highly on implementation specifics. </li>
<li><strong>Direct: </strong>Your router (played by pfSense in this case)
will recognize the Azure site as another routable network within the
boundaries of your own, enabling you to talk to Azure resources as if
they were in your own datacenter. </li>
</ul>
<p>While I'll be using pfSense for the initiator side as it exposes the
options in the most clear way I've found, this article will also be<strong>
useful for non-pfSense </strong>devices since we discuss the details of
the IPsec tunnel; the information here should be applicable to any IPsec
solution. <strong>Update 1/2017</strong>: I've personally tested on
various Cisco, Sonicwall, and pFsense equipment, and Microsoft has added
some great documentation about overall device support <a href="https://docs.microsoft.com/en-us/azure/vpn-gateway/vpn-gateway-about-vpn-devices">here</a>.
</p>
<p>Note: this works for Amazon Web Services (<a target="_blank" href="https://aws.amazon.com/">AWS</a>)
as well but is slightly more complex. Fortunately pfSense includes a <a target="_blank"
href="https://forum.pfsense.org/index.php?topic=81113.0">wizard</a>
that works, but takes a lot of the fun out of it as it strips you of
understanding how it works. In addition the wizard is necessary because of
how Amazon does <a target="_blank" href="https://en.wikipedia.org/wiki/Amazon_Virtual_Private_Cloud">VPC</a>
routing, whereas Azure is a bit more straightforward.</p>
With that, let's get to it! <br>
<h3>Pre-Requisites</h3>
<ul>
<li><strong>pfSense firewall(s)</strong>: The steps in this article were
performed on a pair of HA SG-4860 firewalls running pfSense
2.32p1.</li>
<li><strong>Microsoft Azure account</strong> with adequate permissions:
We'll be performing our actions using the "new" <a href="https://portal.azure.com">portal</a>
based on <strong>Azure Resource Manager</strong> (<a href="http://rickrainey.com/2016/01/19/an-introduction-to-the-azure-resource-manager-arm/">ARM</a>
or AzureRM). </li>
<li><strong>AzureRM PowerShell Cmdlets installed</strong>: On Win10/Server
2016 this can be accomplished with <em>Install-Module AzureRM; </em>for
more info see <a href="https://azure.microsoft.com/en-us/documentation/articles/powershell-install-configure/">this
post</a>.</li>
</ul>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjtX0h_YQTYXfvi85Mo3bhS-QHa9ztD1CGboswHqexBIn0LP9ffkcuoLGTEUiU3TNa7bQOTBF_kaoesKKAjma-_FPcg4X_8hAzbFxWC9H3Tqx7qsNQZLp1FV2OPpJAwNBERtwCsjq8Cm5g/s1600/2016-10-03+13_11_43-Administrator_+Windows+PowerShell.png"
imageanchor="1"
style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjtX0h_YQTYXfvi85Mo3bhS-QHa9ztD1CGboswHqexBIn0LP9ffkcuoLGTEUiU3TNa7bQOTBF_kaoesKKAjma-_FPcg4X_8hAzbFxWC9H3Tqx7qsNQZLp1FV2OPpJAwNBERtwCsjq8Cm5g/s640/2016-10-03+13_11_43-Administrator_+Windows+PowerShell.png"
height="187"
width="640"
border="0"></a></div>
<br>
<h3>Configure Azure IPSec Endpoint</h3>
Before we set up and initiate the connection from pfSense, we need to set up
our endpoint in Azure. To do so, we'll create the following objects: <br>
<ul>
<li>A Resource Group</li>
<li>A Public IP Address</li>
<li>A Virtual Network</li>
<li>A Gateway Subnet</li>
<li>A Virtual Network Gateway</li>
<li>A Local Network Gateway with a Connection</li>
</ul>
<h4>Resource Group</h4>
A <em>Resource Group</em> is a logical grouping of Azure <em>Resources</em>.
This logical group allows for easy organization and clearer billing reports.
We won't get too much into concepts and naming standards here other than to
say groups should be logically tied with similar lifecycle expectancies and
you should be <strong>consistent</strong>. For more information, see <a target="_blank"
href="https://azure.microsoft.com/en-us/documentation/articles/resource-group-overview/#resource-groups">this
Azure article</a>. <br>
<br>
<strong>Note</strong>: We'll be doing most of our steps in the web portal,
but this whole process is much more efficient with PowerShell. <br>
<br>
Let's go!<br>
<ol>
<li>Open and log into the ARM Azure Portal at <a target="_blank" href="https://portal.azure.com">portal.azure.com</a>;
ensure you're working with the <a target="_blank" href="https://channel9.msdn.com/Events/Ignite/2015/BRK2454">subscription</a>
you intend to use. </li>
<li>Navigate to "<em>Resource Groups</em>".</li>
<li>Click "<em>Add</em>" and type a name for your resource group, select
the subscription, and resource group location. Note: The resource group
location has no bearing on where you'll be connecting to as it's just a
location the metadata is stored. </li>
<li>Click "<em>Save</em>".</li>
</ol>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglmpARoV2lakHTSxtTyQe8Uj-DXFuFnvJBwnT7bwg8d7towlpL3tS8CKgaAlG7MOyvg_DA7e3Wt2M4jp6uBWj-RIqKMu2IH_uoUCkvT_iO4WxSga3NIglTjf1BS3ZXlmW11-rznIAoFCI/s1600/Screen+Shot+2016-10-08+at+10.22.54+PM.png"
imageanchor="1"
style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglmpARoV2lakHTSxtTyQe8Uj-DXFuFnvJBwnT7bwg8d7towlpL3tS8CKgaAlG7MOyvg_DA7e3Wt2M4jp6uBWj-RIqKMu2IH_uoUCkvT_iO4WxSga3NIglTjf1BS3ZXlmW11-rznIAoFCI/s320/Screen+Shot+2016-10-08+at+10.22.54+PM.png"
height="206"
width="320"
border="0"></a></div>
<br>
<br>
<ol>
</ol>
<h4>Public IP Address</h4>
While we could do the IP at the time we make the Virtual Network Gateway,
we'll take care of it now to ensure it's provisioned prior to getting to
that step and to discuss the IP details. <br>
<ol>
<li>Navigate to "<em>Public IP addresses".</em></li>
<li>Click "<em>Add</em>" and populate the following: </li>
<ul>
<li><em>Name</em>: Select a name leveraging consistent naming standards.
</li>
<li><em>IP address</em> <em>assignment</em>: Select "<em>Dynamic</em>".
I know what you're saying.. you're saying "but Toby, I'm not saying
anything", and what I'm saying is it seems this should be static.
Unfortunately if we make a static IP we'll be greeted later with the
following: <br>
<br>
<table class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"
cellspacing="0"
cellpadding="0"
align="center">
<tbody>
<tr>
<td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqiV_t9-m86yBYqENck5pkB_9YW2GMRmshlGdlE_CnTcQd7llGnZYKAAmzsh3TQ_92Yq-GecBzDYVi8vSQfCdxdcgp4Eycm5WdBCAw_txK7Y1RJvLQudSvMbOxRxBhPHEK4wmIU2jUuOM/s1600/2016-10-25+21_45_56-Window.png"
imageanchor="1"
style="margin-left: auto; margin-right: auto;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqiV_t9-m86yBYqENck5pkB_9YW2GMRmshlGdlE_CnTcQd7llGnZYKAAmzsh3TQ_92Yq-GecBzDYVi8vSQfCdxdcgp4Eycm5WdBCAw_txK7Y1RJvLQudSvMbOxRxBhPHEK4wmIU2jUuOM/s400/2016-10-25+21_45_56-Window.png"
height="93"
width="400"
border="0"></a></td>
</tr>
<tr>
<td class="tr-caption" style="text-align: center;">Why Azure,
WHY?!</td>
</tr>
</tbody>
</table>
<br>
Now I've been running a tunnel straight for almost a month thus far
and my dynamic IP has not shifted on me; I suspect it will behave the
same as IPs for other resources and stay static so long as it is used.
If this does change, you'll need to change the info in the Phase 1 and
2 setup of the tunnel on the pfSense side as outlined below. For the
record, as of the writing of this article the pricing of IPs in Azure
is a bit odd; dynamic IPs and static IPs beyond the first 5 in any
region are charged the same (pretty trivial), while the first 5 static
in a region are free. See <a target="_blank" href="https://azure.microsoft.com/en-us/pricing/details/ip-addresses/">here</a>
for more info.</li>
<li><em>Idle Timeout</em>: The default of 4 minutes should be fine here.
</li>
<li><em>DNS Name Label</em>: Optionally, specify a DNS alias here,
though we will not reference it again in this guide as I'm not
addressing DNS issues associated with IPsec at this time. </li>
<li><em>Subscription</em>: Select your subscription.</li>
<li><em>Resource Group</em>: Click the resource group we created in the
last step. </li>
<li><em>Location</em>: The IP is our IPsec target, so select a location
close to your local network connection. The <a target="_blank" href="http://azurespeedtest.azurewebsites.net/">Azure
Speed Test</a> comes in quite handy here.</li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9AXW13Q7XcLRrqX9_4St7uhUY7KJ0Johp9vnFgQ2_OHBLIM47hmmMO_VNmfkEXTCraJSXbrrULKHrqhkQJZ23ijE5WwE2aTZmxcQPLlrz95BjGd7e2D4zmFafQ5SOsIcdFhovuHT9Q78/s1600/2016-10-25+21_48_47-Jump+List+for+File+Explorer.png"
imageanchor="1"
style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9AXW13Q7XcLRrqX9_4St7uhUY7KJ0Johp9vnFgQ2_OHBLIM47hmmMO_VNmfkEXTCraJSXbrrULKHrqhkQJZ23ijE5WwE2aTZmxcQPLlrz95BjGd7e2D4zmFafQ5SOsIcdFhovuHT9Q78/s640/2016-10-25+21_48_47-Jump+List+for+File+Explorer.png"
height="640"
width="508"
border="0"></a></div>
<br>
</ul>
<li>Click "<em>Create</em>". This provisioning will take a few minutes
minutes as Azure re-arranges its <a target="_blank" href="https://en.wikipedia.org/wiki/Software-defined_networking">SDN</a>
infrastructure to give you an IP. </li>
<br>
<table class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"
cellspacing="0"
cellpadding="0"
align="center">
<tbody>
<tr>
<td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhultiR1KJXLhyphenhyphencHTI-h_BWBvZaIlTrOjJ08zg-4c_PnPP7Gx1dWZO9Yvipry0WTzfLegC04gvJ4V8AFhQaTMx34YJk395-n2ZwQ_Ymq8TGmvHlB4B4Uok4vplh_b-i6Sb1_JyfGjGd1Es/s1600/Screen+Shot+2016-10-25+at+9.29.23+AM.png"
imageanchor="1"
style="margin-left: auto; margin-right: auto;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhultiR1KJXLhyphenhyphencHTI-h_BWBvZaIlTrOjJ08zg-4c_PnPP7Gx1dWZO9Yvipry0WTzfLegC04gvJ4V8AFhQaTMx34YJk395-n2ZwQ_Ymq8TGmvHlB4B4Uok4vplh_b-i6Sb1_JyfGjGd1Es/s400/Screen+Shot+2016-10-25+at+9.29.23+AM.png"
height="67"
width="400"
border="0"></a></td>
</tr>
<tr>
<td class="tr-caption" style="text-align: center;">What a successful
deployment looks like!</td>
</tr>
</tbody>
</table>
<br>
</ol>
<h4>Virtual Network/Subnet/Gateway Subnet</h4>
A "<em><a href="https://azure.microsoft.com/en-us/documentation/articles/virtual-networks-overview/">Virtual
Network</a></em>" is a network space within Azure that you can carve up
and protect (firewall) to suit your needs. We're required to make one
subnet, and we'll create our "gateway subnet" (landing point) as well. If
this is your first foray into virtual networks on Azure you may want to take
a step back and <a target="_blank" href="https://azure.microsoft.com/en-us/documentation/articles/virtual-network-vnet-plan-design-arm/">consider
your design</a> before proceeding. Oh, you're back already? Let's go. <br>
<ol>
<li>Navigate to "<em>Virtual Networks</em>". </li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4YHFCZtWMMV0V9Cl6sUyH_L5V6hupjOHFhqNOCuwhpV0x7qGOVGy-zvfjNXXVP7nSCYZNX-MzAsTszQ2eiEt6rATAoXykMhuRDJ3xwl5wfS9MQPzJ0IVR4Ikuzwfnst2_Wh1Iub2Z9JQ/s1600/Screen+Shot+2016-10-08+at+10.25.29+PM.png"
imageanchor="1"
style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4YHFCZtWMMV0V9Cl6sUyH_L5V6hupjOHFhqNOCuwhpV0x7qGOVGy-zvfjNXXVP7nSCYZNX-MzAsTszQ2eiEt6rATAoXykMhuRDJ3xwl5wfS9MQPzJ0IVR4Ikuzwfnst2_Wh1Iub2Z9JQ/s320/Screen+Shot+2016-10-08+at+10.25.29+PM.png"
height="80"
width="320"
border="0"></a></div>
<br>
<li>Click "<em>Add</em>" and supply the following: </li>
<ol>
</ol>
<ul>
<li><em>Name</em>: Type a name for your Virtual Network; you should
follow the naming standards as discussed above.</li>
<li><em>Address Space</em>: This is the overall space for your logical
network within Azure. You can create more granular subnets within this
space at any time, so erring on the side of a large subnet would be
wise. If you're unsure, use 10.1.0.0/16.</li>
<li><em>Subnet Name</em>: You're required to create one subnet within
your virtual network off the bat. You need to name it here and ensure
you use a consistent and meaningful naming standard. </li>
<li><em>Subnet Address Range</em>: Specify a subnet range within your
virtual network. This won't be used by our IPsec connection directly,
but we will use it later as a target for testing. If unsure, use
10.1.10.0/24. </li>
<li><em>Subscription</em>: Select your desired subscription. </li>
<li><em>Resource Group</em>: Select "Use Existing" and select the
resource group we created earlier. </li>
<li><em>Location</em>: Select the same location used for the IP
above. </li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDpY3CrV6Tcq9-sEIFkzXwNAAwoDiqPeOgltym4WZ4-DDNYP9M3YXULJmpN71bg4oER06Ux65skAgEosmBSLw4XQ8saREWAfyeYX7lY4PtX4Eb3xCd4OBZM5-yCRBp8gFHUjLZ1JpigNo/s1600/Screen+Shot+2016-10-08+at+10.49.33+PM.png"
imageanchor="1"
style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDpY3CrV6Tcq9-sEIFkzXwNAAwoDiqPeOgltym4WZ4-DDNYP9M3YXULJmpN71bg4oER06Ux65skAgEosmBSLw4XQ8saREWAfyeYX7lY4PtX4Eb3xCd4OBZM5-yCRBp8gFHUjLZ1JpigNo/s400/Screen+Shot+2016-10-08+at+10.49.33+PM.png"
height="400"
width="272"
border="0"></a></div>
<br>
</ul>
<ol>
</ol>
<li>After the Virtual Network has been created (use the refresh key if
necessary), click it to navigate to the next pane, and then click "<em>Subnets</em>".
</li><br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqW0ohnn_3nKhjGaJjE_FdaQlO3qxmABq3Hhz0EL4SDV_vlqV_5amqGz3g2efiISJYcwquXAAX_n9LyyEEBIIWY_zFFdggPL2hBsWPBNclZFKwwH8tNCoExG661hai57d2FDeEEeZQTDE/s1600/Screen+Shot+2016-10-08+at+10.50.40+PM.png"
imageanchor="1"
style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqW0ohnn_3nKhjGaJjE_FdaQlO3qxmABq3Hhz0EL4SDV_vlqV_5amqGz3g2efiISJYcwquXAAX_n9LyyEEBIIWY_zFFdggPL2hBsWPBNclZFKwwH8tNCoExG661hai57d2FDeEEeZQTDE/s640/Screen+Shot+2016-10-08+at+10.50.40+PM.png"
height="339"
width="640"
border="0"></a></div>
<br>
<li>On the next pane, click "<em>+ Gateway Subnet</em>". And specify a
subnet in "<em>Address Range</em>". This subnet needs to be different
than the one we created earlier and should <strong>not</strong> be used
for non-network resources, but rather as an ingress point to your
Virtual Network. If unsure, use 10.1.0.0/24.</li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCgEzUd_3ZtkKcIfVCoI9cdG6xvTq-mnZxchLrFaf3cymjXnDvP9ug1U2tNBIZW549A09SCQ841BZcm-NMEYEMfkZsAl2HvDAFshdBj3agkvb7EgHecZGNNP33g4WVF4rck8Z6O_3jiQI/s1600/Screen+Shot+2016-10-08+at+10.52.18+PM.png"
imageanchor="1"
style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCgEzUd_3ZtkKcIfVCoI9cdG6xvTq-mnZxchLrFaf3cymjXnDvP9ug1U2tNBIZW549A09SCQ841BZcm-NMEYEMfkZsAl2HvDAFshdBj3agkvb7EgHecZGNNP33g4WVF4rck8Z6O_3jiQI/s640/Screen+Shot+2016-10-08+at+10.52.18+PM.png"
height="281"
width="640"
border="0"></a></div>
<br>
</ol>
<h4>Virtual Network Gateway</h4>
The "<a target="_blank" href="https://azure.microsoft.com/en-us/documentation/articles/vpn-gateway-about-vpngateways/">Virtual
Network Gateway</a>" is our configuration element that facilitates the
IPsec tunnel. Microsoft refers to this as a "VPN" gateway (as opposed to <a
href="https://azure.microsoft.com/en-us/documentation/articles/expressroute-introduction/">Express
Route</a>). There are three different VPN gateway SKUs; we'll be doing the
"Standard" offering (of Basic, Standard, High-Performance). It's worth
having a read about the differences <a target="_blank" href="https://azure.microsoft.com/en-us/documentation/articles/vpn-gateway-about-vpn-gateway-settings/#gwsku">here</a>.
<br>
<ol>
<li>Navigate to "<em>Virtual Network Gateways</em>".</li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgW4hFYCj4XEUUrBbY1bwRloPJm10aoDAlyfFMvN6Hd79V2eWxCYeJJBh9xc63RoXLpMqZaXGZ8zahx-AhjejaUGHJ4hZe5ZTQsRCRY0eQVaHRRk82tCQAIbq_9wKhnTsHXv0LavDyU3Po/s1600/Screen+Shot+2016-10-08+at+11.05.13+PM.png"
imageanchor="1"
style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgW4hFYCj4XEUUrBbY1bwRloPJm10aoDAlyfFMvN6Hd79V2eWxCYeJJBh9xc63RoXLpMqZaXGZ8zahx-AhjejaUGHJ4hZe5ZTQsRCRY0eQVaHRRk82tCQAIbq_9wKhnTsHXv0LavDyU3Po/s200/Screen+Shot+2016-10-08+at+11.05.13+PM.png"
height="69"
width="200"
border="0"></a></div>
<br>
<li>Click "<em>Add</em>" and supply the following: </li>
<ul>
<li><em>Name</em>: Again, follow consistent naming standards. </li>
<li><em>Gateway Type</em>: Select "<em>VPN</em>".</li>
<li><em>VPN type: </em>Select "<em>Route-Based</em>" (packets routed by
routing table) in this case; it would be advisable to familiarize
yourself with the difference between route and policy <a target="_blank"
href="https://azure.microsoft.com/en-us/documentation/articles/vpn-gateway-vpn-faq/#gateways">here</a>.
Note that policy requires IKEv1, so if you need to use it note the
settings will be quite a bit different. </li>
<li><em>SKU</em>: <strong>Update 1/2018: </strong>The SKU selection is
now VpnGw<x> or Basic. Note you <strong>cannot </strong>change
a basic VNG to the higher tier (VpnGwX) or vice versa at a later time.
For more information see <a href="https://docs.microsoft.com/en-us/azure/vpn-gateway/vpn-gateway-about-vpngateways#gwsku">this</a>
article. </li>
<li><em>Virtual Network</em>: Select the virtual network we created in
the last step. </li>
<li><em>Public IP address: </em>Select the IP address we created
earlier.</li>
</ul>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgByupM_hxWI9ZQP226XMpa_mH212hdwEKraJM7vuMbwARMQ1vRZUMRHQ1T7r3XeyB95A8kjuJ-hZGybTk1X9WxTD_I_uz6mWsS8RUvJnkAiL18zeB-oPjwhaDwbfK0Qp_1GQNlnZpy8D0/s1600/Screen+Shot+2016-10-08+at+11.09.27+PM.png"
imageanchor="1"
style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgByupM_hxWI9ZQP226XMpa_mH212hdwEKraJM7vuMbwARMQ1vRZUMRHQ1T7r3XeyB95A8kjuJ-hZGybTk1X9WxTD_I_uz6mWsS8RUvJnkAiL18zeB-oPjwhaDwbfK0Qp_1GQNlnZpy8D0/s640/Screen+Shot+2016-10-08+at+11.09.27+PM.png"
height="640"
width="258"
border="0"></a></div>
<br>
<li>Click "<em>Create</em>". </li>
</ol>
<p> <strong>Note:</strong> This step may take up to 45 minutes to complete
provisioning. I've tracked 8 of these and it's averaging almost 40 minutes
per regardless of if you pre-provision the IP or not. You may want to
consider skipping ahead to the pfSense section for a bit and coming back
here. </p>
<br>
<h4>Local Network Gateway/Connection</h4>
<br>
A Local Network Gateway is the specification of our local IP and networks
you would like to route over the tunnel. <br>
<br>
This actually works fine with a dynamic IP if that is your scenario, but
we'll cover the details of that later.<br>
<ol>
<li>Navigate to "<em>Local Network Gateways</em>".</li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEisN_qdFQwCP4scgITFfqw-bviX9j3R1WZHQaps-1pW59SREK-0OaREiOsAqksaEaU4VmjN1KZmtfB0v2AKIP4x5RFwDkfTBCt9ASfm9RIn1_AMich80Lu-sEcwlzgbtSbJgCM5f3xv7mM/s1600/Screen+Shot+2016-10-09+at+9.19.12+PM.png"
imageanchor="1"
style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEisN_qdFQwCP4scgITFfqw-bviX9j3R1WZHQaps-1pW59SREK-0OaREiOsAqksaEaU4VmjN1KZmtfB0v2AKIP4x5RFwDkfTBCt9ASfm9RIn1_AMich80Lu-sEcwlzgbtSbJgCM5f3xv7mM/s200/Screen+Shot+2016-10-09+at+9.19.12+PM.png"
height="63"
width="200"
border="0"></a></div>
<br>
<li>Click "<em>Add</em>" and supply the following: </li>
<ul>
<li><em>Name</em>: Naming? Standards? Consistent? Yeah!</li>
<li><em>IP Address</em>: Enter the public IP address of your device that
will instantiate the tunnel. </li>
<li><em>Address Space</em>: This is where you enter the CIDR notation of
the local networks you would like to route over the tunnel... for
example, if you would like to route 192.168.1.x over the tunnel, then
enter "192.168.1.0/24"</li>
<li><em>Subscription</em>: Enter your desired subscription. </li>
<li><em>Resource Group</em>: Select our resource group we created above.
</li>
<li><em>Location</em>: For consistency, select the same location as you
have selected above. </li>
<li><strong>Update 1/2018</strong>: You can configure BGP settings here
now as well, cool eh?</li>
</ul>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghtIwRJsAAB0dHcTHcshaXd3Ucgb_QGZXLI1C2eUXzsdLzKxdabQzXEyfB_CuKhsc-iRbUOV7x5rASdtdAB6sCYDgS4xXbq-HIv0VYSpg4C1uvIq-2klZVl62LEWwZsEuM41VJKq9KSbA/s1600/Screen+Shot+2016-10-09+at+9.23.45+PM.png"
imageanchor="1"
style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghtIwRJsAAB0dHcTHcshaXd3Ucgb_QGZXLI1C2eUXzsdLzKxdabQzXEyfB_CuKhsc-iRbUOV7x5rASdtdAB6sCYDgS4xXbq-HIv0VYSpg4C1uvIq-2klZVl62LEWwZsEuM41VJKq9KSbA/s400/Screen+Shot+2016-10-09+at+9.23.45+PM.png"
height="400"
width="382"
border="0"></a></div>
<br>
<li>Click "<em>Create</em>".</li>
<li>After provisioning, (you may need to hit "refresh) click your newly
created Local Network Gateway and click "<em>Connections</em>".</li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgacMjkdLbOHqAW9LXcCdUNDeRSaq-MkiIkaUi_h-NjS5F3ebhNW3yBALw760BHZpasVHURpxC8y39bQ1Bz9H5a1Bh9MrOU-e_izbCffBjaaPyefAHd1pFChYuABk8Tky2lgJ04avOvXxk/s1600/Screen+Shot+2016-10-09+at+9.25.36+PM.png"
imageanchor="1"
style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgacMjkdLbOHqAW9LXcCdUNDeRSaq-MkiIkaUi_h-NjS5F3ebhNW3yBALw760BHZpasVHURpxC8y39bQ1Bz9H5a1Bh9MrOU-e_izbCffBjaaPyefAHd1pFChYuABk8Tky2lgJ04avOvXxk/s640/Screen+Shot+2016-10-09+at+9.25.36+PM.png"
height="376"
width="640"
border="0"></a></div>
<br>
<li>On the newly expanded pane, click "<em>Add</em>" and supply the
following: </li>
<ul>
<li><em>Name</em>: You know the drill by now. </li>
<li><em>Connection type</em>: This should be fixed to "Site to Site
(IPsec)"</li>
<li><em>Virtual Network Gateway</em>: Enter the Virtual Network Gateway
we entered in the step above. </li>
<li><em>Shared Key</em>: Specify a unique, randomly generated passphrase
comprised of alphanumeric characters. Some devices have issues with
special characters, hence the omission of. I recommend using at least
30 characters; since it has no impact on tunnel performance I
personally use at least 60 characters for each key. You'll need to
specify this key on your local side as well. </li>
<li><em>Subscription</em>: This should be hard coded to the same
subscription as the LNG. </li>
<li><em>Resource Group</em>: This also should be locked to the same
resource group as the LNG. </li>
<li><em>Location</em>: Locked to that of the LNG. </li>
</ul>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgs-u-EZTC2u2Sxdy1sC8hZV7xczA9-skajftCeHro-cXk2hXheEKqtYiIXofSBE5bRl32Nt4ukOsi-shDsubw5zDejeL0riv7lebsutvpzWSHG3tiU1K6RUkmQwNwoGtDiaTOxTb6KY94/s1600/Screen+Shot+2016-10-09+at+9.35.36+PM.png"
imageanchor="1"
style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgs-u-EZTC2u2Sxdy1sC8hZV7xczA9-skajftCeHro-cXk2hXheEKqtYiIXofSBE5bRl32Nt4ukOsi-shDsubw5zDejeL0riv7lebsutvpzWSHG3tiU1K6RUkmQwNwoGtDiaTOxTb6KY94/s640/Screen+Shot+2016-10-09+at+9.35.36+PM.png"
height="640"
width="249"
border="0"></a></div>
<br>
<li>Click "<em>OK</em>". </li>
</ol>
<h3>Configure pfSense</h3>
<br>
Now we'll set up the IPsec initiator connection on your pfSense firewall(s).
<br>
<br>
<h4>Phase 1 Setup</h4>
<ol>
<li>Login to the firewall and navigate to "<em>VPN->IPsec"</em></li>
<li>Click "<em>Add</em>" and specify the following:</li>
<ul>
<li><em>Key Exchange Version: Auto</em></li>
<li><em>Internet Protocol: IPv4</em></li>
<li><em>Interface</em>: Select the WAN interface from which you would
like to instantiate the connection</li>
<li><em>Remote Gateway: </em>Enter the Azure public IP address created
in the "Public IP Address" section above</li>
<li><em>Description: </em>Whatever you would like; maybe troll your
firewall team with a message here for fun times. </li>
<li><em>Authentication Method: Mutual PSK</em></li>
<li><em>Negotiation Mode: Main </em><strong>Note</strong>: Do not use
"Aggressive" mode as the hash of the PSK is sent over the internet in
clear text. </li>
<li><em>My Identifier</em>: If the WAN interface selected above holds
your public IP address, you can select "<em>My IP Address</em>". If
that interface lies behind another edge device that holds the public
IP, you'll need to select "<em>IP Address</em>" and specify your
external IP. </li>
<li><em>Peer Identifier: Peer IP Address</em></li>
<li><em>Pre-Shared Key: </em>Enter the same Pre-Shared Key used in the
Azure connection specification above. </li>
<li><em>Encryption Algorithm</em>: The strongest available in Azure is <em>AES
256</em> bit, so preferably specify that. For more information on
supported features in Azure, see the References section below. </li>
<li><em>Hashing Algorithm: </em>The best we can do here is <em>SHA256</em>,
so let's go with that. </li>
<li><em>DH Group</em>: <em>2(1024 bit)</em></li>
<li><em>Lifetime (seconds): 10800</em></li>
<li><em>Disable Rekey: Unchecked</em></li>
<li><em>Responder Only: Unchecked</em></li>
<li><em>NAT Traversal: Auto </em>(Even in NAT scenarios Auto usually
works)</li>
<li><em>Dead Peer Detection: Checked</em></li>
<li><em>Delay: 10</em></li>
<li><em>Max Failures: 5</em></li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjsIc7I40C-nXxAFnn4rUHsW-OOt3gheGpl8u3ovEUNzlyfTG6S-gBWaOtiFwU2HL4f_nDG3XE2DGnEFKqlMb5Adk2lAsqyjadGCnlCKWwff_VPPDwCrbWms62zBrLnuNqZl6swTXboMbk/s1600/2016-10-25+16_15_43-scooter.truckchase.lan+-+VPN_+IPsec_+Tunnels_+Edit+Phase+1+-+Chromium.png"
imageanchor="1"
style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjsIc7I40C-nXxAFnn4rUHsW-OOt3gheGpl8u3ovEUNzlyfTG6S-gBWaOtiFwU2HL4f_nDG3XE2DGnEFKqlMb5Adk2lAsqyjadGCnlCKWwff_VPPDwCrbWms62zBrLnuNqZl6swTXboMbk/s640/2016-10-25+16_15_43-scooter.truckchase.lan+-+VPN_+IPsec_+Tunnels_+Edit+Phase+1+-+Chromium.png"
height="640"
width="496"
border="0"></a></div>
<br>
</ul>
<li>Click "<em>Save</em>" to return to the "<em>VPN->IPsec</em>" menu.
</li>
<li>Since we don't want to use this yet, click "<em>Disable</em>" in front
of the new tunnel definition and then "<em>Apply Changes</em>".</li>
</ol>
<h4>Phase 2 Setup</h4>
<ol>
<li>Under our newly created tunnel definition, click "<em>Show Phase 2
Entries</em>"</li>
<li>Click "<em>Add P2</em>" and supply the following information: </li>
<ul>
<li><em>Disabled: Unchecked</em></li>
<li><em>Mode: Tunnel IPv4</em></li>
<li><em>Local Network: </em>Select "<em>Network</em>" and specify the
same network range(s) that you specified during the set up of the
local network gateway on Azure using CIDR notation, i.e.
192.168.1.0/24. This specifies which local network(s) you would like
to route through the tunnel. </li>
<li><em>NAT/BINAT translation: None </em>; <strong>Note: </strong>even
in scenarios where your pfSense device is using NAT behind an upstream
router, this should not be necessary. NAT-T will take care of that
scenario. </li>
<li><em>Remote Network</em>: Select "<em>Network</em>" and specify the
same network range(s) that you specified during the set up of the
target virtual network in Azure using CIDR notation, i.e. 10.1.0.0/16.
This specifies the remote network(s) present in Azure. </li>
<li><em>Description: </em>Put something here to help you remember what
all this fun stuff is about. </li>
<li><em>Protocol: ESP</em></li>
<li><em>Encryption Algorithms</em>: Check only "<em>AES</em>" and "<em>256
bits</em>". </li>
<li><em>Hash Algorithms: </em>Unfortunately Azure <a target="_blank" href="https://azure.microsoft.com/en-us/documentation/articles/vpn-gateway-about-vpn-devices/">only
supports</a> "<em>SHA1</em>" at this time. <strong>Update 1/2017: </strong>SHA256
supported now, use that! </li>
<li><em>PFS Key Group</em>: <a target="_blank" href="https://azure.microsoft.com/en-us/documentation/articles/vpn-gateway-about-vpn-devices/">Azure
documentation</a> states that <a target="_blank" href="https://tools.ietf.org/html/rfc2412">PFS</a>
groups are only supported when Azure acts as responder, and in this
case it is being set up as the initiator. Oddly, I've actually had
luck specifying DH Group 14, but there is no guarantee that will work.
I'm going to stick with it but for this by the book exercise you'll
need to select "<em>off</em>". <strong>Note</strong>: Because of this
setting and the prior Hash Algorithm setting, I do not consider this
tunnel secure against state-level or similarly equipped actors. If
that is a concern you may wish to investigate alternatives. In less
extreme cases, however, this can be considered relatively secure.<strong>
Update 1/2017: </strong>The compatibility has been improved here as
well; match your Encryption/Auth with the right group using the table
<a href="https://docs.microsoft.com/en-us/azure/vpn-gateway/vpn-gateway-about-vpn-devices#ipsec">here</a>.
</li>
<li><em>Lifetime: 3600</em></li>
<li><em>Automatically ping host: blank</em></li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEj5v2O8eBTZbPZSj_hDsPQa7GEF3I5Bp2-aoqGaJ4xAVn2jYy15CXksznjh5dpOQXMbvh0MyRPxHNy0xmzjuLakQQgzg7eDkhRBsV3tEXCqs-udUI8D-iZSz8nmB-TzjIoelgaObm0Jk/s1600/2016-10-25+16_19_06-scooter.truckchase.lan+-+VPN_+IPsec_+Tunnels_+Edit+Phase+2+-+Chromium.png"
imageanchor="1"
style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEj5v2O8eBTZbPZSj_hDsPQa7GEF3I5Bp2-aoqGaJ4xAVn2jYy15CXksznjh5dpOQXMbvh0MyRPxHNy0xmzjuLakQQgzg7eDkhRBsV3tEXCqs-udUI8D-iZSz8nmB-TzjIoelgaObm0Jk/s640/2016-10-25+16_19_06-scooter.truckchase.lan+-+VPN_+IPsec_+Tunnels_+Edit+Phase+2+-+Chromium.png"
height="640"
width="602"
border="0"></a></div>
<br>
</ul>
<li>Click "Save". </li>
</ol>
<strong>Note: <em></em></strong>Depending on your configuration it may be
necessary to navigate to "<em>VPN->IPsec->Advanced Settings</em>" and
check "<em>Enable Maximum MSS</em>", then specify <em>1350</em>. If you get
packet loss with large packets this setting may be needed.<br>
<br>
<h4>Firewall Rules</h4>
<p>Now that our tunnel is set up we have to create local firewall rules that
allow for traffic to pass. First we'll create a network alias for the
Azure side network and then we'll make a rule to allow out Azure based
traffic to pass here. </p>
<ol>
<li>Navigate to "<em>Firewall->Aliases</em>" </li>
<li>Click "<em>Add</em>" and supply the following:</li>
<ul>
<li><em>Name:</em> Supply something that explains this network is to
represent the Azure side of the tunnel; only alphanumeric and "_" are
allowed. </li>
<li><em>Description</em>: Enter full description here; there are no
special character limitations. </li>
<li><em>Type</em>: "<em>Network(s)</em>"</li>
<li><em>Network(s): </em>Enter the CIDR notation of the network you
created for your Virtual Network in Azure. If you followed the example
addresses in this article, that would be 10.1.0.0/16. </li>
</ul>
<li>Click "<em>Save</em>" and then "<em>Apply Changes</em>". </li>
<li>Navigate to "<em>Firewall->Rules->IPsec</em>". </li>
<li>Click "<em>Add -^</em>" and supply the following: </li>
<ul>
<li><em>Action</em>: <em>"Pass</em>"</li>
<li><em>Disable this rule: Unchecked</em></li>
<li><em>Interface</em>: "<em>IPsec</em>"</li>
<li><em>Address Family</em>: "<em>IPv4</em>"</li>
<li><em>Protocol</em>: "<em>Any</em>" <strong>Discussion: </strong>Feel
free to limit the traffic that goes through the tunnel if you like. In
this example I'm allowing all traffic through. </li>
<li><em>Source</em>: "<em>Single host or alias</em>" and then specify
the Azure network alias you created in step 2. </li>
<li><em>Destination</em>: This needs to be the local network(s) to which
you would like to allow traffic. You can either do "<em>network</em>"
with a CIDR notation or specify the entire network represented by an
interface on the firewall. <strong>Note</strong>: if you have
multiple networks you'll need a rule for each, so repeat the last
couple steps for each. </li>
<li><em>Log: Unchecked</em>; keep in mind that should you need to
troubleshoot temporarily logging traffic using this rule can be very
useful. </li>
<li><em>Description</em>: It's a description, so let's do that!</li>
<li>No advanced options necessary unless you would like to do so. </li>
</ul>
<li>Click "<em>Save</em>" and then "<em>Apply Changes</em>". </li>
</ol>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhKAjGyZrFJiMQbhhsaJIQOR6AVvYNVMB3AnpvAjVrDj7CviIISiKGChmIV0IhTdExsJcuJMGVwfXHKxm7ePJI5paPWwST_xLHwYXHIrTKi7EfjzFI9UCSn8LJlwBIXg5i4JheIJkAIJYM/s1600/2016-10-10+14_18_40-Window.png"
imageanchor="1"
style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhKAjGyZrFJiMQbhhsaJIQOR6AVvYNVMB3AnpvAjVrDj7CviIISiKGChmIV0IhTdExsJcuJMGVwfXHKxm7ePJI5paPWwST_xLHwYXHIrTKi7EfjzFI9UCSn8LJlwBIXg5i4JheIJkAIJYM/s640/2016-10-10+14_18_40-Window.png"
height="259"
width="640"
border="0"></a></div>
<br>
<br>
<ol>
</ol>
As long as you have a blanket egress traffic rule we should now be able to
route traffic over the tunnel. If you do not I expect you are aware of how
to make a more specific rule to suit your needs.
<h4>A Note on NAT-T and Upstream Routers</h4>
If your pfSense device is behind another upstream router, you may need some
changes to facilitate the port switchover after initialization. If this
matches your configuration, consider that you may need the following on the
upstream router: <br>
<ul>
<li>A firewall rule that allows <strong>UDP port 4500</strong> into your
pfSense device(s). </li>
<li>A NAT port mapping rule that forwards <strong>UDP port 4500</strong>
to your pfSense device(s). </li>
</ul>
<p>Try without first; some devices are aware enough of the switch to 4500 to
perform the transition without rules, but if it does not work consult the
documentation for the device in question. </p>
<br>
<h3>Enable and Test</h3>
There are several ways to test our connection; in this case I'll be pinging
a VM host in Azure assigned to the same virtual network that this tunnel is
connecting to. We won't go through the provisioning of that; should you need
to refer to <a target="_blank" href="https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-windows-hero-tutorial/">this
basic guide</a> and ensure you place the VM in your target virtual network
and initially created subnet (10.1.10.x in the example above). <br>
<br>
<table class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"
cellspacing="0"
cellpadding="0"
align="center">
<tbody>
<tr>
<td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqzlMXveSpCkkRy1KFCMOgaRiSixM7HRY61ZLwg4RyXCVtWVtolN-358nj3vccY6KF6gmQkomZnOs8-73YAxHS6AFniyx21B2NfuqUNzkFiYAsoAKHSl1swTk6d1aHtd89CjmPnBl-wr8/s1600/2016-10-25+23_57_27-Zoom+Player.png"
imageanchor="1"
style="margin-left: auto; margin-right: auto;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqzlMXveSpCkkRy1KFCMOgaRiSixM7HRY61ZLwg4RyXCVtWVtolN-358nj3vccY6KF6gmQkomZnOs8-73YAxHS6AFniyx21B2NfuqUNzkFiYAsoAKHSl1swTk6d1aHtd89CjmPnBl-wr8/s640/2016-10-25+23_57_27-Zoom+Player.png"
height="226"
width="640"
border="0"></a></td>
</tr>
<tr>
<td class="tr-caption" style="text-align: center;">We're about to go
into a tunnel... a long one. </td>
</tr>
</tbody>
</table>
<br>
<h4>Preparing Your Target</h4>
<ul>
<li>Ensure your VM is up and provisioned in the correct target virtual
network. </li>
<li>Since you can't put anything in the gateway subnet (correctly) this
would be a good opportunity to put the VM in the subnet you were forced
to create when creating the virtual network in the first place.
Check/change <em>VM->Network
interfaces->Details->Settings->IP Configurations</em></li>
<li>Get the private IP address from <em>VM->Network interfaces. </em>It
should be 10.1.10.x if you're following the example addresses in this
article. </li>
<li>Make sure your VM is pingable! If you have instituted <a target="_blank"
href="https://azure.microsoft.com/en-us/documentation/articles/virtual-networks-nsg/">Network
Security Groups</a> that would inhibit access you'll need to modify
them, though this should work by default since we're tunneled in. Make
sure the firewall on the VM allows for incoming ICMP requests as well;
on Win 2012 and higher <em>set-netfirewallrule -DisplayName "File and
Printer Sharing (Echo Request - ICMPv4-In)" -Enabled True</em> will
take care of you. </li>
</ul>
<h4>Bring Up The Tunnel</h4>
<ol>
<li>In the pfSense interface, navigate to <em>VPN->IPsec</em>. </li>
<li>In front of our new tunnel, click "<em>Enable</em>" then "<em>Apply</em>"
toward the top. </li>
<li>Check tunnel status under <em>Status->IPsec</em>. The tunnel
should come up automatically in about a minute. If there is trouble you
can check the <em>Status->System Logs->IPsec</em> section for
more details. </li>
</ol>
<h4>Check Tunnel Status in Azure & Ping Dat VM!</h4>
<p>For this portion we'll use PowerShell; ensure you have the Azure ARM
cmdlets installed. If not, give <em>install-module AzureRM </em>a shot
from an elevated PowerShell prompt. </p>
<ol>
<li>Login to your account: <em>Login-AzureRmAccount</em></li>
<li>Look at your subscriptions and grab the name of the target sub: <em>Get-AzureRmSubscription</em></li>
<li>Change to your correct subscription: <em>Select-AzureRmSubscription
-SubscriptionName <subscription name></em></li>
<li>Check the status: <em>Get-AzureRmVirtualNetworkGatewayConnection
-name <Local Gateway Connection> -ResourceGroupName <Name of
Resourcegroup to which it belongs></em></li>
<li>On the output pane, check the "<em>ConnectionStatus</em>" property. It
should be "<em>Connected</em>". </li>
</ol>
<p>The Get-AzureRmVirtualNetworkGatewayConnection has a series of other
interesting properties as well, including EgressBytesTransferred and
IngressBytesTransferred.</p>
<p>Now proceed to ping your VM by the <strong>private </strong>IP listed
in Azure. As long as everything is configured correctly you should receive
a response!</p>
<br>
<h3>Cost</h3>
VPN Tunnels are subject to a costs from a few different categories: <br>
<ul>
<li><a target="_blank" href="https://azure.microsoft.com/en-us/pricing/details/vpn-gateway/">VPN
Gateway Pricing</a>: This is an hourly cost incurred <strong>while
the tunnel is available</strong>, not necessarily used. This means
once it's provisioned you will incur charges at the hourly rate. As of
the writing the standard performance level that we'll be using is billed
at $0.19/hr in the US. If you have multiple Virtual Networks you will
also be subject to a fee for outgoing traffic destined for another VNet.
This rate depends on the zone and varies between $.035 and $.16 per GB.
Data outbound to your site is charged at the standard data transfer
rates (below) and inbound data is free. <strong>Update 1/2017</strong>:
Updated pricing for the new tiers can be found <a href="https://azure.microsoft.com/en-us/pricing/details/vpn-gateway/">here</a>.
<br>
</li>
<li><a target="_blank" href="https://azure.microsoft.com/en-us/pricing/details/bandwidth/">Data
Transfer Rates</a>: This depends on your level of utilization. The
first 5GB of outgoing/month is free and the prices are set on a curve
thereafter. <br>
</li>
<li><a target="_blank" href="https://azure.microsoft.com/en-us/pricing/details/virtual-network/">Virtual
Network Pricing</a>: VNets are free; you can have up to 50 VNets per
subscription across all regions. <br>
</li>
<li><a target="_blank" href="https://azure.microsoft.com/en-us/pricing/details/ip-addresses/">IP
Addresses</a>: You'll be using at least one IP address. The first 5
static in a given region are free, additional and dynamic are charged at
a rate of $.004/hr. </li>
</ul>
<p>Overall the cost of a "standard" class data tunnel each month for a
single IP address, no additional support, and <strong>without including
outgoing bandwidth</strong>, is about $140/month. <br>
</p>
<p><strong>Note</strong>: Costs as of 10/26/2016, subject to change. Up to
date pricing information is available <a target="_blank" href="https://azure.microsoft.com/en-us/pricing/">here</a>.</p>
<br>
<h3>Dynamic IP? Changing Your IP Address</h3>
There is no reason that this IPsec tunnel will not work without a dnymic IP,
but each time the IP changes you'll need to take a series of steps to
restore tunnel functionality. These are: <br>
<ol>
<li>In Azure, the local network gateway specifies your IP, change it under
"<em>Local network gateway-><NAME>->Configuration->IP
address</em>". </li>
<li>In pfSense, the Phase 1 tunnel definition under "<em>My identifier</em>"
needs to reflect your current external IP address.</li>
<li>If there are any implications on upstream routers you'll need to
handle that as well. </li>
</ol>
<p>After taking care of these you will need to restart the tunnel. This
could all be automated with PowerShell and SSH if you like, but I won't be
covering that here.</p>
<p><strong>Update 1/2017: </strong>FWIW, over the last year none of my
clients have had their IP rotate for an active tunnel. </p>
<br>
<h3>A Note on Effective Security</h3>
As mentioned earlier, this set up does have a couple security issues; the
impact of which I would like to discuss briefly. Without an optimal security
configuration, including the support of Perfect Forward Secrecy, this tunnel
may not be strong enough to stand up to attacks of a state-sponsored actor
over a long period of time. Because of that I cannot recommend this
solution if your traffic may be subject to that level of attack, for example
traffic facilitating substantial financial transaction activity. <strong>Update
1/2017</strong>: As noted above, this situation has improved with the
support of PFS and SHA256 for authentication. <br>
<br>
With that said, this tunnel is still (for better or worse) more secure than
the configuration I have seen at many clients, and should be suitable for
most traffic. It also performs very well; added latency between my modestly
equipped pfSense devices and Azure is trivial. <br>
<br>
<table class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"
cellspacing="0"
cellpadding="0"
align="center">
<tbody>
<tr>
<td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgKHhfsTfK7TItRUee7N6rwvP8iwyOCY02hOt6hvKbCyM81jcVCROaX_Yk-xazW1P6QxyrYv3ALIqMUViZsAiIE6GkEhryW2ollJDvYPrIJ2GorJTz6F8gvP4MCEM8SR5qMZ2J9tzNrHgE/s1600/2001-image-1.jpg"
imageanchor="1"
style="margin-left: auto; margin-right: auto;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgKHhfsTfK7TItRUee7N6rwvP8iwyOCY02hOt6hvKbCyM81jcVCROaX_Yk-xazW1P6QxyrYv3ALIqMUViZsAiIE6GkEhryW2ollJDvYPrIJ2GorJTz6F8gvP4MCEM8SR5qMZ2J9tzNrHgE/s640/2001-image-1.jpg"
height="320"
width="640"
border="0"></a></td>
</tr>
<tr>
<td class="tr-caption" style="text-align: center;">For science!</td>
</tr>
</tbody>
</table>
<br>
<h3>References</h3>
<br>
<h4> Azure</h4>
<a target="_blank" href="https://azure.microsoft.com/en-us/documentation/articles/virtual-networks-overview/">Microsoft:
Azure Virtual Network Overview</a><br>
<a target="_blank" href="https://msdn.microsoft.com/en-us/library/mt125356.aspx">Microsoft:
Azure Resource Manager Cmdlets</a> (PowerShell)<br>
<a href="https://azure.microsoft.com/en-us/documentation/articles/vpn-gateway-howto-site-to-site-resource-manager-portal/">Microsoft:
Create a VNet with a Site-to-Site connection using the Azure portal</a><br>
<a target="_blank" href="https://azure.microsoft.com/en-us/documentation/articles/vpn-gateway-create-site-to-site-rm-powershell/">Microsoft:
Create a VNet with a Site-to-Site connection using PowerShell</a><br>
<a target="_blank" href="http://www.buchatech.com/2016/09/azure-site-to-site-vpn-setup-azure-resource-manager/">Steve
Buchanan: Azure & RRAS Site to Site VPN Setup (Azure Resource Manager)</a><br>
<a target="_blank" href="https://azure.microsoft.com/en-us/documentation/articles/virtual-networks-nsg/">Microsoft:
What is a Network Security Group?</a><br>
<a target="_blank" href="https://azure.microsoft.com/en-us/documentation/articles/vpn-gateway-vpn-faq/">Microsoft:
VPN Gateway FAQ</a><br>
<a target="_blank" href="https://azure.microsoft.com/en-us/documentation/articles/vpn-gateway-about-vpngateways"><span
style="color: #0000ee;">Microsoft:
About VPN Gateway</span></a><br>
<a target="_blank" href="https://azure.microsoft.com/en-us/documentation/articles/powershell-install-configure/">Microsoft:
How to install and configure Azure PowerShell</a><br>
<a target="_blank" href="https://azure.microsoft.com/en-us/documentation/articles/vpn-gateway-about-vpn-devices/">Microsoft:
About VPN Devices for Site-to-Site VPN Gateway Connections</a> (Critical
article; contains IPsec specs of Azure side)<br>
<br>
<h4>pfSense</h4>
<a target="_blank" href="https://doc.pfsense.org/index.php/Routing_internet_traffic_through_a_site-to-site_IPsec_tunnel">pfSense
Doco: Routing internet traffic through a site-to-site IPsec tunnel</a><br>
<a target="_blank" href="https://doc.pfsense.org/index.php/VPN_Capability_IPsec">pfSense
Doco: VPN Capability IPsec</a><br>
<br>
<h4>IPsec</h4>
<a target="_blank" href="https://tools.ietf.org/html/rfc2412">PFS RFC</a><br>
<a target="_blank" href="https://wiki.strongswan.org/projects/strongswan/wiki/SecurityRecommendations">strongSwan
Wiki: Security recommendations</a> <br>
<a target="_blank" href="https://blog.webernetz.net/2015/01/19/considerations-about-ipsec-pre-shared-keys-psks/">Johannes
Webber: Considerations About IPsec Pre-Shared Keys</a><br>
<a target="_blank" href="https://supportforums.cisco.com/document/64281/how-does-nat-t-work-ipsec">Cisco
Support: How Does NAT-T Work With IPsec</a> (Only needed if behind NAT)<br>
Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com3tag:blogger.com,1999:blog-5458589696790815336.post-69252856697469017072016-04-19T21:45:00.009-05:002023-09-23T00:19:40.980-05:00Installing Chef on A Raspberry Pi 2/3<br />
<h3>
Introduction</h3>
So you’ve got 1,103 Raspberry Pis that you need to manage. Two things: <br />
<ol>
<li>Why? </li>
<li>Wanna hang out Saturday? No? You’re busy managing all your mini computers manually? I can help you with that! </li>
</ol>
<a href="https://lh3.googleusercontent.com/-NWzdkcOzCSM/Vy12J1uFr1I/AAAAAAAAJHM/WPTUVnocXMc/s1600-h/RaspChef_Big%25255B14%25255D.jpg"><img alt="RaspChef_Big" height="216" src="https://lh3.googleusercontent.com/-Q3MzKKhaCjc/Vy12KYCFQjI/AAAAAAAAJHQ/bbEPnlpADQo/RaspChef_Big_thumb%25255B10%25255D.jpg?imgmax=800" style="display: block; float: none; margin-left: auto; margin-right: auto;" title="RaspChef_Big" width="620" /></a><br />
<h3>
Scope</h3>
In this article we’ll cover installing and configuring the <a href="https://www.chef.io/chef/">Chef</a> version 12 client on a Raspberry Pi. This has been tested on Raspberry Pi versions 2 and 3; in theory it should work on a 1 as well, albeit slowly. <br />
<h3>
Assumptions</h3>
<ul>
<li>Raspberry Pi with Rasbian, <a href="https://blog.hypriot.com/">Hypriot</a>, or similar build with connections to interwebs </li>
<li>Chef server/org you would like to point clients to </li>
<li>Chef workstation capable of bootstrapping clients; if needed see <a href="https://www.digitalocean.com/community/tutorials/how-to-set-up-a-chef-12-configuration-management-system-on-ubuntu-14-04-servers">this</a> excellent article by Digital Ocean. </li>
</ul>
<h3>
Execution</h3>
The main point of this article is really the installation of <a href="https://www.ruby-lang.org/en/">Ruby</a>, which is the foundation on which Chef is based. Because the Ruby package in the Rasbian locations is out of date (2.1 as of this writing) we need to compile our own from source. Chef 12 <a href="https://docs.chef.io/chef_system_requirements.html">requires Ruby 2.0 or greater</a>, but <a href="https://rack.github.io/">Rack</a>, which is installed with Chef, requires 2.2.2. <b>UPDATE 7/9/2017</b>: Ruby 2.4 or newer is now needed to continue successfully. Thanks Mike (from comments below)!<br />
<h4>
Step 1: Install Ruby</h4>
Clearly this should be scripted for optimal efficiency, but for learning purposes we’ll do it step by step to see exactly what is going on first hand. Log onto your Raspi via SSH and execute the following: <br />
<ol>
<li>Most commands we’ll be executing require root, so let’s elevate our session: <pre style="background: #f0f0f0; border-bottom: #cccccc 1px dashed; border-left: #cccccc 1px dashed; border-right: #cccccc 1px dashed; border-top: #cccccc 1px dashed; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">sudo su
</code></pre>
Where <i><a href="https://www.sudo.ws/intro.html">sudo</a></i> = "super user do" and <i><a href="https://en.wikipedia.org/wiki/Su_%28Unix%29">su</a></i> ="super user"; using root privs to assume root identity. <br /><br />
</li>
<li>Pull the newest package lists from configured repositories to ensure we get the newest packages: <pre style="background: #f0f0f0; border-bottom: #cccccc 1px dashed; border-left: #cccccc 1px dashed; border-right: #cccccc 1px dashed; border-top: #cccccc 1px dashed; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">apt-get update
</code></pre>
<a href="https://lh3.googleusercontent.com/-8OAmTRCfGpE/Vy7G2aAPgQI/AAAAAAAAJJo/q04vC8v58Zo/s1600-h/2016-05-07%25252000_03_01-_home_toby_%25255B7%25255D.png"><img alt="2016-05-07 00_03_01-_home_toby_" border="0" height="192" src="https://lh3.googleusercontent.com/-8Z3XbI1_in4/Vy7G3dB-JMI/AAAAAAAAJJs/8uhOVEQuTxw/2016-05-07%25252000_03_01-_home_toby__thumb%25255B5%25255D.png?imgmax=800" style="background-image: none; border-bottom-width: 0px; border-left-width: 0px; border-right-width: 0px; border-top-width: 0px; display: block; float: none; margin-left: auto; margin-right: auto; padding-left: 0px; padding-right: 0px; padding-top: 0px;" title="2016-05-07 00_03_01-_home_toby_" width="397" /></a><br />
</li>
<li>Install pre-requisites for Ruby: <a href="https://gcc.gnu.org/">gcc</a>, <a href="https://www.gnu.org/software/make/">make</a>, and <a href="https://packages.debian.org/jessie/libssl-dev">libssl-dev</a> with their dependencies. <pre style="background: #f0f0f0; border-bottom: #cccccc 1px dashed; border-left: #cccccc 1px dashed; border-right: #cccccc 1px dashed; border-top: #cccccc 1px dashed; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">apt-get install gcc make libssl-dev
</code></pre>
<a href="https://lh3.googleusercontent.com/-RzOOVHQgn3U/Vy7G34mRFhI/AAAAAAAAJJ0/cH5-xWt37-A/s1600-h/2016-05-07%25252000_06_47-_home_toby_%25255B4%25255D.png"><img alt="2016-05-07 00_06_47-_home_toby_" border="0" height="203" src="https://lh3.googleusercontent.com/-5OZhiwBz5cA/Vy7G4dU7g1I/AAAAAAAAJJ4/6x1ri33sKfg/2016-05-07%25252000_06_47-_home_toby__thumb%25255B2%25255D.png?imgmax=800" style="background-image: none; border-bottom-width: 0px; border-left-width: 0px; border-right-width: 0px; border-top-width: 0px; display: block; float: none; margin-left: auto; margin-right: auto; padding-left: 0px; padding-right: 0px; padding-top: 0px;" title="2016-05-07 00_06_47-_home_toby_" width="356" /></a><br /><b>Note</b>: On some distros, such as Raspbian, gcc and make are installed by default. It won't hurt to include the in the command line none the less, and including it here will cover most distros. <br /><br />
</li>
<li>Download the Ruby source to the /usr/src directory:<pre style="background: #f0f0f0; border-bottom: #cccccc 1px dashed; border-left: #cccccc 1px dashed; border-right: #cccccc 1px dashed; border-top: #cccccc 1px dashed; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">cd /usr/src
wget https://cache.ruby-lang.org/pub/ruby/2.2/ruby-2.2.5.tar.gz
</code></pre>
<b>Note</b>: 2.2.5 is the newest 2.2 version at the time of this writing, but you should make sure there isn't a newer version avialable. Check the Ruby page <a href="https://www.ruby-lang.org/en/downloads/">here</a>.<br /><br />
</li>
<li>Extract the source & navigate to the directory (<b>Make sure you update filenames/directory names for differing versions</b>):<pre style="background: #f0f0f0; border-bottom: #cccccc 1px dashed; border-left: #cccccc 1px dashed; border-right: #cccccc 1px dashed; border-top: #cccccc 1px dashed; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">tar -xvzf ruby-2.2.5.tar.gz
cd ruby-2.2.5
</code></pre>
</li>
<li>Prepare to compile with <a href="https://en.wikipedia.org/wiki/Configure_script">configure</a>, omitting unnecessary components: <pre style="background: #f0f0f0; border-bottom: #cccccc 1px dashed; border-left: #cccccc 1px dashed; border-right: #cccccc 1px dashed; border-top: #cccccc 1px dashed; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">./configure --enable-shared --disable-install-doc --disable-install-rdoc --disable-install-capi
</code></pre>
<a href="https://lh3.googleusercontent.com/-zlUPihpvKis/Vy7G4nNRH5I/AAAAAAAAJJ8/E_3MEoab3wo/s1600-h/2016-05-07%25252000_12_03-_home_toby_%25255B4%25255D.png"><img alt="2016-05-07 00_12_03-_home_toby_" border="0" height="120" src="https://lh3.googleusercontent.com/-Sqy1PjNhhno/Vy7G5I_l9nI/AAAAAAAAJKA/p5G1iWCE7GE/2016-05-07%25252000_12_03-_home_toby__thumb%25255B2%25255D.png?imgmax=800" style="background-image: none; border-bottom-width: 0px; border-left-width: 0px; border-right-width: 0px; border-top-width: 0px; display: block; float: none; margin-left: auto; margin-right: auto; padding-left: 0px; padding-right: 0px; padding-top: 0px;" title="2016-05-07 00_12_03-_home_toby_" width="482" /></a><br /><b>Note</b>: This will take between 2 and 10 minutes depending on which Pi and the speed of your SD card.<br /><br />
</li>
<li>Compile it using <a href="https://www.gnu.org/software/make/manual/make.html">make</a>!<pre style="background: #f0f0f0; border-bottom: #cccccc 1px dashed; border-left: #cccccc 1px dashed; border-right: #cccccc 1px dashed; border-top: #cccccc 1px dashed; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">make -j4 ; make install
</code></pre>
<a href="https://lh3.googleusercontent.com/-MXcUO3PNDvI/Vy7G5Vu3K_I/AAAAAAAAJKE/ftqUVYQcU1c/s1600-h/2016-05-07%25252015_28_19-Window%25255B7%25255D.png"><img alt="2016-05-07 15_28_19-Window" border="0" height="74" src="https://lh3.googleusercontent.com/-Q0uX2YDWdm0/Vy7G51EhheI/AAAAAAAAJKI/meYxH8PJv84/2016-05-07%25252015_28_19-Window_thumb%25255B3%25255D.png?imgmax=800" style="background-image: none; border-bottom-width: 0px; border-left-width: 0px; border-right-width: 0px; border-top-width: 0px; display: block; float: none; margin-left: auto; margin-right: auto; padding-left: 0px; padding-right: 0px; padding-top: 0px;" title="2016-05-07 15_28_19-Window" width="520" /></a><br /><b>Note</b>: "make -j4" will multi-thread the execution, using each of the Raspi's processors. This will take between 15 and 30 minutes depending on your Pi and SD card.</li>
</ol>
<a href="https://lh3.googleusercontent.com/-QvYcUGb8OIU/Vy7FbelLxzI/AAAAAAAAJJM/0r_1jGM_Z2o/s1600-h/hot_hot_hot%25255B20%25255D.jpg"><img alt="hot_hot_hot" border="0" height="267" src="https://lh3.googleusercontent.com/-G0ETHXWI6kI/Vy7Fb5Ra-dI/AAAAAAAAJJQ/rALhW7G_VCs/hot_hot_hot_thumb%25255B18%25255D.jpg?imgmax=800" style="background-image: none; border-bottom-width: 0px; border-left-width: 0px; border-right-width: 0px; border-top-width: 0px; display: block; float: none; margin-left: auto; margin-right: auto; padding-left: 0px; padding-right: 0px; padding-top: 0px;" title="hot_hot_hot" width="519" /></a><br />
<h4>
Step 2: Install Chef</h4>
Now we’ll use the <a href="https://guides.rubygems.org/command-reference/#gem-install">gem install</a> command to get Chef <br />
<ol>
<li>Execute <i>gem install chef </i>as root (sudo if not). <pre style="background: #f0f0f0; border-bottom: #cccccc 1px dashed; border-left: #cccccc 1px dashed; border-right: #cccccc 1px dashed; border-top: #cccccc 1px dashed; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">gem install chef
</code></pre>
<b>Note</b>: This will take between 5 and 25 minutes depending on which Pi, SD card, and network connection. <br /><br /><a href="https://lh3.googleusercontent.com/-gmWd0W2e3CA/Vy7G6AdE1fI/AAAAAAAAJKM/dPeL_59HTY4/s1600-h/2016-05-07%25252016_44_05-Window%25255B4%25255D.png"><img alt="2016-05-07 16_44_05-Window" border="0" height="141" src="https://lh3.googleusercontent.com/-GZV1jqlei2o/Vy7G6rf_XjI/AAAAAAAAJKQ/ekkbPBSGBIw/2016-05-07%25252016_44_05-Window_thumb%25255B2%25255D.png?imgmax=800" style="background-image: none; border-bottom-width: 0px; border-left-width: 0px; border-right-width: 0px; border-top-width: 0px; display: block; float: none; margin-left: auto; margin-right: auto; padding-left: 0px; padding-right: 0px; padding-top: 0px;" title="2016-05-07 16_44_05-Window" width="564" /></a><br />
</li>
<li>Relinquish root privileges as they are no longer needed. This should only exit the root session and not the SSH session itself. If you’re logged in directly as root ignore this, but don’t do that next time! <pre style="background: #f0f0f0; border-bottom: #cccccc 1px dashed; border-left: #cccccc 1px dashed; border-right: #cccccc 1px dashed; border-top: #cccccc 1px dashed; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">exit
</code></pre>
</li>
<li>Test the install to ensure it worked<pre style="background: #f0f0f0; border-bottom: #cccccc 1px dashed; border-left: #cccccc 1px dashed; border-right: #cccccc 1px dashed; border-top: #cccccc 1px dashed; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">chef-client --version
</code></pre>
<a href="https://lh3.googleusercontent.com/-HPpEmKOzbws/Vy7G7Ob3iwI/AAAAAAAAJKU/tDTakHdvwao/s1600-h/2016-05-07%25252016_46_29-Window%25255B7%25255D.png"><img alt="2016-05-07 16_46_29-Window" border="0" height="49" src="https://lh3.googleusercontent.com/-kGKl-dU6WfM/Vy7G7ZMem2I/AAAAAAAAJKY/uyjcG5lTaWk/2016-05-07%25252016_46_29-Window_thumb%25255B5%25255D.png?imgmax=800" style="background-image: none; border-bottom-width: 0px; border-left-width: 0px; border-right-width: 0px; border-top-width: 0px; display: block; float: none; margin-left: auto; margin-right: auto; padding-left: 0px; padding-right: 0px; padding-top: 0px;" title="2016-05-07 16_46_29-Window" width="516" /></a></li>
</ol>
<h4>
Step 3: Configure Chef</h4>
For this step move to your Chef workstation and logon using your account that is configured to manage your organization. <br />
<ol>
<li>Use the knife command to boostrap the newly installed client<pre style="background: #f0f0f0; border-bottom: #cccccc 1px dashed; border-left: #cccccc 1px dashed; border-right: #cccccc 1px dashed; border-top: #cccccc 1px dashed; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">knife bootstrap rasdock02.truckchase.lan -N rasdock02.truckchase.lan -x {user} -P {password}
</code></pre>
<b>Where</b>: {user} is a user on the target platform with root privs and {password} is the password for that account. <br /><br /><b>Note</b>: It is normal to see errors on the first portion on the bootstrap since the Chef ARM client will not be found in the Chef repo, but the second phase should work utilizing the client we just installed.<br /><br /><a href="https://lh3.googleusercontent.com/-EN_T8Ye4xJM/Vy7G7xHDr7I/AAAAAAAAJKc/RLIylGBSM34/s1600-h/2016-05-07%25252016_50_56-Window%25255B6%25255D.png"><img alt="2016-05-07 16_50_56-Window" border="0" height="385" src="https://lh3.googleusercontent.com/-0VpXwqY_CcU/Vy7G8a2NbKI/AAAAAAAAJKg/83oCOL6SJyI/2016-05-07%25252016_50_56-Window_thumb%25255B4%25255D.png?imgmax=800" style="background-image: none; border-bottom-width: 0px; border-left-width: 0px; border-right-width: 0px; border-top-width: 0px; display: block; float: none; margin-left: auto; margin-right: auto; padding-left: 0px; padding-right: 0px; padding-top: 0px;" title="2016-05-07 16_50_56-Window" width="527" /></a></li>
</ol>
<br />
That’s it! For further verification you can check against the Chef server using your workstation (<i>knife client show {node name}</i>) or even better yet, use <a href="https://docs.chef.io/manage.html">Chef Manage</a> if you have it available. <br />
<br />
<a href="https://lh3.googleusercontent.com/-YoRj47s11Rc/Vy7G8wLferI/AAAAAAAAJKk/Rq4MX1bJNnU/s1600-h/2016-05-07%25252016_53_23-Chef%252520Manage%25255B5%25255D.png"><img alt="2016-05-07 16_53_23-Chef Manage" border="0" height="219" src="https://lh3.googleusercontent.com/-G9dks4HU5Zs/Vy7G9TkQfjI/AAAAAAAAJKo/dQF_-Yun43E/2016-05-07%25252016_53_23-Chef%252520Manage_thumb%25255B3%25255D.png?imgmax=800" style="background-image: none; border-bottom-width: 0px; border-left-width: 0px; border-right-width: 0px; border-top-width: 0px; display: block; float: none; margin-left: auto; margin-right: auto; padding-left: 0px; padding-right: 0px; padding-top: 0px;" title="2016-05-07 16_53_23-Chef Manage" width="611" /></a><br />
<h4>
References</h4>
<li><a href="https://docs.chef.io/">Chef Documentation</a>
</li>
<li><a href="https://ruby-doc.org/">Ruby Documentation</a></li>
</ul>
Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com7tag:blogger.com,1999:blog-5458589696790815336.post-57872476884447628962015-04-05T20:25:00.000-05:002015-04-23T00:31:51.779-05:00PowerShell Networking Cheat Sheet <h3>Foreward</h3>
<p>I haven't located a good cheat sheet for basic PowerShell commands to
manage your Network Card/IP stack, so I thought I should make one. If you
like this please share it with your mom because I'm sure she's been
waiting to set a static IP address via PowerShell for some time. Also you
should call her more. </p>
<!--<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivBXruD57JJk_3OiDvmmOspK4hzyCChuNs34_qbyK6BqeJTlhKxfBYIejZ3JuzoVBJn193uDuaapNd93wTI3tSt1sae9oGc5-deqV4dXWbkAtBr0yvOu1VOGhXlZZiHlVP6KdXqFS_4jI/s1600/SUCH_A_CUTE_COUPLE.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivBXruD57JJk_3OiDvmmOspK4hzyCChuNs34_qbyK6BqeJTlhKxfBYIejZ3JuzoVBJn193uDuaapNd93wTI3tSt1sae9oGc5-deqV4dXWbkAtBr0yvOu1VOGhXlZZiHlVP6KdXqFS_4jI/s1600/SUCH_A_CUTE_COUPLE.png"
border="0" height="184" width="640"></a></div>-->
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhcdicw2aPFSW_VKBgPRr8nbNwRub5ioFH7qw2xRlh02zYDhgKvRIyoK6DaJgrKCae4JuTlcg3gL_5o2GfLzvWkTS-M775_LRPloSYe72c9DrNWTB_En9H1RSjbOuG6Thlq4pf_VWcTpYM/s1600/SUCH_A_CUTE_COUPLE2.jpg"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhcdicw2aPFSW_VKBgPRr8nbNwRub5ioFH7qw2xRlh02zYDhgKvRIyoK6DaJgrKCae4JuTlcg3gL_5o2GfLzvWkTS-M775_LRPloSYe72c9DrNWTB_En9H1RSjbOuG6Thlq4pf_VWcTpYM/s1600/SUCH_A_CUTE_COUPLE2.jpg"
border="0" height="184" width="640"></a></div>
<br>
<div style="text-align: center;">My campaign against stock photography
continues. </div>
<br>
<h3>Assumptions</h3>
<br>
<ul>
<li>Windows 2012/Windows 8 or higher</li>
<li>Administrator access to the machine in question</li>
<li>Powershell ran as administrator on the machine in question</li>
</ul>
<p><br>
</p>
<p>CMDLet List:</p>
<p>Most of the commands used to manipulate IP settings can be found by
typing Get-Command -Module NetTCPIP</p>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">PS C:\WINDOWS\system32> Get-Command -Module NetTCPIP
CommandType Name ModuleName
----------- ---- ----------
Function Find-NetRoute NetTCPIP
Function Get-NetCompartment NetTCPIP
Function Get-NetIPAddress NetTCPIP
Function Get-NetIPConfiguration NetTCPIP
Function Get-NetIPInterface NetTCPIP
Function Get-NetIPv4Protocol NetTCPIP
Function Get-NetIPv6Protocol NetTCPIP
Function Get-NetNeighbor NetTCPIP
Function Get-NetOffloadGlobalSetting NetTCPIP
Function Get-NetPrefixPolicy NetTCPIP
Function Get-NetRoute NetTCPIP
Function Get-NetTCPConnection NetTCPIP
Function Get-NetTCPSetting NetTCPIP
Function Get-NetTransportFilter NetTCPIP
Function Get-NetUDPEndpoint NetTCPIP
Function Get-NetUDPSetting NetTCPIP
Function New-NetIPAddress NetTCPIP
Function New-NetNeighbor NetTCPIP
Function New-NetRoute NetTCPIP
Function New-NetTransportFilter NetTCPIP
Function Remove-NetIPAddress NetTCPIP
Function Remove-NetNeighbor NetTCPIP
Function Remove-NetRoute NetTCPIP
Function Remove-NetTransportFilter NetTCPIP
Function Set-NetIPAddress NetTCPIP
Function Set-NetIPInterface NetTCPIP
Function Set-NetIPv4Protocol NetTCPIP
Function Set-NetIPv6Protocol NetTCPIP
Function Set-NetNeighbor NetTCPIP
Function Set-NetOffloadGlobalSetting NetTCPIP
Function Set-NetRoute NetTCPIP
Function Set-NetTCPSetting NetTCPIP
Function Set-NetUDPSetting NetTCPIP
Function Test-NetConnection NetTCPIP
</code></pre>To find out more about any given command, type Get-Help <cmdlet
name> <br>
<br>
<h3> Common Tasks </h3>
<br>
Here are walkthroughs of some of the more common tasks you may want to
perform. <br>
<br>
<span style="text-decoration: underline;">
<h4> List Network Adapters</h4>
</span> Gets a list of all adapters in the machine; you'll need to know your
adapter name or index # (both listed) for some of the commands below. <br>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">Get-NetAdapter
</code></pre><br>
<span style="text-decoration: underline;">
<h4>Change an Adapter Friendly Name</h4>
</span> You may want to consider changing the friendly name of the adapter
you intend to manipulate. By giving it shorter, more meaningful name you'll
have an easier time going forward. Use the old name you got from the last
command. Syntax is <em>Rename-NetAdapter -Name "<Current Name>"
-NewName <"New Name"></em><br>
<code style="color: black; word-wrap: normal;"> </code>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">Rename-NetAdapter -Name "Local Area Connection" -NewName WiFi
</code></pre><br>
<span style="text-decoration: underline;">
<h4> Get the Current IP Address</h4>
</span> Gets all IPv4 addresses on the machine; you'll almost always have
multiple as your loopback interface (127.0.0.1) will be listed. Ignore that
guy. <br>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">Get-NetIPAddress -AddressFamily ipv4
</code></pre> Optionally, you can specify -InterfaceAlias <friendly name>
or -InterfaceIndex <index #> to limit the command to a single adapter.
<br>
<br>
<span style="text-decoration: underline;">
<h4>Assign a Static IP Address to your Network Adapter</h4>
</span> This command will set the one and only (overwriting what is there)
IP address for the specified network adapter. You also can (and should) set
the subnet mask with this command. The subnet mask is set via <a href="http://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing">CIDR</a>
using the -PrefixLength; see the link for more info about CIDR, but if
you're not familiar with CIDR it is likely that you want -PrefixLength 24
which translates to 255.255.255.0 meaning the first three octets are the
network while the last is the host. Syntax is <em>New-NetIPAddress
-InterfaceAlias <name> -IPAddress <IP address> -PrefixLength
<CIDR> -DefaultGateway <Gateway IP> . </em>You can
substitute<em> -InterfaceIndex <index #> </em>for<em> -InterfaceAlias
. </em><br>
<code style="color: black; word-wrap: normal;"> </code>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">New-NetIPAddress -InterfaceAlias WiFi -IPAddress 192.168.1.10 -PrefixLength 24 -DefaultGateway 192.168.1.1
</code></pre><strong> Note</strong>: You will get an error if you already have a
static IP address with a default gateway. To fix this problem see "Delete
Existing Static IP" below and then try again. <br>
<strong>Note2</strong>: We're not using "Set-NetIPAddress" here because it
doesn't allow you to set a default gateway. BOOOO. <br>
<br>
<span style="text-decoration: underline;">
<h4> Set DNS Servers for your Adapter</h4>
</span> To look up names you'll need to set DNS server(s). Syntax is <em>Set-DNSClientServerAddress
-InterfaceAlias <name> -ServerAddresses ("<IP Address 1","IP
Address 2") . </em>You can set as many DNS servers as you like. You can
substitute<em> -InterfaceIndex <index #> </em>for<em> -InterfaceAlias
.</em><code style="color: black; word-wrap: normal;"></code>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">Set-DNSClientServerAddress –interfaceAlias WiFi –ServerAddresses (“192.168.1.5”,”192.168.1.6”)
</code></pre><br>
<span style="text-decoration: underline;">
<h4> Set a Default Gateway</h4>
</span> It's generally easier to set the default gateway as part of the
New-NetIPAddress command above, but if you want to set one separately see
"Set a Static Route" below. <br>
<br>
<span style="text-decoration: underline;">
<h4>Delete Existing Static IP (to prep for a new)</h4>
</span> This is a two step process; you need to delete the IP, then the
gateway. No need to worry about the DNS servers here as it works to
overwrite them with the command above. You will need to know the IP address
you want to delete first; use <em>get-netipaddress</em> (above) to get it
(write it down to use below if necessary). You'll then need to know the
NextHop of the gateway. To get this, use the <em>get-netroute</em> command
and write down the entry(ies) that have a nexthop of the gateway you intend
to remove (see screenshot). <br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPU_PzaJWsS3XxADKyN1EmuTN9JL5L2hmng1j9WjkXoMncHTQ2qslJMR007XxUhzOZHpbV8q09VWkOAj8fDngX_PDy67c3wQnS1F6aWKsWxuMDTwMuB-P6ScHm_qouCNI85SzYni11Fxs/s1600/2015-04-04+22_18_24.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPU_PzaJWsS3XxADKyN1EmuTN9JL5L2hmng1j9WjkXoMncHTQ2qslJMR007XxUhzOZHpbV8q09VWkOAj8fDngX_PDy67c3wQnS1F6aWKsWxuMDTwMuB-P6ScHm_qouCNI85SzYni11Fxs/s1600/2015-04-04+22_18_24.png"
border="0" height="197" width="640"></a></div>
<br>
<br>
The syntax for these commands are <br>
<em>Remove-NetAddress <IPAddress> -Confirm:$False<br>
Remove-NetRoute -NextHop <Gateway IPAddress> -Confirm:$False</em><br>
<code style="color: black; word-wrap: normal;"> </code>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">Remove-NetAddress 192.168.1.10 -Confirm:$False
Remove-NetRoute -NextHop 192.168.1.1 -Confirm:$False
</code></pre><strong> Note</strong>: If you have multiple routes set with that
default gateway it will delete them all. If you haven't manually set routes,
don't worry about it (you just have the one). <br>
<br>
<span style="text-decoration: underline;">
<h4> Set Your Adapter to Use DHCP</h4>
</span> This is another two step process; first set the desired adapter
IP/Gateway to DHCP then set the DNS servers to pull from DHCP as well. <br>
<br>
The syntax for these commands are:<br>
<em>Set-NetIPInterface -InterfaceAlias <name> -Dhcp Enabled<br>
Set-DNSClientServerAddress -InterfaceAlias <name>
-ResetServerAddress<br>
</em>You can substitute <em>-InterfaceIndex <index #> </em>for<em>
-IterfaceAlias </em>if you prefer.<em> </em><br>
<code style="color: black; word-wrap: normal;"> </code>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">Set-NetIPInterface -InterfaceAlias WiFi -Dhcp Enabled
Set-DNSClientServerAddress -InterfaceAlias Wifi -ResetServerAddress
</code></pre><strong> Note</strong>: If you have a static gateway set you'll
need to perform the second step <em>"Remove-NetRoute</em>" from the step
above as well. <br>
<br>
<br>
<h3> Advanced Tasks </h3>
<br>
Here are walkthroughs of some of the more common tasks you may want to
perform. My assumption here is that you know what you want to do so I won't
be discussing the details of what each of these means. <br>
<br>
<span style="text-decoration: underline;">
<h4>Add/Delete a Static Route</h4>
</span> Add: (use -RouteMetric to specify metric or -PolicyStore to control
persistence through reboots)<br>
<code style="color: black; word-wrap: normal;"> </code>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">New-NetRoute -DestinationPrefix 192.168.2.0/24 -InterfaceAlias WiFi -NextHop 192.168.2.1
</code></pre> Add default route:<br>
<code style="color: black; word-wrap: normal;"> </code>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">New-NetRoute -DestinationPrefix 0.0.0.0/0 -InterfaceAlias WiFi -NextHop 192.168.2.1
</code></pre> Delete: (while this command is very specific you can be more
generic; see above)<br>
<code style="color: black; word-wrap: normal;"> </code>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">Remove-NetRoute -DestinationPrefix 192.168.2.0/24 -InterfaceAlias WiFi -NextHop 192.168.2.1
</code></pre><br>
<span style="text-decoration: underline;">
<h4> Test Network Connectivity (Ping)</h4>
</span> <i>Test-Connection</i> replaces ping.exe. In addition to the ping
functionality Test-Connection supports authentication (if firewall is set
accordingly), multiple targets in a single command, running as a job, as
well as more detailed returns. <br>
<code style="color: black; word-wrap: normal;"> </code>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">Test-Connection myhost.mydomain.com
</code></pre><br>
<span style="text-decoration: underline;">
<h4> Assign a DNS Suffix</h4>
</span> This is the domain under which your IP will be registered and under
most circumstances will be used to append to hostname searches. Note this is
per-adapter. You can substitute <em>InterfaceIndex</em> for <em>InterfaceAlias</em>
if you like. <br>
<code style="color: black; word-wrap: normal;"> </code>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">Set-DnsClient -InterfaceAlias WiFi -ConnectionSpecificSuffix mydomain.com
</code></pre><br>
<span style="text-decoration: underline;">
<h4> Assign an Additional IP Address to your NIC</h4>
</span> If you want to add another IP (usually only applicable on a server)<br>
<code style="color: black; word-wrap: normal;"> </code>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">New-NetIPAddress -InterfaceAlias Ethernet1 -IPAddress 192.168.1.101 -PrefixLength 24
</code></pre><br>
<br>
<h3> References/More Information</h3>
<a href="https://technet.microsoft.com/en-us/library/hh826123.aspx">TechNet:
Net TCP/IP Cmdlets in Windows PowerShell</a><br>
<a href="https://technet.microsoft.com/en-us/library/jj590772.aspx">TechNet:
DNS Client Cmdlets in Windows PowerShell</a><br>
<a href="https://technet.microsoft.com/en-us/library/jj134956.aspx">TechNet:
Network Adapter Cmdlets</a><br>
<br>
You did it, hug a puppy! <br>Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com0tag:blogger.com,1999:blog-5458589696790815336.post-53068321662025901212015-01-03T20:22:00.000-06:002015-01-03T20:22:43.616-06:00Remotely Restart Serivices via PowerShell Without Admin Credentials <h3>Scope</h3>
<br>
Ever wanted to allow a non-admin to remotely restart a service? No? K. <br>
<br>
.... well if sometime you feel like it, this article is for you. We'll walk
through enabling <span style="color: #0000ee;"><span style="text-decoration: underline;">PowerShell
Remoting</span></span> (part of <a href="http://blogs.technet.com/b/askperf/archive/2010/09/24/an-introduction-to-winrm-basics.aspx">WinRM</a>),
allowing the non-administrators to connect, and allowing the specified user
or group to restart the service in question. <br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg-2BO4NHC-4ph8hpscQCyh2gDnvs3IVN6DAA2OBP9Zdl4O1hWY7afFG_MWQI-USHeyF0eXeriGeYYJF6Tc-e5CKcTlJGPfWigTNdPgvW40ErUVMB_RdAeMUA1MS9uu90lt-wrmv4GMxP8/s1600/family-guy-facebook-cover_cropped_700.jpg"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg-2BO4NHC-4ph8hpscQCyh2gDnvs3IVN6DAA2OBP9Zdl4O1hWY7afFG_MWQI-USHeyF0eXeriGeYYJF6Tc-e5CKcTlJGPfWigTNdPgvW40ErUVMB_RdAeMUA1MS9uu90lt-wrmv4GMxP8/s1600/family-guy-facebook-cover_cropped_700.jpg"
border="0" height="192" width="640"></a></div>
<br>
<h3>(Install and) Enable PowerShell Remoting</h3>
<br>
On Windows server 2012, this step is already completed by default. On
earlier versions of Windows you may need to install software and then enable
connection. That is an article in and of itself, but fortunately it has
already been <a href="http://blogs.technet.com/b/poshchap/archive/2014/07/04/powershell-remoting-considerations.aspx">published
by Ian Farr</a>. Consult that article to determine what software, if any,
you need to install.<br>
<br>
After installs:<br>
<br>
<strong>Note:</strong> This may not be necessary on 2012 and up but will not
hurt none the less. <br>
<ol>
<li>Remote to the server you wish to remotely manipulate services on</li>
<li>Open an elevated PowerShell prompt</li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQNtNGyakrkcAPq6G81nWCAI7FUCwQGov1zoL7Yc2xKbNSA8vXXpyspw4l6GYScNcM6vdKIUDHvtDo3A1ovb8-l89ta5v8p89Fbmfv627GCArzbEqqi1WN8L7gtFS3MTfG2qsbQDdugRg/s1600/2015-01-02+23_48_32-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQNtNGyakrkcAPq6G81nWCAI7FUCwQGov1zoL7Yc2xKbNSA8vXXpyspw4l6GYScNcM6vdKIUDHvtDo3A1ovb8-l89ta5v8p89Fbmfv627GCArzbEqqi1WN8L7gtFS3MTfG2qsbQDdugRg/s1600/2015-01-02+23_48_32-RD+Tabs+64.png"
border="0"></a></div>
<br>
<li>Run the <em><a href="http://technet.microsoft.com/en-us/library/hh849694.aspx">Enable-PSRemoting</a>
</em>cmdlet</li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhMn9wljINuxy67aiC1rCZAZnpTjWuweculf4POE1EAR8icNZSZexqZplWvEtq1mSGl_yUHc_kBMTbHPoQs0VpyQYsyjikIKzcdzsXhBPo3FYiiwIZlTAN8ryC225m5s6DLD0YtYST_ARc/s1600/2015-01-02+23_49_44-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhMn9wljINuxy67aiC1rCZAZnpTjWuweculf4POE1EAR8icNZSZexqZplWvEtq1mSGl_yUHc_kBMTbHPoQs0VpyQYsyjikIKzcdzsXhBPo3FYiiwIZlTAN8ryC225m5s6DLD0YtYST_ARc/s1600/2015-01-02+23_49_44-RD+Tabs+64.png"
border="0" height="60" width="400"></a></div>
<br>
<li>For each query, type "y" and press enter. </li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhqvhuCsmPlYRp55dXxT2ykE6DWmgOzLm6YZhiSUxnscKcjrijQ2_b1rnTQVOJ6vOJcN4ycYAXvTVaM_8UPeyAVPHOy9rInXz3W1P6zFojJ21aJxyqdS9reNCVQ8GpXIgXTTNYGQOxnLU/s1600/2015-01-02+23_51_23-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhqvhuCsmPlYRp55dXxT2ykE6DWmgOzLm6YZhiSUxnscKcjrijQ2_b1rnTQVOJ6vOJcN4ycYAXvTVaM_8UPeyAVPHOy9rInXz3W1P6zFojJ21aJxyqdS9reNCVQ8GpXIgXTTNYGQOxnLU/s1600/2015-01-02+23_51_23-RD+Tabs+64.png"
border="0" height="408" width="640"></a></div>
<br>
</ol>
<p>The Enable-PSRemoting cmdlet will run a few different sub cmdlets to
enable the proper services and setup initial permissions. </p>
<br>
<h3>Set up Security Principals and Grant Permissions to use PowerShell
Remoting</h3>
<br>
<p><strong>Caution</strong>: This grants access to connect to
the PSRemote interface and attempt to issue commands. While this isn't
enough to stop services, etc. by itself it should be done with caution. </p>
<p>There are two levels of security you will need to address with this
operation: the ability to connect to the PSRemote interface/attempt to
issue commands, and the permissions on the service to be restarted. It is
important to consider that these two groups may not be the same; there is
a lot one can do with PSRemoting outside of restarting a specific service.
Thus it is likely you will want to create a larger (in scope and size)
group to access PSRemoting and a smaller group to restart this specific
service. </p>
<p>If you don't much care and just want to get going you can just make the
principals the same... even just a single user. With that said: </p>
<ol>
<li>Determine what security principals you want to have access to
PSRemoting and create them if necessary. I use an Active Directory group
named appropriately and populated with the desired users, but you could
use anything down to a single user if you like. </li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1RuEaNrbJ6Y9TABVdDFdJxhSiNC6sSvkq_FA6yWVXsjq4Js8ovzht9MfdLX90GKHBI6nDfxy12-Ug9pCHvosNxpB2OI4jWhPn57yY2WVWjcM82zoYrrUVITTTD4zeZpYIFbNNQvU1FXs/s1600/2015-01-02+23_54_34-Active+Directory+Administrative+Center.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1RuEaNrbJ6Y9TABVdDFdJxhSiNC6sSvkq_FA6yWVXsjq4Js8ovzht9MfdLX90GKHBI6nDfxy12-Ug9pCHvosNxpB2OI4jWhPn57yY2WVWjcM82zoYrrUVITTTD4zeZpYIFbNNQvU1FXs/s1600/2015-01-02+23_54_34-Active+Directory+Administrative+Center.png"
border="0" height="27" width="400"></a></div>
<br>
<li>Ensure you still have the PowerShell prompt open on your target
server; if not open it as in the section above. </li>
<li>Execute <em><a href="http://technet.microsoft.com/en-us/library/hh849726.aspx">Set-PSSessionConfiguration</a>
-ShowSecurityDescriptorUI -Name Microsoft.PowerShell </em>which will
bring up the Windows security dialog box. </li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgh3s66WvY2NDOJ_YLXR5tlybNu9TzsQCU4oTcbQAe4V71Sk3Cj7-69QMqQTROx1Y-GtcwjrRMMf40lpcRycUazXsakDXq8CJeI7HhWNczAdFxLWBwK58TshnX6UVqO2R_B2biiPG_Wj6w/s1600/2015-01-02+23_55_37-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgh3s66WvY2NDOJ_YLXR5tlybNu9TzsQCU4oTcbQAe4V71Sk3Cj7-69QMqQTROx1Y-GtcwjrRMMf40lpcRycUazXsakDXq8CJeI7HhWNczAdFxLWBwK58TshnX6UVqO2R_B2biiPG_Wj6w/s1600/2015-01-02+23_55_37-RD+Tabs+64.png"
border="0" height="80" width="640"></a></div>
<br>
<li>Add your desired security principal (AD Group, Local Group, User, etc)
and grant it "Full Control" access. </li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxQrQqbnIkgjikJIV-r1GCh8S10neBE_sSmsE8PCNC65CldJ0Z7-xgdIjuu0J5d141zSIhg0RdNIbTlVKOk40I_X4yFQfp6b7mCxbMgkhMpKWY5H5veOLGI2HYh-X5060mibu4oTg_lr0/s1600/2015-01-02+23_56_30-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxQrQqbnIkgjikJIV-r1GCh8S10neBE_sSmsE8PCNC65CldJ0Z7-xgdIjuu0J5d141zSIhg0RdNIbTlVKOk40I_X4yFQfp6b7mCxbMgkhMpKWY5H5veOLGI2HYh-X5060mibu4oTg_lr0/s1600/2015-01-02+23_56_30-RD+Tabs+64.png"
border="0" height="400" width="331"></a></div>
<br>
<li>Click "OK" to close the box and apply the permissions, then answer y +
<enter> to restart the service. </li>
</ol>
<p>Note that on 64-bit systems where you have elected to use the 32-bit
version of PowerShell you will also need to execute steps 3-5 using the <em>Set-PSSessionConfiguration
-ShowSecurityDescriptorUI -Name Microsoft.PowerShell32 </em>command as
well. </p>
<p>Now you can hit the PSRemote interface with the specified accounts; if
you haven't figured it out yet this could be used for many things other
than just restarting a service... this is why PSRemoting is so great!</p>
<br>
<h3>Grant Service Restart Access to the Appropriate Security Principal</h3>
<br>
<p>In my example my service will be called "MyService". Normally this is an
operation that would be somewhat complex with the built-in Windows tools,
so I'm going to offer up a simpler way. </p>
<ol>
<li>Download and install <a href="http://get-carbon.org/">Carbon</a>
(more below) on your target server(s). </li>
<li>Per the install instructions, unblock the zip, extract, and import the
module after install with <em>Import-Module Carbon </em>or the<em>
import-carbon.ps1</em> script. (<strong>Note</strong>: You may need to
<em><a href="http://technet.microsoft.com/en-us/library/hh849812.aspx">set-executionpolicy</a>
unrestricted</em> to use Carbon; make sure you <em>set-executionpolicy
Restricted</em> when you're done)</li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEik-AJchIWQouEtqBCVcxnmcCfmzI8AkmPGkGdQywK2U_5Nro9_ighPFA_FKjEuhVaOF6AYjMSa_78u_NjjoPNEFIaR1jQiouzqvY0S84Qu6Qb6BVfhkUNJT6uc7waSN_gGzbnuviMomQ0/s1600/2015-01-03+00_14_53-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEik-AJchIWQouEtqBCVcxnmcCfmzI8AkmPGkGdQywK2U_5Nro9_ighPFA_FKjEuhVaOF6AYjMSa_78u_NjjoPNEFIaR1jQiouzqvY0S84Qu6Qb6BVfhkUNJT6uc7waSN_gGzbnuviMomQ0/s1600/2015-01-03+00_14_53-RD+Tabs+64.png"
border="0" height="138" width="640"></a></div>
<br>
<li>Execute the <a href="http://get-carbon.org/help/Grant-ServiceControlPermission.html">Grant-ServiceControlPermission</a>,
i.e. <em>Grant-ServiceControlPermission -ServiceName MyService
-Identity MyDomain\MyGroup </em>where MyService is the short name of
the target service and MyDomain\MyGroup is the target security
principal. </li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjnGAy0EOiySacfwJ0vWO_lHibSYCwkVKK-v23QrPOTgnmUEIm-fQvezK78bsBzUQ2d7_vL1ubs0EAw9cPupponCXjykMaPN5mq_azi8gsK5_pWyd_PyZ5w62lQppDGFe8NQKns8ODHSnQ/s1600/2015-01-03+00_16_30-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjnGAy0EOiySacfwJ0vWO_lHibSYCwkVKK-v23QrPOTgnmUEIm-fQvezK78bsBzUQ2d7_vL1ubs0EAw9cPupponCXjykMaPN5mq_azi8gsK5_pWyd_PyZ5w62lQppDGFe8NQKns8ODHSnQ/s1600/2015-01-03+00_16_30-RD+Tabs+64.png"
border="0" height="24" width="640"></a></div>
<br>
</ol>
<p><a href="https://twitter.com/pshdo">Aaron Jensen</a> has done a great job
with the Carbon module and I highly recommend it to anyone operating in a
DevOps space on Windows Server. That one command in point #3 could have
been an article about SDDLs in and of itself. </p>
<br>
<h3>Test and Provide Restart Commands to Target User(s)</h3>
<br>
<p>Rather than having your target users learn the appropriate PowerShell
commands, it may be easier to provide them with a script. Here's a sample
one that can restart multiple services on multiple servers after executing
the steps above on each. </p>
<br>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">
$servers="Server1","Server2"
$services="MyService1","MyService2"
Write-Warning 'What would you like this script to do? '
Write-Warning '1. Stop Services'
Write-Warning '2. Start Services'
Write-Warning '3. Re-start Services'
Write-Warning '4. Quit'
while($ActionType.length -lt 1)
{
$ActionType=Read-Host "Enter the number to correspond with your desired action. "
$ActionType=[string]$ActionType.trim()
}
switch ($ActionType)
{
1
{
$stop=$true
$start=$false
}
2
{
$stop=$false
$start=$true
}
3
{
$stop=$true
$start=$true
}
4 {Exit}
default {throw ("Unexpected input from action type inquriy.")}
}
foreach ($server in $servers)
{
if ($stop)
{
foreach ($service in $services)
{
Write-Debug "Stopping $service on $Server"
Invoke-Command -ComputerName $server -ScriptBlock {Stop-Service -Name $args[0]} -ArgumentList $service
if ($? -eq $false){throw "TERMINATING SCRIPT; UNABLE TO STOP SERVICE. Check permissions and ensure target machine $server is up"}
Write-Warning "Service $service stopped on $server"
}
}
start-sleep -Milliseconds 500
if ($start)
{
#Start Services
foreach ($service in $services)
{
Write-Debug "Starting $service on $server"
Invoke-Command -ComputerName $server -ScriptBlock {Start-Service -Name $args[0]} -ArgumentList $service
if ($? -eq $false){throw "TERMINATING SCRIPT; UNABLE TO START SERVICE. Check permissions and ensure target machine $server is up"}
Write-Warning "Service $service started on $server"
}
}
}
Write-Warning "All actions complete!"
</code></pre> <br>
This script is really just a starting point; there are many different
directions you could take it based on your needs. For example, you could use
the get-credential cmdlet to allow the user to enter alternative
credentials. To use it as-is simply replace the server(s) and service(s)
names at the top of the script with those you are targeting. <br>
<br>
... and that's it! Thanks to carbon and a couple simple commands this
operation is pretty easy to achieve. This has been a big help in lower
environments for letting users, developers, etc. restart services as needed
without having to interrupt their support personnel. Hopefully it helps you
as well. <br>
<br>
<h3>References/Additional Information</h3>
<br>
<a href="http://technet.microsoft.com/en-us/library/hh849694.aspx">TechNet:
Enable-PSRemoting</a><br>
<a href="http://stackoverflow.com/questions/11153692/wsman-and-basic-authorization">Stack
Overflow: WSMan and Basic Authorization</a><br>
<a href="http://stackoverflow.com/questions/14127050/powershell-remoting-giving-access-is-denied-error">Stack
Overflow: PowerShell Remoting Giving Access is Denied Error</a><br>
<a href="http://blogs.technet.com/b/askperf/archive/2010/09/24/an-introduction-to-winrm-basics.aspx">TechNet:
Introduction to WinRM</a><br>
<a href="http://www.sevecek.com/EnglishPages/Lists/Posts/Post.aspx?ID=10">Ondrej
Sevecek: Enabling remote WMI and PowerShell access over WinRM for
non-administrators</a> (Much of the inspiration for this article)<br>
<a href="http://get-carbon.org/">Carbon</a> (PowerShell Dev-Ops module by
Aaron Jensen)Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com1tag:blogger.com,1999:blog-5458589696790815336.post-23211071626748913042014-12-22T07:30:00.000-06:002014-12-24T00:17:23.466-06:00Mumble + Splunk <h3>Who's on the Server? Splunk it!</h3>
<br>
<a href="http://wiki.mumble.info/wiki/Main_Page">Mumble</a> is a great VOIP
solution for latency sensitive situations; I run several servers for
different applications, and monitoring those has always been a bit of a
challenge since it only generates text logs and doesn't do any historic
usage tracking. Fortunately we've got a tool to solve that problem in both
real time and historic situations: <a href="http://www.splunk.com/">Splunk</a>.
<br>
<br>
In this article we'll walk through a simple example of data ingestion,
parsing, and dashboard creation in Splunk. When we're finished we'll be able
to tell who is online at any given time and how popular your server has been
in the recent past. <br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghWd8NXP7gjoo5D008MKlpOUX0HMnFEZ7QU-iS324TT9qpzI0zrtZB-j7MQ9Ck4IMkfc7v66OxiP4TDTYrbQ239UhNHPxAHEMCZqdM5xHxmbSWk7_cK1OVxmHzVl5tJOWUW5_HS4eA1L4/s1600/2014-12-21+23_27_56-Mumble+Statistics+_+Splunk.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghWd8NXP7gjoo5D008MKlpOUX0HMnFEZ7QU-iS324TT9qpzI0zrtZB-j7MQ9Ck4IMkfc7v66OxiP4TDTYrbQ239UhNHPxAHEMCZqdM5xHxmbSWk7_cK1OVxmHzVl5tJOWUW5_HS4eA1L4/s1600/2014-12-21+23_27_56-Mumble+Statistics+_+Splunk.png"
border="0" height="509" width="640"></a></div>
<br>
<h3>Step 1: Data Ingestion</h3>
This section assumes the following: <br>
<br>
<ul>
<li>Mumble (or another app if using this as a general guideline) installed
and logging to a consistent location.</li>
<li>Logging is not set to rename logfiles as part of the log rollover
process; doing so would complicate our source setup. (note this is
achievable) </li>
<li>The Splunk forwarder is installed on the target machine(s) and is
already configured to output to the indexer(s). (install location
referred to henceforth as $SPLUNK_HOME) What's that? You've got
thousands of boxes and no time to install? We can <a href="http://blog.ittoby.com/2014/09/using-powershell-jobs-to-trigger-remote.html">fix
that</a>!</li>
</ul>
<p>I'm using Windows boxes in this example, but there's no reason all of
this won't work on Linux as well with some minor tweaks.</p>
<br>
To get the data into Splunk we'll first need to identify where the data is
and how to ensure it gets into the target indexer(s). In my case I'll be
targeting the following (4 instances) data sources from one server:<br>
<ul>
<li>C:\Apps\Murmur1LowQual\murmur.log</li>
<li>C:\Apps\Murmur2LowQual\murmur.log</li>
<li>C:\Apps\Murmur1HighQual\murmur.log</li>
<li>C:\Apps\Murmur2HighQual\murmur.log</li>
<li>Performance data: Network interface, CPU Usage</li>
</ul>
<br>
<p>If the application to be monitored is distributed densely and
consistently in your enterprise you will want to make the input changes in
an "<a href="http://docs.splunk.com/Documentation/Splunk/6.2.0/Updating/Updateconfigurations">app</a>"
for deployment assuming you use "<a href="http://docs.splunk.com/Documentation/Splunk/6.2.0/Updating/Forwardermanagementoverview">Forwarder
Management</a>". In this case it is a one-off configuration so I will be
specifying these inputs manually.</p>
<br>
For our one-off case, open the <em>$SPLUNK_HOME/etc/system/local/inputs.conf</em>
file for editing (for background, make sure you understand "<a href="http://docs.splunk.com/Documentation/Splunk/6.2.0/admin/Aboutconfigurationfiles">About
Configuration Files</a>") and add the following entries: <br>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">
[monitor://C:\Apps\MurmurLowQual\murmur.log]
disabled = false
sourcetype = murmur
[monitor://C:\Apps\MurmurHighQual\murmur.log]
disabled = false
sourcetype = murmur
[monitor://C:\Apps\Murmur2HighQual\murmur.log]
disabled = false
sourcetype = murmur
[monitor://C:\Apps\Murmur2LowQual\murmur.log]
disabled = false
sourcetype = murmur
[perfmon://CPU Load]
counters = % Processor Time
instances = _Total
interval = 20
object = Processor
[perfmon://Network Interface]
counters = Bytes Received/sec;Bytes Sent/sec
instances = *
interval = 15
object = Network Interface
</code></pre>
<strong>Note</strong> these entries are Windows specific; the paths and the
perfmon data would need be changed on a Linux host. <br>
<br>
<strong>Where</strong>: <br>
<ul>
<li><em>monitor://<logfilePath> </em>specifies the path to the
logfile to be ingested (Murmur is Mumble's server component) </li>
<li><em>sourcetype = murmur</em> specifies the sourcetype to store this
information under in Splunk. This is critical to properly sorting data.
</li>
<li><em>perfmon://<counter></em> , <em>counters =
<counter1>;<counter2></em> , <em>instances = *</em>, and
<em>object = <object in question> </em>specify the performance
information we want to bring in. This line is Windows specific and needs
to be changed for *nix.</li>
<li><em>interval = <interval in seconds></em> is the interval to
bring that performance data in at. Lower interval = more granular
performance information but more data to store.<em> </em></li>
</ul>
<p>This may or may not be sufficient in your case. For more information, see
<a href="http://docs.splunk.com/Documentation/Splunk/6.2.1/Data/Editinputs.conf">"Edit
inputs.conf" on the Splunk site</a>. </p>
<br>
<h3>Step 2: Pre-Format Data</h3>
<br>
<p>Now we need to take the steps necessary to easily create searches against
this data. At a minimum any Splunk Admin should take the time to do proper
field extractions from your new data source. Creating custom fields ensure
there will be meaningful information for users to search on. This part of
Splunk operation is often the make-or-break point in many organizations,
as proper field extraction can be the difference between an end user
figuring out how to create meaningful searches vs. giving up and going to
the original log files. </p>
<p>In this example I'll use one <a href="http://en.wikipedia.org/wiki/Regular_expression">regex</a>
to extract three fields from the Murmur logfile under the "search" app
context. To set up this field extraction: </p>
<ol>
<li>Log in to your Splunk web interface</li>
<li>Change to the "<em>search</em>" app context</li>
<li>Navigate to "<em>Settings -> Fields</em>" </li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1Zg10s83-vugaWhjJ-wvV48TkP2R-yXu-NPAMdkV_hY0l3HEW5wu7Ypq-VrFjgTPPVaPUISSchSQ4EMkc_hqzmwM4J5nErwoq4WaP2QXOSxcSlIql0FUxtJt-vSbVh2ULQ_9elw1LUy0/s1600/2014-12-21+23_31_15-Search+_+Splunk+6.2.0.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1Zg10s83-vugaWhjJ-wvV48TkP2R-yXu-NPAMdkV_hY0l3HEW5wu7Ypq-VrFjgTPPVaPUISSchSQ4EMkc_hqzmwM4J5nErwoq4WaP2QXOSxcSlIql0FUxtJt-vSbVh2ULQ_9elw1LUy0/s1600/2014-12-21+23_31_15-Search+_+Splunk+6.2.0.png"
border="0" height="258" width="400"></a></div>
<br>
<li>Click on "<em>Field extractions</em>"</li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEinWg43aycj7Xs8MeWeGvWK4lwTiNM3Jq2FRlX_fj30F_a4UwXGT4Cn11uJ84R2k7jPt-cEKKM1okql8VpwUVu3VmCkvV0oU5-xaOKYZyVZgVwf-MKwb6fkHIRNjS1lVf4mKlcJA0WVnHo/s1600/2014-12-21+23_31_35-Settings+_+Splunk.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEinWg43aycj7Xs8MeWeGvWK4lwTiNM3Jq2FRlX_fj30F_a4UwXGT4Cn11uJ84R2k7jPt-cEKKM1okql8VpwUVu3VmCkvV0oU5-xaOKYZyVZgVwf-MKwb6fkHIRNjS1lVf4mKlcJA0WVnHo/s1600/2014-12-21+23_31_35-Settings+_+Splunk.png"
border="0" height="41" width="400"></a></div>
<br>
<li>Click "<em>New</em>"</li>
<li>Leave the "destination app"as "<em>search</em>" (We'll work in search
for now but you could make this into an app) </li>
<li>Name it per your enterprise standard. In my case that is
<AppName_fields extracted>, so "<em>Murmur_sessionID_UserName_AdminStatus</em>".
</li>
<li>Change "Apply to" to "<em>sourcetype</em>" named "<em>murmur</em>"</li>
<li>Keep "Type" to "<em>Inline</em>"</li>
<li>For the Extraction/Transform insert your regex statement. To extract
the Session ID (as session_ID), UserName (as u_name), and AdminStatus
(as userIsAdmin) from a Murmur logfile use this: "<em>=>
<(?<session_ID>[^:]+):(?<u_name>[^\(]+)\((?<userIsAdmin>[^\)]+)</em>"</li>
<li>Click "<em>Save</em>"</li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidTw-HGPc2xvYKOqElMKh-BYtyc8qpkSIcZJ6c3CCS5vMeQh02vy3SP7yYitaleo8Dxnhdb5jsvdXPBXLsxuqZi9AettaLxmtuqMmP3kNa42Kd8Ifk7AnuzXz3dAJa4b8mu-LN7RUpuwM/s1600/2014-12-21+23_33_03-Settings+_+Splunk.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidTw-HGPc2xvYKOqElMKh-BYtyc8qpkSIcZJ6c3CCS5vMeQh02vy3SP7yYitaleo8Dxnhdb5jsvdXPBXLsxuqZi9AettaLxmtuqMmP3kNa42Kd8Ifk7AnuzXz3dAJa4b8mu-LN7RUpuwM/s1600/2014-12-21+23_33_03-Settings+_+Splunk.png"
border="0" height="232" width="400"></a></div>
<br>
<li>To make this usable for everyone, click "<em>Permissions</em>" under
"Sharing" to the right of the name on the "Field extractions"
screen. </li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiM5sxf-am68hMCFTCHVtRgMIBjiQdAhG1hLgpTeJaopBMmrkqDCP7DKMyrsU0kSXJ9yitlozRcQqHYeE354aYRsBO5PsxtzpBqTI7TH4oR4UCghgCD4yCBz_BNjdfedSvIT3wcFteWB6Y/s1600/2014-12-21+23_33_26-Settings+_+Splunk.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiM5sxf-am68hMCFTCHVtRgMIBjiQdAhG1hLgpTeJaopBMmrkqDCP7DKMyrsU0kSXJ9yitlozRcQqHYeE354aYRsBO5PsxtzpBqTI7TH4oR4UCghgCD4yCBz_BNjdfedSvIT3wcFteWB6Y/s1600/2014-12-21+23_33_26-Settings+_+Splunk.png"
border="0"></a></div>
<br>
<li>Select "<em>This app only(search)</em>" (change later if you use a
different app) and check "<em>Read</em>" under "<em>Everyone</em>" Click
"Save". </li>
</ol>
<p>We have now configured field extractions for Mumble. While there is much
more one should do (data sizing, business use cases, <a href="http://wiki.splunk.com/Things_I_wish_I_knew_then">etc.</a>)
to on-board an application this will be enough for now to develop a basic
dashboard. </p>
<br>
<h3>Step 3: Searches and a Dashboard!</h3>
<br>
<p>Now we'll make use of this data. This is the beauty of Splunk; you can
format almost all the data in a meaningful way, and even create new data
points inferred from the other available data. To illustrate what I mean,
one of these searches will determine who is online right now from the
logon/logoff information in the log file. Let's tackle that and a few
others: </p>
<br>
<p><strong>Search 1: Who is online right now? </strong></p>
<em>sourcetype=murmur |transaction session_ID,u_name maxspan=24h|search
authenticated NOT closed|eval AdminRange = case(userIsAdmin < 0,
"False", userIsAdmin >= 1, "True")|table
u_name,session_ID,AdminRange,_time | rename u_name as "User Name",
session_ID as "Session ID", AdminRange as "User is Administrator"</em> <br>
<br>
Will generate a table like this: <br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi2j8VSsxKDtUvJFz_mdpxDYCl9ly6996r1R5Pzvl24A04Nrd9E9UItNnxGXnyEY83E5-KgONh5cqkBDz3IVJZT_AlzE12x3qYWRD2x_gfwi1SSBBMI0lRc7VTBnaXYxJrkNiH3gIndd4w/s1600/2014-12-21+23_39_50-Search+_+Splunk+6.2.0.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi2j8VSsxKDtUvJFz_mdpxDYCl9ly6996r1R5Pzvl24A04Nrd9E9UItNnxGXnyEY83E5-KgONh5cqkBDz3IVJZT_AlzE12x3qYWRD2x_gfwi1SSBBMI0lRc7VTBnaXYxJrkNiH3gIndd4w/s1600/2014-12-21+23_39_50-Search+_+Splunk+6.2.0.png"
border="0" height="137" width="640"></a></div>
<br>
<strong> Where</strong>: <br>
<br>
<em>sourcetype=murmur</em> : only search the appropriate sourcetype. Note
you may want to further limit by host, index, or other locations. <br>
<br>
<em>transaction session_ID,u_name maxspan=24h</em> <strong>:</strong> uses
the powerful "<a href="http://docs.splunk.com/Documentation/Splunk/6.2.1/SearchReference/Transaction">transaction</a>"
command to string events into a transaction by the listed fields. <strong>Note</strong>:
This command can be computationally expensive so be careful when using it! <br>
<br>
<em>search authenticated NOT closed</em> <strong>:</strong> look for
sessions illustrating a user connecting but not yet closed <strong>Note</strong>:
again, since we're using a "NOT" clause this search could be expensive
depending on your data volume from the previous parts of the search.
Fortunately we should be very limited data wise in this search string by
now. <br>
<br>
<em>eval AdminRange = case(userIsAdmin < 0, "False", userIsAdmin >= 1,
"True")</em> <strong>:</strong> use the <a href="http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Eval">eval</a>
command to determine admin status as Boolean <br>
<br>
<em>table u_name,session_ID,AdminRange,_time</em> <strong>:</strong> Render
as a table<br>
<br>
<em>rename u_name as "User Name", session_ID as "Session ID", AdminRange as
"User is Administrator"</em> <strong>:</strong> Rename table fields to be
more meaningful to the end user<br>
<br>
Since we're limiting a session span to 24hours in the transaction search,
you may as well reduce the time for this search to 24 hours as well. For a
slight performance tweak on searches that display a table or graphic of
specific fields you can change the search mode from "smart" to "fast".
Now let's add this to a new dashboard: <br>
<ol>
<li>After the results come up, click "<em>Save As</em>" the "<em>Dashboard
Panel</em>" on the upper right hand side. </li>
<li>Select "<em>New</em>"</li>
<li>Insert an appropriate title, i.e. "<em>Mumble Statistics</em>"
following enterprise standards if applicable. Generally it is OK to let
the Dashboard ID auto-populate with this name. </li>
<li>Write a description if desired and change the Dashboard Permissions to
"<em>Shared in App</em>" so we can share this information with
others. </li>
<li>Type an appropriate Panel Tile such as "<em>Users Online Now!</em>".
Accept the defaults for the remaining and click "Save". </li>
</ol>
For fun you can make a little tachometer displaying this information by
opening the saved search and saving it to the existing dashboard under a
different name, then changing the display to "Radial Gauge". End users
generally like these "at a glance" graphics for important information,
especially at the top of a dashboard. <br>
<br>
<strong>Search 2: How many people have logged on per day for the last 30
days? </strong><br>
<br>
<em>sourcetype=murmur u_name=* authenticated |timechart count</em><br>
<br>
Will generate: <br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeEXj6CB17OpMvj6fcWQUf80FZqwqu8-h1H9Ecl5D5S8kk29nybx8HTPg_nJIprdEaOSN6ztUWkVCS4TgQNCUdutBG1h4WVy-TB78d39NKsS7P7bRd57GVMxkNnBgKeERDZ-ogrJ_ctEU/s1600/2014-12-21+23_42_03-Search+_+Splunk+6.2.0.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeEXj6CB17OpMvj6fcWQUf80FZqwqu8-h1H9Ecl5D5S8kk29nybx8HTPg_nJIprdEaOSN6ztUWkVCS4TgQNCUdutBG1h4WVy-TB78d39NKsS7P7bRd57GVMxkNnBgKeERDZ-ogrJ_ctEU/s1600/2014-12-21+23_42_03-Search+_+Splunk+6.2.0.png"
border="0" height="280" width="640"></a></div>
<br>
<strong> Where</strong>: <br>
<em><br>
sourcetype=murmur u_name=* authenticated</em> <strong>:</strong> only
search the appropriate sourcetype for events where u_name is populated and
the word "authenticated" is present. <br>
<br>
<em>timechart count</em><em></em> <strong>:</strong> charts all results
using the very easy <a href="http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Timechart">timechart</a>
command. Make sure you limit the search scope (below). <br>
<br>
Set the search scope for 30 days, execute, then save to our existing Mumble
Statistics dashboard as a panel named "Logins Per Day - Last 30 Days". I
prefer this as a bar chart, which you can select at the dashboard level. <br>
<br>
<strong>Search 3: How much bandwidth did the server use in the last day? </strong><br>
<br>
<em>host=hostname_here sourcetype="Perfmon:Network Interface"|</em><em><em>eval
DataSrc=case((instance=="Microsoft Hyper-V Network Adapter" AND
counter=="Bytes Received/sec"),"</em><em><em>ETH0</em>bitsSecIN",(instance=="Microsoft
Hyper-V Network Adapter" AND counter=="Bytes
Sent/sec"),"ETH0bitsSecOUT")</em>|eval Kbits_Sec=Value*.008| timechart
span=5m avg(Kbits_Sec) by DataSrc| rename LANbitsSecIN as "LAN Kbits/sec
IN" LANbitsSecOUT as "LAN Kbits/sec OUT" WANbitsSecIN as "WAN Kbits/sec
IN" WANbitsSecOUT as "WAN Kbits/sec OUT"</em><br>
<br>
Will generate: <br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2r_3Qq7Cgu-cATrA4g5hSB1SLOUGG_2hlw-Gxh3D6EKt-NsAsB2-q_epy-FjiaPbxd7xe051d8Yb1-RjO8ZF_ZVb_Ia2bvOhejzg__9KH_zxL0D794qft1PQq0uMAIqkEsVNuE_TE7wU/s1600/2014-12-21+23_42_28-Search+_+Splunk+6.2.0.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2r_3Qq7Cgu-cATrA4g5hSB1SLOUGG_2hlw-Gxh3D6EKt-NsAsB2-q_epy-FjiaPbxd7xe051d8Yb1-RjO8ZF_ZVb_Ia2bvOhejzg__9KH_zxL0D794qft1PQq0uMAIqkEsVNuE_TE7wU/s1600/2014-12-21+23_42_28-Search+_+Splunk+6.2.0.png"
border="0" height="324" width="640"></a></div>
<br>
<strong> Where</strong>: <br>
<em><br>
host=hostname_here sourcetype="Perfmon:Network Interface"</em> <strong>:</strong>
Specify events to search. You will need to change the hostname and
potentially the sourcetype depending on your host platform, etc. <br>
<br>
<em>eval DataSrc=case((instance=="Microsoft Hyper-V Network Adapter" AND
counter=="Bytes Received/sec"),"</em><em><em>ETH0</em>bytesSecIN",(instance=="Microsoft
Hyper-V Network Adapter" AND counter=="Bytes Sent/sec"),"ETH0bytesSecOUT")</em>
<strong>:</strong> Here is where we use eval to map interfaces to
directions. If you have multiple interfaces you'll need to address them on
this line and also note you will need to change the "names" of the adapters
to match your data.<br>
<br>
<em>eval Kbits_Sec=Value*.008</em> <strong>:</strong> Convert Bytes/Sec to
Kbits/Sec<br>
<br>
<em>timechart span=5m avg(Kbits_Sec) by DataSrc</em> <strong>:</strong>
Chart the data by interface direction on a 5 minute average. Note if you
made a longer-term chart you'll need to change the average to calculate on a
wider basis to keep your datapoints low enough to chart. <br>
<br>
<em>rename ETH0bytesSecIN as "ETH0 Kbits/sec IN" </em><em><em>ETH0bytesSecOUT</em>
as "ETH0 Kbits/sec OUT"</em> <strong>:</strong> Rename the datapoints
relative to our eval statement above. <br>
<br>
Set the search scope for 24 hours, execute, then save to our existing Mumble
Statistics dashboard as a panel named "Network Traffic 5m Average Last 24
Hours"<br>
<br>
<table class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"
align="center" cellpadding="0" cellspacing="0">
<tbody>
<tr>
<td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWwI13AQ9Tvg_quzzXu_KhPJRNu9a8FNtHvnYtmkMgO1fIXqUBmeypF-hvSXZAtnKdnE_EFB2T8qyUAVLckrqbBIebsjSlrpp-PPaB5c9GBEW_V1kPiHpXXBh8FYitWnx1D17d_6bbqi4/s1600/steve-urkel-tgif-universe.jpg"
imageanchor="1" style="margin-left: auto; margin-right: auto;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWwI13AQ9Tvg_quzzXu_KhPJRNu9a8FNtHvnYtmkMgO1fIXqUBmeypF-hvSXZAtnKdnE_EFB2T8qyUAVLckrqbBIebsjSlrpp-PPaB5c9GBEW_V1kPiHpXXBh8FYitWnx1D17d_6bbqi4/s1600/steve-urkel-tgif-universe.jpg"
border="0" height="240" width="400"></a></td>
</tr>
<tr>
<td class="tr-caption" style="text-align: center;">How many users on
the sever? Plenty. </td>
</tr>
</tbody>
</table>
<br>
Those three should get you started; clearly there is substantially more one
could do with all the data available to us. After you decide what else to
add make sure you go through your dashboard and reposition/edit each panel
as necessary. Keep in mind that you can rename x/y axis as well as change
the way data is rendered. Hopefully this tutorial has demonstrated that even
with a simple application you can use a tool like Splunk to make the
day-to-day use and impact on an organization much more transparent. This
methodology could easily be turned into an app and distributed throughout
your enterprise and/or the Splunk community.Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com0tag:blogger.com,1999:blog-5458589696790815336.post-48301051934227878672014-11-15T15:26:00.002-06:002014-11-18T23:19:03.969-06:00Microsoft KB 2992611 "Winshock": More Thank You Bargained For <p>(updated 11/18 for re-issue, see below)</p>
<p>Microsoft released one of the most important patches in many years on
Tuesday, and while I would advise you install it right away I also want to
make you aware of some odd behavior I found that could lead to problems.
First, a primer:</p>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxn3eKI5e-5NCNIIj5mFdUonNk4-OaAvdAhGKHxDWj-J-ZxwmUSIfln4uxmmpKxNSsoKta6ET_fT6Gc8qxVO4bvTdaMUSjzcXsng8s05h_HJ5mk2GXU8TrZHhK616UdoMke0jZ2WoH4T8/s1600/2014-11-15+15_13_48-Network+Error.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxn3eKI5e-5NCNIIj5mFdUonNk4-OaAvdAhGKHxDWj-J-ZxwmUSIfln4uxmmpKxNSsoKta6ET_fT6Gc8qxVO4bvTdaMUSjzcXsng8s05h_HJ5mk2GXU8TrZHhK616UdoMke0jZ2WoH4T8/s1600/2014-11-15+15_13_48-Network+Error.png"
border="0"></a></div>
<p><a href="https://technet.microsoft.com/library/security/MS14-066">Here</a>
are the release notes and <a href="https://support.microsoft.com/kb/2992611">here</a>
is more information. This is the worst kind of exploit there can be; a
remote code execution with no workarounds. If one knew the details, they
could easily exploit any Microsoft based internet facing server supporting
TLS and then turn around and use it to infect unpatched Windows based
clients. Obviously you should patch immediately. </p>
<p>That said, you will notice that they mention the addition of four new
cipher suites but there is one other change that may impact you that is
not mentioned. I've found that this patch also re-orders the cipher
suites. Historically Microsoft has notified customers when re-ordering
cipher suites; see <a href="http://support.microsoft.com/kb/2929781">KB2919355</a>
for example. </p>
<p>This is important to understand for two reasons, one theoretical and one
practical. </p>
<ul>
<li>Theoretical is that changing cipher suites impacts your security
posture, and one should always know these things going into a patch.
Fortunately most of the re-order does seem in line with a tighter
security policy.</li>
<li>Practical is that this can break connectivity with some applications.
Specifically, one of my peers found that Java 6 based applications
attempting purposely or otherwise to use the <a href="http://en.wikipedia.org/wiki/Elliptic_curve_Diffie%E2%80%93Hellman">ECDH</a>
key agreement protocol will fail to connect. This happens when Windows
based services present ECDH before the older RSA. <em>Side note</em>:
Oddly the Microsoft JDBC driver tries to negotiate SSL even if it isn't
being used for a connection to SQL. </li>
</ul>
<p>Here are the cipher suite details, first <strong>2008 R2:</strong> </p>
<p> </p>
<table style="width: 800px; height: 1000px;" border="0" cellpadding="0" cellspacing="0">
<colgroup><col style="mso-width-source:userset;mso-width-alt:12763;width:262pt"
width="349"> <col style="mso-width-source:userset;mso-width-alt:13494;width:277pt"
width="369"> </colgroup>
<thead>
<tr style="height:15.0pt" height="20">
<th>2008 R2 Default Before KB299261</th>
<th>2008 R2 Default After KB299261</th>
</tr>
</thead>
<tbody>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_128_CBC_SHA256</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_128_CBC_SHA</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_256_CBC_SHA256</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P521</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_256_CBC_SHA</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P521</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_RC4_128_SHA</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P521</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_3DES_EDE_CBC_SHA</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P521</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256</td>
<td class="xl66">TLS_DHE_RSA_WITH_AES_256_GCM_SHA384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P384</td>
<td class="xl66">TLS_DHE_RSA_WITH_AES_128_GCM_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256</td>
<td class="xl66">TLS_RSA_WITH_AES_256_GCM_SHA384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P384</td>
<td class="xl66">TLS_RSA_WITH_AES_128_GCM_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P521</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P256</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P521</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P256</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384_P521</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P384</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384_P384</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P521</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P256</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P521</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P384</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P521</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P256</td>
<td class="xl66">TLS_RSA_WITH_NULL_MD5</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P384</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_128_CBC_SHA256</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_128_CBC_SHA</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_256_CBC_SHA256</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_256_CBC_SHA</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_RC4_128_MD5</td>
<td class="xl66">TLS_RSA_WITH_AES_256_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">SSL_CK_RC4_128_WITH_MD5</td>
<td class="xl66">TLS_RSA_WITH_AES_128_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">SSL_CK_DES_192_EDE3_CBC_WITH_MD5</td>
<td class="xl66">TLS_RSA_WITH_AES_256_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_NULL_SHA256</td>
<td class="xl66">TLS_RSA_WITH_AES_128_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_NULL_SHA</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_256_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_128_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_256_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_128_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">TLS_RSA_WITH_3DES_EDE_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">TLS_RSA_WITH_RC4_128_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">TLS_RSA_WITH_RC4_128_MD5</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">TLS_RSA_WITH_NULL_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">TLS_RSA_WITH_NULL_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">SSL_CK_RC4_128_WITH_MD5</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">SSL_CK_DES_192_EDE3_CBC_WITH_MD5</td>
</tr>
</tbody>
</table>
<br>
And <strong>2012</strong> (not R2): <br>
<br>
<table style="width: 800px; height: 780px;" border="0" cellpadding="0" cellspacing="0">
<colgroup><col style="mso-width-source:userset;mso-width-alt:10934;width:224pt"
width="299"> <col style="mso-width-source:userset;mso-width-alt:10788;width:221pt"
width="295"> </colgroup>
<thead>
<tr style="height:15.0pt" height="20">
<th>2012 Default Before KB299261</th>
<th>2012 Default After KB299261</th>
</tr>
</thead>
<tbody>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_128_CBC_SHA256</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_128_CBC_SHA</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_256_CBC_SHA256</td>
<td class="xl66">TLS_DHE_RSA_WITH_AES_256_GCM_SHA384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_256_CBC_SHA</td>
<td class="xl66">TLS_DHE_RSA_WITH_AES_128_GCM_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_RC4_128_SHA</td>
<td class="xl66">TLS_RSA_WITH_AES_256_GCM_SHA384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_3DES_EDE_CBC_SHA</td>
<td class="xl66">TLS_RSA_WITH_AES_128_GCM_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P384</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P384</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P256</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P256</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P384</td>
<td class="xl66">TLS_RSA_WITH_AES_256_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384_P384</td>
<td class="xl66">TLS_RSA_WITH_AES_128_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P256</td>
<td class="xl66">TLS_RSA_WITH_AES_256_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P384</td>
<td class="xl66">TLS_RSA_WITH_AES_128_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P256</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P384</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_128_CBC_SHA256</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_128_CBC_SHA</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_256_CBC_SHA256</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_256_CBC_SHA</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_RC4_128_MD5</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">SSL_CK_RC4_128_WITH_MD5</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_256_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">SSL_CK_DES_192_EDE3_CBC_WITH_MD5</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_128_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_NULL_SHA256</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_256_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_NULL_SHA</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_128_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">TLS_RSA_WITH_3DES_EDE_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">TLS_RSA_WITH_RC4_128_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">TLS_RSA_WITH_RC4_128_MD5</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">TLS_RSA_WITH_NULL_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">TLS_RSA_WITH_NULL_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">SSL_CK_RC4_128_WITH_MD5</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20"><br>
</td>
<td class="xl66">SSL_CK_DES_192_EDE3_CBC_WITH_MD5</td>
</tr>
</tbody>
</table>
<br>
2012 R2 is unchanged since the aforementioned April patch. <br>
<br>
The point is that you should ensure your applications & clients don't
have an issue with the cipher suite re-order. It's unlikely that your apps
will have a problem but worthwhile to do a quick connectivity check in a
test environment to be sure. If you do have issues you can re-order your
suites after the patch by manipulating the registry keys listed <a href="https://support.microsoft.com/kb/2992611">here</a>
(not necessarily deleting the keys they list); use the before/after
information above for reference. For more information about prioritizing
cipher suites, see <a href="http://msdn.microsoft.com/en-us/library/windows/desktop/aa374757%28v=vs.85%29.aspx">this</a>.
<br>
<br>
It's unfortunate that they didn't communicate this change as it may have
unanticipated impacts. Here's hoping they return to their generally good
communication in the next cycle. <br>
<br>
<h3>Update 11/18/2014:</h3>
<br>
As I'm sure you've heard, Microsoft has <a href="http://www.infoworld.com/article/2849357/microsoft-windows/microsoft-ms14-066kb-2992611-schannel-ms14-068kb-3011780-kb-3000850.html">released
three patches today</a>, one of which is a re-issue of the patch mentioned
in this post. The re-issue removes the four newly added cipher suites as
there have been multiple problems reported with them. Microsoft also updated
<a href="https://support.microsoft.com/kb/2992611">their article</a> to
include the following statement: <br>
<br>
"<em>Customers who customized their cipher suite priority list should review
their list after they apply this update to make sure that the sequence
meets their expectations.<br>
<br>
Removing these cipher suites does not affect the security updates that are
part of this release. On November 18, 2014, a new secondary package was
added to the release for Windows Server 2008 R2 and Windows Server 2012 to
achieve this. This new package is update 3018238, and it will install
automatically and transparently together with security update 2992611. It
will appear separately in the list of installed updates. If you already
have security update 2992611 installed, you will notice that security
update 2992611 will be reoffered (for Windows Server 2008 R2 or Windows
Server 2012 only) by Windows Update or by Windows Server Update Services
(WSUS) to make sure that update 3018238 is also installed.<br>
<br>
The cipher suites may be re-added to the default priority list in a future
release after the community has had an opportunity to make sure of correct
execution in all customer scenarios.</em>"<br>
<br>
I've just re-applied the newly released patches using WSUS to evaluate them.
Note that you must re-appy using the same method you originally applied
with, meaning that if you downloaded manually you would need to repeat that,
and if you applied via WSUS you would need to use that methodology. Upon
reviewing the "new" cipher suite order I was both surprised and happy with
what I found: <br>
<br>
<table style="width: 800px; height: 1000px;" border="0" cellpadding="0" cellspacing="0">
<colgroup><col style="mso-width-source:userset;mso-width-alt:10971;
width:225pt" span="2" width="300"> </colgroup>
<tbody>
<tr style="height:15.0pt" height="20">
<td class="xl67" style="height: 15pt; width: 225pt; text-align: center;"
height="20" width="300"><span style="font-weight: bold;">Server 2008
R2 2992611 Patch 1 (11/14)</span></td>
<td class="xl67" style="width: 225pt; text-align: center;" width="300"><span
style="font-weight: bold;">Server 2008 R2 2992611 Patch 2 (11/18)</span></td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256</td>
<td class="xl66">TLS_RSA_WITH_AES_128_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384</td>
<td class="xl66">TLS_RSA_WITH_AES_128_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P521</td>
<td class="xl66">TLS_RSA_WITH_AES_256_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P521</td>
<td class="xl66">TLS_RSA_WITH_AES_256_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P521</td>
<td class="xl66">TLS_RSA_WITH_RC4_128_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P521</td>
<td class="xl66">TLS_RSA_WITH_3DES_EDE_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_RSA_WITH_AES_256_GCM_SHA384</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_RSA_WITH_AES_128_GCM_SHA256</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_256_GCM_SHA384</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_128_GCM_SHA256</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P521</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P384</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P521</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384_P521</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P384</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P521</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P521</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P521</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_NULL_MD5</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P384</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_128_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_128_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_256_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_256_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P384</td>
<td class="xl66">TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_256_CBC_SHA256</td>
<td class="xl66">TLS_RSA_WITH_RC4_128_MD5</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_128_CBC_SHA256</td>
<td class="xl66">SSL_CK_RC4_128_WITH_MD5</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_256_CBC_SHA</td>
<td class="xl66">SSL_CK_DES_192_EDE3_CBC_WITH_MD5</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_128_CBC_SHA</td>
<td class="xl66">TLS_RSA_WITH_NULL_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P384</td>
<td class="xl66">TLS_RSA_WITH_NULL_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P256</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384_P384</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P256</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P256</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P384</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P256</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P384</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_256_CBC_SHA256</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_128_CBC_SHA256</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_256_CBC_SHA</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_128_CBC_SHA</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_3DES_EDE_CBC_SHA</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_RC4_128_SHA</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_RC4_128_MD5</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_NULL_SHA256</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_NULL_SHA</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">SSL_CK_RC4_128_WITH_MD5</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">SSL_CK_DES_192_EDE3_CBC_WITH_MD5</td>
<td class="xl66"><br>
</td>
</tr>
</tbody>
</table>
<br>
And now <strong>2012</strong>:<br>
<br>
<table style="width: 800px; height: 780px;" border="0" cellpadding="0" cellspacing="0">
<colgroup><col style="mso-width-source:userset;mso-width-alt:10166;width:209pt"
width="278"> <col style="mso-width-source:userset;mso-width-alt:9837;width:202pt"
width="269"> </colgroup>
<tbody>
<tr style="height:15.0pt" height="20">
<td class="xl67" style="height: 15pt; width: 209pt; text-align: center;"
height="20" width="278"><span style="font-weight: bold;">Server 2012
2992611 Patch 1 (11/14)</span></td>
<td class="xl67" style="width: 202pt; text-align: center;" width="269"><span
style="font-weight: bold;">Server 2012 2992611 Patch 2 (11/18)</span></td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256</td>
<td class="xl66">TLS_RSA_WITH_AES_128_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384</td>
<td class="xl66">TLS_RSA_WITH_AES_128_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_RSA_WITH_AES_256_GCM_SHA384</td>
<td class="xl66">TLS_RSA_WITH_AES_256_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_RSA_WITH_AES_128_GCM_SHA256</td>
<td class="xl66">TLS_RSA_WITH_AES_256_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_256_GCM_SHA384</td>
<td class="xl66">TLS_RSA_WITH_RC4_128_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_128_GCM_SHA256</td>
<td class="xl66">TLS_RSA_WITH_3DES_EDE_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P384</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P384</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P384</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P384</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_256_CBC_SHA256</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_128_CBC_SHA256</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_256_CBC_SHA</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_128_CBC_SHA</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P384</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P256</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384_P384</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_128_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P256</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_128_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P256</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_256_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P384</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_256_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P256</td>
<td class="xl66">TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P384</td>
<td class="xl66">TLS_RSA_WITH_RC4_128_MD5</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_256_CBC_SHA256</td>
<td class="xl66">SSL_CK_RC4_128_WITH_MD5</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_128_CBC_SHA256</td>
<td class="xl66">SSL_CK_DES_192_EDE3_CBC_WITH_MD5</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_256_CBC_SHA</td>
<td class="xl66">TLS_RSA_WITH_NULL_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_128_CBC_SHA</td>
<td class="xl66">TLS_RSA_WITH_NULL_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_3DES_EDE_CBC_SHA</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_RC4_128_SHA</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_RC4_128_MD5</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_NULL_SHA256</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_NULL_SHA</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">SSL_CK_RC4_128_WITH_MD5</td>
<td class="xl66"><br>
</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">SSL_CK_DES_192_EDE3_CBC_WITH_MD5</td>
<td class="xl66"><br>
</td>
</tr>
</tbody>
</table>
<br>
Again, no changes for 2012 R2. If the above looks familiar, good eye.
They're the same as pre-patch: <br>
<br>
<strong>Server 2008</strong> "patch 2" vs. no patch: <br>
<br>
<table style="width: 802px; height: 620px;" border="0" cellpadding="0" cellspacing="0">
<colgroup><col style="mso-width-source:userset;mso-width-alt:10971;
width:225pt" span="2" width="300"> </colgroup>
<tbody>
<tr style="height:15.0pt" height="20">
<td class="xl67" style="height: 15pt; width: 225pt; text-align: center;"
height="20" width="300"><span style="font-weight: bold;">Server 2008
R2 2992611 Patch 2 (11/18)</span></td>
<td class="xl67" style="width: 225pt; text-align: center;" width="300"><span
style="font-weight: bold;">Server 2008 R2 Before Either 2992611
Patch</span></td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_128_CBC_SHA256</td>
<td class="xl66">TLS_RSA_WITH_AES_128_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_128_CBC_SHA</td>
<td class="xl66">TLS_RSA_WITH_AES_128_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_256_CBC_SHA256</td>
<td class="xl66">TLS_RSA_WITH_AES_256_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_256_CBC_SHA</td>
<td class="xl66">TLS_RSA_WITH_AES_256_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_RC4_128_SHA</td>
<td class="xl66">TLS_RSA_WITH_RC4_128_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_3DES_EDE_CBC_SHA</td>
<td class="xl66">TLS_RSA_WITH_3DES_EDE_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P384</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P384</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P256</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P256</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P384</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384_P384</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P256</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P384</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P256</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P384</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_128_CBC_SHA256</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_128_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_128_CBC_SHA</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_128_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_256_CBC_SHA256</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_256_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_256_CBC_SHA</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_256_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA</td>
<td class="xl66">TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_RC4_128_MD5</td>
<td class="xl66">TLS_RSA_WITH_RC4_128_MD5</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">SSL_CK_RC4_128_WITH_MD5</td>
<td class="xl66">SSL_CK_RC4_128_WITH_MD5</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">SSL_CK_DES_192_EDE3_CBC_WITH_MD5</td>
<td class="xl66">SSL_CK_DES_192_EDE3_CBC_WITH_MD5</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_NULL_SHA256</td>
<td class="xl66">TLS_RSA_WITH_NULL_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_NULL_SHA</td>
<td class="xl66">TLS_RSA_WITH_NULL_SHA</td>
</tr>
</tbody>
</table>
<br>
<strong>Server 2012</strong> "patch 2" vs. no patch: <br>
<br>
<table style="width: 800px; height: 620px;" border="0" cellpadding="0" cellspacing="0">
<colgroup><col style="mso-width-source:userset;mso-width-alt:9837;
width:202pt" span="2" width="269"> </colgroup>
<tbody>
<tr style="height:15.0pt" height="20">
<td class="xl67" style="height: 15pt; width: 202pt; text-align: center;"
height="20" width="269"><span style="font-weight: bold;">Server 2012
2992611 Patch 2 (11/18)</span></td>
<td class="xl67" style="width: 202pt; text-align: center;" width="269"><span
style="font-weight: bold;">Server 2012 Before Either 2992611 Patch</span></td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_128_CBC_SHA256</td>
<td class="xl66">TLS_RSA_WITH_AES_128_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_128_CBC_SHA</td>
<td class="xl66">TLS_RSA_WITH_AES_128_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_256_CBC_SHA256</td>
<td class="xl66">TLS_RSA_WITH_AES_256_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_AES_256_CBC_SHA</td>
<td class="xl66">TLS_RSA_WITH_AES_256_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_RC4_128_SHA</td>
<td class="xl66">TLS_RSA_WITH_RC4_128_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_3DES_EDE_CBC_SHA</td>
<td class="xl66">TLS_RSA_WITH_3DES_EDE_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P384</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P384</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384</td>
<td class="xl66">TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P256</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P256</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P384</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384_P384</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P256</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P384</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P256</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P384</td>
<td class="xl66">TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P384</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_128_CBC_SHA256</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_128_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_128_CBC_SHA</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_128_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_256_CBC_SHA256</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_256_CBC_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_AES_256_CBC_SHA</td>
<td class="xl66">TLS_DHE_DSS_WITH_AES_256_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA</td>
<td class="xl66">TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_RC4_128_MD5</td>
<td class="xl66">TLS_RSA_WITH_RC4_128_MD5</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">SSL_CK_RC4_128_WITH_MD5</td>
<td class="xl66">SSL_CK_RC4_128_WITH_MD5</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">SSL_CK_DES_192_EDE3_CBC_WITH_MD5</td>
<td class="xl66">SSL_CK_DES_192_EDE3_CBC_WITH_MD5</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_NULL_SHA256</td>
<td class="xl66">TLS_RSA_WITH_NULL_SHA256</td>
</tr>
<tr style="height:15.0pt" height="20">
<td class="xl66" style="height:15.0pt" height="20">TLS_RSA_WITH_NULL_SHA</td>
<td class="xl66">TLS_RSA_WITH_NULL_SHA</td>
</tr>
</tbody>
</table>
<br>
So as it pertains to the cipher suite order, we're right back where we
started. I'm sure this will alleviate some of the issues some customers
encountered. Keep in mind that there is more to this patch (binaries) than
the cipher suite re-order, so to echo the previous assesment it should be
installed. <br>Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com3tag:blogger.com,1999:blog-5458589696790815336.post-41570633922648844982014-11-11T21:16:00.001-06:002014-11-11T21:16:47.506-06:00Software Review: Altaro Hyper-V Backup <p>Backups can be a tough topic to get excited about. In a forward-looking
technology field even needing to utilize your backup platform is generally
a sign that something went wrong. As a result I haven't delved into the
topic of backups for awhile. I'm taking a break from that dry spell to
examine a Hyper-V backup solution, <a href="http://www.altaro.com/hyper-v-backup/">Altaro
Hyper-V Backup</a>. </p>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj43VOF-Dvstcj2EVydzLcupAQReLyD6hk0Nys0SS877x0JNW7t5VgwxNxMWM4O7UYTxUZrLvyjIzLQuCR9OZtISHu9S8sU1ZEzzqPzVwctrVirpPJrqnhwU57MVdkl06FRhgo-aVkoBW8/s1600/Header2.jpg"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj43VOF-Dvstcj2EVydzLcupAQReLyD6hk0Nys0SS877x0JNW7t5VgwxNxMWM4O7UYTxUZrLvyjIzLQuCR9OZtISHu9S8sU1ZEzzqPzVwctrVirpPJrqnhwU57MVdkl06FRhgo-aVkoBW8/s1600/Header2.jpg"
border="0"></a></div>
<br>
<p> Being that my local lab is Hyper-V based, I was able to take Altaro
through its paces without impacting a production environment; something
affording me the freedom to explore most of the features. Speaking of
features (that's a $2 segue) let's examine the high level features offered
by Altaro: </p>
<ul>
<li>Support for backup of Hyper-V virtual machines</li>
<li>Fully Hypervisor/VSS-based, meaning no worries regarding open files or
system state</li>
<li>Straightforward GUI-Driven backup schedule and retention policies</li>
<li>Offsite replication to another Altaro target (no additional license
required) on a defined schedule, honoring retention policies</li>
<li>Seeding to offsite location using physical disk if desired</li>
<li>Agentless</li>
<li>Granular Restore:</li>
<ul>
<li>File System Level</li>
<li>Exchange Items</li>
</ul>
<li>"Sandbox" Restore for testing backup images</li>
<ul>
<li>Automated sandbox scheduling</li>
</ul>
<li>Cluster Shared Volume support</li>
<li>Boot directly from backup for testing, etc. </li>
<li>E-Mail based alerting</li>
</ul>
<p>Taking the target market (more on that below) into consideration I will
set my priorities as follows (in order of importance):</p>
<ul>
<li>Reliability/General Quality</li>
<li>Labor investment to set up and maintain</li>
<li>Alerting Effectiveness (Not too noisy, alerting when necessary) </li>
<li>Cost</li>
<li>Basic Features (Granular Restore, etc.) </li>
<li>Advanced Features (VM Sandbox Restore, etc.)</li>
</ul>
<p>Having spent over half my career in consulting I've been fortunate to see
all sorts of environments. In my experience the main constraint for
companies targeted by Altaro is usually time (labor) and potentially cost.
That coupled with a predominant and understandable lack of enthusiasm for
backup solutions makes the ease of setup and management crucial to ensure
successful backups. </p>
<h3>Target Market</h3>
<p>Altaro Hyper-V backup supports Hyper-V servers and is likely most
applicable at small to medium size businesses. Traditional backups of
physical servers are not supported, which makes sense given the market gap
Altaro aims to fill. The pricing is <a href="http://www.altaro.com/hyper-v-backup/pricing.php">very
competitive</a> (more below) and comes in <a href="http://www.altaro.com/hyper-v-backup/pricing.php">three
versions</a>: Free, Standard, and Unlimited. </p>
<p>This isn't to say the product may not be applicable in a larger
enterprise, but the tools aren't geared to manage a huge number of servers
and most enterprises are not exclusively utilizing Hyper-V for their
virtualization needs. This review is written with this in mind. As with
all things in IT you're looking for the right tool for the job. </p>
<h3>Test Systems</h3>
<p>I've tested this product in my lab on two Hyper-V hosts, one <a href="http://technet.microsoft.com/en-us/library/jj643303.aspx">2012
R2 Storage Server</a> (with Hyper-V) and one <a href="http://msdn.microsoft.com/en-us/library/dd184075.aspx">2012
Core</a> Hyper-V host. While storage server isn't on the Altaro list of
supported platforms I experienced no problems with it. Collectively this
environment has 16 threads, 64GB RAM, and 12TB of various storage. There
are 14 VMs eligible for backup including different Windows versions and
two flavors of Linux. For test purposes my offsite backup target was an
IaaS VM hosted in Windows Azure (Note: due to cost I do not recommend
using Azure IaaS as a backup target in production).</p>
<h3>Disclosures/Scoring</h3>
<p>When reviewing I feel that full disclosure is critical to maintaining
journalistic integrity. Altaro contacted me and asked if I would review
their software; in the market myself for lab backup software I agreed to
assess it. They provided me with a license to perform the appropriate
testing. I did not accept any payment for this article and as always my
opinions are my own.</p>
<p>I use a two-axes scoring square similar in design and function to the
Gartner "<a href="http://www.gartner.com/technology/research/methodologies/research_mq.jsp">Magic
Quadrant</a>". I use this when the testing is subjective and the target
market of the product is not universal. This allows me to attempt to
assess quality in two ways: </p>
<ul>
<li>Implementation Quality: The quality of the features discussed in this
section, but not relevant to the quantity or specific functionality of
the features. </li>
<li>Feature Set: The quantity and specific functionality of the features
discussed in this section, but not relevant to the quality. </li>
</ul>
<p>Higher on the scale indicates better fit and finish, stability, and
general quality. Farther right on the scale indicates a more
extensive and useful feature set. The best position would be the top right
and the worst the bottom left. It is possible that a given piece of
software may have great features but is very buggy, few features but very
stable, etc. I believe this scoring methodology allows me to communicate
the overall software quality more effectively. </p>
<p>In some cases I will eliminate an axis from scoring if the category
warrants it. In this review, "cost" is the only single axis score. </p>
<ul>
</ul>
<h3>Installation/Basic Configuration</h3>
<p>Installation couldn't be much more straightforward; the installer even <a
href="http://support.altaro.com/customer/portal/articles/877533-how-can-i-install-on-hyper-v-server-server-core-">works
on server core</a>. There are no special service account requirements;
since the agent runs on each Hyper-V host it utilizes the built-in "<a href="https://support.microsoft.com/kb/120929?wa=wsignin1.0">SYSTEM</a>"
principal. Normally I wouldn't care for the use of the admin-equivalent <a
href="https://support.microsoft.com/kb/120929?wa=wsignin1.0">SYSTEM</a>
principal, but since this is backup software there isn't much wiggle room
access wise. </p>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjDRErsKTkqg-qUYnLEUxJnisCr4V_P1KJJRnCOMXClT02rdDFjEfTST_gUs9jloxZAL0pbsVzBwYDH5rmwLDDZGWiKMm8mtCn1vNIkjBw6YJTkovJMUI7tPxKkByZSMgEbat5bHf7wmE/s1600/2014-11-07+23_15_32-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjDRErsKTkqg-qUYnLEUxJnisCr4V_P1KJJRnCOMXClT02rdDFjEfTST_gUs9jloxZAL0pbsVzBwYDH5rmwLDDZGWiKMm8mtCn1vNIkjBw6YJTkovJMUI7tPxKkByZSMgEbat5bHf7wmE/s1600/2014-11-07+23_15_32-RD+Tabs+64.png"
border="0"></a></div>
<p>After installation on each Hyper-V host you will want to create a
centralized management console if you have more than one host. This can be
done on a management workstation if you desire; just install the free
version on your workstation. After installation, connect to the local
agent and select "Central Management Console" from the left pane and click
"yes" when asked if you would like to create a shortcut. </p>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRfr2WXKT6ueUv6Y3iR7unycQt3kwToDQwJ2ccYy3fTjfh81SZcGH57kdXOa8f0P90loAxslNWw2xZoZQtc-Nu7qunMGq8Cot44CBN7JIHb_gcgJSeUBGzlxAQ7lMAFKnd2I9rJ_1Nzwo/s1600/2014-10-27+11_16_19-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRfr2WXKT6ueUv6Y3iR7unycQt3kwToDQwJ2ccYy3fTjfh81SZcGH57kdXOa8f0P90loAxslNWw2xZoZQtc-Nu7qunMGq8Cot44CBN7JIHb_gcgJSeUBGzlxAQ7lMAFKnd2I9rJ_1Nzwo/s1600/2014-10-27+11_16_19-RD+Tabs+64.png"
border="0"></a></div>
<br>
<p>From henceforth you can use the central console if desired. </p>
<p>Assuming your firewall rules were created and enabled you should be able
to click "Add Agent to Group" to add each host. If you have multiple sites
make sure you create appropriate groups for each and then add agents as
appropriate. </p>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgXFS7qNWfzfzWZVJBKvNnHm8gr6SNEB_YY5zPIdHWtm0ezZhTU5Z_-GPvvoesQxMPpQUEl7HKMA0pg-lmRbBoUaNVZWQvCyrsVCpnOc6fOwtk5jnlyls7GbKvMxdw3S4eYCsDmTDolVE8/s1600/2014-10-27+12_19_50-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgXFS7qNWfzfzWZVJBKvNnHm8gr6SNEB_YY5zPIdHWtm0ezZhTU5Z_-GPvvoesQxMPpQUEl7HKMA0pg-lmRbBoUaNVZWQvCyrsVCpnOc6fOwtk5jnlyls7GbKvMxdw3S4eYCsDmTDolVE8/s1600/2014-10-27+12_19_50-RD+Tabs+64.png"
border="0" height="123" width="320"></a></div>
<p>Now you can connect to and manage each host in your environment from a
central location. Note that you will need to launch a console for each
host, but in a small/med environment this should be acceptable as setup is
a one-time occurrence per host. For each host you can now configure the
following: </p>
<ul>
<li>VMs to backup</li>
<li>Primary backup location</li>
<li>Backup schedules and retention</li>
<li>Offsite backup target (see install note below)</li>
<li>Notification</li>
<li>Encryption key</li>
</ul>
<p>Offsite install note: Altaro also provides a WAN target only client that
does not count towards your license. Installation is simple; just click
through, create a user account for your servers, and open firewall ports
as necessary. The software is available <a href="http://www.altaro.com/hyper-v-backup/download-tools.php">here</a>.
</p>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6m1nVU4DB_aiwQSsYpn5Jcqb0_Y5oigrGrzFVj4g5mnrRLeCED8KtbJk5MPvENo4Z0mb6NIbOqG01rs4U9N9Ps2diE43ewLoYLtDJdNpvRiOsIFBVtgxfLlmkMi2De_pCEN2jYzPunHs/s1600/2014-10-26+22_22_10-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img style="width: 697px; height: 397px;"
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6m1nVU4DB_aiwQSsYpn5Jcqb0_Y5oigrGrzFVj4g5mnrRLeCED8KtbJk5MPvENo4Z0mb6NIbOqG01rs4U9N9Ps2diE43ewLoYLtDJdNpvRiOsIFBVtgxfLlmkMi2De_pCEN2jYzPunHs/s1600/2014-10-26+22_22_10-RD+Tabs+64.png"
border="0"></a></div>
<p>When configuring the encryption key, make sure you do so right away as
doing so later will invalidate your previous backups; there is no support
for rotating and retaining encryption keys, but based on the design that
is a good thing (see below as to why). After that you'll want to finish up
by specifying your base schedules and retention as well as notifications
and offsite details. </p>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8M9NSDQBbkU4fV7I-iNEeeBkYzktz3bCw45BnLJN4PvMpzW8xqG-CgyQMNb1w2gtDePKXtj5ftvFrrEcOVaiHuuxk_1dA12dsY4a6v35SUDKt5D8V01y94iGPaNdNO6wo3VMjYnPHb9w/s1600/2014-10-27+22_04_43-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img style="width: 477px; height: 200px;"
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8M9NSDQBbkU4fV7I-iNEeeBkYzktz3bCw45BnLJN4PvMpzW8xqG-CgyQMNb1w2gtDePKXtj5ftvFrrEcOVaiHuuxk_1dA12dsY4a6v35SUDKt5D8V01y94iGPaNdNO6wo3VMjYnPHb9w/s1600/2014-10-27+22_04_43-RD+Tabs+64.png"
border="0"></a></div>
<p>Now let's address the review categories:</p>
<h3>Reliability/General Quality</h3>
<p>As stated, this is the most critical point for backup software. If it
were buggy and didn't work consistently it wouldn't be worth using. I used
the software for just over a month, performing onsite and offsite backups
regularly, restoring full VMs and more granular items, and changing
settings to exercise the software. Backup targets ranged from local disk
through UNC paths and on to attached, dedicated storage. During that month
I did not encounter any issues that concerned me. There were a couple very
minor issues such as backups failing if the administrative console had a
lock on previous backup files associated with the VM in question; these
were to be expected and were addressed by support right away. </p>
<p>Performing a basic restore is almost *too* easy (not that I'm docking
points). You can select which VM to restore, if you want it restored to a
clone or to the original location, backup sources other than the main, as
well as which day you want it restored to. As an added bonus, Altaro can
automatically disable the NIC when restoring to avoid IP address
conflicts. After a non-overwrite restore Altaro even adds the clone to the
console of the Hyper-V host to which you restored! When utilizing any of
these options I found the operation quick and encountered no problems
whatsoever.</p>
<p>As for software updates, the Altaro console will check for updates and
notify when they are released. The GUI provides an upgrade link and takes
you directly to the download page. After downloading and running the
setup, the installer handled the upgrade process flawlessly, automatically
closing open executables and retaining settings. While it would be nice if
the console facilitated the download & install directly, the execution
of the existing system was flawless and in-line with the best its
competitors have to offer. </p>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhIZ9ZHwzMamzj8OcJA46udQBrfEtSSqid7c7LZnRQQzA0YgGVqwcvOwZm6m_D5pTlAH8f2hQHJYqD2RsWIQ-L9MEFJ48ha9SFQigisYx6m2ub9BgSMnH9KYL8YsBH8rbHvE23xDQFmNM0/s1600/2014-10-26+22_56_39-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img style="width: 688px; height: 422px;"
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhIZ9ZHwzMamzj8OcJA46udQBrfEtSSqid7c7LZnRQQzA0YgGVqwcvOwZm6m_D5pTlAH8f2hQHJYqD2RsWIQ-L9MEFJ48ha9SFQigisYx6m2ub9BgSMnH9KYL8YsBH8rbHvE23xDQFmNM0/s1600/2014-10-26+22_56_39-RD+Tabs+64.png"
border="0"></a></div>
<br>
<p>Offsite backups can be a tricky thing to coordinate and pay for;
fortunately Altaro has you covered there as well. Both over the network
(internet) and disk seeding is supported in the standard and unlimited
versions. Disk seeding allows you to push backups to disk and then send
that disk to your offsite location rather than seed the initial full
backups, which can be very useful when sending out many, many GBs of data
for the first go around. After that you can rely on WAN/internet based
uploads for deltas. There is a special software component you use as the
backup target that does not require an additional license. This component
can be installed on a Windows machine; keep in mind you'll need a network
port opened and forwarded to the machine for it to work. To enable offsite
backups Altaro requires that you set an encryption key, which is a welcome
restriction since we don't want to pump unencrypted machine images over
the internet. Once enabled you can schedule automatic offsite backups
anywhere between once a month and every day. Note that if selecting the
weekly/daily backup option in your schedule for normal backups you're
locked to once a week at a minimum for offsite backups. It would be nice
to be able to schedule the offsite backups less often in that case, but
that's not too big a deal. Retention can be set independently between on
and off-site. The only other thing I'd like to see here that isn't in the
software at the time of review is offsite upload bandwidth throttling...
perhaps in a future version. </p>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhx2w1L9prtB4yzzryPzv2OFTA2FEGw6i761qz6D8bc_nLobXqXz3Ps5k6XoC2l7gIFD6ejv-97rQ9fzvifOssVxBhkT7IxYUiWNNI6OvuSyavhWeMxe8VRQWMhhLB2-DtoIfkOBREK4EE/s1600/2014-11-10+22_54_15-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img style="width: 472px; height: 319px;"
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhx2w1L9prtB4yzzryPzv2OFTA2FEGw6i761qz6D8bc_nLobXqXz3Ps5k6XoC2l7gIFD6ejv-97rQ9fzvifOssVxBhkT7IxYUiWNNI6OvuSyavhWeMxe8VRQWMhhLB2-DtoIfkOBREK4EE/s1600/2014-11-10+22_54_15-RD+Tabs+64.png"
border="0"></a></div>
<br>
<p>Altaro also supports "reverse delta" which ensures that when performing
incremental backups the latest backup taken contains the full image rather
than the traditional concept of a full backup followed by deltas until the
next full backup. This is similar to a feature in Veeam to accomplish the
same thing, and one of my favorite things about this product. This means
that the net size for cross site replication is limited to the delta size
and the concept of regular "full" backups is obsolete. Note you still
maintain the ability to restore to an older state; Altaro just down-revs
the image as necessary when you request a restore. </p>
<p>Unfortunately there is no backup level de-duplication across multiple VM
backups, nor is there support for OS level de-duplication at this time.
The lack of de-duplication for regular offsite backups (bandwidth) is
mitigated by the aforementioned reverse delta methodology however. </p>
<p>Hyper-V <a href="http://technet.microsoft.com/en-us/library/jj134199.aspx">share
nothing live migration</a> does pose a couple interesting challenges;
since the schedules and previous backups for a particular VM are tied to a
host they don't follow a migration to a different host. They do remain on
the originating host and work fine after migrating the VM back to the
original host. The implications of this are:</p>
<ul>
<li>The originating host will throw errors until the backup schedule for
the missing VM is disabled. </li>
<li>Restores on the new host will not be available. </li>
<li>If you backup the VM on the new host, the initial backup will be full
and offsite replication will be impacted as such. </li>
</ul>
<p>While these issues are theoretically fixable it would be a mess to do so
and I suspect it wouldn't impact customers much. The most common use case
for share nothing live migration is to maintain uptime across host
reboots, transferring the VMs back thereafter. In that case this is not an
issue; the impact in your environment will be up to you to judge. </p>
<p>A more robust Hyper-V environment would likely utilize <a href="http://technet.microsoft.com/en-us/library/dd630633%28v=ws.10%29.aspx">Hyper-V
Cluster Shared Volumes</a>, which Altaro Ultimate edition has native
support for. While I did not test this feature extensively, the
support and features is in line with what I would expect. This support is
likely more important than the share nothing live migration issues I noted
above. </p>
<p>The built in reporting is easy to consume and effective. It is not geared
for notification of eternal parties, but barring that use case I don't see
much of anything deficiency wise. The dashboard has impressively effective
visualizations of backup times and volumes on a per VM or overall basis.
The porting section has records of backups, restores, and errors. </p>
<p>Reliability and General Quality score:</p>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg3jpNOa8UX0Q3QZYyz7W2igGkG6shbas_nTEqrHM6ppamP-xPbio1EjuoSL7mp2U3ldoqhAn22jaoBrJKtKzsznqoo87SucqNXctNmIfKl45X9q5f3QZSKmIcUX2ozhE-ThsAWsQu8Fns/s1600/_Rating_Reliability.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img style="width: 200px; height: 200px;"
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg3jpNOa8UX0Q3QZYyz7W2igGkG6shbas_nTEqrHM6ppamP-xPbio1EjuoSL7mp2U3ldoqhAn22jaoBrJKtKzsznqoo87SucqNXctNmIfKl45X9q5f3QZSKmIcUX2ozhE-ThsAWsQu8Fns/s1600/_Rating_Reliability.png"
border="0"></a></div>
<br>
<h3>Labor Investment to Set Up and Maintain</h3>
<p>This is a category where Altaro really shines. Installation couldn't be
much more straightforward, and usage/maintenance is impressively easy.
This fact has many positive ramifications on a labor constrained
environment and is why I have it ranked so highly. Altaro has an
impressive collection of online documentation including <a href="http://support.altaro.com/customer/portal/topics/397045-user-guides/articles">user
guides</a>, a <a href="http://support.altaro.com/">knowledgebase/FAQ</a>,
and they maintain a <a href="http://support.altaro.com/customer/portal/topics/370968-change-logs/articles">detailed
change log</a>. </p>
<p>As a demonstration of the ease of use, take for example the setup and
association of a backup schedule: </p>
<ol>
<li>Under the schedules tab, click to create a new schedule. </li>
<li>Define schedule frequency, time, and offsite schedule. </li>
<li>Drag the desired VMs to the schedule. </li>
</ol>
<p>Most of the software is as intuitive as what you see here. I know I can
reasonably delegate responsibility for this tool without having to
provided dedicated training or even significant time in most target
environments. </p>
<p>Ease of Set Up/Maintenance score: </p>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi09LL_f3eUx_vYlXdpFiogairn-5kgmP_UUBUykx8RiiY3O1ZRrqOOS8skGfsNPWJf-P9aOl8RXSq70L_OcHEp3_6VyRZlk2TIUwGqosAg19mIXgZJXYCtKU5llOSfv-H7rWMkAlrSl2Q/s1600/_Rating_Setup_Maintain.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img style="width: 200px; height: 200px;"
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi09LL_f3eUx_vYlXdpFiogairn-5kgmP_UUBUykx8RiiY3O1ZRrqOOS8skGfsNPWJf-P9aOl8RXSq70L_OcHEp3_6VyRZlk2TIUwGqosAg19mIXgZJXYCtKU5llOSfv-H7rWMkAlrSl2Q/s1600/_Rating_Setup_Maintain.png"
border="0"></a></div>
<br>
<h3>Alerting Effectiveness</h3>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQTv9uwulCCzYaVV3HBatgh25whsbvp5gYUKHpCtyTLdUkNoSoeJAPV_suGe05-WlCpxd4-uYxp3B7ht5hnPTgX2FuLrKHj1E4v0woc_b32El_XtXHLgpd0kKcfAtIuIP1MbLtUNlsmfc/s1600/2014-10-26+22_28_31-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img style="width: 673px; height: 407px;"
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQTv9uwulCCzYaVV3HBatgh25whsbvp5gYUKHpCtyTLdUkNoSoeJAPV_suGe05-WlCpxd4-uYxp3B7ht5hnPTgX2FuLrKHj1E4v0woc_b32El_XtXHLgpd0kKcfAtIuIP1MbLtUNlsmfc/s1600/2014-10-26+22_28_31-RD+Tabs+64.png"
border="0"></a></div>
<br>
<p>It is critical that backup software alert effectively; a silent failure
in this space is a much larger problem waiting to happen. Altaro natively
supports individually configuring the following categories for backup
notifications: </p>
<ul>
<li>Successful Backups/Offsite Copies</li>
<li>Backups/Offsite Copies with Skipped Files</li>
<li>Failed Backups/Offsite Copies</li>
<li>Completed Restore Operations</li>
</ul>
<p>The following notification methodologies are supported:</p>
<ul>
<li>E-Mail</li>
<li>Windows Event Log</li>
</ul>
<p>Additionally, anyone connected to the console will receive alerts as
well. While e-mail notification will generally be sufficient in most
cases, large installations may want to log to the event log and use
another tool to consolidate job status and deliver one consolidated
report. I found all available notification methodologies to be reliable;
the only minor nitpick I have is that the retry interval and thus the
failure notification timing cannot be configured.</p>
<p>Alerting Effectiveness score: </p>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgPBWouVedx8SQk-B0dEiC-GfcCJ8rg8xuwQHVIAXbKfJp__sCDhG2v49nDtPOFBLQWpJHsUYns9du83ywaL4PruR2U0E8-QVDI0f4Wt2qEMc8vmNUQn9Kwao0sYz3r0Niaey4lEWDE-EU/s1600/_Rating_alerting.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img style="width: 200px; height: 200px;"
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgPBWouVedx8SQk-B0dEiC-GfcCJ8rg8xuwQHVIAXbKfJp__sCDhG2v49nDtPOFBLQWpJHsUYns9du83ywaL4PruR2U0E8-QVDI0f4Wt2qEMc8vmNUQn9Kwao0sYz3r0Niaey4lEWDE-EU/s1600/_Rating_alerting.png"
border="0"></a></div>
<br>
<h3>Cost</h3>
<p><a href="http://www.altaro.com/hyper-v-backup/pricing.php">Altaro pricing</a>
is very competitive. In my research I've found it cheaper than most of its
main competitors, and the beauty of the licensing is that it is very
straightforward. As of this writing the standard version is $395 per
Hyper-V host (5 guests per maximum, no core/socket limit) and $585 for
unlimited (unlimited VMs per guest, no core/socket limit). That price
includes one year of upgrades and support, and you can choose to purchase
additional years of upgrades and support for 25% of the purchase price
($99/$146). The free version that can backup two VMs per host is, oddly
enough, free. The free version does not support some of the more advanced
features which is to be expected. For the sake of comparison Veeam Hyper-V
essentials 2CPU Socket licenses with a year of maintenance and support
cost the following: "Standard" $820, "Enterprise" $1500, and "Enterprise
Plus" $2300. </p>
<p>Cost score:</p>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRM0pI6Naz_R7YW6nCVGwRXto-uPv3b5nMNHWR1Eu8wwOUpOd89kcH_-T3TUkzvV1624ppZdJEtx2CzPz5SMp335fyG96JDhTEtmBJ-v7Y4yHS_HUkTkIuf-Z_sAsVW7t1GiRmqFeBfkM/s1600/_Rating_cost.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img style="width: 200px; height: 200px;"
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRM0pI6Naz_R7YW6nCVGwRXto-uPv3b5nMNHWR1Eu8wwOUpOd89kcH_-T3TUkzvV1624ppZdJEtx2CzPz5SMp335fyG96JDhTEtmBJ-v7Y4yHS_HUkTkIuf-Z_sAsVW7t1GiRmqFeBfkM/s1600/_Rating_cost.png"
border="0"></a></div>
<br>
<h3>Basic Features</h3>
<p>Altaro supports granular file level restore of any drive in a backup
image; after selecting the VM you want a file from it loads up a file
browser where you can select what files or directories you would like and
where to restore them to (Standard and Unlimited). It also supports
Exchange message level restore (Unlimited version). This option scans a VM
backup image for Exchange databases in the default install locations. If
your DB is in a non-standard location you can browse to and then mount it.
You are then presented with a list of mailboxes; upon selecting one you
can restore messages by their from, time, and/or subject criteria. </p>
<p>Score: </p>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjR5fToapDxih70Gq3prcZ6ji0V3uQ_0YxliLJvbL8zklm71zuZ0roctLQ2yYHR86d9sRHpFJlIty66DJumMFBj8zLkxFXbZ9YkTLwajJJC22gVv6ecDNSX41glhVwtOwY8rdisGMgP7AA/s1600/_Rating_Basic_Features.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img style="width: 200px; height: 200px;"
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjR5fToapDxih70Gq3prcZ6ji0V3uQ_0YxliLJvbL8zklm71zuZ0roctLQ2yYHR86d9sRHpFJlIty66DJumMFBj8zLkxFXbZ9YkTLwajJJC22gVv6ecDNSX41glhVwtOwY8rdisGMgP7AA/s1600/_Rating_Basic_Features.png"
border="0"></a></div>
<br>
<h3>Advanced Features</h3>
<p>In addition to the file level and Exchange object level restores, Altaro
Standard and Unlimited support restoring to a sandbox.The Sanbox restore
is a great start; it allows you to restore & boot selected VMs on a
schedule to ensure the backups being taken are successful from a high
level. This sort of thing can be a great feature to help business lines
gain confidence in your ability to restore mission-critical virtual
machines. It would be nice if this feature supported custom notifications
(for non-IT interested parties) and eventually restoring multiple VMs to a
shared sandbox and then running automated test scripts, but that's likely
a big ask at this point in time. Point is they've already got great
functionality and are moving in the right direction. </p>
<p>Altaro also supports booting directly from a backup drive (image), but
this feature is hamstrung by the fact that it requires the backup image to
be neither compressed or encrypted. I can't imagine a scenario where you
wouldn't at least want to compress backups, so while this feature looks
good on paper it isn't practical in most situations. That said, restoring
a VM to a clone is so easy and quick that wouldn't fret too much about it.
Frankly if I were Altaro I would consider removing this feature; not
because it's not interesting but because assuming compression is the norm
it isn't practical and could serve to disappoint a customer who thought
too much of this feature on paper.</p>
<p>Score: </p>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrWnVuj5azq763CiM_whwXp8Q9KVBYfMkMaevpwZG_2F6dGJjFVKVwMOQClU5vLXP_DzISIklwbM0PvDu6fId4jWp01iNSwMQuvihDjnf0JF-hz8wiv_YipsUhLmptoWz_hvf1CzHSuKQ/s1600/_Rating_Advanced_Features.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img style="width: 200px; height: 200px;"
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrWnVuj5azq763CiM_whwXp8Q9KVBYfMkMaevpwZG_2F6dGJjFVKVwMOQClU5vLXP_DzISIklwbM0PvDu6fId4jWp01iNSwMQuvihDjnf0JF-hz8wiv_YipsUhLmptoWz_hvf1CzHSuKQ/s1600/_Rating_Advanced_Features.png"
border="0"></a></div>
<br>
<h3>Bonus Super Feature: Encryption</h3>
<p>The concept of "<a href="http://en.wikipedia.org/wiki/Trust_no_one_%28internet_security%29">Trust
No One</a>" relies on software allowing one to specify their own key (or
seed) to protect data. If implemented correctly then only the
individual(s) with the key would be able to decrypt the data. Adherence to
this concept can be critical for some businesses trusted with protecting
not only their data but the data of their customers. I'm pleased to say
that Altaro, at least in theory, supports "Trust No One" style AES
encryption for both local (ultimate only) and offsite (standard &
ultimate). </p>
<p>First a note: while the theory of trust no one results in unbreakable
encryption, the actual security relies on the details of the
implementation. While the key is specified by you, you still are actually
trusting that the software doesn't intentionally or unintentionally store
the encryption key in a way that could reveal it to anyone else. That can
be difficult and I have no way to examine the implementation specifics at
a code level. </p>
<p>That said, I'm very impressed that Altaro takes this seriously. One
should be able to rest much easier knowing that if their backups are
compromised their data is safe. To inquire about the implementation of
this encryption, I asked the following question of Altaro support: </p>
<p>"<em>If one enables onsite and remote encryption and provides their own
key, but then somehow loses that key, is there any way to recover the
backups? </em>"</p>
<p>I received this response within hours: </p>
<p>"<em>Hi Toby,<br>
With regards to encrypted backups I’m afraid that the key cannot be
recovered in any way and must thereby be remembered. This is designed
this way in order to ensure that the sensitive information contained
within the backups doesn’t fall into the wrong hands.</em>"</p>
<p>Perfect answer! This illustrates to me that not only is the software
designed to be secure, but the support department is aware of the fact
that it is and also aware of why. This makes me much more comfortable with
offsite backups in particular. This to me is one of the most important
aspects of Altaro; the fact that they get this right gives me a huge
confidence in their overall design.</p>
<p><strong>Note</strong>: Remember that as stated this encryption is
optional and not enabled by default. If you don't want to encrypt you
don't need to worry about storing a key. </p>
<h3>Conclusion</h3>
<p>When I set out to do this review I was afraid I would be one of two
things: disappointed or bored. The quality of the software prevented me
from being disappointed and the ease of use prevented me from being bored.
This tool is a powerful backup solution for a small to medium size
business implementation of a Hyper-V based private cloud, and the pricing
is right in line, if not below that level. Additionally, Altaro's support
department was impressive and their documentation is thorough. There are a
few missing features, but taking the full picture into account I consider
those minor. I have no reservations recommending Altaro for the
Hyper-V based small to medium sized business. </p>Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com3tag:blogger.com,1999:blog-5458589696790815336.post-60065901944132691452014-09-15T22:04:00.001-05:002014-09-15T23:36:26.765-05:00Using PowerShell Jobs to Trigger Remote MSI Installs <script type="text/javascript">
// toggle specific
function toggleDiv(id1,id2) {
var tag = document.getElementById(id1).style;
var tagicon = document.getElementById(id2);
if(tag.display == "none") {
tag.display = "block";
tagicon.innerHTML = " - ";
} else {
tag.display = "none";
tagicon.innerHTML = " + ";
}
}
function expandAll(cnt) {
for(var x=1; x<=cnt; x++) {
document.getElementById('content'+x).style.display="block";
document.getElementById('icon'+x).innerHTML=" - ";
}
}
function collapseAll(cnt) {
for(var x=1; x<=cnt; x++) {
document.getElementById('content'+x).style.display="none";
document.getElementById('icon'+x).innerHTML=" + ";
}
}
</script><style type="text/css">
.Clicktitle {
padding:5px;
border:1px solid #CCCCCC;
font:15px arial;
width:300px;
cursor:pointer;
height:20px;
}
.Clickitem {
padding:5px;
border:1px solid #CCCCCC;
font:12px verdana;
}
.item div {
border-bottom:1px dashed #CCCCCC;
padding:5px;
}
.button {
border:1px solid #CCCCCC;
padding:5px;
cursor:pointer;
}
</style>
So you want to deploy an <a href="http://technet.microsoft.com/en-us/library/cc978328.aspx">MSI
package</a> to potentially thousands of machines using <a href="http://blogs.msdn.com/b/powershell/">PowerShell</a>?
Odd, me too.<br>
<br>
<h3>The Goal</h3>
<br>
Sometimes package management solutions aren't the right tool for the job;
say for example you want to push/install packages as part of a single, one
time effort. This has been the case for me on more than a few contracts; we
have a piece of software we intend on distributing across a class of
machines generally not managed by <a href="http://en.wikipedia.org/wiki/System_Center_Configuration_Manager">SCCM</a>
or a similar tool. For example one may want to install something like <a href="http://www.splunk.com">Splunk</a>
on all servers in an organization. <br>
<br>
To accomplish this, my tool of choice is PowerShell. As a control mechanism
it has come a long way in the past few years. Jobs can be used for huge
deployments to asynchronously process multiple steps on many machines
simultaneously. One of the trickier things to do, however, has been to
install MSI packages as jobs using <a href="http://blogs.technet.com/b/jonjor/archive/2009/01/09/winrm-windows-remote-management-troubleshooting.aspx">WinRM</a>
(remotely). <br>
<br>
<br>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjEdfPgD8o1NC7BPs1cwUz4elV6UEnP-lErkCEN0YsOnt_cTYp3mdK5a98fzMLKuf3tNsrde9e0-_T315_m_AVEp2bxPGxOBFgEHf0ghovSJy0LtkMbEuhQPaYdlG6Q9SHhvaLXn0gyaEA/s1600/BabyFlash.jpg"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img
style="width: 536px; height: 354px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjEdfPgD8o1NC7BPs1cwUz4elV6UEnP-lErkCEN0YsOnt_cTYp3mdK5a98fzMLKuf3tNsrde9e0-_T315_m_AVEp2bxPGxOBFgEHf0ghovSJy0LtkMbEuhQPaYdlG6Q9SHhvaLXn0gyaEA/s1600/BabyFlash.jpg"
border="0"></a></div>
<br>
<h3>Concepts</h3>
<br>
Here's what we'll be working with in this article: <br>
<ul>
<li><a href="http://en.wikipedia.org/wiki/Windows_PowerShell">PowerShell</a></li>
<li>SSH (<a href="http://www.powershellmagazine.com/2014/07/03/posh-ssh-open-source-ssh-powershell-module/">1</a>)(<a
href="http://www.powershelladmin.com/wiki/SSH_from_PowerShell_using_the_SSH.NET_library">2</a>)
if managing *nix as well (for installing .deb, .etc)</li>
<li><a href="http://blogs.technet.com/b/heyscriptingguy/archive/2012/12/31/using-windows-powershell-jobs.aspx">PowerShell
Jobs</a></li>
<li><a href="http://msdn.microsoft.com/en-us/library/aa384426%28v=vs.85%29.aspx">WinRM</a></li>
<li><a href="http://en.wikipedia.org/wiki/Windows_Installer">MSI Packages</a></li>
<li><a href="http://en.wikipedia.org/wiki/Carl_Sagan#Phrase_.27billions_and_billions.27">Billions
and Billions</a> of endpoints<br>
</li>
</ul>
<h3>Approach<br>
</h3>
<br>
As is the case for all scripts that manage massive numbers of endpoints, we
need to make sure we can scale our approach. Most often this means split out
all tasks into <a href="http://msdn.microsoft.com/en-us/library/dd878288%28v=vs.85%29.aspx">jobs</a>
and move on; this includes determining platform specifics, file
distribution, and triggering installations. To accommodate this strategy
per-machine information is generally stored in <a href="http://ss64.com/ps/syntax-hash-tables.html">hash
tables</a> where it can be quickly referenced by downstream tasks. Take
the following bare-bones example; this is a subset of a script I commonly
use to distribute files. Note there is quite a bit that can be done to
enhance the functionality here; my only purpose with this is to illustrate
how to trigger and then track many jobs:<br>
<br>
<div class="Clicktitle" onclick="javascript:toggleDiv('content1','icon1');">
<span style="float: left;">Copy Jobs Example (Click to Expand)</span> <span
id="icon1" style="float: right;"> + </span> </div>
<div class="Clickitem" id="content1" style="display: none;">
<div>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">
#prerequisites include a $Connect_Success array populated with target systems that passed previous checks, a $tempdir variable to specify the directory name to copy to,
# $Forwarder64Exe and $Forwarder32Exe that represent the filenames of the x64 and x86 msi packages respectively, and $ForwarderLocation which contains the share/folder that
# the installers are stored in. $Server_Exe_Local_Path and $Server_Exe_Path are hashtables where we store the local and remote paths of the installer file on the target
# systems for later use.
Write-Debug "determining platform and downloading client software to temp directory"
[System.Collections.ArrayList]$Needs_Copy=$Connect_Success
#We need to make a one-time copy or it will throw an error when we modify it in the loop.... that can be accomplished by encapsulating the array in Parens. The loop does not track it... zoop!
Do {
foreach ($server in $($Needs_Copy))
{
$jobname="$server Copy"
Write-Debug "Looking for Job: $server Copy"
$thisjob=Get-Job $jobname -ErrorAction SilentlyContinue
if ($thisjob)
# JOB ALREADY EXISTS SO WAS STARTED IN PRIOR RUN
{
Write-Debug ("Job found and is in the following state:" + $thisjob.state)
switch ($thisjob.State)
{
"Completed"
{
if ($thisjob.HasMoreData)
{
Write-Debug "Completed but with more data, assuming failed."
$Copy_Failure+=$server
$Needs_Copy.Remove($server)
#Insert logic here to receive-job and report on the failure
#NOT REMOVING JOB YET BECAUSE WE COULD LOG FAILURE
#Remove-Job $thisjob
}
else
{
Write-Debug "Completed with no data, assuming success."
$Copy_Success+=$server
Remove-Job $thisjob
$Needs_Copy.Remove($server)
}
}
"Running"
{
Write-Debug "Pending $jobname already in queue, assuming in progress"
}
"Failed"
{
Write-Debug "Failed status."
$Copy_Failure+=$server
#Insert logic here to receive-job and report on the failure
#NOT REMOVING JOB YET BECAUSE WE COULD LOG FAILURE
#Remove-Job $thisjob
$Needs_Copy.Remove($server)
}
}
}
else
{
Write-Debug "No job found for $server, starting one..."
$sysdrive=Invoke-Command -ComputerName $server -scriptblock {$env:SystemDrive}
$sysdrive=$sysdrive.substring(0,1)
$target='\\' + $server + '\' + $sysdrive + '$' + '\' + $tempDir + '\'
#Check to see if target is 64 or 32 bit
$Platform=Invoke-Command -ComputerName $server -ScriptBlock {${env:ProgramFiles(x86)}}
If ($Platform)
{
Write-Debug "Detected $server is 64 bit"
$source="$ForwarderLocation$Forwarder64Exe"
$tmpVar=$Server_Exe_Path.Get_Item($server)
$Server_Exe_Path.Set_Item($server,"$tmpVar$Forwarder64Exe")
$tmpVar=$Server_Exe_Local_Path.Get_Item($server)
$Server_Exe_Local_Path.Set_Item($server,"$tmpVar$Forwarder64Exe")
Remove-Variable tmpVar
}
else
{
Write-Debug "Detected $server is 32 bit"
$source="$ForwarderLocation$Forwarder32Exe"
$tmpVar=$Server_Exe_Path.Get_Item($server)
$Server_Exe_Path.Set_Item($server,"$tmpVar$Forwarder32Exe")
$tmpVar=$Server_Exe_Local_Path.Get_Item($server)
$Server_Exe_Local_Path.Set_Item($server,"$tmpVar$Forwarder32Exe")
Remove-Variable tmpVar
}
Write-Debug "Copying $source to $target with job $jobname"
Start-Job -name $jobname -ScriptBlock {Copy-Item -Path $args[0] -Destination $args[1] -force} -ArgumentList @($source,$target)
if (-not($?))
{
#INSERT ERROR HANDLING HERE
}
Write-Debug "Copy job $jobname added for $server"
}
}
}
until ($Needs_Copy.count -eq 0)
</code></pre> </div>
</div>
<br>
So note in the example above the system running the PowerShell command will
launch as many threads as possible (limiting would be easy with a few lines
of code) and then circle back and check the status of each. This basic
framework can work for nearly any remote operation. Obvious enhancements to
the code above would be: <br>
<ul>
<li>Logging</li>
<li>Error handling of each condition</li>
<li>Throttling the entire operation to x outstanding jobs</li>
<li>Using a round-robin or geo associated file copy sources to distribute
load (specific to this file copy) </li>
<li>... and more!©</li>
</ul>
<h3>The Reason for This Article</h3>
<p>This framework is the basis for my "major operations" using PowerShell
and works well in many situations, however I ran into a serious problem
with using this strategy to install MSI packages. While it should be easy
to use <a href="http://technet.microsoft.com/en-us/library/hh849719.aspx">invoke-command</a>
-asjob or something similar to launch an install job as a job remotely, I
found that the tracking mechanism and the session created for the command
were often broken by the behavior of msiexec.exe. </p>
<p>As it turns out this is due to the fact that due to their layout, some
MSI packages quickly terminate the calling msiexec.exe and launch a few
instances thereafter. Since the launched instances aren't tracked as child
processes of the calling .exe, PowerShell considers the job "done" and
terminates the remote session, killing the sub processes before the
install is finished. The following solution is a modular (i.e. re-usable
code) approach to addressing this issue. </p>
<h3>Solving the Problem</h3>
<p>To solve the problem, we need to launch our own session manually and
track success with an external criteria that we devise. This can be as
complex as a specific line in a specific log file or as simple as a timer.
I won't cover all that external criteria here because that's for you to
decide. We will cover the base strategy and give an example of a
timer-based session execution. </p>
<p>Before I go into the code that does work, let's cover what doesn't. The
following remote execution strategies will not work with an MSI that
branches: </p>
<ul>
<li><a href="http://technet.microsoft.com/en-us/library/hh849719.aspx">Invoke-Command</a>
(-asjob)</li>
<li>Invoke-Command (-asjob) ; {<a href="http://technet.microsoft.com/en-us/library/hh849848.aspx">Start-Process</a>}</li>
<li><a href="http://technet.microsoft.com/en-us/library/hh849717.aspx">New-PSSession</a>
; Invoke-Command (-asjob) ; {Start-Process} ; <a href="http://technet.microsoft.com/en-us/library/hh849722.aspx">Remove-PSSession</a></li>
</ul>
<p>Here is the code that <strong>does</strong> work: </p>
<ul>
<li>New-PSSession ; Invoke-Command (-asjob) ; {Start-Process} ; wait based
on external criteria ; Remove-PSSession</li>
</ul>
<p> </p>
We'll get on with the real code example here, but before I do let me make a
note of a feature of the preceeding and to-follow code: You'll note I use
the line <em>[System.Collections.ArrayList]$Needs_Install=$Copy_Success</em>
followed by <em>foreach ($server in $($Needs_Install))</em> . The reason
for this is because this <a href="http://msdn.microsoft.com/en-us/library/system.collections.arraylist%28v=vs.110%29.aspx">.NET
array type</a>, unlike the standard PowerShell array, allows for easy
removal of elements. In the "foreach" line I enclose the array variable name
in an extra set of parens to render it a copy for each iteration, avoiding
errors when I remove an element. This allows me to use my original array as
a dynamically sized list of servers to operate on. <br>
<br>
That said, here's a code example:<br>
<br>
<div class="Clicktitle" onclick="javascript:toggleDiv('content2','icon2');">
<span style="float: left;">MSI Install Job Example (Click to Expand)</span>
<span id="icon1" style="float: right;"> + </span> </div>
<div class="Clickitem" id="content2" style="display: none;">
<div>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">
#prerequisites include a $Copy_Success array populated with target systems that passed previous checks, $Mins_Per_Job specifying how many minutes to wait per job, $argumentList which
# is used to specify the command line arguments for this installer package, $server_Install_Session_Start hashtable for storing sesion start times, and $Server_Exe_Local_Path hashtable
# that contains the installer location for each server. This is used so we can programatically determine the appropriate temp space to operate in on a per machine basis.
[System.Collections.ArrayList]$Needs_Install=$Copy_Success
Do {
foreach ($server in $($Needs_Install))
{
$jobname="$server Install"
Write-Debug "Looking for Job: $server Install"
$thisjob=Get-Job $jobname -ErrorAction SilentlyContinue
if ($thisjob)
#Job already exists so it was started in a prior run
{
Write-Debug ("Job found and is in the following state:" + $thisjob.state)
switch ($thisjob.State)
{
"Completed"
{
if ($thisjob.HasMoreData)
{
# Note: Depending on your MSI additional data may or may not be an indicator of failure. You should test yours to see.
Write-Debug "Completed but with more data, assuming failed."
$Install_Failure+=$server
$Needs_Install.Remove($server)
#NOT REMOVING JOB YET BECAUSE WE COULD LOG FAILURE
}
else
{
Write-Debug "Completed with no data, assuming success."
# Now we need to do some verification of the install since the job does not track that accurately
# this section will check for job expiration. If it was not set at the top of the script this will be skipped.
if ($Mins_Per_Job)
{
if (($server_Install_Session_Start.Get_Item($server)) -le [datetime]::Now.AddMinutes(-$Mins_Per_Job))
{
$Install_Success+=$server
$Needs_Install.Remove($server)
Remove-PSSession $server
Remove-Job $thisjob
}
else{Write-Debug "$jobname marked complete but known to do so incorrectly. Sleeping for $Mins_Per_Job minutes."}
}
ElseIf ($OtherTrackingMechanism)
{
#INSERT OTHER TRACKING LOGIC HERE
}
else
{
#No control mechanism found, assuming success and moving on.
Write-Debug "No session control mechanism found, assuming success and removing job."
$Install_Success+=$server
$Needs_Install.Remove($server)
Remove-PSSession $server
Remove-Job $thisjob
}
}
}
"Running"
{
Write-Debug "Pending $jobname already in queue, assuming in progress"
}
"Failed"
{
#We will log this as well but wont investigate why as of yet
Write-Debug "Failed status."
$Install_Failure+=$server
#NOT REMOVING JOB YET BECAUSE WE COULD LOG FAILURE
#Remove-Job $thisjob
$Needs_Install.Remove($server)
}
}
}
else
{
Write-Debug "No install job found for $server, starting one..."
# Get the installer location
$tmpVar=$Server_Exe_Local_Path.Get_Item($server)
# Start the session for this server
$session=New-PSSession -ComputerName $server
# Create the installer command line
$script=[ScriptBlock]::Create("msiexec.exe /i $tmpVar $argumentList")
# Kick it off
Invoke-Command –session $session –ScriptBlock $script -AsJob -JobName $jobname
if (-not($?)){$Install_Failure+=$server}
$server_Install_Session_Start.Set_Item($server,(Get-Date))
else
{
#do something if it works
}
}
}
}
until ($Needs_Install.count -eq 0)
</code></pre> </div>
</div>
<br>
<h3> Code Discussion</h3>
<br>
(Note: some of my variable names clearly won't make sense for your
adaptation) So, examining the code we see a couple key lines: <br>
<ul>
<li><em>[System.Collections.ArrayList]$Needs_Install=$Copy_Success </em>:
There's that .NET array type we're talking about</li>
<li><em>Do { ... }until ($Needs_Install.count -eq 0)</em> : And this is
why. This whole process takes place until the to-be-processed array is
empty. Note you could easily wrap all parts of a given install script in
a larger array for tracking all parts of the process. </li>
<li><em>foreach ($server in $($Needs_Install))</em> : Double parens makes
removal of items within the loop possible since it creates the list as a
copy rather than a reference</li>
<li><em>$session=New-PSSession -ComputerName $server</em> : Here's the
start of the session we're talking about. You could if desired use a
hash table to track session names per server if desired (<em>New-PSSession
-name ($hashtable.get_item($server))</em>)</li>
<li><em>$script=[ScriptBlock]::Create("msiexec.exe /i $tmpVar
$argumentList")</em> : create the script to be executed remotely. Note
that $tmpVar includes the machine specific location for execution. </li>
<li><em>$server_Install_Session_Start.Set_Item($server,(Get-Date))</em> :
track the install time for this endpoint. Only need this line if using
time tracking for session expiry</li>
<li><em>if ($Mins_Per_Job)</em> ; If we specify this at the top of the
script as a variable then we're using it. This allows easy code-reuse,
adding more specific completion detection routines as necessary. Note
that minutes/job would probably work in most cases where you're
processing few enough endpoints that a single machine can handle all
connections simultaneously. Once you surpass the outgoing session
capacity you'll need to be more aggressive. </li>
<li><em>if (($server_Install_Session_Start.Get_Item($server)) -le
[datetime]::Now.AddMinutes(-$Mins_Per_Job)) {....}</em> : A bit of
date logic here to test the session age. If it is past the configured
then we terminate the session, remove the job, and take other end-of-job
relative steps!</li>
</ul>
<h3>In Closing</h3>
<p>Using this methodology you can easily scale up a more complex solution
with full error tracking, verification, etc. It's amazing how far we've
come in the automation front in the last ten years, and I can't wait to see what the future holds.
For example, think of the possibilities when combined with
things like <a href="http://blogs.technet.com/b/privatecloud/archive/2013/08/30/introducing-powershell-desired-state-configuration-dsc.aspx">Desired
State Configuration</a>.</p>Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com0tag:blogger.com,1999:blog-5458589696790815336.post-84870058181058475412014-07-30T23:46:00.000-05:002014-08-17T22:00:05.594-05:00Why Schannel EventID 36888 / 36874 Occurs and How to Fix It <table style="width: 685px; height: 265px;" border="0" cellpadding="0" cellspacing="0">
<tbody>
<tr>
<td style="width: 712px; height: 265px; vertical-align: top;"><br>
<br>
Having trouble talking to your webserver? Seeing the aforementioned
errors? Are you hungry? <br>
<br>
I can fix two of those. I ran into this error at a large, highly
distributed client site. Because of the nature of the problem
(sporadic) it took longer to solve than I would have liked.
Hopefully this article will save you that time. </td>
<td style="width: 334.367px;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAOUvyDwk4t6I3hqrB-SgteR18elJHaThQ7wWQaCLKP8UGTA5VOl3riJxcizZw9B7Vu3KY9G66qBVMNKFN6McE7RVwBMhUtw0WP1_hMpHtyBuJXkQQgJCrIb5CH_U7iJKq56kxbCR6Pe8/s1600/option2.jpg"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAOUvyDwk4t6I3hqrB-SgteR18elJHaThQ7wWQaCLKP8UGTA5VOl3riJxcizZw9B7Vu3KY9G66qBVMNKFN6McE7RVwBMhUtw0WP1_hMpHtyBuJXkQQgJCrIb5CH_U7iJKq56kxbCR6Pe8/s1600/option2.jpg"
border="0" height="228" width="320"></a></td>
</tr>
</tbody>
</table>
<p><br>
</p>
<h3>What Components are Involved? </h3>
This error involves two sides: a "client" and a server. Client is in quotes
because it can be, and often is, an application consuming a web service or
similar. On the server side this problem generally occurs on Windows 2008 or
newer. The "client" can be any platform.<br>
<br>
<h3>What Errors Again? </h3>
<br>
Generally, but not always, these errors are manifested into following
events: <br>
<ul>
<li>System Log, Schannel source, EventID 36888</li>
<li>System Log, Schannel source, EventID 36874</li>
</ul>
<table style="width: 672px; height: 236px;" border="0" cellpadding="0" cellspacing="0">
<tbody>
<tr>
<td>
<div class="separator" style="clear: both; text-align: center;"> <a
href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEik7SUZ5tRDKxixGpEej7vFw0uVT1t2RqUDScVu4OXD9DtBO2v4Xzl1xqVXbNbFXderN_AePxeSln_mGkDaiN3Yzl6VGUMOEkxCLMUnwY3aABuXW3q2tesv7NubyO1FnWnnmZTVyfHKuzg/s1600/2014-07-24+23_20_48-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEik7SUZ5tRDKxixGpEej7vFw0uVT1t2RqUDScVu4OXD9DtBO2v4Xzl1xqVXbNbFXderN_AePxeSln_mGkDaiN3Yzl6VGUMOEkxCLMUnwY3aABuXW3q2tesv7NubyO1FnWnnmZTVyfHKuzg/s1600/2014-07-24+23_20_48-RD+Tabs+64.png"
border="0" height="222" width="320"></a></div>
</td>
<td>
<div class="separator" style="clear: both; text-align: center;"> <a
href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi2j0uEA9ledna7IiQn-pSozB0Xbfp6GR1J5eXyj2IVa9itIQHxFWCYzkODjIHKrXBuhQXvPmDr8_V70JM2kSlMN2pt_Un2B19CgvVn0OmHlbrnX-HtSHt8AJ1_F7os_Unopt4BmB6joH0/s1600/2014-07-24+23_21_09-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi2j0uEA9ledna7IiQn-pSozB0Xbfp6GR1J5eXyj2IVa9itIQHxFWCYzkODjIHKrXBuhQXvPmDr8_V70JM2kSlMN2pt_Un2B19CgvVn0OmHlbrnX-HtSHt8AJ1_F7os_Unopt4BmB6joH0/s1600/2014-07-24+23_21_09-RD+Tabs+64.png"
border="0" height="222" width="320"></a></div>
</td>
</tr>
</tbody>
</table>
<div class="separator" style="clear: both; text-align: center;"> </div>
<br>
These errors can occur on either side, provided obviously that side is
Windows. What errors you receive on the other side depend entirely on the
platform. <br>
<br>
<h3>What is Happening? </h3>
At a high level, the client and server are failing to agree on a way to talk
to each other securely. To communicate securely, the server and client must
agree on a methodology to communicate involving 4 main components. Those
are: <br>
<ul>
<li>How to authenticate each other (Key Exchange)</li>
<li>How to encrypt data to be exchanged (Encryption Cipher)</li>
<li>How to verify the message hasn't been tampered with (Message
Authentication Code)</li>
<li>How to determine random numbers for seeding keys (Pseudorandom
Function)</li>
</ul>
<p>The client and server must agree to the same implementation of each of
these items. Bundled together, these are referred to as a <a href="http://en.wikipedia.org/wiki/Cipher_suite">cipher
suite</a>. </p>
<p>The client and server each have preferences as to which portions of the
cipher suite hold which priority. Based on this prioritization, a set of
supported cipher suites is compiled and proposed at the beginning of any
SSL/TLS connection. The client first proposes what it would like, then the
server compares the client list to its own list and selects the first
matching suite. </p>
<p>So therein lies the problem: <strong>Your server doesn't like any of the
proposals from the client. </strong></p>
<h3>Why?</h3>
<p>This is why I decided to write this article. While there are several hits
on the internet regarding this problem, I have yet to see one that nails
it. Initially (and originally published in this article) I suspected the
problem was due to an incorrect <a href="http://msdn.microsoft.com/en-us/library/windows/desktop/bb931380%28v=vs.85%29.aspx">cryptographic
service provider</a> but thanks to some insights from one of my
colleagues I took another look. Turns out that due to the nature of this
problem it can appear sporadically and be difficult to troubleshoot. If
you're experiencing this problem the following may be true of your
environment: </p>
<ul>
<li>Internal CA (<a href="http://en.wikipedia.org/wiki/Certificate_signing_request">Certificate
Authority</a>)</li>
<li>You're using certreq.exe to create a CSR (<a href="http://en.wikipedia.org/wiki/Certificate_signing_request">Certificate
Signing Request</a>)</li>
<li>Your template for the CSR is "quite short"</li>
</ul>
<p>If those things aren't true, don't worry because here are the details: If
your CSR is requesting a certificate that is valid for signing only,
rather than signing and encryption, and your CA has a policy that allows
for encryption even when the request was signing only, then you will
likely see this problem... sometimes. Clearly a certificate requested for
signature only shouldn't work at all when used for encryption, but if your
CA overrides the request to allow for encryption that will create a
situation where encryption will work, but only under circumstances when
the client supports a couple specific protocol suites. Identifying
certificates causing this problem is complicated; since the CA overrode
the We'll cover the specifics further in the next two sections...</p>
<h3>Detecting The Problem</h3>
<p>Feel free to skip this section if you want to jump to the fix. Detection
can be pretty easy using tools like <a href="http://www.wireshark.org/">Wireshark</a>.
Fire up the tool on either the client or server with the proper <a href="http://wiki.wireshark.org/CaptureFilters">capture
filters</a> to reduce noise, and then attempt the failing connection.
You will see only a handful of packets (5 or so) as the rejection happens
pretty quickly. To see the detail appropriately, you'll need to tell
Wireshark this is SSL/TLS by right clicking->decode as->SSL. </p>
<p>If a protocol negotiation is the issue, you'll see the connection reset
by the server immediately after the client suggests a list of cipher
suites. This packet from the client will have the info of "client hello"
followed immediately with a TCP RST (reset) from the server. </p>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiptH43Ux51AcDvE-9EWZaAZba-hpp9ftrWE3cv-FQo3Tm0m_WJ4PWhTgazSm6Dz9Ep5MPR7qD-OTnuj-0Ymj1txP6knO0jE3ChgE4UK8jhVECoqSvgJ3J14QD0N2OkiQnMVzRdeiuK4kY/s1600/2014-07-30+00_30_15-Remote+Desktop+Connection.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiptH43Ux51AcDvE-9EWZaAZba-hpp9ftrWE3cv-FQo3Tm0m_WJ4PWhTgazSm6Dz9Ep5MPR7qD-OTnuj-0Ymj1txP6knO0jE3ChgE4UK8jhVECoqSvgJ3J14QD0N2OkiQnMVzRdeiuK4kY/s1600/2014-07-30+00_30_15-Remote+Desktop+Connection.png"
border="0" height="22" width="640"></a></div>
<p>If you drill into the details of the "client hello" packet you will be
able to see the suites the client is proposing. </p>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3s2iI-d4B94RpFbB6jFavzTbreDqTlv98RiD48TEJ3qXWgb3i0jiNWrGI7ixB59LBGXYbkUGYSTfw7YXXIevhRf4PPuc0N5yUiltxlxPLm9TE3rzcGrzRzDUTnBgN2OSlzzbIpOmhIBQ/s1600/2014-07-30+00_38_31-Remote+Desktop+Connection.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3s2iI-d4B94RpFbB6jFavzTbreDqTlv98RiD48TEJ3qXWgb3i0jiNWrGI7ixB59LBGXYbkUGYSTfw7YXXIevhRf4PPuc0N5yUiltxlxPLm9TE3rzcGrzRzDUTnBgN2OSlzzbIpOmhIBQ/s1600/2014-07-30+00_38_31-Remote+Desktop+Connection.png"
border="0" height="200" width="400"></a></div>
<p>You can then attempt a successful TLS connection if you are able to
produce one (if not just jump to the fix and try it) using the same
methodology. I found that while using the affected cert type listed above,
my server <strong>only</strong> supported
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA and
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, clearly a very limited subset. </p>
<h3>The Fix</h3>
<p>To remediate this issue you'll need to make sure that certificate
ordered is for the correct purpose. Rather than recreate that article I'll
direct you to my favorite one <a href="http://blogs.technet.com/b/pki/archive/2009/08/05/how-to-create-a-web-server-ssl-certificate-manually.aspx">here</a>,
however note that the [strings],[Extensions],and [RequestAttributes]
sections may not be needed depending on your situation. The main takeaway
from that article is that at the very least the KeySpec and KeyUsage
settings need to be specified (see link under references for more info).
Request, retrieve, and install this certificate.</p>
<p>You can use any other method you would like to obtain a certificate
(perhaps you do), but it's critical to ensure your request has the correct
parameters including the certificate usage. If you are using Windows PKI
with AD integrated templates, you can "hard code" this in the templates if
you like. </p>
<p>If this fix didn't work for you, wait for the "Wait There's More" section
because it's likely due to a misconfigured set of cipher suites. Speaking
of that...</p>
<h3>Wait There's More</h3>
<p>As a security best practice, you should also control (restrict) your
available cipher suites on Windows/IIS. This is pretty easy to do; it can
be done via Group Policy for large sets of servers and one-by-one with
registry settings or better yet with <a href="https://www.nartac.com/Products/IISCrypto/Default.aspx">this
easy tool</a> from Nartac. For more guidance check out <a href="https://www.ssllabs.com/downloads/SSL_TLS_Deployment_Best_Practices_1.0.pdf">these</a>
<a href="http://security.stackexchange.com/questions/48325/identify-and-disable-weak-cipher-suites">three</a>
<a href="https://wiki.mozilla.org/Security/Server_Side_TLS">links</a>. </p>
<p>Thanks for reading and feel free to add your own experience below!</p>
<h3>References<br>
</h3>
<p><a href="http://support.microsoft.com/kb/299520">Microsoft Support: How
to Determine the Cipher Suite for the Server and Client</a></p>
<p><a href="http://support.microsoft.com/default.aspx?scid=kb;EN-US;245030">Microsoft
Support: How to restrict the use of certain cryptographic algorithms and
protocols in Schannel.dll</a></p>
<p><a href="http://msdn.microsoft.com/en-us/library/aa374757%28VS.85%29.aspx">MSDN:
Cipher Suites in Schannel</a></p>
<p><a href="http://technet.microsoft.com/en-us/library/cc736326%28v=ws.10%29.aspx">Technet:
Certreq.exe syntax</a><br>Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com9tag:blogger.com,1999:blog-5458589696790815336.post-2789462402992391742014-07-06T21:04:00.000-05:002015-02-25T23:37:16.952-06:00Setup Your Own Chocoloatey/NuGet Repository <p>In this article we'll examine setting up a NuGet/Chocolatey <a href="http://en.wikipedia.org/wiki/Software_repository">repository</a>
in your enterprise to distribute software. This will allow you to easily
distribute development and software packages throughout your network. </p>
<h3>NuGet? I don't need any more candy. </h3>
<a href="http://nuget.codeplex.com/">NuGet</a> started life <a href="http://weblogs.asp.net/scottgu/announcing-nupack-asp-net-mvc-3-beta-and-webmatrix-beta-2">NuPack</a>
(not to be confused with <a href="http://www.nupack.org/">NuPack</a>), an
open source solution for managing .NET packages. Since then it has evolved
into a mature platform with numerous interfaces including a Visual Studio
plugin, command line, and Mono support. Chocolatey and <a href="http://blogs.msdn.com/b/powershell/archive/2014/05/14/windows-management-framework-5-0-preview-may-2014-is-now-available.aspx">PowerShellGe</a>t
are built on that framework. Speaking of Chocolate...<br>
<br>
<h3>Chocolatey? I told you already, no more candy. <br>
</h3>
Where NuGet was meant for .NET packages, <a href="https://chocolatey.org/">Chocolatey</a>,
which is built on the same infrastructure, is meant for machine (Windows)
packages. Think of it like <a href="http://en.wikipedia.org/wiki/Advanced_Packaging_Tool"><em>apt</em></a>
or <a href="https://fedoraproject.org/wiki/Yum"><em>yum</em></a> for
windows. PowerShell has also already shipped a preview of <a href="http://blogs.technet.com/b/windowsserver/archive/2014/04/03/windows-management-framework-v5-preview.aspx">PowerShell
OneGet</a> which can use Chocolatey repositories. <br>
<br>
<h3>But There are Already Repositories for These and I'm Hungry! Why Would I
Make My Own?<br>
</h3>
Yup! There are great public repositories for <a href="http://chocolatey.org/packages">Chocolatey</a>
and <a href="https://www.nuget.org/packages">NuGet</a>, but those are
geared at freely available public software that wouldn't be built for an
explicit purpose. By hosting your own you can host custom .NET packages
specific to your business unit or even package commercially available
software for distribution with Chocolatey provided your licensing is up to
snuff. <br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4tjwqtez2dX_lM-7uw9FdrA-Tk1uMJXL2DwFm7tzGqW9_A5pEbwbJBWh0a2bFaeTjee_XncD7lcXc91tO_HhJ8x0B0F_HsvU2D2Xt7Wd9qEMMa-50X4nhLWLnhFx7sGpXXR8ZeMyuB3A/s1600/ChocolateyNuget.jpg"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4tjwqtez2dX_lM-7uw9FdrA-Tk1uMJXL2DwFm7tzGqW9_A5pEbwbJBWh0a2bFaeTjee_XncD7lcXc91tO_HhJ8x0B0F_HsvU2D2Xt7Wd9qEMMa-50X4nhLWLnhFx7sGpXXR8ZeMyuB3A/s1600/ChocolateyNuget.jpg"
border="0"></a></div>
<br>
<h3>Hosting Options</h3>
<br>
There are several options to get going that vary in terms of hosting
location, ease of installation, and scalability. Some of the more popular
options include: <br>
<ul>
<li><a href="http://www.nuget.org/packages/NuGet.Server">NuGet Server</a>
: A basic server that runs on-premise and is easy to setup. Doesn't have
granular security features and will only scale so far before it slows
down. </li>
<li><a href="https://github.com/NuGet/NuGetGallery">NuGet Gallery</a>:
More complex NuGet server package that includes advanced security
features and will scale for larger implementations (this is what the
main public NuGet repo uses). </li>
<li><a href="http://docs.myget.org/">MyGet</a>: A commercial NuGet repo
service hosted in Azure. Has a limited free tier and reasonably priced
paid tiers. Worth consideration if you don't want to host your own
infrastructure. </li>
</ul>
<p>The assumption with this article is that you're hosting your first
NuGet/Chocolately repo for the purposes of your enterprise or team use.
Since the NuGet server includes most needed functionality for that purpose
and can be replaced with the NuGet Gallery as you grow, we'll set up a <strong>NuGet
Server</strong>(the first option) in this article.</p>
<span style="font-weight: bold;">Let's Get Started!</span><br>
<h3>Assumptions</h3>
<ul>
<li>We'll be setting up on a <strong>Windows Server 2012 R2</strong>.
This will work on Windows Server 2008 and up. I'm assuming you have one
set up and ready to go. </li>
<li>(Updated, see note below) You will need <strong><a href="http://www.visualstudio.com/en-us/products/visual-studio-express-vs.aspx">Visual
Studio Express</a> 2010/2012</strong> or <strong><a href="http://www.visualstudio.com/en-us/news/vs2013-community-vs.aspx">Visual
Studio Community</a> 2013</strong> or newer on your
workstation. (Preferably not on the server)</li>
<li>You will need <strong>Admin rights</strong> on both the server and
your workstation. </li>
<li>In enterprise environments you often have to make due with the
resources you have available, so we'll be setting up the repo as a <strong>virtual
application</strong> in IIS rather than its own site so that it can
share port 80 for the sake of simplicity. </li>
</ul>
<h3>IIS Setup on Server</h3>
<p>We'll walk through installing the minimum IIS requirements to run the
NuGet Server package. Everything here could be very easily done with
PowerShell but we'll use the GUI to make for a more visual tutorial. </p>
<ol>
<li>On the server where you will host the application, start the "<strong>Add
Roles and Features Wizard</strong>"</li>
<li>Click "<strong>Next</strong>" until you advance to the "<strong>Server
Roles</strong>" section. If you're executing remotely make sure you
select the correct server.</li>
<li>Of the Roles listed, select "<strong>Web Server (IIS)</strong>" and
select "<strong>Add Features</strong>" when prompted. Click "<strong>Next</strong>". </li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgPH9efIWQQdQMjdjz8ZsiQj2e757iDlZO3Vd1E3TvNomCrLKsvgxP3AtHqqObRhQ4w5ilkIGqbMusPIo1kIiFXwIlgEAtLXIpX_c8ZI7iZqPqnREOR4QVPrkHNE9yW3KoZR1TmVp4xG7A/s1600/2014-07-04+23_26_18-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgPH9efIWQQdQMjdjz8ZsiQj2e757iDlZO3Vd1E3TvNomCrLKsvgxP3AtHqqObRhQ4w5ilkIGqbMusPIo1kIiFXwIlgEAtLXIpX_c8ZI7iZqPqnREOR4QVPrkHNE9yW3KoZR1TmVp4xG7A/s1600/2014-07-04+23_26_18-RD+Tabs+64.png"
border="0" height="282" width="400"></a></div>
<br>
<li>On the "Features" page, expand "<strong>.NET Framework 4.5 Features</strong>"
and ensure "<strong>.NET Framework 4.5</strong>" and "<strong>ASP.NET
4.5</strong>" are checked. Click "<strong>Next</strong>".</li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgF5NyDZPy1K-HvgRvg925rknvg9OLjdsIL6zLcmks_z8MImJRFRURc-GRFkwEbxoo7e9FUQrAMIfPD2OV4FhRxZO2vIq3QZZMFhEEzxoQwhqKzViKW2hp_A04mnEYvphFg2ywtWA2N9Jk/s1600/2014-07-04+23_27_01-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgF5NyDZPy1K-HvgRvg925rknvg9OLjdsIL6zLcmks_z8MImJRFRURc-GRFkwEbxoo7e9FUQrAMIfPD2OV4FhRxZO2vIq3QZZMFhEEzxoQwhqKzViKW2hp_A04mnEYvphFg2ywtWA2N9Jk/s1600/2014-07-04+23_27_01-RD+Tabs+64.png"
border="0" height="282" width="400"></a></div>
<br>
<li>Click "<strong>Next</strong>" to advance to the "Role Services"
section under "Web Server Role (IIS)" and <strong>select the following</strong>
(only the most granular required, not the headings):</li>
<ul>
<li>Web Server</li>
<ul>
<li>Common HTTP Features</li>
<ul>
<li>Default Document</li>
<li>Static Content</li>
</ul>
<li>Health and Diagnostics</li>
<ul>
<li>HTTP Logging</li>
</ul>
<li>Performance</li>
<ul>
<li>Static Content Compression</li>
</ul>
<li>Security</li>
<ul>
<li>Request Filtering</li>
</ul>
<li>Application Development</li>
<ul>
<li>.NET Extensibility 4.5</li>
<li>ASP.NET 4.5</li>
<li>ISAPI Extensions</li>
<li>ISAPI Filters</li>
</ul>
</ul>
<li>Management Tools</li>
<ul>
<li>IIS Management Console</li>
</ul>
</ul>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhu_ZpGJqfwtfP3pQi2o4XC8ZU6rCyvWO2-FJx87FsnSuPva8j1EblcCuqLZEYu4tUJHpqUp8tg4smbHE-mCs8EnAOBKIFNEiEiUkoVZrln-IJ14z6Vk5_WHBsi8FwrfBNE87DcSpoOVA/s1600/2014-07-04+23_28_00-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhu_ZpGJqfwtfP3pQi2o4XC8ZU6rCyvWO2-FJx87FsnSuPva8j1EblcCuqLZEYu4tUJHpqUp8tg4smbHE-mCs8EnAOBKIFNEiEiUkoVZrln-IJ14z6Vk5_WHBsi8FwrfBNE87DcSpoOVA/s1600/2014-07-04+23_28_00-RD+Tabs+64.png"
border="0" height="282" width="400"></a></div>
<br>
<li>Click "<strong>Next</strong>" and then click "<strong>Install</strong>".
<br> </li>
<li>You shouldn't need to reboot, but check the installation status and do
so if requested. </li>
</ol>
<h3>IIS Setup on Server</h3>
Now we'll set up the site that the NuGet Server will be served from. As
mentioned earlier, I will walk you through setting it up as a virtual
application off of the default web site. This configuration would allow you
to share port 80 with an existing site as well as show you how to configure
this application below the root, which does need a bit of special
consideration worth mentioning. <br>
<ol>
<li>Create the directory structure for your site. I always put my IIS
sites on a non-system drive with the permissions locked down. In this
example, I'll be using <strong>D:\Sites\NuGetRepo </strong>. </li>
<li>Create the directory for the NuGet/Chocolately package repo. We'll
configure this below. This directory uses a different permissions
structure and could potentially be shared out over your LAN, so it may
be beneficial to place this separate from the site. In my example I'll
be using <strong>D:\NuGetRepo</strong> .</li>
<li>(<em>Optional/Best Practice</em>) At the D:\Sites and D:\NuGetRepo
levels disable inheritance ensure only Administrators and SYSTEM have
write access. Do not allow any other access at this time, we'll get to
that below. </li>
<li>On the server to host NuGet Server, <strong>open the IIS management
tool</strong>. </li>
<li><strong>Right click</strong> the Default Web Site (note this could be
any web site) and select "<strong>Add Application...</strong>" (A
virtual directory will <em>not</em> work!)</li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzm4_k9N-Rg7g8iJBQfwH9jPdNYu8gPh48kjEm9jEdvAyczXl7SRY8wwYtUOTX63u2tJq2kZ4wzstEUQ3hYHgelW-yHWN43GIX1lGK4-b9LE2Hsumn7hQPODlOnRnIq3iFh-P9F2KUINM/s1600/2014-07-04+23_37_45-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzm4_k9N-Rg7g8iJBQfwH9jPdNYu8gPh48kjEm9jEdvAyczXl7SRY8wwYtUOTX63u2tJq2kZ4wzstEUQ3hYHgelW-yHWN43GIX1lGK4-b9LE2Hsumn7hQPODlOnRnIq3iFh-P9F2KUINM/s1600/2014-07-04+23_37_45-RD+Tabs+64.png"
border="0" height="398" width="400"></a></div>
<br>
<li>Set the <strong>alias to</strong> "<strong>NuGet</strong>", leave the
"<strong>Application Pool</strong>" on "<strong>DefaultApplicationPool</strong>"
(again, you could change this if desired) and set the <strong>physical
path to</strong> what you created for the site. For our example we're
using "<strong>D:\Sites\NuGetRepo</strong>" .<strong> </strong></li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8aZXAqW4Z2fbmB1uCqIwjB0BjWzyeh4Asug34sa6fJdq75KVSI-d6fdj-nqIy4tzXOoq63dvZkiziwMIhYeU-GtOek6v7Cq_jOJjaA-vMVMvtd310b0BIsKEVlEpwMJxOAAq1Zv-oo30/s1600/2014-07-04+23_38_21-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8aZXAqW4Z2fbmB1uCqIwjB0BjWzyeh4Asug34sa6fJdq75KVSI-d6fdj-nqIy4tzXOoq63dvZkiziwMIhYeU-GtOek6v7Cq_jOJjaA-vMVMvtd310b0BIsKEVlEpwMJxOAAq1Zv-oo30/s1600/2014-07-04+23_38_21-RD+Tabs+64.png"
border="0" height="295" width="400"></a></div>
<br>
<li>The default out-of-box settings should work for the site, but in case
the Default Web Site settings have been changed you may want to refresh
your view and ensure IIS authentication is set to "Anonymous". If
desired, change the logging location as well (D:\logfiles\IIS\NuGetRepo
for example). </li>
</ol>
<h3>Setting File System Permissions on IIS Server</h3>
<p>For clients to access the site and repo successfully we need to set file
system level permissions. If you have used different directory names above
substitute them here accordingly.</p>
<p><strong>Note</strong>: We're assuming you have administrative access to
this server from your Workstation as well to deploy the code in the steps
below. If not, you'll need to grant whoever will be deploying the site
access to the site folder. If you are admin, don't worry about it. </p>
<ol>
<li>The directory containing the website needs to be read by the AppPool
account and the Anon user account. <strong>Right click</strong> <strong>D:\Sites\NuGetRepo</strong>
and select "<strong>Properties</strong>".</li>
<li>Click "<strong>Security</strong>", "<strong>Edit</strong>", and then "<strong>Add</strong>".
Change the "<strong>Location</strong>" <strong>to the local system </strong>name.
</li>
<li>Give the default web site application pool <a href="http://technet.microsoft.com/en-us/library/dd548356%28v=WS.10%29.aspx">virtual
service account</a> and <a href="http://www.iis.net/learn/get-started/planning-for-security/understanding-built-in-user-and-group-accounts-in-iis">anonymous
account</a> permissions by typing "<strong>IIS APPPOOL\DefaultAppPool</strong>;<strong>IUSR</strong>",
clicking "<strong>Check Names</strong>" and then "<strong>OK</strong>".
Again, if you have elected to use a different site/pool/acct you will
need to take that into account. This should resolve to two accounts,
"DefaultAppPool" and IUSR". </li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjK-Hjgtz9tvAOpBY18NzKKszfg6bXT7iNGhYM9cgLFi2m1mmOMxP1EaAifcKBPfxuqbEd5X7JrAnKHqCydCeBqBRMx52l0_QcY5-8BwOIdn589pVKehq5qWb2dRYT-w3fFnDzVXh02ydg/s1600/2014-07-04+23_34_27-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjK-Hjgtz9tvAOpBY18NzKKszfg6bXT7iNGhYM9cgLFi2m1mmOMxP1EaAifcKBPfxuqbEd5X7JrAnKHqCydCeBqBRMx52l0_QcY5-8BwOIdn589pVKehq5qWb2dRYT-w3fFnDzVXh02ydg/s1600/2014-07-04+23_34_27-RD+Tabs+64.png"
border="0" height="218" width="400"></a></div>
<br>
<li>Give each of the added users "<strong>Read & Execute</strong>", "<strong>List
folder contents</strong>", and "<strong>Read</strong>" permissions and
then click "<strong>OK</strong>".</li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvK1de9YsF5NotOAVf1J-mU4HNvm-svvfogkzioTyvOuIA_pzr-GYG6ZjSJmGqH9PArMPaS7oI-mRpWDg5_EIxvfq1GzqzTbLJ1T17DI3k3iBsXdCAV39zUGd8VD_OrWypVrNdBY85yuk/s1600/2014-07-04+23_35_14-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvK1de9YsF5NotOAVf1J-mU4HNvm-svvfogkzioTyvOuIA_pzr-GYG6ZjSJmGqH9PArMPaS7oI-mRpWDg5_EIxvfq1GzqzTbLJ1T17DI3k3iBsXdCAV39zUGd8VD_OrWypVrNdBY85yuk/s1600/2014-07-04+23_35_14-RD+Tabs+64.png"
border="0" height="400" width="328"></a></div>
<br>
<li>The directory containing the actual repo only needs to be read by the
AppPool account. <strong>Right click</strong> <strong>D:\NuGetRepo</strong>
and select "<strong>Properties</strong>".</li>
<li>Click "<strong>Security</strong>", "<strong>Edit</strong>", and then "<strong>Add</strong>".
Change the "<strong>Location</strong>" <strong>to the local system</strong>
name. <br>
</li>
<li>Give the default web site application pool virtual service account
permissions by typing "<strong>IIS APPPOOL\DefaultAppPool</strong>",
clicking "<strong>Check Names</strong>" and then "<strong>OK</strong>".
Again, if you have elected to use a different site/pool/acct you will
need to take that into account. This should resolve to one account,
"DefaultAppPool". <br>
</li>
<li>Grant it "<strong>Read & Execute</strong>", "<strong>List folder
contents</strong>", and "<strong>Read</strong>" permissions and then
click "<strong>OK</strong>".</li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVnoOAJZswieEksdIfvojmD5IqP59DDrtvlnxxLnp9Ag0yretO_47XTu9KneJtLrJbb1wsjbrmhbfr_PVGPlRUsdUGxFyzsEUxoiKKAU6koJxVGLBdXA9ly2BmwGdX_7vzucf3QFOXlCI/s1600/2014-07-04+23_37_00-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVnoOAJZswieEksdIfvojmD5IqP59DDrtvlnxxLnp9Ag0yretO_47XTu9KneJtLrJbb1wsjbrmhbfr_PVGPlRUsdUGxFyzsEUxoiKKAU6koJxVGLBdXA9ly2BmwGdX_7vzucf3QFOXlCI/s1600/2014-07-04+23_37_00-RD+Tabs+64.png"
border="0" height="400" width="308"></a></div>
<br>
</ol>
<h3>NuGet Server Config on Workstation</h3>
<p>Now we'll grab the NuGet Server package and configure it accordingly.
Note some of these options will vary slightly depending on which version
of Visual Studio you are using. I'm using 2012 Premium but everything is
possible in 2010 Express and up. Edit: As pointed out by Nik below it looks as if Microsoft changed this starting in 2013. For 2013 and higher you'll need the <a href="http://www.visualstudio.com/en-us/news/vs2013-community-vs.aspx">community edition</a>.</p>
<p><strong>Note:</strong> We are assuming your IIS server is accessible to
you and has file sharing turned on to push the site. If you are unable to
get to the filesystem of the server from this machine you will need to use
a different deployment mechanism when we get to that step. </p>
<ol>
<li>On your workstation, open Visual Studio and start a new Project by
selecting "<strong>File</strong>"->"<strong>New</strong>"->"<strong>Project</strong>"</li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-MeFX8UQpbj8D3Y1_Z2Rpug3iz3qPkNxB0nHZ9dI8slh2QeCq1yT8wWjO8ukdsxNZYLdxefGcKXhzCW6UIdcVhcEkMZl1RgOUouf5LurpmAFDvkh79G9EJRc0FdxXEr9srInLJPHJjWM/s1600/2014-07-04+17_37_34-.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-MeFX8UQpbj8D3Y1_Z2Rpug3iz3qPkNxB0nHZ9dI8slh2QeCq1yT8wWjO8ukdsxNZYLdxefGcKXhzCW6UIdcVhcEkMZl1RgOUouf5LurpmAFDvkh79G9EJRc0FdxXEr9srInLJPHJjWM/s1600/2014-07-04+17_37_34-.png"
border="0" height="102" width="320"></a></div>
<br>
<li>Navigate to "<strong>Installed</strong>"->"<strong>Templates</strong>"->"<strong>Visual
C#</strong>"->"<strong>Web</strong>" and select "<strong>ASP.NET
Empty Web Application</strong>"</li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdXFJxlLNj_OXS57kOnh6bnphIyLFgAUv-hqJWWMtfq4ZUBmun-VvZkip31gOOKuFwkzz8QVZkaUjPCcRmfi0oeQVbDLxSLk9ReLHfNOEzftrTcMhhIlodN9HLWfZ5QiVn1frjWjdL0mQ/s1600/2014-07-04+17_38_19-New+Project.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdXFJxlLNj_OXS57kOnh6bnphIyLFgAUv-hqJWWMtfq4ZUBmun-VvZkip31gOOKuFwkzz8QVZkaUjPCcRmfi0oeQVbDLxSLk9ReLHfNOEzftrTcMhhIlodN9HLWfZ5QiVn1frjWjdL0mQ/s1600/2014-07-04+17_38_19-New+Project.png"
border="0" height="112" width="320"></a></div>
<br>
<li>Right click on your newly created application under the solution and
select "<strong>Manage NuGet Packages</strong>"</li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjY53SqssGtanhzhRKfsCuY7jv2UKR0eBxMRf-9-9EsgbPtX4DXXdgojsMwVT9DwcJ8sskF9ElI6zoKC8Dm98XAWpaNZqmMQ24peOlE-v1RYFNMwHM9xDMpmKfgfE6kDKfhUqA3_6e9-Y0/s1600/2014-07-04+17_38_55-WebApplication4+-+Microsoft+Visual+Studio.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjY53SqssGtanhzhRKfsCuY7jv2UKR0eBxMRf-9-9EsgbPtX4DXXdgojsMwVT9DwcJ8sskF9ElI6zoKC8Dm98XAWpaNZqmMQ24peOlE-v1RYFNMwHM9xDMpmKfgfE6kDKfhUqA3_6e9-Y0/s1600/2014-07-04+17_38_55-WebApplication4+-+Microsoft+Visual+Studio.png"
border="0" height="290" width="320"></a></div>
<br>
<li>Assuming the defaults of the nuget.org feed and "Stable Only" are
selected, type "<strong>nuget.server</strong>" in the search box and hit
<strong>Enter</strong>. </li>
<li>Select the "<strong>NuGet.Server</strong>" package and click "<strong>Install</strong>".
This will install the NuGet server package and any dependencies. Accept
license agreements associated with the other packages to continue and
then close the package management window. </li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9xgDRXqbbWr7HdtfSM4FRGU911SID8X0F9vUGAtLwNo2w6UoditmKIRePtyUc0A5Vl40PsOwIm16KO1Ny7DRx8mMulfn8zg5E6a1VX5Lm8GZtqaqfWQXglRWGj6MDmEq1ctAaMTXQvsc/s1600/2014-07-04+17_39_39-WebApplication4+-+Manage+NuGet+Packages.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9xgDRXqbbWr7HdtfSM4FRGU911SID8X0F9vUGAtLwNo2w6UoditmKIRePtyUc0A5Vl40PsOwIm16KO1Ny7DRx8mMulfn8zg5E6a1VX5Lm8GZtqaqfWQXglRWGj6MDmEq1ctAaMTXQvsc/s1600/2014-07-04+17_39_39-WebApplication4+-+Manage+NuGet+Packages.png"
border="0" height="112" width="400"></a></div>
<br>
<li>The only thing we need to customize is the web.config file for
our installation. In the Solution Explorer <strong>click</strong> "<strong>Web.config</strong>"
under the web application. Note: This file is also where you can control
<a href="http://docs.nuget.org/docs/creating-packages/creating-and-publishing-a-package#api-key">API
Key</a> behavior, but that is outside the scope of this article. </li>
<li>Look for the add key="packagesPath" entry in the web.config file under
the "<appSettings>" heading. We need to set this to the location
of our repository. <strong>Change <add key="packagesPath"
value=""/> </strong>to<strong> <add key="packagesPath"
value="D:\NuGetRepo"/></strong> (or other directory if
appropriate). Note that there is no trailing slash. <strong>Save your
project</strong>.</li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWsb9BMH7K-WxzxrIbz9w2mNSCoKvgqHYKVmLYn86fTkAL-kv0Y3V9s-peGKdXISEzlWqK1exSyFwRMMkVfkwxuskaXv0OngG77x4Zxdoncg-wVuLehjQszHu5iiL0ALdCbvjgPd738kc/s1600/2014-07-04+17_42_01-WebApplication4+-+Microsoft+Visual+Studio.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img style="width: 338px; height: 257px;"
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWsb9BMH7K-WxzxrIbz9w2mNSCoKvgqHYKVmLYn86fTkAL-kv0Y3V9s-peGKdXISEzlWqK1exSyFwRMMkVfkwxuskaXv0OngG77x4Zxdoncg-wVuLehjQszHu5iiL0ALdCbvjgPd738kc/s1600/2014-07-04+17_42_01-WebApplication4+-+Microsoft+Visual+Studio.png"
border="0"></a></div>
<br>
<li>Now we need to publish. Click "<strong>Build</strong>"->"<strong>Publish
WebApplication...</strong>"</li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgISk6OsAROt4FPf4BUQJ0n055YlxxV5iVMfTw7eVbJew2TVtlt5V9ZeObsFEEWcRx78GSRLFtbVeJ8U4c7JtfpBrlhMN7tcyzs70Rq0CG9LanW6-HBXTbO_mAo1epxrD5OOrJAr48gLuY/s1600/2014-07-04+18_18_16-.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgISk6OsAROt4FPf4BUQJ0n055YlxxV5iVMfTw7eVbJew2TVtlt5V9ZeObsFEEWcRx78GSRLFtbVeJ8U4c7JtfpBrlhMN7tcyzs70Rq0CG9LanW6-HBXTbO_mAo1epxrD5OOrJAr48gLuY/s1600/2014-07-04+18_18_16-.png"
border="0" height="182" width="320"></a></div>
<br>
<li>If you already have a working publishing profile for the web server,
select it and skip to step 12. Otherwise, <strong>select <new
profile></strong> from the drop-down box, <strong>enter a name</strong>,
and click "<strong>OK</strong>". </li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhr8mWd7j8QJ92skv7su0835FKXLadIpXIjJAv5PcG2MzP6hYydAyinJqNNBOb5uDQXen9J_7TfElRYW9CdCaQGWtzefuBho6cDi6YuaTHUMqFRSqTxxT75zeDxRU4vI3JFgsCtEVnGxXY/s1600/2014-07-04+18_20_24-New+Profile.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhr8mWd7j8QJ92skv7su0835FKXLadIpXIjJAv5PcG2MzP6hYydAyinJqNNBOb5uDQXen9J_7TfElRYW9CdCaQGWtzefuBho6cDi6YuaTHUMqFRSqTxxT75zeDxRU4vI3JFgsCtEVnGxXY/s1600/2014-07-04+18_20_24-New+Profile.png"
border="0" height="212" width="320"></a></div>
<br>
<li><strong>Change "Publish method" to "File System"</strong> and enter
the full path to the web server site location, I.E. "<strong>\\<server>\d$\sites\NuGetRepo\</strong>"
. Click "<strong>Next</strong>".</li>
<li>Accept the default publishing settings and click "<strong>Next</strong>". </li>
<li>Review the settings and click "<strong>Publish</strong>". </li>
<li><strong>Review the Output window</strong> to ensure there weren't any
errors. </li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj06-EwvZET2iIABz6BLxNVX2LsITaND8Hku3RIvgzWh9kahkB3tH-aBrd9_NQTe4oHiyMXVlRjq0jhz4Rc7Qsx7kzZNZGSHr8oTxhFKU6yiEopROwZwsl-n3MmD3kp2-j-xB6m33hu5EA/s1600/2014-07-04+18_26_10-WebApplication1+-+Microsoft+Visual+Studio.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj06-EwvZET2iIABz6BLxNVX2LsITaND8Hku3RIvgzWh9kahkB3tH-aBrd9_NQTe4oHiyMXVlRjq0jhz4Rc7Qsx7kzZNZGSHr8oTxhFKU6yiEopROwZwsl-n3MmD3kp2-j-xB6m33hu5EA/s1600/2014-07-04+18_26_10-WebApplication1+-+Microsoft+Visual+Studio.png"
border="0" height="91" width="352"></a></div>
<br>
<li>Test your NuGet Server by navigating to
http://<servername>/NuGet/ . If you encounter errors be sure to
browse to it locally on the server to get the full error information. </li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiTAmwlBxxAWzPwHtPZ1xpoxivNES2LDW4v3oqgh2vBLluSfb4R2FjtBdngQqHrQ5TIjPP_69SM_83q_yDMRmqRpItV2WBGnAjdQCp2yDid9rPg7StSqQHsdjLzPoSI24AtBqrdv0oKVa8/s1600/2014-07-04+23_43_09-NuGet+Private+Repository.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiTAmwlBxxAWzPwHtPZ1xpoxivNES2LDW4v3oqgh2vBLluSfb4R2FjtBdngQqHrQ5TIjPP_69SM_83q_yDMRmqRpItV2WBGnAjdQCp2yDid9rPg7StSqQHsdjLzPoSI24AtBqrdv0oKVa8/s1600/2014-07-04+23_43_09-NuGet+Private+Repository.png"
border="0" height="306" width="640"></a></div>
<br>
</ol>
<p>That's it! Now all you need to do is configure the source in your
clients, make packages, and enjoy! For instructions on those steps see
below, and stay tuned for more. Thanks for reading!</p>
<h3>Creating NuGet/Chocolatey Packages</h3>
<a href="https://chocolatey.org/">Chocolatey.org : all things Chocolatey!</a><br>
<a href="https://github.com/chocolatey/chocolatey/wiki/CreatePackagesQuickStart">
Chocolatey Docs: Create Packages Quick Start</a><br>
<a href="docs.nuget.org/docs/reference/nuspec-reference">NuGet Docs: Nuspec
Reference</a><br>
<a href="https://github.com/chocolatey/chocolateytemplates/blob/master/_templates/chocolatey/__NAME__.nuspec">Chocolatey
Docs: Chocolately Templates</a><br>
<a href="http://docs.nuget.org/docs/creating-packages/creating-and-publishing-a-package">NuGet
Docs: Creating and Publishing a Package</a><br>
<a href="http://www.hanselman.com/blog/CreatingANuGetPackageIn7EasyStepsPlusUsingNuGetToIntegrateASPNETMVC3IntoExistingWebFormsApplications.aspx">Scott
Hanselman: Creating a NuGet Package in 7 Easy Steps!</a><br>
<a href="https://github.com/chocolatey/chocolatey/wiki/CreatePackages">Chocolatey
Docs: Creating Chocolatey Packages</a><br>
<a href="http://www.topbug.net/blog/2012/07/02/a-simple-tutorial-create-and-publish-chocolatey-packages/">Hong
Xu: Create and Publish Chocolatey Packages</a><br>
<a href="http://docs.nuget.org/docs/creating-packages/configuration-file-and-source-code-transformations">
NuGet Docs: Configuration File and Source Code Transformations</a><br>
<h3>Configuring Sources</h3>
<a href="https://github.com/chocolatey/chocolatey/wiki/CommandsInstall#source-optional">
Chocolatey Docs: Source command</a><br>
<a href="http://docs.nuget.org/docs/start-here/managing-nuget-packages-using-the-dialog">NuGet
Docs: Visual Studio Package Sources</a><br>
<h3>References!</h3>
<a href="http://docs.nuget.org/docs/creating-packages/hosting-your-own-nuget-feeds">
NuGet Docs: Hosting Your Own Feeds</a><br>
<a href="http://www.hanselman.com/blog/IsTheWindowsUserReadyForAptget.aspx">Scott
Hanselman: Is the Windows User Ready for Apt-Get? </a><br>
<a href="http://mbrownnyc.wordpress.com/2013/09/06/create-your-own-nuget-server-to-serve-packages/">MBrownNYC:
Create Your Own NuGet Server to Serve Packages for Chocolatey</a><br>
<a href="http://docs.nuget.org/docs/reference/ecosystem">NuGet Docs: An
Overview of the NuGet Ecosystem</a><br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQjJF8NqWR9KFzDJxdXy1xKBVx4FpplJ-Z7E3euIJH0NrY0Lsuh2BCnpSoknbIrFgvvZV9De_P1L65xbyEtPBr5qJdtxC1ZfyQvxhznyxLEe-VUt8y-L3GbIefP_q54oEUUkjGBCQOrHI/s1600/IMG_20140705_233726.jpg"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQjJF8NqWR9KFzDJxdXy1xKBVx4FpplJ-Z7E3euIJH0NrY0Lsuh2BCnpSoknbIrFgvvZV9De_P1L65xbyEtPBr5qJdtxC1ZfyQvxhznyxLEe-VUt8y-L3GbIefP_q54oEUUkjGBCQOrHI/s1600/IMG_20140705_233726.jpg"
border="0" height="296" width="400"></a></div>Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com12tag:blogger.com,1999:blog-5458589696790815336.post-78172273499668498882014-05-27T19:17:00.008-05:002023-05-10T23:46:26.527-05:00Cloudy I/O Performance - Increasing Azure IOPS (Part 2 of 2)<i>Notes:</i> This is part 2 of a 2 part post. You can find part 1 <a href="http://blog.ittoby.com/2014/05/cloudy-io-performance-deciphering-iops.html">here</a>. Since publication of this article Microsoft has introduced new machine types with higher performing storage. More information can be found <a href="https://azure.microsoft.com/en-us/pricing/details/virtual-machines/">here</a>. <br />
<br />
<h3>
Foreword</h3>
<br />
In the last article we discussed a repeatable testing methodology to
quantify storage performance in the cloud, and in this article we'll put
that methodology into practice. I've done substantial testing in Azure and
aim to illustrate what your options are for scaling performance at this
point in time.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh6mWTsArpsrVrorMlT3SMAyzs81F0ifwt1tkSjDsk9iAVvwgvKplvdKnSVEnQ-3MIODvOMOt3RL3I2UyeV91IrTExRZtyf_Hi23WqRa-ry23YGo3u-N1LpKQjaO0v8Go9TH2kK1DPRLJw/s1600/70s_mainframe_crew_MOD_2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="448" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh6mWTsArpsrVrorMlT3SMAyzs81F0ifwt1tkSjDsk9iAVvwgvKplvdKnSVEnQ-3MIODvOMOt3RL3I2UyeV91IrTExRZtyf_Hi23WqRa-ry23YGo3u-N1LpKQjaO0v8Go9TH2kK1DPRLJw/s1600/70s_mainframe_crew_MOD_2.jpg" width="640" /></a></div>
<h3>
<b> Scope</b></h3>
I undertook this project to see what can be done to increase disk I/O in
Windows Azure IaaS. Upon researching the topic I found several interesting
articles. Among those are:
<br />
<ul>
<li><a href="https://google.com">Timothy
Khouri: Windows Azure IaaS Performance – SQL and IOps (Link removed, site was exploited) <br />
</a></li>
<li><a href="http://google.com">Timothy
Khouri: Configuring Disks in Azure for More Performance (Link removed, site was exploited) <br />
</a></li>
<li><a href="http://msdn.microsoft.com/en-us/library/windowsazure/dn248436.aspx">MSDN:
Performance Guidance for SQL Server in Windows Azure Virtual Machines</a></li>
<li><a href="http://msdn.microsoft.com/en-us/library/windowsazure/dn133151.aspx">MSDN:
Getting Started with SQL Server in Windows Azure Virtual Machines</a></li>
<li><a href="http://www.windowsazure.com/en-us/pricing/details/storage/?rnd=1">Azure:
Storage Pricing Details</a></li>
<li><a href="http://www.windowsazure.com/en-us/pricing/details/virtual-machines/">Azure:
VM Pricing Details</a></li>
<li><a href="http://msdn.microsoft.com/en-us/library/windowsazure/dn197896.aspx">Virtual
Machine and Cloud Service Sizes for Windows Azure</a></li>
<li><a href="http://blogs.msdn.com/b/windowsazure/archive/2012/11/02/windows-azure-s-flat-network-storage-and-2012-scalability-targets.aspx">Azure
Blogs: Windows Azure’s Flat Network Storage and 2012 Scalability
Targets</a><br />
</li>
</ul>
There seems to be little consensus regarding disk striping in Windows
Azure IaaS. Some blogs recommend this while some of Microsoft's own
writing seems to discourage it. After combing through the options the
following points stand out: <br />
<ul>
<li>Disk Striping (Software <a href="https://en.wikipedia.org/wiki/RAID_0#RAID_0">RAID
0</a>) may or may not increase performance based on your workload. </li>
<li>Striping will increase I/O capacity to a degree (which we'll test
here).</li>
<li>What software striping solution works better: legacy (Windows software
RAID from 2000 to present) or Storage Spaces (new software "RAID" in
Windows 2012 and up)?</li>
<li>How does NTFS cluster size impact performance?</li>
<li>If striping, disable geo-replication as Microsoft explicitly warns
against the use of geo-replication with this solution.</li>
<li>If possible, use native application load distribution rather that disk
striping to split I/O. (For example, split DB files in SQL across
disks)</li>
<li>Some articles reference needing to use multiple storage accounts to
get maximum performance. This is <b>not</b> true; as of
6/7/2012 storage account targets are 20,000 IOPS per account. Unless you
will exceed the 20,000 keep all your disks on one account for the sake
of simplicity. We will prove that does not have an impact on
performance.</li>
</ul>
With that said, I want to quantify the solution for my given scaling
problem with the notion that if the tests are simple enough to run, this
approach can be used for any future scaling problem as well. <br />
<h3>
Putting it All Together</h3>
We'll use the testing methodology outlined in <a href="http://blog.ittoby.com/2014/05/cloudy-io-performance-deciphering-iops.html">part
1</a> of this article to collect the results. In this case we need to
first add disks and set up stripes in Azure Windows VMs.<br />
Note: To jump straight to Azure disk performance tests, scroll to the
bottom of this article. <br />
<h4>
Create New Disks and Attach to Designated VM</h4>
In order to run all the tests listed below, you need to know how to create
new disks and attach them to your virtual machine. My favorite solution to
this is to use a locally created dynamic VHD and upload it to the location
you would like using PowerShell. Let's go through the process of attaching
one disk as a primer: <br />
<ol>
<li>Decide which storage account you will use for these disks. If you plan
on doing striping of any kind, ensure the storage account is set to
"Locally Redundant" replication (<i>Storage->Desired Storage
Account->Configure)</i>, as "Geo Redundant" is not supported. Since
the replication setting applies to all blobs (Azure's terminology;
disks) in that account you may want to have a dedicated account for
these disks to keep your others Geo Redundant.</li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXXQ_q5mMP9cVtVbPiYUzXm59Qzh5d2e9_f9qSyYof-s9nptT7H6nuzk592WdVshyphenhyphenB8qnpe_IZ5ZNCJ2S_x9Dkrj69My7mkQCzoVGQZcwJptZr2yVwnXVW1SWAIL-xBmoSgMX1fe1b3Yw/s1600/2014-05-26+16_43_38-Storage+-+Windows+Azure.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="132" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXXQ_q5mMP9cVtVbPiYUzXm59Qzh5d2e9_f9qSyYof-s9nptT7H6nuzk592WdVshyphenhyphenB8qnpe_IZ5ZNCJ2S_x9Dkrj69My7mkQCzoVGQZcwJptZr2yVwnXVW1SWAIL-xBmoSgMX1fe1b3Yw/s1600/2014-05-26+16_43_38-Storage+-+Windows+Azure.png" width="400" /> </a></div>
<br />
<li>Determine what container you would like to store your Azure disk blob
by opening the Azure management portal and navigating to <i>Storage->Desired
Storage Account->Containers</i> and copy the URL to your clipboard.
To keep things simple you may want to create a new storage container, so
do so now and use that URL if desired.</li>
<li>Using Hyper-V (On Windows 2008 or higher including Windows 8) create
an empty dynamically expanding VHD disk of your desired size. For my
testing I have been using 10GB disks. <b>Note 1</b>: Do not
create a VHDX; Azure uses the older VHD format. <b>Note 2</b>:
You'll need to re-create the VHD for each disk if you intend on using
Storage Spaces as each disk must have a unique ID. </li>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">#create a dynamically expanding 10GB VHD; change size as appropriate
New-VHD –Path $sourceVHD –SizeBytes 10GB -Dynamic
</code></pre>
<li>This disk will be uploaded to the container we selected in step 1.
Determine the name you want the disk to be referenced by in Azure and
execute the following script: </li>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">#import Azure cmdlets
import-module azure.psd1
#specify your subscription so PS knows what account to upload the data to
select-azuresubscription "mysubscriptionname"
#$sourceVHD should be the location of your empty vhd file
$sourceVHD = "D:\Skydrive\Projects\Azure\AzureEmpty10G_Disk.vhd"
#$destinationVHD should be the URL of the container and the name of the vhd you want created in your account. Obviously for subsequent disks you need to change the VHD name.
$destinationVHD = "http://yourstorageacctname.blob.core.windows.net/vhds/data02.vhd"
#now upload it.
Add-AzureVhd -LocalFilePath $sourceVHD -Destination $destinationVHD
</code></pre>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhP3rl-d8DszMAc8fKGc-GbFerpU0Qd-bVQFR-c0lpb-N1bw_n6zsJqmToCHeU83bkWDFPTTHI0KkagHa41j59EgJEy7hbbs_NLwNkaSw6ek91NhDU6ai9AloHIBt9AY_jMdNOEkay1LZw/s1600/2014-02-26+01_09_12-Untitled+-+Notepad.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="282" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhP3rl-d8DszMAc8fKGc-GbFerpU0Qd-bVQFR-c0lpb-N1bw_n6zsJqmToCHeU83bkWDFPTTHI0KkagHa41j59EgJEy7hbbs_NLwNkaSw6ek91NhDU6ai9AloHIBt9AY_jMdNOEkay1LZw/s1600/2014-02-26+01_09_12-Untitled+-+Notepad.png" width="400" /></a></div>
<br />
<li>Add this new disk as available to VMs by navigating to <i>Virtual
Machines->Disks->Create</i></li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgUuf-scyMRoivhUprfieWO2KDQVTZx1hJ0zN0saADOCcJIz9ivxL8YlaLj9_qhLKzlNo3-AFHApo_zd1XVXPb_wcEZqSK4KfwLL8gaq8GKVVUiZQ5cKGzmMt-RDWQa7E1J-cDxBszchGE/s1600/2014-02-26+01_20_30-Virtual+machines+-+Windows+Azure.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="358" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgUuf-scyMRoivhUprfieWO2KDQVTZx1hJ0zN0saADOCcJIz9ivxL8YlaLj9_qhLKzlNo3-AFHApo_zd1XVXPb_wcEZqSK4KfwLL8gaq8GKVVUiZQ5cKGzmMt-RDWQa7E1J-cDxBszchGE/s1600/2014-02-26+01_20_30-Virtual+machines+-+Windows+Azure.png" width="400" /></a></div>
<br />
<li>Enter the desired management name for this disk and input or browse to
the URL of the VHD you just uploaded and click the check box. </li>
<li>Attach the disk to your VM by navigating to <i>Virtual
Machines->Ensure your desired VM is
highlighted->Attach->Attach Disk</i></li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxAXIWns9ye3PlvyFZlyUQE6KIY9WHqJvW_F9DQStkGcwNONl0jZ43rcQvrcXLhAr5ANyDlX2VblDAXOp8B-w8OJmHGwjqvZavZezgndkHt8fUXyRFHQMNvrByI0exhTGGncfGYelFMnE/s1600/2014-02-26+01_23_55-Virtual+machines+-+Windows+Azure.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxAXIWns9ye3PlvyFZlyUQE6KIY9WHqJvW_F9DQStkGcwNONl0jZ43rcQvrcXLhAr5ANyDlX2VblDAXOp8B-w8OJmHGwjqvZavZezgndkHt8fUXyRFHQMNvrByI0exhTGGncfGYelFMnE/s1600/2014-02-26+01_23_55-Virtual+machines+-+Windows+Azure.png" width="400" /></a></div>
<br />
<li><span style="font-style: italic;"></span>Select the disk we just
added. Your cache preference will depend on the application. In my case
this is off but you will want to use the methodology outlined in the
first part of this article to test caching impact for your application.
Note a change of cache status requires a VM reboot. </li>
</ol>
Now for a brief tutorial on how to set up our two types of striped disks;
you'll likely only be using one of the two but I'll cover both just in
case. Performance results of each are outlined later in this article. <br />
<ol>
</ol>
<h4>
Set Up a Traditional Software Stripe in Windows</h4>
Setting up a traditional software stripe is easy. I've tested this on
Windows 2003 and higher. <br />
<h4>
</h4>
<ol>
<li>Logon to your VM as an admin and open the Disk Management tool. </li>
<li>If prompted, allow the initialization of the disks.</li>
<li>Right-click on one of the newly created empty volumes and select <i>New
Striped Volume</i>.</li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg2O65MBMUWVhL06iHSFHzIGu8yVF7BafDkBYYp15SqjeM-uIrSqd_wVy7KwxyYTSs4Fgo6nTP7f550jAtUpIz8SUCRGAw0F6nzT4SICcIPnM9BrSNx_Gbb2U-xHMANRBtch7ty0rfQhi4/s1600/2014-02-26+02_01_27-RD+Tabs+64.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="217" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg2O65MBMUWVhL06iHSFHzIGu8yVF7BafDkBYYp15SqjeM-uIrSqd_wVy7KwxyYTSs4Fgo6nTP7f550jAtUpIz8SUCRGAw0F6nzT4SICcIPnM9BrSNx_Gbb2U-xHMANRBtch7ty0rfQhi4/s1600/2014-02-26+02_01_27-RD+Tabs+64.png" width="400" /></a></div>
<br />
<li>Select the desired disks and continue.</li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXNz0Y2WxVA-Z70ZZfX_KB6mokD2KqIloZXxZNepmgo4jN94Q8D0jCp_qMKJOcChyuhPmuS0vdmQW4jdnSxb-jN73d3MJF92iT7A27LAmk3GhY5yzTkP7iVtC7UVPr58gQ6SiKbUuchOM/s1600/2014-02-26+02_02_06-RD+Tabs+64.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="259" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXNz0Y2WxVA-Z70ZZfX_KB6mokD2KqIloZXxZNepmgo4jN94Q8D0jCp_qMKJOcChyuhPmuS0vdmQW4jdnSxb-jN73d3MJF92iT7A27LAmk3GhY5yzTkP7iVtC7UVPr58gQ6SiKbUuchOM/s1600/2014-02-26+02_02_06-RD+Tabs+64.png" width="320" /></a></div>
<br />
<li>Create and format a new NTFS disk using your striped volume. Make sure
to pay attention to the cluster size (results below).</li>
</ol>
<h4>
Setup a Storage Spaces Software Stripe in Windows 2012 or Higher<br />
</h4>
Microsoft introduced a new approach to disk pooling in Windows Server
2012 and Windows 8 called <a href="http://technet.microsoft.com/en-us/library/hh831739.aspx">Storage
Spaces</a>. This interesting new tech allows for a myriad of different
configuration options including disk tiering which can be useful for
on-premise servers. In this case we'll be using the "simple" pool type
which is similar to disk striping. <br />
<ol>
<li>Open Server Manager and navigate to <i>File and Storage Services
-> Volumes -> Storage Pools</i></li>
<li>Under <i>Storage Pools</i> you should see "<i>Primordial</i>".
(As opposed to "Unused Disks". I'm guessing someone was pretty proud of
that.) Right click it and select "<i>New Storage Pool</i>". </li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimnPQgJIIqY59xyOOsoZURG7M3uLh9zExg-Wizv8BiUJdOwC-hNe6patve2phL_jgxMLlVYAuUPtppXybdEnU1mKWc2GtmoZbJlkO1djVZAjDBX_4yckUzYadhwou8AbSQvj7gQ2JvxsU/s1600/2014-03-04+21_45_51-infraspace01+-+infraspace01.cloudapp.net_54837+-+Remote+Desktop+Connection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimnPQgJIIqY59xyOOsoZURG7M3uLh9zExg-Wizv8BiUJdOwC-hNe6patve2phL_jgxMLlVYAuUPtppXybdEnU1mKWc2GtmoZbJlkO1djVZAjDBX_4yckUzYadhwou8AbSQvj7gQ2JvxsU/s1600/2014-03-04+21_45_51-infraspace01+-+infraspace01.cloudapp.net_54837+-+Remote+Desktop+Connection.png" /></a></div>
<br />
<li>Walk through the Wizard selecting each disk you would like to be part
of the pool. </li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgp3kxHAxQ93pqqnciUHASx2rNfw9xhVtMOSxEVUq-OloilLC8nCdNLNZMTUQjGxyvjjPYoZ2tys6cQrkQEo67_jCH4jGWXkUctzMl8FIH3aLMo3Q6A3a7PT5NkV4FxJMwUFOgjzZQ1vCM/s1600/2014-03-04+21_47_23-infraspace01+-+infraspace01.cloudapp.net_54837+-+Remote+Desktop+Connection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="260" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgp3kxHAxQ93pqqnciUHASx2rNfw9xhVtMOSxEVUq-OloilLC8nCdNLNZMTUQjGxyvjjPYoZ2tys6cQrkQEo67_jCH4jGWXkUctzMl8FIH3aLMo3Q6A3a7PT5NkV4FxJMwUFOgjzZQ1vCM/s1600/2014-03-04+21_47_23-infraspace01+-+infraspace01.cloudapp.net_54837+-+Remote+Desktop+Connection.png" width="400" /></a></div>
<br />
<li>On the results page, ensure "<i>Create a virtual disk when this
wizard closes</i>" is selected and click "<i>Close</i>". </li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhfnPxctbkz_11FNKRYtckwfIOsQmrL49gWxDrvZZiF-07zyqUmcuvrGJhLf4-V8xsGFjF3qS1fv4iXi-ogCTYcZoSIiml-RAiWZqGnx4fAYDC42LQ_6FOdVj9r6LwJKQxCNAmhUTvvwds/s1600/2014-03-04+21_47_56-infraspace01+-+infraspace01.cloudapp.net_54837+-+Remote+Desktop+Connection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="260" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhfnPxctbkz_11FNKRYtckwfIOsQmrL49gWxDrvZZiF-07zyqUmcuvrGJhLf4-V8xsGFjF3qS1fv4iXi-ogCTYcZoSIiml-RAiWZqGnx4fAYDC42LQ_6FOdVj9r6LwJKQxCNAmhUTvvwds/s1600/2014-03-04+21_47_56-infraspace01+-+infraspace01.cloudapp.net_54837+-+Remote+Desktop+Connection.png" width="400" /></a></div>
<br />
<li>Walk through the Virtual Disk Wizard, specifying a meaningful name and
selecting simple storage layout and fixed provisioning. </li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4MfUCozY7BrjiaLwzrQrIs3mVSipMcjEWiISMa_-Gzic_e64uGbC2bN_PhcdPDEI3rHo6UtbRyYb7OpB0MQcYcMr6v6mGGvhdesPqXlrWNMLgt2Ne3TOYfpJtEoacAskBfmlVGTv1RT0/s1600/2014-03-04+21_49_35-infraspace01+-+infraspace01.cloudapp.net_54837+-+Remote+Desktop+Connection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="292" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4MfUCozY7BrjiaLwzrQrIs3mVSipMcjEWiISMa_-Gzic_e64uGbC2bN_PhcdPDEI3rHo6UtbRyYb7OpB0MQcYcMr6v6mGGvhdesPqXlrWNMLgt2Ne3TOYfpJtEoacAskBfmlVGTv1RT0/s1600/2014-03-04+21_49_35-infraspace01+-+infraspace01.cloudapp.net_54837+-+Remote+Desktop+Connection.png" width="400" /></a></div>
<br />
<li>On the results page, ensure "<i>Create a volume when this wizard
closes</i>" is selected and click "<i>Close</i>".</li>
<li>Complete the New Volume Wizard specifying your desired drive letter
and desired NTFS cluster size. </li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmJfyavm2-ABCEtRiF0ZdkxYQg2lYAniZnVVNHRP_wJw0QWXhsXNRUa2wMAcmTASTMNUXU6sApLKDN0RM8E-vEPWALYIuK24frjXMdi6IIEJQPJBhNWTTqQCwYfv318moMw9vZdwgQtOc/s1600/2014-03-04+21_51_33-infraspace01+-+infraspace01.cloudapp.net_54837+-+Remote+Desktop+Connection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="292" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmJfyavm2-ABCEtRiF0ZdkxYQg2lYAniZnVVNHRP_wJw0QWXhsXNRUa2wMAcmTASTMNUXU6sApLKDN0RM8E-vEPWALYIuK24frjXMdi6IIEJQPJBhNWTTqQCwYfv318moMw9vZdwgQtOc/s1600/2014-03-04+21_51_33-infraspace01+-+infraspace01.cloudapp.net_54837+-+Remote+Desktop+Connection.png" width="400" /></a></div>
<br />
</ol>
<h3>
Run Tests/Collect Results!</h3>
Now that we have our disks configured, we need to run our tests. For
instructions how how to do so, see part 1 of this topic <a href="http://blog.ittoby.com/2014/05/cloudy-io-performance-deciphering-iops.html">here</a>.
<br />
When analyzing the IOMeter output you will want to pay special attention to
the following metrics:
<br />
<ul>
<li>IOPS (Cumlative, Read, Write, Higher is better)</li>
<li>MBps (Cumlative, Read, Write, Higher is better)</li>
<li>Response Time (Avg, Avg Read, Avg Write, lower is better) </li>
</ul>
If putting the data together for a report, Excel works nicely
as I'll display below. <br />
<h4>
Results</h4>
Now for the most important part, the findings. Tests performed: <br />
<h4>
Sector Size Tests: </h4>
<ul>
<li>1 Disk, 4k Sector Size (default)</li>
<li>1 Disk, 8k Sector Size</li>
<li>1 Disk, 16k Sector Size</li>
<li>1 Disk, 32k Sector Size</li>
<li>1 Disk, 64k Sector Size</li>
<li>3 Disks, 4k Sector Size (results confirmation test)</li>
<li>3 Disks, 32k Sector Size (results confirmation test)</li>
</ul>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEje5j4QymNNX5Y-2qux6MRrhBBTlZrnpBRRfidL1BuyFyubQuYS9L-U_AZw5hRTbNww5TrhFaIpcd82U46AZigHPuzYTbzGptzv4UHRnSlItpgxczbAEjH3QB-TEx253QfrAKGnM2Ko_ss/s1600/2014-05-26+21_38_50-IOTesting.xlsx+-+Excel.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="276" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEje5j4QymNNX5Y-2qux6MRrhBBTlZrnpBRRfidL1BuyFyubQuYS9L-U_AZw5hRTbNww5TrhFaIpcd82U46AZigHPuzYTbzGptzv4UHRnSlItpgxczbAEjH3QB-TEx253QfrAKGnM2Ko_ss/s1600/2014-05-26+21_38_50-IOTesting.xlsx+-+Excel.png" width="640" /></a></div>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;">
<tbody>
<tr>
<td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhlIxOXjHhVfe2FhS3tZCKHeaeOS-JXX1B43A1LqlfV9YnXwPvD3EabcO0G85It14LR01dvCR-1mlY1ermp7rFW9YcT6CDnPhzjeqj3Zk1Jbh4fJvNwV9dJIPnGH2vxeK-Bvx_OrhdjVfk/s1600/2014-05-26+14_56_36-IOTesting.xlsx+-+Excel.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="101" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhlIxOXjHhVfe2FhS3tZCKHeaeOS-JXX1B43A1LqlfV9YnXwPvD3EabcO0G85It14LR01dvCR-1mlY1ermp7rFW9YcT6CDnPhzjeqj3Zk1Jbh4fJvNwV9dJIPnGH2vxeK-Bvx_OrhdjVfk/s1600/2014-05-26+14_56_36-IOTesting.xlsx+-+Excel.png" width="640" /></a></td>
</tr>
<tr>
<td class="tr-caption" style="text-align: center;">Table 1-Cluster
Size Tests</td>
</tr>
</tbody>
</table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;">
<tbody>
<tr>
<td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiR4CmrtpYewhRVmRggfyRdFF7K19NPfOgP3NLy7I1ERmvJEAo67p2493_QrPx8yBJB9vD2QYcKGyrMW5VSLlfUMusQqPw5HNtPNyGfdCJhp8LPG77gCDZGQYEmk6iVOsm96mt-lgp33_8/s1600/2014-05-26+14_57_01-IOTesting.xlsx+-+Excel.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="60" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiR4CmrtpYewhRVmRggfyRdFF7K19NPfOgP3NLy7I1ERmvJEAo67p2493_QrPx8yBJB9vD2QYcKGyrMW5VSLlfUMusQqPw5HNtPNyGfdCJhp8LPG77gCDZGQYEmk6iVOsm96mt-lgp33_8/s1600/2014-05-26+14_57_01-IOTesting.xlsx+-+Excel.png" width="640" /></a></td>
</tr>
<tr>
<td class="tr-caption" style="text-align: center;">Table 2-Cluster
Size Verification</td>
</tr>
</tbody>
</table>
Sector size tests echo what others have observed with Azure; since IOPS
are capped at 500 (or 300 for basic VMs) larger sector sizes can result in
higher throughput. In my case 32k was the sweet spot; depending on your
workload your results will vary slightly. I have seen consistently (albeit
slightly) higher performance with larger sector sizes in Azure. <br />
<ul>
</ul>
<h4>
Legacy Disk Striping Tests:</h4>
<ul>
<li>1 Disk, 32k Sector Size</li>
<li>2 Disks, Striped Volume, 32k Sector Size</li>
<li>2 Disks in 2 Storage Accounts, Striped Volume, 32k Sector Size
(Multiple Storage Account Test)</li>
<li>3 Disks, Striped Volume, 32k Sector Size</li>
<li>4 Disks, Striped Volume, 32k Sector Size</li>
</ul>
<See Bar Charts Below Under Disk Striping Methodology><br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;">
<tbody>
<tr>
<td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhZ_XG7B_fy61dEci9me6WL2PVsNKeRdJ0hUChPr6l0NklNVGwA7vP6nO3Ts6HSHDyRIolg5izbeO1Wb77_OKgLWFVGA74tskmVeUdT_KHItlgZRXjBRiQDhNkoPMKOwr8kNOjiWtzqXc/s1600/2014-05-26+16_07_27-IOTesting.xlsx+-+Excel.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="100" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhZ_XG7B_fy61dEci9me6WL2PVsNKeRdJ0hUChPr6l0NklNVGwA7vP6nO3Ts6HSHDyRIolg5izbeO1Wb77_OKgLWFVGA74tskmVeUdT_KHItlgZRXjBRiQDhNkoPMKOwr8kNOjiWtzqXc/s1600/2014-05-26+16_07_27-IOTesting.xlsx+-+Excel.png" width="640" /></a></td>
</tr>
<tr>
<td class="tr-caption" style="text-align: center;">Table 3-Legacy
Striping and Storage Account Tests</td>
</tr>
</tbody>
</table>
You can see with one disk we get 500 IOPS as expected. From there we can
see a scaling trend that is most definitely not linear. Two disks result
in 33% higher performance, while three disks add an additional 23% (64%
from one disk). Adding the fourth disk actually results in a drop from
three disks, coming it at 5% lower than three and 56% higher than one. <br />
Additionally, we also see that splitting disks across storage accounts
makes no appreciable difference. Note: Bar charts for this results
section have been combined into the graphs below. <br />
<ul>
</ul>
<h4>
Disk Striping Methodology Tests:</h4>
<ul>
<li>2 Disks, Striped Volume 32k Sector Size</li>
<li>2 Disks, Storage Spaces Simple, 32k Sector Size</li>
<li>3 Disks, Striped Volume, 32k Sector Size</li>
<li>3 Disks, Storage Spaces Simple, 32k Sector Size</li>
<li>4 Disks, Striped Volume, 32k Sector Size</li>
<li>4 Disks, Storage Spaces Simple, 32k Sector Size</li>
</ul>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhywSbwP1ev2knDC3ofaS_NIWuK2xj-RILStCUbsN05rivrfoQ863zUFRL3T1nxur2RF4Qq2jnrGV4tVodyLQ5irJYIjl8wblJII78OtSgOJtUR7s_j1cAwXtZIAjRzu9r0jdCrajmaXoA/s1600/2014-05-26+21_20_58-IOTesting.xlsx+-+Excel.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="458" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhywSbwP1ev2knDC3ofaS_NIWuK2xj-RILStCUbsN05rivrfoQ863zUFRL3T1nxur2RF4Qq2jnrGV4tVodyLQ5irJYIjl8wblJII78OtSgOJtUR7s_j1cAwXtZIAjRzu9r0jdCrajmaXoA/s1600/2014-05-26+21_20_58-IOTesting.xlsx+-+Excel.png" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiu_j_smMVu_DJQGiHjXf1v7HV63viqEvGMO2_qanCdGn1b65v8MJf-guhmOpeNqDJv7G0YB2hyphenhyphenDFyXE8GReiQ1Xr-1d25e_BcAltjtMKXy3f7cOrzfwWGiK84gDc8b4o38TT77GLyr-2s/s1600/2014-05-26+21_21_25-IOTesting.xlsx+-+Excel.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="504" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiu_j_smMVu_DJQGiHjXf1v7HV63viqEvGMO2_qanCdGn1b65v8MJf-guhmOpeNqDJv7G0YB2hyphenhyphenDFyXE8GReiQ1Xr-1d25e_BcAltjtMKXy3f7cOrzfwWGiK84gDc8b4o38TT77GLyr-2s/s1600/2014-05-26+21_21_25-IOTesting.xlsx+-+Excel.png" width="640" /></a></div>
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;">
<tbody>
<tr>
<td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhy7fJLbsmSRAvpHOK1CW1TrJcxEmaJZDPH3wvRON2w6qu2xmriBLCOAYe4xAE2wR1JjNy9bvkm2_hTwviuEftCB50NIGcX4TsqjcyrvgxXnfm6PyWiht542ZkbcIyv9KQlIJDxwF4IfYE/s1600/2014-05-26+14_58_01-IOTesting.xlsx+-+Excel.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="114" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhy7fJLbsmSRAvpHOK1CW1TrJcxEmaJZDPH3wvRON2w6qu2xmriBLCOAYe4xAE2wR1JjNy9bvkm2_hTwviuEftCB50NIGcX4TsqjcyrvgxXnfm6PyWiht542ZkbcIyv9KQlIJDxwF4IfYE/s1600/2014-05-26+14_58_01-IOTesting.xlsx+-+Excel.png" width="640" /></a></td>
</tr>
<tr>
<td class="tr-caption" style="text-align: center;">Table 4-Legacy
Striping vs. Storage Spaces Test</td>
</tr>
</tbody>
</table>
Now we compare legacy striping to the newly introduced Storage Spaces.
Two disk scaling is a definitive win for Storage Spaces, while beyond that
legacy striping generally performs better (save max latency). In my
opinion two disk Storage Spaces stripe is the sweet spot here (56% IOPS
improvement!) when considering that the with more disks we add complexity
that doesn't pan out on the performance side. <br />
<h3>
Conclusion</h3>
I hope you have found these results interesting; I certainly have. Even
if you choose not to run these tests yourself I hope my results prove
helpful when sizing your machines. Since the access pattern I used is
relatively universal it should be applicable in most scenarios. <br />
Software level disk striping works relatively well in Microsoft Azure to
increase per-disk performance in lieu of a provider level solution similar
to Amazon EBS <a href="http://aws.typepad.com/aws/2012/08/fast-forward-provisioned-iops-ebs.html">provisioned
IOPS</a>. Splitting the workload across logical disks or VMs is
preferred but not applicable to all workloads. When employing this
solution make sure you select only locally redundant replication because
Micrsoft warns that geo-redundant replication may cause data consistency
issues on the replication target. <br />
For additional information see the links near the top of this article.
Thanks for reading!Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com13tag:blogger.com,1999:blog-5458589696790815336.post-14142759280685072312014-05-20T21:37:00.000-05:002014-05-28T01:13:45.361-05:00Cloudy I/O Performance - Deciphering IOPS in IaaS (Part 1 of 2) <p><em>Note:</em> This is part 1 of a 2 part post. Part 2 can be found <a href="http://blog.ittoby.com/2014/05/cloudy-io-performance-increasing-azure.html">here</a>.<br>
</p>
<h3> Foreword</h3>
<br>
Disk performance scaling options in the public cloud seem limited
(particularly in Azure as of this writing), but there are ways to increase
your IOPS in IaaS solutions. To add to the performance problem,
transactional costs of running application tests can be not only time
consuming but expensive. To tune your storage performance reliably you will
need a fast, consistent way to test different configurations. This article
will cover that methodology and lead into a results/guidance article for
Azure (but applicable to others) IaaS storage performance. <br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5geaEIXrGvrtVS34s6q7m8YAA0f6d4WKTJ0RFrjhMG-SNAgLKNAxU0RjB_P97Vms3h21oIa0ZdfQOJHcEq94taO-6oKNbXhgCJE_6N4yllfYDoqNCIwBDz3W2Q2vEswoTmbaB1jFVdFM/s1600/cloudyWorkDoneish.jpg"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5geaEIXrGvrtVS34s6q7m8YAA0f6d4WKTJ0RFrjhMG-SNAgLKNAxU0RjB_P97Vms3h21oIa0ZdfQOJHcEq94taO-6oKNbXhgCJE_6N4yllfYDoqNCIwBDz3W2Q2vEswoTmbaB1jFVdFM/s1600/cloudyWorkDoneish.jpg"
border="0" height="484" width="640"></a></div>
<br>
<br>
We'll be doing this testing on Windows, but you could also easily do this on
Linux and the results that I'll be sharing are just as applicable there. To
accomplish this testing we'll be using the following tools:<br>
<ul>
<li><a href="http://www.iometer.org/">IOMeter</a> (for Linux try <a href="http://iorate.org/">iorate</a>
or <a href="http://www.linuxintro.org/wiki/Iometer">iometer on Wine</a>)</li>
<li><a href="http://blog.ittoby.com/2013/03/azure-powershell-setup-and-iaas-mass.html">PowerShell
cmdlets for Azure</a></li>
<li><a href="http://technet.microsoft.com/library/dd163558.aspx">Disk
Management</a></li>
</ul>
<p>Let's begin!</p>
<h3> <b> Execution</b></h3>
We will proceed in the following order:<br>
<ol>
<li>Analyze Workload</li>
<li>Create Test Scenarios</li>
<li>Collect and Analyze Results (Mainly in Part 2)</li>
<li>Findings (In Part 2) </li>
</ol>
<br>
<h4> Assumptions</h4>
If you plan on emulating my tests you'll need to have access to the
following:<br>
<ul>
<li>Microsoft Windows Azure account (note this methodology will worth with
EC2 or any other platform including standard hardware/on-prem VMs)</li>
<li>IaaS VM Configured. A medium size is recommended for testing 4 disks
or fewer to limit the available memory. More on that below. </li>
<li>Administrator access to your VM. </li>
<li>Your workload is in fact disk I/O bound. If you're not sure of that
you may want to start with <a href="http://technet.microsoft.com/en-us/magazine/2008.08.pulse.aspx">this
article</a>. </li>
<li>Awareness that you will incur additional storage transaction costs by
running these tests. </li>
</ul>
<br>
<ul>
</ul>
<h4>Analysis/Create Workload</h4>
<p><em>Note:</em> If you're just trying to get a general sense for your VM
I/O performance capability, you don't need to collect data for a custom
access specification. IOMeter includes several tests you can use so skip
to the "Install IOMeter..." section below. </p>
<p>The first thing we need to do is create our workload. By using IOMeter we
can develop custom access patterns that model common workloads and have
the tool and workloads installed and configured in minutes on any machine.
There is nearly endless information on this topic, so I won't attempt to
create a definitive source here. For details on how to configure and use
IOMeter, see the following videos/articles: </p>
<ul>
<li><a href="http://www.youtube.com/watch?v=lnztswVcxLg">Video Tutorial
from iTechStorm</a></li>
<li><a href="http://kristau.net/blog/814/">Kristau.net Short
Tutorial/Intro</a></li>
<li><a href="http://www.iometer.org/doc/documents.html">IOMeter Users
Guide</a></li>
<li><a href="https://communities.vmware.com/docs/DOC-3961"><span style="color: #0000ee;"><span
style="text-decoration: underline;">VMWare Community:Storage
System Performance Analysis with Iometer</span></span></a></li>
</ul>
<p> To create an accurate workload you will need a good understanding
of the access pattern of your application. If you don't have that
information you can use a tool like Perfmon to do analysis on a fully
configured platform. The following counters will be of interest when
creating your access specification: </p>
<ul>
<li>Physical or Logical Disk: Average Disk Bytes per Read</li>
<li>Physical or Logical Disk: Average Disk Bytes per Write</li>
<li>Physical or Logical Disk: Disk Read Bytes/sec</li>
<li>Physical or Logical Disk: Disk Write Bytes/sec</li>
<li>Physical or Logical Disk: Disk Reads/sec</li>
<li>Physical or Logical Disk: Disk Writes/sec</li>
</ul>
<p>For further information, see <a href="http://technet.microsoft.com/en-us/library/cc722466.aspx">this
excellent Technet Article</a>. </p>
<p>By collecting this data during the access pattern you wish to emulate you
can accurately estimate (with one caveat) the information needed to create
the IOMeter access specification. That caveat is determining the
sequential vs. random access pattern of the platform since Perfmon
analysis will reveal the rest. To determine that, you'll need an
understanding of how the platform stores and accesses/writes data. In my
case I'm tuning my VM for <a href="http://www.splunk.com">Splunk</a>,
which uses a Map/Reduce functionality that has a highly sequential
read/write pattern. If you are unsure of your access pattern then err on
the side of configuring for mostly random access (90% or so) since it is
generally more common and demanding of the underlying storage
subsystem. </p>
<h4>Install IOMeter and Config Access Specification</h4>
<p>The following actions can be done on your target testing platform or a
different machine to stage settings. We'll be saving our settings for
quick use later. </p>
<ol>
<li>Download and install IOMeter on your server. There are a series of
ways to stage files on any VM, but if you're looking for a quick way in
the Microsoft ecosystem check out my <a href="http://blog.ittoby.com/2013/02/azure-iaas-vm-tip-use-skydrive-to.html">Onedrive/Azure
post</a>.</li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjvcZC7o8yXisKhmp7Ln9roSi1MuvUHAWBHhck9Bv7azGa1DLJo_pu81ADFj8CXCl-xiioqJT_ae7eplnzbjtINSQmRUL0F4YDdd_KLAmsRe5mRbmYm6NzahgN1K8A1kNuP0N3_M6EQjdI/s1600/2014-02-26+01_35_05-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjvcZC7o8yXisKhmp7Ln9roSi1MuvUHAWBHhck9Bv7azGa1DLJo_pu81ADFj8CXCl-xiioqJT_ae7eplnzbjtINSQmRUL0F4YDdd_KLAmsRe5mRbmYm6NzahgN1K8A1kNuP0N3_M6EQjdI/s1600/2014-02-26+01_35_05-RD+Tabs+64.png"
border="0" height="311" width="400"></a></div>
<br>
<li>Open IOMeter as administrator.</li>
<br>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiY5dC-nEkjHh3sxQnOrS6Ba5ICpMmm1zkki69yoTy5Q6KTwXK0MAHwxpHkB_7WIV61BS2PtB_jzx2Yk1UFBcnQ67NQrj_ZA6P7RoQ1Nat7F43zxqk999nCQYHG_NZTsS1hFid6NLFmZAQ/s1600/2014-02-26+01_35_54-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiY5dC-nEkjHh3sxQnOrS6Ba5ICpMmm1zkki69yoTy5Q6KTwXK0MAHwxpHkB_7WIV61BS2PtB_jzx2Yk1UFBcnQ67NQrj_ZA6P7RoQ1Nat7F43zxqk999nCQYHG_NZTsS1hFid6NLFmZAQ/s1600/2014-02-26+01_35_54-RD+Tabs+64.png"
border="0" height="320" width="296"></a></div>
<br>
<li>Under "Topology" configure your workers. Each worker represents one
thread generating I/O. By default it will create one per CPU thread
available, but in most cases you will only want one worker per process
you are emulating. In my case I'm assuming one large query at a time
(and we'll scale from there), so I'll be testing with one worker. If you
are unsure stick to one worker and you can move up from there when you
become more familiar. </li>
<br>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhazuEGZxhi8kHD03VwkysAPNwe1ItyenkouG8WuiA-lA-Y92tbAvwW5kG_hCr1Kx01GwejOCA0NSOs2hPE-R-QStz9D2fZ3jth7hZUd3w4wAaKaSPAzxKQaqc1TdINbk2bF_BGexQbi7c/s1600/2014-02-26+01_39_06-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhazuEGZxhi8kHD03VwkysAPNwe1ItyenkouG8WuiA-lA-Y92tbAvwW5kG_hCr1Kx01GwejOCA0NSOs2hPE-R-QStz9D2fZ3jth7hZUd3w4wAaKaSPAzxKQaqc1TdINbk2bF_BGexQbi7c/s1600/2014-02-26+01_39_06-RD+Tabs+64.png"
border="0"></a></div>
<br>
<li>Under "Disk Targets" select the disk you wish to test. This can change
in later runs so if the disk you want to test isn't present here select
a placeholder. </li>
<li>Under "Disk Targets" configure your "Maximum Disk Size". This
configures the size of your test file in sectors, which are considered
to be <a href="http://en.wikipedia.org/wiki/Disk_sector">512 bytes</a>
each. To lessen the impact of OS caching you need to ensure this value
exceeds the amount of RAM present on the machine to be tested. In my
case I'll be testing on a 6GB RAM machine with a (approx) 7.5GB file, so
I've configured it for 15000000 sectors. (15000000 sectors * 512 bytes
per sector=7,680,000,000 bytes) To do this quickly take your total
desired size (in bytes!) and divide it by 512. (If you aren't certain
you got it right, check the size iobw.tst file created at the root of
your target drive after the first test is complete)</li>
<br>
<table class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"
align="center" cellpadding="0" cellspacing="0">
<tbody>
<tr>
<td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBBhwrO_M4f4N4aswH1jNMsHYtgLmUkPnINt5xCS4lelZd4_aLZXPcckQwQ6DrUG-TKsWaUQO-XuKNwOsKFpr6chelc9K2r_Td8NSm-hmlKC682jTeCQSHHOklZI2kcINoMdyfJmEi6Iw/s1600/2014-02-26+01_38_46-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBBhwrO_M4f4N4aswH1jNMsHYtgLmUkPnINt5xCS4lelZd4_aLZXPcckQwQ6DrUG-TKsWaUQO-XuKNwOsKFpr6chelc9K2r_Td8NSm-hmlKC682jTeCQSHHOklZI2kcINoMdyfJmEi6Iw/s1600/2014-02-26+01_38_46-RD+Tabs+64.png"
border="0" height="130" width="400"></a></td>
</tr>
<tr>
<td class="tr-caption" style="text-align: center;">Testing T: With a
4.5GB Test File</td>
</tr>
</tbody>
</table>
<br>
<li>Under "Disk Targets" configure your maximum outstanding I/O. This
varies depending on access spec and OS, but I've had consistent (with
real application access) results testing with 16 maximum outstanding I/O
on windows. </li>
<li>Under "Test Setup" configure your "Ramp Up Time" and "Run Time". Ramp
up need only be about 20 seconds for most scenarios and run time is best
between 1 and 10 minutes. My results are based on (many per config) 5
minute tests. </li>
<li>Under "Access Specification" select your access spec. There is far too
much to get into here; either select one or many existing access
specifications that suit you needs ("4k 75% read" is a good start if you
don't care) or create your own based on your findings from the
Analysis/Create workload section. For the purposes of my test I made a
"_Splunk" access spec with the following characteristics ascertained
from my earlier performance testing: </li>
<ol>
<li>Transfer Request Size: 32kB (NOTE: My access spec may not reflect
yours. Most won't be this large)</li>
<li>Percent Read/Write Distribution: 53% Write/47% Read (NOTE: My access
spec may not reflect yours. Most specs won't be this write heavy)</li>
<li>Percent Random/Sequential Distribution: 75% Sequential/25% Random
(NOTE: My access spec may not reflect yours. Most specs won't be this
sequential)</li>
<div class="separator" style="clear: both; text-align: center;"> <br>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXUv3jFxopA0cHV1K3P_DRW9oI4if5H7KFzz9NhpgNkCVbtauw5HT5lZU9sZY342Mhz0SctYIBlibN5ZGLYNyIeGptjn1LQwxt1sFlb5M77rZEHFXQf3OQIJLk_1i4PxUJnIub80oH8u4/s1600/2014-05-19+23_54_54-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXUv3jFxopA0cHV1K3P_DRW9oI4if5H7KFzz9NhpgNkCVbtauw5HT5lZU9sZY342Mhz0SctYIBlibN5ZGLYNyIeGptjn1LQwxt1sFlb5M77rZEHFXQf3OQIJLk_1i4PxUJnIub80oH8u4/s1600/2014-05-19+23_54_54-RD+Tabs+64.png"
border="0" height="292" width="400"></a></div>
<br>
</ol>
<li>Add your access specification to the list of queued tests if you
haven't done so already (removing all others).</li>
<br>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgyjNWXBZqcpH6-uuMZjo3KPNV96X54wRdWcrqsT11kZiW4-7JkOrAz6u7vli895kxnUILXLsIdcKDyRiJ-IuTMcGCE0simD3dakbKD70a7a_xGIJY77ol7uoIwe_7fwl73J7GOpIfDuoE/s1600/2014-02-26+01_39_20-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgyjNWXBZqcpH6-uuMZjo3KPNV96X54wRdWcrqsT11kZiW4-7JkOrAz6u7vli895kxnUILXLsIdcKDyRiJ-IuTMcGCE0simD3dakbKD70a7a_xGIJY77ol7uoIwe_7fwl73J7GOpIfDuoE/s1600/2014-02-26+01_39_20-RD+Tabs+64.png"
border="0" height="258" width="400"></a></div>
<br>
<li>Click the disk icon to save the settings to an ICF file. This file
will save all your settings including custom access specifications if
applicable. Since this file is what you'll use to shortcut future
testing, save it somewhere easy to transfer to other VMs such as
OneDrive, Dropbox, SpiderOak, etc.</li>
</ol>
<h4>Run the Test</h4>
<p>After setting up or loading your test settings, all you need do is click
the green flag to start the test and then select where you would like to
save the results. Make sure you don't overwrite any previous results and
give the file a meaningful name so you remember what this test represents
later, i.e.
"results_3disk_1_StorAcct_Striped_32k_sectors_noCache_run1.csv" or
similar. </p>
The test will run for the configured time and then you will be able to run
additional tests or analyze results. Since the output is in CSV format, the
natural place to look at this data is Excel. When IOMeter starts for the
first time on a given disk it needs to create the test file. This will take
quite awhile in both Amazon EC2 and Azure. (15 mins for my 7.5G for example)
I believe this is due to the way space is allocated on the backend storage.
Once this is created, however, you can run subsequent tests on the same
volume without needing to wait for the test file to be created. Once the run
is done I recommend running several more to ensure your tests aren't subject
to wild performance swings. More on analysis in part 2 of this
article. <br>
<br>
<h3>How Much Will This Cost?<br>
</h3>
<p>Since you're charged by transaction I'm sure you will be wondering how
much this will cost. Let's break down your above baseline (system running)
cost in Azure: </p>
<p>IOPS are currently capped at <a href="http://msdn.microsoft.com/en-us/library/windowsazure/dn197896.aspx">500</a>
for standard tier machines (300 for basic). Storage transactions are
currently $0.01 per 100,000. (<a href="http://www.windowsazure.com/en-us/pricing/details/storage/pricing-changes/">halved
on 3/14/14</a>) For every 5 minute test per disk you access you will
then execute a maximum of 150,000 transactions. As a one time per
configuration cost, you will need to build the test file which will be
(test file size/volume sector size) transactions. For example, a 7.5 Gib
test file will be approximately 1,875,000 transactions assuming a default
4kb sector size. (7,500,000,000/4000)</p>
<p>Test transactions + creation transactions = 2 million or so IOPS, or <strong>$0.20</strong>
@ .01 per 100,000. So... not much. The amount is generally trivial on
Amazon EC2 as well. While this methodology will save you some in
transaction costs, the main savings will be in time & labor. (which is
usually our real cost anyhow!) </p>
<h3>Further Optimization<br>
</h3>
<p>Once you are comfortable with this process I would advise doing the
following to further optimize this process. After doing so you may be able
to automate the whole routine!</p>
<ul>
<li>Create standard Perfmon counter sets for disk access and save/import
them <a href="http://technet.microsoft.com/en-us/library/cc766318.aspx">as
a template</a>. </li>
<li>Script the Perfmon analysis with <a href="http://technet.microsoft.com/en-us/library/hh849686.aspx">PowerShell</a>. </li>
<li>Create or download IOMeter templates for common access routines and
include them with your set. </li>
<li>Script the installation and running of IOMeter, including multiple
runs and uploading results to a common location. Easy to do with
PowerShell and refer to the <a href="http://csis.pace.edu/%7Elombardi/sciences/computer/systems/windows/docs/iometer.pdf">IOMeter
manual</a> for command line options (page 75 or so).</li>
<li>Package up all your assets with a custom installer and put it in an
easy to get location. (mmmm... <a href="https://github.com/chocolatey/chocolatey">Chocolatey</a>)</li>
<li>If you want angry followers and think digital bits are out there to be
wasted, auto tweet your results! (maybe not this) </li>
</ul>
<h3>In Closing<br>
</h3>
<p>I/O testing in the cloud is certainly feasible but requires a little
extra discipline. With several access specifications in your toolkit you
can conquer most performance problems quickly. What to do if your cloud
platform doesn't provide your desired IOPS? Coming up in part 2! </p>Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com0tag:blogger.com,1999:blog-5458589696790815336.post-59366278382647674422014-04-21T20:49:00.000-05:002014-08-09T15:57:43.443-05:00Web Application Proxy Server in 2012 R2<!DOCTYPE html>
<html>
<head>
<meta content="text/html; charset=windows-1252" http-equiv="content-type">
<title>Setup of Web Application Proxy Server in Windows 2012 R2</title>
</head>
<body> When Microsoft <a href="http://blog.ittoby.com/2012/10/microsoft-clarifies-tmgforefront-death.html">discontinued</a>
<a href="http://en.wikipedia.org/wiki/Microsoft_Forefront_Threat_Management_Gateway">Threat
Management Gateway</a> (which once was <a href="http://en.wikipedia.org/wiki/Microsoft_Forefront_Threat_Management_Gateway#Microsoft_Proxy_Server">Proxy</a>
and then <a href="http://en.wikipedia.org/wiki/Microsoft_Forefront_Threat_Management_Gateway#ISA_Server_2000">ISA</a> server)
I must admit I was disappointed; it was a relatively inexpensive <b>authenticated</b> reverse
proxy that worked with Exchange Server as well as many other complicated
products. In the interim we were told that <a href="http://en.wikipedia.org/wiki/Microsoft_Forefront_Unified_Access_Gateway">Unified
Access Gateway</a> would be the replacement, but that product isn't as
well suited to the task.<br>
<br>
Several alternatives are out there, including: Kemp, F5, Nginx, and Squid
but either the price or the relative difficulty of setup isn't in line with
TMG. Fortunately starting in Windows 2012R2 Microsoft introduced Web
Application Proxy which largely fills the gap. <br>
<br>
<br>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhB5Yi28-Xed9xUjUk0FEqLxOUraBrTTryLmHuCdMP3GO48Yk0LLvBbYbcAAtVqTZ-hqWmGx8Th6Wx1wekdz6zl7TB9MzPZk_HEmjldOOWXzUHpeUDsyGVoYHpij2IvlGcUcTfgJIKApgI/s1600/web_App_Proxy_UR_Welcome.jpg"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhB5Yi28-Xed9xUjUk0FEqLxOUraBrTTryLmHuCdMP3GO48Yk0LLvBbYbcAAtVqTZ-hqWmGx8Th6Wx1wekdz6zl7TB9MzPZk_HEmjldOOWXzUHpeUDsyGVoYHpij2IvlGcUcTfgJIKApgI/s1600/web_App_Proxy_UR_Welcome.jpg"
border="0"></a></div>
<div style="text-align: center;">Web Application Proxy/Server 2012r2 release
party.Trust me, I paid big bucks for this insider photo.</div>
<br>
<h3>What is Web Application Proxy?</h3>
<strong>W</strong>eb <strong>A</strong>pplication <strong>P</strong>roxy
(WAP from henceforth) is based on and replaces <a href="http://technet.microsoft.com/en-us/windowsserver/dd448613.aspx">Active
Directory Federation Services</a> Proxy 2.0. In addition to the ADFS Proxy
functionality it also introduces the ability to expose internal resources to
external users. These users can be pre-authenticated (and then impersonated
for <a href="http://en.wikipedia.org/wiki/Single_sign-on">SSO</a>) against
your Active Directory infrastructure using ADFS prior to being allowed
access to resources. <br>
<br>
<h3>Wait, This is ADFS Proxy 3.0?</h3>
<p>Yup! That and more. Here's what you can do with it:</p>
<ul>
<li>Authorize external users for access to other claims-aware external or
internal resources (Generally <a href="http://en.wikipedia.org/wiki/Software_as_a_service">SaaS</a>).</li>
<li>Allow access (by "reverse" proxy) to multiple internal applications on
the same port. </li>
<li>Pre-Authenticate users against Active Directory via <a href="http://en.wikipedia.org/wiki/Kerberos_%28protocol%29">Kerberos</a>
or <a href="http://msdn.microsoft.com/en-us/library/windows/desktop/aa378749%28v=vs.85%29.aspx">NTLM</a>
to facilitate SSO and access to internal applications (if desired) </li>
<li>Expose multiple internal resources on a single IP address/port
(generally 443) differentiated by hostname</li>
<li>Loadbalance using a session affinity based solution in front of WAP</li>
</ul>
<p> </p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEilNy9ifn_3kBXr6kRZ5b8pwAiqQlqlpsNSWuRt4D7BYOU088L_Ju3dlNsmZIdukT6O7tnsi2eU3eWzSn5thDlBbdIQgP7H4fc4r9VqvFkKerCeiETHF5o9akE1-Jf9NaxC7hV8r7q8m0Y/s1600/WebAppProxy.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEilNy9ifn_3kBXr6kRZ5b8pwAiqQlqlpsNSWuRt4D7BYOU088L_Ju3dlNsmZIdukT6O7tnsi2eU3eWzSn5thDlBbdIQgP7H4fc4r9VqvFkKerCeiETHF5o9akE1-Jf9NaxC7hV8r7q8m0Y/s640/WebAppProxy.png"
border="0"></a></div>
<ul>
</ul>
<h3>Let's Go!</h3>
<p>This article will cover the following: </p>
<ul>
<li>WAP requirements</li>
<li>Set up</li>
<li>Forwarding a couple of sample applications</li>
<li>Troubleshooting</li>
</ul>
<h3>Software Requirements</h3>
<p>Web application proxy is available on Windows Server 2012 R2 and higher,
and it <a href="http://technet.microsoft.com/en-us/library/dn383650.aspx">requires
ADFS 3.0</a> to be available on the back end. For assistance in setting
up ADFS 3.0, see my article <a href="http://blog.ittoby.com/2014/01/cloud-authentication-primer-basic.html">here</a>.
If you would like to proxy authentication for non-claims aware
applications, I.E. Exchange OWA pre-2013 SP1 (<a href="http://technet.microsoft.com/library/dn635116%28EXCHG.150%29.aspx">SP1
Claims</a>) or Kerberos/NTLM apps, you will need to have the WAP server<strong>
joined to your domain</strong>. </p>
<p>Additionally, you'll need the certificate (private and public key) from
your ADFS server and one certificate (again, private and public) for each
application you intend to proxy. These certificates must be trusted by
your clients, so generally external globally trusted (Digicert for
example) certificate authorities are preferred. The certificates need to
be installed under the "Personal" portion of the "Local Machine" store on
the machine you intend to use as your WAP proxy. If you only intend to
host internal resources to domain-joined computers connecting remotely you
can use an internal PKI provided your clients trust your issuing CA(s).
For information on how to setup an internal CA, see my article <a href="http://blog.ittoby.com/2012/04/creating-two-tier-pki-windows-2008r2.html">here</a>.
If you need help exporting your public and private key from your ADFS
server and other services, see <a href="http://technet.microsoft.com/en-us/library/cc737187%28v=ws.10%29.aspx">this</a>
article. Note that if these certificates are marked as non-exportable you
will need new certificates for those services, so make sure you plan
accordingly. </p>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiuderMCbpK7NJcOy2X47_PtgEzd5KwOa2zGvk13KVDjynJ_Yqz12Ipl1L9x0mV_xuE3_C_PVauSTfBZXDNLgP8R2cGgKMcyTcBFzEtY677XP_qT2D9e_aZgH-i11xFDI2zz20SbrK6xt4/s1600/2014-01-31+23_53_11-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiuderMCbpK7NJcOy2X47_PtgEzd5KwOa2zGvk13KVDjynJ_Yqz12Ipl1L9x0mV_xuE3_C_PVauSTfBZXDNLgP8R2cGgKMcyTcBFzEtY677XP_qT2D9e_aZgH-i11xFDI2zz20SbrK6xt4/s1600/2014-01-31+23_53_11-RD+Tabs+64.png"
border="0" height="383" width="400"></a></div>
<br>
<h3>Connectivity and Hardware/VM Requirements</h3>
<p>Preferably, your WAP server should be placed in a <strong>D</strong>e-<strong>M</strong>ilitarized
<strong>Z</strong>one with a firewall on either side of it. The machine
can operate with either one or two <strong>N</strong>etwork <strong>I</strong>nterface
<strong>C</strong>ards, but for proper security I recommend two NICs; one
internal and one external. Other connectivity options will work, including
branching into your internal network on the inside interface, but I won't
be covering those scenarios in detail. For all connectivity options see
the following diagram:</p>
<p> </p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipFf-ERZ6W5G52-JdxMn2pkIP9X9aOIsosm5l-o1Qwh5Qjxn8xerI8QX1tr6kUgz6nsJ_2ybER3fkaKVf6bzS9T3ko4I8W3UAwKl5Syoa7gpYOAa5AJQd31fgqFiHckYF4EX2RbQ2Zf5g/s1600/WebAppProxyConnectivity.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipFf-ERZ6W5G52-JdxMn2pkIP9X9aOIsosm5l-o1Qwh5Qjxn8xerI8QX1tr6kUgz6nsJ_2ybER3fkaKVf6bzS9T3ko4I8W3UAwKl5Syoa7gpYOAa5AJQd31fgqFiHckYF4EX2RbQ2Zf5g/s640/WebAppProxyConnectivity.png"
border="0" height="509" width="533"></a></div>
<p>As for the hardware you can use either real hardware or a VM assuming you
have a proper DMZ NIC setup on your Hyper-V/ESX/Xen/whatever host(s). WAP
is not a particularly demanding application and uses very little I/O. It
is also horizontally scalable with a network level load balancer (f5) so I
won't give direct guidance on specifications since it would likely have
little relevance to your configuration. As in most cases, performance
evaluation and configuration change is the way to go. </p>
<p>After deciding on your hardware and installing the OS, you'll need to
configure the NICs. We'll cover that in the next section...</p>
<h3>Installation</h3>
<p>Now that the hardware and OS are ready to go, let's configure the NICs: </p>
<h4>Network Configuration</h4>
<ol>
<li>First open the "Network and Sharing Center" and click "Change Adapter
Settings". Re-name the NICs "External" and "Internal" according to how
they are connected to avoid confusion during set up and troubleshooting.<br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgG6H7H4-EE7Uqz3MS3V9ZJQLHjdZYb68KX7AmNTU-Gk2QKNtSP9s9MQeZ5M0MYnX32km8zAHW3i4AGbQBCfx6MFzwzEBI5-eD1gJWKNaS9kp0_XCwcT-sCVJrlomJcfcPSPxFa0UwQie8/s1600/2014-01-31+02_18_57-server+on+server+-+Virtual+Machine+Connection.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgG6H7H4-EE7Uqz3MS3V9ZJQLHjdZYb68KX7AmNTU-Gk2QKNtSP9s9MQeZ5M0MYnX32km8zAHW3i4AGbQBCfx6MFzwzEBI5-eD1gJWKNaS9kp0_XCwcT-sCVJrlomJcfcPSPxFa0UwQie8/s1600/2014-01-31+02_18_57-server+on+server+-+Virtual+Machine+Connection.png"
border="0" height="39" width="320"></a></div>
<br>
</li>
<li>Give each NIC appropriate IP address settings. The subnet for each
will depend on your firewall/switch configuration. Some firewall
configurations may require communication stay on a single subnet but if
given a choice it is generally better to have them on different subnets.
(2 NICs) Leave the default gateway on the internal NIC blank. If your
WAP server is not domain joined because you intend on using only claims
auth or passthrough (not delegation) then leave your DNS servers blank
on the internal NIC as well and be sure to execute step 4. </li>
<li>If the WAP server needs to access resources (ADFS, DC, App) on a
subnet other than that the internal NIC is connected to, you will need
to add a static route to the server so it knows how to get to that
network. For example, if your WAP server is on 192.168.1.10/24, your
ADFS server is 192.168.2.5/24, and your gateway is 192.168.1.1, you
would issue the following command from an elevated command prompt: <em>route
ADD 192.168.2.0 MASK 255.255.255.0 192.168.1.1 IF 192.168.1.10 -p </em>.
For more information, see <a href="http://technet.microsoft.com/en-us/library/cc757323%28v=WS.10%29.aspx">this</a>
article.<strong></strong></li>
<li><Only if you haven't specified DNS servers on the internal
NIC>To look up the ADFS server for claims verification you will need
to add each internal ADFS server address to your
%SYSTEMROOT%\system32\drivers\etc\hosts file. Do this now; if you need
further instructions see <a href="http://www.rackspace.com/knowledge_center/article/how-do-i-modify-my-hosts-file#Windows_Vista">this</a>
article.</li>
<li>Now we'll secure the external NIC. Open the properties of that NIC and
on the "Networking" tab unbind everything except for "QoS Packet
Scheduler" and the protocol you intend on using (IPv4 or IPv6). </li>
<li>If using IPv4, drill into the properties of that protocol and select
"Disable NetBIOS over TCP/IP" under the "WINS" tab. Also ensure you
disable "Register this connection's address in DNS" on the "DNS" tab. <br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrvgZ9jjLIM3je_IIo80FFXB9dE928JyhQS9oaC5T9cdvEXG3l852GMS3J8VGKjbToKt9n2JXRpETJrj6bl4eeBr0h3E0Wv24RZ_eb5BS8Xv03jM1YEoaPl-hS-eu1QHcbI76rtSWkV2I/s1600/2014-01-31+02_20_12-server+on+server+-+Virtual+Machine+Connection.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrvgZ9jjLIM3je_IIo80FFXB9dE928JyhQS9oaC5T9cdvEXG3l852GMS3J8VGKjbToKt9n2JXRpETJrj6bl4eeBr0h3E0Wv24RZ_eb5BS8Xv03jM1YEoaPl-hS-eu1QHcbI76rtSWkV2I/s1600/2014-01-31+02_20_12-server+on+server+-+Virtual+Machine+Connection.png"
border="0" height="320" width="267"></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDO9sWsb0sc_nv5Tnyp97inW0up8RKmlOg3INu-Swlg_EGQ6ft0_bb1z95gsFfsuci0YBpjOxqjjZz9z2v5jHZiKVMSHYJrN5i_MhrReTdsN7r3puss_NIcveftU31wa-mB80C7JKaLjg/s1600/2014-01-31+22_40_28-server+on+server+-+Virtual+Machine+Connection.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDO9sWsb0sc_nv5Tnyp97inW0up8RKmlOg3INu-Swlg_EGQ6ft0_bb1z95gsFfsuci0YBpjOxqjjZz9z2v5jHZiKVMSHYJrN5i_MhrReTdsN7r3puss_NIcveftU31wa-mB80C7JKaLjg/s1600/2014-01-31+22_40_28-server+on+server+-+Virtual+Machine+Connection.png"
border="0" height="400" width="282"></a></div>
<br>
</li>
<li>On your external firewall, open the ports for the services you wish to
forward. (443 would be common)</li>
<li>On your internal firewall, open ports necessary for AD/other
communication. <a href="http://msmvps.com/blogs/acefekay/archive/2011/11/01/active-directory-firewall-ports-let-s-try-to-make-this-simple.aspx">Here</a>
is an excellent guide. </li>
</ol>
<h4>WAP Installation</h4>
<ol>
<li>In server manager, click "Manage->Add Roles and Features".</li>
<li>Click "Next" on the "Before you begin" screen.</li>
<li>For "Installation Type" select "Role-based or feature-based
installation" & click "Next".<br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1kBgVzXlXCdYNvZzxIXfJLyp23zT1M2pYDHX-LCF1nPmN4mVRW_dC_JaYMPjuAzLZb5g3m863iywQVdDsj5Bmn_3BsAxZ_Z1rNbyWZB96mh0Ucqn_1S4y6A9CsbARB4HHzb1DA8V3woA/s1600/2014-01-31+22_23_13-server+on+server+-+Virtual+Machine+Connection.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1kBgVzXlXCdYNvZzxIXfJLyp23zT1M2pYDHX-LCF1nPmN4mVRW_dC_JaYMPjuAzLZb5g3m863iywQVdDsj5Bmn_3BsAxZ_Z1rNbyWZB96mh0Ucqn_1S4y6A9CsbARB4HHzb1DA8V3woA/s1600/2014-01-31+22_23_13-server+on+server+-+Virtual+Machine+Connection.png"
border="0" height="281" width="400"></a></div>
<br>
</li>
<li>Select your desired WAP server and click "Next".</li>
<li>On "Add Roles and Features Wizard", select the "Remote Access" role
and click "Next".<br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcuWx2w3L-ICWh4PZ2iEM-sJq4jOMUZQPq4geZIelng2EEG9i0_3RRV0aQcTj-K3QZlE_dZjY9nGlzYR_k7CgDabeny3NSCOy2CrSeK8Z53AaSP9GB3gkdWqu54W9S1fjdPiDr60_Lixg/s1600/2014-01-31+22_29_41-_.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcuWx2w3L-ICWh4PZ2iEM-sJq4jOMUZQPq4geZIelng2EEG9i0_3RRV0aQcTj-K3QZlE_dZjY9nGlzYR_k7CgDabeny3NSCOy2CrSeK8Z53AaSP9GB3gkdWqu54W9S1fjdPiDr60_Lixg/s1600/2014-01-31+22_29_41-_.png"
border="0" height="282" width="400"></a></div>
</li>
<br>
<li>You do not need to select any features; click "Next" on the "Select
features" page. </li>
<li>Read the dialog presented on the "Remote Access" screen and click
"Next".</li>
<li>Leave "Include management tools" checked and click "Add Features".<br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFZ-vB_5ItPjvrSjmdJ_VnVcD0gWmZEwEppXPpv30YR08Nm9AwCzkWdipPGJ1OXPLRP_tzD7NjJJB1YSroqaWhama35I6-l5ys0MOAIf9D_qutojHR7P42G-cB4Nv5-eqLew1wKfHtFGI/s1600/2014-01-31+22_31_19-_.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFZ-vB_5ItPjvrSjmdJ_VnVcD0gWmZEwEppXPpv30YR08Nm9AwCzkWdipPGJ1OXPLRP_tzD7NjJJB1YSroqaWhama35I6-l5ys0MOAIf9D_qutojHR7P42G-cB4Nv5-eqLew1wKfHtFGI/s1600/2014-01-31+22_31_19-_.png"
border="0" height="282" width="400"></a></div>
<br>
</li>
<li>On the "Select role services" page select "Web Application Proxy" and
click "Next".<br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEinp_RafFd6O6gFa13ccVAeHOoN-vu0B8ooZ_j5e3IrR04LkBv9ZvSC-BmkmDo_BfrMEVVo19JC8hzexPuXsZmSmCPdXDh0mhSycOMo0Q1qpksxAbKLrFw_XabHDKUqbfbue01PyeLB16s/s1600/2014-01-31+22_31_37-_.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEinp_RafFd6O6gFa13ccVAeHOoN-vu0B8ooZ_j5e3IrR04LkBv9ZvSC-BmkmDo_BfrMEVVo19JC8hzexPuXsZmSmCPdXDh0mhSycOMo0Q1qpksxAbKLrFw_XabHDKUqbfbue01PyeLB16s/s1600/2014-01-31+22_31_37-_.png"
border="0" height="282" width="400"></a></div>
<br>
</li>
<li>When presented with the confirmation screen, click "Install".<br>
</li>
</ol>
<h4>WAP Configuration</h4>
<p><strong>Prerequisite Note</strong>: For this step you will need the
public and private key for your internal ADFS server(s) installed to the
"Personal" section of the "Local Computer" store on your WAP server. For
more information, refer to "Software Requirements" above. </p>
<ol>
<li> After installation, server manager will notify you that configuration
is required. Click the notification flag and select "Open the Web
Application Proxy Wizard".<br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjH4QyefA61IM7Bb3F0csRZgRSAQdcavUHnZmT9_ueWXIqFuuxa4jMWDwDNc9SLR-W7t8gmchch3c6Rzs1hTlQYIUxHOFSD2LBuXm2GPb5-9BUjgNisB896xeuWkCV-SPqup3IS-hTXg7A/s1600/2014-01-01+03_53_05-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjH4QyefA61IM7Bb3F0csRZgRSAQdcavUHnZmT9_ueWXIqFuuxa4jMWDwDNc9SLR-W7t8gmchch3c6Rzs1hTlQYIUxHOFSD2LBuXm2GPb5-9BUjgNisB896xeuWkCV-SPqup3IS-hTXg7A/s1600/2014-01-01+03_53_05-RD+Tabs+64.png"
border="0" height="206" width="320"></a></div>
<br>
</li>
<li>On the "Welcome" screen of the "Web Application Proxy Wizard" click
"Next".</li>
<li>On the "Federation Server" screen, enter the <strong>external</strong>
fully qualified domain name of your federation service. This needs to be
registered in external DNS (i.e. resolvable from the internet).
For more information, see my article linked under "Software
Requirements". Insert the username/password of a domain administrator
account to properly register this as a proxy server. This account will <strong>not</strong>
be used after this point, so a service account is not necessary. Click
"Next".<br>
<br>
<div class="separator" style="clear: both; text-align: center;">
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAqVU_teuCL4ydNGjiDMH5N9Z41tJh0KoDPIEkgyq7RKHOLdI6JggRu4pz48WUMlZonK9PYCJipB1ZqcQYfdLiru59LOZSznUYkuSu9NB52XDHRs85PJD9_-QcKWMlnYx_itbZ6fgNbKk/s1600/2014-01-31+23_16_11-D__Skydrive_Projects_ScreenCaps_Animal.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAqVU_teuCL4ydNGjiDMH5N9Z41tJh0KoDPIEkgyq7RKHOLdI6JggRu4pz48WUMlZonK9PYCJipB1ZqcQYfdLiru59LOZSznUYkuSu9NB52XDHRs85PJD9_-QcKWMlnYx_itbZ6fgNbKk/s1600/2014-01-31+23_16_11-D__Skydrive_Projects_ScreenCaps_Animal.png"
border="0" height="325" width="400"></a></div>
</div>
<br>
</li>
<li>Select the ADFS certificate you installed earlier from the dropdown
and click "Next".<br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTINxTH8mSX0yqsNR1HM7ODVShgpXtGGnX5P0JVNaX6wh6N9w7yOgZvkiwM8U0ZU_zaMVudJ7uKVA9AdH38rOL1ljtCi3wLo3oJnAFhiNtbBhlilvvYC0zig7o-q-aNa_nQcYcJd9u6u8/s1600/2014-01-01+03_55_01-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTINxTH8mSX0yqsNR1HM7ODVShgpXtGGnX5P0JVNaX6wh6N9w7yOgZvkiwM8U0ZU_zaMVudJ7uKVA9AdH38rOL1ljtCi3wLo3oJnAFhiNtbBhlilvvYC0zig7o-q-aNa_nQcYcJd9u6u8/s1600/2014-01-01+03_55_01-RD+Tabs+64.png"
border="0" height="325" width="400"></a></div>
<br>
</li>
<li>You'll be presented with the configuration details. If you intend on
setting up another WAP server for load balancing copy the powershell
command down for later use. Click "Configure" to continue. <br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_BRNEwS9DgUC_dRwtVmvPEdliTYLeggl8tABovgaV9rw2PyqSTyx48cplmO2DiFSbL-7O8Mk0XU8MrI6FFsb7OC25JEpFrHhTtqVUp1fGsjX22vKJK3LkfKEWRXcFVJn_JMlSqUOVA8g/s1600/2014-01-01+03_55_40-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_BRNEwS9DgUC_dRwtVmvPEdliTYLeggl8tABovgaV9rw2PyqSTyx48cplmO2DiFSbL-7O8Mk0XU8MrI6FFsb7OC25JEpFrHhTtqVUp1fGsjX22vKJK3LkfKEWRXcFVJn_JMlSqUOVA8g/s1600/2014-01-01+03_55_40-RD+Tabs+64.png"
border="0" height="325" width="400"></a></div>
<br>
</li>
<li>You should see the message "Web Application Proxy was configured
successfully".<br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjv-FIhBmxfZsBQ3hgGn0DhXay2v3KEFR4xbnhNaGpdMp2Uwe1TxU2Mschdd9zFaDodoILayATyMHDE3D198EBlDlNMpETHUDs_gnxgXUBYkdX3D_-F6s8a5fvO5qJ028Ps1n4NRmdSIro/s1600/2014-02-01+00_38_18-D__Skydrive_Projects_ScreenCaps_Animal.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjv-FIhBmxfZsBQ3hgGn0DhXay2v3KEFR4xbnhNaGpdMp2Uwe1TxU2Mschdd9zFaDodoILayATyMHDE3D198EBlDlNMpETHUDs_gnxgXUBYkdX3D_-F6s8a5fvO5qJ028Ps1n4NRmdSIro/s1600/2014-02-01+00_38_18-D__Skydrive_Projects_ScreenCaps_Animal.png"
border="0" height="323" width="400"></a></div>
<br>
</li>
</ol>
<h3>Setup Verification</h3>
<p>To verify basic functionality: </p>
<ol>
<li>On the WAP server, open up Tools->Remote Access Management Console</li>
<li>On the left-hand navigation pane, select "Operations Status"</li>
<li>The status of the WAP server will be relayed in the middle pane. Do
not be surprised to see the server listed twice, once for the FQDN and
once for netbios. This is normal. </li>
</ol>
<p>Now that setup is complete, let's move on to publishing!</p>
<h3>Example A: Proxying Exchange 2010 OWA (Pre-auth/Non-Claims/Delegated
Authentication)</h3>
<p>Now that we've completed the ADFS/WAP setup, let's walk through the setup
of a non-claims aware application using Kerberos/NTLM delegation. A
popular example would be Exchange Outlook Web Access; I'll be using
version 2010 SP3. </p>
<p><strong>Prerequisite Note</strong>: For this step you will need the
public and private key for the services you wish to host (Exchange OWA in
this case) installed to the "Personal" section of the "Local Computer"
store on your WAP server. Requests destined for your back-end service are
decrypted and re-encrypted at the WAP server. For more information, refer
to "Software Requirements" above.</p>
<h4>Trust Setup</h4>
<p>First, we must set up the new trust on the ADFS server. On your back-end
ADFS server (not the WAP server) do the following: </p>
<ol>
<li>Open the AD FS management tool and click the "Trust Relationships"
folder on the left hand navigation pane. </li>
<li>In the right hand action pane, click "Add Non-Claims-Aware Relaying
Party Trust".<br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6ULJBjevOHEV4jnwggEwAbJ8-fxXk08ZJ3REKBQe2LaIz6SYXj0H9H55leo8cL0JUBJVXQdg6rWqLNJcvAwG-CZCOT1_q_RqCJ9p3gdhUV6YNR5ioYwz3dtLz0RTWyjuL5QOjAiW7uuU/s1600/2014-02-13+23_32_53-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6ULJBjevOHEV4jnwggEwAbJ8-fxXk08ZJ3REKBQe2LaIz6SYXj0H9H55leo8cL0JUBJVXQdg6rWqLNJcvAwG-CZCOT1_q_RqCJ9p3gdhUV6YNR5ioYwz3dtLz0RTWyjuL5QOjAiW7uuU/s1600/2014-02-13+23_32_53-RD+Tabs+64.png"
border="0" height="247" width="400"></a></div>
<br>
</li>
<li>A wizard will pop up; click "Start" on the welcome screen. <br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhe8HkFi0-g8hTL13_50grnH0PVOGRHl_stibNmSW5p1s3oo4Qsal6S12hyphenhyphen_ZGXK9f1MuekB13AQkCARYj4QX05R5nHul2ED8FwAhGHrafs4IPDxhyphenhyphenN6t_diyo0wAhpgciziOcsbfXNvdk/s1600/2014-02-13+23_33_21-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhe8HkFi0-g8hTL13_50grnH0PVOGRHl_stibNmSW5p1s3oo4Qsal6S12hyphenhyphen_ZGXK9f1MuekB13AQkCARYj4QX05R5nHul2ED8FwAhGHrafs4IPDxhyphenhyphenN6t_diyo0wAhpgciziOcsbfXNvdk/s1600/2014-02-13+23_33_21-RD+Tabs+64.png"
border="0" height="322" width="400"></a></div>
<br>
</li>
<li>Give this rule a (human) meaningful name, i.e." <Servername>
Exchange OWA" along with a description if desired and click "Next".<br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-m_5V4qsRXXVfKUEWwCLRWVQAnIUievUFL9mjYs66aUHZbGMoLOyyO2yyhXa84qOi0OxJaAAzi3XKmG55eJq-LW4P_wikc4868ghv5SSBZBFOk188GJarF6htyCAL7mqbSAAg39QofPY/s1600/2014-02-13+23_33_49-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-m_5V4qsRXXVfKUEWwCLRWVQAnIUievUFL9mjYs66aUHZbGMoLOyyO2yyhXa84qOi0OxJaAAzi3XKmG55eJq-LW4P_wikc4868ghv5SSBZBFOk188GJarF6htyCAL7mqbSAAg39QofPY/s1600/2014-02-13+23_33_49-RD+Tabs+64.png"
border="0" height="322" width="400"></a></div>
<br>
</li>
<li>Now we'll add the non-claims aware relaying trust party identifier
(which in this case is simply a URL). Enter the external fully qualified
domain name of the server complete with url path ending in a trailing
forward slash, i.e. https://mail.company.com/owa/ and click "Next".
Note: WAP identifiers must end in a trailing slash even though the MSFT
example doesn't look that way.<br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhenVZN4D6DridpC838eDJUoZQ4PiuK1WfJs8yo7rVAMwoIO0pxr-HGuMJDyzuATF-ZKslUfFg_RydLMxXaKTf3z8iijSZUVD2HnBW6F2aUBNc8hEDPZMzJXTQuqo3GYdCVwWqpeYI_aq8/s1600/2014-02-13+23_34_09-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhenVZN4D6DridpC838eDJUoZQ4PiuK1WfJs8yo7rVAMwoIO0pxr-HGuMJDyzuATF-ZKslUfFg_RydLMxXaKTf3z8iijSZUVD2HnBW6F2aUBNc8hEDPZMzJXTQuqo3GYdCVwWqpeYI_aq8/s1600/2014-02-13+23_34_09-RD+Tabs+64.png"
border="0" height="322" width="400"></a></div>
<br>
<strong></strong></li>
<li>On the next screen, "Configure Multi-Factor Authentication Now?", you
can set up multi-factor authentication should you desire. I will not be
configuring multi-factor for this demonstration, but note you can always
set it up later if desired. Leave "I do not want to configure..."
selected and click "Next". <br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSN6_Ob3PDBNOvgVaAZpIHvuvWJWbBnDFEKaL55L-xApPlf31VO7dDmmt4fiejahbPAAk4I3jISx3-Piz_tS62qAKC0GOq6NzzGE1pee1axk39EuHWvaHzCJYcB0qVgj3G4UW0tenV_-g/s1600/2014-02-13+23_34_16-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSN6_Ob3PDBNOvgVaAZpIHvuvWJWbBnDFEKaL55L-xApPlf31VO7dDmmt4fiejahbPAAk4I3jISx3-Piz_tS62qAKC0GOq6NzzGE1pee1axk39EuHWvaHzCJYcB0qVgj3G4UW0tenV_-g/s1600/2014-02-13+23_34_16-RD+Tabs+64.png"
border="0" height="322" width="400"></a></div>
<br>
</li>
<li>Review your configuration on the "Ready to Add Trust" screen and click
"Next".</li>
<li>The "Finish" screen will have a checkbox starting with "Open the Edit
Authorization Rules dialog..." that is checked by default. Leave it
checked because we will want to specify who is allowed access through to
the back-end via this rule right away. Click "Finish". <br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYpEI32Np6Krn-XVmNulXT2HXqwWqO2nzz6ieD-T7YxbxM7-ZgFuNbWyzbmr7gHeTN5r2YDCm5VhP5a86njY62SsyqCW40b7CnKT_voBW1nDetIlOF0wlwhZ2Zmf8yGBEVeLKXhOJCFj4/s1600/2014-02-13+23_34_46-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYpEI32Np6Krn-XVmNulXT2HXqwWqO2nzz6ieD-T7YxbxM7-ZgFuNbWyzbmr7gHeTN5r2YDCm5VhP5a86njY62SsyqCW40b7CnKT_voBW1nDetIlOF0wlwhZ2Zmf8yGBEVeLKXhOJCFj4/s1600/2014-02-13+23_34_46-RD+Tabs+64.png"
border="0" height="322" width="400"></a></div>
<br>
</li>
<li>A dialog box titled "Edit Claim Rules for <Rule Name>" will come
up allowing us to define who should be allowed access via this rule.
Click "Add Rule'.<br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgVOhOZQLy6e7GH-GEHf2L4WqgPO8I55lRzyLeVHwGgDWZYllc6HGvEYyBGwkapejjQ4ogj7ikjpcHSjaoqE9CIakPu9YPn_34A45HJLQU7JtF31KsRUor2itxsRJy94IwDgZ_l52Q3f6Y/s1600/2014-02-13+23_34_53-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgVOhOZQLy6e7GH-GEHf2L4WqgPO8I55lRzyLeVHwGgDWZYllc6HGvEYyBGwkapejjQ4ogj7ikjpcHSjaoqE9CIakPu9YPn_34A45HJLQU7JtF31KsRUor2itxsRJy94IwDgZ_l52Q3f6Y/s1600/2014-02-13+23_34_53-RD+Tabs+64.png"
border="0" height="400" width="365"></a></div>
<br>
</li>
<li>You will be asked to select a rule template. What you select here will
depend on what is reasonable for your environment. You should create (a)
rule(s) that correspond with the least access required possible as
anyone getting past this point will be able to attempt to authenticate
directly against the target internal resource. You may, for example,
want to use a specific Active Directory group with only the users that
need access to this resources. For the purposes of testing and this
article, however, I will be using a simple "Permit All Users" rule. This
will allow anyone in AD through and is suitable for testing or in
addition to other rules. Select the rule template and click "Next".<br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHasmFjqhVhf-Goljql2n_4btAgP3VrTH2Hbb8W-4eCyRJ1R5tfPo5IL30rzcJuISALnmgDutgMGPcYa_qisTh8CvBts3I-eX7t0geHLfj84YZKM6RMcB1xpSjXALhCdenyoa_2x0a3Hg/s1600/2014-02-13+23_35_12-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHasmFjqhVhf-Goljql2n_4btAgP3VrTH2Hbb8W-4eCyRJ1R5tfPo5IL30rzcJuISALnmgDutgMGPcYa_qisTh8CvBts3I-eX7t0geHLfj84YZKM6RMcB1xpSjXALhCdenyoa_2x0a3Hg/s1600/2014-02-13+23_35_12-RD+Tabs+64.png"
border="0" height="322" width="400"></a></div>
<br>
</li>
<li>Click "Finish" to close the "Add Issuance Authorization Claim Rule
Wizard"</li>
<li>So long as you do not want any additional rules, click "OK" to close
the Edit Claim Rules dialog box. <br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgK0r4dLFZgxweuCKkOix9Ie-i0EFbiLCfW9beakQfRhjbeqgof0V1kPsAXYw3vGouX6ZhtRdF_DDpnwLu11O7BfLY9nuNwPKscvoXdObdG9sNzQtgJiN6mN1wBSHOvhlpZhi_sQjyoWPw/s1600/2014-02-13+23_35_44-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgK0r4dLFZgxweuCKkOix9Ie-i0EFbiLCfW9beakQfRhjbeqgof0V1kPsAXYw3vGouX6ZhtRdF_DDpnwLu11O7BfLY9nuNwPKscvoXdObdG9sNzQtgJiN6mN1wBSHOvhlpZhi_sQjyoWPw/s1600/2014-02-13+23_35_44-RD+Tabs+64.png"
border="0" height="400" width="365"></a></div>
<br>
</li>
</ol>
<h4>Back-end Service Configuration</h4>
<p>Now we need to configure our back-end service to accept the
authentication coming from the WAP server. In our case we will need to
change the authentication mechanism allowed by Exchange from forms
based to integrated authentication.Your steps here will differ depending
on what service you are offering up. </p>
<ol>
<li>Open the Exchange management console and Click on "Server
Configuration"->"Client Access"</li>
<li>For each server in your Exchange farm, click the "Outlook Web App"
tab, then right click "owa (Default Web Site)" and click "properties". <br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEilgIheko5ILFTfMexC__-f_cQmwRshoWZyQL1N-VAgv_otihWa-Lsvfrl8wqCRTCTtAFeH9vwV0KNwCMD11kEs_D1GXTRYfdZhFHykYjKdIIbsBsDUwLoYDmkIo9qLJ8ddCRz761dG3-M/s1600/2014-02-14+01_45_18-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEilgIheko5ILFTfMexC__-f_cQmwRshoWZyQL1N-VAgv_otihWa-Lsvfrl8wqCRTCTtAFeH9vwV0KNwCMD11kEs_D1GXTRYfdZhFHykYjKdIIbsBsDUwLoYDmkIo9qLJ8ddCRz761dG3-M/s1600/2014-02-14+01_45_18-RD+Tabs+64.png"
border="0" height="357" width="400"></a></div>
<br>
</li>
<li>Select the "Authentication" tab and click "Use one or more standard
authentication methods:" then select only "Integrated Windows
authentication".<br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMo2VZC9nF6IrfIwVPX44o9wAedwdMLsXMYDEDjIBrkz2-dDXy15-twQ4bGi1dQD6QYZ-f7KtlxyA6CPDQemHbYbnY5qW15hZ0tKtXa2O24IvWU7J4RTxyTyMQhZniURuKwmHZTXp9Wwo/s1600/2014-02-14+01_45_44-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMo2VZC9nF6IrfIwVPX44o9wAedwdMLsXMYDEDjIBrkz2-dDXy15-twQ4bGi1dQD6QYZ-f7KtlxyA6CPDQemHbYbnY5qW15hZ0tKtXa2O24IvWU7J4RTxyTyMQhZniURuKwmHZTXp9Wwo/s1600/2014-02-14+01_45_44-RD+Tabs+64.png"
border="0" height="400" width="343"></a></div>
<br>
</li>
<li>Click "OK" on the warning. </li>
<li>Repeat steps 2 and 3 for the "ecp (Default Web Site)" under "Exchange
Control Panel" on each server<strong>. </strong></li>
<li>Using an elevated command prompt or PowerShell, execute "<em>iisreset
-noforce</em>" to restart IIS. (This should be done in a maintenance
window)</li>
</ol>
<h4>Configure Delegation </h4>
<br>
Now we'll configure the WAP server AD computer object so that it can pass
authentication to your back-end server(s). Note the SPNs referenced to not
need to be manually registered at a domain level. <br>
<ol>
<li>With domain administrator privileges, open the Active Directory
Administrative Center. (Active Directory Users and Computers if you
prefer)</li>
<li>Navigate to and open the properties of the WAP server computer object.
<br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_3h8MYsHJIg9t90poRfVBfBU_2AAKtkS5nQrmQn0vWW9UPP_JAPz0GxrhRtlVAyIp_ieUyI8sMOadZOFw-CkwfZ2_jmPnlQuARlTlXvO-5OjNO1UQyO57CBTrmYxX3pmZb7cp4vvdp3A/s1600/2014-02-19+23_28_29-Active+Directory+Administrative+Center.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_3h8MYsHJIg9t90poRfVBfBU_2AAKtkS5nQrmQn0vWW9UPP_JAPz0GxrhRtlVAyIp_ieUyI8sMOadZOFw-CkwfZ2_jmPnlQuARlTlXvO-5OjNO1UQyO57CBTrmYxX3pmZb7cp4vvdp3A/s1600/2014-02-19+23_28_29-Active+Directory+Administrative+Center.png"
border="0" height="273" width="400"></a></div>
<br>
</li>
<li>Click or scroll down to the "Delegation" section of the object. <br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgnFGVQ2LwZnowVjxKp3kKghHi3CtfabYehmK6eeTKDnmW1zgtdamlLovy1J3O4eNd5lLMo26rmTc4Orw2XE_1rOy0O8HGJ-BIFCZhe-9XArp9xvrAjea3QPcO868IQoquDEkmU693wbqE/s1600/2014-02-19+23_29_20.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgnFGVQ2LwZnowVjxKp3kKghHi3CtfabYehmK6eeTKDnmW1zgtdamlLovy1J3O4eNd5lLMo26rmTc4Orw2XE_1rOy0O8HGJ-BIFCZhe-9XArp9xvrAjea3QPcO868IQoquDEkmU693wbqE/s1600/2014-02-19+23_29_20.png"
border="0"></a></div>
<br>
</li>
<li>Select "Trust this computer for delegation to specified servers only"
and "Use any authentication protocol" (since we'll be using NTLM here;
select Kerberos only for applications that support it) then click
"Add..."</li>
<li>When presented with the "Add Services" dialog, click "Add Users or
Computers...".<br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi8cS8-u05HtnGC9q5EcMcpP2kMwh0Mht_MAO-idUxLqlMZ0v62WpZZFEhe5AeTvvXxQp_N2mALmzhEkD5L0byfxfn907kz8GpgKdBzv8Yb8j-DM9b_qM44oKcCdeQOLGdmdFJdbaRhCnA/s1600/2014-02-19+23_30_37-Add+Services.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi8cS8-u05HtnGC9q5EcMcpP2kMwh0Mht_MAO-idUxLqlMZ0v62WpZZFEhe5AeTvvXxQp_N2mALmzhEkD5L0byfxfn907kz8GpgKdBzv8Yb8j-DM9b_qM44oKcCdeQOLGdmdFJdbaRhCnA/s1600/2014-02-19+23_30_37-Add+Services.png"
border="0" height="230" width="400"></a></div>
<br>
</li>
<li>Type the name of the back-end Exchange server(s) and click "Check
Names" and then "OK"</li>
<li>Scroll to the http/SERVERNAME.domain.ext (since we're serving up the
HTTP protocol; change if your app differs) and select it, then click
"OK". <strong>Note: </strong>If using Active Directory Administrative
Center you need to add the <a href="http://en.wikipedia.org/wiki/Fully_qualified_domain_name">FQDN</a>
name <em>and</em> the NETBIOS name; if using Active Directory Users
Computers you need only add the FQDN and both will be added. <br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHK3sPsBTDqAIV9tO3XimwwM6Z2B5CcQYVh-aPVM0b2m_u-Efz77BZ1Re5jwpvWPNTqIWhWN01P39CgZ93cidneWecE0jpG-ZIw4JozgxT4qf1ZdJBfhCFYiNRwNIvqWHmlWTwZmC74S4/s1600/2014-02-19+23_33_17-Add+Services.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHK3sPsBTDqAIV9tO3XimwwM6Z2B5CcQYVh-aPVM0b2m_u-Efz77BZ1Re5jwpvWPNTqIWhWN01P39CgZ93cidneWecE0jpG-ZIw4JozgxT4qf1ZdJBfhCFYiNRwNIvqWHmlWTwZmC74S4/s1600/2014-02-19+23_33_17-Add+Services.png"
border="0" height="217" width="400"></a></div>
<br>
</li>
</ol>
<h4>Configure Application Publishing on WAP Server<br>
</h4>
<br>
Finally we'll configure WAP publishing for this application. <br>
<ol>
<li>On the WAP server, open the Remote Access Management Console (can be
found in admin tools or tools from Server Manager)</li>
<li>In the left hand navigation plane, select "Configuration"->"Web
Application Proxy"</li>
<li>On the right hand action pane, click "Publish"<br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhyL3OrNC5wx9QwVvAHmZoFLrOJIMi65MCXwxQYnOW1dyxXwjQmt0nM4sBnyb6XCnsOn6wfWidtKNc8hAB_6K_SbZi0ivnb8MbF4Me-ZZiP5UhEQqrb7QnILo4McXLNs7EySj3pet5Ce1M/s1600/2014-02-19+23_21_01-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhyL3OrNC5wx9QwVvAHmZoFLrOJIMi65MCXwxQYnOW1dyxXwjQmt0nM4sBnyb6XCnsOn6wfWidtKNc8hAB_6K_SbZi0ivnb8MbF4Me-ZZiP5UhEQqrb7QnILo4McXLNs7EySj3pet5Ce1M/s1600/2014-02-19+23_21_01-RD+Tabs+64.png"
border="0" height="130" width="400"></a></div>
<br>
</li>
<li>A wizard will come up. Click "Next" on the welcome screen. </li>
<li>When prompted for preauthentication type, select "Active Directory
Federation Services (AD FS)". This ensures requests are authenticated by
ADFS prior to being passed onto the back-end server. Click "Next".<br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjG3hSIGcjWdqUDkwbY4XHdwyAukg5kHueQlyIkwAq565lFwvs_DcuKJ9w7aOEUTJ4P_CR6WSWW9WWoosJS5b2Wp0621B6AQ2Iz9byK2PVMhpdjfi2nSvO8zMvfPur9s1fV1EM_2ZqwvZs/s1600/2014-02-19+23_22_35-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjG3hSIGcjWdqUDkwbY4XHdwyAukg5kHueQlyIkwAq565lFwvs_DcuKJ9w7aOEUTJ4P_CR6WSWW9WWoosJS5b2Wp0621B6AQ2Iz9byK2PVMhpdjfi2nSvO8zMvfPur9s1fV1EM_2ZqwvZs/s1600/2014-02-19+23_22_35-RD+Tabs+64.png"
border="0" height="325" width="400"></a></div>
<br>
</li>
<li>For "Relying Party", select the trust rule we created earlier under
the "Trust Setup" section above and click "Next". <br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhr2I8ht6mM4929L8ZWk3ro9jFafRvFksbl9uEUicP3MlZ_MWMa-vDp7z-w83sgitjLyAFnHtP-0ew5Li34n-76vUfj8UqI9L06gmOURrio1zAcJ8WUkDBJMeZIuRyVPSuuekn2Kwd-AoM/s1600/2014-02-19+23_22_42-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhr2I8ht6mM4929L8ZWk3ro9jFafRvFksbl9uEUicP3MlZ_MWMa-vDp7z-w83sgitjLyAFnHtP-0ew5Li34n-76vUfj8UqI9L06gmOURrio1zAcJ8WUkDBJMeZIuRyVPSuuekn2Kwd-AoM/s1600/2014-02-19+23_22_42-RD+Tabs+64.png"
border="0" height="325" width="400"></a></div>
<br>
</li>
<li>Now the meat of the settings; on the "Publishing Settings" step enter
a meaningful name for this connection (i.e. Exchange 2010 OWA), the
external URL it will be accessed by (i.e.
https://mail.company.com/owa/), select the external certificate for that
service (see "Software Requirements" above), the internal URL
(preferably should match the external but doesn't have to in all cases),
and the server SPN that we specified on the step above, then click
"Next".<br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgVfKs7dRkullsDbLGDWJ6_gSz3nbXCLBK-oX1uxZBQ2LDYAyMwlcGM8Cdvb6CmGIWt-eXdkkh3zkyFjPU6Azx3jRl8a-0eAS_4vqVoHvRM9l5u1OdXA9R6sTKjbkytRBokBJJ8Aqlplrg/s1600/2014-02-24+01_17_33-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgVfKs7dRkullsDbLGDWJ6_gSz3nbXCLBK-oX1uxZBQ2LDYAyMwlcGM8Cdvb6CmGIWt-eXdkkh3zkyFjPU6Azx3jRl8a-0eAS_4vqVoHvRM9l5u1OdXA9R6sTKjbkytRBokBJJ8Aqlplrg/s1600/2014-02-24+01_17_33-RD+Tabs+64.png"
border="0" height="325" width="400"></a></div>
<br>
</li>
<li>You will be shown the confirmation screen with the appropriate
PowerShell command line for the options you have configured. Should you
want to repeat a similar publishing step, copy and retain this command
line for use later. Click "Publish". <br>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjIPulLfalqBhmSgdQBUiZQe0jt1dPW1Kma8-QgtFEP0p7_aoKdx58ooOApCRmsw7klZEnlUbxCNWXXIYb_j8Sk5muVnucqjeYrB6OPEX1hSWcu6qLF70RsfMdLynVaI7-qzj9XygX24pM/s1600/2014-02-24+01_18_10-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjIPulLfalqBhmSgdQBUiZQe0jt1dPW1Kma8-QgtFEP0p7_aoKdx58ooOApCRmsw7klZEnlUbxCNWXXIYb_j8Sk5muVnucqjeYrB6OPEX1hSWcu6qLF70RsfMdLynVaI7-qzj9XygX24pM/s1600/2014-02-24+01_18_10-RD+Tabs+64.png"
border="0" height="325" width="400"></a></div>
<br>
</li>
<li>The results screen will display the publishing status. Assuming all is
well, click "Close" to close the wizard. </li>
</ol>
<ol>
</ol>
<ol>
</ol>
<h3>Example B: RDP Proxy (No Pre-auth/Passthrough)</h3>
<p>Passthrough applications are substantially easier (and less secure)
because they do not require any set up in ADFS and do not subject the user
connection attempt to any authentication before passing it on. This isn't
to say the back-end service won't require authentication, however, but it
is still less secure since you are opening your back-end service up to
processing logon requests directly from the internet. </p>
<h4>Publish RDP Proxy on WAP Server<br>
</h4>
<br>
In this example I will publish RDP proxy direct to the internet proxied
through the WAP server. This allows me to serve up this application on the
same IP address and port as other services assuming the hostname requested
is unique. Again, this section assumes the public and private keys
associated with the URL you intend to use installed in the WAP server's
"personal" store. In my example I use a hostname of "rdp.company.com"<br>
<ol>
<li>On the WAP server, open the Remote Access Management Console (can be
found in admin tools or tools from Server Manager)</li>
<li>In the left hand navigation plane, select "Configuration"->"Web
Application Proxy"</li>
<li>On the right hand action pane, click "Publish"</li>
<li>A wizard will come up. Click "Next" on the welcome screen. </li>
<li>When prompted for preauthentication type, select "Pass-through" and
click "Next".<br>
<br>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj70uQnj5kR048gyKNytQDA4CCr_ekPeYdGHTNjXt2CN-wKULwEavYZUcvtfqllp6pj-OfdRlUG60KKc0gypF9gczMngqftODcveE36A7B-rda78Wqvt_z0V_-Z4pOfu3S8uDXkuEVHNR4/s1600/2014-04-20+21_36_36-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj70uQnj5kR048gyKNytQDA4CCr_ekPeYdGHTNjXt2CN-wKULwEavYZUcvtfqllp6pj-OfdRlUG60KKc0gypF9gczMngqftODcveE36A7B-rda78Wqvt_z0V_-Z4pOfu3S8uDXkuEVHNR4/s400/2014-04-20+21_36_36-RD+Tabs+64.png"
border="0"></a></div>
<br>
</li>
<li>On the "Publishing Settings" step enter a meaningful name for this
connection (i.e. RDProxy), the external URL it will be accessed by (i.e.
https://rdp.company.com/), select the external certificate for that
service (see "Software Requirements" above), and the internal URL
(preferably should match the external but doesn't have to in all cases).
Click "Next". <br>
<br>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGKRM-0IazexwKXBM6iSEhjyYUntLiZeXhDG_i2VRVa_hTEG7RWYSHEHPoyGyMIp5kjAuJXhpNRCjuktRCsfF-TULHNEV1YK87EeFw-aG5ZISbUhDtSg4WJD1NVoCjvYR_ITmC34N06e4/s1600/2014-04-20+21_37_10-RD+Tabs+64.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGKRM-0IazexwKXBM6iSEhjyYUntLiZeXhDG_i2VRVa_hTEG7RWYSHEHPoyGyMIp5kjAuJXhpNRCjuktRCsfF-TULHNEV1YK87EeFw-aG5ZISbUhDtSg4WJD1NVoCjvYR_ITmC34N06e4/s400/2014-04-20+21_37_10-RD+Tabs+64.png"
border="0"></a></div>
<br>
</li>
<li>You will be given a summary of the publishing rule about to be created
and a Powershell command of it's equivalent. If you are satisfied with
the details click "Publish". </li>
</ol>
<h3>Troubleshooting</h3>
<ul>
</ul>
Something not working? Check out the following locations:<br>
<h4>Event Logs</h4>
<br>
Applications and Services Logs->AD FS/Admin<br>
Applications and Services
Logs->Microsoft->Windows->WebApplicationProxy/Admin<br>
<br>
<h4>Other</h4>
<br>
Should you need to enable debug logging, there is an excellent article <a href="http://jorgequestforknowledge.wordpress.com/2014/02/05/enabling-debug-tracing-in-adfs-v2-1-and-v3-0/">here</a>
demonstrating how to do so. One word of caution, however; should you edit
the C:\Windows\ADFS\Config\microsoft.identityServer.proxyservice.exe.config
referenced therein I recommend backing it up first. If not formatted
correctly WAP will start up successfully with the values listed in the file,
but when it comes time to rotate the ADFS Proxy Trust certificate (an
automatic action that happens once every 3 weeks) the configuration of the
new cert will fail. In that case you would see an Event ID 422 logged to AD
FS/Admin stating "Unable to retrieve proxy configuration data from the
Federation Service.". <br>
<br>
<h3> (Excellent!) References</h3>
<br>
Want more? Here are some wonderful resources!<br>
<br>
<a href="%20http://technet.microsoft.com/en-us/library/dn280944.aspx">Technet:
Web Application Proxy Overview</a><br>
<a href="http://technet.microsoft.com/en-us/library/dn383662.aspx">Technet:
Install and Configure the Web Application Proxy Server</a><br>
<a href="%20http://technet.microsoft.com/en-us/library/dn383650.aspx">Technet:
Installing and Configuring Web Application Proxy for Publishing Internal
Applications</a><br>
<a href="http://technet.microsoft.com/en-us/library/dn280942.aspx"> Technet
Overview Guide: Connect to Applications and Services from Anywhere with
Web Application Proxy</a><br>
<a href="http://social.technet.microsoft.com/Forums/windowsserver/en-US/0bdf918d-7477-420e-84cc-b6f9ebeada0c/web-app-proxy-and-ipv6">Technet
Social: On WAP and IPv6</a><br>
<a href="http://social.technet.microsoft.com/Forums/windowsserver/en-US/d3feeeac-bbc9-4acd-a382-5d6bd92112a5/adfs-wap-and-logging">
Technet Social: ADFS, WAP, and Logging</a><br>
<a href="http://blogs.technet.com/b/applicationproxyblog/archive/2014/06/19/how-to-support-non-sni-capable-clients-with-web-application-proxy-and-ad-fs-2012-r2.aspx">Technet
Blog: How to support non-SNI capable Clients with Web Application Proxy
and AD FS 2012 R2</a> (Needed to support Android clients for
Exchange ActiveSync or other clients that don't support SNI hosted through
WAP)<br>
<a href="http://blogs.technet.com/b/askpfeplat/archive/2013/07/22/faq-on-adfs-part-1.aspx">Technet
Ask PFE: FAQ on ADFS Part 1, Excellent coverage of SQL vs. Internal DB and
certificates for AD FS</a> (Not WAP per se) <br>
<a href="http://blog.kloud.com.au/2013/07/03/windows-2012-r2-preview-web-application-proxy-exchange-2013-publishing-tests/">Marc
Terblanche: Windows 2012 R2 Preview Web Application Proxy - Exchange 2013
Publishing Tests</a><br>
<a href="http://blogs.technet.com/b/askds/archive/2012/01/05/understanding-the-ad-fs-2-0-proxy.aspx">Ask
the DS Team: Understanding the ADFS 2.0 Proxy</a> (Not about WAP but
excellent coverage of AD FS proxy functionality) <br>
<a href="%20http://dotnet.dzone.com/articles/troubleshooting-adfs-20">Rob
Sanders: Troubleshooting ADFS 2.0</a> (Not about 3.0/WAP but too good not
to be mentioned) <br>
<a href="http://technet.microsoft.com/pt-pt/library/cc756046%28v=ws.10%29.aspx">
Technet: Configure Event Logging on a Federation Server Proxy</a> (Still
partially relevant) <br>
<a href="%20http://technet.microsoft.com/en-us/library/adfs2-troubleshooting-things-to-check%28v=ws.10%29.aspx">Technet:
Things to check before troubleshooting ADFS 2.0</a> (Still partially
relevant) <br>
<a href="http://technet.microsoft.com/en-us/library/adfs2-troubleshooting-configuring-computers%28v=ws.10%29.aspx">Technet:
Configuring Computers for Troubleshooting AD FS 2.0</a> (Still partially
relevant) <br>
<br>
Thanks for reading, if you have questions or comments leave them below!<br>
</body>
</html>
Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com9tag:blogger.com,1999:blog-5458589696790815336.post-79431872394304116762014-02-17T23:20:00.000-06:002017-03-08T23:51:05.534-06:00Generate a Certificate for Exchange 2010 using Microsoft PKIIf creating a certificate request for OWA (and other associated services) with the intention of processing this request on a Microsoft enterprise PKI structure, you'll meet a series of challenges. Here is how to overcome them in the order you will encounter them.<br />
<br />
After <a href="http://technet.microsoft.com/en-us/library/dd351057%28v=exchg.141%29.aspx">creating the request</a> and attempting to submit it, you will first see this error message:<br />
<br />
<b>ASN1 bad tag value met. 0x8009310b (ASN: 267).</b><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAo5gC-jOKh9mhgcl3c4CBINAqBhnI2UhlPyPKlNpwv0NqEF4q12vZY_wmRLD8nwYQaXhbCvVEoANWt7IiPtwvtuVGPVKzfjz-3B3M4EZfOd0oEF0uwW1RCp1xHcpeFpyAHpT49RYXBLc/s1600/2014-02-17+22_48_24-RD+Tabs+64.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="137" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAo5gC-jOKh9mhgcl3c4CBINAqBhnI2UhlPyPKlNpwv0NqEF4q12vZY_wmRLD8nwYQaXhbCvVEoANWt7IiPtwvtuVGPVKzfjz-3B3M4EZfOd0oEF0uwW1RCp1xHcpeFpyAHpT49RYXBLc/s1600/2014-02-17+22_48_24-RD+Tabs+64.png" width="320" /></a></div>
<br />
<br />
This error is because the request is by default encoded in Unicode while 2008r2 and lower PKI can only process ANSI. To convert, either open the request with notepad and select "<i>File->Save As</i>" and change the encoding to ANSI before saving.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiyBATv2EZXqL-QIuWbNUlUAESbw2gVvLo_XAR9NHwgIY6CszhLEuc1Mpy2cwXelSft6Vbb5UiRsSY9V6rnWvzhmrqLsOV3V-DHKUDOhAhy2qtumCsj-CNcB0WUIkeD4heh87gfjMZwbIA/s1600/2014-02-17+22_49_49-RD+Tabs+64.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="313" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiyBATv2EZXqL-QIuWbNUlUAESbw2gVvLo_XAR9NHwgIY6CszhLEuc1Mpy2cwXelSft6Vbb5UiRsSY9V6rnWvzhmrqLsOV3V-DHKUDOhAhy2qtumCsj-CNcB0WUIkeD4heh87gfjMZwbIA/s1600/2014-02-17+22_49_49-RD+Tabs+64.png" width="400" /></a></div>
<br />
<br />
If you submit at this point, you will see:<br />
<br />
<b>The request contains no certificate template information. 0x80094801 (-2146875391)<br />Denied by Policy Module 0x80094801, the request does not contain a certificate template extension or the Certificate Template request attribute.</b><br />
<br />
This is because Microsoft enterprise PKI does not process unqualified (by means of a template) requests. We need to force a template (WebServer will work) by using the following command:<br />
<br />
<i>certreq -submit -attrib "CertificateTemplate:WebServer" NameOfMyRequest.req</i><br />
<br />
Select your CA. If your template is configured correctly, the cert request will be successful. If not, you will receive the warning:<br />
<br />
<b>Certificate not issued (Denied) Denied by Policy Module 0x80094800, The request was for a certificate template that is not supported by the Active Directory Certificate Services policy: Web Server.<br /><br />The requested certificate template is not supported by this CA. 0x80094800 (-2146875392)</b><br />
<br />
To remediate this issue you will need to create/enable your template as desired. I created a custom 2k8 compatible Web Server template that allows for exporting the private key (for Web Application Proxy) and it worked well. For more information, see <b>Section 4</b> of <a href="http://blog.ittoby.com/2012/04/creating-two-tier-pki-windows-2008r2.html">my article</a>. <br />
<br />
<b>Update 2017: </b>I have confirmed that this solution works for Exchange 2016 as well. <br />
<br />
After saving the request, use either powershell or the management console to finish processing the request, and then enjoy your e-net i-mail over the hinterlandnet.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhh1hKiCl5ZC24hq4V1vxHy3Gj2snKY09m9K0_qGwPUOgVRJ0zZEIXBhcL7cvgN68RFACv0P8CZkdacTlyvTDyf-N5yvoeDdI9lm1HQ5edeQEEtBmqRofqhJ2ptcqHwzfFVOAQeLGyBqF0/s1600/FonzieCert.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="298" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhh1hKiCl5ZC24hq4V1vxHy3Gj2snKY09m9K0_qGwPUOgVRJ0zZEIXBhcL7cvgN68RFACv0P8CZkdacTlyvTDyf-N5yvoeDdI9lm1HQ5edeQEEtBmqRofqhJ2ptcqHwzfFVOAQeLGyBqF0/s1600/FonzieCert.jpg" width="400" /></a></div>
<br />Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com0tag:blogger.com,1999:blog-5458589696790815336.post-38514888965561632702014-02-09T21:04:00.001-06:002014-02-09T21:04:16.874-06:00Windows Server Failover Clustering Quorum Behavior Guide“<i>A <strike>Republic</strike> Quorum, if you can keep it.</i>” - Ben Franklin.<br />
<br />
Your <a href="http://technet.microsoft.com/en-us/library/cc770737.aspx"><b>WSFC</b></a> Quorum is like a Republic, or more accurately, a Democracy. There are many articles out there regarding Quorum voting logic but most are somewhat lengthy. I decided to set out to see how few words I could effectively explain Quorum rules in, so here we go. Don't count this part. Or this. Wait... er... start counting.... <b>NOW</b>.<br />
<br />
<h3>
Windows 2008 and higher:</h3>
<br />
<ul>
</ul>
<br />
<ul>
<li>A Quorum is the act of <b>n</b> <i>nodes</i> agreeing on a majority. </li>
<li>A <i>node</i> is a cluster member, <a href="http://technet.microsoft.com/en-us/library/jj612870.aspx#BKMK_witness">shared disk</a>, or <a href="http://technet.microsoft.com/en-us/library/jj612870.aspx#BKMK_witness">fileshare</a> witness. </li>
<li>A cluster can have a shared disk or fileshare, but not both. </li>
<li>A majority is defined by <b>greater</b> than 50% consensus. (a tie is <b>not</b> majority)</li>
<li><b>Fractions</b> (only) are rounded up to the nearest integer. (2.5=3)</li>
<li>There is a legacy quorum method called "disk only" wherein one (defined quorum) disk is the only vote. This is considered obsolete because it creates a single point of failure. </li>
</ul>
<br />
<ul>
</ul>
<div>
<h3>
Windows 2008/r2 with Hotfix 2494036 or Higher:</h3>
</div>
<div>
</div>
<div>
<ul>
</ul>
</div>
<div>
"Nodeweight" was added to revoke a node of its voting privileges (NodeWeight=0). You can use this for nodes in a different site or to ensure that shared disk/fileshare casts the deciding vote. This is generally used in cross-site clusters. </div>
<div>
<br />
<ul>
</ul>
<div>
<h3>
Windows 2012 or Higher:</h3>
<br />
<b>Dynamic Clustering </b></div>
</div>
<div>
</div>
<div>
<ul>
</ul>
</div>
<div>
"Dynamic Clustering" changes the nodeweight of downed cluster member and effectively reduces the number of participating nodes by one. This works under the following circumstances:<br />
<ul>
<li>Prior to the outage, the cluster has achieved quorum (normal under most circumstances)</li>
<li>Nodes must go down one at a time so the remaining nodes can agree to removed the downed member. If multiple nodes go down simultaneously the dynamic removal will not take place. </li>
</ul>
<br />
<ul>
</ul>
<div>
<h3>
Examples/Illustration: </h3>
</div>
</div>
<div>
<br /></div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhUtRJFUFmX9l6Or3Wb6YkWyUhNM2iRDc0t9mLXkYG6uS-mz6FD85xH13ehifqMPBbmoSZDqE2855t7m6W2VOWIYG9LUz0aucWzPT5_oC9ZPnKFqWoXxKSJWPKHNtiTDbB5rKHCz03zEbc/s1600/ClusterFailure.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhUtRJFUFmX9l6Or3Wb6YkWyUhNM2iRDc0t9mLXkYG6uS-mz6FD85xH13ehifqMPBbmoSZDqE2855t7m6W2VOWIYG9LUz0aucWzPT5_oC9ZPnKFqWoXxKSJWPKHNtiTDbB5rKHCz03zEbc/s1600/ClusterFailure.png" height="314" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<h3>
Closing</h3>
<br />
Windows Server Failover Clustering is an excellent option for <a href="http://technet.microsoft.com/en-us/library/hh270278.aspx">SQL Server</a>, <a href="http://technet.microsoft.com/en-us/library/cc732181%28v=ws.10%29.aspx">Hyper-V</a>, and other services. Hopefully this understanding of cluster failover behavior enables you to design solutions that better meet the needs of your clients.<br />
<br />
Note: It is important to consider how a Quorum is formed when considering patching strategy. <br />
<br />
<h3>
References</h3>
<br />
<a href="http://technet.microsoft.com/en-us/library/cc731739.aspx">TechNet: Understanding Quorum Configurations in a Failover Cluster</a><br />
<a href="http://blogs.technet.com/b/aevalshah/archive/2012/08/21/windows-server-2012-failover-clustering-dynamic-quorum.aspx">Aeval Shah's Blog: Windows Server 2012 Failover Clustering Dynamic Quorum</a><br />
<a href="http://technet.microsoft.com/en-us/library/hh270281.aspx">Configure Cluster Quorum NodeWeight Settings</a><br />
<a href="http://support.microsoft.com/kb/2494036">Microsoft Support: Cluster NodeWeight hotfix for 2008/r2</a><br />
<br /></div>
Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com0tag:blogger.com,1999:blog-5458589696790815336.post-57505427654756775832014-01-28T19:20:00.000-06:002014-02-04T00:15:03.182-06:00Cloud Authentication Primer: Basic Active Directory Federation Services SetupCloud computing. Cool stuff eh? Just pay your (not quite) local provider and all your problems are a thing of the past.<br />
<br />
How are we going to authenticate/authorize our users? I dunno, but sign the contract now and you get a special price for a limited time!<br />
<br />
Fortunately you're in luck. Microsoft released Active Directory Federation Services (ADFS) all the way back with 2003 r2 and released the much appreciated version 2.0 a bit after the release of 2008 r2 (install-able as an upgrade). Version 3.0 has just shipped with 2012 r2, and each release has brought many welcome new features.<br />
<br />
Put (probably too) simply, ADFS allows you to extend your Active Directory space to other platforms, including Azure, Amazon EC2, and other cloud services including those supporting <a href="http://en.wikipedia.org/wiki/Security_Assertion_Markup_Language">SAML</a>. By doing so, you can grant your already set up internal users access to new services located elsewhere. ADFS is also required for some Microsoft products including the new <a href="http://technet.microsoft.com/en-us/library/dn280944.aspx">Web Application Proxy</a>, which I'll be covering later. This isn't the only way to accomplish AD auth to other parties (<a href="http://www.windowsazure.com/en-us/documentation/articles/virtual-networks-install-replica-active-directory-domain-controller/">AD Replica</a>; IDaaS), but it's a great place to start.<br />
<br />
The main components of ADFS are as follows:<br />
<br />
<ul>
<li><b>Federation Service</b>: This service on one to many servers facilitates the core functionality; sending/receiving authentication requests to/from third parties. </li>
<li><b>Federation Service Proxy</b>: Sits in a perimeter network (DMZ) and sends requests to Federation Service servers on the interior of your network on behalf of clients outside the interior. </li>
</ul>
<div class="separator" style="clear: both; text-align: center;">
</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxt8Fr0KKfaLnOyOXImujE8Gj6OsMI9ZitYRXbY1FDwZkVz9FkcNbRMvfuJQgZfMmGHsOyvFHUptf6W0BJCe4o98tsPgXAngk37Hhq2tV1LyCfhWK-cYQ_J-CgHD5iDZj0LGdyuIIkAU4/s1600/ADFS_Basic.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxt8Fr0KKfaLnOyOXImujE8Gj6OsMI9ZitYRXbY1FDwZkVz9FkcNbRMvfuJQgZfMmGHsOyvFHUptf6W0BJCe4o98tsPgXAngk37Hhq2tV1LyCfhWK-cYQ_J-CgHD5iDZj0LGdyuIIkAU4/s1600/ADFS_Basic.png" height="588" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Highly simplified view of ADFS In Use; I'm guessing Bob is cool with that. </td></tr>
</tbody></table>
<br />
With that primer set, let's walk through a basic install/test of a single instance of Active Directory Federation Services 3.0 on Windows 2012 r2. I will not be covering the proxy or services integration at this time.<br />
<br />
<h3>
Requirements</h3>
<br />
Other than domain admin rights as listed, you'll also need:<br />
<ul>
<li>An Active Directory domain you would like to extend</li>
<li>2012 r2 media and licensing taken care of. </li>
<li>Domain Admin privileges; minor AD updates will be necessary and you'll need a new service account.</li>
<li>The ability to create new trusted certificates from either an internal or external source. (Need a new PKI setup? See my article <a href="http://blog.ittoby.com/2012/04/creating-two-tier-pki-windows-2008r2.html">here</a>.)</li>
<li>At least one VM/Physical machine; One for the ADFS server and another as a shared database infrastructure if you desire. More on those options below. </li>
<li>Desired server on the internal network and domain joined with access to Active Directory servers. </li>
<li>Access to add records to DNS. (May not be required but I recommend it) </li>
<li>(If applicable) Ability to create a new DB in a shared infrastructure. Note this is only a requirement if you do not intend to use the Windows Internal Database. </li>
</ul>
<h3>
Sizing</h3>
<div>
<br /></div>
<div>
ADFS, like Active Directory and most related services, is not very demanding from a hardware perspective. A modestly sized VM is generally a good solution in most cases and scaling usually is realized horizontally, dictated by geographic concerns. You can use either the built in Windows Internal Database for ADFS or place the database on a shared infrastructure if desired. I will be using a shared DB infrastructure for this example. For more information on sizing, see <a href="http://technet.microsoft.com/en-us/library/gg749917.aspx">this</a> link. </div>
<div>
<br /></div>
<h3>
Step 1: Install ADFS Binaries</h3>
<div>
<ol>
<li>Log on to the machine you would like to set up as an ADFS server with your administrative logon. </li>
<li>If not launched already, launch Server Manager and click <i>Manage</i>-> <i>Add Roles and Features</i></li>
<li>Click <i>Next </i>on the <i>Before you begin</i> page. </li>
<li>Select <i>Role-based or feature-based installation </i>and click <i>Next</i>.</li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjksKCdXDsQfeX4D6F63uC234ubgBhsnMrHILqBVAdbIa8x_dMRFi-T8ueCXMcpqoqkYaYSItAsT2OolC0uQ82CjnaT-NFeEMc7i_yorTzAofBTId_Y-c1QhtVoF9a8iP9CUhwc3BPrQjI/s1600/2013-12-30+01_20_43-Thog+on+LINK+-+Virtual+Machine+Connection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjksKCdXDsQfeX4D6F63uC234ubgBhsnMrHILqBVAdbIa8x_dMRFi-T8ueCXMcpqoqkYaYSItAsT2OolC0uQ82CjnaT-NFeEMc7i_yorTzAofBTId_Y-c1QhtVoF9a8iP9CUhwc3BPrQjI/s1600/2013-12-30+01_20_43-Thog+on+LINK+-+Virtual+Machine+Connection.png" height="283" width="400" /></a></div>
<br />
<li>Select the appropriate server and click <i>Next</i>.</li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgoY3HfpkckGxrWBfnEodkPZUkaaYaqmOijUVPV4FpjcXCTYie5zCpD1DwhpzBfdOfgP0ZoCcDKWziXxruhfwBgrduBl1Mrr8xEK1ZgIcF5FgDWs1ZwJSyGMBmW10ld1YS8AllDNwP8An4/s1600/2013-12-30+01_21_15-Thog+on+LINK+-+Virtual+Machine+Connection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgoY3HfpkckGxrWBfnEodkPZUkaaYaqmOijUVPV4FpjcXCTYie5zCpD1DwhpzBfdOfgP0ZoCcDKWziXxruhfwBgrduBl1Mrr8xEK1ZgIcF5FgDWs1ZwJSyGMBmW10ld1YS8AllDNwP8An4/s1600/2013-12-30+01_21_15-Thog+on+LINK+-+Virtual+Machine+Connection.png" height="281" width="400" /></a></div>
<br />
<li>Select the <i>Active Directory Federation Services</i> role and click <i>Next</i>. (Note that despite serving up some services via HTTPS IIS is <b>not </b>needed)</li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEif_053tjwJhIDJQqjD-OAdXrxpwi5cVtn7yBxd7FrbUi4ydNTt8LIC7Iu8NjfEYTiyo9ONIU0b3k1dIKqAueXrshO3evPy9xOowAtKEZ01J6NFTs75lbtEuvYzveUZcQxA7WCsgB8urgc/s1600/2013-12-30+01_21_47-Thog+on+LINK+-+Virtual+Machine+Connection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEif_053tjwJhIDJQqjD-OAdXrxpwi5cVtn7yBxd7FrbUi4ydNTt8LIC7Iu8NjfEYTiyo9ONIU0b3k1dIKqAueXrshO3evPy9xOowAtKEZ01J6NFTs75lbtEuvYzveUZcQxA7WCsgB8urgc/s1600/2013-12-30+01_21_47-Thog+on+LINK+-+Virtual+Machine+Connection.png" height="283" width="400" /></a></div>
<br />
<li>No additional features will be needed; click <i>Next.</i></li>
<li>Read the ADFS notes and then click <i>Next</i> to proceed.</li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7FrRWChxD-CIbh3N2yn5K3PE7hGYEWYs6xa1wZzub_C2jO3q0Ap9RBrowT-hOCfn_7hyphenhyphen2o6C6vyGGjLEnLNwQ7Zjf-MoHBSpAGePUx1gBB-dbDp-Ku-xiOkDkcPSkokXRw_XxHu7FLO0/s1600/2013-12-30+01_22_42-Thog+on+LINK+-+Virtual+Machine+Connection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7FrRWChxD-CIbh3N2yn5K3PE7hGYEWYs6xa1wZzub_C2jO3q0Ap9RBrowT-hOCfn_7hyphenhyphen2o6C6vyGGjLEnLNwQ7Zjf-MoHBSpAGePUx1gBB-dbDp-Ku-xiOkDkcPSkokXRw_XxHu7FLO0/s1600/2013-12-30+01_22_42-Thog+on+LINK+-+Virtual+Machine+Connection.png" height="283" width="400" /></a></div>
<br />
<li>Click <i>Install</i> to execute. When completed, do <b>not</b> configure ADFS yet.</li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfdqF4Kv8vXCvPavE6EgzSqdvCKF7_ZeW-q6heZ9VfrscdmaKgZG6iOvc6OfjdHmPEIOKJiilWf1JGw68SsIgQeljUkri4C2pu7TYTdxQpbcHw_3L45OSM-PZWnF_hyphenhyphensaEAFjXGYpwd4k/s1600/2013-12-30+01_23_05-Thog+on+LINK+-+Virtual+Machine+Connection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfdqF4Kv8vXCvPavE6EgzSqdvCKF7_ZeW-q6heZ9VfrscdmaKgZG6iOvc6OfjdHmPEIOKJiilWf1JGw68SsIgQeljUkri4C2pu7TYTdxQpbcHw_3L45OSM-PZWnF_hyphenhyphensaEAFjXGYpwd4k/s1600/2013-12-30+01_23_05-Thog+on+LINK+-+Virtual+Machine+Connection.png" height="281" width="400" /></a></div>
<br />
</ol>
<h3>
Step 2 : Acquire Certificates </h3>
</div>
<div>
<br /></div>
<div>
For ADFS to function correctly you'll need at least one certificate. The certificates needed are for service communications and Token-signing. For the purpose of this tutorial I will not be replacing the auto-generated Token-signing certificate because the install will work fine without doing so. For a large scale, production installation, however, I highly recommend installing a custom Token-signing certificate after installation. For more information on how to do so, see <a href="http://technet.microsoft.com/en-us/library/hh341466%28v=ws.10%29">this TechNet link</a>.<br />
<br />
As for the service communications certificate, this must be created to proceed. This can be either an internal PKI or third party cert so long as <b>all</b> clients intending to use ADFS trust the certificate. Obviously if you're working with a large third party (say salesforce.com) you'll need another third party's certificate (<a href="http://www.digicert.com/">digicert</a> for example). If you're working across your enterprise only, you can use your PKI but you will need to ensure all devices (mobile included for workplace join) trust your CA and your CRL needs to be accessible externally (see <a href="http://blog.ittoby.com/2012/04/creating-two-tier-pki-windows-2008r2.html">Publish Root CA CRL & CRT to Web</a>) For the purposes of this example I'll be using the company PKI so expect specifics of certificate application to differ accordingly.<br />
<br />
As for the certificate name you should pick the eventual external name of the ADFS service. Should you configure an ADFS proxy in the future, Microsoft instructions mandate the use of the same certificate. For this reason plan accordingly. Other SANs are necessary as well; we'll cover that below:</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgNXPuv6-yj7ailR8CCn0K9PReADXibiNz2XLQVdcgGRI5GrqIGtMs4Ufj0HHBw1LRwbK__aiUQi0k54dX3KtFsWAbo9qNuK01Ct8INr_LxlRMaCz-vdnhuMvNX6JCLHb_murSzpUDL6uc/s1600/2013-12-30+09_23_53-Thog+on+LINK+-+Virtual+Machine+Connection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgNXPuv6-yj7ailR8CCn0K9PReADXibiNz2XLQVdcgGRI5GrqIGtMs4Ufj0HHBw1LRwbK__aiUQi0k54dX3KtFsWAbo9qNuK01Ct8INr_LxlRMaCz-vdnhuMvNX6JCLHb_murSzpUDL6uc/s1600/2013-12-30+09_23_53-Thog+on+LINK+-+Virtual+Machine+Connection.png" height="218" width="400" /></a></div>
<br />
<div>
<ol>
<li>Apply for a certificate; if using internal Microsoft AD PKI the web server template will work fine. </li>
<li>Set the common name to your desired external connection URL. For this example I'll be using <i>fs.companyname.com</i></li>
<li>Add the following DNS SANs (subject alternative names)</li>
<ol>
<li><i>fs.companyname.com </i>(Yes, it's in the CN but needed here again to work properly)</li>
<li><i>internalservername.internaldomain.lan </i>(Unless you plan on splitting DNS internally) </li>
<li><i>enterpriseregistration.companyname.com</i> (This is needed should you ever want to utilize <a href="http://technet.microsoft.com/en-us/library/dn280945.aspx">Workplace Join</a> and won't hurt you even if you don't) </li>
</ol>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhKPXDVg7wfWjNMfdpk7D-JFp8WFIDgEfCpsS25bu9z9BHl3fMfqlVzyNaQaTDYfT_3xpez3DrAoH_bF1hFCBtZtjemCBrRXXJ_zuOMX0Nkyo22vbcJH33HZv2-x-8ToUWYuk-UktWzo9s/s1600/2013-12-31+02_36_55-RD+Tabs+64.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhKPXDVg7wfWjNMfdpk7D-JFp8WFIDgEfCpsS25bu9z9BHl3fMfqlVzyNaQaTDYfT_3xpez3DrAoH_bF1hFCBtZtjemCBrRXXJ_zuOMX0Nkyo22vbcJH33HZv2-x-8ToUWYuk-UktWzo9s/s1600/2013-12-31+02_36_55-RD+Tabs+64.png" height="397" width="400" /></a></div>
<br />
<li>Ensure the private key is marked as exportable as you'll need this cert for an ADFS Proxy in the likely event that you need it.</li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6naLJjV-FxkffIMdX-ZdpUUl6lGSufko8UGEddWE7CoJ4RCW_KmLMVnEdE-uOrWBlbQGkbfF95o4mb5fUhIXR3hP0MFmAfGyScGSHMTyxSKxTLrfkL64CDocsXnExicvFBY57IOJZh40/s1600/2013-12-30+09_28_55-Thog+on+LINK+-+Virtual+Machine+Connection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6naLJjV-FxkffIMdX-ZdpUUl6lGSufko8UGEddWE7CoJ4RCW_KmLMVnEdE-uOrWBlbQGkbfF95o4mb5fUhIXR3hP0MFmAfGyScGSHMTyxSKxTLrfkL64CDocsXnExicvFBY57IOJZh40/s1600/2013-12-30+09_28_55-Thog+on+LINK+-+Virtual+Machine+Connection.png" height="398" width="400" /></a></div>
<br />
<li>Acquire the certificate and ensure it is installed correctly in the personal store of the local computer. </li>
</ol>
<h3>
Step 3: Configure ADFS</h3>
</div>
<div>
<br />
With your cert in hand we're ready to config ADFS. Be prepared to create a new service account for this step. </div>
<div>
<ol>
<li>On Server Manager on the ADFS server, click the flag and then select <i>Configure the federation server on this server.</i></li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEicmQX-JnyCLd2dur7-fQpy7K_zcMQ-9gsvuVPBZsWxYkznUFT8PmPWJe2_6EpBc3XSxeIvuAL3ErVXfLg0vFknolCCB_IO01sETYX3lJbNze8UaQUzblTyO6eOIFeFaELNznUprATk0qE/s1600/2013-12-30+09_41_33-Thog+on+LINK+-+Virtual+Machine+Connection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEicmQX-JnyCLd2dur7-fQpy7K_zcMQ-9gsvuVPBZsWxYkznUFT8PmPWJe2_6EpBc3XSxeIvuAL3ErVXfLg0vFknolCCB_IO01sETYX3lJbNze8UaQUzblTyO6eOIFeFaELNznUprATk0qE/s1600/2013-12-30+09_41_33-Thog+on+LINK+-+Virtual+Machine+Connection.png" /></a></div>
<br />
<li>On the <i>Welcome</i> screen, select <i>Create the first federation server in a federation server farm </i>and click <i>Next</i>.</li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcIA1YQR-bh0E78WWsYfZObiOhuZIr3NXstqCrTXhrtGWnNnbgVWcaGRa9tWmfrI7jbRKXGz2LzAN-xZ5VeRzVR6qUMVauKrp1AmtW2aJbFEMqrbbrTgtzUKjKLtZlbKrkA2j_OvJID0s/s1600/2013-12-30+09_41_44-Thog+on+LINK+-+Virtual+Machine+Connection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcIA1YQR-bh0E78WWsYfZObiOhuZIr3NXstqCrTXhrtGWnNnbgVWcaGRa9tWmfrI7jbRKXGz2LzAN-xZ5VeRzVR6qUMVauKrp1AmtW2aJbFEMqrbbrTgtzUKjKLtZlbKrkA2j_OvJID0s/s1600/2013-12-30+09_41_44-Thog+on+LINK+-+Virtual+Machine+Connection.png" height="292" width="400" /></a></div>
<br />
<li>For <i>Connect to AD DS</i> ensure you have the appropriate user selected. If it's the current user you won't need to change anything. Click <i>Next</i>. </li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhh_0z_tGBQfJ5PKgRkGfQ3oHkMdwtvPGXRs2VryAHRGje6O2SzH-B3Rj1gTKwSH9S7b05eWp2sRFGE3aukUvW1KjvzExssz4PRgQ_eDSp_VdKUuhxRjjLDeDJVPgbKjMtj-hX0S5iIFCk/s1600/2013-12-30+09_42_17-Thog+on+LINK+-+Virtual+Machine+Connection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhh_0z_tGBQfJ5PKgRkGfQ3oHkMdwtvPGXRs2VryAHRGje6O2SzH-B3Rj1gTKwSH9S7b05eWp2sRFGE3aukUvW1KjvzExssz4PRgQ_eDSp_VdKUuhxRjjLDeDJVPgbKjMtj-hX0S5iIFCk/s1600/2013-12-30+09_42_17-Thog+on+LINK+-+Virtual+Machine+Connection.png" height="292" width="400" /></a></div>
<br />
<li>On <i>Specify Service Properties</i> select the SSL certificate you imported earlier or import it now if you have yet to do so. </li>
<li>Ensure the <i>Federation Service Name</i> matches the external URL reference associated with the SSL certificate name. </li>
<li>Set the <i>Federation Service Display Name</i> to what you would like your users to see when using the service explicitly, i.e. "My Awesome Company". Click <i>Next</i>.</li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkAXjxQZMtOcDbaIVskNGPzp5wNtzxkwv2Mv4ZTj9JEMDZmXfwafz447AwtmcFMqdry-DROfDSX77W36zv_jVs-s4n52XNSAZjc65Kbgb3kqinKXmQF9NXN0e_r1oPYTzDrcUG_P6imNg/s1600/2013-12-30+09_43_03-Thog+on+LINK+-+Virtual+Machine+Connection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkAXjxQZMtOcDbaIVskNGPzp5wNtzxkwv2Mv4ZTj9JEMDZmXfwafz447AwtmcFMqdry-DROfDSX77W36zv_jVs-s4n52XNSAZjc65Kbgb3kqinKXmQF9NXN0e_r1oPYTzDrcUG_P6imNg/s1600/2013-12-30+09_43_03-Thog+on+LINK+-+Virtual+Machine+Connection.png" height="292" width="400" /></a></div>
<br />
<li>As for the Service Account, I recommend using a <a href="http://technet.microsoft.com/en-us/library/hh831782.aspx">Group Managed Service Account</a> if your domain supports it (Windows 2012). If not, fall back to a Managed Service Account or standard Service Account. As I'll be using a managed service account we'll need to do the following; if not, substitute your own account creation needs for the sub steps below. </li>
<ol>
<li>To facilitate the GMSAccount Open a Powershell prompt as administrator and execute <i>Add-KdsRootKey -EffectiveTime (Get-Date).AddHours(-10)</i></li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWdeAXB2tMzaAiRpNUYUvFrOgCqizH3I-RnwRL6FVyDN4o51yB0xFfM6sgL1XXTP1fl79D7NrXPginKzYcTcEIWvPf6LHeTJXTlxDMDzSagjrAFWe7w5zhpnxhlWq_3IN3A47L1cb7zng/s1600/2013-12-30+09_59_23-Thog+on+LINK+-+Virtual+Machine+Connection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWdeAXB2tMzaAiRpNUYUvFrOgCqizH3I-RnwRL6FVyDN4o51yB0xFfM6sgL1XXTP1fl79D7NrXPginKzYcTcEIWvPf6LHeTJXTlxDMDzSagjrAFWe7w5zhpnxhlWq_3IN3A47L1cb7zng/s1600/2013-12-30+09_59_23-Thog+on+LINK+-+Virtual+Machine+Connection.png" height="144" width="640" /></a></div>
<br />
<li>Close Powershell and return to the Wizard</li>
</ol>
<li>Select the appropriate service account type and specify a name. If using a GMSAccount the Wizard can create it for you. Be sure you give it a name that suits your standards (I use GS_<description> where G=Group,S=Service, _<desc>=meaningful) and click <i>Next</i>. </desc></description></li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQSpqRN1u2hpw3C5g9DqVgXlNL6NLGfRXcSqUfGtIRkJ1dxlxBmd0BdaNKZ835jv6uSUAPJvm-blD7ohzPJpPN639z-VhdpMCYiqKGruHtg4THdm-cMZeGm4fKS_2kS3GfmJOY6acpxGY/s1600/2013-12-30+10_02_28-Thog+on+LINK+-+Virtual+Machine+Connection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQSpqRN1u2hpw3C5g9DqVgXlNL6NLGfRXcSqUfGtIRkJ1dxlxBmd0BdaNKZ835jv6uSUAPJvm-blD7ohzPJpPN639z-VhdpMCYiqKGruHtg4THdm-cMZeGm4fKS_2kS3GfmJOY6acpxGY/s1600/2013-12-30+10_02_28-Thog+on+LINK+-+Virtual+Machine+Connection.png" height="292" width="400" /></a></div>
<br />
<li>Select if you would like to use the Windows Internal Database or a SQL install/instance on a different server. In my example I'm using a dedicated SQL server. Click <i>Next</i>.</li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWbmmXX5bAJnxt7DNrCvqVMj61A-QELHZNI2wnhJHgjQR-7YcW6s_MrMITrHxAMDRdXn8RCvRgEzFzWef-y9uvnC7y9qor2iBAWAJB8cF2gezJUFgu4D-fdpAU4wtYEr3o_P-fREjCz-k/s1600/2013-12-30+10_14_25-Thog+on+LINK+-+Virtual+Machine+Connection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWbmmXX5bAJnxt7DNrCvqVMj61A-QELHZNI2wnhJHgjQR-7YcW6s_MrMITrHxAMDRdXn8RCvRgEzFzWef-y9uvnC7y9qor2iBAWAJB8cF2gezJUFgu4D-fdpAU4wtYEr3o_P-fREjCz-k/s1600/2013-12-30+10_14_25-Thog+on+LINK+-+Virtual+Machine+Connection.png" height="292" width="400" /></a></div>
<br />
<li>On the <i>Review Options</i> page you will be presented with the information you have just entered. Ensure it is correct and click the <i>View Script </i>button. This is the Powershell equivalent for this installation; save it for later reference and perhaps use for another member in the farm. Click <i>Next</i>.</li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_dgV8emp0FNPKwroD209AKlxKaIJwbmEL1wu585hBW0y1iEFcQIqO8bPVor7W4glK4EWoK1pt9bG9xtHlh2uOGFB96ntGTj7I0aPeBqbYpSUyofpONlr_pgj_Ms2RqKnsBtWVQncOjr4/s1600/2013-12-30+10_15_25-Thog+on+LINK+-+Virtual+Machine+Connection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_dgV8emp0FNPKwroD209AKlxKaIJwbmEL1wu585hBW0y1iEFcQIqO8bPVor7W4glK4EWoK1pt9bG9xtHlh2uOGFB96ntGTj7I0aPeBqbYpSUyofpONlr_pgj_Ms2RqKnsBtWVQncOjr4/s1600/2013-12-30+10_15_25-Thog+on+LINK+-+Virtual+Machine+Connection.png" height="400" width="362" /></a></div>
<br />
<li>The <i>Pre-Requisite</i> checks will run. Provided it passes, click <i>Configure</i>. </li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgm6OHlKnrNcNwf3535tVpWIQ9zS0rF21DYubL8zf1PoAAu8D6sST0A8mKhpU8oG_Uky1Ig_qnRUz5YHWJkpexTzQzYF_80itpxvlPc6dIN45c2i017EClYHxDdEv4-qYFJKyRx2HqKL9k/s1600/2013-12-30+10_16_07-Thog+on+LINK+-+Virtual+Machine+Connection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgm6OHlKnrNcNwf3535tVpWIQ9zS0rF21DYubL8zf1PoAAu8D6sST0A8mKhpU8oG_Uky1Ig_qnRUz5YHWJkpexTzQzYF_80itpxvlPc6dIN45c2i017EClYHxDdEv4-qYFJKyRx2HqKL9k/s1600/2013-12-30+10_16_07-Thog+on+LINK+-+Virtual+Machine+Connection.png" height="291" width="400" /></a></div>
<br />
</ol>
<div>
<h3>
Step 4: Configure Networking</h3>
</div>
<div>
<ol>
<li>Add a DNS record to your internal network to point the public URL you specified in the SSL certificate to the internal address of the ADFS server. </li>
<li>If testing workplace join internally, add the appropriate enterprise registration DNS entry as well. </li>
<li>If you proceed further with an ADFS Proxy you will need to add an external DNS record at that time. </li>
</ol>
</div>
<h3>
Step 5: Test and Troubleshoot</h3>
</div>
<div>
<br /></div>
<div>
After installation you can perform a simple test by navigating to:<br />
<br />
<a href="https://fs.%28companyname%29.com/adfs/ls/idpinitiatedsignon">https://fs.(companyname).com/adfs/ls/idpinitiatedsignon</a></div>
<div>
<br /></div>
<div>
Where the fs.(companyname).<companyname>com represents the URL specified in the DNS SAN/CN of the cert. This should present you with the following logon screen: </companyname></div>
<div>
<br /></div>
<div>
All you need do is click the <i>Sign In</i> button and you will receive a login prompt. If entered correctly and all is working you should get the message <i>You are signed in. </i>when completed. </div>
<div>
<br /></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzPugYAEYRE5VJYxn5aHIfdmjumN9lx-OwSJ7CrTHuPshnN9yZuyuf0hSjrMxTiGTLujlrK1UPtP08bPFgihpNjcMwXil5SJLp1wsGtfWNmMcB9gIJnPmJ-DSyp4nvGtfPFhAGMw6xCJ0/s1600/2014-01-28+01_23_54-Sign+In+-+Internet+Explorer.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzPugYAEYRE5VJYxn5aHIfdmjumN9lx-OwSJ7CrTHuPshnN9yZuyuf0hSjrMxTiGTLujlrK1UPtP08bPFgihpNjcMwXil5SJLp1wsGtfWNmMcB9gIJnPmJ-DSyp4nvGtfPFhAGMw6xCJ0/s1600/2014-01-28+01_23_54-Sign+In+-+Internet+Explorer.png" height="411" width="640" /></a></div>
<br />
<div>
If you have any additional problems I can offer the following: </div>
<div>
<br /></div>
<div>
<ul>
<li>The most interesting event logs are stored in <i>Event Viewer</i>-><i>Applications and Services Logs</i>-><i>AD FS</i>-><i>Admin. </i></li>
<li>If the configuration fails with a cryptic error message, ensure you don't have anything taking up port 80 or port 443. If either is taken the config routine crashes. </li>
<li>If you need to re-do the installation for any reason that is fine, but make sure you overwrite the database. The easiest way to do so is use the powershell script we mentioned above, but add the <i>-OverwriteConfiguration True </i>parameter. </li>
</ul>
</div>
<div>
<br /></div>
<h3>
In Closing</h3>
<div>
Clearly this is just the beginning. The next logical step would be to setup an ADFS Proxy and then establish relationships with your SaaS providers. We'll have more on that in the future. To get started now, check these thoroughly vetted and highly appreciated links, now with 86% less sand.<br />
<br />
Edit: <a href="https://plus.google.com/101698498606612076671">Dan Salmon</a> from the good folks at <a href="http://www.rbaconsulting.com/">RBA Consulting</a> sent over <a href="https://social.technet.microsoft.com/wiki/contents/articles/20235.windows-azure-active-directory-application-gallery-sso-ready-saas-apps.aspx">this</a> excellent Microsoft link that lists many SSO ready SaaS offerings. A great place to start if you're looking...<br />
<br />
<h4>
ADFS</h4>
<a href="http://technet.microsoft.com/en-us/library/cc772593(WS.10).aspx">WindowServer: ADFS Overview</a><br />
<a href="http://technet.microsoft.com/en-us/library/gg749917.aspx">TechNet: Planning for Federation Server Capacity</a><br />
<a href="http://social.technet.microsoft.com/Forums/en-US/158b80b3-84fb-4940-a3d0-374bbaee38b7/adfs-ssl-certificates?forum=winserversecurity">TechNet Security Forum: ADFS SSL Certificates</a><br />
<a href="http://blogs.technet.com/b/adfs/archive/2007/07/23/adfs-certificates-ssl-token-signing-and-client-authentication-certs.aspx">ADFS Product Support Blog: SSL, Token Signing, and Client Authentication Certs</a><br />
<a href="http://social.msdn.microsoft.com/Forums/vstudio/en-US/32c87fed-65ed-47ad-87cd-b3bd0106c5e0/adfs-token-signing-token-decrypting-certificate-3rd-party-certificate-required?forum=Geneva">MSDN Social: ADFS Token Cert - 3rd Party Cert Required?</a><br />
<a href="http://technet.microsoft.com/en-us/library/dd807040%28v=ws.10%29">TechNet: Certificate Requirements for Federation Services</a><br />
<a href="http://technet.microsoft.com/en-us/library/hh341466%28v=ws.10%29">TechNet: Token Signing Certificates</a><br />
<a href="http://technet.microsoft.com/en-us/library/cc779508(v=ws.10).aspx">TechNet: Federated Web SSO Example</a><br />
<a href="http://technet.microsoft.com/en-us/library/cc780038(v=ws.10).aspx">TechNet: Installing the ADFS Web Agent component of ADFS</a><br />
<a href="http://technet.microsoft.com/en-us/magazine/ff721824.aspx">Jeffrey Schwartz: ADFS 2.0 Open Doors to the Cloud</a><br />
<a href="http://wiki.servicenow.com/index.php?title=Configuring_ADFS_2.0_to_Communicate_with_SAML_2.0">Example in Action: ServiceNow ADFS/SAML config</a><br />
<h4>
ADFS Proxy</h4>
</div>
<div>
<a href="http://technet.microsoft.com/en-us/library/dd807054(v=WS.10).aspx">TechNet: Certificate Requirements for Federation Server Proxies</a></div>
<div>
<a href="http://blogs.technet.com/b/askds/archive/2012/01/05/understanding-the-ad-fs-2-0-proxy.aspx">Ask the DS Team: Understanding ADFS Proxy</a> (2.0, still applicable)</div>
<div>
<br /></div>
<h4>
Azure + AD</h4>
<div>
<a href="http://www.windowsazure.com/en-us/documentation/articles/virtual-networks-install-replica-active-directory-domain-controller/">Windows Azure: Install a Replica Active Directory DC</a> (not ADFS)</div>
<div>
<br /></div>
<h4>
Misc</h4>
<div>
<a href="http://blogs.technet.com/b/matthewms/archive/2013/11/01/why-windows-server-2012-r2-step-by-step-workplace-join-bringing-peace-of-mind-for-byod.aspx">Matt Hester's Weblog: Step-by-step Workplace Join</a></div>
<div>
<a href="http://blogs.technet.com/b/kevinremde/archive/2013/10/30/what-s-new-for-active-directory-in-server-2012-r2.aspx">Kevin Remde: What's new in AD on 2012R2</a></div>
<div>
<a href="http://technet.microsoft.com/en-us/library/dn280945.aspx">TechNet: Overview of SSO/Workplace Join</a></div>
<div>
<br /></div>
<div>
Thanks for reading!</div>
<div>
<br /></div>
Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com1tag:blogger.com,1999:blog-5458589696790815336.post-85082718552402743212013-12-17T22:14:00.000-06:002014-11-15T22:48:19.054-06:00Monitoring Windows: Granular Service Rights in an Enterprise Environment <script type="text/javascript">
// toggle specific
function toggleDiv(id1,id2) {
var tag = document.getElementById(id1).style;
var tagicon = document.getElementById(id2);
if(tag.display == "none") {
tag.display = "block";
tagicon.innerHTML = " - ";
} else {
tag.display = "none";
tagicon.innerHTML = " + ";
}
}
function expandAll(cnt) {
for(var x=1; x<=cnt; x++) {
document.getElementById('content'+x).style.display="block";
document.getElementById('icon'+x).innerHTML=" - ";
}
}
function collapseAll(cnt) {
for(var x=1; x<=cnt; x++) {
document.getElementById('content'+x).style.display="none";
document.getElementById('icon'+x).innerHTML=" + ";
}
}
</script> <style type="text/css">
.Clicktitle {
padding:5px;
border:1px solid #CCCCCC;
font:15px arial;
width:300px;
cursor:pointer;
height:20px;
}
.Clickitem {
padding:5px;
border:1px solid #CCCCCC;
font:12px verdana;
}
.item div {
border-bottom:1px dashed #CCCCCC;
padding:5px;
}
.button {
border:1px solid #CCCCCC;
padding:5px;
cursor:pointer;
}
</style><br>
<h3>
Foreword</h3>
<br>
I'm a big fan of <a href="http://www.splunk.com/" target="_blank">Splunk</a>
and similar tools for network and systems monitoring. This new generation of
machine data analysis brings a myriad of product and monitoring
opportunities. These systems work by indexing large swaths of machine
logging (logs, performance information, etc.) and providing an incredibly
effective interface for reporting of that information.To achieve this,
however, you need to provide very broad access to target systems.<br>
<br>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikXEBZgxki5su2vjqeHHaE2DwXnJscZlmhBTdQJuzSanHBun6PM5e9h6GQdBsy0vkmoDwaOlo_gQI0axNjVMNTCU8mdg3Bol8dnBwDu_GudH9apaI5Rr647R3s9_5va_q9KHPqBx7uZPU/s1600/splunk1.jpg"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikXEBZgxki5su2vjqeHHaE2DwXnJscZlmhBTdQJuzSanHBun6PM5e9h6GQdBsy0vkmoDwaOlo_gQI0axNjVMNTCU8mdg3Bol8dnBwDu_GudH9apaI5Rr647R3s9_5va_q9KHPqBx7uZPU/s400/splunk1.jpg"
border="0" height="275" width="400"></a></div>
<br>
If for any reason you can't use <a href="http://docs.splunk.com/Splexicon:Universalforwarder">Universal
Forwarders</a> on Windows platforms, data collection can be
difficult. While Splunk provides <a href="http://docs.splunk.com/Documentation/Splunk/6.0/Data/MonitorWMIdata">some</a>
<a href="http://docs.splunk.com/Documentation/Splunk/latest/Installation/InstallonWindows"
target="_blank">great</a> <a href="http://docs.splunk.com/Documentation/Splunk/6.0/Data/ConsiderationsfordecidinghowtomonitorWindowsdata"
target="_blank">documentation</a>, it suffers from the same access (and
granting) problem as other platforms needing similar access. Generally this
problem is solved in one of two ways:<br>
<br>
<ul>
<li>Grant local administrator access on every machine to be monitored to a
single service account. Clearly this won't fly in organizations
concerned with granting the least amount of access needed for their
application portfolio. A single service account/group that has
administrator everywhere is a significant security risk. </li>
<li>Use Group Policy to grant rights, <a href="http://en.wikipedia.org/wiki/Access_control_list">ACL</a>s,
etc. The problem with this approach is that some things cannot be
addressed by <a href="http://technet.microsoft.com/en-us/windowsserver/bb310732.aspx">GPO</a>
(<a href="http://en.wikipedia.org/wiki/Windows_Management_Instrumentation">WMI</a>)
and if granting <a href="http://technet.microsoft.com/en-us/library/dd349804%28v=ws.10%29.aspx"
target="_blank">user rights</a> often you cannot be granular enough
since each right defined must list <b>all</b> accounts/groups that get
that right. Across an enterprise that is a difficult and potentially
costly proposition. The main problem with setting user rights via Group
Policy is that the rights are not additive; they overwrite all entries
including required system or other application entries. This makes
central management of thousands of machines impractical. Say you need to
define "act as part of the operating system" for your monitoring
software; you would then need to centrally define every account/group
that has that right and any machine that needs to define that right in a
different way would need to obtain an override GPO for the machines in
question. While this is possible, many very large organizations find it
difficult to execute.</li>
</ul>
Neither one of these solutions is desirable in a medium to large sized
enterprise due to security requirements and management complexities. To
address this issue in a more granular way than the two options listed above,
we can use scripts to automate local configuration where necessary.<br>
<br>
I'll be using Powershell scripts to perform the following tasks:<br>
<ul>
<li>Add Active Directory groups/users to local groups</li>
<li>Grant WMI rights to Active Directory groups/users</li>
<li>Grant local user rights to Active Directory groups/users</li>
</ul>
Assuming the affected security principals are not set elsewhere, we can set
these items with scripts that can be triggered via group policy. I'll be
using the GPO startup script functionality, but you can feel free to use any
other trigger mechanism you prefer (logon script, etc.). Note that this
solution <b>will</b> work for products similar to Splunk that require
similar rights.<br>
<br>
<h3>
<b>
Execution</b></h3>
<br>
First things first, we need to establish what we'll be doing and principals
to which we want to grant access.<br>
<br>
<h4>
Assumptions</h4>
<br>
<ul>
<li>Splunk indexer (or your other monitoring tool) installed and ready to
go</li>
<li>You have administrative rights on all machines in question and Active
Directory rights sufficient to create the service account/group(s)</li>
<li>You would rather use the Splunk Indexer to collect information rather
than install the <a href="http://docs.splunk.com/Documentation/Storm/Storm/User/SetupauniversalforwarderonWindows">Splunk
Universal Installer</a> on your target machines (Good option as
well). </li>
<li>Powershell 2.0 or newer & Windows 2003 or newer. This could be
done with batch/executables or VBScript but I won't be covering that.</li>
</ul>
<br>
<h3>
Service Account/Group Setup</h3>
<br>
Per the <a href="http://docs.splunk.com/Documentation/Splunk/6.0/Installation/ChoosetheuserSplunkshouldrunas#About_the_.22Local_System_user.22_and_.22other_user.22_choices"
target="_blank">Splunk instructions</a>, create a service account to run
Splunk. This will require administrative access on the machine running
Splunk, but nothing else (yet). It would be best to use a <a href="http://technet.microsoft.com/en-us/library/dd560633%28v=ws.10%29.aspx"
target="_blank">managed service account</a> but if you cannot a standard
account will be fine.<br>
<br>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjFs18vrqVX7UxKYKPH1riEAlbQq0Ax26aLTYW_RfDlAXGpSN85O3iMTzcJ9MAUnN5dteyjh4J_6-YSjtM7y9s-XwGGItj_LPkjvNNoqiM18w4FC1wf0RYn3aMkVz3924JI-PJVl8_vjI4/s1600/2013-12-11+03_07_34-.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjFs18vrqVX7UxKYKPH1riEAlbQq0Ax26aLTYW_RfDlAXGpSN85O3iMTzcJ9MAUnN5dteyjh4J_6-YSjtM7y9s-XwGGItj_LPkjvNNoqiM18w4FC1wf0RYn3aMkVz3924JI-PJVl8_vjI4/s640/2013-12-11+03_07_34-.png"
border="0" height="377" width="640"></a></div>
<br>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgcpePf_sroqv71DpAhIn84qWG3crYa_vBre-b5fVJZx32NK8IpCowO60DFtyDP5sXq6TnvWpB8D-o1Kf8FzEEmaOoNblkQVCHR9xohv5meWpsvu-vInGZdHozEP5wYyR6UziEoxLdOOLA/s1600/SplunkRights.png"
imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgcpePf_sroqv71DpAhIn84qWG3crYa_vBre-b5fVJZx32NK8IpCowO60DFtyDP5sXq6TnvWpB8D-o1Kf8FzEEmaOoNblkQVCHR9xohv5meWpsvu-vInGZdHozEP5wYyR6UziEoxLdOOLA/s1600/SplunkRights.png"
border="0"></a></div>
<br>
Splunk instructions deem that an Active Directory group need be created to
contain the service account. This isn't really necessary, but there are
potential advantages so we'll stick to it.<br>
<br>
After creating/configuring that service account, <a href="http://technet.microsoft.com/en-us/library/cc783256%28v=ws.10%29.aspx"
target="_blank">create the service account group</a> and make your service
account a member. For the purposes of our demonstration I've named this
group "Splunk Accounts"<br>
<br>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjupsJCdDsnDYt4PnvKXz1HxPscx-ngRHg_HqWOgDOO_uzukFJvD8E6t-qF4zhyphenhyphenUbGEGs2E4rh529DZwI19GQkHhr03ZpHswU_ddD6vChtIkjy2Sg1dUKEDvNlOSOaRGB8W6Gdm23TYCCo/s1600/2013-12-11+03_12_29-.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjupsJCdDsnDYt4PnvKXz1HxPscx-ngRHg_HqWOgDOO_uzukFJvD8E6t-qF4zhyphenhyphenUbGEGs2E4rh529DZwI19GQkHhr03ZpHswU_ddD6vChtIkjy2Sg1dUKEDvNlOSOaRGB8W6Gdm23TYCCo/s640/2013-12-11+03_12_29-.png"
border="0" height="352" width="640"></a></div>
<br>
<br>
You should monitor this group to ensure no other accounts are added since
doing so would grant those accounts unwanted rights.<br>
<br>
Now let's configure services. The service account <b>will</b> need
to be local administrator on machines running the Splunk software (<a href="http://docs.splunk.com/Splexicon:Splunkd">splunkd</a>,
<a href="http://docs.splunk.com/Splexicon:Splunkweb">splunkweb</a>) directly
(more on this another time). After granting local admin, stop the Splunkd
and splunkweb services, change the logon to your new service account and
start them back up.<br>
<br>
Now let's tackle the interesting part: granting granular rights to the
service account group.<br>
<br>
<h3>
Objective </h3>
<div>
</div>
<div>
Our objective is to grant the appropriate rights to the service account
group on every machine in our enterprise that we wish to monitor. Per the
documentation from Splunk, the minimum rights to collect Event Log and WMI
data remotely on a Windows platform are: </div>
<div>
<br>
<h4>
<b>Misc</b></h4>
</div>
<div>
<ul>
<li>WMI rights under root/cimv2</li>
<li>DCOM launch permissions</li>
</ul>
<div>
</div>
</div>
<div>
<h4>
<b>User Rights</b></h4>
</div>
<div>
<ul>
<li>Access this Computer from the Network</li>
<li>Act as part of the operating system (I'd like to confirm this one;
not sure why)</li>
<li>Log on as a batch job</li>
<li>Log on as a service</li>
<li>Profile System Performance</li>
<li>Replace a process level token (not sure about this one either)</li>
</ul>
</div>
<div>
<h4>
<b>Local Group Membership</b></h4>
</div>
<div>
<ul>
<li>Distributed COM Users</li>
<li>Event Log Readers</li>
<li>Performance Log Users</li>
</ul>
<div>
</div>
I'm sure you can see the why the official documentation prefers local
admin rights. :) Fortunately, we've got automation to do our heavy
lifting.<br>
<div>
</div>
<h4>
Scripts</h4>
<div>
</div>
<div>
Let's examine the scripts we'll be using. I've prepared four scripts for
this task: </div>
<div>
<ul>
<li>Powershell script to grant WMI and DCOM rights (Grant-WMIACL.ps1)</li>
<li>Powershell script to grant User Rights (Grant-Rights.ps1)</li>
<li>Powershell script to add local group membership and tie the others
together (Setup-Splunkuser.ps1)</li>
<li>Batch file to launch powershell script correctly on all platforms
(SetupSplunkuser.bat)</li>
</ul>
<div>
</div>
<div>
<h4>
<b>Grant-WMIACL.ps1</b></h4>
</div>
</div>
<div>
</div>
<div>
As discussed, this script will grant the appropriate WMI and DCOM
access. This is based heavily on the work of <a href="http://blogs.msdn.com/b/wmi/archive/2009/07/27/scripting-wmi-namespace-security-part-3-of-3.aspx"
target="_blank">Steve Lee</a> and <a href="http://unlockpowershell.wordpress.com/2009/11/20/script-remote-dcom-wmi-access-for-a-domain-user/"
target="_blank">Karl Mitschke</a>. Note that I've left options in the
script and commented out to change the level of access given to the WMI
objects, including the ability to toggle inheritance. I've left these in
to help you decipher the <a href="http://msdn.microsoft.com/en-us/library/windows/desktop/aa374928%28v=vs.85%29.aspx"
target="_blank">SDDL</a> structure. That said, the line that is not
commented out in this will work fine for our application. This script
could be run remotely (hence the $strComputer="localhost" line) but
we'll be using it locally (more on that below). </div>
<br>
<br>
<div class="Clicktitle" onclick="javascript:toggleDiv('content1','icon1');">
<span style="float: left;">Grant-WMIACL.ps1 (Click to Expand)</span> <span
id="icon1" style="float: right;"> + </span>
</div>
<div class="Clickitem" id="content1" style="display: none;">
<div>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">if ($args[0] -eq $null)
{
Write-Warning "No account or group name provided as an argument. Re-launch with the principal to grant rights to as the first argument."
Throw "I don't know what to do, please specify username, i.e. mydomain\myuser"
}
$Uname=$args[0]
Write-Warning "Granting WMI and DCOM access to $Uname..."
#This function gets a SID from an account name
function get-sid
{
Param ($DSIdentity)
$ID = new-object System.Security.Principal.NTAccount($DSIdentity)
return $ID.Translate( [System.Security.Principal.SecurityIdentifier] ).toString()
}
$sid = get-sid $Uname
#Enable Account, Remote Enable
#maps to "CC" SDDL_CREATE_CHILD ADS_RIGHT_DS_CREATE_CHILD , "WP" SDDL_WRITE_PROPERTY ADS_RIGHT_DS_WRITE_PROP
#$SDDL = "A;;CCWP;;;$sid"
#Provider Write(RP), Enable Account, Remote Enable
#maps to "RP" SDDL_READ_PROPERTY ADS_RIGHT_DS_READ_PROP , "CC" SDDL_CREATE_CHILD ADS_RIGHT_DS_CREATE_CHILD , "WP" SDDL_WRITE_PROPERTY ADS_RIGHT_DS_WRITE_PROP
#$SDDL = "A;;CCWPRP;;;$sid"
#Execute Methods(DC), Provider Write(RP), Enable Account, Remote Enable
#maps to "DC" SDDL_DELETE_CHILD ADS_RIGHT_DS_DELETE_CHILD , "RP" SDDL_READ_PROPERTY ADS_RIGHT_DS_READ_PROP , "CC" SDDL_CREATE_CHILD ADS_RIGHT_DS_CREATE_CHILD , "WP" SDDL_WRITE_PROPERTY ADS_RIGHT_DS_WRITE_PROP
#$SDDL = "A;;CCWPRPDC;;;$sid"
#Execute Methods(DC), Provider Write(RP), Enable Account, Remote Enable WITH INHERITENCE!! (note CI is in the header)
#maps to "CI" SDDL_CONTAINER_INHERIT CONTAINER_INHERIT_ACE , "DC" SDDL_DELETE_CHILD ADS_RIGHT_DS_DELETE_CHILD , "RP" SDDL_READ_PROPERTY ADS_RIGHT_DS_READ_PROP , "CC" SDDL_CREATE_CHILD ADS_RIGHT_DS_CREATE_CHILD , "WP" SDDL_WRITE_PROPERTY ADS_RIGHT_DS_WRITE_PROP
$SDDL = "A;CI;CCWPRPDC;;;$sid"
#Local Launch and Remote Activation
$DCOMSDDL = "A;;CCDCRP;;;$sid"
#Local Launch, Local Activation, Remote Launch, remote Activation
#$DCOMSDDL = “A;;CCDCLCSWRP;;;$sid”
#$computers = Get-Content "computers.txt"
#foreach ($strcomputer in $computers)
#{
$strComputer="localhost"
$Reg = [WMIClass]"\\$strComputer\root\default:StdRegProv"
$DCOM = $Reg.GetBinaryValue(2147483650,"software\microsoft\ole","MachineLaunchRestriction").uValue
$security = Get-WmiObject -ComputerName $strcomputer -Namespace root/cimv2 -Class __SystemSecurity
$converter = new-object system.management.ManagementClass Win32_SecurityDescriptorHelper
$binarySD = @($null)
$result = $security.PsBase.InvokeMethod("GetSD",$binarySD)
$outsddl = $converter.BinarySDToSDDL($binarySD[0])
$outDCOMSDDL = $converter.BinarySDToSDDL($DCOM)
$newSDDL = $outsddl.SDDL += "(" + $SDDL + ")"
$newDCOMSDDL = $outDCOMSDDL.SDDL += "(" + $DCOMSDDL + ")"
$WMIbinarySD = $converter.SDDLToBinarySD($newSDDL)
$WMIconvertedPermissions = ,$WMIbinarySD.BinarySD
$DCOMbinarySD = $converter.SDDLToBinarySD($newDCOMSDDL)
$DCOMconvertedPermissions = ,$DCOMbinarySD.BinarySD
#Set WMI Permissions
$result = $security.PsBase.InvokeMethod("SetSD",$WMIconvertedPermissions)
#Set DCOM Permissions
$result = $Reg.SetBinaryValue(2147483650,"software\microsoft\ole","MachineLaunchRestriction", $DCOMbinarySD.binarySD)
</code></pre>
</div>
</div>
<br>
She's a beaut eh? Now for <b>Grant-Rights.ps1</b>:<br>
<br>
This one is huge because we're using full .NET code in the beginning. That
code is a combination of snippets from <a href="http://stackoverflow.com/questions/10187837/granting-seservicelogonright-to-a-user-from-powershell"
target="_blank">here</a>, <a href="http://stackoverflow.com/questions/1147914/why-might-lsaaddaccountrights-return-status-invalid-parameter/14469248"
target="_blank">here</a>, and <a href="http://www.hightechtalks.com/csharp/lsa-functions-276626.html"
target="_blank">here</a>. Note the comments displaying the short names
for some of the rights we can assign which I've left in to facilitate
applying this approach to systems other than Spunk. More can be found <a
href="http://technet.microsoft.com/en-us/library/bb457125.aspx" target="_blank">here</a>.<br>
<br>
<br>
<div class="Clicktitle" onclick="javascript:toggleDiv('content2','icon2');">
<span style="float: left;">Grant-Rights.ps1 (Click to Expand)</span> <span
id="icon2" style="float: right;"> + </span>
</div>
<div class="Clickitem" id="content2" style="display: none;">
<div>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">Add-Type @'
using System;
using System.Collections.Generic;
using System.Text;
namespace MyLsaWrapper
{
using System.Runtime.InteropServices;
using System.Security;
using System.Management;
using System.Runtime.CompilerServices;
using System.ComponentModel;
using LSA_HANDLE = IntPtr;
[StructLayout(LayoutKind.Sequential)]
struct LSA_OBJECT_ATTRIBUTES
{
internal int Length;
internal IntPtr RootDirectory;
internal IntPtr ObjectName;
internal int Attributes;
internal IntPtr SecurityDescriptor;
internal IntPtr SecurityQualityOfService;
}
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
struct LSA_UNICODE_STRING
{
internal ushort Length;
internal ushort MaximumLength;
[MarshalAs(UnmanagedType.LPWStr)]
internal string Buffer;
}
sealed class Win32Sec
{
[DllImport("advapi32", CharSet = CharSet.Unicode, SetLastError = true),
SuppressUnmanagedCodeSecurityAttribute]
internal static extern uint LsaOpenPolicy(
LSA_UNICODE_STRING[] SystemName,
ref LSA_OBJECT_ATTRIBUTES ObjectAttributes,
int AccessMask,
out IntPtr PolicyHandle
);
[DllImport("advapi32", CharSet = CharSet.Unicode, SetLastError = true),
SuppressUnmanagedCodeSecurityAttribute]
internal static extern uint LsaAddAccountRights(
LSA_HANDLE PolicyHandle,
IntPtr pSID,
LSA_UNICODE_STRING[] UserRights,
int CountOfRights
);
[DllImport("advapi32", CharSet = CharSet.Unicode, SetLastError = true),
SuppressUnmanagedCodeSecurityAttribute]
internal static extern int LsaLookupNames2(
LSA_HANDLE PolicyHandle,
uint Flags,
uint Count,
LSA_UNICODE_STRING[] Names,
ref IntPtr ReferencedDomains,
ref IntPtr Sids
);
[DllImport("advapi32")]
internal static extern int LsaNtStatusToWinError(int NTSTATUS);
[DllImport("advapi32")]
internal static extern int LsaClose(IntPtr PolicyHandle);
[DllImport("advapi32")]
internal static extern int LsaFreeMemory(IntPtr Buffer);
}
/// <summary>
/// This class is used to grant "Log on as a service", "Log on as a batchjob", "Log on localy" etc.
/// to a user.
/// </summary>
public sealed class LsaWrapper : IDisposable
{
[StructLayout(LayoutKind.Sequential)]
struct LSA_TRUST_INFORMATION
{
internal LSA_UNICODE_STRING Name;
internal IntPtr Sid;
}
[StructLayout(LayoutKind.Sequential)]
struct LSA_TRANSLATED_SID2
{
internal SidNameUse Use;
internal IntPtr Sid;
internal int DomainIndex;
uint Flags;
}
[StructLayout(LayoutKind.Sequential)]
struct LSA_REFERENCED_DOMAIN_LIST
{
internal uint Entries;
internal LSA_TRUST_INFORMATION Domains;
}
enum SidNameUse : int
{
User = 1,
Group = 2,
Domain = 3,
Alias = 4,
KnownGroup = 5,
DeletedAccount = 6,
Invalid = 7,
Unknown = 8,
Computer = 9
}
enum Access : int
{
POLICY_READ = 0x20006,
POLICY_ALL_ACCESS = 0x00F0FFF,
POLICY_EXECUTE = 0X20801,
POLICY_WRITE = 0X207F8
}
const uint STATUS_ACCESS_DENIED = 0xc0000022;
const uint STATUS_INSUFFICIENT_RESOURCES = 0xc000009a;
const uint STATUS_NO_MEMORY = 0xc0000017;
IntPtr lsaHandle;
public LsaWrapper()
: this(null)
{ }
// // local system if systemName is null
public LsaWrapper(string systemName)
{
LSA_OBJECT_ATTRIBUTES lsaAttr;
lsaAttr.RootDirectory = IntPtr.Zero;
lsaAttr.ObjectName = IntPtr.Zero;
lsaAttr.Attributes = 0;
lsaAttr.SecurityDescriptor = IntPtr.Zero;
lsaAttr.SecurityQualityOfService = IntPtr.Zero;
lsaAttr.Length = Marshal.SizeOf(typeof(LSA_OBJECT_ATTRIBUTES));
lsaHandle = IntPtr.Zero;
LSA_UNICODE_STRING[] system = null;
if (systemName != null)
{
system = new LSA_UNICODE_STRING[1];
system[0] = InitLsaString(systemName);
}
uint ret = Win32Sec.LsaOpenPolicy(system, ref lsaAttr,
(int)Access.POLICY_ALL_ACCESS, out lsaHandle);
if (ret == 0)
return;
if (ret == STATUS_ACCESS_DENIED)
{
throw new UnauthorizedAccessException();
}
if ((ret == STATUS_INSUFFICIENT_RESOURCES) || (ret == STATUS_NO_MEMORY))
{
throw new OutOfMemoryException();
}
throw new Win32Exception(Win32Sec.LsaNtStatusToWinError((int)ret));
}
public void AddPrivileges(string account, string privilege)
{
LSA_UNICODE_STRING[] names = new LSA_UNICODE_STRING[1];
LSA_TRANSLATED_SID2 lts;
IntPtr tsids = IntPtr.Zero;
IntPtr tdom = IntPtr.Zero;
names[0] = InitLsaString(account);
lts.Sid = IntPtr.Zero;
Console.WriteLine("String account: {0}", names[0].Length);
int ret1 = Win32Sec.LsaLookupNames2(lsaHandle, 0, 1, names, ref tdom, ref tsids);
if (ret1 != 0)
throw new Win32Exception(Win32Sec.LsaNtStatusToWinError(ret1));
lts = (LSA_TRANSLATED_SID2)Marshal.PtrToStructure(tsids, typeof(LSA_TRANSLATED_SID2));
IntPtr pSid = lts.Sid;
//IntPtr pSid = GetSIDInformation(account);
LSA_UNICODE_STRING[] privileges = new LSA_UNICODE_STRING[1];
privileges[0] = InitLsaString(privilege);
uint ret = Win32Sec.LsaAddAccountRights(lsaHandle, pSid, privileges, 1);
Win32Sec.LsaFreeMemory(tsids);
Win32Sec.LsaFreeMemory(tdom);
if (ret == 0)
return;
if (ret == STATUS_ACCESS_DENIED)
{
throw new UnauthorizedAccessException();
}
if ((ret == STATUS_INSUFFICIENT_RESOURCES) || (ret == STATUS_NO_MEMORY))
{
throw new OutOfMemoryException();
}
throw new Win32Exception(Win32Sec.LsaNtStatusToWinError((int)ret));
}
public void Dispose()
{
if (lsaHandle != IntPtr.Zero)
{
Win32Sec.LsaClose(lsaHandle);
lsaHandle = IntPtr.Zero;
}
GC.SuppressFinalize(this);
}
~LsaWrapper()
{
Dispose();
}
// helper functions
IntPtr GetSIDInformation(string account)
{
LSA_UNICODE_STRING[] names = new LSA_UNICODE_STRING[1];
LSA_TRANSLATED_SID2 lts;
IntPtr tsids = IntPtr.Zero;
IntPtr tdom = IntPtr.Zero;
names[0] = InitLsaString(account);
lts.Sid = IntPtr.Zero;
Console.WriteLine("String account: {0}", names[0].Length);
int ret = Win32Sec.LsaLookupNames2(lsaHandle, 0, 1, names, ref tdom, ref tsids);
if (ret != 0)
throw new Win32Exception(Win32Sec.LsaNtStatusToWinError(ret));
lts = (LSA_TRANSLATED_SID2)Marshal.PtrToStructure(tsids,
typeof(LSA_TRANSLATED_SID2));
Win32Sec.LsaFreeMemory(tsids);
Win32Sec.LsaFreeMemory(tdom);
return lts.Sid;
}
static LSA_UNICODE_STRING InitLsaString(string s)
{
// Unicode strings max. 32KB
if (s.Length > 0x7ffe)
throw new ArgumentException("String too long");
LSA_UNICODE_STRING lus = new LSA_UNICODE_STRING();
lus.Buffer = s;
lus.Length = (ushort)(s.Length * sizeof(char));
lus.MaximumLength = (ushort)(lus.Length + sizeof(char));
return lus;
}
}
public class LsaWrapperCaller
{
public static void AddPrivileges(string account, string privilege)
{
using (LsaWrapper lsaWrapper = new LsaWrapper())
{
lsaWrapper.AddPrivileges(account, privilege);
}
}
}
}
'@
#Access this Computer from the Network
#SeNetworkLogonRight
#
#Act as part of the operating system
#SeTcbPrivilege
#
#Log on as a batch job
#SeBatchLogonRight
#
#Log on as a service
#SeServiceLogonRight
#
#Profile System Performance
#SeSystemProfilePrivilege
#
#Replace a process level token
#SeAssignPrimaryTokenPrivilege
if ($args[0] -eq $null -or $args[1] -eq $null)
{
Write-Warning "Available privs:"
Write-Host ""
Write-Host "Access this Computer from the Network"
Write-Warning "SeNetworkLogonRight"
Write-Host ""
Write-Host "Act as part of the operating system"
Write-Warning "SeTcbPrivilege"
Write-Host ""
Write-Host "Log on as a batch job"
Write-Warning "SeBatchLogonRight"
Write-Host ""
Write-Host "Log on as a service"
Write-Warning "SeServiceLogonRight"
Write-Host ""
Write-Host "Profile System Performance"
Write-Warning "SeSystemProfilePrivilege"
Write-Host ""
Write-Host "Replace a process level token"
Write-Warning "SeAssignPrimaryTokenPrivilege"
Throw "I don't know what to do, please specify username,right, i.e. mydomain\myuser, SeServiceLogonRight"
}
$Uname=$args[0]
$Right=$args[1]
Write-Warning "Giving $Right to $Uname..."
[MyLsaWrapper.LsaWrapperCaller]::AddPrivileges($Uname, $Right)
</code></pre>
</div>
</div>
<br>
<br>
Don't let the length of that code scare you off; the only part you need to
understand is the last few lines. Now let's examine <b>Setup-Splunkuser.ps1</b>:<br>
<br>
This script does a few things: defines the target for these privileges,
adds those to a series of groups, and ties the other scripts together. The
one thing you'll definitely want to modify is the entry at the top, <i>$SplunkAcct="MYDOMAIN\Splunk
Accounts"</i>. This should be the domain and group you created to host
the service account(s). The next line, <i>$localGroups="Distributed COM
Users","Event Log Readers","Performance Log Users"</i> local groups
to which you want to add AD groups to, each in quotes and separated by
commas. What I've set it to here is perfect for Splunk, but feel free to
change it for your needs. <br>
<br>
<br>
<div class="Clicktitle" onclick="javascript:toggleDiv('content3','icon3');">
<span style="float: left;">Setup-Splunkuser.ps1 (Click to Expand)</span>
<span id="icon3" style="float: right;"> + </span>
</div>
<div class="Clickitem" id="content3" style="display: none;">
<div>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">$SplunkAcct="MYDOMAIN\Splunk Accounts"
$localGroups="Distributed COM Users","Event Log Readers","Performance Log Users"
#Get-ScriptDirectory Function #
#############################################################
function Get-ScriptDirectory
{
$Invocation = (Get-Variable MyInvocation -Scope 1).Value
$path=Split-Path $Invocation.MyCommand.Path
# INSERT ESCAPE CHARACTERS FOR USING PATHS IN COMMANDS
$path=$path.replace('{','`{')
$path=$path.replace('}','`}')
$path=$path.replace('[','`[')
$path=$path.replace(']','`]')
$path
}
#**********************************************************************************************************************
#get directory
$ScriptDir=Get-ScriptDirectory
#grant rights
Invoke-Expression "$scriptdir\Grant-Rights.ps1 '$SplunkAcct' 'SeNetworkLogonRight'"
Invoke-Expression "$scriptdir\Grant-Rights.ps1 '$SplunkAcct' 'SeTcbPrivilege'"
Invoke-Expression "$scriptdir\Grant-Rights.ps1 '$SplunkAcct' 'SeBatchLogonRight'"
Invoke-Expression "$scriptdir\Grant-Rights.ps1 '$SplunkAcct' 'SeServiceLogonRight'"
Invoke-Expression "$scriptdir\Grant-Rights.ps1 '$SplunkAcct' 'SeSystemProfilePrivilege'"
Invoke-Expression "$scriptdir\Grant-Rights.ps1 '$SplunkAcct' 'SeAssignPrimaryTokenPrivilege'"
#grant WMI ACLs
Invoke-Expression "$scriptdir\Grant-WMIACL.ps1 '$SplunkAcct'"
#add local group access
#NOTE WE ARE CHANGING THE ACCT VARIABLE TO A FORWARDSLASH NOW DO ALL OTHER FUNCTIONS BEFORE THIS
$SplunkAcct=$SplunkAcct.replace('\','/')
foreach ($localGroup in $localGroups)
{
$lclGrp=[ADSI]"WinNT://$env:COMPUTERNAME/$localGroup,group"
$lclGrp.psbase.Invoke("Add",([ADSI]"WinNT://$SplunkAcct").path)
}
</code></pre>
</div>
</div>
<br>
Now for the final piece to the puzzle: <b>SetupSplunkuser.bat</b>:<br>
<br>
This script exists to provide compatibility with 2008 (not R2) and lower.
Native Powershell scripts run correctly on 2008R2 and higher, but a batch
launched script will run from 2003 up with no problem. This will be the
script we'll use to kick everything off.<br>
<br>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">powershell.exe -nologo -noexit -file %~dp0\Setup-SplunkUser.ps1
</code></pre>
<br>
Now that we have our scripts in place, we need to do the hard part. Run
them everywhere we would like in the enterprise.<br>
<br>
<h4>
Running the Scripts on Target Machines</h4>
<div>
<br>
As discussed earlier, there are several ways to accomplish this. The
solution I'll be using is <b>Group Policy</b> by utilizing the <b>Startup
Script</b> functionality in the <b>Computer</b> portion of the <b>policy</b>.
Setting up the group policy is easy, the difficult part is getting the
scripts to run correctly, but we'll worry about that when we get
there. </div>
<div>
</div>
<div>
<h4>
Determine Your Target Machines</h4>
<br>
If you want to configure Splunk access rights across all machines in
your domain you can plan on using the default domain policy GPO to roll
out these changes. If you're reading this article, however, I suspect
you want to be more targeted with your application of changes.<br>
<br>
I recommend using <a href="http://technet.microsoft.com/en-us/library/cc781988%28v=ws.10%29.aspx"
target="_blank">security filtering</a> against the desired groups of
computers. If you don't already have groups you would like to use,
create a security group in Active Directory and add your desired
machines to it. To do this on a large scale, you can use something like
<a href="http://gallery.technet.microsoft.com/Add-Computers-To-A-abc99ab6"
target="_blank">this</a>.<br>
<br>
After you have your target group(s) set up, you need to configure the
group policy you plan on using to execute the script. If this GPO
doesn't yet exist, <a href="http://technet.microsoft.com/en-us/library/cc776678%28v=ws.10%29.aspx"
target="_blank">create it</a> and apply the security filtering listed
above. <a href="http://technet.microsoft.com/en-us/library/cc759123%28v=ws.10%29.aspx"
target="_blank">Edit the GPO</a> using GPMC:<br>
<br>
<ol>
<li>After opening the GPO, navigate to "<i>Computer Configuration</i>-><i>Policies</i>-><i>Windows
Settings</i>-><i>Scripts (Startup/Shutdown)</i></li>
<br>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh58MUnj-lqQWKh45G3leTdHE4CJ-Dsr-_7MWGPL9J04MXYHWD7uqNjslkfjm0cOlx-tNYPQwqzQlimSQhkFQk0tBHVyTlfcBof5_IrjD4hiaam0WnHKYKV2LEz7Cjg7qJLpI2SuMCqjTY/s1600/2013-12-12+21_04_47-Group+Policy+Management.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh58MUnj-lqQWKh45G3leTdHE4CJ-Dsr-_7MWGPL9J04MXYHWD7uqNjslkfjm0cOlx-tNYPQwqzQlimSQhkFQk0tBHVyTlfcBof5_IrjD4hiaam0WnHKYKV2LEz7Cjg7qJLpI2SuMCqjTY/s400/2013-12-12+21_04_47-Group+Policy+Management.png"
border="0" height="285" width="400"></a></div>
<br>
<li>Select "<i>Startup</i>". </li>
<li>Click "<i>Show Files...</i>"; this will open a folder on your <a
href="http://en.wikipedia.org/wiki/File_Replication_Service" target="_blank">sysvol</a> volume
associated with this Group Policy Object. </li>
<br>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPNMU2te8h5jKno4VtlKVGoCDCAykORbEWHS4fSfdhl5BRsqv0a-MXqnKfgDbowxsTgSAOkbUT8jru8J69itGNlFBzXYO4XBM8Vn6DiCcnnxi6Rb0KABg6WRv5K2LU2vwzxtGixCcn9SM/s1600/2013-12-12+21_05_32-Startup+Properties.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPNMU2te8h5jKno4VtlKVGoCDCAykORbEWHS4fSfdhl5BRsqv0a-MXqnKfgDbowxsTgSAOkbUT8jru8J69itGNlFBzXYO4XBM8Vn6DiCcnnxi6Rb0KABg6WRv5K2LU2vwzxtGixCcn9SM/s1600/2013-12-12+21_05_32-Startup+Properties.png"
border="0"></a></div>
<br>
<li>After modifying accordingly (Setup-Splunkuser.ps1) copy all four
scripts into the folder. These will be automatically replicated to
all domain controllers in the domain. </li>
<br>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjM-CFSHV-GebsPyRXBQ77xyiSShqfls5u5UDKKPV5wV84ipQePviAve0i-g_9qWtD-LcZOz1gxji8TH8buKKAF0ngEv6agY7FF6P2WtGpvR_-J-kb97FHKD4Qf1qLwADFEuskpIJa5Hog/s1600/2013-12-12+21_07_16-Group+Policy+Management.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjM-CFSHV-GebsPyRXBQ77xyiSShqfls5u5UDKKPV5wV84ipQePviAve0i-g_9qWtD-LcZOz1gxji8TH8buKKAF0ngEv6agY7FF6P2WtGpvR_-J-kb97FHKD4Qf1qLwADFEuskpIJa5Hog/s400/2013-12-12+21_07_16-Group+Policy+Management.png"
border="0" height="136" width="400"></a></div>
<br>
<li>Back on the "Startup Properties" screen, ensure the "Scripts" tab
is selected (not Powershell Scripts) and click "<i>Add</i>". </li>
<li>Type or browse to SetupSplunkUser.bat. And click "<i>OK</i>"
twice. </li>
</ol>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAyn_kRZ-y1MEJQuW_4e7eCRkasx_0Bs1SVcJ79sf4BecVj8pyHH9hI1d3jSAKO_4_3I1OToP8gFyzVSl71FzkQdqP10FUCV7J3aj77RIULSS-hL-520UxdCDnoanwEt0jS7yU2CYigNg/s1600/2013-12-12+21_32_29-.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAyn_kRZ-y1MEJQuW_4e7eCRkasx_0Bs1SVcJ79sf4BecVj8pyHH9hI1d3jSAKO_4_3I1OToP8gFyzVSl71FzkQdqP10FUCV7J3aj77RIULSS-hL-520UxdCDnoanwEt0jS7yU2CYigNg/s320/2013-12-12+21_32_29-.png"
border="0" height="164" width="320"></a></div>
<br>
<br>
This now is triggered to run on reboot for all impacted machines, but
we're not done yet. We still have to navigate the oddly challenging
world of running Powershell scripts in the Windows environment.<br>
<br>
<h3>
Running Powershell Scripts in a Distributed Environment - The Problem</h3>
<br>
The hurdle using this solution is a larger problem: By default,
Powershell scripts are super scary and won't run in your environment for
two reasons:<br>
<br>
<ul>
<li>By default, running scripts requires those scripts be
signed. </li>
<li>By default, scripts will not run from the "internet", and for some
reason the UNC path to the domain controller that may have just
authenticated you is considered the "internet" by Windows. </li>
</ul>
<div>
</div>
<div>
More <a href="http://blogs.msdn.com/b/powershell/archive/2008/09/30/powershell-s-security-guiding-principles.aspx"
target="_blank">here</a>. There are two ways to solve this
problem. </div>
<div>
<ul>
<li><b>Secure Way/PKI</b>(Recommended): If you have an internal <a
href="http://blog.ittoby.com/2012/04/creating-two-tier-pki-windows-2008r2.html"
target="_blank">PKI</a> you can sign all your scripts AND add
the signer cert(s) to the trusted publisher store on all targeted
machines in Active Directory. </li>
<li><b>Not-So Secure Way/Disable Security</b>: You can run scripts
from your DC by setting the execution policy in Powershell to
"Unrestricted" and adding your domain controller sysvol share to
the trusted sites zone in Internet Explorer configuration. </li>
</ul>
<div>
I will briefly discuss each solution. </div>
</div>
<div>
</div>
<div>
<h4>
Secure Solution - PKI</h4>
<br>
This is the preferred and most secure method. While you could do this
with certificates from a public authority, it is recommended that you
use your own <a href="http://blog.ittoby.com/2012/04/creating-two-tier-pki-windows-2008r2.html"
target="_blank">Active Directory integrated PKI</a> for this. Here
is a very high-level overview of the steps:<br>
<br>
<ol>
<li>Acquire a <a href="http://en.wikipedia.org/wiki/Code_signing" target="_blank">code
signer cert</a> from your PKI. If using a Windows PKI you can
use the "Code Signing" template, but it really should be
customized first. Export or save the public key (.cer) because
we'll need it later. </li>
<li>Sign each script with the following code (substituting
scriptname.ps1 with the location of your script): </li>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">#This assumes you have only one code signing script
#use Get-Childitem cert:\CurrentUser\My -codesigning to see all and change index if necessary
$cert=@(Get-Childitem cert:\CurrentUser\My -codesigning)[0]
#Do this for each script and change scriptname.ps1 to the actual script name
Set-AuthenticodeSignature .\scriptname.ps1 $cert</code></pre>
<li>Copy the scripts into the sysvol share as outlined above. </li>
<li>Edit the GPO you would like to use to distribute the script...
the same one you're using to run the scripts above would be the
perfect match. </li>
<li>Navigate to "<i>Computer Configuration</i>-><i>Policies</i>-><i>Windows
Settings</i>-><i>Security Settings</i>-><i>Public Key
Policies</i>-><i>Trusted Publishers</i>"</li>
<li><i>Right click</i>-><i>Import</i> and select your public
key (.cer) corresponding to your code signing cert. </li>
</ol>
<div>
Close the GPO and you should now be good. Once the certificate
propagates to the target machines they'll run the signed scripts
without issue. </div>
<br>
<h4>
Alternative Solution A - Disable Security for this Script</h4>
<h4>
<div style="font-weight: normal;">
</div>
<div style="font-weight: normal;">
</div>
<ol style="font-weight: normal;">
<li>Edit the <i>SetupSplunkUser.bat</i> script and change
the line to:</li>
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code
style="color: black; word-wrap: normal;">powershell.exe -ExecutionPolicy Bypass -nologo -noexit -file %~dp0\Setup-SplunkUser.ps1
</code></pre>
<li>Upload the file to your sysvol share as described above. </li>
<li>If you have issues running the other Powershell scripts that
are called from the first, you may have to use some trickery:
such as <a href="http://stackoverflow.com/questions/9271681/how-to-run-powershell-script-even-if-set-executionpolicy-is-banned">this</a>.</li>
</ol>
</h4>
<h4>
Alternative Solution B - Disable Security Permanently</h4>
</div>
<ol>
<li>Edit the GPO you would like to use to distribute the script... the
same one you're using to run the scripts would be the perfect
match. </li>
<li>Navigate to "<i>Computer Configuration</i>-><i>Policies</i>-><i>Administrative
Templates</i>-><i>Windows Components</i>-><i>Windows
Powershell</i>"</li>
<li>Edit "<i>Turn on Script Execution</i>" and change it to "<i>Enabled</i>"
and "<i>Allow all scripts</i>". Click OK and close the GPO. </li>
<li>Add your domain sysvol path (ensure you use your domain name, not
your domain controller name) to the trusted sites zone: see <a href="http://community.spiceworks.com/topic/326140-add-trusted-sites-via-gpo-but-still-allow-users-to-add-trusted-sites"
target="_blank">here</a> and <a href="http://forums.techarena.in/active-directory/1134794.htm"
target="_blank">here</a>. OR change the UACAsInternet property;
see <a href="http://www.scritube.com/stiinta/tutorials/windows-en/Internet-Explorer-UrlAction-Se14468711.php"
target="_blank">here</a>. </li>
</ol>
<br>
That should do it! Remote eventlog, WMI, and most other types of remote
collection should now work using Splunk.<br>
<br>
<h3>
Troubleshooting</h3>
<br>
If you have issues getting or determining if the scripts are running
successfully, try the following:<br>
<br>
<ul>
<li>Try executing SetupSplunkUser.bat from an elevated command prompt
on one of the target machines. Note any issues. </li>
<li>To audit running in a test group, you can add a line like "<i>ran
script Setup-SplunkUser.ps1" | Out-File "c:\temp\temp.txt</i>" to
create a log of the script execution on each machine. </li>
<li>Script execution should be logged in the Application Log of each
server. </li>
</ul>
<div>
With this information you should be able to better control the service
rights of Splunk or similar software in a large enterprise environment
and still keep your auditors happy. For an added bonus, you could use
Splunk itself for <a href="http://blogs.splunk.com/2013/07/08/audit-file-access-and-change-in-windows/">security
auditing</a> now that your're done. </div>
<div>
</div>
<div>
<br>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjIJ75jE34IREizYIyKT_nH5aeuU4LFhsCwF9KJoA5IhEy7zm54evUJfIPLqz75oO1efl5QlB7zIHg8r39Ry-t8AoJee9c2SRWlr0XgrwYyR12TrBP2VV4snflcKgb7BgHwu0iIiZjCMwE/s1600/screenshot_ES_security_posture_dashboard.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjIJ75jE34IREizYIyKT_nH5aeuU4LFhsCwF9KJoA5IhEy7zm54evUJfIPLqz75oO1efl5QlB7zIHg8r39Ry-t8AoJee9c2SRWlr0XgrwYyR12TrBP2VV4snflcKgb7BgHwu0iIiZjCMwE/s1600/screenshot_ES_security_posture_dashboard.png"
border="0"></a></div>
</div>
</div>
</div>Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com0tag:blogger.com,1999:blog-5458589696790815336.post-69406261755675805092013-12-08T23:25:00.000-06:002013-12-08T23:25:21.051-06:00Windows 2012 to 2012R2 In-Place Upgrade (Not recommended) Wipes out Network Teams, vSwitches<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjvduG1ScpyR75OxIZwxbjPg06tiz7qZkzIFD5l1v76AonRxJeskK9sRRWKdHkwwnm9XZ349QlTeJ-mZ-73eDjAvCx9ZUGnClTvJdLb5njS3r5N6S5KDhiGWnMUWFCF3B1KHRvMs7AxdNI/s1600/826465813.gif" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="179" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjvduG1ScpyR75OxIZwxbjPg06tiz7qZkzIFD5l1v76AonRxJeskK9sRRWKdHkwwnm9XZ349QlTeJ-mZ-73eDjAvCx9ZUGnClTvJdLb5njS3r5N6S5KDhiGWnMUWFCF3B1KHRvMs7AxdNI/s320/826465813.gif" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Preparing for an in-place upgrade. @ NFL I own the rights to this? :)</td></tr>
</tbody></table>
<br />
<span style="font-size: large;"><b>W</b></span>ord to the wise: if you plan on doing an in-place upgrade from server 2012 to 2012R2 be prepared to re-create all your OS level network teams. If you upgrade a Hyper-V host and your teams were used for switches, be ready to re-create those as well.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgqKYEmNBViksFkw-Z-Z_JalIlglm_tKIMQ_iOrZCRI2jFJhthm7byUfh02csZEpYxKE8JG6s0lN2vLtGizARt6_bHxrI2qV2LMkw5iqrZM8rHakThLLddYHuYW3I2Wg0Hd3Vkk0sOXcb0/s1600/2013-12-08+23_16_09-RD+Tabs+64.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgqKYEmNBViksFkw-Z-Z_JalIlglm_tKIMQ_iOrZCRI2jFJhthm7byUfh02csZEpYxKE8JG6s0lN2vLtGizARt6_bHxrI2qV2LMkw5iqrZM8rHakThLLddYHuYW3I2Wg0Hd3Vkk0sOXcb0/s1600/2013-12-08+23_16_09-RD+Tabs+64.png" /></a></div>
<br />I don't recommend doing an in-place upgrade in the first place for reasons like this, but I figured I'd give it a shot on one of the hosts in my lab. Every OS level network team was wiped out, but easily recreated. <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjEIm8jzs0bfNYEptqIoWMwIbVeBXQO1GEENJilJ5WwYlUzTNV65ib0RjIPOKmtapN4Igy-cvSrO4ntYgSW9hBEalC3dfT7QfsdE6SF2L-vNq0EF7ObgbVAydIGcdwaLpbxGqwmh-MwtEQ/s1600/2013-12-08+23_15_29-RD+Tabs+64.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjEIm8jzs0bfNYEptqIoWMwIbVeBXQO1GEENJilJ5WwYlUzTNV65ib0RjIPOKmtapN4Igy-cvSrO4ntYgSW9hBEalC3dfT7QfsdE6SF2L-vNq0EF7ObgbVAydIGcdwaLpbxGqwmh-MwtEQ/s1600/2013-12-08+23_15_29-RD+Tabs+64.png" /></a></div>
<br />
Other than this I haven't found any other in-place upgrade issues... yet, and that's why I'm always hesitant to recommend the practice. It's rare you do find what the issue are until the become a problem. <br />
<br />
Any issues you've encountered? Leave 'em below!Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com1tag:blogger.com,1999:blog-5458589696790815336.post-8843131207840486272013-10-28T20:59:00.001-05:002013-11-09T00:05:15.268-06:00Windows 8.1 Exhibits Gaming Performance DeficitMicrosoft just released <a href="http://technet.microsoft.com/en-us/windows/dn140266.aspx" target="_blank">Windows 8.1</a>, and with that I subjected the latest version of the operating system to some gaming performance testing. The results illustrate a small decline in the performance of Windows when moving from 8.0 to 8.1. <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_v5ZYsYq6MXT8pW6x95ljCVJ1OO-JJ1gZDgNZVKhhitcGhd2cq97tPqxjVOBdM-xmynwI2YwtUasYraGN1TXm9ncqoUGG41QhoHv7Xxx2sL-6k2x3QI-cYC-34GWw1hHtr9ay6J7nvdc/s1600/elizabeth_win3.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_v5ZYsYq6MXT8pW6x95ljCVJ1OO-JJ1gZDgNZVKhhitcGhd2cq97tPqxjVOBdM-xmynwI2YwtUasYraGN1TXm9ncqoUGG41QhoHv7Xxx2sL-6k2x3QI-cYC-34GWw1hHtr9ay6J7nvdc/s1600/elizabeth_win3.jpg" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<h3>
System as Tested</h3>
<br />
<ul>
<li>Asus z87-Pro Motherboard</li>
<li>Intel i7-4770k @ 4.7Ghz/4.4Ghz Uncore</li>
<li>Corsair Dominator Pro 16GB (8x2) @ 2133, 10-11-11-27</li>
<li>Nvidia 670 2GB @ 995/3204</li>
<ul>
<li>ForceWare 320.49 on both 8.0 and 8.1</li>
</ul>
<li>Sound Blaster ZxR Rendering in 5.1 </li>
<li>"Upgrade Install" of 8.1 from 8.0. 8.0 was installed fresh. </li>
<li>All drivers the same </li>
<li>App/Service footprint nearly identical </li>
</ul>
<h3>
Testing Methodology</h3>
<br />
For the last 13 years or so, I've used a benchmarking script to test
gaming performance. This script performs automated testing and logs
results from each game to a central file. The script runs each game <i>n</i>
(i.e. as many as I would like) times and then generates a
median (rather than average). I have added games to this
periodically since its inception.<br />
<br />
I say all this
because you'll notice there are a few old games listed. I have found
including these results useful to judge CPU/Memory performance and
identify bottlenecks specific to different APIs. <br />
<br />
For these tests, each game was run at least three times (most more) for each set of settings. There are 4 tested settings that apply to most games: 4xAA/16xAnisotropic filtering, 0x/0x, and all combinations thereof. Normally these results would be split, but for the purposes of this platform comparison they have been combined. Bioshock Infinite does not support any AA, so that is not affected by the AA setting. All tests are performed at 1600x1200 which maintains compatibility with the older titles and while being a less than ten percent deviation from the 1080p effective pixel count. <br />
<br />
For the first set of results there were over 250 test runs. These runs have been compared in total between Windows 8.0 and Windows 8.1 on a per game basis to generate a % performance change.<br />
<br />
Games tested as follows: (in order of age, oldest->newest)<br />
<ul>
<li>Quake 3 (1999, OpenGL)</li>
<li>Comanche 4 (2001, D3D8)</li>
<li>Doom 3 (2004 OpenGL) </li>
<li>Serious Sam 2 (2005 OpenGL)</li>
<li>Company of Heroes (2006 DX10)</li>
<li>Crysis (2007 DX10 Very High Settings)</li>
<li>Dirt 2 Ultra (2009 DX11Ultra Settings)</li>
<li>Alien vs. Predator (2010 DX11)</li>
<li>Bioshock Infinite Ultra (2013 DX11 Ultra Settings)</li>
<li>Unigine Valley (2013 DX11)</li>
</ul>
<h3>
Results </h3>
<br />
As illustrated in the bar chart below, the results indicate that Windows 8.1 lags in performance when compared to Windows 8.0 original release. <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEja_h7JiM31fat17WaQM_BAr4FNH19dDCCXBETvNT6t6kqENNJnamLgBvsmNgAg4FLOvGoBEHNsxzyknuCz7jjuz1WeKDhRNuAQ6qtCdl4kORh1ZlPARLWxUC3UZpL69xn1c90dG1xhN40/s1600/2013-10-27+22_47_08-Win8_1_Perf.xlsx+-+Excel.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="464" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEja_h7JiM31fat17WaQM_BAr4FNH19dDCCXBETvNT6t6kqENNJnamLgBvsmNgAg4FLOvGoBEHNsxzyknuCz7jjuz1WeKDhRNuAQ6qtCdl4kORh1ZlPARLWxUC3UZpL69xn1c90dG1xhN40/s640/2013-10-27+22_47_08-Win8_1_Perf.xlsx+-+Excel.png" width="640" /></a></div>
<br />
<br />
<table border="0" cellpadding="0" cellspacing="0" style="width: 744px;"><colgroup><col style="mso-width-alt: 5485; mso-width-source: userset; width: 113pt;" width="150"></col>
<col style="mso-width-alt: 2011; mso-width-source: userset; width: 41pt;" width="55"></col>
<col style="mso-width-alt: 2048; mso-width-source: userset; width: 42pt;" width="56"></col>
<col style="mso-width-alt: 1243; mso-width-source: userset; width: 26pt;" width="34"></col>
<col style="mso-width-alt: 2121; mso-width-source: userset; width: 44pt;" width="58"></col>
<col style="mso-width-alt: 2157; mso-width-source: userset; width: 44pt;" width="59"></col>
<col style="mso-width-alt: 1389; mso-width-source: userset; width: 29pt;" width="38"></col>
<col span="2" style="mso-width-alt: 2048; mso-width-source: userset; width: 42pt;" width="56"></col>
<col style="mso-width-alt: 1280; mso-width-source: userset; width: 26pt;" width="35"></col>
<col style="mso-width-alt: 2084; mso-width-source: userset; width: 43pt;" width="57"></col>
<col style="mso-width-alt: 2011; mso-width-source: userset; width: 41pt;" width="55"></col>
<col style="mso-width-alt: 1280; mso-width-source: userset; width: 26pt;" width="35"></col>
</colgroup><tbody>
<tr height="35" style="height: 26.25pt; mso-height-source: userset;">
<td class="xl65" height="35" style="border-color: rgb(112, 173, 71); border-style: solid; border-width: 0.5pt 0.5pt 1pt; color: black; font-family: Calibri; font-size: 11pt; font-weight: 700; height: 26.25pt; text-align: left; text-decoration: none; width: 113pt;" width="150">Game; All Scores Median of Multiple Runs</td>
<td class="xl66" style="border-color: rgb(112, 173, 71); border-style: solid; border-width: 0.5pt 0.5pt 1pt; color: black; font-family: Calibri; font-size: 10pt; font-weight: 700; text-align: right; text-decoration: none; width: 41pt;" width="55">8.0 4xAA
16xAni</td>
<td class="xl66" style="border-color: rgb(112, 173, 71); border-style: solid; border-width: 0.5pt 0.5pt 1pt; color: black; font-family: Calibri; font-size: 10pt; font-weight: 700; text-align: right; text-decoration: none; width: 42pt;" width="56">8.1 4xAA
16xAni</td>
<td class="xl66" style="border-color: rgb(112, 173, 71); border-style: solid; border-width: 0.5pt 0.5pt 1pt; color: black; font-family: Calibri; font-size: 10pt; font-weight: 700; text-align: right; text-decoration: none; width: 26pt;" width="34">% Diff</td>
<td class="xl66" style="border-color: rgb(112, 173, 71); border-style: solid; border-width: 0.5pt 0.5pt 1pt; color: black; font-family: Calibri; font-size: 10pt; font-weight: 700; text-align: right; text-decoration: none; width: 44pt;" width="58">8.0 0xAA
0xAni</td>
<td class="xl66" style="border-color: rgb(112, 173, 71); border-style: solid; border-width: 0.5pt 0.5pt 1pt; color: black; font-family: Calibri; font-size: 10pt; font-weight: 700; text-align: right; text-decoration: none; width: 44pt;" width="59">8.1 0xAA
0xAni</td>
<td class="xl66" style="border-color: rgb(112, 173, 71); border-style: solid; border-width: 0.5pt 0.5pt 1pt; color: black; font-family: Calibri; font-size: 10pt; font-weight: 700; text-align: right; text-decoration: none; width: 29pt;" width="38">% Diff</td>
<td class="xl66" style="border-color: rgb(112, 173, 71); border-style: solid; border-width: 0.5pt 0.5pt 1pt; color: black; font-family: Calibri; font-size: 10pt; font-weight: 700; text-align: right; text-decoration: none; width: 42pt;" width="56">8.0 4xAA
0xAni</td>
<td class="xl66" style="border-color: rgb(112, 173, 71); border-style: solid; border-width: 0.5pt 0.5pt 1pt; color: black; font-family: Calibri; font-size: 10pt; font-weight: 700; text-align: right; text-decoration: none; width: 42pt;" width="56">8.1 4xAA
0xAni</td>
<td class="xl66" style="border-color: rgb(112, 173, 71); border-style: solid; border-width: 0.5pt 0.5pt 1pt; color: black; font-family: Calibri; font-size: 10pt; font-weight: 700; text-align: right; text-decoration: none; width: 26pt;" width="35">% Diff</td>
<td class="xl66" style="border-color: rgb(112, 173, 71); border-style: solid; border-width: 0.5pt 0.5pt 1pt; color: black; font-family: Calibri; font-size: 10pt; font-weight: 700; text-align: right; text-decoration: none; width: 43pt;" width="57">8.0 0xAA
16xAni</td>
<td class="xl66" style="border-color: rgb(112, 173, 71); border-style: solid; border-width: 0.5pt 0.5pt 1pt; color: black; font-family: Calibri; font-size: 10pt; font-weight: 700; text-align: right; text-decoration: none; width: 41pt;" width="55">8.1 0xAA
16xAni</td>
<td class="xl66" style="border-color: rgb(112, 173, 71); border-style: solid; border-width: 0.5pt 0.5pt 1pt; color: black; font-family: Calibri; font-size: 10pt; font-weight: 700; text-align: right; text-decoration: none; width: 26pt;" width="35">% Diff</td>
</tr>
<tr height="20" style="height: 15.0pt;">
<td class="xl67" height="20" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; height: 15.0pt; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">Quake3</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">904.4</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">904.73</td>
<td align="right" class="xl69" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">0.04</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">912.37</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">910.97</td>
<td align="right" class="xl69" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">-0.15</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">903.43</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">903.8</td>
<td align="right" class="xl69" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">0.04</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">911.1</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">908.6</td>
<td align="right" class="xl69" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">-0.27</td>
</tr>
<tr height="20" style="height: 15.0pt;">
<td class="xl67" height="20" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; height: 15.0pt; text-decoration: none; text-line-through: none; text-underline-style: none;">Comanche4</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">198.47</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">198.5</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">0.02</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">198.53</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">198.97</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">0.22</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">198.33</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">199.87</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">0.78</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">199.5</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">198.93</td>
<td align="right" class="xl65" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">-0.29</td>
</tr>
<tr height="20" style="height: 15.0pt;">
<td class="xl67" height="20" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; height: 15.0pt; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">Serious Sam 2</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">486.7</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">469.6</td>
<td align="right" class="xl69" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">-3.51</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">549.3</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">536.2</td>
<td align="right" class="xl69" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">-2.38</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">521.5</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">507.5</td>
<td align="right" class="xl69" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">-2.68</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">547.9</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">538.7</td>
<td align="right" class="xl69" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">-1.68</td>
</tr>
<tr height="20" style="height: 15.0pt;">
<td class="xl67" height="20" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; height: 15.0pt; text-decoration: none; text-line-through: none; text-underline-style: none;">Doom3</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">338</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">334.7</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">-0.98</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">473.2</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">472.1</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">-0.23</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">339.8</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">337.7</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">-0.62</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">468.5</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">472.7</td>
<td align="right" class="xl65" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">0.90</td>
</tr>
<tr height="20" style="height: 15.0pt;">
<td class="xl67" height="20" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; height: 15.0pt; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">Company of Heroes_DX10</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">261.55</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">246.5</td>
<td align="right" class="xl69" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">-5.75</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">290.25</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">268.05</td>
<td align="right" class="xl69" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">-7.65</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">267.3</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">248.7</td>
<td align="right" class="xl69" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">-6.96</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">285.15</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">264.05</td>
<td align="right" class="xl69" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">-7.40</td>
</tr>
<tr height="20" style="height: 15.0pt;">
<td class="xl67" height="20" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; height: 15.0pt; text-decoration: none; text-line-through: none; text-underline-style: none;">Crysis_Island_DX10</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">63.06</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">62.12</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">-1.49</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">81.16</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">79.59</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">-1.93</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">66.03</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">64.73</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">-1.97</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">77.82</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">76.32</td>
<td align="right" class="xl65" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">-1.93</td>
</tr>
<tr height="20" style="height: 15.0pt;">
<td class="xl67" height="20" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; height: 15.0pt; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">Dirt 2 DX11_Ultra</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">129.46</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">127.62</td>
<td align="right" class="xl69" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">-1.42</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">167.47</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">165.05</td>
<td align="right" class="xl69" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">-1.45</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">142.65</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">139.97</td>
<td align="right" class="xl69" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">-1.88</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">155.99</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">154.13</td>
<td align="right" class="xl69" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">-1.19</td>
</tr>
<tr height="20" style="height: 15.0pt;">
<td class="xl67" height="20" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; height: 15.0pt; text-decoration: none; text-line-through: none; text-underline-style: none;">Alien vs. Predator DX11</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">63</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">62.2</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">-1.27</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">110.3</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">108.6</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">-1.54</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">65.4</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">64.6</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">-1.22</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">104.6</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">103.3</td>
<td align="right" class="xl65" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">-1.24</td>
</tr>
<tr height="20" style="height: 15.0pt;">
<td class="xl67" height="20" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; height: 15.0pt; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">Bioshock Infinite Ultra_DX11</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">100.4</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">99.29</td>
<td align="right" class="xl69" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">-1.11</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">99.73</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">99.11</td>
<td align="right" class="xl69" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">-0.62</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">99.53</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">97.93</td>
<td align="right" class="xl69" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">-1.61</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">100.2</td>
<td align="right" class="xl68" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">98.48</td>
<td align="right" class="xl69" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">-1.72</td>
</tr>
<tr height="20" style="height: 15.0pt;">
<td class="xl67" height="20" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; height: 15.0pt; text-decoration: none; text-line-through: none; text-underline-style: none;">Unigine_Valley</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">58.63</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">58.28</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">-0.60</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">79.75</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">79.67</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">-0.10</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">60.13</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">59.08</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">-1.75</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">78.25</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">77.6</td>
<td align="right" class="xl65" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">-0.83</td>
</tr>
</tbody></table>
<br />
<br />
While most percentages are relatively low, the trend appears consistent and the high sample size lends to the reliability of the data. I find this a bit troubling since there are no obvious reasons performance degradation should be introduced when looking at the 8.1 changes. Note there is still work to do; I'd like to get additional opinions including ATI results. <br />
<br />
Now let's take a look at Futuremark 3dMark Advanced Edition. These tests were run through a couple times each and averaged. They were performed on default settings. (1080p/720p/720p)<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjamDuVmY9uHdqrZJa9U2zRsr3POoncQfCxbj57K7rRP1kd6z620JE6mZV_QB7t3FFkox6Cw4R1d6C6YqgDGsJuMsG_T7TDebEVD8-6PeCXWGIH3dy5JSeebCvuaS0qhKmf1KViuJHMj4Y/s1600/2013-10-22+22_33_32-Win8_1_Perf.xlsx+-+Excel.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="420" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjamDuVmY9uHdqrZJa9U2zRsr3POoncQfCxbj57K7rRP1kd6z620JE6mZV_QB7t3FFkox6Cw4R1d6C6YqgDGsJuMsG_T7TDebEVD8-6PeCXWGIH3dy5JSeebCvuaS0qhKmf1KViuJHMj4Y/s640/2013-10-22+22_33_32-Win8_1_Perf.xlsx+-+Excel.png" width="640" /></a></div>
<br />
<br />
<br />
<br />
<table align="center" border="0" cellpadding="0" cellspacing="0" style="width: 341px;"><colgroup><col style="mso-width-alt: 5485; mso-width-source: userset; width: 113pt;" width="150"></col>
<col span="2" style="mso-width-alt: 2413; mso-width-source: userset; width: 50pt;" width="66"></col>
<col style="mso-width-alt: 2157; mso-width-source: userset; width: 44pt;" width="59"></col>
</colgroup><tbody>
<tr height="20" style="height: 15.0pt;">
<td class="xl66" height="20" style="border-bottom: 1.0pt solid #70AD47; border-left: .5pt solid #70AD47; border-right: .5pt solid #70AD47; border-top: .5pt solid #70AD47; color: windowtext; font-family: Calibri; font-size: 11.0pt; font-weight: 700; height: 15.0pt; text-decoration: none; text-line-through: none; text-underline-style: none; width: 113pt;" width="150">3DMark Test</td>
<td style="border-color: rgb(112, 173, 71); border-style: solid; border-width: 0.5pt 0.5pt 1pt; color: black; font-family: Calibri; font-size: 11pt; font-weight: 700; text-align: right; text-decoration: none; width: 50pt;" width="66">win 8.0</td>
<td style="border-color: rgb(112, 173, 71); border-style: solid; border-width: 0.5pt 0.5pt 1pt; color: black; font-family: Calibri; font-size: 11pt; font-weight: 700; text-align: right; text-decoration: none; width: 50pt;" width="66">win 8.1</td>
<td style="border-color: rgb(112, 173, 71); border-style: solid; border-width: 0.5pt 0.5pt 1pt; color: black; font-family: Calibri; font-size: 11pt; font-weight: 700; text-align: right; text-decoration: none; width: 44pt;" width="59">% Diff</td>
</tr>
<tr height="20" style="height: 15.0pt;">
<td class="xl66" height="20" style="background: #E2EFDA; border: .5pt solid #70AD47; color: windowtext; font-family: Calibri; font-size: 11.0pt; font-weight: 400; height: 15.0pt; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">Fire Strike</td>
<td align="right" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">6560</td>
<td align="right" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">6502</td>
<td align="right" class="xl65" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">-0.88</td>
</tr>
<tr height="20" style="height: 15.0pt;">
<td class="xl67" height="20" style="background: white; border: .5pt solid #70AD47; color: windowtext; font-family: Calibri; font-size: 11.0pt; font-weight: 400; height: 15.0pt; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">Cloud Gate</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">25138</td>
<td align="right" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">24331</td>
<td align="right" class="xl65" style="border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; text-decoration: none; text-line-through: none; text-underline-style: none;">-3.21</td>
</tr>
<tr height="20" style="height: 15.0pt;">
<td class="xl66" height="20" style="background: #E2EFDA; border: .5pt solid #70AD47; color: windowtext; font-family: Calibri; font-size: 11.0pt; font-weight: 400; height: 15.0pt; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">Ice Storm</td>
<td align="right" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">179351</td>
<td align="right" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">176476</td>
<td align="right" class="xl65" style="background: #E2EFDA; border: .5pt solid #70AD47; color: black; font-family: Calibri; font-size: 11.0pt; font-weight: 400; mso-pattern: #E2EFDA none; text-decoration: none; text-line-through: none; text-underline-style: none;">-1.60</td>
</tr>
</tbody></table>
<br />
<br />
Again, consistent downturns in performance.<br />
<br />
This is where I was going to publish this article, but as my finger hovered over the "Publish" button I had one lingering concern: What if I'm wrong? What if I missed a setting or something in the upgrade that might have this impact?<br />
<br />
<h3>
So Then We Did it Over Again</h3>
<br />
I wouldn't feel right publishing this article without doing a clean install and absolute parity of all operating systems involved. For the sake of being complete, I included a Windows 7 install as well. What I found stunned me... 8.1 isn't the start of a downturn, it's a continuation. On a smaller subset of tests I performed the same amount of runs. (average of 14 runs per title) The operating system/drivers were as follows:<br />
<br />
<ul>
<li>Windows 7 SP1 fully patched, clean install</li>
<li>Windows 8 fully patched, clean install</li>
<li>Windows 8.1 fully patched, clean install</li>
<li>Nvidia driver version 327.23 on all platforms</li>
</ul>
Here are the results using Windows 7 as baseline:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiY0hT1gBWWix0qO22k6RiHzHQiREjWLGdP3EAZNaJ8XR1ddIMkQRd7B3qH7kwGckS6sLTbxc4Oe6NrWO4r8rCW_wkCPUZyA1KhgnaCW_cFaO1GjJKucb0FNZJsZ7pVZhFqmOhQvfjKuGg/s1600/2013-10-25+09_47_53-Day2Tests.xlsx+-+Excel.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="402" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiY0hT1gBWWix0qO22k6RiHzHQiREjWLGdP3EAZNaJ8XR1ddIMkQRd7B3qH7kwGckS6sLTbxc4Oe6NrWO4r8rCW_wkCPUZyA1KhgnaCW_cFaO1GjJKucb0FNZJsZ7pVZhFqmOhQvfjKuGg/s640/2013-10-25+09_47_53-Day2Tests.xlsx+-+Excel.png" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
As you can see, Windows 8.1 lags behind Windows 8 and Windows 8 lags behind Windows 7. This is a troubling trend. Here's 3dMark (at least 3 runs per test):<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgpO92FEAH2Co70DgSQhjsSa9TMjAchmlxPhgAv3vmRqsxitZIrXTJ7Pzr4Qq1BiUhopBrhdLlnduavS8gR2A6PWvT2lrB4D4BGa2PrXsGWKjiTjip_jNbOiUeD1WdWpZvDU4asYa-iMHk/s1600/2013-10-25+09_48_29-Day2Tests.xlsx+-+Excel.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="382" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgpO92FEAH2Co70DgSQhjsSa9TMjAchmlxPhgAv3vmRqsxitZIrXTJ7Pzr4Qq1BiUhopBrhdLlnduavS8gR2A6PWvT2lrB4D4BGa2PrXsGWKjiTjip_jNbOiUeD1WdWpZvDU4asYa-iMHk/s640/2013-10-25+09_48_29-Day2Tests.xlsx+-+Excel.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
The "Fire Strike" test gives us the only real hope of progress in Windows 8, but Windows 8.1 gives that gain right back. Overall, more troubling results. <br />
<br />
This, coupled with the lingering <a href="http://www.reddit.com/r/windows/comments/1oor43/windows_81_warning_for_gamers_issues_with/" target="_blank">mouse</a> and <a href="http://steamcommunity.com/discussions/forum/1/864975399591571685/" target="_blank">input issues</a> make for a bit of a shaky upgrade to 8.1 from a gaming perspective. As I outlined in my <a href="http://blog.ittoby.com/2013/10/the-gathering-storm-in-pc-gaming.html" target="_blank">previous article</a>, I believe it's time for Microsoft to step up their game (hah!) in light of new market pressures. This trend is unfortunate since there are a lot of minor changes in 8.1 that are a step in the right direction. (UI, etc.) Let's hope they iron out this performance issue quickly.<br />
<br />
Update (11/4/2013): It looks as if Microsoft has <a href="http://answers.microsoft.com/en-us/windows/forum/windows8_1-gaming/mouse-lag-in-video-games-since-upgrading-to/85c28c06-b13c-43be-8282-88528c42e400" target="_blank">acknowledged the mouse issue</a> which I expect means a fix would be coming. Let's hope performance and other issues are addressed as well.<br />
<br />
Update (11/9/2013): Microsoft has <a href="http://support.microsoft.com/kb/2908279" target="_blank">released a patch</a> to deal with mouse issues. Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com4tag:blogger.com,1999:blog-5458589696790815336.post-71425344071877841852013-10-03T23:46:00.000-05:002013-10-19T14:44:09.563-05:00The Gathering Storm in the PC Gaming IndustryAs a result of recent events in the computing world, I think we are entering into a very special time in PC gaming. This industry has been close to my heart since I loaded up <a href="http://en.wikipedia.org/wiki/Police_Quest_II:_The_Vengeance" target="_blank">Police Quest 2</a> on my first <a href="http://en.wikipedia.org/wiki/Turbo_button" target="_blank">turbo-button</a> equipped and ultra-beige <a href="http://en.wikipedia.org/wiki/IBM_PC_compatible" target="_blank">PC clone</a>. Since then, there have been many evolutions of the gaming platform but no revolutions. That may finally change in the near future. Despite the <a href="http://www.bbc.co.uk/news/business-23251285" target="_blank">decline in PC sales</a>, the <a href="http://www.forbes.com/sites/jasonevangelho/2013/03/28/despite-mobile-and-console-competition-pc-gaming-industry-rakes-in-20-billion-during-2012/" target="_blank">PC gaming industry continues to grow</a>, and the PC as we know it is morphing into device types that don't fit the molds to which we're accustomed. <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhA_HXUhpaigJPJ1v_uvpXBV_3UKmq_g4ry8tr4NLKFJlHl5kw9WtH-_F8_ZrG_2jPiddlu0ttdO9pVYgfkydcN6RcJPlJZcmWoed6PHUQOlHBlLPlNvwh3ciz1-SoIKGk78lkB_gaBnfs/s1600/SteamBox_010913_coverwikiheader.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="256" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhA_HXUhpaigJPJ1v_uvpXBV_3UKmq_g4ry8tr4NLKFJlHl5kw9WtH-_F8_ZrG_2jPiddlu0ttdO9pVYgfkydcN6RcJPlJZcmWoed6PHUQOlHBlLPlNvwh3ciz1-SoIKGk78lkB_gaBnfs/s640/SteamBox_010913_coverwikiheader.jpg" width="640" /></a></div>
<br />
<br />
With this article I aim to illustrate why we're about to go through a particularly intriguing time in gaming by covering how we got here, where we are now, what has recently changed, and finally why the future is so exciting.<br />
<br />
With that said, let's start with the foundation of the modern gaming platform; we'll cover a bit of history and the pace of innovation. <br />
<br />
<h3>
Hardware</h3>
<span style="font-size: x-small;"><span style="font-family: Verdana,sans-serif;"><i>My new PC Gets 35 MPG. </i></span></span><br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<span style="font-size: x-small;"><span style="font-family: Verdana,sans-serif;"><i></i></span></span>
<br />
<ul>
<li>CPU Performance has increased steadily and closely followed <a href="http://en.wikipedia.org/wiki/Moore%27s_law" target="_blank">Moore's law</a> since the adoption of the Intel <a href="http://en.wikipedia.org/wiki/Intel_8080" target="_blank">8080</a> in the late 1970s. Intel has largely dominated this space from then until now save a few year interruption in the mid 2000s when AMD introduced the <a href="http://en.wikipedia.org/wiki/AMD_K8" target="_blank">K8</a>. </li>
<li>Gaming relevant display adapters started with<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWVWsDYIOE29tjbIXdORiS8mXor-zWWT7uyaiNL0u-55ZlgPmAaOPMTybiUcs4a4MPEV-wYIhDzULBDI04vpXwoUU4u7dWxjytaN6GmrTSYcwNJjMXmzo6G3rjai13IZ8FHdEMgWm5rKU/s1600/537_orchid_righteous_3d_3dfx_voodoo1_top_hq.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="123" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWVWsDYIOE29tjbIXdORiS8mXor-zWWT7uyaiNL0u-55ZlgPmAaOPMTybiUcs4a4MPEV-wYIhDzULBDI04vpXwoUU4u7dWxjytaN6GmrTSYcwNJjMXmzo6G3rjai13IZ8FHdEMgWm5rKU/s200/537_orchid_righteous_3d_3dfx_voodoo1_top_hq.jpg" width="200" /></a> "<a href="http://en.wikipedia.org/wiki/Diamond_stealth" target="_blank">Windows Accelerator</a>" boards (Late 1980s) and since the introduction of <a href="http://vintage3d.org/verite1.php" target="_blank">the</a> <a href="http://en.wikipedia.org/wiki/S3_ViRGE" target="_blank">first</a> <a href="http://en.wikipedia.org/wiki/3dfx_Interactive" target="_blank">3d</a> <a href="http://en.wikipedia.org/wiki/Riva_128" target="_blank">accelerators </a>(Mid 1990s), the progress has been steady with minor interruptions by things like programmable shaders. </li>
<li>Audio progression was initially rapid and has stagnated significantly in the last ten years. Things got kicked off in 1988 with the <a href="http://www.youtube.com/watch?v=qPS4udeuEfc" target="_blank">Roland LAPC-I</a>, then the <a href="http://en.wikipedia.org/wiki/Ad_Lib,_Inc." target="_blank">Adlib</a> and then on to the <a href="http://en.wikipedia.org/wiki/Sound_Blaster#First_generation_Sound_Blasters.2C_8-bit_ISA_.26_MCA_cards" target="_blank">Creative Labs cards</a> which is where we remain today. We nearly had a major evolution in gaming sound in the late 90s with the introduction of <a href="http://en.wikipedia.org/wiki/A3D" target="_blank">A3d</a> by Aureal, but unfortunately they were forced out of business by the legal maneuvering of Creative. </li>
</ul>
<h3>
Operating Systems</h3>
<i><span style="font-size: x-small;"><span style="font-family: Verdana,sans-serif;">Everything Old is New Again.</span></span></i><br />
<i><span style="font-size: x-small;"><span style="font-family: Verdana,sans-serif;"><br /></span></span></i>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi18YbD7HXMUzXpGfgRDunRkU3l0Jb44mWra4x0NGzRO0RL0vlJvgEpEEKen5r__shu5f6rc_3ovncug-Ss9etE3iBvprXwsfo5KQ7RQh5T8v8DINs61mUi6iW-xV8b_zItxFjc927AQC8/s1600/PCGamer-Aug1995-750.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi18YbD7HXMUzXpGfgRDunRkU3l0Jb44mWra4x0NGzRO0RL0vlJvgEpEEKen5r__shu5f6rc_3ovncug-Ss9etE3iBvprXwsfo5KQ7RQh5T8v8DINs61mUi6iW-xV8b_zItxFjc927AQC8/s320/PCGamer-Aug1995-750.jpg" width="236" /></a></div>
<i><span style="font-size: x-small;"><span style="font-family: Verdana,sans-serif;"><br /></span></span></i>
<br />
For the popular gaming platform this one is easy. After the <a href="http://en.wikipedia.org/wiki/Commodore_64" target="_blank">C64</a> and other early platforms faded away it's been Microsoft, Microsoft, Microsoft. DOS to Windows. DOS was new then it was old. Windows was new then it was old. NT was new now it's aging, and the commitment to the OS as a gaming platform has waned for years.<br />
<br />
<h3>
Input and Output Devices</h3>
<i><span style="font-size: x-small;"><span style="font-family: Verdana,sans-serif;">Because the <a href="http://www.youtube.com/watch?v=dZ80bIUEMyQ" target="_blank">IBM Model M</a> Still goes for Sixty Bucks on Ebay.</span></span></i><br />
<br />
Little has changed with the mouse and keyboard in the last 30 years. As with everything else, performance and ability to customize has increased bit by bit. At least most gamepad supporting games these days have standardized on the Xbox360 controller layout rather than forcing you to map buttons to your gamepad du jour. <br />
<br />
The monitor hasn't progressed much since <a href="http://en.wikipedia.org/wiki/Video_Graphics_Array" target="_blank">VGA</a>
was introduced in 1987. Slow and steady improvements, with a dip in
performance when LCDs became the popular solution. As it pertains to
pure gaming performance (input lag/pixel persistence/response time),
<a href="http://wecravegamestoo.com/forums/monitor-reviews-discussion/13215-asus-vg248qe-review-144hz-3d-vision-2-lightboost-gaming-monitor.html" target="_blank">LCDs are just now catching up to CRTs</a>. <br />
<br />
<h3>
</h3>
<h3>
Evolution Summary</h3>
<i><span style="font-size: x-small;"><span style="font-family: Verdana,sans-serif;">Putting it together. </span></span></i><br />
<br />
As you can see, not much has really changed given we're talking about nearly 30 years. We're still playing similar games on similar platforms using similar interfaces. That's not a complaint mind you, but relative to the change we have seen in other sectors in recent years this space seems long in the tooth. Now let's shift to the present...<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDvkhFJHYz6o1SPwqhLWDvBdhsYHDQfPVXC9JscKD7TLM7Ob9DTvcr5-P78Lolz5sV8ZehTWRffCUF49bJyeKG9HfIiDcTMiqhvJcBggCDfQEtoDjiyvm965_bTinVpMBz7EcAY443YTw/s1600/community_image_1380364383.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDvkhFJHYz6o1SPwqhLWDvBdhsYHDQfPVXC9JscKD7TLM7Ob9DTvcr5-P78Lolz5sV8ZehTWRffCUF49bJyeKG9HfIiDcTMiqhvJcBggCDfQEtoDjiyvm965_bTinVpMBz7EcAY443YTw/s400/community_image_1380364383.png" width="400" /></a></div>
<br />
<br />
<h3>
Setting the Stage for Change </h3>
<i><span style="font-size: x-small;"><span style="font-family: Verdana,sans-serif;">It takes a nation of <strike>millions</strike> a few companies to hold us back. </span></span></i><br />
<br />
First, a few facts: Depending on who/where/how you ask, the <i>average</i> gamer ranges from <a href="http://www.theesa.com/facts/" target="_blank">30</a> to <a href="http://arstechnica.com/gaming/2012/07/how-gamings-demographics-reverted-back-to-2005/" target="_blank">37</a> years old. Nearly half are women, and over half of Americans play video games. <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_EybIGVu_0_YWnkf7kI4pDI9ydBOENsoWc5xCah0-cDMSrQOld3y5I7dpcvcpsjwtA9YVqkhLelItrNPnm3qZUq8MmYih6zjjh67Lgw0YhvDPe4iESdBiGISacDen5KUIg0lhqZy-3yo/s1600/193015-half-life-half-life.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_EybIGVu_0_YWnkf7kI4pDI9ydBOENsoWc5xCah0-cDMSrQOld3y5I7dpcvcpsjwtA9YVqkhLelItrNPnm3qZUq8MmYih6zjjh67Lgw0YhvDPe4iESdBiGISacDen5KUIg0lhqZy-3yo/s320/193015-half-life-half-life.jpg" width="320" /></a></div>
<br />
<br />
The gaming industry is an odd thing when you look at it as a whole; while we take the status quo for granted it really has not fully evolved. My generation (<a href="http://en.wikipedia.org/wiki/Generation_X" target="_blank">Gen X</a>) is the first generation that has grown up with this entertainment medium, and while we've aged & settled down the market has not. There still exists a perception among the baby boomer generation that games are "kids stuff", and with boomers still in control of many large companies it's no surprise that these issues exist. A single viewing of the "<a href="http://en.wikipedia.org/wiki/Spike_Video_Game_Awards" target="_blank">Spike Video Game Awards</a>" is all you need to illustrate the damage of this ill effect. This is improving with time, but like all new mediums before it, the gaming industry still needs time to realize its potential. We're still waiting for anything the likes of <a href="http://www.imdb.com/title/tt0062622/" target="_blank">2001: A Space Odyssey</a> or <a href="http://www.imdb.com/title/tt0068646/" target="_blank">The Godfather</a>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkxFWNOrHVIqpkxiz-Vhwy_ygce19Af7noJ1Cis-gZFZ5K8Tc8KFBp7aGXSl0Ci_rhUIWmz6aUFq31A8HjdjSh4k3UoOcI6g7zQ9XhltzHyEgCHK5hutE_Im8U_SXmaylbH3Rf6D0M3CE/s1600/sshock2d.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="161" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkxFWNOrHVIqpkxiz-Vhwy_ygce19Af7noJ1Cis-gZFZ5K8Tc8KFBp7aGXSl0Ci_rhUIWmz6aUFq31A8HjdjSh4k3UoOcI6g7zQ9XhltzHyEgCHK5hutE_Im8U_SXmaylbH3Rf6D0M3CE/s320/sshock2d.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">We're Getting Close Though</td></tr>
</tbody></table>
<br />
Couple this with extreme confusion on the part of the gaming system providers and the conflict of interest that it creates. With Microsoft being the predominant PC gaming platform one would think they would be eager to lead the space. I think their own conflict of interest and lack of a credible competitor has caused them to become complacent. Time and time again Microsoft has claimed to <a href="http://www.microsoft.com/en-us/news/press/2007/mar07/03-14g4wandxboxlivepr.aspx" target="_blank">commit</a> to the platform, but since the release of the original Xbox in <a href="http://en.wikipedia.org/wiki/Xbox" target="_blank">2001</a> they back down in favor of that lone consumer-image win. The most important and perfect example of this is DirectX, which started out strong in 1995 with <a href="http://en.wikipedia.org/wiki/DirectX#Releases" target="_blank">ten major releases</a> in the first ten years, but since 2006 has seen only <b>one</b> major release (DX11). To help put this in perspective, Internet Explorer 7 was <a href="http://en.wikipedia.org/wiki/Internet_Explorer_7" target="_blank">released the same year</a> as DirectX 10. Many developers in the industry have been complaining for years that DirectX is <a href="http://www.bit-tech.net/hardware/graphics/2011/03/16/farewell-to-directx/1" target="_blank">holding the industry back</a> and it even appears at times that Microsoft uses relatively minor updates in an attempt to push gamers to the newest version of their operating system. (See <a href="http://www.techpowerup.com/186671/directx-11-2-exclusive-to-windows-8-1-and-xbox-one.html" target="_blank">Windows 8.1</a> and in a more defensible example, <a href="http://arstechnica.com/information-technology/2007/02/7060/" target="_blank">Windows Vista</a>) <i>Update 10/17/2013: <a href="http://blogs.windows.com/windows/b/appbuilder/archive/2013/10/14/raising-the-bar-with-direct3d.aspx" target="_blank">Microsoft has published</a> an article in defense of the Direct3d progress.</i><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
Additionally, Microsoft <a href="http://thedigitallifestyle.com/w/index.php/2012/06/19/tv-tuner-and-wmc-remote-control-and-receiver-no-longer-included-in-windows-hardware-certification/" target="_blank">doesn't</a> <a href="http://www.engadget.com/2012/06/23/two-more-nails-in-the-coffin-for-media-center-start-up-options/" target="_blank">want</a> you to hook up a PC to your TV. While home theater computing has never been a major market, it was inevitable that Microsoft would be a major player because of industry partnerships and the hardware support picture. Now they want to cut costs in that area and push the sale of the Xbox One instead, and that leaves space in this, albeit small, market segment. <br />
<br />
And this is just the start. The desire to move away from the traditional PC platform has been an undercurrent for years, but without a viable competitor there is no reason for hardware manufacturers to spend money developing for an alternative. That's not to say others haven't tried though, and their frustration can be best summarized by <a href="http://www.youtube.com/watch?v=iYWzMvlj2RQ" target="_blank">this</a> (NSFW, Language/Text) statement from Linus Torvalds. <br />
<br />
<h3>
Market Trends and Impact on Hardware Support</h3>
<span style="font-size: x-small;"><span style="font-family: Verdana,sans-serif;"><i>"Close the watertight doors." -Capt. EJ Smith</i></span></span><i><span style="font-size: x-small;"><span style="font-family: Verdana,sans-serif;"> </span></span></i><br />
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhg2332ChtGafcURnCRIYolzqlk2Nc0s_v_cvxQ3E-WG8i-oAvn5HHk8dn0sVpCiDhra2LkGZe7RSoZD4NLz-26Ft_BVB50-HqIM_kj-irc3dasclnzhNVyyKEnjU7u837g6olT8zX-kBQ/s1600/BulkyMobileComputing.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhg2332ChtGafcURnCRIYolzqlk2Nc0s_v_cvxQ3E-WG8i-oAvn5HHk8dn0sVpCiDhra2LkGZe7RSoZD4NLz-26Ft_BVB50-HqIM_kj-irc3dasclnzhNVyyKEnjU7u837g6olT8zX-kBQ/s320/BulkyMobileComputing.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Mobile Computing is Gaining in Popularity</td></tr>
</tbody></table>
<br />
With the monumental rise of portable computing (Smartphones, etc.) there is an effect here that is easily overlooked. Android (Linux) and iOS are absolutely <a href="http://www.forbes.com/sites/chuckjones/2013/09/30/iphones-market-share-down-prior-to-5s-5c-launch-with-windows-almost-double-digits-in-europe/" target="_blank">dominating</a> the mobile space, and with that comes the demand for and expertise in coding for other operating systems. This sort of experience doesn't directly undermine the current PC gaming picture, but it helps to build an environment more suitable for change.<br />
<br />
Microsoft seems primed to slowly <a href="http://www.microsoft.com/investor/reports/ar13/shareholder-letter/index.html" target="_blank">abandon the traditional PC</a> space in favor of pursuing the built-for-purpose device model of Apple coupled with the software-as-a-service model of Google. It isn't that they want to abandon gaming, they just want you to use a device for that. Over the course of the last year and a half they have <a href="http://techland.time.com/2013/09/03/microsoft-buys-nokia/" target="_blank">taken definitive steps</a> to <a href="http://www.microsoft.com/en-us/news/press/2013/jul13/07-11onemicrosoft.aspx" target="_blank">reorganize</a> and re-purpose themselves as a devices and services company. <br />
<br />
The problem with this strategy is they're trying to sell a platform that, when compared to the traditional PC or newer solutions (below), limits functionality. PC gaming thrives in part due to the community's ability to customize, add-on, and change the experience. As of now, nearly anything one may want to contribute, be it <a href="http://en.wikipedia.org/wiki/Modding" target="_blank">modding</a>, <a href="http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&ved=0CC4QtwIwAA&url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DLkCNJRfSZBU&ei=fRpUUrOUFtShqwH6w4A4&usg=AFQjCNGiahd5HwefDWgkZtjGyoiqphNvvQ&sig2=z2kIC7nJkmDbyuOIuU-jpA&bvm=bv.53537100,d.aWM" target="_blank">culture</a>, or even <a href="https://minecraft.net/" target="_blank">whole</a> <a href="http://privateer.sourceforge.net/" target="_blank">games</a> is open more often than not. In a future Xbox (or PS4) gaming space like the one Microsoft has portrayed those things become more difficult or impossible to do for the average lone and unassociated contributor. <br />
<br />
<h3>
Enter the Contender</h3>
<span style="font-size: x-small;"><span style="font-family: Verdana,sans-serif;"><i>Is that the protagonist from Mike Tyson's Punchout? </i></span></span><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg-jqcp96Jg84ohHBFBvycV4jx2nvR1CFEacM7YnEGcFQfa-m99elc7r_4jh0J_FiAUM7FzTAi8ozv_J1XsHxNiGlpqCYrDAfUHCC45uLK8rNjKCa4ZDrKmHD57V-CM5eA02NRzVyg4X4g/s1600/steam_os.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg-jqcp96Jg84ohHBFBvycV4jx2nvR1CFEacM7YnEGcFQfa-m99elc7r_4jh0J_FiAUM7FzTAi8ozv_J1XsHxNiGlpqCYrDAfUHCC45uLK8rNjKCa4ZDrKmHD57V-CM5eA02NRzVyg4X4g/s320/steam_os.jpg" width="320" /></a></div>
<span style="font-size: x-small;"><span style="font-family: Verdana,sans-serif;"><i><br /></i></span></span>
<br />
As I'm sure you're aware, <a href="http://en.wikipedia.org/wiki/Valve_Corporation" target="_blank">Valve</a> recently announced the <a href="http://www.joystiq.com/2013/09/25/steam-box-exists/" target="_blank">SteamMachines</a>, <a href="http://store.steampowered.com/livingroom/SteamController/" target="_blank">Steam Controller</a>, and most importantly (to my story at least), <a href="http://store.steampowered.com/livingroom/SteamOS/" target="_blank">SteamOS</a>. This article isn't about an in-depth analysis of these announcements, but for the sake of context I'll introduce each:<br />
<br />
<b><u>SteamMachines/Controller</u></b><br />
<br />
The SteamMachines aren't <b><i>a</i></b> platform but rather a series of similar platforms. It's more of a hardware spec for a living room focused PC. While Valve will be making (and offering in a limited 300 person beta) their own "SteamBox", the playing field is wide open for any manufacturer to create and ship a SteamMachine. Boy if Bissell ever wants to do a market sector transition this one would be easy. <a href="http://www.youtube.com/watch?v=eeAjkbNq4xI" target="_blank">The controller</a> is a stab at providing a traditional controller footprint that will more easily map to keyboard/mouse focused games. <a href="http://www.ign.com/articles/2013/09/30/steam-controller-developers-weigh-in" target="_blank">Initial impressions</a> on the <a href="http://electronics.howstuffworks.com/everyday-tech/haptic-technology.htm" target="_blank">haptic feedback</a> mechanism and other details have been relatively positive. That said, the controller is really an attempt to bridge the "couch gap" created by targeting this platform to the living-room/entertainment space. Should you choose you can still use an Xbox360 controller with the appropriate receiver or a traditional keyboard/mouse. <br />
<br />
<b><u>SteamOS</u></b><br />
<br />
This piece is what I find the most interesting and has the largest potential for impact. The software that will be run on any SteamMachine, SteamOS, has the following characteristics: <br />
<br />
<ul>
<li>An entire operating system <a href="http://en.wikipedia.org/wiki/Linux_distribution" target="_blank">distribution</a> based on the open-source juggernaut <a href="http://en.wikipedia.org/wiki/Linux" target="_blank">Linux</a></li>
<li>Will run and be supported (to an extent) on nearly <b>any</b> standard PC platform</li>
<li>Capable of streaming games from your existing Windows gaming PC, which not only leverages any existing heavy duty hardware but also helps to bridge the Linux compatibility gap. </li>
</ul>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgglbTAQdSxrevPgT24dAHr10tM4VhYKhFkZhhFSrFt0KJMkJMcCbqAc87LZIh9nBmAO36p1lkbwpHpu4kRl8_tR4DsnNKh0smKeE1R-1Eioui7cq7Qcc5HrPplOmzorTW8QAE-isHCPNo/s1600/tf2update.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="151" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgglbTAQdSxrevPgT24dAHr10tM4VhYKhFkZhhFSrFt0KJMkJMcCbqAc87LZIh9nBmAO36p1lkbwpHpu4kRl8_tR4DsnNKh0smKeE1R-1Eioui7cq7Qcc5HrPplOmzorTW8QAE-isHCPNo/s320/tf2update.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Valve CEO Gabe Newell and His Heavy Friend</td></tr>
</tbody></table>
<ul>
</ul>
Since this platform is based on Linux it creates quite an opportunity to leverage community improvements to its core. Say you want an API to have a certain feature for an upcoming piece of hardware... the vendor, the community, and Valve could all work in tandem to provide the solution quickly and efficiently for all. If exercised correctly, the partnership between private sector and community development can be very powerful. All that's needed to facilitate that partnership is financial incentive mixed with a motivated, intelligent user base. Both those elements stand ready to contribute should the pieces fall into place. <br />
<br />
<h3>
Contender to Whom?</h3>
<span style="font-size: x-small;"><span style="font-family: Verdana,sans-serif;"><i>I want you to be nice until it's time to not be nice.</i></span></span><br />
<br />
So this is where I'm supposed to make grand statements about head-to-head match-ups with consoles and/or PCs, but I won't. I don't think those match-ups will really happen. This platform isn't taking on any of the established players directly.... at least not yet.<br />
<br />
The beauty of this idea is that it is potentially more sustainable than any other model. While the purely private entities need to fight over exclusives and drop lots of cash developing their platforms, the SteamOS initiative can leverage an existing and growing library of top-quality gaming experiences and could rely on the slow, steady pace of innovation of the underlying platform to gradually drain market share from its competitors. <br />
<br />
<h3>
Wildcards</h3>
<span style="font-size: x-small;"><span style="font-family: Verdana,sans-serif;"><i>Nobody gave the 2007 Giants a chance. </i></span></span><br />
<span style="font-size: x-small;"><span style="font-family: Verdana,sans-serif;"><i><br /></i></span></span>
This is the particularly interesting part of this equation. Seeing as how the PC platform is arguably (in theory) the most adaptable gaming platform that is or will be available, it is worth tracking several new technologies that may contribute to the success or failure of the SteamOS or traditional Windows based gaming PC platform:<br />
<br />
<h4>
Oculus Rift and other Output Devices</h4>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhchZWplt4OSff0J-qn7_6ZLIhp043uPUiingoDXYfjRdmVjwZv4or1iUp6cDAjQ0n1tgjq2nE567hjtlF2RPB8wuDfEqPueCMuGVaBJ_4PeRZivK4FBLZdmZclLcSA-ulghyphenhyphengx1kNxW30/s1600/Occulus.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="226" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhchZWplt4OSff0J-qn7_6ZLIhp043uPUiingoDXYfjRdmVjwZv4or1iUp6cDAjQ0n1tgjq2nE567hjtlF2RPB8wuDfEqPueCMuGVaBJ_4PeRZivK4FBLZdmZclLcSA-ulghyphenhyphengx1kNxW30/s400/Occulus.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Jimmy Fallon Models the Oculus Rift and Looks... Comfortable?</td></tr>
</tbody></table>
<div>
<br /></div>
The <a href="http://www.oculusvr.com/" target="_blank">Oculus Rift</a> is the first VR headset that stands to be both cost effective and <a href="http://boulter.com/blog/2004/08/19/performant-is-not-a-word/" target="_blank">performant</a> enough to catch on in the consumer space. This device is a potential slam dunk for the living room space (space to move around + reliance on controller). Whoever has the most effective support out of the gate could dominate a small market of enthusiasts and genre devotees (flight sims, racing sims) right off the bat. <br />
<br />
<h4>
New Input Controls</h4>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNcJN76RjQB7OSl8RU7cOoQjhJIv5Z25ImCW4vKJffuWH-WcyvvIWbkz8y6JnK7LMykbNakS9G2V6m8a409D-SnBXPKZWQpbsWCSi84wX9aAE4nmX-RhPGCr0fr60Y6b66-a6Ii-_OgEM/s1600/stem.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="160" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNcJN76RjQB7OSl8RU7cOoQjhJIv5Z25ImCW4vKJffuWH-WcyvvIWbkz8y6JnK7LMykbNakS9G2V6m8a409D-SnBXPKZWQpbsWCSi84wX9aAE4nmX-RhPGCr0fr60Y6b66-a6Ii-_OgEM/s320/stem.jpg" width="320" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
Not to be left behind, the input space also has some new VR initiatives. There are quite a few out there, but the most interesting I've seen are the relatively portable treadmill solution <a href="http://www.kickstarter.com/projects/1944625487/omni-move-naturally-in-your-favorite-game" target="_blank">Omni</a>, the <a href="http://www.kickstarter.com/projects/89577853/stem-system-the-best-way-to-interact-with-virtual" target="_blank">Stem VR controller system</a>, and the <a href="http://tacticalhaptics.com/" target="_blank">Tactical Haptics</a> controller feedback system. Again, these are longshot devices that will only drive a small market initially, but the concept of a PC in the living room is a much better fit for any of these than a PC in the office or a console with limited support. <br />
<br />
<h4>
Rebirth of the <a href="http://en.wikipedia.org/wiki/Home_theater_PC" target="_blank">Home Theater PC</a></h4>
By targeting the living room sp<span id="goog_1463804608"></span><span id="goog_1463804609"></span><a href="http://www.blogger.com/"></a>ace the SteamMachine/SteamOS combo also taps into another potential market: the aforementioned HTPC crowd and additionally, if the price and functionality are right, the <a href="http://allthingsd.com/20130924/more-cord-cutting-drumbeats-very-likely-cutters-growing-in-number/" target="_blank">growing</a> "<a href="http://en.wikipedia.org/wiki/Multichannel_video_programming_distributor#Cord_cutters" target="_blank">cord cutters</a>" crowd. <br />
<br />
Should Valve's <a href="http://steamdb.info/blog/25/" target="_blank">media playback efforts</a> prove not only sufficient, but best in class, it would open up the market for low-spec/low-cost devices from hardware partners targeted explicitly at that space. The prospect of a roku-like device that can also play casual games and stream more power hungry ones could be a very interesting alternative to the bevy of devices found in many living rooms now, especially if a solution was provided to play games on low-end hardware. Speaking of transitions:<br />
<br />
<h4>
Network Streaming</h4>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAoi0m76xaVpCkKOK86YNfBBhKHYzOCIOdQAtjbh5h33HUK_0DuFc9dS_jqP1lM9fWyi8hjCmYpzrI3UZrp8vRE8eKsqdAOl6sGOmuQmtU8jwu_VTuJFVxYpw8dUbOmta8nBaqaaFc1hM/s1600/wow-screenshot.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAoi0m76xaVpCkKOK86YNfBBhKHYzOCIOdQAtjbh5h33HUK_0DuFc9dS_jqP1lM9fWyi8hjCmYpzrI3UZrp8vRE8eKsqdAOl6sGOmuQmtU8jwu_VTuJFVxYpw8dUbOmta8nBaqaaFc1hM/s400/wow-screenshot.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">"I'm not really here, I'm in a datacenter."</td></tr>
</tbody></table>
<div>
<br /></div>
Valve has already announced that you'll be able to stream games from another PC on your local network, but what if they were to introduce a "cloud" gaming service down the line? The technology is feasible; <a href="http://www.onlive.com/" target="_blank">Onlive</a> has been doing it for some time already. This would be an interesting alternative for people that don't want to spend a lot of money up front on a gaming caliber box, but still be able to play "AAA" titles provided they aren't bothered by the <a href="http://en.wikipedia.org/wiki/Input/output" target="_blank">I/O</a> latency. This space would be a natural transition for Valve's gaming model, and create all sorts of new opportunities for charging models such as rental/pay by the hour without leaving your living room.<br />
<br />
Note too that while Valve has stated they are working closely with both AMD and Nvidia, the first announced Steam Machine platform directly from Valve <a href="http://steamcommunity.com/groups/steamuniverse#announcements/detail/2145128928746175450" target="_blank">includes Nvidia hardware</a>. Keep in mind Nvidia <a href="http://www.nvidia.com/object/cloud-gaming.html" target="_blank">already offers</a> cloud gaming solutions and has the local streaming thing pretty much <a href="http://kotaku.com/theres-only-one-really-good-reason-to-buy-nvidias-shi-1208940700" target="_blank">figured out</a> (running on Linux no less). It would seem all the pieces may already be there. <a href="http://www.theverge.com/2013/9/26/4774418/microsoft-demonstrates-halo-4-streaming-from-the-cloud-to-windows-and" target="_blank">Microsoft</a> and <a href="http://www.engadget.com/2013/09/19/sony-ps3-titles-to-gaikai-2014/" target="_blank">Sony</a> are obviously in the race on this one as well. <br />
<br />
<h4>
The Performance Problem/Possibilities</h4>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div>
<br /></div>
Currently, the performance picture on Linux is somewhat spotty. There have been strides made in the last few months but the performance still generally trails its Windows counterpart. These performance differences are due almost entirely to drivers from hardware manufacturers. Of those, video card drivers have the most impact. For further reading on the current state of things, see: <a href="http://www.phoronix.com/scan.php?page=article&item=nvidia_windows8_geforce&num=1" target="_blank">Nvidia</a>, <a href="http://www.phoronix.com/scan.php?page=article&item=amd_win8_ubuntu13&num=1" target="_blank">AMD</a>, <a href="http://www.phoronix.com/scan.php?page=article&item=intel_iris5200_win8redux&num=1" target="_blank">Intel</a>. As for the future, there have <a href="http://www.pcworld.com/article/2049369/amd-nvidia-ramp-up-linux-driver-support-after-valves-steamos-announcement.html" target="_blank">already</a> been announcements pledging better support. <br />
<br />
Most games on the Windows platform utilize the <a href="http://en.wikipedia.org/wiki/Application_programming_interface" target="_blank">API</a> Direct3d, which acts as a unifying layer between the game and the video card. This allows for the same code to run on hardware from different vendors (in theory) without issue. This API is created by Microsoft, so it is unavailable on the Linux (SteamOS, Linux, MacOSX) platform. The standard API utilized by those platforms for 3d acceleration is and has been <a href="http://en.wikipedia.org/wiki/OpenGL" target="_blank">OpenGL</a> for quite some time, which also runs on Windows. This would be the obvious choice for games going forward should they want to target both major PC platforms.<br />
<br />
Oddly though, that isn't what a lot of developers would want to target anyhow. The more potentially lucrative pursuit would be to target consoles and the PC space, and anything coded to OpenGL and, to a lesser extent, Direct3d, would need work to be ported to consoles. <br />
<br />
And now there's <a href="http://www.hardwarecanucks.com/forum/hardware-canucks-reviews/63428-closer-look-amds-mantle.html" target="_blank">Mantle</a>.<br />
<br />
Mantle is the product of AMD's "next generation" console wins. The AMD Jaguar platform powers both the PS4 and Xbox One. To facilitate a unified API level and expose all the power possible on the platform, AMD created the "Mantle" API. Now AMD plans to bring this API to the PC. (Windows first, but it could work on any OS) It has already been adopted by the folks at <a href="http://dice.se/" target="_blank">Dice</a> and they reported substantial performance gains. We'll have to wait for December to find out, but it will be very interesting to see if these performance promises pan out. <br />
<br />
The potential here, however, is the possibility that this could further upset the entrenched standards in the industry. I don't think a hardware proprietary API is a good solution for the gaming community (remember <a href="http://en.wikipedia.org/wiki/Glide_API" target="_blank">Glide</a>?), but if it catches on in any way the competition will be forced to respond in kind. AMD claims they'll <a href="http://vr-zone.com/articles/mantling-alliances-ritchie-corpus-amd-interview/58215.html" target="_blank">open the platform</a> eventually but I'd wager that will end up like the Nvidia <a href="http://www.bit-tech.net/custompc/news/602205/nvidia-offers-physx-support-to-amd--ati.html" target="_blank">"offer" to share PhysX</a>. If the performance promises do pan out, one has to expect that <a href="http://gaben.tv/" target="_blank">Gabe</a> (Valve CEO) would love to nudge its adoption if only to further undermine Direct3d.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhL0VGzVb0dbY7JtMgEN6LBKTesdg0q8i-BF2hPz3XiKma5f5OfzZtxx7pv-Cg7fbBZtN8_UoiMagkJ1d2ZEqMVFo70vNeeiUytWGgdtmVty2Iw31V50UhU60q0iinXPMFPGi0nAPdkzh8/s1600/bf4.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhL0VGzVb0dbY7JtMgEN6LBKTesdg0q8i-BF2hPz3XiKma5f5OfzZtxx7pv-Cg7fbBZtN8_UoiMagkJ1d2ZEqMVFo70vNeeiUytWGgdtmVty2Iw31V50UhU60q0iinXPMFPGi0nAPdkzh8/s400/bf4.jpg" width="400" /></a></div>
<br />
<h4>
Unintended Consequences</h4>
<br />
With all of the pressures introduced by the SteamOS introduction, Windows 8 lukewarm reception, the rise of the mobile market, and everything else mentioned above there will undoubtedly be changes across the board in an attempt to gain new or retain old customers. All these factors combined lead us to...<br />
<br />
<h3>
The Point.</h3>
<span style="font-size: x-small;"><span style="font-family: Verdana,sans-serif;"><i>So why do we care about this?</i></span></span><br />
<span style="font-size: x-small;"><span style="font-family: Verdana,sans-serif;"><i><br /></i></span></span>
<br />
The pace of PC innovation hasn't kept up with other similar markets and it is now at its weakest point since its inception. Microsoft has done a poor job of keeping focus on the Windows based gaming space. Valve has introduced not only a hardware spec but a potential sans-Microsoft OS solution. Input, output, and consumption models are on the verge of being flipped on their head, and hardware manufacturers are moving to take more control of the API space. The probability of change to this platform stands higher than ever.<br />
<br />
We don't know what will happen. <b>For the first time in many years</b> we don't know what will happen. The opportunities are many and the space is an incredibly exciting one to witness and participate in.<br />
<br />
And this is the point. I'm intrigued by what this market could become for the first time in a long, long time. Perhaps Microsoft will re-invigorate the DirectX platform, or maybe SteamOS will finally bring the "<a href="http://en.wikipedia.org/wiki/Desktop_Linux#Year_of_Desktop_Linux" target="_blank">year of the Linux desktop</a>". Maybe the Oculus Rift combined with a Kinnect will launch the Xbox One to dominance, or maybe Nvidia releases a competing API that obsoletes everything prior. <br />
<br />
Or maybe there's room for all of this. The market is growing and we're finally getting some competition in the field that is long overdue.<br />
<br />
In times of rapid change the smallest actions can have large, lasting impacts. Even your product selection will impact the direction of video gaming in a small way, and any contributions you make to gaming communities even more-so. Now is <a href="http://www.youtube.com/watch?v=0atATbXbQ9g" target="_blank">our time</a> to choose the direction, so go and contribute what you can. I know we'll all have fun along the way.Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com0tag:blogger.com,1999:blog-5458589696790815336.post-19588111218797300852013-09-25T23:20:00.000-05:002013-09-28T19:40:09.859-05:00Overclocking Haswell Quick Fail MethodOverclocking is the process of increasing the clock speed on a system processor with the idea of getting higher performance. Given the time, this can be an interesting pursuit if you like pushing hardware performance to the highest level possible. With proper testing, this can be done once on a system and it will remain stable for 24/7/365 use for the life of the platform.<br />
<br />
With a week off of work I decided to upgrade my aging i7-920 x58 platform to a <a href="http://anandtech.com/show/7001/intels-haswell-quadcore-desktop-processor-skus">Haswell</a> based i7-4770k/z87. Overclocking this platform proved to have a steep learning curve, so I figured I'd share my general approach so others could save time getting started.<br />
<br />
<h3>
Overclocking Quick Fail Method</h3>
The general idea of the quick fail method (not sure if that's a name, but let's just pretend it is now) is to isolate each component of your build and then push it quickly to failure, then back it off and find the appropriate voltage/speed combo. <br />
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiwtoqquGPOMY1mJ0p2aAlGVf8LdyrC5oDgKuZx0mW-xJM6AmaBsqVxwiEAolX5rx0Q5CoYdB0x1m9j9PVRYsTt07GVEN7m6_YvEYcBEjWbkPaYXbfunH11Qil4ORewTdPpg47s9awUHeU/s1600/haswell_demo.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="265" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiwtoqquGPOMY1mJ0p2aAlGVf8LdyrC5oDgKuZx0mW-xJM6AmaBsqVxwiEAolX5rx0Q5CoYdB0x1m9j9PVRYsTt07GVEN7m6_YvEYcBEjWbkPaYXbfunH11Qil4ORewTdPpg47s9awUHeU/s400/haswell_demo.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Overclocking is hard with the Paparazzi on your tail. </td></tr>
</tbody></table>
<br />
<br />
<h4>
Software/Hardware Needed</h4>
<br />
<ul>
<li><a href="http://www.aida64.com/" target="_blank">ADIA64</a> for Voltage/Temp monitoring</li>
<li><a href="http://download.mersenne.ca/gimps/p95v279.win64.zip" target="_blank">Prime95</a> v.27.9 for Stress Testing (Discussion Below); Alternatively you can use <a href="http://www.ocbase.com/" target="_blank">OCCT</a>.</li>
<li>Intel K series processor (K series has an unlocked multiplier)</li>
<li>z87 chipset motherboard that supports overclocking; I used Asus but there are good boards from Gigabyte, MSI, and others as well. </li>
</ul>
<br />
<ul>
</ul>
<h4>
Getting Started/Clocking Approach</h4>
<br />
Make Sure you have <a href="http://www.youtube.com/watch?v=EyXLu1Ms-q4" target="_blank">applied your thermal interface material</a> correctly and disabled the IGP in the BIOS(assuming you're not using it). After your hardware is ready to go, we need to isolate components for testing. By doing so, we can more effectively ensure that each piece is stable before we combine them. We'll tweak/test in the following order:<br />
<br />
<ul>
<li>Core</li>
<li><a href="http://en.wikipedia.org/wiki/Uncore" target="_blank">Uncore</a> (Cache & Mem controller)</li>
<li>Memory</li>
<li>Combined</li>
</ul>
<br />
Those are also listed in order of performance. The Uncore, for example, does not need to be synced to the core clock and is <a href="http://www.overclock.net/t/1398975/official-haswell-owners-thread/2510#post_20587225" target="_blank">not nearly as important</a> from a performance perspective. <br />
<br />
<ul>
</ul>
<h4>
Familiarize Yourself With Voltage and Temperature Norms</h4>
<br />
Before getting started, I recommend familiarizing yourself with the voltage/temperature behavior of you CPU at stock. To do so, we'll use the ADIA64 sensor display and the built in stress-test. (Note I don't recommend ADIA for stress testing other than here to gen heat/voltage) Launch ADIA64 and open the sensor display by expanding <i>Computer</i>->"<i>Sensor</i>". Take note of the CPU Core (vCore), CPU Cache (vCache, sometimes called vRing), VCCSA (or Vsa, System Agent), and CPU VRM (vRIN) settings, as we'll be targeting those for modification later. Launch the system stability test tool by selecting "<i>Tools</i>"->"<i>System Stability Test</i>" and click "<i>Start</i>". Watch the behavior of the voltages and take note of which are dynamic, and how they behave under load, and what they top out at. You'll need to know this for comparison purposes when we start adjusting. <br />
<br />
<br />
<h3>
Setting Voltages</h3>
<div>
<br /></div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMpbnXptFnlt8FaXIS0x6DHtPPorCAOnvWEOQTyeOv4Fm2KZNaaXCnUF3HZvnUgo5cVziNWIAg37zTRLL_i6dyPvDNzcrh9aicBm5PAlPCQffa392R2Y5YlqKCUPuzjAZGwtHIUb2G3Zs/s1600/haswell_voltage.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMpbnXptFnlt8FaXIS0x6DHtPPorCAOnvWEOQTyeOv4Fm2KZNaaXCnUF3HZvnUgo5cVziNWIAg37zTRLL_i6dyPvDNzcrh9aicBm5PAlPCQffa392R2Y5YlqKCUPuzjAZGwtHIUb2G3Zs/s640/haswell_voltage.jpg" width="640" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<br />
Most z87 motherboards offer three or four options for overriding voltages: Offset, Adaptive, Manual and Auto. Let's take a moment to discuss each: <br />
<br />
<ul>
<li><b>Auto</b>: It's wise to avoid Auto unless you don't plan on spending much of any time testing. Most boards will apply far too much voltage to many aspects of the chip, creating additional heat which will waste energy and negatively effect your clocking potential. </li>
<li><b>Manual</b> will set voltage to a static number, which I'd shy away from as well for regular 24/7 use as it will counter alot of the benefit of the Haswell chip by not allowing for a dynamic reduction in voltage. Reducing the voltage reduces power consumption and heat mainly while idle. </li>
<li><b>Offset</b>: Now we're getting somewhere... Offset will apply a modifier to the target voltage on top of Intel's scaling algorithm. For example, if the vCore were set to .95 at a given point in time, and you specified a +.10 offset, then the resulting voltage would be 1.05. The only downside here is that the offset is always applied, but at least the voltage can scale downward. That leads us to:</li>
<li><b>Adaptive</b> adds additional voltage on top of the offset voltage, but only while the turbo multiplier is active. This is a great new addition as it allows whatever is set here to be entirely rolled off when not in turbo mode. Unfortunately, you will still need to use offset for most of the voltage boost in most cases to keep things stable. A good rule of thumb to start with is putting the desired extra voltage 75% into the offset and 25% into the turbo offset. </li>
</ul>
Note that many have found that adjusting the LLC ramp is not recommended when using Offset/Adaptive.<br />
<br />
<ul>
</ul>
<h3>
Stress Testing, Throttling, and the AVX Problem</h3>
<br />
<h4>
Why Prime95?</h4>
<br />
Of all the tools I've tried, including ADIA64, Prime95 has provided me the most consistent and reproducible results. Prime seems to do a better job of bombing an unstable Haswell than a lot of other tools. For example, I was able to complete an ADIA64 stress test for two hours while the same setup locked up my system nearly immediately on Prime95SmallFFT.<br />
<br />
More importantly, Prime allows for an effective customization of what portions of the CPU are being tested. SmallFFT will test the CPU and a bit of the cache(s) while Blend will effectively test the uncore and quite a bit of the system memory.<br />
<br />
<br />
<h4>
Haswell Thermal Throttling</h4>
<br />
The Haswell has an internal throttling mechanism (<i>Thermal Control Circuit</i>) to protect the chip that cannot be disabled. This throttling reduces the speed of the processor until the temperature drops below the temperature specified by Intel. For more information on the throttling point (at most 100 deg C) see <a href="http://www.intel.com/content/www/us/en/processors/core/4th-gen-core-family-desktop-vol-1-datasheet.html" target="_blank">table 26 in this document from Intel</a>. <br />
<br />
<h4>
The AVX Issue</h4>
<br />
<a href="http://en.wikipedia.org/wiki/Advanced_Vector_Extensions" target="_blank">AVX</a> are a series of extensions originally introduced in the SandyBridge platform that facilitate accelerating certain math functions. The problem with AVX is that when utilized by stress testing programs they're generally used in an "unrealistic" way (much more often than one would see) generating substantially more heat than the non-AVX equivalent processing. This can be enough heat to trigger thermal throttling of the Haswell chip which invalidates stress testing because it causes the chip to run at a slower speed.<br />
<br />
We'll start testing with AVX enabled because we do want to ensure the cooling solution can handle as much heat as theoretically could be thrown out, but if it causes the chip to throttle we will disable it. That will be covered below. <br />
<br />
<h3>
Determine Max Core Speed</h3>
<br />
<h4>
Lower Speed of Non-Core Components</h4>
<br />
To ensure we can definitively find the failure point of the core we need to take the uncore and memory out of the picture. <br />
<ol>
<li>Reduce the "Uncore" (sometimes called CPU Cache) multiplier to something substantially (5-8 depending on how aggressive) lower than your target overclock. For example if you are targeting 4.5Ghz, set the max Uncore multiplier to 38. If there is an option for minimum Uncore multiplier, leave it at "Auto". </li>
<li>Reduce the memory speed to something your RAM can easily handle, i.e. if you have DDR-2133, set it to DDR-1600 speed. </li>
</ol>
<h4>
Overclock the Core</h4>
<br />
<b>Primary Voltages</b>: vCore<br />
<b>Secondary Voltages</b>: vRIN, vCache<br />
<b>Test</b>: Prime95 SmallFFT <br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGhPigbbtGaKIpGPijpcspKNlLJKwk5M46nP2xp9ASIEwbS9Mbbx7Ts8oPmmB0fOo2DeWV6vDR4Pi_o1XJ0S4_zMX5oHUp4_pvF0lR-8QebdcqZy8VpwrGLtJaBGZWfl_s7WoR8rQ26nU/s1600/2013-09-23+23_03_17-.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGhPigbbtGaKIpGPijpcspKNlLJKwk5M46nP2xp9ASIEwbS9Mbbx7Ts8oPmmB0fOo2DeWV6vDR4Pi_o1XJ0S4_zMX5oHUp4_pvF0lR-8QebdcqZy8VpwrGLtJaBGZWfl_s7WoR8rQ26nU/s400/2013-09-23+23_03_17-.png" width="400" /></a></div>
<br />
<br />
<br />
<ol>
</ol>
<ol>
<li>Up the multiplier of the cores (if you have the option just sync all) to a reasonable starting point. On the i7-4770, for example, a good starting point would be 40 or so. </li>
<li>To ensure that your system isn't being throttled launch the ADIA64 stress testing tool but don't start it. The lower portion of the display will monitor and alert on throttling. </li>
<li>Perform a 20 minute SmallFFT stress test using Prime95. To do so, launch the 64 bit Prime95 client and select the "SmallFFT" option. Make sure you monitor and take notes of the system voltages and temperatures during the test. If there is a problem with the overclock at this stage, it will generally manifest itself as a BSOD. If your system throttles more than occasionally you need to disable AVX (see Appendix A) and re-start testing. </li>
<li>If the system seems stable thus far feel free to up the multiplier and repeat. If the system crashed and you want to push further with the understanding that with more voltage and speed comes more power consumption and heat, go ahead and add voltage (see above for methods) and then repeat testing. Voltage to vCore should be added in steps of .01v to .02v or so. I wouldn't exceed 1.35v under load for 24/7 running. If vCore doesn't seem to have an impact here, you may want to bump CPU Input Voltage (vRIN) or the uncore voltage (vCache) just a bit... just don't go to high with those because they don't matter nearly as much as vCore. </li>
<li>After working your multiplier and voltage up to where you're willing to go, perform an extended stability test by saving the BIOS settings and then running the Prime95 SmallFFT test for at least 4 hours. If the extended test fails, bump voltage up or multiplier down and re-start testing. </li>
</ol>
<h4>
Overclock the UnCore</h4>
<br />
<b>Primary Voltages</b>: vCache <br />
<b>Secondary Voltages</b>: vRIN, vCore, vSCCA <br />
<b>Tests</b>: Prime95 SmallFFT, Prime95 Blend<br />
<br />
Use the same methodology listed above to clock the Uncore. The following modifications exist: the main voltage to change is vCache rather than vCore. To ensure the uncore is stable you'll need to run Prime95 Blend after you finish the SmallFFT test. Don't be surprised if SmallFFT is stable and Blend requires more voltage and/or lowering the multiplier.<br />
<br />
<h4>
Clock Your Memory</h4>
<br />
<b>Primary Voltages</b>: vDIMM<br />
<b>Secondary Voltages</b>: vSCCA, vRIN, vCache<br />
<b>Test</b>: Prime95 Blend<br />
<br />
Finally, clock your RAM using the correct voltage, speed, and timing adjustments. (Details outside the scope of this article) Use an extended Prime95 Blend test to finalize everything. Personally I'm not comfortable with anything less than 24 hrs for the final test, but opinions on that vary.<br />
<br />
<br />
<h4>
Results/Final Thoughts</h4>
<br />
Obviously there is a ton here I'm not covering, but hopefully this information will save you time when starting your overclock. In some cases you may need to manipulate other voltages and voltage ramps, but those vary between motherboards so I won't touch on them.<br />
<br />
Using this methodology I was able to get my water-cooled i4770k 24/7 stable @ 4.8Ghz/4/4Ghz Uncore without de-lidding. Not too shabby I should think. Finally I can get the performance I've been looking for when running <a href="http://en.wikipedia.org/wiki/Lotus_1-2-3" target="_blank">Lotus 1-2-3</a>.<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgf1VQnv_v81LkxJllgCZj10R5nAf3hdZGVLWMdpqJkgnM011uI-LXvPuIN4sCplFIBZEeBQlAl0IfUGy0Cd3jbYzSAizK5TZsvQ5Gw6Pj9CGBNZNU6Z2IdYN4lzben7i6501d9BJ8J-Y8/s1600/2013-09-23+08_08_11-.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="441" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgf1VQnv_v81LkxJllgCZj10R5nAf3hdZGVLWMdpqJkgnM011uI-LXvPuIN4sCplFIBZEeBQlAl0IfUGy0Cd3jbYzSAizK5TZsvQ5Gw6Pj9CGBNZNU6Z2IdYN4lzben7i6501d9BJ8J-Y8/s640/2013-09-23+08_08_11-.png" width="640" /></a></div>
<br />
<br />
<h3>
Appendix A: (If Needed) Disable AVX</h3>
<ol>
<li>Launch command prompt as administrator</li>
<li>Execute "<i>bcdedit /set xsavedisable 1</i>"</li>
<li>Reboot </li>
</ol>
<b>Remember to re-enable</b> when done testing by using the same methodology, only executing bcdedit /set xsavedisable 0 rather than 1. <br />
<br />
<h3>
Appendix B: General Usage Notes</h3>
<br />
<ul>
<li>If you want to take full advantage of Haswell's power management make sure you use a power profile for 24/7 use that includes reducing the core processor speed. (Stored in advanced options)</li>
<li>If using the IGP for media playback applications, note that the Haswell <a href="http://www.avsforum.com/t/1477460/theory-about-intels-hdmi-quantization-range-setting-full-0-255" target="_blank">doesn't display the full 0-255 Quantization Range</a> without major workarounds. </li>
</ul>
<br />
<br />
<h3>
Additional Reading</h3>
<div>
<a href="http://www.techpowerup.com/reviews/Intel/Haswell_OC_Guide/" target="_blank">TechPowerUp Intel Z87 & Haswell OC Guide</a></div>
<div>
<a href="http://techreport.com/review/24889/haswell-overclocked-the-core-i7-4770k-at-4-7ghz" target="_blank">TechReport: Haswell Overclocked</a></div>
<div>
<a href="http://www.xtremesystems.org/forums/showthread.php?286614-Haswell-what-voltages-are-safe" target="_blank">XtremeSystems: Haswell: what voltages are safe?</a></div>
<div>
<a href="http://www.overclockers.com/3step-guide-to-overclock-intel-haswell" target="_blank">Overclockers: 3 Step Guide to Overclock Haswell</a></div>
<div>
<a href="http://www.hardwareluxx.com/index.php/reviews/hardware/cpu/26405-haswell-test-intel-core-i7-4770k-and-i5-4670k.html?start=7" target="_blank">HardwareLuxx: Haswell Review</a></div>
Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com2tag:blogger.com,1999:blog-5458589696790815336.post-69411388470729430832013-08-18T21:55:00.001-05:002013-08-20T14:14:33.927-05:00Hyper-V Port Mirroring and Network Capture <h3>Introduction</h3>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhuRMjy-9VGYcxlZv2xJeOn1NN0Zn0qPtU_oEfw5WGM9jHKV1S0CMH4Q_w8kaO8waF3l8kbu8Wa2uBJl-o3os3Z-JVRdt54IkO6eRCRZQGlazPQ0HPXMdUffYDOW9uV5am_MjY73cMA9Qc/s1600/wiretap.jpg"
imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img
style="width: 276px; height: 154px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhuRMjy-9VGYcxlZv2xJeOn1NN0Zn0qPtU_oEfw5WGM9jHKV1S0CMH4Q_w8kaO8waF3l8kbu8Wa2uBJl-o3os3Z-JVRdt54IkO6eRCRZQGlazPQ0HPXMdUffYDOW9uV5am_MjY73cMA9Qc/s320/wiretap.jpg"
border="0"></a>
<p>Hyper-V port mirroring, introduced in Windows Server 2012, allows you to
easily monitor traffic on virtual machines without having to capture the
traffic directly on that VM. It's exactly the same thing as standard <a href="http://en.wikipedia.org/wiki/Port_mirroring">port
mirroring</a>, only we're doing it using a virtual switch rather than a
physical one. You can use this in your Hyper-V environment for all sorts
of network troubleshooting. It only takes a couple of minutes to setup, so
let's do it and then capture some packets!</p>
<h3>Assumptions</h3>
<ul>
<li>Administrative Access to 2012 Hyper-V Host/Cluster</li>
<li>At least one vSwitch already configured (though we'll cover multiple)
</li>
<li>At least two VMs configured; one to monitor and one to collect the
data</li>
</ul>
<br>
<h3>Set Up Port Mirroring (GUI)</h3>
<br>
<h4>Configure Machine to be Mirrored</h4>
<ol>
<li><strong>Open</strong> the <strong>Hyper-V Manager</strong></li>
<li><strong>Right click</strong> the machine that you would like to
capture <strong>from</strong> and select "<strong><em>Settings</em></strong>".</li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj09dvW6C3zBCB5H11hSfCHjuLv_wUFuMHUcchQSdJD8QZrcFaDjuwzziZeE7nocYOv3_3LaDqipY7FzbkqlujG3ishYyhGV39oDiAknr01ikCFt7mof06a7HgmDpVeWm48YF3ByYQ6-_U/s1600/2013-08-17+23_32_04-.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj09dvW6C3zBCB5H11hSfCHjuLv_wUFuMHUcchQSdJD8QZrcFaDjuwzziZeE7nocYOv3_3LaDqipY7FzbkqlujG3ishYyhGV39oDiAknr01ikCFt7mof06a7HgmDpVeWm48YF3ByYQ6-_U/s640/2013-08-17+23_32_04-.png"
border="0" height="337" width="640"></a></div>
<br>
<li><strong>Expand the properties<em> </em>of the NIC</strong> (Network
Interface Card) you would like to mirror by clicking the plus sign to
its left and <strong>click</strong> "<strong><em>Advanced Features</em></strong>". </li>
<li>Under "<strong><em>Port mirroring</em>"</strong>->"<strong><em>Mirroring
Mode</em></strong>" click the drop down and <strong>select</strong>
"<strong><em>Source</em></strong>". This sets this NIC as the source of
mirroring on the Hyper-V switch it is connected to. </li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTbdVuncih7b_zYQGVZJnxpqor2IYzgttdVhpS5DnvDdt2TQoZy0m-oGp-zvw5BEI4TJzTRVQLPfXlYowy8-EyWYwzOfOzbL3ztJgQe3CNi54OY3jY_LRhMTVNHc-B6HfSGRqow4nd8_w/s1600/2013-08-10+14_39_40-Server+Manager.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTbdVuncih7b_zYQGVZJnxpqor2IYzgttdVhpS5DnvDdt2TQoZy0m-oGp-zvw5BEI4TJzTRVQLPfXlYowy8-EyWYwzOfOzbL3ztJgQe3CNi54OY3jY_LRhMTVNHc-B6HfSGRqow4nd8_w/s640/2013-08-10+14_39_40-Server+Manager.png"
border="0" height="601" width="640"></a></div>
<br>
<li><strong>Make note of the vSwitch</strong> the NIC is connected to
(right below "Network Adapter") and <strong>click<em> </em></strong>"<strong><em>OK</em></strong>".</li>
</ol>
<br>
<h4>Configure the Mirror Target</h4>
<br>
<p>It makes setting up a network capture substantially easier if you add a
dedicated NIC for each source machine. This NIC must be on the same
virtual switch. A dedicated NIC allows unbinding all services/protocols in
the guest OS, which will facilitate an entirely clean capture. More on
that below...</p>
<ol>
<li><strong>Shut down the virtual machine</strong> you intend on being the
<strong>target</strong> of the network capture. </li>
<li>After shutdown, <strong>right click</strong> that machine and <strong>select</strong>
"<strong><em>Settings</em></strong>".</li>
<li>Under "<strong><em>Add Hardware</em></strong>" in the right hand plane
<strong>select</strong> "<strong><em>Network Adapter</em></strong>" and
<strong>click</strong> "<strong><em>Add</em></strong>". </li>
<li>The network adapter properties page for the new NIC will come up.
Under "<strong><em>Virtual switch:</em></strong>" <strong>select</strong>
the <strong>same switch that the source machine/NIC</strong> is
connected to. </li>
<li>Expand the <strong>properties</strong> of the new NIC and <strong>select</strong>
"<strong><em>Advanced Features</em></strong>". </li>
<li>Under "<strong><em>Port mirroring</em></strong>"->"<strong><em>Mirroring
Mode</em></strong>" click the drop down and <strong>select</strong>
"<strong><em>Destination</em></strong>". This sets this NIC as the
source of mirroring on the Hyper-V switch it is connected to. <strong>Click</strong><em>
</em>"<strong><em>OK</em></strong>".</li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgCrzZB_UTpz4giux5tG7UP4AmiySR1mR1kLKjTtIC0CiGJaEoG_o2rm6Vk44xK2EZvv4d-5DkLvgPwRlz7MIV7iWUJ4ZcJSB_auPCvBSQxlavNaL1Nfouzujbw7rmwtDET8vj2cBBKGN8/s1600/2013-08-10+15_33_20-Server+Manager.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgCrzZB_UTpz4giux5tG7UP4AmiySR1mR1kLKjTtIC0CiGJaEoG_o2rm6Vk44xK2EZvv4d-5DkLvgPwRlz7MIV7iWUJ4ZcJSB_auPCvBSQxlavNaL1Nfouzujbw7rmwtDET8vj2cBBKGN8/s640/2013-08-10+15_33_20-Server+Manager.png"
border="0" height="601" width="640"></a></div>
<br>
<li><strong>Start the VM</strong> back up. </li>
</ol>
<br>
<h4>Configure the Mirror NIC in Capturing VM OS</h4>
<br>
<p><strong>Note</strong>: The instructions for this portion are somewhat
generic because your guest OS on the capturing VM may differ from mine. </p>
<ol>
<li>After the capturing VM starts back up, <strong>log on via RDP or
otherwise</strong>. </li>
<li><strong>Open</strong> your <strong>network connections</strong> and
determine <strong>which NIC</strong> is the added one for mirroring. <strong>Rename
it</strong> something like "<em>{VSwitchName} Port Mirror</em>" for
easy identification.</li>
<li><strong>Open </strong>the<strong> properties</strong> of that NIC. </li>
<li><strong>Un-bind all protocols and services</strong> from that NIC and
<strong>click</strong> "<strong><em>OK</em></strong>". By removing all
bindings we'll be able to ensure a clean capture without interfering
with the existing network connection. None of the standard protocols or
services are used in the mirror process; Hyper-V takes care of
everything for you. If you already have your network
sniffing software installed, you may need to reboot the capture machine
in order to see the NIC. </li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjC2aapXtw4bPoIgH7IGilw8yAD7nIMqTY7hERHzEuDIID3v32dN3pyZlSSYMSZnskO1bs0ZVdauFesATLhTVGUJIdBlMsp55WpG0hWDYRNReCeKprz1cVNy_mHrZAcAg94cuBMvSKUTGY/s1600/2013-08-10+15_55_43-Control+Panel_Network+and+Internet_Network+Connections.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjC2aapXtw4bPoIgH7IGilw8yAD7nIMqTY7hERHzEuDIID3v32dN3pyZlSSYMSZnskO1bs0ZVdauFesATLhTVGUJIdBlMsp55WpG0hWDYRNReCeKprz1cVNy_mHrZAcAg94cuBMvSKUTGY/s400/2013-08-10+15_55_43-Control+Panel_Network+and+Internet_Network+Connections.png"
border="0" height="400" width="317"></a></div>
<br>
</ol>
<h4>Install and Use Packet Capture Software</h4>
<br>
<p>I'll be using Wireshark for Windows; if desired you could substitute
something like <a href="http://www.microsoft.com/en-us/download/details.aspx?id=4865">Microsoft
Network Monitor</a> or <a href="https://connect.microsoft.com/site216">Microsoft
Message Analyzer</a> on Windows; <a href="https://www.archlinux.org/packages/?sort=&q=wireshark">Wireshark</a>
or <a href="http://www.tcpdump.org/">tcpdump</a> on Linux. </p>
<ol>
<li><strong>Download and install <a href="http://www.wireshark.org/download.html">Wireshark</a></strong>.
The portable version works just as well if you prefer. </li>
<li><strong>Open Wireshark</strong> and <strong>click</strong> the "<strong><em>Interface
List</em></strong>" button on the upper left hand corner. </li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8x9OYY4bu7MAq6zRyAcrLTqI2CSjrBRO82Mw82-_1YUPjcmK0z3CKhl9QMd3KhGKglFjVvXPgYquwUkiy_P-ckBQm4fLJaEZKM7Fy9K3SGK9zXBaIRB2oPC9ugELbDMtxO2UCtbHirFc/s1600/2013-08-17+22_20_23-The+Wireshark+Network+Analyzer+++%5BWireshark+1.10.1++%28SVN+Rev+50926+from+_trunk-1.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img style="width: 310px; height: 304px;"
src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8x9OYY4bu7MAq6zRyAcrLTqI2CSjrBRO82Mw82-_1YUPjcmK0z3CKhl9QMd3KhGKglFjVvXPgYquwUkiy_P-ckBQm4fLJaEZKM7Fy9K3SGK9zXBaIRB2oPC9ugELbDMtxO2UCtbHirFc/s400/2013-08-17+22_20_23-The+Wireshark+Network+Analyzer+++%5BWireshark+1.10.1++%28SVN+Rev+50926+from+_trunk-1.png"
border="0"></a></div>
<br>
<li><strong>Select</strong> the <strong>dedicated capture NIC</strong>
(which we renamed earlier), ensure it is the only selected, and <strong>click</strong>
"<strong><em>Start</em></strong>".</li>
<br>
<div class="separator" style="clear: both; text-align: center;"> <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigQ8fSJLnJXyLPI_QbgYgzTS2cDtCWkBFADIiaAyB_Vp5qoYEd4XWV-SUPxkMiI21krEVT1O3wcvRdHUsKGHr62AjI2KXe6dcu_wPQPeq79xBCjeqTE4EwxnJb-0VNAarhlMNBLeCitlY/s1600/2013-08-17+22_21_26-The+Wireshark+Network+Analyzer+++%5BWireshark+1.10.1++%28SVN+Rev+50926+from+_trunk-1.png"
imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigQ8fSJLnJXyLPI_QbgYgzTS2cDtCWkBFADIiaAyB_Vp5qoYEd4XWV-SUPxkMiI21krEVT1O3wcvRdHUsKGHr62AjI2KXe6dcu_wPQPeq79xBCjeqTE4EwxnJb-0VNAarhlMNBLeCitlY/s640/2013-08-17+22_21_26-The+Wireshark+Network+Analyzer+++%5BWireshark+1.10.1++%28SVN+Rev+50926+from+_trunk-1.png"
border="0" height="225" width="640"></a></div>
<br>
<li><a href="http://www.wired.com/images_blogs/underwire/2009/06/tron_guy_roflcon.jpg">Enjoy</a>
<a href="http://www.wireshark.org/docs/">all</a> <a href="http://wiki.wireshark.org/CaptureFilters">your</a>
<a href="http://wiki.wireshark.org/DisplayFilters">packets</a>. </li>
</ol>
<h3>Powershell Commands for NIC Setup/Mirroring</h3>
<ul>
<li><a href="http://technet.microsoft.com/en-us/library/hh848564.aspx">Add-VMNetworkAdapter</a>:
(Host) Add an adapter to a VM</li>
<li><a href="http://technet.microsoft.com/en-us/library/hh848457.aspx">Set-VMNetworkAdapter</a>:
(Host) Change properties of NIC to mirror source, destination, etc. </li>
<li><a href="http://technet.microsoft.com/en-us/library/jj130867.aspx">Get-NetAdapter</a>:
(Guest) Get list of NICs</li>
<li><a href="http://technet.microsoft.com/en-us/library/jj130868.aspx">Rename-Netadapter</a>:
(Guest) Rename NICs</li>
</ul>
<ol>
</ol>
<h3>Caveats</h3>
<ul>
<li>You won't be able to decrypt encrypted packets unless you get the
private key from the target server for decryption, which obviously may
be a security issue given we're not on that machine. </li>
<li>Make sure you de-config (new word!) the port mirroring in HyperV when
you're done as the packet replication continues even if you're not
capturing. </li>
<li>After unbinding all services/protocols in Windows the adapter won't
appear in the "Network and Sharing Center" anymore. You'll have to click
"Change adapter settings" to get to the NIC. </li>
</ul>
<h3>More Reading</h3>
<ul>
<li><a href="http://technet.microsoft.com/en-us/library/jj679878.aspx">What's
New in Hyper-V Virtual Switch for Windows Server 2012</a></li>
<li><a href="http://jeffwouters.nl/index.php/2013/01/powershell-configure-port-mirroring-in-hyper-v/">Jeff
Wouter: Powershell Port Mirroring in Hyper-V</a></li>
</ul>
<ul>
</ul>
<br>
<br>
<p>That's it, easy eh? Questions/Comments, leave 'em below!</p>Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com2tag:blogger.com,1999:blog-5458589696790815336.post-34395734618506114692013-08-08T22:00:00.000-05:002015-05-05T00:17:45.678-05:00Starting Small: Set Up a Hadoop Compute Cluster Using Raspberry Pis<h4>
What is <a href="http://en.wikipedia.org/wiki/Apache_Hadoop">Hadoop</a>?
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwA935-gFuMMdr4H_22exEhZE9YWgSuk3i9u_xbywhJJALc0GJiOLqRvJGkdBxqeOWtOOHxRoP8ZRA5foldtViOyD35RHFjGRIbEtZsQd9LAAV4kNoYyJekZEn3otEuHlsatGjD9GG-FM/s1600/hadoop-elephant.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="149" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwA935-gFuMMdr4H_22exEhZE9YWgSuk3i9u_xbywhJJALc0GJiOLqRvJGkdBxqeOWtOOHxRoP8ZRA5foldtViOyD35RHFjGRIbEtZsQd9LAAV4kNoYyJekZEn3otEuHlsatGjD9GG-FM/s200/hadoop-elephant.jpg" width="200" /></a></div>
</h4>
<br />
Hadoop is a big data computing framework that generally refers to the main
components: the core, <a href="http://www-01.ibm.com/software/data/infosphere/hadoop/hdfs/">HDFS</a>,
and <a href="http://en.wikipedia.org/wiki/MapReduce">MapReduce</a>. There
are several other projects under the umbrella as well. For more information,
see <a href="http://www.youtube.com/watch?v=S9xnYBVqLws">this interview</a>
with <a href="http://www.cloudera.com/content/cloudera/en/home.html">Cloudera</a>
CSO Mike Olson. <br />
<br />
<h4>
What is a <a href="http://www.raspberrypi.org/">Raspberry Pi</a>?</h4>
<br />
The Pi is a small, inexpensive ($39) <a href="http://en.wikipedia.org/wiki/ARM_architecture">ARM</a>
based computer. It is meant primarily as an educational tool. <br />
<br />
<h4>
Is Hadoop on the Pi practical?</h4>
<br />
<b>Nope!</b> Compute performance is horrendous. <br />
<br />
<h4>
Then why?</h4>
<br />
It's a great learning opportunity to work with Hadoop and multiple nodes.
It's also cool to be able to put your "compute cluster" in a lunch box. <br />
In reality though, this article is much more about setting up your first
Hadoop compute cluster than it is about the Pi. <br />
<br />
<h4>
Could I use this guide to setup a non-Raspberry Pi based (Real) Hadoop
Cluster?</h4>
<br />
Absolutely, please do. I'll make notes to that effect throughout. <br />
<br />
<h4>
I've been wanting to do this anyhow, let's get started!</h4>
<br />
Yeah, that's what I said. <br />
<br />
<br />
<h3>
Hardware Used</h3>
<br />
<ul>
<li>3x Raspberry Pis</li>
<li>1x 110v Power Splitter</li>
<li>1x 4 port 10/100 Switch</li>
<li>1x Powered 7 port USB hub</li>
<li>3x 2' Cat 5 cables</li>
<li>3x 2' USB "power" cables</li>
<li>3x Class 10 16GB SDHC Cards</li>
</ul>
<br />
Total cost: about $170 bucks. I picked it all up at my local Microcenter,
including the Pis!<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;">
<tbody>
<tr>
<td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEifuw9NTAzOJEYc1UY778P7NLAbuSYFJELoeu_946G0mwo2hvEswzVk0eC-POuYcuplL3kVqLMCi-Q2jHloTJfknTNWJ-DKvxdDj-6wpnmr_9MXwa8OPbA7TJi0fWcF9eeR0T7s5QJKM3A/s1600/WP_000028.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEifuw9NTAzOJEYc1UY778P7NLAbuSYFJELoeu_946G0mwo2hvEswzVk0eC-POuYcuplL3kVqLMCi-Q2jHloTJfknTNWJ-DKvxdDj-6wpnmr_9MXwa8OPbA7TJi0fWcF9eeR0T7s5QJKM3A/s400/WP_000028.jpg" width="400" /></a></td>
</tr>
<tr>
<td class="tr-caption" style="text-align: center;">Pi Hadoop Cluster;
caution: may be slow. </td>
</tr>
</tbody>
</table>
<br />
If you would like a a high performance Hadoop cluster just pick one of these
up on the way home: <br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;">
<tbody>
<tr>
<td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKVyOhyCoLwqcPhRk-SgvVqnr8Gc-Dlf-C5xdRJ-9T8I-GL9JmVQAaXHRh-fExZhw_3wQ9N11uHBiukjYPRcG1O0WXlE89aTl0NLjAklJ05PSUGWTHllTMCdRf-sPHsgxwwN5UYb_Z2y0/s1600/Yahoo-hadoop-cluster_OSCON_2007.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="172" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKVyOhyCoLwqcPhRk-SgvVqnr8Gc-Dlf-C5xdRJ-9T8I-GL9JmVQAaXHRh-fExZhw_3wQ9N11uHBiukjYPRcG1O0WXlE89aTl0NLjAklJ05PSUGWTHllTMCdRf-sPHsgxwwN5UYb_Z2y0/s400/Yahoo-hadoop-cluster_OSCON_2007.jpg" width="400" /></a></td>
</tr>
<tr>
<td class="tr-caption" style="text-align: center;">This will cost at
least $59.95. Maybe more. Probably more.</td>
</tr>
</tbody>
</table>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<h3>
Raspberry Pi Preparation</h3>
<br />
We'll do the master node by itself first so that this guide can be used
for a single or multi-node setup. <b>Note</b>: On a non-Pi
installation skip to Networking Preparation (though you may want to update
your OS manually).<br />
<br />
<h4>
Initial Config</h4>
<ol>
<li>Download the "Raspbian wheezy" image from <a href="http://www.raspberrypi.org/downloads">here</a>
and follow <a href="http://elinux.org/RPi_Easy_SD_Card_Setup">these</a>
directions to set up your first SD card. Soft-float isn't necessary. </li>
<li>Hook up a monitor and keyboard and start up the device by connecting
up the USB power.</li>
<li>When the Raspberry Pi config tool launches, change the keyboard layout
first. If you set passwords, etc. with the wrong KB layout you may have
a hard time with that later. </li>
<li>Change the timezone, then the locale. (Language, etc.)</li>
<li>Change the default user password</li>
<li>Configure the Pi to disable the X environment on boot.
(boot_behavior->straight to desktop->no)</li>
<li>Enable the SSH Daemon (Advanced-> A4 SSH) then exit the
raspi-config tool</li>
<li>Reboot the Pi. (sudo reboot)</li>
</ol>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFvRBRMfDmimIIOg0KAKc71dAhfVexFbyYcZFBir_smQwtyFZ7fnrtMmJFt74mNlbly3dQcXzjWvQxRsybXw1l_Pu3UAuG5bMWNy6KvcOw_A4mulGnJXTWDHQbwyIhyRsbKp0vpakE5P4/s1600/2013-07-26+23_49_36-hduser@rasdoop1_+_.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFvRBRMfDmimIIOg0KAKc71dAhfVexFbyYcZFBir_smQwtyFZ7fnrtMmJFt74mNlbly3dQcXzjWvQxRsybXw1l_Pu3UAuG5bMWNy6KvcOw_A4mulGnJXTWDHQbwyIhyRsbKp0vpakE5P4/s400/2013-07-26+23_49_36-hduser@rasdoop1_+_.png" width="400" /></a></div>
<ol>
</ol>
<h4>
Update The OS</h4>
<b>Note</b>: It is assumed from henceforth you are connecting
to the Pi or your OS via <a href="http://www.wikihow.com/Use-SSH">SSH</a>.
You can continue on the direct terminal if you like until we get to
multiple nodes. <br />
<ol>
<li>Log on to your Pi using the Raspberry user with the password you set
earlier. </li>
<li>To pull the newest sources, execute sudo apt-get update</li>
<li>To pull upgrades for the OS and packages, execute sudo apt-get upgrade</li>
<li>Reboot the pi. (sudo reboot)</li>
</ol>
<h4>
Split the Memory/Overclock</h4>
<ol>
<li>After logging back into the Pi, execute sudo raspi-config to bring up
the Rasperry Pi config tool. </li>
<li>Select Option 8, "Advanced Options"</li>
<li>Select Option A3, "Memory Split"</li>
<li>When prompted for how much memory the GPU should have, enter the
minimum, "16" and hit enter. </li>
<li>(If you would like to overclock) Select Option 7, "Overclock", hit
"OK" to the warning, and select what speed you would like.</li>
<li>On the main menu, select "finish" and hit "Yes" to reboot. </li>
</ol>
<h3>
Networking Preparation</h3>
Each Hadoop node must have a unique name and static IP. I'll be
following the Debian instructions since the Rasbian build is based on <a href="https://wiki.debian.org/NetworkConfiguration#Configuring_the_interface_manually">Debian</a>.
If you're running another distro follow the instructions for that. (<a href="https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/3/html/Reference_Guide/s1-networkscripts-interfaces.html">Redhat</a>,
<a href="https://gist.github.com/fernandoaleman/2172388">CentOS</a>, <a href="http://www.howtoforge.com/linux-basics-set-a-static-ip-on-ubuntu">Ubuntu</a>)<br />
<br />
First we need to change the machine hostname by editing /etc/hostname<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">sudo nano /etc/hostname</code></pre>
<b> Note</b>: I'll be using nano throughout the article; I'm sure
some of you will be substituting vi instead. :) <br />
Change the hostname to what you would like and save the file; I used "<b>Node
1</b>" throughout this document. If you're setting up the second node
make sure to use a different hostname.<br />
<br />
Now to change to a static IP by editing /etc/network/interfaces<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">sudo nano /etc/network/interfaces</code></pre>
Change the iface eth0 to a static IP by replacing "auto eth0" and/or "iface
eth0 inet dhcp" with: <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">iface eth0 inet static
address 192.168.1.40
netmask 255.255.255.0
gateway 192.168.1.1</code></pre>
<b>Note</b>: Substitute the correct address (IP), netmask, and
gateway for your environment. <br />
<br />
Make sure you can resolve DNS queries correctly by editing your
/etc/resolv.conf:<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">sudo nano /etc/resolv.conf</code></pre>
Change the content to match below, substituting the correct information for
your environment:<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">domain company.com
search company.com
nameserver 192.168.1.20
nameserver 192.168.1.30
</code></pre>
domain=domain suffix for this machine<br />
search=appends when FQDN not specified<br />
nameserver=list DNS servers in order of precedence <br />
<br />
We could restart networking etc, but for the sake of simplicity we'll bounce
the box: <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">sudo reboot</code></pre>
<h3>
Install Java</h3>
If you're going for performance you can install the SunJDK, but getting
that to work requires a bit of extra effort. Since we're on the Pi and
performance isn't our goal, I'll be using OpenJDK which installs easily.
If you're installing on a "real" machine/VM, you may want to diverge a bit
here and go for the real deal (Probably Java ver 6). <br />
After connecting to the new IP via SSH again, execute: <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">sudo apt-get install openjdk-7-jdk</code></pre>
Ensure that 7 is the version you want. If not substitute the package
accordingly. <br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjgG1izalv0m37a-AtLFpQttIcHaIP-nRVpBPLp3PHkEuxPAbxNxsOkYBTveIgX20n_A1OWYBY227bOJnZQtJ90jRcnxNtCZHI21LlE_pUtOtOSP6CudCpUgPm86P9lWSGt5eO7MXihRzM/s1600/2013-06-25+01_26_56-Blogger_+itToby+-+Edit+post.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="392" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjgG1izalv0m37a-AtLFpQttIcHaIP-nRVpBPLp3PHkEuxPAbxNxsOkYBTveIgX20n_A1OWYBY227bOJnZQtJ90jRcnxNtCZHI21LlE_pUtOtOSP6CudCpUgPm86P9lWSGt5eO7MXihRzM/s640/2013-06-25+01_26_56-Blogger_+itToby+-+Edit+post.png" width="640" /></a></div>
<br />
<br />
<h3>
Create and Config Hadoop User and Groups</h3>
Create the hadoop group:
<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">sudo addgroup hadoop</code></pre>
Create hadoop user "hduser" and place it in the hadoop group; make sure you
remember the password!<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">sudo adduser --ingroup hadoop hduser</code></pre>
Give hduser the ability to sudo: <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">sudo adduser hduser sudo</code></pre>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi9-jUkOBqxs0DK_e1aG32kNsD3AHUAPMhXVBAUUv1JHePa5KVajHjZMNksPTwcqoLgpNMIs6dbOiiYp553jD0179rn0tb6baAnpldSJ2YZ8Sfy2GQ75Kz_TB-8tk8Sij1OXWOOkhqjiyQ/s1600/2013-06-25+01_34_36-Blogger_+itToby+-+Edit+post.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="310" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi9-jUkOBqxs0DK_e1aG32kNsD3AHUAPMhXVBAUUv1JHePa5KVajHjZMNksPTwcqoLgpNMIs6dbOiiYp553jD0179rn0tb6baAnpldSJ2YZ8Sfy2GQ75Kz_TB-8tk8Sij1OXWOOkhqjiyQ/s640/2013-06-25+01_34_36-Blogger_+itToby+-+Edit+post.png" width="640" /></a></div>
Now logout: <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">logout</code></pre>
Log back in as the newly created hduser. <b>From hence forth
everything will be executed as that user. </b><br />
<span style="font-weight: bold;"></span><br />
<h3>
Setup SSH Certs</h3>
SSH is used between Hadoop nodes and services (even on the same node) to
coordinate work. These SSH sessions will be run as the Hadoop user, "hduser"
that we created earlier. We need to generate a keypair to use for EACH node.
Let's do this one first:<br />
<br />
Create the certs for use with ssh:
<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">ssh-keygen -t rsa -P ""</code></pre>
When prompted, save the key to the default directory (will be <i>/home/USERNAME/.ssh/id_rsa</i>).
If done correctly you will be shown the fingerprint and randomart image. <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhJygjgxWormaih1sz0mGqt0AYZtEH_L1ZjoOiBO85rqYoTYoJnhtMx1bsoZlIAlpa6wwmCxECP5pEjDogkqTHr21WROMpboji6gIapydFKVUanmqSRb9-PPZvNr5HQm2l3pIhFpnxZdH0/s1600/2013-06-25+01_37_14-Blogger_+itToby+-+Edit+post.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="507" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhJygjgxWormaih1sz0mGqt0AYZtEH_L1ZjoOiBO85rqYoTYoJnhtMx1bsoZlIAlpa6wwmCxECP5pEjDogkqTHr21WROMpboji6gIapydFKVUanmqSRb9-PPZvNr5HQm2l3pIhFpnxZdH0/s640/2013-06-25+01_37_14-Blogger_+itToby+-+Edit+post.png" width="640" /></a></div>
<br />
Copy the public key into the user's authorized keys store (~ represents the
home directory of the current user, >> appends to that file to
preserve any keys already there)
<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys</code></pre>
SSH to localhost and add to the list of known hosts. <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">ssh localhost</code></pre>
When prompted, type "yes" to add to the list of known hosts. <br />
<br />
<h3>
Download/Install Hadoop!</h3>
<b>Note</b>: As of this writing, the newest version of the 1.x
branch (which we're covering) is 1.2. (Ahem, 1.2.1, they're quick) You should check to see what the
newest stable release is and change the download link below accordingly.
Versions and mirrors can be found <a href="http://hadoop.apache.org/releases.html">here</a>.
<br />
<br />
Download to home dir: <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">cd ~
wget http://mirror.catn.com/pub/apache/hadoop/core/hadoop-1.2.0/hadoop-1.2.0.tar.gz</code></pre>
Unzip to the /usr/local dir:<code></code><br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">sudo tar vxzf hadoop-1.2.0.tar.gz -C /usr/local</code></pre>
Change the dir name of the unzipped hadoop dir. Note that if your version #
is different you'll need to adjust the command<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">cd /usr/local
sudo mv hadoop-1.2.0/ hadoop</code></pre>
Set the hadoop dir owner to be the
hadoop user (hduser):<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">sudo chown -R hduser:hadoop hadoop</code></pre>
<br />
<h3>
Configure the User Environment</h3>
Now we'll config the environment for the user (hduser) to run Hadoop. This
assumes you're logged on as that user. <br />
Change to the home dir and open the .bashrc file for editing:<code></code><br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">cd ~
nano .bashrc</code></pre>
Add the following to the .bashrc file. It doesn't matter where: (note if
you're using Sun Java on a real box you need to put in the right dir for
JAVA_HOME)<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code><code>export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-armhf
export HADOOP_INSTALL=/usr/local/hadoop
export PATH=$PATH:$HADOOP_INSTALL/bin</code></code></pre>
Bounce the box (you could actually just log off/on, but what the hell eh?):<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">sudo reboot</code></pre>
<code style="color: black; word-wrap: normal;"></code><br />
<h3>
Configure Hadoop!</h3>
(Finally!) It's time to setup Hadoop on the machine. With these steps we'll
set up a single node and we'll discuss multiple node configuration
afterward. <br />
First let's ensure your Hadoop install is in place correctly. After logging
in as hduser, execute:<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">hadoop version</code></pre>
you should see something like: (differs depending on version)<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code>Hadoop 1.2.0
Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1479473
Compiled by hortonfo on Mon May 6 06:59:37 UTC 2013
From source with checksum 2e0dac51ede113c1f2ca8e7d82fb3405
This command was run using /usr/local/hadoop/hadoop-core-1.2.0.jar</code>
</pre>
Good. Now let's configure. First up we'll set up the environment file
that defines some overall runtime parameters:<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">nano /usr/local/hadoop/conf/hadoop-env.sh</code></pre>
Add the following lines to set the Java Home parameter (specifies where Java
is) and how much memory the Hadoop processes are allowed to use. There
should be sample lines in your existing hadoop-env.sh that you can just
un-comment and modify. <br />
<b>Note</b>: If you're using a "real" machine rather than a Pi,
change the heapsize to a number appropriate to your machine (physical mem -
(OS + caching overhead)) and ensure you have the right directory for
JAVA_HOME because it will likely be different. If you have a Raspberry Pi version "A" rather than "B", you'll heapsize will need to be much lower since the RAM is halved.<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-armhf
export HADOOP_HEAPSIZE=272</code></pre>
Now let's configure core-site.xml. This file defines critical operational
parameters for a Hadoop site. <br />
<b>Note</b>: You'll notice we're using "localhost" in the
configuration. That will change when we go to multi-node, but we'll leave it
as localhost for a demonstration of an exportable local only configuration.
<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">nano /usr/local/hadoop/conf/core-site.xml</code></pre>
Add the following lines. If there is already information in the file make
sure you respect <a href="http://www.w3schools.com/xml/xml_whatis.asp">XML
format rules</a>. The "description" field is not required, but I've
included it to help this tutorial make sense. <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code><configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/fs/hadoop/tmp</value></code><code><code><code> <description>Sets the operating directory for Hadoop data.</code></code><code><code><property></property>
</description></code>
</code> </property>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:54310</value></code><code><code> <description></code></code><code><code><property><description>The name of the default file system. A URI whose
scheme and authority determine the FileSystem implementation.
The URI's scheme determines the config property (fs.SCHEME.impl) naming
the FileSystem implementation class. The URI's authority is used to
determine the host, port, etc. for a filesystem.</description> </property>
</description></code>
</property>
</configuration>
</code><code></code></pre>
It's time to configure the mapred-site.xml. This file
determines where the mapreduce job tracker(s) run. Again, we're using
localhost for now and we'll change it when we go to multi-node in a bit. <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">nano /usr/local/hadoop/conf/mapred-site.xml</code></pre>
Add the following lines: <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code><configuration>
<property>
<name>mapred.job.tracker</name>
<value>localhost:54311</value>
<description>The host and port that the MapReduce job tracker runs
at. If "local", then jobs are run in-process as a single map
and reduce task.
</description>
</property>
</configuration></code></pre>
Now on to hdfs-site.xml. This file configures the HDFS parameters. <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">nano /usr/local/hadoop/conf/hdfs-site.xml</code></pre>
Add the following lines: <br />
Note: The dfs.replication value sets how many copies of any given file
should exist in the configured HDFS site. We'll do 1 for now since there is
only one node, but again when we configure multiple nodes we'll change this.
<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code><configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
<description>Default block replication.
The actual number of replications can be specified when the file is created.
The default is used if replication is not specified in create time.
</description>
</property>
</configuration></code><property></property><description></description></pre>
Create the working directory and set permissions<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">sudo mkdir -p /fs/hadoop/tmp
sudo chown hduser:hadoop /fs/hadoop/tmp
sudo chmod 750 /fs/hadoop/tmp/
</code></pre>
Format the working directory <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">/usr/local/hadoop/bin/hadoop namenode -format</code></pre>
<br />
<h3>
Start Hadoop and Run Your First Job</h3>
Let's roll, this will be fun. <br />
Start it up: <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">cd /usr/local/hadoop
bin/start-all.sh</code></pre>
<div>
</div>
<div>
</div>
Now let's make sure it's running successfully. First we'll use the <i>jps</i>
command (Java PS) to determine what Java processes are running: <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">jps</code></pre>
You should see the following processes: (ignore the number in front, that's
a process ID) <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">4863 Jps
4003 SecondaryNameNode
4192 TaskTracker
3893 DataNode
3787 NameNode
4079 JobTracker</code></pre>
If there are any missing processes, you'll need to review logs. By default,
logs are located in: <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">/usr/local/hadoop/logs</code></pre>
There are two types of log files: .out and .log. .out files detail process
information while .log files are the logging output from the process.
Generally .log files are used for troubleshooting. The logfile name standard
is: <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">hadoop-(username)-(processtype)-(machinename).log</code></pre>
i.e.<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">hadoop-hduser-jobtracker-node1.log</code></pre>
In this, like many other articles, we'll use the included wordcount example.
For the wordcount example you will need plain text large enough to be
interesting. For this purpose, you can download plain text books from <a href="http://www.gutenberg.org/">Project
Gutenberg</a>. <br />
<br />
After downloading 1 or more books (make sure you selected "Plain Text UTF-8"
format) copy them to the local filesystem on your Hadoop node using SSH (via
<a href="http://en.wikipedia.org/wiki/Secure_copy">SCP</a> or similar). In
my example I've copied the books to <i><b>/tmp/books</b></i>. <br />
<code></code><property></property><description></description>
<br />
<pre><code><b></b></code><b></b></pre>
Now let's copy the books onto the HDFS filesystem by using the Hadoop <a href="http://hadoop.apache.org/docs/stable/file_system_shell.html">dfs
command</a>. <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">cd /usr/local/hadoop
bin/hadoop dfs -copyFromLocal /tmp/books /fs/hduser/books</code></pre>
After copying in the book(s), execute the wordcount example:<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code>bin/hadoop jar hadoop*examples*.jar wordcount /fs/hduser/books /fs/hduser/books-output</code></pre>
You'll see the job run; it may take awhile... remember these Pis don't
perform all that well. Upon completion you should see something like this: <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code>13/06/17 15:46:54 INFO mapred.JobClient: Job complete: job_201306170244_0001
13/06/17 15:46:55 INFO mapred.JobClient: Counters: 29
13/06/17 15:46:55 INFO mapred.JobClient: Job Counters
13/06/17 15:46:55 INFO mapred.JobClient: Launched reduce tasks=1
13/06/17 15:46:55 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=566320
13/06/17 15:46:55 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
13/06/17 15:46:55 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
13/06/17 15:46:55 INFO mapred.JobClient: Launched map tasks=1
13/06/17 15:46:55 INFO mapred.JobClient: Data-local map tasks=1
13/06/17 15:46:55 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=112626
13/06/17 15:46:55 INFO mapred.JobClient: File Output Format Counters
13/06/17 15:46:55 INFO mapred.JobClient: Bytes Written=229278
...</code></pre>
The results from the run, if you're curious, should be in the last part of
the command you executed earlier. (/fs/hduser/books-output in our case) Update: to check the output use the hadoop dfs command a la: <br /><pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code>hduser@rasdoop1 /usr/local/hadoop $ bin/hadoop dfs -ls /fs/hduser/books-output
Found 3 items
-rw-r--r-- 3 hduser supergroup 0 2015-05-05 05:00 /fs/hduser/books-output/_SUCCESS
drwxr-xr-x - hduser supergroup 0 2015-05-05 04:35 /fs/hduser/books-output/_logs
-rw-r--r-- 3 hduser supergroup 926451 2015-05-05 04:58 /fs/hduser/books-output/part-r-00000
hduser@rasdoop1 /usr/local/hadoop $bin/hadoop dfs -cat /fs/hduser/books-output/part-r-00000
...</code></pre>
Congrats! You just ran your first Hadoop job, welcome to the world of
(tiny)big data!<br />
<br />
<h3>
Multiple Nodes</h3>
<br />
Hadoop has been successfully run on thousands of nodes; this is where we
derive the power of the platform. The first step to getting to many nodes is
getting to 2, so let's do that. <br />
<br />
Set up the Secondary Pi<br />
<br />
I'll cover cloning a node for the third Pi, so for this one we'll assume
you've set up another Pi the same as the first using the instructions above.
After that, we just need make appropriate changes to the configuration
files. <b>Important Note</b>: I'll be referring to the nodes as
Node 1 and Node 2 and making the assumption that you have named them that.
Feel free to use different names, just make sure you substitute them in
below. Name resolution will be an issue; we'll touch on that below. <br />
<br />
Node 1 will run: <br />
<ul>
<li>NameNode</li>
<li>Secondary NameNode (In a large production cluster this would be
somewhere else)</li>
<li>DataNode</li>
<li>JobTracker (In a large production cluster this would be somewhere
else)</li>
<li>TaskTracker</li>
</ul>
Node 2 will run: <br />
<ul>
<li>DataNode</li>
<li>TaskTracker</li>
</ul>
Normally this is where I planned on writing up an overview of the
different pieces of Hadoop, but upon researching I found an article that
exceeded what I planned to write it by such a magnitude I figured why
bother, I'll just <a href="http://bradhedlund.com/2011/09/10/understanding-hadoop-clusters-and-the-network/">link
it here</a>. Excellent post by <a href="https://twitter.com/bradhedlund">Brad
Hedlund</a>, be sure to check it out. <br />
On <b>Node 1</b>, edit masters<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code>nano /usr/local/hadoop/conf/masters</code></pre>
remove "localhost" and add the first node FQDN and save the file:<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code>Node1.domain.ext</code></pre>
Note that conf/masters does NOT determine which node holds "master" roles,
i.e. NameNode, JobTracker, etc. It only defines which node will attempt to
contact the nodes in slaves (below) to initiate start-up. <br />
On <b>Node 1,</b> edit slaves<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code>nano /usr/local/hadoop/conf/slaves</code></pre>
remove "localhost" add the internal FQDN of all nodes in the cluster and
save the file:<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code>Node1.domain.ext
Node2.domain.ext</code></pre>
On <b>ALL</b> (both) cluster members edit core-site.xml<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code>nano /usr/local/hadoop/conf/core-site.xml</code></pre>
and change the fs.default.name value to reflect the location of the
NameNode:<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code><property>
<name>fs.default.name</name>
<value>hdfs://node1:54310</value>
</property></code></pre>
On <b>ALL</b> (both) cluster members edit mapred-site.xml<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code>nano /usr/local/hadoop/conf/mapred-site.xml</code></pre>
and change the mapred.job.tracker value to reflect the location of the
JobTracker:<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code><property>
<name>mapred.job.tracker</name>
<value>node1:54311</value>
</property></code></pre>
<div>
</div>
<code></code>On <b>ALL</b> (both) cluster members edit
hdfs-site.xml<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code>nano /usr/local/hadoop/conf/hdfs-site.xml</code></pre>
and change the dfs.replication value to reflect the location of the
JobTracker:<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code><property>
<name>dfs.replication</name>
<value>2</value>
</property></code></pre>
<div>
</div>
<code></code>The <b>dfs.replication value</b> determines <b>how
many copies</b> of any given piece of data will exist in the HDFS
site. By default this is set to 3 which is optimal for many small->medium
clusters. In this case we set it to the number of nodes we'll have right
now, 2. <br />
<br />
Now we need to ensure the master node can talk to the slave node(s) by
adding the master key to the authorized keys file on each node (just 1 for
now)<br />
On <b>Node 1,</b> cat your public key and copy it to the
clipboard or some other medium for transfer to the slave node<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">cat ~/.ssh/id_rsa.pub</code></pre>
On <b>Node 2</b><br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code>nano ~/.ssh/authorized_keys</code></pre>
and <b>paste in</b> the key from the master and <b>save the
file</b> to allow it to take commands. <br />
<br />
<h3>
Name Lookup</h3>
The Hadoop nodes should be able to resolve the names of the nodes in the
cluster. To accomplish this we have two options: the right way and the quick
way. <br />
<br />
<b>The right way: DNS</b><br />
<br />
The best way to enable name lookup works correctly is by adding the nodes
and their IPs to your internal DNS. Not only does this facilitate standard
Hadoop operation, but it makes it possible to browse around the Hadoop
status web pages (more on that below) from a non-node member, which I
suspect you'll want to do. Those sites are automatically created by Hadoop
all the hyperlinks use the node DNS names, so you need to be able to resolve
those names from a "client" machine. This can scale too; with the right tool
set it's very easy to automate DNS updates when spinning up a node. <br />
<br />
To do this option, add your Hadoop hostnames to the DNS zone they reside in.
<br />
<br />
<b>The quick way: HOSTS</b><br />
<br />
If you don't have access to update your internal DNS zone, you'll need to
use the hosts files on the nodes. This will allow the nodes to talk to each
other via name. There are scenarios where you may want to automate the
updating of node based HOSTS files for performance or fault tolerance
reasons as well, but I won't go into that here. <br />
<br />
To use this option do the following: <br />
<br />
On <b>ALL</b> (both) cluster members edit /etc/hosts<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code>nano /etc/hosts</code></pre>
and add each of your nodes with the appropriate IP addresses (mine are only
examples)<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code>192.168.1.40 node1
192.168.1.41 node2</code></pre>
Now that the configuration is done we need to format the HDFS filesystem. <b>Note:
THIS IS DESTRUCTIVE</b>. Anything currently on your single node
Hadoop system will be deleted.<br />
On <b>Node 1</b><br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code>bin/hadoop namenode -format</code></pre>
When that is done, let's start the cluster: <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">cd /usr/local/hadoop
bin/start-all.sh</code></pre>
Just as before, use the jps command to check processes, but this time on
both nodes. <br />
On <b>Node 1</b><br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">jps</code></pre>
You should see the following processes: (ignore the number in front, that's
a process ID) <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">4863 Jps
4003 SecondaryNameNode
4192 TaskTracker
3893 DataNode
3787 NameNode
4079 JobTracker</code></pre>
On <b>Node 2</b><br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">jps</code></pre>
You should see the following processes: (ignore the number in front, that's
a process ID) <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">6365 TaskTracker
7248 Jps
6279 DataNode
</code></pre>
If there are problems review the logs. (for detailed location see
reference above) Note that by default each node contains its own logs. <br />
<br />
Now let's run the wordcount example in a cluster. To facilitate this you'll
need enough data to split, so when downloading UTF-8 books from <a href="http://www.gutenberg.org/">Project
Gutenberg</a> get at least 6 very long books (that should do with default
settings). <br />
<br />
Copy them to the local filesystem on your master Hadoop node using SSH (via
<a href="http://en.wikipedia.org/wiki/Secure_copy">SCP</a> or similar). In
my example I've copied the books to <i><b>/tmp/books</b></i>. <br />
<code></code><property></property><description></description>
<br />
<pre><code><b></b></code><b></b></pre>
Now let's copy the books onto the HDFS filesystem by using the Hadoop <a href="http://hadoop.apache.org/docs/stable/file_system_shell.html">dfs
command</a>. This will automatically replicate data to all nodes. <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">cd /usr/local/hadoop
bin/hadoop dfs -copyFromLocal /tmp/books /fs/hduser/books</code></pre>
After copying in the book(s) wait a couple minutes for all blocks to
replicate (more on monitoring below) then execute the wordcount example:<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code>bin/hadoop jar hadoop*examples*.jar wordcount /fs/hduser/books /fs/hduser/books-output</code></pre>
<br />
<h3>
Monitoring HDFS/Jobs</h3>
<br />
Hadoop includes a great set of management web sites that will allow you to
monitor jobs, check logfiles, browse the filesystem, etc. Let's take a
moment to examine the three sites. <br />
<br />
<b>JobTracker</b><br />
By default, the job tracker site can be found at <b>http://(jobtrackernodename.domain.ext):50030/jobtracker.jsp</b>
; i.e. http://node1.company.com:50030/jobtracker.jsp<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEifhEVgDCMedostArWkkVw4DgsIfjm6Biowoqnwn1esmx3GBVGAoHlZWcngGlf_16Ie9TcAr5hU_qUbhVKjwKlImQiLgvVdwibyRaQOGCXphSAbE7BQBckPRbcza7IhzMp9oKo2jSKqf3c/s1600/2013-08-07+15_01_00-Program+Manager.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="547" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEifhEVgDCMedostArWkkVw4DgsIfjm6Biowoqnwn1esmx3GBVGAoHlZWcngGlf_16Ie9TcAr5hU_qUbhVKjwKlImQiLgvVdwibyRaQOGCXphSAbE7BQBckPRbcza7IhzMp9oKo2jSKqf3c/s640/2013-08-07+15_01_00-Program+Manager.png" width="640" /></a></div>
<br />
<br />
On the Job Tracker you can see information regarding the cluster, running
map and reduce tasks, node status, running and completed jobs, and you can
also drill into specific node and task status. <br />
<br />
<b>DFSHealth</b><br />
By default, the DFSHealth site can be found at <b>http://(NameNodename.domain.ext):50070/dfshealth.jsp</b>
; i.e. http://node1.company.com:50070/dfshealth.jsp<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4_0IX5GDtTAH7aGZ3bdvmASvuAzcpcpY2rxZl3t03NlXOlNjejmXUg2T_-XspJP-yVHnMc0YXkhn3Ljp49dkqAq9XM7dbGNtmJvLUuyZVou53h2x9F0evoazFzCTeoF5iKeLw5L2UDU4/s1600/2013-08-07+14_59_52-Program+Manager.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="548" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4_0IX5GDtTAH7aGZ3bdvmASvuAzcpcpY2rxZl3t03NlXOlNjejmXUg2T_-XspJP-yVHnMc0YXkhn3Ljp49dkqAq9XM7dbGNtmJvLUuyZVou53h2x9F0evoazFzCTeoF5iKeLw5L2UDU4/s640/2013-08-07+14_59_52-Program+Manager.png" width="640" /></a></div>
<br />
The DFSHealth site allows you to browse the HDFS system, view nodes, look at
space consumption, and check NameNode logs. <br />
<br />
<b>TaskTracker</b><br />
By default, the task tracker site can be found at <b>http://(nodename.domain.ext):50060/tasktracker.jsp</b>
; i.e. http://node2.company.com:50060/tasktracker.jsp<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixQQ9IBuHF1Q5ESsAtuaQjOlhyphenhyphenn1pgeohVECupbXCFhovSlNzhfydnJZisB4ZzmPM1V48-p7d2IKvHvkyJB8pGEjBAz3kCUoUwuRNY-JMD-LS_0nh_w43LijofbquD9hETQiVQkymdD7s/s1600/2013-08-07+14_59_17-Program+Manager.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="546" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixQQ9IBuHF1Q5ESsAtuaQjOlhyphenhyphenn1pgeohVECupbXCFhovSlNzhfydnJZisB4ZzmPM1V48-p7d2IKvHvkyJB8pGEjBAz3kCUoUwuRNY-JMD-LS_0nh_w43LijofbquD9hETQiVQkymdD7s/s640/2013-08-07+14_59_17-Program+Manager.png" width="640" /></a></div>
<br />
<br />
The Task Tracker site runs on each node that can run a task to provide more
detailed information about what task is running on that node at the time.<br />
<br />
<br />
<h3>
Cloning a Node and Adding it to the Cluster</h3>
<br />
Now we'll explore how to get our third node into the cluster the quickest
way possible: cloning the drive. This will cover, at a high level, the SD
card cloning process for the Pi. If you're using a real machine replace the
drive cloning steps with your favorite (*cough*Clonzilla*cough*) cloning
software, or copy the virtual drive if you are using VMs. <br />
<br />
You should always clone a slave node unless you intend on making a new
cluster. To this end, do the following:<br />
<br />
<ol>
<li>Shut down <b>Node 2</b> (<i>sudo init 0</i>) and remove
the SD card. </li>
<li>Insert the SD card into a PC to so we can clone it; I'm using a
Windows 7 machine so if you're using Linux we'll diverge here. (I'll
meet you at the pass!)</li>
<li>Capture an image to your hdd using your favorite SDcard imaging
software. I use either <a href="http://www.roadkil.net/program.php?ProgramID=12">Roadkill's
Disk Image</a> or <a href="http://www.chrysocome.net/dd">dd for
windows</a>. </li>
<li>Remove the Node 2 SD card and place it back into Node 2. Do not power
up yet.</li>
<li>Insert a new SD card of the same size to your imaging PC.</li>
<li>Delete any partitions on the card to ensure error-free imaging. I use
<i>Diskpart</i> , <i>Select Disk x</i> , <i>Clean</i> , <i>exit</i>
where "x" is the disk number. Use <i>list disk</i> to find it...
don't screw up or you'll wipe out your HDD!</li>
<li>Image the new SD card with the image from Node 2 and remove it from
the PC when complete. </li>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiLQ4yaU3jaK3dfF1ZoNL3WMH8bYtTISU7eozh3FMPesQASStHjtlqG2YQmr5P3efG3jP5yaT5AyfihbuXlkzhsmcd1Y3LeCYaeSR1NpYX8r1L0cTgEsPSKDjoBeCPD0Nf0wuqmn4aDK4Q/s1600/2013-08-07+20_51_51-.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="326" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiLQ4yaU3jaK3dfF1ZoNL3WMH8bYtTISU7eozh3FMPesQASStHjtlqG2YQmr5P3efG3jP5yaT5AyfihbuXlkzhsmcd1Y3LeCYaeSR1NpYX8r1L0cTgEsPSKDjoBeCPD0Nf0wuqmn4aDK4Q/s400/2013-08-07+20_51_51-.png" width="400" /></a></div>
<br />
<li>Insert the newly cloned SD card to Node 3. Power up that node and
leave Node 2 off for now or the IPs will conflict. </li>
</ol>
<br />
Log into the new node (same IP as the cloned node) via ssh with the hduser
(Hadoop user) account and perform the following tasks: <br />
<br />
Change the hostname:<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">sudo nano /etc/hostname</code></pre>
For this example change "Node2" to <b></b>"<b>Node3</b>"
and save the file. <br />
<br />
Change the IP address:<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">sudo nano /etc/network/interfaces</code></pre>
For this example change "192.168.1.41" to "<b>192.168.1.42</b>"
and save the file. Switch to your own IP address range if necessary. <br />
<br />
If you're using HOSTS for name resolution, add Node 3 with the proper IP
address to the hosts file<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">sudo nano /etc/hosts</code></pre>
<b> Repeat this on the master node</b> and for the sake of
completeness the other node(s) should probably be updated as well. In a
large scale deployment this would all be automated. <br />
If you are using DNS, make sure to add the Node3 entry to your zone. <br />
<br />
Reboot Node 3 to enact the IP and hostname changes<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">sudo reboot</code></pre>
Power up Node 2 again at this time if you had it powered down. Log back into
Node 3 (now with the new IP!) with the hduser account via SSH. <br />
<br />
Generate a new key:<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">ssh-keygen -t rsa -P ""</code>
</pre>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjSosp3DPiwW_qJm4M7Zs9Uy2EIS5jpQ0OdcYb1tbZj2vkif-jeyycTKspyp77e9tDDK7r_G8G6dBx0EX2PXupRK_TDm8ufw-jn8YUD0iWKGf3uiwQ-sFrJdYgdCn85UsIvVpLeI5ej9w0/s1600/2013-08-07+21_34_34-Raspberry+Pi+Hadoop+Compute+Cluster+%5Bfile__..._Raspberry+Pi+Hadoop+Compute+Clust.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="432" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjSosp3DPiwW_qJm4M7Zs9Uy2EIS5jpQ0OdcYb1tbZj2vkif-jeyycTKspyp77e9tDDK7r_G8G6dBx0EX2PXupRK_TDm8ufw-jn8YUD0iWKGf3uiwQ-sFrJdYgdCn85UsIvVpLeI5ej9w0/s640/2013-08-07+21_34_34-Raspberry+Pi+Hadoop+Compute+Cluster+%5Bfile__..._Raspberry+Pi+Hadoop+Compute+Clust.png" width="640" /></a></div>
Copy the key to its own trusted store:<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys</code></pre>
<br />
If you aren't already, log into Node1 (Master) as hduser via SSH and do the
following:<br />
<b>Add Node 3</b> to the slaves list<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">nano /usr/local/hadoop/conf/slaves</code></pre>
<br />
SSH to Node 3 to add it to known hosts<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;">ssh rasdoop3</pre>
When prompted re: continue connecting type <b>yes</b><br />
<br />
An <b>ALL</b> nodes edit hdfs-site.xml<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;">nano /usr/local/hadoop/conf/hdfs-site.xml</pre>
Change it to replicate to 3 nodes. Note if you add more nodes to the cluster
after this you would most likely NOT increase this above 3. It's not 1:1
that we're going for, it's the number of replicas we want in the entire
cluster. Once you're running a cluster with > 3 nodes you'll generally
know if you want dfs.replication set to more than 3. <br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code><configuration>
<property>
<name>dfs.replication</name>
<value>3</value>
<description>Default block replication.
The actual number of replications can be specified when the file is created.
The default is used if replication is not specified in create time.
</description>
</property>
</configuration></code>
</pre>
<br />
Now we need to format the filesystem. It is possible, but slightly more
complicated, to clone/add a node without re-formatting the whole HDFS
cluster, but that's a topic for another blog post. Since we have virtually
no data here, we'll address all concerns by wiping clean: <br />
On <b>Node 1</b>:<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code>cd /usr/local/hadoop
bin/hadoop namenode -format</code></pre>
<br />
Now we should be able to start up all three nodes. Let's give it a shot!<br />
On <b>Node 1</b>:<br />
<pre style="background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code>bin/start-all.sh</code></pre>
<br />
<b>Note</b>: If you have issues starting up the Datanodes you may
need to delete the sub-directories under the directory listed in
core-site.xml as "<i>hadoop.tmp.dir</i>" and then re-format again (see
above). <br />
<br />
As for startup troubleshooting and running the wordcount example, copy out
the instructions above after the initial cluster setup. There should be
nothing different from running with two nodes save the fact that you may
need more books to get it working on all 3 nodes. <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgEDlKbWTfh5RsVs1GuTS70g9KvdkgmQn0egsq0Qlb42gTlG4cWZPa84MhUKkBHboqeZjfDm3aws7EDnnTHluFmibzs-FdN4POIDAZsEg2jqrjk9_cZ9jv0gAhaY8nKSCGVAcJkkIDJ2A0/s1600/HadoopChurn.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="458" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgEDlKbWTfh5RsVs1GuTS70g9KvdkgmQn0egsq0Qlb42gTlG4cWZPa84MhUKkBHboqeZjfDm3aws7EDnnTHluFmibzs-FdN4POIDAZsEg2jqrjk9_cZ9jv0gAhaY8nKSCGVAcJkkIDJ2A0/s640/HadoopChurn.png" width="640" /></a></div>
<br />
<h3>
Postmortem/Additional Reading</h3>
You did it! You've now got the experience of setting up a Hadoop cluster
under your belt. There's a ton of directions to go from here; this
relatively new tech is changing the way a lot of companies view data and
business intelligence. <br />
<br />
There is so much great community content out there; here's just a small list
of the references I used and some additional reading. <br />
<h3>
<code>References: </code></h3>
<a href="http://www.raspberrypi.org/phpBB3/viewtopic.php?f=41&t=37190">Raspberry
Pi forums: Hadoop + HDFS + MR on Pi cluster - works!</a><br />
<a href="http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/">Michael
G. Noll: Running Hadoop on Ubuntu Linux Single-Node Cluster</a><br />
<a href="http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/">Michael
G. Noll: Running Hadoop on Ubuntu Linux Multi-Note Cluster</a><br />
<a href="http://wiki.apache.org/hadoop/GettingStartedWithHadoop">Hadoop
Wiki: Getting Started with Hadoop</a><br />
<a href="http://wiki.apache.org/hadoop/WordCount">Hadoop Wiki: Wordcount
Example</a><br />
<a href="http://raspberrypicloud.wordpress.com/2013/04/25/getting-hadoop-to-run-on-the-raspberry-pi/">University
of Glasgow's Raspberry Pi Hadoop Project</a><br />
<a href="http://hadoop.apache.org/docs/stable/cluster_setup.html">Hadoop
Wiki: Cluster Setup</a><br />
<a href="http://wiki.apache.org/hadoop/HadoopJavaVersions">Hadoop Wiki:
Java Versions</a><br />
<a href="http://allthingshadoop.com/2010/04/28/map-reduce-tips-tricks-your-first-real-cluster/">All
Things Hadoop: Tips, Tricks and Pointers When Setting Up..</a><br />
<a href="http://wiki.apache.org/hadoop/FAQ#A17">Hadoop Wiki: FAQ</a><br />
<a href="http://code.google.com/p/hadoop-toolkit/wiki/HadoopPerformanceMonitoring">Code/Google
Hadoop Toolkit: Hadoop Performance Monitoring</a><br />
<a href="http://www.jeremymorgan.com/tutorials/raspberry-pi/how-to-overclock-raspberry-pi/">Jeremy
Morgan: How to Overclock the Raspberry Pi</a><br />
<div class="separator" style="clear: both; text-align: center;">
</div>
Toby Meyerhttp://www.blogger.com/profile/11479868060801724272noreply@blogger.com70