Do you have multi-continent datacenters with gobs of bandwidth, IOps, and
processing capability? No? I can help get you part of the way there... a
network presence in one.
 |
| A tunnel. |
A Site to Site Connection?
It's easier to think of this as an extension to your network into another
datacenter over the internet. Using
IPsec
we can provide a relatively (comments at the end) secure, direct connection
between on on-premises datacenter and
Azure
hosted resources by encrypting the traffic that flows between the two.
What do I mean by:
- Secure: IPsec tunnels all your traffic so it is
encrypted over the internet; in reality, this is really "more secure"
rather than definitively secure, as the effective security depends
highly on implementation specifics.
- Direct: Your router (played by pfSense in this case)
will recognize the Azure site as another routable network within the
boundaries of your own, enabling you to talk to Azure resources as if
they were in your own datacenter.
While I'll be using pfSense for the initiator side as it exposes the
options in the most clear way I've found, this article will also be
useful for non-pfSense devices since we discuss the details of
the IPsec tunnel; the information here should be applicable to any IPsec
solution. Update 1/2017: I've personally tested on
various Cisco, Sonicwall, and pFsense equipment, and Microsoft has added
some great documentation about overall device support here.
Note: this works for Amazon Web Services (AWS)
as well but is slightly more complex. Fortunately pfSense includes a wizard
that works, but takes a lot of the fun out of it as it strips you of
understanding how it works. In addition the wizard is necessary because of
how Amazon does VPC
routing, whereas Azure is a bit more straightforward.
With that, let's get to it!
Pre-Requisites
- pfSense firewall(s): The steps in this article were
performed on a pair of HA SG-4860 firewalls running pfSense
2.32p1.
- Microsoft Azure account with adequate permissions:
We'll be performing our actions using the "new" portal
based on Azure Resource Manager (ARM
or AzureRM).
- AzureRM PowerShell Cmdlets installed: On Win10/Server
2016 this can be accomplished with Install-Module AzureRM; for
more info see this
post.
Configure Azure IPSec Endpoint
Before we set up and initiate the connection from pfSense, we need to set up
our endpoint in Azure. To do so, we'll create the following objects:
- A Resource Group
- A Public IP Address
- A Virtual Network
- A Gateway Subnet
- A Virtual Network Gateway
- A Local Network Gateway with a Connection
Resource Group
A
Resource Group is a logical grouping of Azure
Resources.
This logical group allows for easy organization and clearer billing reports.
We won't get too much into concepts and naming standards here other than to
say groups should be logically tied with similar lifecycle expectancies and
you should be
consistent. For more information, see
this
Azure article.
Note: We'll be doing most of our steps in the web portal,
but this whole process is much more efficient with PowerShell.
Let's go!
- Open and log into the ARM Azure Portal at portal.azure.com;
ensure you're working with the subscription
you intend to use.
- Navigate to "Resource Groups".
- Click "Add" and type a name for your resource group, select
the subscription, and resource group location. Note: The resource group
location has no bearing on where you'll be connecting to as it's just a
location the metadata is stored.
- Click "Save".
Public IP Address
While we could do the IP at the time we make the Virtual Network Gateway,
we'll take care of it now to ensure it's provisioned prior to getting to
that step and to discuss the IP details.
- Navigate to "Public IP addresses".
- Click "Add" and populate the following:
- Name: Select a name leveraging consistent naming standards.
- IP address assignment: Select "Dynamic".
I know what you're saying.. you're saying "but Toby, I'm not saying
anything", and what I'm saying is it seems this should be static.
Unfortunately if we make a static IP we'll be greeted later with the
following:
 |
| Why Azure,
WHY?! |
Now I've been running a tunnel straight for almost a month thus far
and my dynamic IP has not shifted on me; I suspect it will behave the
same as IPs for other resources and stay static so long as it is used.
If this does change, you'll need to change the info in the Phase 1 and
2 setup of the tunnel on the pfSense side as outlined below. For the
record, as of the writing of this article the pricing of IPs in Azure
is a bit odd; dynamic IPs and static IPs beyond the first 5 in any
region are charged the same (pretty trivial), while the first 5 static
in a region are free. See here
for more info.
- Idle Timeout: The default of 4 minutes should be fine here.
- DNS Name Label: Optionally, specify a DNS alias here,
though we will not reference it again in this guide as I'm not
addressing DNS issues associated with IPsec at this time.
- Subscription: Select your subscription.
- Resource Group: Click the resource group we created in the
last step.
- Location: The IP is our IPsec target, so select a location
close to your local network connection. The Azure
Speed Test comes in quite handy here.
- Click "Create". This provisioning will take a few minutes
minutes as Azure re-arranges its SDN
infrastructure to give you an IP.
 |
| What a successful
deployment looks like! |
Virtual Network/Subnet/Gateway Subnet
A "
Virtual
Network" is a network space within Azure that you can carve up
and protect (firewall) to suit your needs. We're required to make one
subnet, and we'll create our "gateway subnet" (landing point) as well. If
this is your first foray into virtual networks on Azure you may want to take
a step back and
consider
your design before proceeding. Oh, you're back already? Let's go.
- Navigate to "Virtual Networks".
- Click "Add" and supply the following:
- Name: Type a name for your Virtual Network; you should
follow the naming standards as discussed above.
- Address Space: This is the overall space for your logical
network within Azure. You can create more granular subnets within this
space at any time, so erring on the side of a large subnet would be
wise. If you're unsure, use 10.1.0.0/16.
- Subnet Name: You're required to create one subnet within
your virtual network off the bat. You need to name it here and ensure
you use a consistent and meaningful naming standard.
- Subnet Address Range: Specify a subnet range within your
virtual network. This won't be used by our IPsec connection directly,
but we will use it later as a target for testing. If unsure, use
10.1.10.0/24.
- Subscription: Select your desired subscription.
- Resource Group: Select "Use Existing" and select the
resource group we created earlier.
- Location: Select the same location used for the IP
above.
- After the Virtual Network has been created (use the refresh key if
necessary), click it to navigate to the next pane, and then click "Subnets".
- On the next pane, click "+ Gateway Subnet". And specify a
subnet in "Address Range". This subnet needs to be different
than the one we created earlier and should not be used
for non-network resources, but rather as an ingress point to your
Virtual Network. If unsure, use 10.1.0.0/24.
Virtual Network Gateway
The "
Virtual
Network Gateway" is our configuration element that facilitates the
IPsec tunnel. Microsoft refers to this as a "VPN" gateway (as opposed to
Express
Route). There are three different VPN gateway SKUs; we'll be doing the
"Standard" offering (of Basic, Standard, High-Performance). It's worth
having a read about the differences
here.
- Navigate to "Virtual Network Gateways".
- Click "Add" and supply the following:
- Name: Again, follow consistent naming standards.
- Gateway Type: Select "VPN".
- VPN type: Select "Route-Based" (packets routed by
routing table) in this case; it would be advisable to familiarize
yourself with the difference between route and policy here.
Note that policy requires IKEv1, so if you need to use it note the
settings will be quite a bit different.
- SKU: Update 1/2018: The SKU selection is
now VpnGw<x> or Basic. Note you cannot change
a basic VNG to the higher tier (VpnGwX) or vice versa at a later time.
For more information see this
article.
- Virtual Network: Select the virtual network we created in
the last step.
- Public IP address: Select the IP address we created
earlier.
- Click "Create".
Note: This step may take up to 45 minutes to complete
provisioning. I've tracked 8 of these and it's averaging almost 40 minutes
per regardless of if you pre-provision the IP or not. You may want to
consider skipping ahead to the pfSense section for a bit and coming back
here.
Local Network Gateway/Connection
A Local Network Gateway is the specification of our local IP and networks
you would like to route over the tunnel.
This actually works fine with a dynamic IP if that is your scenario, but
we'll cover the details of that later.
- Navigate to "Local Network Gateways".
- Click "Add" and supply the following:
- Name: Naming? Standards? Consistent? Yeah!
- IP Address: Enter the public IP address of your device that
will instantiate the tunnel.
- Address Space: This is where you enter the CIDR notation of
the local networks you would like to route over the tunnel... for
example, if you would like to route 192.168.1.x over the tunnel, then
enter "192.168.1.0/24"
- Subscription: Enter your desired subscription.
- Resource Group: Select our resource group we created above.
- Location: For consistency, select the same location as you
have selected above.
- Update 1/2018: You can configure BGP settings here
now as well, cool eh?
- Click "Create".
- After provisioning, (you may need to hit "refresh) click your newly
created Local Network Gateway and click "Connections".
- On the newly expanded pane, click "Add" and supply the
following:
- Name: You know the drill by now.
- Connection type: This should be fixed to "Site to Site
(IPsec)"
- Virtual Network Gateway: Enter the Virtual Network Gateway
we entered in the step above.
- Shared Key: Specify a unique, randomly generated passphrase
comprised of alphanumeric characters. Some devices have issues with
special characters, hence the omission of. I recommend using at least
30 characters; since it has no impact on tunnel performance I
personally use at least 60 characters for each key. You'll need to
specify this key on your local side as well.
- Subscription: This should be hard coded to the same
subscription as the LNG.
- Resource Group: This also should be locked to the same
resource group as the LNG.
- Location: Locked to that of the LNG.
- Click "OK".
Configure pfSense
Now we'll set up the IPsec initiator connection on your pfSense firewall(s).
Phase 1 Setup
- Login to the firewall and navigate to "VPN->IPsec"
- Click "Add" and specify the following:
- Key Exchange Version: Auto
- Internet Protocol: IPv4
- Interface: Select the WAN interface from which you would
like to instantiate the connection
- Remote Gateway: Enter the Azure public IP address created
in the "Public IP Address" section above
- Description: Whatever you would like; maybe troll your
firewall team with a message here for fun times.
- Authentication Method: Mutual PSK
- Negotiation Mode: Main Note: Do not use
"Aggressive" mode as the hash of the PSK is sent over the internet in
clear text.
- My Identifier: If the WAN interface selected above holds
your public IP address, you can select "My IP Address". If
that interface lies behind another edge device that holds the public
IP, you'll need to select "IP Address" and specify your
external IP.
- Peer Identifier: Peer IP Address
- Pre-Shared Key: Enter the same Pre-Shared Key used in the
Azure connection specification above.
- Encryption Algorithm: The strongest available in Azure is AES
256 bit, so preferably specify that. For more information on
supported features in Azure, see the References section below.
- Hashing Algorithm: The best we can do here is SHA256,
so let's go with that.
- DH Group: 2(1024 bit)
- Lifetime (seconds): 10800
- Disable Rekey: Unchecked
- Responder Only: Unchecked
- NAT Traversal: Auto (Even in NAT scenarios Auto usually
works)
- Dead Peer Detection: Checked
- Delay: 10
- Max Failures: 5
- Click "Save" to return to the "VPN->IPsec" menu.
- Since we don't want to use this yet, click "Disable" in front
of the new tunnel definition and then "Apply Changes".
Phase 2 Setup
- Under our newly created tunnel definition, click "Show Phase 2
Entries"
- Click "Add P2" and supply the following information:
- Disabled: Unchecked
- Mode: Tunnel IPv4
- Local Network: Select "Network" and specify the
same network range(s) that you specified during the set up of the
local network gateway on Azure using CIDR notation, i.e.
192.168.1.0/24. This specifies which local network(s) you would like
to route through the tunnel.
- NAT/BINAT translation: None ; Note: even
in scenarios where your pfSense device is using NAT behind an upstream
router, this should not be necessary. NAT-T will take care of that
scenario.
- Remote Network: Select "Network" and specify the
same network range(s) that you specified during the set up of the
target virtual network in Azure using CIDR notation, i.e. 10.1.0.0/16.
This specifies the remote network(s) present in Azure.
- Description: Put something here to help you remember what
all this fun stuff is about.
- Protocol: ESP
- Encryption Algorithms: Check only "AES" and "256
bits".
- Hash Algorithms: Unfortunately Azure only
supports "SHA1" at this time. Update 1/2017: SHA256
supported now, use that!
- PFS Key Group: Azure
documentation states that PFS
groups are only supported when Azure acts as responder, and in this
case it is being set up as the initiator. Oddly, I've actually had
luck specifying DH Group 14, but there is no guarantee that will work.
I'm going to stick with it but for this by the book exercise you'll
need to select "off". Note: Because of this
setting and the prior Hash Algorithm setting, I do not consider this
tunnel secure against state-level or similarly equipped actors. If
that is a concern you may wish to investigate alternatives. In less
extreme cases, however, this can be considered relatively secure.
Update 1/2017: The compatibility has been improved here as
well; match your Encryption/Auth with the right group using the table
here.
- Lifetime: 3600
- Automatically ping host: blank
- Click "Save".
Note: Depending on your configuration it may be
necessary to navigate to "
VPN->IPsec->Advanced Settings" and
check "
Enable Maximum MSS", then specify
1350. If you get
packet loss with large packets this setting may be needed.
Firewall Rules
Now that our tunnel is set up we have to create local firewall rules that
allow for traffic to pass. First we'll create a network alias for the
Azure side network and then we'll make a rule to allow out Azure based
traffic to pass here.
- Navigate to "Firewall->Aliases"
- Click "Add" and supply the following:
- Name: Supply something that explains this network is to
represent the Azure side of the tunnel; only alphanumeric and "_" are
allowed.
- Description: Enter full description here; there are no
special character limitations.
- Type: "Network(s)"
- Network(s): Enter the CIDR notation of the network you
created for your Virtual Network in Azure. If you followed the example
addresses in this article, that would be 10.1.0.0/16.
- Click "Save" and then "Apply Changes".
- Navigate to "Firewall->Rules->IPsec".
- Click "Add -^" and supply the following:
- Action: "Pass"
- Disable this rule: Unchecked
- Interface: "IPsec"
- Address Family: "IPv4"
- Protocol: "Any" Discussion: Feel
free to limit the traffic that goes through the tunnel if you like. In
this example I'm allowing all traffic through.
- Source: "Single host or alias" and then specify
the Azure network alias you created in step 2.
- Destination: This needs to be the local network(s) to which
you would like to allow traffic. You can either do "network"
with a CIDR notation or specify the entire network represented by an
interface on the firewall. Note: if you have
multiple networks you'll need a rule for each, so repeat the last
couple steps for each.
- Log: Unchecked; keep in mind that should you need to
troubleshoot temporarily logging traffic using this rule can be very
useful.
- Description: It's a description, so let's do that!
- No advanced options necessary unless you would like to do so.
- Click "Save" and then "Apply Changes".
As long as you have a blanket egress traffic rule we should now be able to
route traffic over the tunnel. If you do not I expect you are aware of how
to make a more specific rule to suit your needs.
A Note on NAT-T and Upstream Routers
If your pfSense device is behind another upstream router, you may need some
changes to facilitate the port switchover after initialization. If this
matches your configuration, consider that you may need the following on the
upstream router:
- A firewall rule that allows UDP port 4500 into your
pfSense device(s).
- A NAT port mapping rule that forwards UDP port 4500
to your pfSense device(s).
Try without first; some devices are aware enough of the switch to 4500 to
perform the transition without rules, but if it does not work consult the
documentation for the device in question.
Enable and Test
There are several ways to test our connection; in this case I'll be pinging
a VM host in Azure assigned to the same virtual network that this tunnel is
connecting to. We won't go through the provisioning of that; should you need
to refer to
this
basic guide and ensure you place the VM in your target virtual network
and initially created subnet (10.1.10.x in the example above).
 |
| We're about to go
into a tunnel... a long one. |
Preparing Your Target
- Ensure your VM is up and provisioned in the correct target virtual
network.
- Since you can't put anything in the gateway subnet (correctly) this
would be a good opportunity to put the VM in the subnet you were forced
to create when creating the virtual network in the first place.
Check/change VM->Network
interfaces->Details->Settings->IP Configurations
- Get the private IP address from VM->Network interfaces. It
should be 10.1.10.x if you're following the example addresses in this
article.
- Make sure your VM is pingable! If you have instituted Network
Security Groups that would inhibit access you'll need to modify
them, though this should work by default since we're tunneled in. Make
sure the firewall on the VM allows for incoming ICMP requests as well;
on Win 2012 and higher set-netfirewallrule -DisplayName "File and
Printer Sharing (Echo Request - ICMPv4-In)" -Enabled True will
take care of you.
Bring Up The Tunnel
- In the pfSense interface, navigate to VPN->IPsec.
- In front of our new tunnel, click "Enable" then "Apply"
toward the top.
- Check tunnel status under Status->IPsec. The tunnel
should come up automatically in about a minute. If there is trouble you
can check the Status->System Logs->IPsec section for
more details.
Check Tunnel Status in Azure & Ping Dat VM!
For this portion we'll use PowerShell; ensure you have the Azure ARM
cmdlets installed. If not, give install-module AzureRM a shot
from an elevated PowerShell prompt.
- Login to your account: Login-AzureRmAccount
- Look at your subscriptions and grab the name of the target sub: Get-AzureRmSubscription
- Change to your correct subscription: Select-AzureRmSubscription
-SubscriptionName <subscription name>
- Check the status: Get-AzureRmVirtualNetworkGatewayConnection
-name <Local Gateway Connection> -ResourceGroupName <Name of
Resourcegroup to which it belongs>
- On the output pane, check the "ConnectionStatus" property. It
should be "Connected".
The Get-AzureRmVirtualNetworkGatewayConnection has a series of other
interesting properties as well, including EgressBytesTransferred and
IngressBytesTransferred.
Now proceed to ping your VM by the private IP listed
in Azure. As long as everything is configured correctly you should receive
a response!
Cost
VPN Tunnels are subject to a costs from a few different categories:
- VPN
Gateway Pricing: This is an hourly cost incurred while
the tunnel is available, not necessarily used. This means
once it's provisioned you will incur charges at the hourly rate. As of
the writing the standard performance level that we'll be using is billed
at $0.19/hr in the US. If you have multiple Virtual Networks you will
also be subject to a fee for outgoing traffic destined for another VNet.
This rate depends on the zone and varies between $.035 and $.16 per GB.
Data outbound to your site is charged at the standard data transfer
rates (below) and inbound data is free. Update 1/2017:
Updated pricing for the new tiers can be found here.
- Data
Transfer Rates: This depends on your level of utilization. The
first 5GB of outgoing/month is free and the prices are set on a curve
thereafter.
- Virtual
Network Pricing: VNets are free; you can have up to 50 VNets per
subscription across all regions.
- IP
Addresses: You'll be using at least one IP address. The first 5
static in a given region are free, additional and dynamic are charged at
a rate of $.004/hr.
Overall the cost of a "standard" class data tunnel each month for a
single IP address, no additional support, and without including
outgoing bandwidth, is about $140/month.
Note: Costs as of 10/26/2016, subject to change. Up to
date pricing information is available here.
Dynamic IP? Changing Your IP Address
There is no reason that this IPsec tunnel will not work without a dnymic IP,
but each time the IP changes you'll need to take a series of steps to
restore tunnel functionality. These are:
- In Azure, the local network gateway specifies your IP, change it under
"Local network gateway-><NAME>->Configuration->IP
address".
- In pfSense, the Phase 1 tunnel definition under "My identifier"
needs to reflect your current external IP address.
- If there are any implications on upstream routers you'll need to
handle that as well.
After taking care of these you will need to restart the tunnel. This
could all be automated with PowerShell and SSH if you like, but I won't be
covering that here.
Update 1/2017: FWIW, over the last year none of my
clients have had their IP rotate for an active tunnel.
A Note on Effective Security
As mentioned earlier, this set up does have a couple security issues; the
impact of which I would like to discuss briefly. Without an optimal security
configuration, including the support of Perfect Forward Secrecy, this tunnel
may not be strong enough to stand up to attacks of a state-sponsored actor
over a long period of time. Because of that I cannot recommend this
solution if your traffic may be subject to that level of attack, for example
traffic facilitating substantial financial transaction activity.
Update
1/2017: As noted above, this situation has improved with the
support of PFS and SHA256 for authentication.
With that said, this tunnel is still (for better or worse) more secure than
the configuration I have seen at many clients, and should be suitable for
most traffic. It also performs very well; added latency between my modestly
equipped pfSense devices and Azure is trivial.
 |
| For science! |
References
Azure
Microsoft:
Azure Virtual Network Overview
Microsoft:
Azure Resource Manager Cmdlets (PowerShell)
Microsoft:
Create a VNet with a Site-to-Site connection using the Azure portal
Microsoft:
Create a VNet with a Site-to-Site connection using PowerShell
Steve
Buchanan: Azure & RRAS Site to Site VPN Setup (Azure Resource Manager)
Microsoft:
What is a Network Security Group?
Microsoft:
VPN Gateway FAQ
Microsoft:
About VPN Gateway
Microsoft:
How to install and configure Azure PowerShell
Microsoft:
About VPN Devices for Site-to-Site VPN Gateway Connections (Critical
article; contains IPsec specs of Azure side)
pfSense
pfSense
Doco: Routing internet traffic through a site-to-site IPsec tunnel
pfSense
Doco: VPN Capability IPsec
IPsec
PFS RFC
strongSwan
Wiki: Security recommendations
Johannes
Webber: Considerations About IPsec Pre-Shared Keys
Cisco
Support: How Does NAT-T Work With IPsec (Only needed if behind NAT)
