Using GRE Tunnels with Open vSwitch
I'mback with another "how to" article on Open vSwitch (OVS), this time taking a lookat using GRE (Generic Routing Encapsulation) tunnels with OVS. OVS can use GREtunnels between hosts as a way of encapsulating traffic and creating an overlaynetwork. OpenStack Quantum can (and does) leverage this functionality, in fact,to help separate different "tenant networks" from one another. In this write-up,I'll walk you through the process of configuring OVS to build a GRE tunnel tobuild an overlay network between two hypervisors running KVM.Naturally,any sort of "how to" such as this always builds upon the work of others. Inparticular, I found a couple of Brent Salisbury's articles (hereand here)especially useful.
Thisprocess has 3 basic steps:
[*]Createan isolated bridge for VM connectivity.
[*]Createa GRE tunnel endpoint on each hypervisor.
[*]Adda GRE interface and establish the GRE tunnel.
Thesesteps assume that you've already installed OVS on your Linux distribution ofchoice. I haven't explicitly done a write-up on this, but there are numerousposts from a variety of authors (in this regard, Google is your friend).
We'llstart with an overview of the topology, then we'll jump into the specificconfiguration steps.
Reviewing the Topology
Thegraphic below shows the basic topology of what we have going on here:
http://blog.scottlowe.org/wp-content/uploads/2013/05/ovs-gre-tun-overview.png
Wehave two hypervisors (CentOS 6.3 and KVM, in my case), both running OVS (anolder version, version 1.7.1). Each hypervisor has one OVS bridge that has atleast one physical interface associated with the bridge (shown asbr0 connected to eth0 in the diagram). As part of thisprocess, you'll create the other internal interfaces (the tep andgre interfaces, as well as the second, isolated bridge to which VMswill connect. You'll then create a GRE tunnel between the hypervisors and testVM-to-VM connectivity.
Creating an Isolated Bridge
Thefirst step is to create the isolated OVS bridge to which the VMs will connect. Icall this an "isolated bridge" because the bridge has no physical interfacesattached. (Side note: this>integrationbridge. The concept is the same.)
Thecommand is very simple, actually:
ovs-vsctl add-br br2
Yes,that's it. Feel free to substitute a different name for br2 in thecommand above, if you like, but just make note of the name as you'll need itlater.
Tomake things easier for myself, once I'd created the isolated bridge I then createda libvirt network for it so that it was dead-easy to attach VMs to this newisolated bridge.
Configuring the GRE Tunnel Endpoint
TheGRE tunnel endpoint is an interface on each hypervisor that will, as the nameimplies, serve as the endpoint for the GRE tunnel. My purpose in creating aseparate GRE tunnel endpoint is to separate hypervisor management traffic fromGRE traffic, thus allowing for an architecture that might leverage a separatemanagement network (which is typically considered a recommended practice).
Tocreate the GRE tunnel endpoint, I'm going to use the same technique I describedin my post on runninghost management traffic through OVS. Specifically, we'll create an internalinterface and assign it an IP address.
Tocreate the internal interface, use this command:
ovs-vsctl add-port br2 tep0 -- set interface tep0 type=internal
Inyour environment, you'll substitute br2 with the name of theisolated bridge you created earlier. You could also use a different name thantep0. Since this name is essentially for human consumption only,use what makes sense to you. Since this is a tunnel endpoint, tep0made sense to me.
Oncethe internal interface is established, assign it with an IP address usingifconfig or ip, whichever you prefer. I'm stillgetting used to using ip (more on that in a future post, mostlikely), so I tend to use ifconfig, like this:
ifconfig tep0 192.168.200.20 netmask 255.255.255.0
Obviously,you'll want to use an IP addressing scheme that makes sense for yourenvironment. One important note: don't use the same subnet as you've assigned toother interfaces on the hypervisor, or else you can't control that the GREtunnel will originate (or terminate) on the interface you specify. This isbecause the Linux routing table on the hypervisor will control how the trafficis routed. (You could use source routing, a topic I plan to discuss in a futurepost, but that's beyond the scope of this article.)
Repeatthis process on the other hypervisor, and be sure to make note of the IPaddresses assigned to the GRE tunnel endpoint on each hypervisor; you'll needthose addresses shortly. Once you've established the GRE tunnel endpoint on eachhypervisor, test connectivity between the endpoints using ping or asimilar tool. If connectivity is good, you're clear to proceed; if not, you'llneed to resolve that before moving on.
Establishing the GRE Tunnel
Bythis point, you've created the isolated bridge, established the GRE tunnelendpoints, and tested connectivity between those endpoints. You're now ready toestablish the GRE tunnel.
Usethis command to add a GRE interface to the isolated bridge on eachhypervisor:
ovs-vsctl add-port br2 gre0 -- set interface gre0 type=gre \
options:remote_ip=
Substitutethe name of the isolated bridge you created earlier here for br2and feel free to use something other than gre0 for the interfacename. I think using gre as the base name for the GRE interfacesmakes sense, but run with what makes sense to you.
Onceyou repeat this command on both hypervisors, the GRE tunnel should be up andrunning. (Troubleshooting the GRE tunnel is one area where my knowledge is weak;anyone have any suggestions or commands that we can use here?)
Testing VM Connectivity
Aspart of this process, I spun up an Ubuntu 12.04 server image on each hypervisor(using virt-install as I outlined here),attached each VM to the isolated bridge created earlier on that hypervisor, andassigned each VM an IP address from an entirely different subnet than thephysical network was using (in this case, 10.10.10.x).
Here'sthe output of the route -n command on the Ubuntu guest, to showthat it has no knowledge of the "external" IP subnet—it knows onlyabout its own interfaces:
ubuntu:~ root$ route -n
Kernel IP routing table
DestinationGateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.10.10.2540.0.0.0 UG 100 0 0 eth0
10.10.10.0 0.0.0.0 255.255.255.0U 0 0 0 eth0
Similarly,here's the output of the route -n command on the CentOS host,showing that it has no knowledge of the guest's IP subnet:
centos:~ root$ route -n
Kernel IP routing table
DestinationGateway Genmask Flags Metric Ref Use Iface
192.168.2.00.0.0.0 255.255.255.0U 0 0 0 tep0
192.168.1.00.0.0.0 255.255.255.0U 0 0 0 mgmt0
0.0.0.0 192.168.1.2540.0.0.0 UG 0 0 0 mgmt0
Inmy case, VM1 (named web01) was given 10.10.10.1; VM2 (namedweb02) was given 10.10.10.2. Once I went through the steps outlinedabove, I was able to successfully ping VM2 from VM1, as you can see in thisscreenshot:
http://blog.scottlowe.org/wp-content/uploads/2013/05/ovs-gre-tun-vm-connectivity.png
(Althoughit's not shown here, connectivity from VM2 to VM1 was obviously successful aswell.)
"OK,that's cool, but why do I care?" you might ask.
Inthis particular context, it's a bit of a science experiment. However, if youtake a step back and begin to look at the bigger picture, then (hopefully)something starts to emerge:
[*]Wecan use an encapsulation protocol (GRE in this case, but it could have just aseasily been STT or VXLAN) to isolate VM traffic from the physical network andfrom other VM traffic. (Think multi-tenancy.)
[*]Whilethis process was manual, think about some sort of controller (an OpenFlowcontroller, perhaps?) that could help automate this process based on itsknowledge of the VM topology.
[*]Usinga virtualized firewall or virtualized firewall, I could easily provideconnectivity into or out of this isolated (encapsulated) private network. (Thisis probably something I'll experiment with later.)
[*]Whatif we wrapped some sort of orchestration framework around this, to help deployVMs, create networks, add routers/firewalls automatically, all based on thecustomer's needs? (OpenStack Networking, anyone?)
Anyway,I hope this is helpful to someone. As always, I welcome feedback and suggestionsfor improvement, so feel free to speak up in the comments below. Vendordisclosures, where appropriate, are greatly appreciated. Thanks!
页:
[1]