设为首页 收藏本站
查看: 1264|回复: 0

[经验分享] OpenStack网络知识片断(持续更新)

[复制链接]

尚未签到

发表于 2016-1-9 10:33:23 | 显示全部楼层 |阅读模式
  
   
OpenStack网络知识片断(持续更新)

 

 
  
LinuxBridge创建虚拟机
  生成设备名:tap+portid
如果不是Xen,执行ip link show dev “tap+portid”看设备是否已存在,若存在返回;若不存在:
ip tuntap add “tap+portid” mode tap
ip link set “tap+portid” address <mac_address>
ip link set “tap+portid” up
 
LinuxBridge agent
使用pydev库获取本机所有以tap开头的设备
对每一个tap设备(port)循环:
    Quantum获取port的详细信息
    ip lingk show dev <dev>
   
获取设备所在的网桥(对于新增设备应该是空):在/sys/devices/virtual/net/目录下找到以brq开头的网桥名,对每一个网桥:
        获取/sys/devices/virtual/net/<bridge>/brif/目录下所有设备
    生成network对应的网桥(brq+networkid)
   
获取network对应phynet所对应的phyinterface(必须已存在)
    ip link add link <phyinterface> name <phyinterface.vlanid> type vlan id <vlanid>
    ip link set <phyinterface.vlanid> up
    brctl addbr <brq+networkid>
    brctl setfd <brq+networkid>
    brctl stp <brq+networkid> off
    ip link set <brq+networkid> up
    brctl addif <brq+networkid>
<phyinterface.vlanid>
   
brctl addif <brq+networkid> <dev>
循环结束

OVS run_instance(准备网络)
创建虚拟机,例如网卡portid1e2b09d7-e9d6-4aba-964e-79fdd8bcf3ba
ip link show dev qbr1e2b09d7-e9 #
判断
brctl addbr qbr1e2b09d7-e9 #增加Linux网桥
ip link show dev qvo1e2b09d7-e9 #判断
ip link show dev qvb1e2b09d7-e9 #判断
ip link add qvb1e2b09d7-e9 type veth peer name qvo1e2b09d7-e9 #增加对等设备
ip link set qvb1e2b09d7-e9 up #激活设备
ip link set qvb1e2b09d7-e9 promisc on #混杂模式
ip link set qvo1e2b09d7-e9 up
ip link set qvo1e2b09d7-e9 promisc on
ip link set qbr1e2b09d7-e9 up #
激活网桥
brctl addif qbr1e2b09d7-e9 qvb1e2b09d7-e9 #Linux网桥添加设备
#下面的命令向OVS添加port
ovs-vsctl -- --may-exist add-port br-int qvo1e2b09d7-e9 -- set Interface qvo1e2b09d7-e9 external-ids:iface-id=1e2b09d7-e9d6-4aba-964e-79fdd8bcf3ba external-ids:iface-status=active external-ids:attached-mac=fa:16:3e:ea:ad:8d external-ids:vm-uuid=49b6d841-163f-4aab-b309-149727c227b4

OVS agent
初始化:
ovs-vsctl  -- --if-exists del-port br-int patch-tun
ovs-ofctl del-flows br-int
ovs-ofctl add-flow br-int hard_timeout=0,idle_timeout=0,priority=1,actions=normal
循环主体:
1)    ovs-vsctl list-ports br-int
输出:qvo1e2b09d7-e9\nqvo2d58d5dc-db\nqvo2e505b97-bb\nqvo5739b2dc-78\nqvo69121bea-6a\nqvod58fde4e-5f\nqvoe0a0b269-53\n
2)   
循环调用:ovs-vsctl get Interface qvo1e2b09d7-e9 external_ids,获取iface-idportid
输出:
{attached-mac="fa:16:3e:ea:ad:8d", iface-id="1e2b09d7-e9d6-4aba-964e-79fdd8bcf3ba", iface-status=active, vm-uuid="49b6d841-163f-4aab-b309-149727c227b4"}\n
3)   
根据portid循环:
a)      Quantum查询信息,调用get_device_details接口
b)      ovs-vsctl -- --columns=external_ids,name,ofport find Interface external_ids:iface-id="1e2b09d7-e9d6-4aba-964e-79fdd8bcf3ba",输出:
external_ids     : {attached-mac="fa:16:3e:ea:ad:8d", iface-id="1e2b09d7-e9d6-4aba-964e-79fdd8bcf3ba", iface-status=active, vm-uuid="49b6d841-163f-4aab-b309-149727c227b4"}
name               : "qvo1e2b09d7-e9"
ofport              : 6
c)     
port所属的network自动分配(如果已记录过该net,直接跳到下一步)local vlan id1-4094),且对于network对应的physical net,节点上要有一个OVS网桥与之对应,如果是vlan模式,需要做如下操作:
# outbound,出口的vlan转换
br.add_flow(priority=4,
in_port=self.phys_ofports[physical_network], #
br-int连接的port的标号
dl_vlan=lvid, #自动分配的local vlan,从1开始
actions="mod_vlan_vid:%s,normal" % segmentation_id) #segmentation_idplugin分配的vlan
# inbound,入口的vlan转换
self.int_br.add_flow(priority=3,
in_port=self.int_ofports[physical_network], #
与上面的br连接的port的标号
dl_vlan=segmentation_id,
actions="mod_vlan_vid:%s,normal" % lvid)
d)      ovs-vsctl set Port qvo1e2b09d7-e9 tag=1 #
这里的1是为network分配的local vlan id
e)      ovs-ofctl del-flows br-int in_port=6 #
这里的6port的标号,表示不允许数据流入
dhcp agent
dhcp agent需要为不同的plugin配置不同的interface_driver
OVS
quantum.agent.linux.interface.OVSInterfaceDriver
LinuxBridge
quantum.agent.linux.interface.BridgeInterfaceDriver

dhcp agent
执行的命令及输出:
#
查看设备是否存在
ip netns exec qdhcp-c6e38a5a-2adf-42a5-8c6f-5eab99208869 ip -o link show tap9739ea30-d6  
       '17: tap9739ea30-d6: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN \\    link/ether fa:16:3e:de:19:12 brd ff:ff:ff:ff:ff:ff\n'
#
查看设备IP
ip netns exec qdhcp-c6e38a5a-2adf-42a5-8c6f-5eab99208869 ip addr show tap9739ea30-d6 permanent scope global   
       '17: tap9739ea30-d6: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN \n    link/ether fa:16:3e:de:19:12 brd ff:ff:ff:ff:ff:ff\n    inet 10.10.11.2/24 brd    10.10.11.255 scope global tap9739ea30-d6\n'
#
查看DHCP进程信息,其中的进程号是从文件中获取/var/lib/quantum/dhcp/{netid}/pid
cat /proc/13695/cmdline  
       'dnsmasq\x00--no-hosts\x00--no-resolv\x00--strict-order\x00--bind-interfaces\x00--interface=tap9739ea30-d6\x00--except-interface=lo\x00--domain=openstacklocal\x00--pid-file=/var/lib/quantum/dhcp/c6e38a5a-2adf-42a5-8c6f-5eab99208869/pid\x00--dhcp-hostsfile=/var/lib/quantum/dhcp/c6e38a5a-2adf-42a5-8c6f-5eab99208869/host\x00--dhcp-optsfile=/var/lib/quantum/dhcp/c6e38a5a-2adf-42a5-8c6f-5eab99208869/opts\x00--dhcp-script=/usr/bin/quantum-dhcp-agent-dnsmasq-lease-update\x00--leasefile-ro\x00--dhcp-range=set:tag0,10.10.11.0,static,120s\x00'
#
停止进程
ip netns exec qdhcp-c6e38a5a-2adf-42a5-8c6f-5eab99208869 kill -9 13695
#
启动进程
QUANTUM_RELAY_SOCKET_PATH=/var/lib/quantum/dhcp/lease_relay QUANTUM_NETWORK_ID=c6e38a5a-2adf-42a5-8c6f-5eab99208869 ip netns exec qdhcp-c6e38a5a-2adf-42a5-8c6f-5eab99208869 dnsmasq --no-hosts --no-resolv --strict-order --bind-interfaces --interface=tap9739ea30-d6 --except-interface=lo --domain=openstacklocal --pid-file=/var/lib/quantum/dhcp/c6e38a5a-2adf-42a5-8c6f-5eab99208869/pid --dhcp-hostsfile=/var/lib/quantum/dhcp/c6e38a5a-2adf-42a5-8c6f-5eab99208869/host --dhcp-optsfile=/var/lib/quantum/dhcp/c6e38a5a-2adf-42a5-8c6f-5eab99208869/opts --dhcp-script=/usr/bin/quantum-dhcp-agent-dnsmasq-lease-update --leasefile-ro --dhcp-range=set:tag0,10.10.11.0,static,120s

dhcp agent需要为不同的plugin配置不同的interface_driver              
OVSquantum.agent.linux.interface.OVSInterfaceDriver
LinuxBridge
quantum.agent.linux.interface.BridgeInterfaceDriver
 
dhcp agent执行的命令及输出:
ip netns exec qdhcp-c6e38a5a-2adf-42a5-8c6f-5eab99208869 ip -o link show tap9739ea30-d6
    '17: tap9739ea30-d6: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN \\    link/ether fa:16:3e:de:19:12 brd ff:ff:ff:ff:ff:ff\n'
ip netns exec qdhcp-c6e38a5a-2adf-42a5-8c6f-5eab99208869 ip addr show tap9739ea30-d6 permanent scope global
    '17: tap9739ea30-d6: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN \n    link/ether fa:16:3e:de:19:12 brd ff:ff:ff:ff:ff:ff\n    inet 10.10.11.2/24 brd     10.10.11.255 scope global tap9739ea30-d6\n'
cat /proc/13695/cmdline
    'dnsmasq\x00--no-hosts\x00--no-resolv\x00--strict-order\x00--bind-interfaces\x00--interface=tap9739ea30-d6\x00--except-interface=lo\x00--domain=openstacklocal\x00--pid-file=/var/lib/quantum/dhcp/c6e38a5a-2adf-42a5-8c6f-5eab99208869/pid\x00--dhcp-hostsfile=/var/lib/quantum/dhcp/c6e38a5a-2adf-42a5-8c6f-5eab99208869/host\x00--dhcp-optsfile=/var/lib/quantum/dhcp/c6e38a5a-2adf-42a5-8c6f-5eab99208869/opts\x00--dhcp-script=/usr/bin/quantum-dhcp-agent-dnsmasq-lease-update\x00--leasefile-ro\x00--dhcp-range=set:tag0,10.10.11.0,static,120s\x00'
ip netns exec qdhcp-c6e38a5a-2adf-42a5-8c6f-5eab99208869 kill -9 13695
 
QUANTUM_RELAY_SOCKET_PATH=/var/lib/quantum/dhcp/lease_relay QUANTUM_NETWORK_ID=c6e38a5a-2adf-42a5-8c6f-5eab99208869 ip netns exec qdhcp-c6e38a5a-2adf-42a5-8c6f-5eab99208869 dnsmasq --no-hosts --no-resolv --strict-order --bind-interfaces --interface=tap9739ea30-d6 --except-interface=lo --domain=openstacklocal --pid-file=/var/lib/quantum/dhcp/c6e38a5a-2adf-42a5-8c6f-5eab99208869/pid --dhcp-hostsfile=/var/lib/quantum/dhcp/c6e38a5a-2adf-42a5-8c6f-5eab99208869/host --dhcp-optsfile=/var/lib/quantum/dhcp/c6e38a5a-2adf-42a5-8c6f-5eab99208869/opts --dhcp-script=/usr/bin/quantum-dhcp-agent-dnsmasq-lease-update --leasefile-ro --dhcp-range=set:tag0,10.10.11.0,static,120s

 

 l3 agent
  l3 agent需要配置与plugin对应的interface_driver:
OVS: quantum.agent.linux.interface.OVSInterfaceDriver
,此时external_network_bridgebr-ex
LinuxBridge: quantum.agent.linux.interface.BridgeInterfaceDriver


  
  
初始化:
  1. 加载interface_driver
2. ip netns list  #
列出以'qrouter-'开头的namespace
输出:
qdhcp-487f81ab-98d3-457a-b712-b29e71e89b52
qdhcp-084ae80a-b108-4f8a-90ca-f44aa1ca738a
qdhcp-7c25296d-bc81-45f6-bcc0-37fa44588b83
qdhcp-c6e38a5a-2adf-42a5-8c6f-5eab99208869
qrouter-0e38e30f-4fae-4f48-be4c-76d2fb803b23
qrouter-ccf5f323-2a41-41d1-8bb6-b772a8ae17fc
循环:  ip netns exec qrouter-ccf5f323-2a41-41d1-8bb6-b772a8ae17fc ip -o link list
    '9: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN \\    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00\n25: qr-012c9d13-85: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN \\    link/ether fa:16:3e:f2:a8:56 brd ff:ff:ff:ff:ff:ff\n26: qg-388798a1-55: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN \\    link/ether fa:16:3e:f6:23:b9 brd ff:ff:ff:ff:ff:ff\n'
   
对命名空间内的设备名作循环(加粗字体):
        如果是qr开头:ovs-vsctl --timeout=2 -- --if-exists del-port br-int qr-012c9d13-85
        
如果是qg开头:ovs-vsctl --timeout=2 -- --if-exists del-port br-ex qg-388798a1-55
   
循环结束
循环结束


  
  
工作任务:
  1. 保证br-ex存在
2. Quantum获取router:external的网络(一个l3 agent只处理一个external网络,默认是br-ex,可以在配置文件中配置external_network_bridge
3.
循环获取Quantum中的router对象(只处理连接外网的router):
    增加qrouter-routerid命名空间
    ip netns exec qrouter-0e38e30f-4fae-4f48-be4c-76d2fb803b23 sysctl -w net.ipv4.ip_forward=1
   
如果配置了metadata_ip,执行
        rules.append(('INPUT', '-s 0.0.0.0/0 -d %s -p tcp -m tcp --dport %s -j ACCEPT' % (self.conf.metadata_ip, self.conf.metadata_port)))
        rules.append(('PREROUTING', '-s 0.0.0.0/0 -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination %s:%s' % (self.conf.metadata_ip, self.conf.metadata_port)))
   
应用iptable规则
        ip netns exec qrouter-0e38e30f-4fae-4f48-be4c-76d2fb803b23 /sbin/iptables-save -t filter
        ip netns exec qrouter-0e38e30f-4fae-4f48-be4c-76d2fb803b23 /sbin/iptables-restore
        ip netns exec qrouter-0e38e30f-4fae-4f48-be4c-76d2fb803b23 /sbin/iptables-save -t nat
        ip netns exec qrouter-0e38e30f-4fae-4f48-be4c-76d2fb803b23 /sbin/iptables-restore
   
对于router的新增内部port循环:
        ip netns exec qrouter-ccf5f323-2a41-41d1-8bb6-b772a8ae17fc ip -o link show qr-012c9d13-85,如果设备不存在:
            ovs-vsctl -- --may-exist add-port br-int qr-012c9d13-85
                       -- set Interface qr-012c9d13-85 type=internal
                       -- set Interface qr-012c9d13-85 external-ids:iface-id=012c9d13-8554-4b39-96b8-e4bd2e787559
                       -- set Interface qr-012c9d13-85 external-ids:iface-status=active
                       -- set Interface qr-012c9d13-85 external-ids:attached-mac=fa:16:3e:f2:a8:56
        ip link set qr-012c9d13-85 address fa:16:3e:f2:a8:56
        ip link set qr-012c9d13-85 netns qrouter-ccf5f323-2a41-41d1-8bb6-b772a8ae17fc
        ip netns exec qrouter-ccf5f323-2a41-41d1-8bb6-b772a8ae17fc ip link set qr-012c9d13-85 up
        ip netns exec qrouter-ccf5f323-2a41-41d1-8bb6-b772a8ae17fc ip addr show qr-012c9d13-85 permanent scope global
        ip netns exec qrouter-ccf5f323-2a41-41d1-8bb6-b772a8ae17fc ip -4 addr add 10.10.10.1/24 brd 10.10.10.255 scope global dev qr-012c9d13-85
       
如果router连接到外部网络,增加snat规则(将内部的IP转换为外网的IP),应用iptable规则
    循环结束
   
   
对于router上删除的port循环:
        OVS上删除port
       
删除nat规则
    循环结束

    初始化gw_portip netns exec qrouter-ccf5f323-2a41-41d1-8bb6-b772a8ae17fc ip -o link show qg-388798a1-55,如果设备不存在,同上(操作br-ex
    ip netns exec qrouter-ccf5f323-2a41-41d1-8bb6-b772a8ae17fc route add default <gw_port的网关IP>
   
增加snat规则

    处理router上的floatingIP,对br-ex上的port(qg-388798a1-55)配置外网地址,配置snat/dnat规则
循环结束

  
  

运维网声明 1、欢迎大家加入本站运维交流群:群②:261659950 群⑤:202807635 群⑦870801961 群⑧679858003
2、本站所有主题由该帖子作者发表,该帖子作者与运维网享有帖子相关版权
3、所有作品的著作权均归原作者享有,请您和我们一样尊重他人的著作权等合法权益。如果您对作品感到满意,请购买正版
4、禁止制作、复制、发布和传播具有反动、淫秽、色情、暴力、凶杀等内容的信息,一经发现立即删除。若您因此触犯法律,一切后果自负,我们对此不承担任何责任
5、所有资源均系网友上传或者通过网络收集,我们仅提供一个展示、介绍、观摩学习的平台,我们不对其内容的准确性、可靠性、正当性、安全性、合法性等负责,亦不承担任何法律责任
6、所有作品仅供您个人学习、研究或欣赏,不得用于商业或者其他用途,否则,一切后果均由您自己承担,我们对此不承担任何法律责任
7、如涉及侵犯版权等问题,请您及时通知我们,我们将立即采取措施予以解决
8、联系人Email:admin@iyunv.com 网址:www.yunweiku.com

所有资源均系网友上传或者通过网络收集,我们仅提供一个展示、介绍、观摩学习的平台,我们不对其承担任何法律责任,如涉及侵犯版权等问题,请您及时通知我们,我们将立即处理,联系人Email:kefu@iyunv.com,QQ:1061981298 本贴地址:https://www.yunweiku.com/thread-162107-1-1.html 上篇帖子: 使用haproxy作为OpenStack loadbalancer时通信异常 下篇帖子: openstack nova源码分析之api(二)
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

扫码加入运维网微信交流群X

扫码加入运维网微信交流群

扫描二维码加入运维网微信交流群,最新一手资源尽在官方微信交流群!快快加入我们吧...

扫描微信二维码查看详情

客服E-mail:kefu@iyunv.com 客服QQ:1061981298


QQ群⑦:运维网交流群⑦ QQ群⑧:运维网交流群⑧ k8s群:运维网kubernetes交流群


提醒:禁止发布任何违反国家法律、法规的言论与图片等内容;本站内容均来自个人观点与网络等信息,非本站认同之观点.


本站大部分资源是网友从网上搜集分享而来,其版权均归原作者及其网站所有,我们尊重他人的合法权益,如有内容侵犯您的合法权益,请及时与我们联系进行核实删除!



合作伙伴: 青云cloud

快速回复 返回顶部 返回列表