Integrate ONOS(emu) with OpenStack (Kilo)

scvmm

　　2015年12月10日 发布
　　http://uni2u.tistory.com/58
　　http://blog.night9.cc/onos-on-centos7/
　　https://www.youtube.com/watch?v=qP8nPYhz_Mo
　　https://wiki.onosproject.org/display/ONOS/Simple+workthrough
　　http://forum.onosfw.com/t/how-to-integrate-onos-master-1-4-0-with-openstack-kilo/80
　　https://github.com/hyunsun/documentations/wiki/Neutron-ONOS-Integration-for-CORD-VTN
　　About onosfw：
　　Flash video：https://www.youtube.com/watch?v=7bxjWrR4peI
　　On Controller Node:
　　[iyunv@controller ~]# yum install python-pip
　　[iyunv@controller ~]# sudo pip install networking-onos
　　...
　　Successfully installed networking-onos-1.0.0 pbr-1.8.1 pytz-2015.7
　　[iyunv@controller ~]# ls /usr/lib/python2.7/site-packages/networking_onos
　　[iyunv@controller ~]# vi /usr/lib/python2.7/site-packages/networking_onos-1.0.0.dist-info/entry_points.txt
　　[neutron.ml2.mechanism_drivers]
　　onos_ml2 = networking_onos.plugins.ml2.driver:ONOSMechanismDriver
　　[neutron.service_plugins]
　　onos_router = networking_onos.plugins.l3.driver:ONOSL3Plugin
　　[iyunv@controller ~]# vi /usr/lib/python2.7/site-packages/neutron-2015.1.1-py2.7.egg-info/entry_points.txt
　　[neutron.ml2.mechanism_drivers]
　　......
　　mlnx = neutron.plugins.ml2.drivers.mlnx.mech_mlnx:MlnxMechanismDriver
　　opendaylight = neutron.plugins.ml2.drivers.opendaylight.driver:OpenDaylightMechanismDriver
　　onos_ml2 = networking_onos.plugins.ml2.driver:ONOSMechanismDriver
　　[neutron.service_plugins]
　　......
　　brocade_mlx_l3 = neutron.services.l3_router.brocade.mlx.l3_router_plugin:BrocadeRouterPlugin
　　lbaas = neutron_lbaas.services.loadbalancer.plugin:LoadBalancerPlugin
　　onos_router = networking_onos.plugins.l3.driver:ONOSL3Plugin
　　[iyunv@controller ~]# vi /etc/neutron/neutron.conf
　　[DEFAULT]
　　verbose = True
　　rpc_backend = rabbit
　　auth_strategy = keystone
　　core_plugin = ml2
　　#service_plugins = router
　　service_plugins = networking_onos.plugins.l3.driver.ONOSL3Plugin
　　[iyunv@controller ~]# vi /etc/neutron/plugins/ml2/ml2_conf.ini
　　[ml2]
　　type_drivers = flat,vlan,gre,vxlan
　　tenant_network_types = gre
　　#mechanism_drivers = openvswitch
　　mechanism_drivers = onos_ml2
　　... ...
　　[onos]
url_path = http://192.168.0.3:8181/onos/vtn
username = karaf
password = karaf
　　[iyunv@controller ~]# systemctl restart neutron-server.service
　　+--------------------------------------+--------------------+------------+-------+----------------+---------------------------+
| id                                   | agent_type         | host       | alive | admin_state_up | binary                    |
+--------------------------------------+--------------------+------------+-------+----------------+---------------------------+
| 75001fc6-fa0f-4ab1-9267-73089d8d3ae1 | DHCP agent         | controller | :-)   | True           | neutron-dhcp-agent        |
| 7d549c58-e73f-45c2-9e84-6b78896af4ce | Metadata agent     | controller | :-)   | True           | neutron-metadata-agent    |
| 9a7941f1-5ed6-4957-b5bf-f1e9674a5bf5 | Open vSwitch agent | compute    | :-)   | True           | neutron-openvswitch-agent |
| fa34f576-f9e2-4457-8264-cbbe2756e5d8 | Open vSwitch agent | controller | :-)   | True           | neutron-openvswitch-agent |
| fd72b5b0-561d-4cb0-8e88-9f445fdaa271 | L3 agent           | controller | :-)   | True           | neutron-l3-agent          |
+--------------------------------------+--------------------+------------+-------+----------------+---------------------------+
　　On Network Node:
　　[iyunv@controller ~]# systemctl stop neutron-openvswitch-agent.service
　　[iyunv@controller ~]# systemctl stop neutron-l3-agent.service
　　[iyunv@controller ~]# systemctl restart neutron-dhcp-agent.service
　　[iyunv@controller ~]# systemctl restart neutron-metadata-agent.service
　　[iyunv@controller ~]# ovs-vsctl del-manager
　　[iyunv@controller ~]# ovs-vsctl del-br br-int
　　On Compute Node:
　　[iyunv@compute ~]# systemctl stop neutron-openvswitch-agent.service
　　[iyunv@compute ~]# systemctl restart openstack-nova-compute.service
　　[iyunv@compute ~]# ovs-vsctl del-manager
　　[iyunv@compute ~]# ovs-vsctl del-br br-int
　　On Controller Node:
　　[iyunv@controller ~]# source admin-openrc.sh
　　[iyunv@controller ~]# neutron agent-list
　　+--------------------------------------+--------------------+------------+-------+----------------+---------------------------+
| id                                   | agent_type         | host       | alive | admin_state_up | binary                    |
+--------------------------------------+--------------------+------------+-------+----------------+---------------------------+
| 75001fc6-fa0f-4ab1-9267-73089d8d3ae1 | DHCP agent         | controller | :-)   | True           | neutron-dhcp-agent        |
| 7d549c58-e73f-45c2-9e84-6b78896af4ce | Metadata agent     | controller | :-)   | True           | neutron-metadata-agent    |
| 9a7941f1-5ed6-4957-b5bf-f1e9674a5bf5 | Open vSwitch agent | compute    | xxx   | True           | neutron-openvswitch-agent |
| fa34f576-f9e2-4457-8264-cbbe2756e5d8 | Open vSwitch agent | controller | xxx   | True           | neutron-openvswitch-agent |
| fd72b5b0-561d-4cb0-8e88-9f445fdaa271 | L3 agent           | controller | xxx   | True           | neutron-l3-agent          |
+--------------------------------------+--------------------+------------+-------+----------------+---------------------------+
　　Install onos controller on Controller Node:
　　Firewall Setting (Here i just stop the firewalld service)
　　[iyunv@controller ~]# systemctl start firewalld
　　[iyunv@controller ~]# firewall-cmd --zone=public --add-port=8181/tcp --permanent
　　[iyunv@controller ~]# firewall-cmd --zone=public --add-port=6633/tcp --permanent
　　Download onos from git
　　[iyunv@controller ~]# yum install git
　　[iyunv@controller ~]# git clone https://gerrit.onosproject.org/onos
　　Other version Download Site
　　Install Java 8
　　[iyunv@controller ~]# cd ~ && mkdir Downloads Applications
　　[iyunv@controller ~]# cd Downloads
　　[iyunv@controller Downloads]# wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u45-b14/jdk-8u45-linux-x64.rpm"
　　[iyunv@controller Downloads]# rpm -ivh jdk-8u45-linux-x64.rpm
　　Download Karaf & Maven
　　[iyunv@controller Downloads]# wget http://download.nextag.com/apache/karaf/3.0.3/apache-karaf-3.0.3.tar.gz
　　[iyunv@controller Downloads]# wget https://archive.apache.org/dist/maven/maven-3/3.3.1/binaries/apache-maven-3.3.1-bin.tar.gz
　　[iyunv@controller Downloads]# tar -zxvf apache-karaf-3.0.3.tar.gz -C ../Applications/
　　[iyunv@controller Downloads]# tar -zxvf apache-maven-3.3.1-bin.tar.gz -C ../Applications/
　　Create a CELL definition
　　[iyunv@controller ~]# cat >>/root/onos/tools/test/cells/sdnds-tw<<EOF

　　# ONOS from Scratch tutorial cell

　　# Cell name

　　export ONOS_CELL=sdnds-tw

　　# the address of the VM to install the package onto

　　export OC1="192.168.0.3"

　　# the default address used by ONOS utilities when none are supplied

　　export OCI="192.168.0.3"

　　# the ONOS apps to load at startup

　　export ONOS_APPS="drivers,openflow,fwd,proxyarp,mobility"

　　# the Mininet VM (if you have one)

　　export OCN="192.168.0.3"

　　# pattern to specify which address to use for inter-ONOS node communication (not used with single-instance core)

　　export ONOS_NIC="192.168.0.*"

　　# User

　　export ONOS_USER=karaf

　　export ONOS_GROUP=karaf

　　EOF:
　　Configure bash
　　[iyunv@controller ~]# cat>>~/.bashrc<<EOF
　　export ONOS_ROOT=/root/onos
　　source /root/onos/tools/dev/bash_profile
　　export JAVA_HOME=/usr/java/jdk1.8.0_45/
　　export JRE_HOME=/usr/java/jdk1.8.0_45/jre/
　　export KARAF_ROOT=/root/Applications/apache-karaf-3.0.3
　　export M2_HOME=/root/Applications/apache-maven-3.3.1
　　export ONOS_USER=karaf
　　export ONOS_GROUP=karaf
　　ONOS_CELL=sdnds-tw
　　EOF
　　[iyunv@controller ~]# source ~/.bashrc
　　Build onos
　　[iyunv@controller ~]# vi ~/Applications/apache-karaf-3.0.3/etc/org.apache.karaf.features.cfg
　　After featuresRepositories, add , mvn:org.onosproject/onos-features/1.4.0-SNAPSHOT/xml/features
　　[iyunv@controller ~]# cd /root/onos
　　[iyunv@controller onos]# mvn clean install
　　Run onos
　　[iyunv@controller onos]# onos-setup-karaf clean 192.168.0.3
　　Unpacking /root/Downloads/apache-karaf-3.0.3.tar.gz to /root/Applications...
　　Adding ONOS feature repository...
　　Adding ONOS boot features webconsole,onos-api,onos-core,onos-incubator,onos-cli,onos-rest,onos-gui...
　　Branding as ONOS...
　　Creating local cluster configs for IP 192.168.0.3...
　　Copying package configs...
　　Staging builtin apps...
　　Customizing apps to be auto-activated: drivers,openflow,fwd,proxyarp,mobility...
　　[iyunv@controller onos]# karaf clean
　　Welcome to Open Network Operating System (ONOS)!
　　____  _  ______  ____
　　/ __ \/ |/ / __ \/ __/
　　/ /_/ /    / /_/ /\ \
　　\____/_/|_/\____/___/
　　Hit '<tab>' for a list of available commands
　　and '[cmd] --help' for help on a specific command.
　　Hit '<ctrl-d>' or type 'system:shutdown' or 'logout' to shutdown ONOS.
　　onos> apps
　　...
　　* id=46, name=org.onosproject.fwd, version=1.4.0.SNAPSHOT, origin=ON.Lab, description=Reactive forwarding application using flow subsystem, features=[onos-app-fwd], featuresRepo=mvn:org.onosproject/onos-app-fwd/1.4.0-SNAPSHOT/xml/features, apps=[], permissions=[]
　　onos> feature:install onos-ovsdatabase
　　onos> feature:install onos-app-vtn-onosfw
　　onos> devices
　　id=of:0000000000000002, available=true, role=MASTER, type=SWITCH, mfr=Nicira, Inc., hw=Open vSwitch, sw=2.3.1, serial=None, managementAddress=192.168.0.3, protocol=OF_13, channelId=192.168.0.3:41297
id=of:000001bf6d14126c, available=true, role=MASTER, type=SWITCH, mfr=Nicira, Inc., hw=Open vSwitch, sw=2.3.1, serial=None, managementAddress=192.168.0.4, protocol=OF_13, channelId=192.168.0.4:41292
　　id=ovsdb:192.168.0.3, available=true, role=MASTER, type=CONTROLLER, mfr=unknown, hw=unknown, sw=unknown, serial=unknown, ipaddress=192.168.0.3
　　id=ovsdb:192.168.0.4, available=true, role=MASTER, type=CONTROLLER, mfr=unknown, hw=unknown, sw=unknown, serial=unknown, ipaddress=192.168.0.4
　　Q: Why it brings ovsdb info? And there can not get the detail info.
　　Set flow manager and check
　　[iyunv@controller ~]# ovs-vsctl set-manager tcp:192.168.0.3:6640
　　[iyunv@controller ~]# ovs-vsctl show
　　[iyunv@controller onos]# ovs-vsctl show
　　fe309351-12f1-4a17-bd8b-19d1050f5862
   Manager "tcp:192.168.0.3:6640"
        is_connected: true
    Bridge br-ex
        Port br-ex
            Interface br-ex
                type: internal
        Port "enp3s0"
            Interface "enp3s0"
    Bridge br-int
       Controller "tcp:192.168.0.3:6653"
            is_connected: true
        fail_mode: secure
        Port "tape180f131-2c"
            Interface "tape180f131-2c"
                type: internal
        Port "vxlan-0.0.0.0"
            Interface "vxlan-0.0.0.0"
                type: vxlan
                options: {key=flow, local_ip="192.168.0.3", remote_ip=flow}
        Port vxlan
            Interface vxlan
                type: vxlan
                options: {key=flow, remote_ip=flow}
        Port "eth0"
            Interface "eth0"
        Port br-int
            Interface br-int
    ovs_version: "2.3.1"
　　Q: I don't know why red font part type is vxlan, cause I configured as gre.... why?
　　[iyunv@compute ~]# ovs-vsctl set-manager tcp:192.168.0.3:6640
　　[iyunv@compute ~]# ovs-vsctl show
　　e9091a13-03ec-46be-8bfc-9eb18bcfef57
　　Manager "tcp:192.168.0.3:6640"
        is_connected: true
　　Bridge br-int
        Controller "tcp:192.168.0.3:6653"
            is_connected: true
　　fail_mode: secure
　　Port br-int
　　Interface br-int
　　Port "eth0"
　　Interface "eth0"
　　Port "vxlan-0.0.0.0"
　　Interface "vxlan-0.0.0.0"
　　type: vxlan
　　       options: {key=flow, local_ip="192.168.0.4", remote_ip=flow}
　　ovs_version: "2.3.1"
　　Dump flows and check whether it works well
　　[iyunv@controller onos]# ovs-ofctl dump-flows br-int -O openflow13
　　OFPST_FLOW reply (OF1.3) (xid=0x2):
　　cookie=0x2f0000487fc234, duration=852.940s, table=0, n_packets=0, n_bytes=0, send_flow_rem priority=40000,dl_type=0x88cc actions=CONTROLLER:65535
　　cookie=0x2f00004644b005, duration=852.940s, table=0, n_packets=0, n_bytes=0, send_flow_rem priority=5,ip actions=CONTROLLER:65535
　　cookie=0x2f0000487fd07e, duration=852.938s, table=0, n_packets=0, n_bytes=0, send_flow_rem priority=40000,dl_type=0x8942 actions=CONTROLLER:65535
　　cookie=0x2f0000488f2a3a, duration=852.923s, table=0, n_packets=0, n_bytes=0, send_flow_rem priority=40000,arp actions=CONTROLLER:65535
　　cookie=0x2f00004644b0bf, duration=852.940s, table=0, n_packets=0, n_bytes=0, send_flow_rem priority=5,arp actions=CONTROLLER:65535
　　[iyunv@compute ~]# ovs-ofctl dump-flows br-int -O openflow13
　　OFPST_FLOW reply (OF1.3) (xid=0x2):
　　cookie=0x2f0000e838c012, duration=820.103s, table=0, n_packets=0, n_bytes=0, send_flow_rem priority=40000,dl_type=0x88cc actions=CONTROLLER:65535
　　cookie=0x2f0000e5fdade3, duration=820.103s, table=0, n_packets=0, n_bytes=0, send_flow_rem priority=5,ip actions=CONTROLLER:65535
　　cookie=0x2f0000e838ce5c, duration=820.003s, table=0, n_packets=0, n_bytes=0, send_flow_rem priority=40000,dl_type=0x8942 actions=CONTROLLER:65535
　　cookie=0x2f0000e8482818, duration=820.003s, table=0, n_packets=0, n_bytes=0, send_flow_rem priority=40000,arp actions=CONTROLLER:65535
　　cookie=0x2f0000e5fdae9d, duration=820.003s, table=0, n_packets=0, n_bytes=0, send_flow_rem priority=5,arp actions=CONTROLLER:65535
　　Access to Web GUI
　　http://192.168.0.3:8181/onos/ui/index.html

분명히 controller/network 노드 한대, compute 노드 한대인데 왜 이렇게 표시 되는지 ... networking-onos패키지에서 제공하는 ml2드라이버와 l3드라이버 문제인지 ...

클릭하면 포터 정보 확인 가능한데 포트 정보도 문제가 많음 분명히 gre로 설정햇는데 왜 vxlan으로 표시되고 표시 되는 것마저 이상함

　　OpenStack Dashboard Access: http://117.16.149.115/dashboard

이상하거나 말거나 ! 일단 네트워크 만들어봄 adnin tenant 로 !

같은 tenant 로 같은 admin-net 에서 vm끼티 통신 가능 여부 확인하기 위해서 인스턴스 admin-01과 admin-02만들어봄

인스턴스 두대 추가하면 마지막 두 정보 추가됨 첫번째 호스트 정보는 머임???

가상머신이 정보를 받아오지 못하고 잇음