[ovs-discuss] duplicate option: of_interface
Gabe Black
Gabe.Black at viavisolutions.com
Wed Aug 26 18:43:30 UTC 2015
Interesting... it seems that the ovs-vsctl set bridge br-int datapath_type=netdev does take, but very soon after it is changed back to system. I'll try and figure out how that is happening.
Gabe
> -----Original Message-----
> From: Gabe Black
> Sent: Wednesday, August 26, 2015 12:42 PM
> To: 'Mooney, Sean K'; bugs at openvswitch.org
> Subject: RE: duplicate option: of_interface
>
> Hi Sean,
>
> Yes, I did have the OVS_DATAPATH_TYPE set to netdev. Even after setting
> the [ovs] datapath_type=netdev and trying to manually set the
> datapath_type to netdev (ovs-vsctl set Bridge br-int
> datapath_type=netdev), the br-int still shows system.
>
> The interesting thing is that the other bridges (br-ex and br-p6p1 are set to
> netdev (automatically during stacking), but just br-int is system and
> immutable it seems via manual ovs-vsctl commands).
>
> I'll keep playing around and see if there is some way to reconcile.
>
> Thanks,
> Gabe
>
> > -----Original Message-----
> > From: Mooney, Sean K [mailto:sean.k.mooney at intel.com]
> > Sent: Wednesday, August 26, 2015 5:12 AM
> > To: Gabe Black; bugs at openvswitch.org
> > Subject: RE: duplicate option: of_interface
> >
> > I know the feeling. I think your horizon issue can be fixed by Running
> > sudo iptables -F sudo iptables -X iptables based security groups do
> > not work with vhost-user so this will have no effect on vm security
> >
> > I think the ovs-ofctl: br-int is not a bridge or a socket error is
> > because the datapath_type on the bridge is not currently set to netdev.
> >
> > Looking at your previous error I noticed the follow command the
> > datapath_type is being set to system.
> > ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--',
> > > '--may-exist', 'add- br', 'br-int', '--', 'set', 'Bridge', 'br-int',
> > > 'datapath_type=system']
> >
> >
> > I think the reason the bridge does not exists is because it has the
> > wrong datapath type.
> > You can validate this by running
> > sudo ovs-vsctl list-br
> >
> > when using kernel ovs the output will be similar to this
> >
> > _uuid : 50c64fb3-3c0c-40f9-8e45-6384a99687aa
> > controller : []
> > datapath_id : "0000b24fc650f940"
> > datapath_type : system
> > external_ids : {bridge-id=br-int}
> > fail_mode : secure
> > flood_vlans : []
> > flow_tables : {}
> > ipfix : []
> > mirrors : []
> > name : br-int
> > netflow : []
> > other_config : {}
> > ports : [1e2aeb85-01a2-4f1d-a0ef-f4954acab3f2, b8c7a19e-c510-
> 47c1-
> > 874c-9d99ec6085f5, c370d053-6aea-4f16-a6bc-f54f1aabfcb5,
> > e1a7ef91-53a4- 4222-9bf7-249888ad48ef, e917d939-44db-4661-9311-
> bb68a6c132c4]
> > protocols : ["OpenFlow10"]
> > sflow : []
> > status : {}
> > stp_enable : false
> >
> > when using ovs with dpdk the datapath_type should be netdev.
> >
> > If it is currently system and assuming you have
> > OVS_DATAPATH_TYPE=netdev set in your local.conf this could be caused,
> > as one of my patches to neutron merged Saturday and we have not
> > updated our deployment code To add the appropriate configuration.
> >
> https://github.com/openstack/neutron/commit/5b708d5f0e9a5ddb4675148
> > 9a52665e673a5cb0b
> >
> >
> > If you add
> > [ovs]
> > datapath_type = netdev
> > to the ml2_conf.ini the q-agt will configure the dpdk netdev datapath
> > instead of the system kernel datapath.
> >
> > For your running system you can run the following commands to set the
> > datapath
> >
> > sudo ovs-vsctl set bridge <bridge name> datapath=netdev.
> >
> > This should be run for all ovs bridges.
> >
> > On huge table memory as the system is left running main memory gets
> > fragment.
> > when you start ovs with our service file it allocates hugempage memory
> > in the kernel from the current free memory.
> >
> > When you compile dpdk there is a hugepage segment limit of 256 segment.
> > If the dpdk application cannot allocated its hugepage memory in less
> > segment then this limit the application will exit.
> > You can adjust this limit by setting OVS_DPDK_MEM_SEGMENTS=<value>
> in
> > your local.conf The more often you stack and unstack between reboots
> > the more fragmented your memory will be so for development I tend to
> > set the OVS_DPDK_MEM_SEGMENTS quite high.
> >
> > Regards
> > Sean.
> > -----Original Message-----
> > From: Gabe Black [mailto:Gabe.Black at viavisolutions.com]
> > Sent: Tuesday, August 25, 2015 9:45 PM
> > To: Gabe Black; Mooney, Sean K; bugs at openvswitch.org
> > Subject: RE: duplicate option: of_interface
> >
> > Sorry, that was premature. Apparently hugetable memory gets
> > fragmented or leaks or something as the ovs-dpdk failed to start.
> Rebooting fixed that.
> >
> > I still get many errors in the q-agt log file similar to this:
> > ERROR neutron.agent.linux.utils [-]
> > Command: ['ovs-ofctl', 'add-flows', 'br-int', '-'] Exit code: 1
> > Stdin:
> > hard_timeout=0,idle_timeout=0,priority=0,table=0,cookie=0x0,actions=no
> > r
> > mal
> > Stdout:
> > Stderr: ovs-ofctl: br-int is not a bridge or a socket
> >
> > and
> >
> > ERROR neutron.agent.common.ovs_lib [req-ab1e1f65-d14c-4209-ac20-
> > d29e450eda38 None None] Unable to execute ['ovs-ofctl', 'dump-flows',
> > 'br- int', 'table=23']. Exception:
> > Command: ['ovs-ofctl', 'dump-flows', 'br-int', 'table=23'] Exit code:
> > 1
> > Stdin:
> > Stdout:
> > Stderr: ovs-ofctl: br-int is not a bridge or a socket
> >
> > But at least q-agt isn't dying. I remember why I hate fedora though..
> > Even though horizon says it is running and we turned the firewall to
> > permissive, I can never access the website. Seems like something else
> > is protecting or acting as a firewall.
> >
> >
> >
> > > -----Original Message-----
> > > From: discuss [mailto:discuss-bounces at openvswitch.org] On Behalf Of
> > > Gabe Black
> > > Sent: Tuesday, August 25, 2015 2:06 PM
> > > To: Mooney, Sean K; bugs at openvswitch.org
> > > Subject: Re: [ovs-discuss] duplicate option: of_interface
> > >
> > > Thanks for the reply and hints on changes... Armed with that I
> > > undid the changes from git commit 053bfc5a (neutron) and that
> > > allowed the q-agt process to at least not die there with that
> > > callstack. The commit message seemed to indicate it was for better
> > > restarting/cleanup; so I hope it is relatively low impact to back out.
> > >
> > > However, it now dies because it is still unable to create the br-int bridge:
> > >
> > > Unable to execute ['ovs-vsctl', '--timeout=10', '--oneline',
> > > '--format=json', '--', '--may-exist', 'add-br', 'br-int', '--',
> > > 'set', 'Bridge', 'br-int', 'datapath_type=system'].
> > > 2015-08-25 12:59:44.777 TRACE neutron.agent.ovsdb.impl_vsctl
> > > Traceback (most recent call last):
> > > 2015-08-25 12:59:44.777 TRACE neutron.agent.ovsdb.impl_vsctl File
> > > "/opt/stack/neutron/neutron/agent/ovsdb/impl_vsctl.py", line 63, in
> > > run_vsctl
> > > 2015-08-25 12:59:44.777 TRACE neutron.agent.ovsdb.impl_vsctl
> > > log_fail_as_error=False).rstrip()
> > > 2015-08-25 12:59:44.777 TRACE neutron.agent.ovsdb.impl_vsctl File
> > > "/opt/stack/neutron/neutron/agent/linux/utils.py", line 153, in execute
> > > 2015-08-25 12:59:44.777 TRACE neutron.agent.ovsdb.impl_vsctl raise
> > > RuntimeError(m)
> > > 2015-08-25 12:59:44.777 TRACE neutron.agent.ovsdb.impl_vsctl
> > > RuntimeError:
> > > 2015-08-25 12:59:44.777 TRACE neutron.agent.ovsdb.impl_vsctl
> Command:
> > > ['ovs-vsctl', '--timeout=10', '--oneline', '--format=json', '--',
> > > '--may-exist', 'add- br', 'br-int', '--', 'set', 'Bridge', 'br-int',
> > > 'datapath_type=system']
> > > 2015-08-25 12:59:44.777 TRACE neutron.agent.ovsdb.impl_vsctl Exit
> > > code: -14
> > > 2015-08-25 12:59:44.777 TRACE neutron.agent.ovsdb.impl_vsctl Stdin:
> > > 2015-08-25 12:59:44.777 TRACE neutron.agent.ovsdb.impl_vsctl Stdout:
> > > 2015-08-25 12:59:44.777 TRACE neutron.agent.ovsdb.impl_vsctl Stderr:
> > > 2015- 08-25T19:59:44Z|00002|fatal_signal|WARN|terminating with
> > > signal
> > > 14 (Alarm
> > > clock)
> > > 2015-08-25 12:59:44.777 TRACE neutron.agent.ovsdb.impl_vsctl
> > > 2015-08-25 12:59:44.777 TRACE neutron.agent.ovsdb.impl_vsctl
> > >
> > > I noticed a difference between a multi-node setup and a single node,
> > > where multi-node's controller has openvswitch and ovsdpdk for the
> > > Q_ML2_PLUGIN_MECHANISM_DRIVERS. Is it perhaps related to that?
> Or
> > > should it be trying to create br-int as datapath_type netdev instead?
> > > Just throwing things out there because I'm ignorant :-) I'll try
> > > variations of that in hopes I get lucky.
> > >
> > >
> > > Gabe
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Mooney, Sean K [mailto:sean.k.mooney at intel.com]
> > > > Sent: Tuesday, August 25, 2015 1:30 PM
> > > > To: Gabe Black; bugs at openvswitch.org
> > > > Subject: RE: duplicate option: of_interface
> > > >
> > > > Hi Gabe
> > > > We have started to see that message in our ci since this weekend.
> > > > We are currently investigating it but I belive a change has merged
> > > > to neuton that we need to back port to Our agent.
> > > >
> > > > A lot of code has merged in the last 2 weeks as the code freeze
> > > > for the liberty release is moday.
> > > > The stable kilo branch should be unaffected but we are actively
> > > > looking into this at present.
> > > >
> > > > Regards
> > > > Sean.
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Gabe Black [mailto:Gabe.Black at viavisolutions.com]
> > > > Sent: Tuesday, August 25, 2015 7:50 PM
> > > > To: bugs at openvswitch.org
> > > > Cc: Mooney, Sean K
> > > > Subject: duplicate option: of_interface
> > > >
> > > > I have followed the getting started guide
> > > > (http://git.openstack.org/cgit/stackforge/networking-ovs-
> > > > dpdk/tree/doc/source/getstarted.rst) on both fedora 21 and Ubuntu
> > > > 15.04 to get a single-node set up with dpdk ovs.
> > > >
> > > > My local.conf file is identical to the one provided as the single
> > > > node
> > > template:
> > > > http://git.openstack.org/cgit/stackforge/networking-ovs-
> > > > dpdk/tree/doc/source/_downloads/local.conf.single_node
> > > >
> > > > I set HOST_IP_IFACE=eno1, HOST_IP=10.3.73.124,
> > > > OVS_BRIDGE_MAPPINGS="default:br-enp4s0f0", and
> > > > ML2_VLAN_RANGES=default:1000:1010
> > > >
> > > > eno1 and associated IP is the interface/ip address of the server
> > > > (i.e. what
> > > we
> > > > use to ssh to the box). enp4s0f0 is the 10G intel nic interface that will
> > > > eventually be used for the data interface in a multi-node setup.
> > > > Finally the vlan range was just arbitrarily chosen.
> > > >
> > > > Other than that, there isn't anything else modified other than
> > > > following instructions of the getting started guide. However for
> > > > both Fedora 21, and Ubuntu 15.04 (Ubuntu there were some mods that
> > > > needed to take place like disabling apparmor, symlinking
> > > > /var/run/openstack, and fixing ovs-dpdk-init
> > > > script) result in the following error message in q-agt:
> > > >
> > > > Traceback (most recent call last):
> > > > File "/usr/bin/networking-ovs-dpdk-agent", line 10, in <module>
> > > > sys.exit(main())
> > > > File "/usr/lib/python2.7/site-
> > > >
> > packages/networking_ovs_dpdk/eventlet/ovs_dpdk_neutron_agent.py",
> > > > line 20, in main
> > > > agent_main.main()
> > > > File "/usr/lib/python2.7/site-
> > > > packages/networking_ovs_dpdk/agent/main.py", line 43, in main
> > > > mod = importutils.import_module(mod_name)
> > > > File
> > > > "/usr/lib/python2.7/site-packages/oslo_utils/importutils.py",
> > > > line 57, in import_module
> > > > __import__(import_str)
> > > > File "/usr/lib/python2.7/site-
> > > >
> > >
> >
> packages/networking_ovs_dpdk/agent/openflow/ovsdpdk_ofctl/main.py",
> > > > line 17, in <module>
> > > > from networking_ovs_dpdk.agent import ovs_dpdk_neutron_agent
> > > > File "/usr/lib/python2.7/site-
> > > > packages/networking_ovs_dpdk/agent/ovs_dpdk_neutron_agent.py",
> > > line
> > > > 47, in <module>
> > > > from neutron.plugins.ml2.drivers.openvswitch.agent import
> > > > ovs_dvr_neutron_agent
> > > > File
> > > >
> > >
> >
> "/opt/stack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_
> > > > dvr_neutron_agent.py", line 29, in <module>
> > > > cfg.CONF.import_group('AGENT',
> > > > 'neutron.plugins.ml2.drivers.openvswitch.'
> > > > File "/usr/lib/python2.7/site-packages/oslo_config/cfg.py", line
> > > > 2088, in import_group
> > > > __import__(module_str)
> > > > File
> > > >
> > >
> >
> "/opt/stack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/com
> > > > mon/config.py", line 111, in <module>
> > > > cfg.CONF.register_opts(ovs_opts, "OVS")
> > > > File "/usr/lib/python2.7/site-packages/oslo_config/cfg.py", line
> > > > 1824, in __inner
> > > > result = f(self, *args, **kwargs)
> > > > File "/usr/lib/python2.7/site-packages/oslo_config/cfg.py", line
> > > > 1983, in register_opts
> > > > self.register_opt(opt, group, clear_cache=False)
> > > > File "/usr/lib/python2.7/site-packages/oslo_config/cfg.py", line
> > > > 1828, in __inner
> > > > return f(self, *args, **kwargs)
> > > > File "/usr/lib/python2.7/site-packages/oslo_config/cfg.py", line
> > > > 1967, in register_opt
> > > > return group._register_opt(opt, cli)
> > > > File "/usr/lib/python2.7/site-packages/oslo_config/cfg.py", line
> > > > 1345, in _register_opt
> > > > if _is_opt_registered(self._opts, opt):
> > > > File "/usr/lib/python2.7/site-packages/oslo_config/cfg.py", line
> > > > 574, in _is_opt_registered
> > > > raise DuplicateOptError(opt.name)
> > > > oslo_config.cfg.DuplicateOptError: duplicate option: of_interface
> > > > q-agt failed to start
> > > >
> > > > I thought this error message was just because Ubuntu
> > > > testing/support hasn't been fleshed out yet with ovs-dpdk, but
> > > > then I got the exact same error on Fedora 21. I tried editing
> > > > both
> > > >
> > >
> >
> /opt/stack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/comm
> > > > on/config.py and /opt/stack/networking-ovs-
> > > > dpdk/networking_ovs_dpdk/common/config.py to get past the error,
> > but
> > > > then there are complaints about not finding br-int... So I'm
> > > > guessing that isn't the correct workaround. Anyone have any
> > > > suggestions of what I might have misconfigured?
> > > >
> > > > Thank you for your help!
> > > > Gabriel Black
> > > >
> > >
> > > _______________________________________________
> > > discuss mailing list
> > > discuss at openvswitch.org
> > > http://openvswitch.org/mailman/listinfo/discuss
More information about the discuss
mailing list