[ovs-discuss] [bug]ovs-dpdk can't open dpdk port and dpdkvhostuser port added before

王华夏 wanghuaxia at jd.com
Thu Feb 25 07:49:30 UTC 2016


Thanks for your sharing , maybe this


Syslog :

Feb 24 18:44:43 A06-R12-302F0111-I32-13 ovs-vswitchd[123203]: ovs|00513|dpdk|ERR|vhost-user socket device setup failure for socket /var/run/openvswitch/tap57b09341-c9
Feb 24 18:44:43 A06-R12-302F0111-I32-13 ovs-vswitchd[123203]: ovs|00515|dpdk|ERR|vhost-user socket device setup failure for socket /var/run/openvswitch/tap6e8f58fa-c1
Feb 24 18:44:43 A06-R12-302F0111-I32-13 ovs-vswitchd[123203]: ovs|00517|dpdk|ERR|vhost-user socket device setup failure for socket /var/run/openvswitch/tap1522aef1-14
Feb 24 18:44:43 A06-R12-302F0111-I32-13 ovs-vswitchd[123203]: ovs|00519|dpdk|ERR|vhost-user socket device setup failure for socket /var/run/openvswitch/tap78ec2293-ae
Feb 24 18:44:43 A06-R12-302F0111-I32-13 ovs-vswitchd[123203]: ovs|00521|dpdk|ERR|vhost-user socket device setup failure for socket /var/run/openvswitch/tapb6496d1c-4c
Feb 24 18:44:43 A06-R12-302F0111-I32-13 ovs-vswitchd[123203]: ovs|00523|dpdk|ERR|vhost-user socket device setup failure for socket /var/run/openvswitch/tap511dcfea-e1
Feb 24 18:44:43 A06-R12-302F0111-I32-13 ovs-vswitchd[123203]: ovs|00525|dpdk|ERR|vhost-user socket device setup failure for socket /var/run/openvswitch/tapb94777b3-5d
Feb 24 18:44:44 A06-R12-302F0111-I32-13 ovs-vswitchd[123203]: ovs|00528|dpdk|ERR|vhost-user socket device setup failure for socket /var/run/openvswitch/tap57b09341-c9
Feb 24 18:44:44 A06-R12-302F0111-I32-13 ovs-vswitchd[123203]: ovs|00530|dpdk|ERR|vhost-user socket device setup failure for socket /var/run/openvswitch/tap6e8f58fa-c1
Feb 24 18:44:44 A06-R12-302F0111-I32-13 ovs-vswitchd[123203]: ovs|00532|dpdk|ERR|vhost-user socket device setup failure for socket /var/run/openvswitch/tap1522aef1-14
Feb 24 18:44:44 A06-R12-302F0111-I32-13 ovs-vswitchd[123203]: ovs|00534|dpdk|ERR|vhost-user socket device setup failure for socket /var/run/openvswitch/tap78ec2293-ae
Feb 24 18:44:44 A06-R12-302F0111-I32-13 ovs-vswitchd[123203]: ovs|00536|dpdk|ERR|vhost-user socket device setup failure for socket /var/run/openvswitch/tapb6496d1c-4c
Feb 24 18:44:44 A06-R12-302F0111-I32-13 ovs-vswitchd[123203]: ovs|00538|dpdk|ERR|vhost-user socket device setup failure for socket /var/run/openvswitch/tapb94777b3-5d

Ovs-vswitch log:

2016-02-24T10:24:09.555Z|00152|bridge|WARN|could not open network device tap9f4bd15d-51 (No such device)
2016-02-24T10:24:09.603Z|00153|dpdk|INFO|Socket /var/run/openvswitch/tap9f4bd15d-51 created for vhost-user port tap9f4bd15d-51
2016-02-24T10:24:09.604Z|00002|dpif_netdev(pmd71)|INFO|Core 10 processing port 'tap6e8f58fa-c1'
2016-02-24T10:24:09.604Z|00003|dpif_netdev(pmd71)|INFO|Core 10 processing port 'tap9f4bd15d-51'
2016-02-24T10:24:09.604Z|00154|bridge|INFO|bridge br-int: added interface tap9f4bd15d-51 on port 8
2016-02-24T10:24:10.471Z|00155|connmgr|INFO|br-tun<->unix: 1 flow_mods in the last 0 s (1 modifications)
2016-02-24T10:24:10.475Z|00156|connmgr|INFO|br-tun<->unix: 1 flow_mods in the last 0 s (1 adds)
2016-02-24T10:24:10.687Z|00157|connmgr|INFO|br-int<->unix: 1 flow_mods in the last 0 s (1 deletes)
2016-02-24T10:24:10.692Z|00158|connmgr|INFO|br-int<->unix: 1 flow_mods in the last 0 s (1 deletes)
2016-02-24T10:24:10.697Z|00159|ofp_util|INFO|Dropped 1 log messages in last 4012 seconds (most recently, 4012 seconds ago) due to excessive rate
2016-02-24T10:24:10.697Z|00160|ofp_util|INFO|normalization changed ofp_match, details:
2016-02-24T10:24:10.697Z|00161|ofp_util|INFO| pre: in_port=8,nw_proto=58,tp_src=136
2016-02-24T10:24:10.697Z|00162|ofp_util|INFO|post: in_port=8
2016-02-24T10:24:10.697Z|00163|connmgr|INFO|br-int<->unix: 1 flow_mods in the last 0 s (1 deletes)
2016-02-24T10:24:10.702Z|00164|connmgr|INFO|br-int<->unix: 1 flow_mods in the last 0 s (1 deletes)
2016-02-24T10:24:10.707Z|00165|connmgr|INFO|br-int<->unix: 1 flow_mods in the last 0 s (1 adds)
2016-02-24T10:24:10.712Z|00166|connmgr|INFO|br-int<->unix: 1 flow_mods in the last 0 s (1 adds)
2016-02-24T10:24:10.717Z|00167|connmgr|INFO|br-int<->unix: 1 flow_mods in the last 0 s (1 adds)
2016-02-24T10:24:10.722Z|00168|connmgr|INFO|br-int<->unix: 1 flow_mods in the last 0 s (1 adds)
2016-02-24T10:24:14.718Z|00003|daemon_unix(monitor)|ERR|1 crashes: pid 86760 died, killed (Aborted), core dumped, restarting
2016-02-24T10:24:14.743Z|00004|ovs_numa|INFO|Discovered 16 CPU cores on NUMA node 0
2016-02-24T10:24:14.743Z|00005|ovs_numa|INFO|Discovered 16 CPU cores on NUMA node 1
2016-02-24T10:24:14.743Z|00006|ovs_numa|INFO|Discovered 2 NUMA nodes and 32 CPU cores
2016-02-24T10:24:14.743Z|00007|memory|INFO|55836 kB peak resident set size after 4137.6 seconds
2016-02-24T10:24:14.743Z|00008|reconnect|INFO|unix:/var/run/openvswitch/db.sock: connecting...
2016-02-24T10:24:14.747Z|00009|reconnect|INFO|unix:/var/run/openvswitch/db.sock: connected
2016-02-24T10:24:14.781Z|00010|ofproto_dpif|INFO|netdev at ovs-netdev: Datapath supports recirculation
2016-02-24T10:24:14.781Z|00011|ofproto_dpif|INFO|netdev at ovs-netdev: MPLS label stack length probed as 3
2016-02-24T10:24:14.781Z|00012|ofproto_dpif|INFO|netdev at ovs-netdev: Datapath supports unique flow ids
2016-02-24T10:24:14.781Z|00013|ofproto_dpif|INFO|netdev at ovs-netdev: Datapath does not support ct_state
2016-02-24T10:24:14.781Z|00014|ofproto_dpif|INFO|netdev at ovs-netdev: Datapath does not support ct_zone
2016-02-24T10:24:14.781Z|00015|ofproto_dpif|INFO|netdev at ovs-netdev: Datapath does not support ct_mark
2016-02-24T10:24:14.781Z|00016|ofproto_dpif|INFO|netdev at ovs-netdev: Datapath does not support ct_label
2016-02-24T10:24:14.815Z|00017|dpdk|ERR|vhost-user socket device setup failure for socket /var/run/openvswitch/tap9f4bd15d-51
2016-02-24T10:24:14.815Z|00018|bridge|WARN|could not open network device tap9f4bd15d-51 (Unknown error -1)
2016-02-24T10:24:14.815Z|00019|dpdk|ERR|vhost-user socket device setup failure for socket /var/run/openvswitch/tap57b09341-c9
2016-02-24T10:24:14.815Z|00020|bridge|WARN|could not open network device tap57b09341-c9 (Unknown error -1)
2016-02-24T10:24:14.815Z|00021|dpdk|ERR|vhost-user socket device setup failure for socket /var/run/openvswitch/tap1522aef1-14
2016-02-24T10:24:14.816Z|00022|bridge|WARN|could not open network device tap1522aef1-14 (Unknown error -1)
2016-02-24T10:24:14.816Z|00023|dpdk|ERR|vhost-user socket device setup failure for socket /var/run/openvswitch/tap6e8f58fa-c1
2016-02-24T10:24:14.816Z|00024|bridge|WARN|could not open network device tap6e8f58fa-c1 (Unknown error -1)
2016-02-24T10:24:14.816Z|00025|dpdk|ERR|vhost-user socket device setup failure for socket /var/run/openvswitch/tap78ec2293-ae
2016-02-24T10:24:14.816Z|00026|bridge|WARN|could not open network device tap78ec2293-ae (Unknown error -1)
2016-02-24T10:24:14.816Z|00027|bridge|INFO|bridge br-int: added interface patch-tun on port 1
2016-02-24T10:24:14.816Z|00028|dpdk|ERR|vhost-user socket device setup failure for socket /var/run/openvswitch/tapb6496d1c-4c
2016-02-24T10:24:14.816Z|00029|bridge|WARN|could not open network device tapb6496d1c-4c (Unknown error -1)
2016-02-24T10:24:14.817Z|00030|bridge|INFO|bridge br-int: added interface br-int on port 65534
2016-02-24T10:24:14.817Z|00031|dpdk|ERR|vhost-user socket device setup failure for socket /var/run/openvswitch/tapb94777b3-5d
2016-02-24T10:24:14.817Z|00032|bridge|WARN|could not open network device tapb94777b3-5d (Unknown error -1)
2016-02-24T10:24:14.817Z|00033|bridge|INFO|bridge br-tun: added interface patch-int on port 1
2016-02-24T10:24:14.817Z|00034|bridge|INFO|bridge br-tun: added interface vxlan-0a001f8d on port 3
2016-02-24T10:24:14.818Z|00035|bridge|INFO|bridge br-tun: added interface br-tun on port 65534
2016-02-24T10:24:14.818Z|00036|bridge|INFO|bridge br-tun: added interface vxlan-0a002016 on port 4
2016-02-24T10:24:14.818Z|00037|bridge|INFO|bridge br-tun: added interface vxlan-0a001f7e on port 2
2016-02-24T10:24:14.818Z|00038|bridge|WARN|could not open network device dpdk0 (Cannot allocate memory)
2016-02-24T10:24:14.818Z|00039|bridge|INFO|bridge br-dpdk: added interface br-dpdk on port 65534
2016-02-24T10:24:14.819Z|00040|bridge|INFO|bridge br-int: using datapath ID 00000ac7455f064c
2016-02-24T10:24:14.819Z|00041|connmgr|INFO|br-int: added service controller "punix:/var/run/openvswitch/br-int.mgmt"
2016-02-24T10:24:14.819Z|00042|bridge|INFO|bridge br-tun: using datapath ID 00008affbc1d314f
2016-02-24T10:24:14.819Z|00043|connmgr|INFO|br-tun: added service controller "punix:/var/run/openvswitch/br-tun.mgmt"
2016-02-24T10:24:14.819Z|00044|bridge|INFO|bridge br-dpdk: using datapath ID 00001e3c7b157245
2016-02-24T10:24:14.819Z|00045|connmgr|INFO|br-dpdk: added service controller "punix:/var/run/openvswitch/br-dpdk.mgmt"
2016-02-24T10:24:14.850Z|00046|dpdk|ERR|vhost-user socket device setup failure for socket /var/run/openvswitch/tap9f4bd15d-51


Indeed I have set as --socket-mem 400  and tested for long time and did not have the same problem ,so it would not be lack of memory



Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: Christian Ehrhardt<mailto:christian.ehrhardt at canonical.com>
Sent: 2016Äê2ÔÂ25ÈÕ 15:07
To: Íõ»ªÏÄ<mailto:wanghuaxia at jd.com>
Cc: discuss at openvswitch.org<mailto:discuss at openvswitch.org>
Subject: Re: [ovs-discuss] [bug]ovs-dpdk can't open dpdk port and dpdkvhostuser port added before

Hi Eric,
when debugging similar issues I realized that while the openvswitch log (like /var/log/openvswitch/ovs-vswitchd.log depending on your Distro) is great.
The DPDK messages can bypass that and at least in my case ended up in syslog.

Therein you will find messages like
ovs-ctl[3560]: EAL: ...
ovs-vswitchd[3592]: EAL: ...

And DPDK there is rather verbose on its initialization (which is good).
Since you can trigger the issue by different --socket-mem options I'd recommend you to check these logs.
Most of the time, once I looked there it was eventually rather obvious.

Regarding the memory allocation you will find for example (in my case non socket based -m 4096)
ovs-ctl[3560]: EAL: Requesting 4 pages of size 1024MB from socket 0

If it is not obvious please share the logs you find there and people on the list might be able to help you more.


Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd

On Thu, Feb 25, 2016 at 7:22 AM, Íõ»ªÏÄ <wanghuaxia at jd.com<mailto:wanghuaxia at jd.com>> wrote:

HI

I am trying to restart ovs ,find  error as fellows, all thess bridges and ports I have added before

Bridge br-dpdk
        Port "dpdk0"
            Interface "dpdk0"
                type: dpdk
                error: "could not open network device dpdk0 (Cannot allocate memory)"
        Port br-dpdk
            Interface br-dpdk
                type: internal
    Bridge br-tun
        fail_mode: secure
        Port "vxlan-0a001f8d"
            Interface "vxlan-0a001f8d"
                type: vxlan
                options: {df_default="true", in_key=flow, local_ip="10.0.32.13", out_key=flow, remote_ip="10.0.31.141"}
        Port "vxlan-0a001f7e"
            Interface "vxlan-0a001f7e"
                type: vxlan
                options: {df_default="true", in_key=flow, local_ip="10.0.32.13", out_key=flow, remote_ip="10.0.31.126"}
        Port "vxlan-0a002016"
            Interface "vxlan-0a002016"
                type: vxlan
                options: {df_default="true", in_key=flow, local_ip="10.0.32.13", out_key=flow, remote_ip="10.0.32.22"}
        Port patch-int
            Interface
 patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
    Bridge br-int
        fail_mode: secure
        Port br-int
            Interface br-int
                type: internal
        Port "tap78ec2293-ae"
            tag: 1
            Interface "tap78ec2293-ae"
                type: dpdkvhostuser
                error: "could not open network device tap78ec2293-ae (Unknown error -1)"
        Port "tapb94777b3-5d"
            tag: 1


When start the ovs like this ,this problem will occurs:

ovs-vswitchd --dpdk -c 0x1 -n 4 --socket-mem=2048,2048 -- unix:/var/run/openvswitch/db.sock --pidfile --detach -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log ¨Cmonitor

if change the --socket-mem=2048,2048   to   ¨Csocket-mem= 2048 ,  it will have no problem

if it  means the numa node1 can not use ?

my ovs version is 2.5 and dpdk version is 2.2 with kernel Linux version 3.10.0-229.20.1.el7.x86_64 (builder at kbuilder.dev.centos.org<mailto:builder at kbuilder.dev.centos.org>) (gcc version 4.8.3 20140911 (Red Hat 4.8.3-9) (GCC) ) #1 SMP Tue Nov 3 19:10:07 UTC 2015


Thanks & regards

Eric wang

·¢ËÍ×Ô Windows 10 °æÓʼþ<https://go.microsoft.com/fwlink/?LinkId=550986>Ó¦ÓÃ


_______________________________________________
discuss mailing list
discuss at openvswitch.org<mailto:discuss at openvswitch.org>
http://openvswitch.org/mailman/listinfo/discuss


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20160225/c8b20b5d/attachment-0002.html>


More information about the discuss mailing list