[ovs-discuss] ovs-vswitch Huge number of netlink file descriptors open

Ben Pfaff blp at ovn.org
Mon Oct 23 16:22:29 UTC 2017


Can you provide your configuration database, that is,
ovs-vswitchd.conf.db?

On Sat, Oct 21, 2017 at 01:44:42PM +0800, quan_hping at heetian.com wrote:
> Hi,All,
> The  file descriptors are "netlink".
>  $sudo lsof -p $(pidof ovs-vswitchd)
> ovs-vswit 2197 root *297u  netlink                         0t0   42309398 GENERIC
> ovs-vswit 2197 root *298u  netlink                         0t0   42309399 GENERIC
> ovs-vswit 2197 root *299u     sock                0,7      0t0   42309400 protocol: NETLINK
> ovs-vswit 2197 root *300u  netlink                         0t0   42309401 GENERIC
> ovs-vswit 2197 root *301u  netlink                         0t0   42309402 GENERIC
> ...More...
> ovs-vswit 2197 root *351u  netlink                         0t0   39326247 GENERIC
> ovs-vswit 2197 root *352u  netlink                         0t0   39121841 GENERIC
> ovs-vswit 2197 root *354w      REG              253,0 10086253 1342248977 /var/log/openvswitch/ovs-vswitchd.log
> $sudo reboot
> When I restart the machine ,the file descriptor will increasing.Some time more then 12K record.
> 
> Recenty log about openvswitch.
> ovsdb-server.log
> 2017-10-21T04:01:02.559Z|01308|jsonrpc|WARN|Dropped 7 log messages in last 17 seconds (most recently, 10 seconds ago) due to excessive rate
> 2017-10-21T04:01:02.559Z|01309|jsonrpc|WARN|unix: receive error: Connection reset by peer
> 2017-10-21T04:01:02.560Z|01310|reconnect|WARN|unix: connection dropped (Connection reset by peer)
> 2017-10-21T04:01:03.282Z|01311|reconnect|WARN|unix: connection dropped (Connection reset by peer)
> 2017-10-21T04:01:10.075Z|01312|jsonrpc|WARN|Dropped 1 log messages in last 7 seconds (most recently, 7 seconds ago) due to excessive rate
> 2017-10-21T04:01:10.075Z|01313|jsonrpc|WARN|unix: send error: Broken pipe
> 2017-10-21T04:01:10.076Z|01314|reconnect|WARN|unix: connection dropped (Broken pipe)
> 2017-10-21T04:01:14.907Z|01315|reconnect|WARN|unix: connection dropped (Connection reset by peer)
> 2017-10-21T04:01:20.134Z|01316|reconnect|WARN|unix: connection dropped (Broken pipe)
> 2017-10-21T04:01:27.499Z|01317|jsonrpc|WARN|Dropped 2 log messages in last 13 seconds (most recently, 8 seconds ago) due to excessive rate
> 2017-10-21T04:01:27.499Z|01318|jsonrpc|WARN|unix: receive error: Connection reset by peer
> 2017-10-21T04:01:27.499Z|01319|reconnect|WARN|unix: connection dropped (Connection reset by peer)
> 2017-10-21T04:01:33.758Z|01320|jsonrpc|WARN|unix: receive error: Connection reset by peer
> 2017-10-21T04:01:33.758Z|01321|reconnect|WARN|unix: connection dropped (Connection reset by peer)
> 2017-10-21T04:01:34.489Z|01322|reconnect|WARN|unix: connection dropped (Connection reset by peer)
> 2017-10-21T04:01:39.198Z|01323|reconnect|WARN|unix: connection dropped (Connection reset by peer)
> 2017-10-21T04:01:41.569Z|01324|reconnect|WARN|unix: connection dropped (Broken pipe)
> 2017-10-21T04:01:41.913Z|01325|reconnect|WARN|unix: connection dropped (Broken pipe)
> 2017-10-21T04:01:42.365Z|01326|reconnect|WARN|unix: connection dropped (Connection reset by peer)
> 
> ovs-vswitchd.log
> 2017-10-21T03:56:10.511Z|422666|netlink_socket|ERR|connect(0): Argument list too long
> 2017-10-21T03:56:10.513Z|422667|netlink_socket|ERR|connect(0): Argument list too long
> 2017-10-21T03:56:10.516Z|422668|netlink_socket|ERR|connect(0): Argument list too long
> 2017-10-21T03:56:10.518Z|422669|netlink_socket|ERR|connect(0): Argument list too long
> 2017-10-21T03:56:10.520Z|422670|netlink_socket|ERR|connect(0): Argument list too long
> 2017-10-21T03:56:10.523Z|422671|netlink_socket|ERR|connect(0): Argument list too long
> 2017-10-21T03:56:10.525Z|422672|netlink_socket|ERR|connect(0): Argument list too long
> 
> 
> 
> 合天网安实验室-您身边的信息安全实验室 | quan_hping at heetian.com
>  
> From: Ben Pfaff
> Date: 2017-10-21 01:41
> To: quan_hping at heetian.com
> CC: ovs-discuss
> Subject: Re: [ovs-discuss] ovs-vswitch Huge number of netlink file descriptors open
> On Fri, Oct 20, 2017 at 06:56:26PM +0800, quan_hping at heetian.com wrote:
> > Hi All,
> > I searched a mail list like my problem.
> > https://mail.openvswitch.org/pipermail/ovs-discuss/2017-March/043817.html 
> > 
> > $sudo lsof -p $(pidof ovs-vswitchd) | wc -l
> > 115536
> > $cat /proc/$(cat /var/run/openvswitch/ovs-vswitchd.pid)/limits | grep open
> > Max open files            999999                999999                files
> > $ovs-vsctl show | grep -c Port
> > 905
> > If anyone has any suggestions for how to solve it?Thanks.
>  
> Can you find out what kinds of file descriptors are open?  Are they
> sockets (and what kind), etc.?
>  
> This could be a file descriptor leak of some kind.  If it is, then
> restarting OVS would fix it; if it is not, then restarting OVS will not
> help, or at least not for long.  Can you figure out whether restarting
> reduces the number of file descriptors, and by how much?


More information about the discuss mailing list