[ovs-discuss] 答复: 答复: ????: ????: [ovs]deleting interfaces connnected to ports results in ovs-vswitchd memory growing

[pengyi Peng(Yi)]

Yes, after backlog comes back to zero, memory is freed, and the memory of ovs-vswitchd stops growing.

However, if stream_send always returns -EAGAIN, the messages are held: "The send function will not block.  If no bytes can be immediately accepted for transmission, it returns -EAGAIN immediately."
In this situation, more messages waits to be sent,more memory is alloced, the memory of ovs-vswitchd will keep growing until messages are sent successfully and memory is freed. 

There is such a possibility indeed. 
1. create a bridge (ovs-vsctl add-br br-0). 
2.create a port in br-0 connected to physical interface eth0. 
3.create two Virtual machines which have 10 virtaual interfaces each. 
4.create 2000 ports connected to taps created by tunctl command. 

After an hour, the memory of ovs-vswitchd is about 310MB. And the backlog is over 200 million. I execute "ifconfig eth0 down". Soon, the messages are sent successfully and backlog slowly decreases. The ovs-vswitchd momory decreases too. If I just let it there and didn't ifconfig eth0 down, I believe the memory of ovs-vswitchd will be a horrible size. Can ovs prevent this situation? And why stream_send returns -EAGAIN ? 

-----邮件原件-----
发件人: Ben Pfaff [mailto:blp at nicira.com] 
发送时间: 2013年4月24日 1:37
收件人: pengyi Peng(Yi)
抄送: discuss at openvswitch.org; Liuyongan; Zhangkun (K); dev at openvswitch.org
主题: Re: 答复: ????: ????: [ovs-discuss] [ovs]deleting interfaces connnected to ports results in ovs-vswitchd memory growing

On Tue, Apr 23, 2013 at 01:56:17AM +0000, pengyi Peng(Yi) wrote:
> I find that when ovs-vswitchd memory is abnormally growing, backlog
> in struct rpc is growing fast too. Almost every time in jsonrpc_run,
> stream_send returns -EAGAIN, the msg holds and waits next time to
> send. The backlog pluses the strlen of next msg. So the backlog can
> be as big as 1228252. But after all ports are deleted, the backlog
> will slowly decrease and in the end be zero again. However, at this
> time, with top command, the memory of ovs-vswitchd doesn't change at
> all. It is still bigger than the memory before all interfaces are
> deleted.

None of the above is unexpected.  In particular, it is normal for
Unixs implementation of malloc() to hold on to allocated memory even
after it has been freed.

> And last night, I tested another situation. 
> First I create 200 ports which try to connect to interfaces that don't exist.
> (ovs-vsctl --timeout=5 --no-wait add-port br-0 tap-no-exist$i vlan_mode=access tag=0 -- set interface tap-no-exist$i type=system)
> 
> Second, I create 1000 ports connected to interfaces that exist.
> ( tunctl -t vif$i
> ifconfig vif$i up
> ovs-vsctl --timeout=5 --no-wait add-port br-0 vif$i vlan_mode=access tag=0 -- set interface vif$i type=system)
> 
> After all ports finish connecting to interfaces, the memory of
> ovs-vswitchd is 530832KB. Today it changes to 524444KB. But the
> backlog in struct rpc is still very big(455532592).

It is unusual to see the backlog grow so high.  We haven't observed
that behavior before when we've created and destroyed many interfaces,
or at least this is the first time it's been reported to me.  Can you
figure out why ovs-vswitchd is trying to commit so many transactions?
It would be better for it to do avoid doing that.  Maybe we can do
something to coalesce transactions or to skip some of them.