[ovs-discuss] low bandwidth for gre

Li, Chen chen.li at intel.com
Tue Jan 7 05:51:48 UTC 2014


eth4 MTU = 1500, and br-int MTU = 1200   =>  2.76 Gbits/sec

Also, I tested in Openstack environment, while all virtual ports MTU are 1500, and eth4 MTU = 9000
The bandwidth between  2 instances is about 2Gb/s.

Thanks.
-chen


-----Original Message-----
From: Zhou, Han [mailto:hzhou8 at ebay.com] 
Sent: Tuesday, January 07, 2014 1:46 PM
To: Li, Chen; Chris Luke; 'Jesse Gross'
Cc: discuss at openvswitch.org
Subject: RE: [ovs-discuss] low bandwidth for gre

Hi Chen,

Are you sure it is only MTU change resulted in such a huge difference?
It is hard to believe that fragmentation itself would lead to 9G -> 122K performance degradation.
Could you try eth4 MTU = 1500, and br-int MTU = 1200?

Best regards,
Han

-----Original Message-----
From: discuss-bounces at openvswitch.org [mailto:discuss-bounces at openvswitch.org] On Behalf Of Li, Chen
Sent: Tuesday, January 07, 2014 12:46 PM
To: Chris Luke; 'Jesse Gross'
Cc: discuss at openvswitch.org
Subject: Re: [ovs-discuss] low bandwidth for gre

O.
Thanks for the reply.
I just tried... And it works great !! 
I can achieve 9.84 Gbits/sec now while br-int has MTU 8000, while eth4 has MTU 9000.

Thanks a lot for the help !

-chen

-----Original Message-----
From: Chris Luke [mailto:chrisy at flirble.org]
Sent: Tuesday, January 07, 2014 11:46 AM
To: Li, Chen; 'Jesse Gross'
Cc: discuss at openvswitch.org
Subject: RE: [ovs-discuss] low bandwidth for gre

No, he means the physical interface needs a larger MTU than the logical one. The physical one has to transport the packets your logical interface accepts plus the header data for GRE. If there isn't enough headroom then it must split the packet in two and reassemble at the far end - a slow process.

Chris.




> -----Original Message-----
> From: discuss-bounces at openvswitch.org [mailto:discuss- 
> bounces at openvswitch.org] On Behalf Of Li, Chen
> Sent: Monday, January 06, 2014 22:23
> To: Jesse Gross
> Cc: discuss at openvswitch.org
> Subject: Re: [ovs-discuss] low bandwidth for gre
> 
> Sorry, I'm really new to network area.
> 
> So, you mean I should keep the sender under the low MTU, while the 
> receiver with higher MTU size ??
> 
> But, how I can make sure the sender is always the sender ????
> 
> Thanks.
> -chen
> 
> -----Original Message-----
> From: Jesse Gross [mailto:jesse at nicira.com]
> Sent: Tuesday, January 07, 2014 11:20 AM
> To: Li, Chen
> Cc: discuss at openvswitch.org
> Subject: Re: [ovs-discuss] low bandwidth for gre
> 
> If you change the MTU of both interfaces then it will still result in 
> fragmentation.
> 
> On Mon, Jan 6, 2014 at 9:50 PM, Li, Chen <chen.li at intel.com> wrote:
> > Yes, fragmentation do affect the bandwidth.
> >
> > I run command  to change interfaces' MTU size to test if 
> > fragmentation
> makes a difference.
> >         ifconfig eth4 mtu ${mtu_size}
> >         ifconfig br-int mtu ${mtu_size}
> >
> > The default MTU = 1500    =>    122 Kbits/sec
> >                     MTU = 3000    =>     419 Kbits/sec
> >                     MTU = 6000    =>     780 Kbits/sec
> >                     MTU = 9000    =>     1.14 Mbits/sec
> >
> >
> > But, what I don’t understand is, I didn’t see really high CPU%.
> >
> > Also, there is no other way to improve the bandwidth ?
> > Even under MTU=9000, 1.14 Mb/s vs. 10 Gb/s, the gap between the test
> result we can get and the physical NIC is unacceptable!!
> >
> >
> > Thanks.
> > -chen
> >
> >
> > -----Original Message-----
> > From: Jesse Gross [mailto:jesse at nicira.com]
> > Sent: Tuesday, January 07, 2014 10:32 AM
> > To: Li, Chen
> > Cc: discuss at openvswitch.org
> > Subject: Re: [ovs-discuss] low bandwidth for gre
> >
> > Well did you at least see if fragmentation makes a difference?
> >
> > On Mon, Jan 6, 2014 at 8:14 PM, Li, Chen <chen.li at intel.com> wrote:
> >> But 122 Kbits/s vs. 9.41Gb/s ??
> >>
> >> Is this correct ???
> >>
> >> How can I improve it ??
> >>
> >>
> >> Thanks.
> >> -chen
> >>
> >>
> >> -----Original Message-----
> >> From: Jesse Gross [mailto:jesse at nicira.com]
> >> Sent: Tuesday, January 07, 2014 12:24 AM
> >> To: Li, Chen
> >> Cc: discuss at openvswitch.org
> >> Subject: Re: [ovs-discuss] low bandwidth for gre
> >>
> >> On Sun, Jan 5, 2014 at 11:58 PM, Li, Chen <chen.li at intel.com> wrote:
> >>> Hi list,
> >>>
> >>>
> >>>
> >>> I’m working under CentOS, so the kernel is 
> >>> 2.6.32-358.123.2.openstack.el6.x86_64.
> >>>
> >>> openvswitch.x86_64      1.11.0_8ce28d-1.el6ost
> >>>
> >>>
> >>>
> >>> I have two physical machine in the test, each of them have a 10 Gb 
> >>> NIC
> card.
> >>>
> >>> The bandwidth between the two machine using iperf can achieve
> 9.41Gb/s.
> >>>
> >>>
> >>>
> >>> After enable gre follow guide
> >>> http://networkstatic.net/configuring-vxlan-and-gre-tunnels-on-open
> >>> vs
> >>> w
> >>> i
> >>> tch/#!prettyPhoto
> >>>
> >>> The bandwitdth between the two nodes are really low.
> >>>
> >>> Only  122 Kbits/s !!!!
> >>
> >> At a minimum, fragmentation is presumably occurring.
> _______________________________________________
> discuss mailing list
> discuss at openvswitch.org
> http://openvswitch.org/mailman/listinfo/discuss

_______________________________________________
discuss mailing list
discuss at openvswitch.org
http://openvswitch.org/mailman/listinfo/discuss


More information about the discuss mailing list