[ovs-discuss] performance drop 60% when using vxlan tunnel

Jesse Gross jesse at nicira.com
Wed Nov 12 21:18:15 UTC 2014


On Tue, Nov 11, 2014 at 10:52 PM, FengYu LeiDian
<fengyuleidian0615 at gmail.com> wrote:
> 于 2014年11月06日 23:27, Jesse Gross 写道:
>
>> On Wed, Nov 5, 2014 at 10:58 PM, FengYu LeiDian
>> <fengyuleidian0615 at gmail.com> wrote:
>>>
>>> 于 2014年11月06日 00:08, Jesse Gross 写道:
>>>
>>>> On Tue, Nov 4, 2014 at 11:03 PM, FengYu LeiDian
>>>> <fengyuleidian0615 at gmail.com> wrote:
>>>>>
>>>>>
>>>>> Hi
>>>>>
>>>>> Env: redhat 6.4, OpenvSwitch-2.1.2, using native
>>>>> datapath/linux/openvswitch.ko module
>>>>>
>>>>> VM1 on host1, VM2 on host2, host1 and host2 are connected by a switch.
>>>>> both VM enable virtio/vhost when launching VM
>>>>>
>>>>> case1:
>>>>> VM -> tap -> ovs-bridge -> eth1
>>>>>
>>>>> case2:
>>>>> VM -> tap -> ovs-bridge -> vxlan -> eth1
>>>>>
>>>>> When using vxlan in case2, iperf performance drop 60%,
>>>>
>>>>
>>>>
>>>> Your NIC probably doesn't support offloads (checksum, TSO, etc.) in
>>>> the presence of VXLAN.
>>>
>>>
>>>
>>> This is the default features supported by my NIC 82599 both in case1 and
>>> case2.
>>
>>
>> I can assure you that this NIC does not support VXLAN and the stack is
>> being forced to do segmentation in software.
>
> Sorry for the late reply.
>
> Do you mean NIC supports VXLAN offload?
> Yes, my 82599EB does not have this feature.
>
>
> After some debugging, when VM sending small packet,
>
> In case2: performance 80Mbps
> VM -> tap -> ovs-bridge -> vxlan -> eth1
>
> tcpdump tap0 observed packet size bigger than 1000 Bytes because
> of VM eth TSO involved.
>
> While tcpdump in eth1 observed small packets with the same size 192Bytes.
> Looks like skb segmentation happens somewhere before vxlan.
>
> I tried to set vxlan1 interface MTU by below, no luck.
> ovs-vsctl add-port ovs-br0 vxlan1 -- set interface vxlan1 type=vxlan
> options:remote_ip=192.168.1.5 options:key=99 options:dst_port=4789
> options:df_default=true mtu=9000
>
>
> In case1: performance 400Mbps
> VM -> tap -> ovs-bridge -> eth1
>
> Configuration is same as Case2, only without vxlan interface.
> Both tap and eth1 observed bigger packet size.

The packets appear to be larger because segmentation is being done on
the NIC instead of in software.



More information about the discuss mailing list