[ovs-discuss] Gre tunnel segmenting packets to 1500 bytes
Kristoffer Egefelt
kristoffer at itoc.dk
Mon Jul 28 09:29:23 UTC 2014
Do you mean that GRE offloading is needed for OVS to forward GSO/GRO packets ?
The funny thing is, that it seems to be the VMs MTU that determines what the GRE MTU will be.
So at VM mtu 1500, packet sizes are:
VM: 64K
VIF: 64K
Bridge: 64K
GRE: 1500 bytes
NIC: 1500 bytes
At VM mtu 9000, packet sizes are:
VM: 64K
VIF: 64K
Bridge: 64K
GRE: 9000 bytes
NIC: 9000 bytes
Without GRE and and mtu whatever, sizes are:
VM: 64K
VIF: 64K
Bridge: 64K
GRE: 64K
NIC: 64K
Does this makes sense?
Thanks
Regards
Kristoffer
On 25/07/2014, at 16.34.48, Gurucharan Shetty <shettyg at nicira.com> wrote:
> If you are talking about why you are getting ~2 Gbps and not 10 Gbps
> using GRE tunnel, then I think it is expected. I haven't gotten more
> than 4 Gbps using GRE tunnel (with a lot of network optimizations like
> setting affinity etc). If you expect to get 10 Gbps, you should use a
> tunnel that has some help from the NIC with offloading.
>
> On Fri, Jul 25, 2014 at 4:05 AM, Kristoffer Egefelt <kristoffer at itoc.dk> wrote:
>> I just confirmed that without the GRE tunnel, it works.
>> So it seems the problem is in the GRE handling in or after openvswitch.
>>
>> I’m not sure if this is actually supposed to work with GRE packets - do the NIC drivers need to support GRE offload for this to work?
>> Or could it be a kernel issue - as I understand it OVS 2.1 uses the kernels GRE implementation?
>>
>>
>>
>> On 25/07/2014, at 11.15.30, Kristoffer Egefelt <kristoffer at itoc.dk> wrote:
>>
>>> Hi Flavio,
>>>
>>>> On Thu, Jul 24, 2014 at 01:47:32PM +0200, Kristoffer Egefelt wrote:
>>>> [...]
>>>>> - Packets egressing openvswitch over a gre tunnel is segmented to
>>>>> 1500 bytes. If I configure mtu 9000 in the VM, the packets are
>>>>> segmented to 9000 bytes on the gre tunnel - in any case tcpdump in
>>>>> the sending VM shows packet sizes of 62636 bytes - strange ?
>>>>
>>>> Packets can go up to 64k when segmentation offloading is enabled
>>>> on the interface.
>>>
>>>
>>> Right, so the trigger here could be netback/netfront or the kernel or the NIC driver transporting GRE packets?
>>> I’m not sure what to expect here - if all offloading is enabled in the entire path, should GSO/GRO 64K packets be able to traverse:
>>>
>>> VM1-> Netfront -> Netback -> OVS bridge -> OVS GRE bridge -> NIC -> switch and
>>> Switch -> NIC -> OVS GRE bridge -> OVS bridge -> Netback -> Netfront -> VM2
>>>
>>> all without being segmented, other than below the NIC driver and in the switch ?
>>>
>>> Do you know if this is supposed to just work - I might be hitting some weird bug here. Using intel IXGBE, kernel 3.13, ovs 2.1.
>>>
>>> Offloading in NICs and VIFs and VMs eth0 included below:
>>>
>>> — Hypervisor:
>>> Features for eth0:
>>> rx-checksumming: on
>>> tx-checksumming: on
>>> tx-checksum-ipv4: on
>>> tx-checksum-ip-generic: off [fixed]
>>> tx-checksum-ipv6: on
>>> tx-checksum-fcoe-crc: on [fixed]
>>> tx-checksum-sctp: on
>>> scatter-gather: on
>>> tx-scatter-gather: on
>>> tx-scatter-gather-fraglist: off [fixed]
>>> tcp-segmentation-offload: on
>>> tx-tcp-segmentation: on
>>> tx-tcp-ecn-segmentation: off [fixed]
>>> tx-tcp6-segmentation: on
>>> udp-fragmentation-offload: off [fixed]
>>> generic-segmentation-offload: on
>>> generic-receive-offload: on
>>> large-receive-offload: off
>>> rx-vlan-offload: on
>>> tx-vlan-offload: on
>>> ntuple-filters: off
>>> receive-hashing: on
>>> highdma: on [fixed]
>>> rx-vlan-filter: on [fixed]
>>> vlan-challenged: off [fixed]
>>> tx-lockless: off [fixed]
>>> netns-local: off [fixed]
>>> tx-gso-robust: off [fixed]
>>> tx-fcoe-segmentation: on [fixed]
>>> tx-gre-segmentation: off [fixed]
>>> tx-udp_tnl-segmentation: off [fixed]
>>> tx-mpls-segmentation: off [fixed]
>>> fcoe-mtu: off [fixed]
>>> tx-nocache-copy: on
>>> loopback: off [fixed]
>>> rx-fcs: off [fixed]
>>> rx-all: off [fixed]
>>> tx-vlan-stag-hw-insert: off [fixed]
>>> rx-vlan-stag-hw-parse: off [fixed]
>>> rx-vlan-stag-filter: off [fixed]
>>>
>>>
>>> — vif
>>> Features for vif1.0:
>>> rx-checksumming: off [fixed]
>>> tx-checksumming: on
>>> tx-checksum-ipv4: on
>>> tx-checksum-ip-generic: off [fixed]
>>> tx-checksum-ipv6: off [fixed]
>>> tx-checksum-fcoe-crc: off [fixed]
>>> tx-checksum-sctp: off [fixed]
>>> scatter-gather: on
>>> tx-scatter-gather: on
>>> tx-scatter-gather-fraglist: off [fixed]
>>> tcp-segmentation-offload: on
>>> tx-tcp-segmentation: on
>>> tx-tcp-ecn-segmentation: off [fixed]
>>> tx-tcp6-segmentation: off [fixed]
>>> udp-fragmentation-offload: off [fixed]
>>> generic-segmentation-offload: off
>>> generic-receive-offload: on
>>> large-receive-offload: off [fixed]
>>> rx-vlan-offload: off [fixed]
>>> tx-vlan-offload: off [fixed]
>>> ntuple-filters: off [fixed]
>>> receive-hashing: off [fixed]
>>> highdma: off [fixed]
>>> rx-vlan-filter: off [fixed]
>>> vlan-challenged: off [fixed]
>>> tx-lockless: off [fixed]
>>> netns-local: off [fixed]
>>> tx-gso-robust: off [fixed]
>>> tx-fcoe-segmentation: off [fixed]
>>> tx-gre-segmentation: off [fixed]
>>> tx-udp_tnl-segmentation: off [fixed]
>>> tx-mpls-segmentation: off [fixed]
>>> fcoe-mtu: off [fixed]
>>> tx-nocache-copy: on
>>> loopback: off [fixed]
>>> rx-fcs: off [fixed]
>>> rx-all: off [fixed]
>>> tx-vlan-stag-hw-insert: off [fixed]
>>> rx-vlan-stag-hw-parse: off [fixed]
>>> rx-vlan-stag-filter: off [fixed]
>>>
>>> — VM
>>> Features for eth0:
>>> rx-checksumming: on [fixed]
>>> tx-checksumming: on
>>> tx-checksum-ipv4: on
>>> tx-checksum-ip-generic: off [fixed]
>>> tx-checksum-ipv6: off [fixed]
>>> tx-checksum-fcoe-crc: off [fixed]
>>> tx-checksum-sctp: off [fixed]
>>> scatter-gather: on
>>> tx-scatter-gather: on
>>> tx-scatter-gather-fraglist: off [fixed]
>>> tcp-segmentation-offload: on
>>> tx-tcp-segmentation: on
>>> tx-tcp-ecn-segmentation: off [fixed]
>>> tx-tcp6-segmentation: off [fixed]
>>> udp-fragmentation-offload: off [fixed]
>>> generic-segmentation-offload: on
>>> generic-receive-offload: on
>>> large-receive-offload: off [fixed]
>>> rx-vlan-offload: off [fixed]
>>> tx-vlan-offload: off [fixed]
>>> ntuple-filters: off [fixed]
>>> receive-hashing: off [fixed]
>>> highdma: off [fixed]
>>> rx-vlan-filter: off [fixed]
>>> vlan-challenged: off [fixed]
>>> tx-lockless: off [fixed]
>>> netns-local: off [fixed]
>>> tx-gso-robust: on [fixed]
>>> tx-fcoe-segmentation: off [fixed]
>>> tx-gre-segmentation: off [fixed]
>>> tx-ipip-segmentation: off [fixed]
>>> tx-sit-segmentation: off [fixed]
>>> tx-udp_tnl-segmentation: off [fixed]
>>> tx-mpls-segmentation: off [fixed]
>>> fcoe-mtu: off [fixed]
>>> tx-nocache-copy: on
>>> loopback: off [fixed]
>>> rx-fcs: off [fixed]
>>> rx-all: off [fixed]
>>> tx-vlan-stag-hw-insert: off [fixed]
>>> rx-vlan-stag-hw-parse: off [fixed]
>>> rx-vlan-stag-filter: off [fixed]
>>> l2-fwd-offload: off [fixed]
>>>
>>> Thanks for your help.
>>>
>>> Regards
>>> Kristoffer
>>> _______________________________________________
>>> discuss mailing list
>>> discuss at openvswitch.org
>>> http://openvswitch.org/mailman/listinfo/discuss
>>>
>>
>> _______________________________________________
>> discuss mailing list
>> discuss at openvswitch.org
>> http://openvswitch.org/mailman/listinfo/discuss
> _______________________________________________
> discuss mailing list
> discuss at openvswitch.org
> http://openvswitch.org/mailman/listinfo/discuss
More information about the discuss
mailing list