[ovs-discuss] How does MTU work with tunnels?
gvrose8192 at gmail.com
Tue Sep 10 21:57:08 UTC 2019
On 9/10/2019 7:07 AM, Heim, Dennis wrote:
> Your MTU on VXLAN needs to allow for give or take around 50 bytes. Either, you would need to move up to larger frames (9k frames) or decrease your MTU on the end stations. If I could I would have enabled Jumbo frames, but that wasn't an option for various reasons. My understanding is by default you get 1500 byte ethernet frames. Then, you add tunnel overhead ~50 bytes, which varies between VXLAN, GENEVE, GRE. Then you get a packet with 1550 bytes. Your underlay network (physical network), needs to support MTU of 1550 or greater. Otherwise you get fragmentation, and unexpected performance. I am not a network engineer, so this is mostly what I learned through sleepless nights.
> Dennis Heim | Domain Architect (Collaboration Labs)
> World Wide Technology, Inc. | +1 314-212-1814
Recommended MTU for end stations on overlay network is 1600 bytes. See
Begin quote -------------------------------
When leveraging encapsulation technologies, it is important to increase
the MTU supported both on transport nodes as well as on all interfaces
of the devices deployed in the physical network. For NSX-T, the
documentation refers to a preferred MTU size of 1600 or above.
Actually, my personal recommendation is 9000: why bothering increasing
to 1600 when you can do 9000? It’s only a maximum after all! :)
Different possible scenarios have different requirements:
When leveraging NSX-T only for the distributed firewall, you don’t need
to increase the MTU. Nevertheless, I always recommend it for long-term
plans (in case of network virtualization is introduced later).
When you want to carry overlay traffic for network virtualization,
increasing the MTU to a minimum of 1600 bytes is recommended (same rule
I hope this helps.
> “The most powerful person in the world is the story teller. The storyteller sets the vision, values and agenda of an entire generation that is to come” – Steve Jobs
> “Leadership isn’t a different maker. It is the difference maker” – Tim Kight
> "Leaders who don't listen will eventually be surrounded by people who have nothing to say" --- Andy Stanley
> "Worry less about who you might offend, and more about who you might inspire" -- Tim Allen
> “Imagination is more important than knowledge.” -- Albert Einstein
> “If you can raise the level of effort and performance in those around you, you are officially a leader” – Urban Meyer
> “The greatest danger for most of us is not that our aim is too high and we miss it, but that it is too low and we reach it.” -- Michelangelo Buonarroti
> “Mediocore managers play checkers (assuming everyone is the same). Great managers play chess (acknowledging that everyone is unique)” – Marcus Buckingham
> “If you’re not failing every now and again, it’s a sign you’re not doing anything very innovative” – Woody Allen
> Click here to join me in my Collaboration Meeting Room
> -----Original Message-----
> From: Benjamin <benjamin.reis at vates.fr>
> Sent: Tuesday, September 10, 2019 9:49 AM
> To: Heim, Dennis <Dennis.Heim at wwt.com>; ovs-discuss at openvswitch.org
> Subject: Re: [ovs-discuss] How does MTU work with tunnels?
> Thanks for your quick answer.
> But i'm not sure I understand what you're saying: max MTU for VxLAN would be 1320?
> Le 10/09/2019 à 15:26, Heim, Dennis a écrit :
>> I run VXLAN over a DMVPN setup, as it is part of our Lab/PoC architecture (distributed). From an MTU perspective, Windows is happy with 1360, but some linux/apache servers require it set to 1320 to work properly.
>> Dennis Heim | Domain Architect (Collaboration Labs) World Wide
>> Technology, Inc. | +1 314-212-1814
>> "The most powerful person in the world is the story teller. The
>> storyteller sets the vision, values and agenda of an entire generation
>> that is to come" - Steve Jobs "Leadership isn't a different maker. It
>> is the difference maker" - Tim Kight "Leaders who don't listen will
>> eventually be surrounded by people who have nothing to say" --- Andy
>> Stanley "Worry less about who you might offend, and more about who you
>> might inspire" -- Tim Allen "Imagination is more important than
>> knowledge." -- Albert Einstein "If you can raise the level of effort
>> and performance in those around you, you are officially a leader" -
>> Urban Meyer "The greatest danger for most of us is not that our aim is
>> too high and we miss it, but that it is too low and we reach it." --
>> Michelangelo Buonarroti "Mediocore managers play checkers (assuming
>> everyone is the same). Great managers play chess (acknowledging that
>> everyone is unique)" - Marcus Buckingham "If you're not failing every
>> now and again, it's a sign you're not doing anything very innovative"
>> - Woody Allen
>> Click here to join me in my Collaboration Meeting Room
>> -----Original Message-----
>> From: ovs-discuss-bounces at openvswitch.org
>> <ovs-discuss-bounces at openvswitch.org> On Behalf Of Benjamin
>> Sent: Tuesday, September 10, 2019 9:04 AM
>> To: ovs-discuss at openvswitch.org
>> Subject: [ovs-discuss] How does MTU work with tunnels?
>> Hi all,
>> I working with openvswitch 2.5.3 (tried with 2.11.0 same behavior) on XCP-ng (XenServer fork).
>> I'm trying to undersand how the mtu works with GRE and VxLAN tunnels.
>> As for now when I create a GRE tunnel with any MTU, I think the MTU is not taken in consideration because i can ping -s 15000 on the corresponding interface.
>> However, with VxLAN if I set a MTU greater than 1450 then the MTU is
>> 1450 and if I set a MTU lower than 1450 I have same behavior as for GRE.
>> So I'm a but confused.
>> When I check the mtu value with : ovs-vsctl get int xapi0 mtu_request or ovs-vsctl get int xapi0 mtu the returned value is what I expect.
>> Thanks in advance,
>> discuss mailing list
>> discuss at openvswitch.org
> discuss mailing list
> discuss at openvswitch.org
More information about the discuss