[ovs-dev] HA: SNAT on OVN logical_router in userspace works for ICMP but not TCP or UDP

Rostyslav Fridman Rostyslav_Fridman at epam.com
Fri Feb 8 20:04:07 UTC 2019


> How about dumping flows and conntrack entries and checking stats at various points ?
> ovs-ofctl dump-flows <bridge>
> ovs-appctl dpif/dump-flows <bridge>
> ovs-appctl dpctl/dump-conntrack

Please find flow dumps at the following link: https://pastebin.com/raw/epKAhTKm

> How are you sending said UDP/TCP packets ?

Curl and telnet requests from the container.

--
Best regards,
Rostyslav Fridman

-----Исходное сообщение-----
От: Darrell Ball [mailto:dball at vmware.com] 
Отправлено: 8 февраля 2019 г. 20:59
Кому: Ben Pfaff <blp at ovn.org>; Rostyslav Fridman <Rostyslav_Fridman at epam.com>
Копия: ovs-dev at openvswitch.org; Vasyl Samoilov <Vasyl_Samoilov at epam.com>
Тема: Re: [ovs-dev] SNAT on OVN logical_router in userspace works for ICMP but not TCP or UDP


We have advanced system tests for userspace datapath to test OVN, including tcp packets.

system-ovn

124: ovn -- 2 LRs connected via LS, gateway router, SNAT and DNAT ok
125: ovn -- 2 LRs connected via LS, gateway router, easy SNAT ok
126: ovn -- multiple gateway routers, SNAT and DNAT  ok
127: ovn -- load-balancing                           ok
128: ovn -- load-balancing - same subnet.            ok
129: ovn -- load balancing in gateway router         ok
130: ovn -- multiple gateway routers, load-balancing ok
131: ovn -- load balancing in router with gateway router port ok
132: ovn -- DNAT and SNAT on distributed router - N/S ok
133: ovn -- DNAT and SNAT on distributed router - E/W ok

Let us define the problem first since the context is mostly undefined

How about dumping flows and conntrack entries and checking stats at various points ?

ovs-ofctl dump-flows <bridge>
ovs-appctl dpif/dump-flows <bridge>
ovs-appctl dpctl/dump-conntrack

How are you sending said UDP/TCP packets ?



On 2/8/19, 10:15 AM, "ovs-dev-bounces at openvswitch.org on behalf of Ben Pfaff" <ovs-dev-bounces at openvswitch.org on behalf of blp at ovn.org> wrote:

    Darrell, is this something you can help with?
    
    On Fri, Feb 08, 2019 at 02:18:53PM +0000, Rostyslav Fridman via dev wrote:
    > I've encountered the issue that SNAT on OVN logical_router in userspace works for ICMP but not TCP or UDP. I am seeing this behavior on version 2.10.1 as well as on top of the git tree.
    > 
    > I try to access internet (216.58.215.110) from container (10.0.0.2). On the external-router I have SNAT configured. On the external server I see that container address is translated for ICMP request, but not for TCP.
    > container:/# ping 216.58.215.110
    > PING 216.58.215.110 (216.58.215.110) 56(84) bytes of data.
    > 64 bytes from 216.58.215.110: icmp_seq=1 ttl=53 time=140 ms
    > ^C
    > --- 216.58.215.110 ping statistics ---
    > 1 packets transmitted, 1 received, 0% packet loss, time 0ms
    > rtt min/avg/max/mdev = 140.663/140.663/140.663/0.000 ms
    > container:/# curl 216.58.215.110
    > ^C
    > ---
    > external-server:~# tcpdump -i vlan111 host 216.58.215.110
    > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    > listening on vlan111, link-type EN10MB (Ethernet), capture size 262144 bytes
    > 07:53:22.393289 IP 10.250.111.40 > waw02s17-in-f14.1e100.net: ICMP echo request, id 218, seq 1, length 64
    > 07:53:22.533574 IP waw02s17-in-f14.1e100.net > 10.250.111.40: ICMP echo reply, id 218, seq 1, length 64
    > 07:53:24.830595 IP 10.0.0.2.58050 > waw02s17-in-f14.1e100.net.http: Flags [S], seq 219699121, win 29200, options [mss 1460,sackOK,TS val 2742820693 ecr 0,nop,wscale 7], length 0
    > 
    > In the bridge flows I see that NAT should be performed since flow packet count is increasing.
    > ovs-appctl bridge/dump-flows br-int
    > ...
    > table_id=41, duration=5135s, n_packets=28, n_bytes=2408, priority=9,ip,metadata=0x1,nw_src=10.0.0.0/8,actions=ct(commit,table=42,zone=NXM_NX_REG12[0..15],nat(src=10.250.111.40))
    > 
    > ovn-trace also confirms that it should be working.
    > 
    > I have the following scheme:
    > OVS: trunked bonded port --- netdev bridge (br-ext) --- patch --- netdev bridge (br-int)
    > OVN: container --- logical_switch (internal-switch) --- logical_router (internal-router) --- logical_switch (interconnect) --- logical_router (external-router) --- logical_switch (external-switch with localnet port to br-ext)
    > 
    > OVN configuration:
    > switch d0f22f65-214f-422e-a5ba-68b7ef66581b (interconnect)
    >     port interconnect_to_internal-router
    >         type: router
    >         addresses: ["00:00:00:73:a8:30 100.64.1.2/24"]
    >         router-port: internal-router_to_interconnect
    >     port interconnect_to_external-router
    >         type: router
    >         addresses: ["00:00:00:da:6b:85 100.64.1.1/24"]
    >         router-port: external-router_to_interconnect
    > switch bcdc365a-7c2c-4c32-9a51-8107864e879a (internal-switch)
    >     port internal-switch_to_internal-router
    >         type: router
    >         addresses: ["00:00:00:6b:83:b1 10.0.3.253/22"]
    >         router-port: internal-router_to_internal-switch
    >     port default_aaa_eth0
    >         addresses: ["0a:00:00:00:00:03 10.0.0.2"]
    > switch 3feba85f-4c6f-4550-9435-7f27837c1fd8 (external-switch)
    >     port vlan111-mgmt
    >         addresses: ["a2:dc:3c:76:8f:27"]
    >     port vlan111
    >         type: localnet
    >         tag: 111
    >         addresses: ["unknown"]
    >     port external-switch_to_external-router
    >         type: router
    >         addresses: ["00:00:00:61:f0:c0 10.250.111.40/24"]
    >         router-port: external-router_to_external-switch
    > router f97f9421-c727-488d-8575-bfaf7a7bde6b (vlan111-80973513-f2fe-48cb-904a-b205fb0bcc6f)
    >     port external-router_to_interconnect
    >         mac: "00:00:00:da:6b:85"
    >         networks: ["100.64.1.1/24"]
    >     port external-router_to_external-switch
    >         mac: "00:00:00:61:f0:c0"
    >         networks: ["10.250.111.40/24"]
    >     nat 486f81b0-491f-4c90-9ddd-04ea27e70ac5
    >         external ip: "10.250.111.40"
    >         logical ip: "10.0.0.0/8"
    >         type: "snat"
    > router 5ca8fc47-1860-43c9-a0ee-a285fd877db5 (overlay-vlan111-80973513-f2fe-48cb-904a-b205fb0bcc6f)
    >     port internal-router_to_interconnect
    >         mac: "00:00:00:73:a8:30"
    >         networks: ["100.64.1.2/24"]
    >     port internal-router_to_internal-switch
    >         mac: "00:00:00:6b:83:b1"
    >         networks: ["10.0.3.253/22"]
    > 
    > OVS configuration:
    >     Bridge br-int
    >         Port patch-br-int-br-ext
    >             Interface patch-br-int-br-ext
    >                 type: patch
    >                 options: {peer=patch-br-ext-br-int}
    >         Port "patch-br-int-to-vlan111 "
    >             Interface "patch-br-int-to-vlan111 "
    >                 type: patch
    >                 options: {peer="patch-vlan111-to-br-int"}
    >         Port "vlan111-mgmt"
    >             Interface "vlan111-mgmt"
    >                 type: internal
    >         Port br-int
    >             Interface br-int
    >                 type: internal
    >         Port "veth51a477d8"
    >             Interface "veth51a477d8"
    >     Bridge br-ext
    >         Port "patch-vlan111-to-br-int"
    >             Interface "patch-vlan111-to-br-int"
    >                 type: patch
    >                 options: {peer="patch-br-int-to-vlan111 "}
    >         Port "bond0"
    >             trunks: [111]
    >             Interface "enp4s0f0"
    >                 type: dpdk
    >                 options: {dpdk-devargs="0000:04:00.0"}
    >             Interface "enp4s0f1"
    >                 type: dpdk
    >                 options: {dpdk-devargs="0000:04:00.1"}
    >         Port patch-br-ext-br-int
    >             Interface patch-br-ext-br-int
    >                 type: patch
    >                 options: {peer=patch-br-int-br-ext}
    >         Port br-ext
    >             Interface br-ext
    >                 type: internal
    >     ovs_version: "2.11.90"
    > 
    > What else should I try? Have I missed anything?
    > Thanks in advance
    > 
    > --
    > Best regards,
    > Rostyslav Fridman
    > _______________________________________________
    > dev mailing list
    > dev at openvswitch.org
    > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.openvswitch.org%2Fmailman%2Flistinfo%2Fovs-dev&amp;data=02%7C01%7Cdball%40vmware.com%7Ce953e5faf1624665554208d68df158a6%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636852465099359454&amp;sdata=mvXAd1EkqjRmk4y20rEDGqBKodUiFxN%2B6IflSpv2gik%3D&amp;reserved=0
    _______________________________________________
    dev mailing list
    dev at openvswitch.org
    https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.openvswitch.org%2Fmailman%2Flistinfo%2Fovs-dev&amp;data=02%7C01%7Cdball%40vmware.com%7Ce953e5faf1624665554208d68df158a6%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636852465099359454&amp;sdata=mvXAd1EkqjRmk4y20rEDGqBKodUiFxN%2B6IflSpv2gik%3D&amp;reserved=0
    



More information about the dev mailing list