[ovs-dev] HA: SNAT on OVN logical_router in userspace works for ICMP but not TCP or UDP

Rostyslav Fridman Rostyslav_Fridman at epam.com
Fri Feb 8 20:52:58 UTC 2019


# ovs-appctl dpif/dump-flows br-int
recirc_id(0x1),dp_hash(0x9eeb76ae/0xff),in_port(8),packet_type(ns=0,id=0),eth_type(0x8100),vlan(vid=111,pcp=0),encap(eth_type(0x0800),ipv4(frag=no)), packets:20, bytes:2040, used:0.942s, actions:4
ct_state(-new-est-rel-inv-trk),recirc_id(0),in_port(8),packet_type(ns=0,id=0),eth(src=0a:00:00:00:00:03/01:00:00:00:00:00,dst=00:00:00:6b:83:b1),eth_type(0x0800),ipv4(src=10.0.0.2/255.255.254.0,dst=216.58.215.110/224.0.0.0,ttl=64,frag=no), packets:25, bytes:2354, used:0.942s, flags:S, actions:ct_clear,ct(zone=5,nat),recirc(0xb1)
ct_state(+new-est-rel-inv+trk),recirc_id(0xb2),in_port(8),packet_type(ns=0,id=0),eth(src=00:00:00:73:a8:30,dst=00:00:00:da:6b:85),eth_type(0x0800),ipv4(src=10.0.0.2/255.0.0.0,dst=216.58.215.110/128.0.0.0,ttl=63,frag=no), packets:20, bytes:1960, used:0.942s, actions:set(eth(src=00:00:00:61:f0:c0,dst=00:25:90:e7:23:94)),set(ipv4(src=10.0.0.0/255.0.0.0,dst=128.0.0.0/128.0.0.0,ttl=62)),ct(commit,zone=3,nat(src=10.250.111.40)),recirc(0xb3)
ct_state(+new-est-rel-inv+trk),recirc_id(0xb1),in_port(8),packet_type(ns=0,id=0),eth(src=0a:00:00:00:00:03,dst=00:00:00:6b:83:b1),eth_type(0x0800),ipv4(src=10.0.0.2/255.255.254.0,dst=216.58.215.110/224.0.0.0,ttl=64,frag=no), packets:20, bytes:1960, used:0.942s, actions:ct_clear,ct_clear,set(eth(src=00:00:00:73:a8:30,dst=00:00:00:da:6b:85)),set(ipv4(src=10.0.0.0/255.255.254.0,dst=192.0.0.0/224.0.0.0,ttl=63)),ct(zone=2,nat),recirc(0xb2)
ct_state(-new+est-rel-inv+trk),recirc_id(0xb3),in_port(8),packet_type(ns=0,id=0),eth(src=00:00:00:61:f0:c0,dst=00:25:90:e7:23:94),eth_type(0x0800),ipv4(frag=no), packets:19, bytes:1862, used:0.942s, actions:ct_clear,push_vlan(vid=111,pcp=0),hash(l4(0)),recirc(0x1)

==================================

# ovs-appctl dpctl/dump-conntrack 
icmp,orig=(src=10.0.0.2,dst=216.58.215.110,id=246,type=8,code=0),reply=(src=216.58.215.110,dst=10.250.111.40,id=246,type=0,code=0),zone=3


-----Исходное сообщение-----
От: Darrell Ball [mailto:dball at vmware.com] 
Отправлено: 8 февраля 2019 г. 22:40
Кому: Rostyslav Fridman <Rostyslav_Fridman at epam.com>; Ben Pfaff <blp at ovn.org>
Копия: ovs-dev at openvswitch.org; Vasyl Samoilov <Vasyl_Samoilov at epam.com>
Тема: Re: [ovs-dev] SNAT on OVN logical_router in userspace works for ICMP but not TCP or UDP

Could you dump the datapath flows and conntrack entries while your test is running (i.e. sending packets) ?


==================================

# ovs-appctl dpif/dump-flows br-int
<no output>


==================================

# ovs-appctl dpctl/dump-conntrack 
<no output>

Also besides arp, could you limit traffic thru. the SUT to the test traffic, like the curl triggered packets ?



On 2/8/19, 12:04 PM, "Rostyslav Fridman" <Rostyslav_Fridman at epam.com> wrote:

    > How about dumping flows and conntrack entries and checking stats at various points ?
    > ovs-ofctl dump-flows <bridge>
    > ovs-appctl dpif/dump-flows <bridge>
    > ovs-appctl dpctl/dump-conntrack
    
    Please find flow dumps at the following link: https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpastebin.com%2Fraw%2FepKAhTKm&amp;data=02%7C01%7Cdball%40vmware.com%7Cd20b9c1d51e44df4519808d68e00971f%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636852530531092223&amp;sdata=RAl%2Fe3ktUSwxqCIY5BwGL%2Fdylgp59f%2B3vheX3wRr%2FUM%3D&amp;reserved=0
    
    > How are you sending said UDP/TCP packets ?
    
    Curl and telnet requests from the container.
    
    --
    Best regards,
    Rostyslav Fridman
    
    -----Исходное сообщение-----
    От: Darrell Ball [mailto:dball at vmware.com] 
    Отправлено: 8 февраля 2019 г. 20:59
    Кому: Ben Pfaff <blp at ovn.org>; Rostyslav Fridman <Rostyslav_Fridman at epam.com>
    Копия: ovs-dev at openvswitch.org; Vasyl Samoilov <Vasyl_Samoilov at epam.com>
    Тема: Re: [ovs-dev] SNAT on OVN logical_router in userspace works for ICMP but not TCP or UDP
    
    
    We have advanced system tests for userspace datapath to test OVN, including tcp packets.
    
    system-ovn
    
    124: ovn -- 2 LRs connected via LS, gateway router, SNAT and DNAT ok
    125: ovn -- 2 LRs connected via LS, gateway router, easy SNAT ok
    126: ovn -- multiple gateway routers, SNAT and DNAT  ok
    127: ovn -- load-balancing                           ok
    128: ovn -- load-balancing - same subnet.            ok
    129: ovn -- load balancing in gateway router         ok
    130: ovn -- multiple gateway routers, load-balancing ok
    131: ovn -- load balancing in router with gateway router port ok
    132: ovn -- DNAT and SNAT on distributed router - N/S ok
    133: ovn -- DNAT and SNAT on distributed router - E/W ok
    
    Let us define the problem first since the context is mostly undefined
    
    How about dumping flows and conntrack entries and checking stats at various points ?
    
    ovs-ofctl dump-flows <bridge>
    ovs-appctl dpif/dump-flows <bridge>
    ovs-appctl dpctl/dump-conntrack
    
    How are you sending said UDP/TCP packets ?
    
    
    
    On 2/8/19, 10:15 AM, "ovs-dev-bounces at openvswitch.org on behalf of Ben Pfaff" <ovs-dev-bounces at openvswitch.org on behalf of blp at ovn.org> wrote:
    
        Darrell, is this something you can help with?
        
        On Fri, Feb 08, 2019 at 02:18:53PM +0000, Rostyslav Fridman via dev wrote:
        > I've encountered the issue that SNAT on OVN logical_router in userspace works for ICMP but not TCP or UDP. I am seeing this behavior on version 2.10.1 as well as on top of the git tree.
        > 
        > I try to access internet (216.58.215.110) from container (10.0.0.2). On the external-router I have SNAT configured. On the external server I see that container address is translated for ICMP request, but not for TCP.
        > container:/# ping 216.58.215.110
        > PING 216.58.215.110 (216.58.215.110) 56(84) bytes of data.
        > 64 bytes from 216.58.215.110: icmp_seq=1 ttl=53 time=140 ms
        > ^C
        > --- 216.58.215.110 ping statistics ---
        > 1 packets transmitted, 1 received, 0% packet loss, time 0ms
        > rtt min/avg/max/mdev = 140.663/140.663/140.663/0.000 ms
        > container:/# curl 216.58.215.110
        > ^C
        > ---
        > external-server:~# tcpdump -i vlan111 host 216.58.215.110
        > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
        > listening on vlan111, link-type EN10MB (Ethernet), capture size 262144 bytes
        > 07:53:22.393289 IP 10.250.111.40 > waw02s17-in-f14.1e100.net: ICMP echo request, id 218, seq 1, length 64
        > 07:53:22.533574 IP waw02s17-in-f14.1e100.net > 10.250.111.40: ICMP echo reply, id 218, seq 1, length 64
        > 07:53:24.830595 IP 10.0.0.2.58050 > waw02s17-in-f14.1e100.net.http: Flags [S], seq 219699121, win 29200, options [mss 1460,sackOK,TS val 2742820693 ecr 0,nop,wscale 7], length 0
        > 
        > In the bridge flows I see that NAT should be performed since flow packet count is increasing.
        > ovs-appctl bridge/dump-flows br-int
        > ...
        > table_id=41, duration=5135s, n_packets=28, n_bytes=2408, priority=9,ip,metadata=0x1,nw_src=10.0.0.0/8,actions=ct(commit,table=42,zone=NXM_NX_REG12[0..15],nat(src=10.250.111.40))
        > 
        > ovn-trace also confirms that it should be working.
        > 
        > I have the following scheme:
        > OVS: trunked bonded port --- netdev bridge (br-ext) --- patch --- netdev bridge (br-int)
        > OVN: container --- logical_switch (internal-switch) --- logical_router (internal-router) --- logical_switch (interconnect) --- logical_router (external-router) --- logical_switch (external-switch with localnet port to br-ext)
        > 
        > OVN configuration:
        > switch d0f22f65-214f-422e-a5ba-68b7ef66581b (interconnect)
        >     port interconnect_to_internal-router
        >         type: router
        >         addresses: ["00:00:00:73:a8:30 100.64.1.2/24"]
        >         router-port: internal-router_to_interconnect
        >     port interconnect_to_external-router
        >         type: router
        >         addresses: ["00:00:00:da:6b:85 100.64.1.1/24"]
        >         router-port: external-router_to_interconnect
        > switch bcdc365a-7c2c-4c32-9a51-8107864e879a (internal-switch)
        >     port internal-switch_to_internal-router
        >         type: router
        >         addresses: ["00:00:00:6b:83:b1 10.0.3.253/22"]
        >         router-port: internal-router_to_internal-switch
        >     port default_aaa_eth0
        >         addresses: ["0a:00:00:00:00:03 10.0.0.2"]
        > switch 3feba85f-4c6f-4550-9435-7f27837c1fd8 (external-switch)
        >     port vlan111-mgmt
        >         addresses: ["a2:dc:3c:76:8f:27"]
        >     port vlan111
        >         type: localnet
        >         tag: 111
        >         addresses: ["unknown"]
        >     port external-switch_to_external-router
        >         type: router
        >         addresses: ["00:00:00:61:f0:c0 10.250.111.40/24"]
        >         router-port: external-router_to_external-switch
        > router f97f9421-c727-488d-8575-bfaf7a7bde6b (vlan111-80973513-f2fe-48cb-904a-b205fb0bcc6f)
        >     port external-router_to_interconnect
        >         mac: "00:00:00:da:6b:85"
        >         networks: ["100.64.1.1/24"]
        >     port external-router_to_external-switch
        >         mac: "00:00:00:61:f0:c0"
        >         networks: ["10.250.111.40/24"]
        >     nat 486f81b0-491f-4c90-9ddd-04ea27e70ac5
        >         external ip: "10.250.111.40"
        >         logical ip: "10.0.0.0/8"
        >         type: "snat"
        > router 5ca8fc47-1860-43c9-a0ee-a285fd877db5 (overlay-vlan111-80973513-f2fe-48cb-904a-b205fb0bcc6f)
        >     port internal-router_to_interconnect
        >         mac: "00:00:00:73:a8:30"
        >         networks: ["100.64.1.2/24"]
        >     port internal-router_to_internal-switch
        >         mac: "00:00:00:6b:83:b1"
        >         networks: ["10.0.3.253/22"]
        > 
        > OVS configuration:
        >     Bridge br-int
        >         Port patch-br-int-br-ext
        >             Interface patch-br-int-br-ext
        >                 type: patch
        >                 options: {peer=patch-br-ext-br-int}
        >         Port "patch-br-int-to-vlan111 "
        >             Interface "patch-br-int-to-vlan111 "
        >                 type: patch
        >                 options: {peer="patch-vlan111-to-br-int"}
        >         Port "vlan111-mgmt"
        >             Interface "vlan111-mgmt"
        >                 type: internal
        >         Port br-int
        >             Interface br-int
        >                 type: internal
        >         Port "veth51a477d8"
        >             Interface "veth51a477d8"
        >     Bridge br-ext
        >         Port "patch-vlan111-to-br-int"
        >             Interface "patch-vlan111-to-br-int"
        >                 type: patch
        >                 options: {peer="patch-br-int-to-vlan111 "}
        >         Port "bond0"
        >             trunks: [111]
        >             Interface "enp4s0f0"
        >                 type: dpdk
        >                 options: {dpdk-devargs="0000:04:00.0"}
        >             Interface "enp4s0f1"
        >                 type: dpdk
        >                 options: {dpdk-devargs="0000:04:00.1"}
        >         Port patch-br-ext-br-int
        >             Interface patch-br-ext-br-int
        >                 type: patch
        >                 options: {peer=patch-br-int-br-ext}
        >         Port br-ext
        >             Interface br-ext
        >                 type: internal
        >     ovs_version: "2.11.90"
        > 
        > What else should I try? Have I missed anything?
        > Thanks in advance
        > 
        > --
        > Best regards,
        > Rostyslav Fridman
        > _______________________________________________
        > dev mailing list
        > dev at openvswitch.org
        > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.openvswitch.org%2Fmailman%2Flistinfo%2Fovs-dev&amp;data=02%7C01%7Cdball%40vmware.com%7Cd20b9c1d51e44df4519808d68e00971f%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636852530531102232&amp;sdata=axSfaReIM7aD1cZ1BpOTwI1PjRe7LGDH6JZ70iYbxBw%3D&amp;reserved=0
        _______________________________________________
        dev mailing list
        dev at openvswitch.org
        https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.openvswitch.org%2Fmailman%2Flistinfo%2Fovs-dev&amp;data=02%7C01%7Cdball%40vmware.com%7Cd20b9c1d51e44df4519808d68e00971f%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636852530531102232&amp;sdata=axSfaReIM7aD1cZ1BpOTwI1PjRe7LGDH6JZ70iYbxBw%3D&amp;reserved=0
        
    
    



More information about the dev mailing list