[ovs-dev] HA: SNAT on OVN logical_router in userspace works for ICMP but not TCP or UDP

Darrell Ball dlu998 at gmail.com
Wed Feb 13 16:54:00 UTC 2019


On Wed, Feb 13, 2019 at 7:25 AM Rostyslav Fridman <
Rostyslav_Fridman at epam.com> wrote:

> Okay, I have managed to make it work, however I do not know if this is how
> it is supposed to work.
>
>
>
> Basically, I reduced OVN scheme from
>
> container – internal switch – (container subnet gateway) internal router –
> interconnect switch – external router (NAT IP address) – external switch
>
> to
>
> container – internal switch – (container subnet gateway) external router
> (NAT IP address) – external switch
>
> The difference being that local subnet for NAT resides on the same external
> router as an external NAT IP address. In this case both ICMP and TCP/UDP
> NAT translations work. It did not work for TCP when local subnet and
> external IP resided on different OVN routers.
>

Assuming the rules requested are not L4 protocol specific, TCP having
different flow generation than ICMP does not look right to me.
afaik
 container – internal switch – (container subnet gateway) internal router –
interconnect switch – external router (NAT IP address) – external switch
is supported
However, I think it is probably better for someone actively working on this
aspect of OVN at the moment to answer this; I added a couple of people to
the thread.


>
>
> This behavior seems strange, am I missing something here?
>
>
>
> *От:* Darrell Ball [mailto:dlu998 at gmail.com]
> *Отправлено:* 13 февраля 2019 г. 17:11
> *Кому:* Rostyslav Fridman <Rostyslav_Fridman at epam.com>
> *Копия:* Darrell Ball <dball at vmware.com>; Ben Pfaff <blp at ovn.org>;
> ovs-dev at openvswitch.org; Vasyl Samoilov <Vasyl_Samoilov at epam.com>
> *Тема:* Re: [ovs-dev] HA: SNAT on OVN logical_router in userspace works
> for ICMP but not TCP or UDP
>
>
>
>
>
>
>
> On Wed, Feb 13, 2019 at 5:16 AM Rostyslav Fridman via dev <
> ovs-dev at openvswitch.org> wrote:
>
> Hi Darrel,
>
> I've reduced the amount of traffic. Here is dump-flows for ICMP traffic:
> ovs-appctl  dpif/dump-flows br-int
> ct_state(-new+est-rel-inv+trk),recirc_id(0x303b),in_port(8),packet_type(ns=0,id=0),eth(src=00:00:00:61:f0:c0,dst=00:25:90:e7:23:94),eth_type(0x0800),ipv4(frag=no),
> packets:6, bytes:588, used:0.994s,
> actions:ct_clear,push_vlan(vid=111,pcp=0),hash(l4(0)),recirc(0x1)
> ct_state(-new-est-rel-inv-trk),recirc_id(0),in_port(8),packet_type(ns=0,id=0),eth(src=0a:00:00:00:00:03,dst=00:00:00:6b:83:b1),eth_type(0x0806),arp(sip=10.0.0.2,tip=10.0.3.253,op=1/0xff,sha=0a:00:00:00:00:03,tha=00:00:00:00:00:00),
> packets:0, bytes:0, used:never, actions:userspace(pid=0,slow_path(action))
> recirc_id(0x1),dp_hash(0x9eeb76ae/0xff),in_port(8),packet_type(ns=0,id=0),eth_type(0x8100),vlan(vid=111,pcp=0),encap(eth_type(0x0800),ipv4(frag=no)),
> packets:7, bytes:714, used:0.994s, actions:4
> ct_state(+new-est-rel-inv+trk),recirc_id(0x303b),in_port(8),packet_type(ns=0,id=0),eth(src=00:00:00:61:f0:c0,dst=00:25:90:e7:23:94),eth_type(0x0800),ipv4(frag=no),
> packets:0, bytes:0, used:never,
> actions:ct_clear,push_vlan(vid=111,pcp=0),hash(l4(0)),recirc(0x1)
>
> ct_state(+new-est-rel-inv+trk),recirc_id(0x303a),in_port(8),packet_type(ns=0,id=0),eth(src=00:00:00:73:a8:30,dst=00:00:00:da:6b:85),eth_type(0x0800),ipv4(src=
> 10.0.0.2/255.0.0.0,dst=216.58.215.110/128.0.0.0,ttl=63,frag=no),
> packets:7, bytes:686, used:0.994s,
> actions:set(eth(src=00:00:00:61:f0:c0,dst=00:25:90:e7:23:94)),set(ipv4(src=
> 10.0.0.0/255.0.0.0,dst=128.0.0.0/128.0.0.0,ttl=62)
> ),ct(commit,zone=3,nat(src=10.250.111.40)),recirc(0x303b)
>
> ct_state(+new-est-rel-inv+trk),recirc_id(0x3039),in_port(8),packet_type(ns=0,id=0),eth(src=0a:00:00:00:00:03,dst=00:00:00:6b:83:b1),eth_type(0x0800),ipv4(src=
> 10.0.0.2/255.255.254.0,dst=216.58.215.110/224.0.0.0,ttl=64,frag=no),
> packets:7, bytes:686, used:0.994s,
> actions:ct_clear,ct_clear,set(eth(src=00:00:00:73:a8:30,dst=00:00:00:da:6b:85)),set(ipv4(src=
> 10.0.0.0/255.255.254.0,dst=192.0.0.0/224.0.0.0,ttl=63)
> ),ct(zone=2,nat),recirc(0x303a)
>
> ct_state(-new-est-rel-inv-trk),recirc_id(0),in_port(8),packet_type(ns=0,id=0),eth(src=0a:00:00:00:00:03/01:00:00:00:00:00,dst=00:00:00:6b:83:b1),eth_type(0x0800),ipv4(src=
> 10.0.0.2/255.255.254.0,dst=216.58.215.110/224.0.0.0,ttl=64,frag=no),
> packets:7, bytes:686, used:0.994s,
> actions:ct_clear,ct(zone=5,nat),recirc(0x3039)
>
>
> Here is dump-flows for TCP traffic:
> ovs-appctl  dpif/dump-flows br-int
> ct_state(-new-est-rel+inv+trk),recirc_id(0x3038),in_port(8),packet_type(ns=0,id=0),eth(src=00:00:00:61:f0:c0,dst=00:25:90:e7:23:94),eth_type(0x0800),ipv4(frag=no),
> packets:2, bytes:148, used:3.064s, flags:S,
> actions:ct_clear,push_vlan(vid=111,pcp=0),hash(l4(0)),recirc(0x1)
> ct_state(-new-est-rel-inv-trk),recirc_id(0),in_port(8),packet_type(ns=0,id=0),eth(src=0a:00:00:00:00:03,dst=00:00:00:6b:83:b1),eth_type(0x0806),arp(sip=10.0.0.2,tip=10.0.3.253,op=1/0xff,sha=0a:00:00:00:00:03,tha=00:00:00:00:00:00),
> packets:0, bytes:0, used:never, actions:userspace(pid=0,slow_path(action))
> recirc_id(0x1),dp_hash(0x5be46ea1/0xff),in_port(8),packet_type(ns=0,id=0),eth_type(0x8100),vlan(vid=111,pcp=0),encap(eth_type(0x0800),ipv4(frag=no)),
> packets:2, bytes:156, used:3.064s, flags:S, actions:5
>
> ct_state(-new-est-rel+inv+trk),recirc_id(0x3037),in_port(8),packet_type(ns=0,id=0),eth(src=00:00:00:73:a8:30,dst=00:00:00:da:6b:85),eth_type(0x0800),ipv4(src=
> 10.0.0.2/255.0.0.0,dst=216.58.215.110/128.0.0.0,ttl=63,frag=no),
> packets:2, bytes:148, used:3.064s, flags:S,
> actions:set(eth(src=00:00:00:61:f0:c0,dst=00:25:90:e7:23:94)),set(ipv4(src=
> 10.0.0.0/255.0.0.0,dst=128.0.0.0/128.0.0.0,ttl=62)
> ),ct(commit,zone=3,nat(src=10.250.111.40)),recirc(0x3038)
>
> ct_state(-new-est-rel+inv+trk),recirc_id(0x3036),in_port(8),packet_type(ns=0,id=0),eth(src=0a:00:00:00:00:03,dst=00:00:00:6b:83:b1),eth_type(0x0800),ipv4(src=
> 10.0.0.2/255.255.254.0,dst=216.58.215.110/224.0.0.0,ttl=64,frag=no),
> packets:2, bytes:148, used:3.064s, flags:S,
> actions:ct_clear,ct_clear,set(eth(src=00:00:00:73:a8:30,dst=00:00:00:da:6b:85)),set(ipv4(src=
> 10.0.0.0/255.255.254.0,dst=192.0.0.0/224.0.0.0,ttl=63)
> ),ct(zone=2,nat),recirc(0x3037)
>
> ct_state(-new-est-rel-inv-trk),recirc_id(0),in_port(8),packet_type(ns=0,id=0),eth(src=0a:00:00:00:00:03/01:00:00:00:00:00,dst=00:00:00:6b:83:b1),eth_type(0x0800),ipv4(src=
> 10.0.0.2/255.255.254.0,dst=216.58.215.110/224.0.0.0,ttl=64,frag=no),
> packets:2, bytes:148, used:3.064s, flags:S,
> actions:ct_clear,ct(zone=5,nat),recirc(0x3036)
>
>
>
> This flow looks particularly wrong; it would appear the flow generation
> has a problem.
>
>
>
>
> ct_state(-new-est-rel+inv+trk),recirc_id(0x3037),in_port(8),packet_type(ns=0,id=0),eth(src=00:00:00:73:a8:30,dst=00:00:00:da:6b:85),eth_type(0x0800),ipv4(src=
> 10.0.0.2/255.0.0.0,dst=216.58.215.110/128.0.0.0,ttl=63,frag=no),
> packets:2, bytes:148, used:3.064s, flags:S,
> actions:set(eth(src=00:00:00:61:f0:c0,dst=00:25:90:e7:23:94)),set(ipv4(src=
> 10.0.0.0/255.0.0.0,dst=128.0.0.0/128.0.0.0,ttl=62)),
> ct(commit,zone=3,nat(src=10.250.111.40)),recirc(0x3038)
>
>
>
> btw; I don't see something corresponding to this in previous OF flow dump
> link you sent.
>
>
>
> Probably, you need to dump both OVN Southbound flows and Openflow flows to
> see where these flows originate.
>
>
>
>
>
>
> As can be seen from output no conntrack entry is created for new state.
>
>
> -----Исходное сообщение-----
> От: Darrell Ball [mailto:dball at vmware.com]
> Отправлено: 9 февраля 2019 г. 0:29
> Кому: Rostyslav Fridman <Rostyslav_Fridman at epam.com>; Ben Pfaff <
> blp at ovn.org>
> Копия: ovs-dev at openvswitch.org; Vasyl Samoilov <Vasyl_Samoilov at epam.com>
> Тема: Re: [ovs-dev] SNAT on OVN logical_router in userspace works for ICMP
> but not TCP or UDP
>
> If TCP packets do not go thru conntrack, then that would explain why the
> TCP traffic is not natted (since you don't have any other rules that could
> do that)
>
> You need to find out where the TCP packets are going.
> Try making the rules L4 protocol specific (i.e. look for TCP and also do
> something different for ICMP) Maybe add some other debug rules to trace the
> TCP packets otherwise
>
>
>
> On 2/8/19, 1:47 PM, "Rostyslav Fridman" <Rostyslav_Fridman at epam.com>
> wrote:
>
>     I have sent TCP traffic. It does not show up in dump-conntrack for
> some reason. However, I see it on the external server.
>
>     -----Исходное сообщение-----
>     От: Darrell Ball [mailto:dball at vmware.com]
>     Отправлено: 8 февраля 2019 г. 23:29
>     Кому: Rostyslav Fridman <Rostyslav_Fridman at epam.com>; Ben Pfaff <
> blp at ovn.org>
>     Копия: ovs-dev at openvswitch.org; Vasyl Samoilov <
> Vasyl_Samoilov at epam.com>
>     Тема: Re: [ovs-dev] SNAT on OVN logical_router in userspace works for
> ICMP but not TCP or UDP
>
>     I thought the problem was with TCP/UDP traffic ?
>     Did you send TCP traffic for this test ?; if not, can you run the test
> with TCP ?
>
>
>
>     On 2/8/19, 12:53 PM, "Rostyslav Fridman" <Rostyslav_Fridman at epam.com>
> wrote:
>
>         # ovs-appctl dpif/dump-flows br-int
>
> recirc_id(0x1),dp_hash(0x9eeb76ae/0xff),in_port(8),packet_type(ns=0,id=0),eth_type(0x8100),vlan(vid=111,pcp=0),encap(eth_type(0x0800),ipv4(frag=no)),
> packets:20, bytes:2040, used:0.942s, actions:4
>
> ct_state(-new-est-rel-inv-trk),recirc_id(0),in_port(8),packet_type(ns=0,id=0),eth(src=0a:00:00:00:00:03/01:00:00:00:00:00,dst=00:00:00:6b:83:b1),eth_type(0x0800),ipv4(src=
> 10.0.0.2/255.255.254.0,dst=216.58.215.110/224.0.0.0,ttl=64,frag=no),
> packets:25, bytes:2354, used:0.942s, flags:S,
> actions:ct_clear,ct(zone=5,nat),recirc(0xb1)
>
> ct_state(+new-est-rel-inv+trk),recirc_id(0xb2),in_port(8),packet_type(ns=0,id=0),eth(src=00:00:00:73:a8:30,dst=00:00:00:da:6b:85),eth_type(0x0800),ipv4(src=
> 10.0.0.2/255.0.0.0,dst=216.58.215.110/128.0.0.0,ttl=63,frag=no),
> packets:20, bytes:1960, used:0.942s,
> actions:set(eth(src=00:00:00:61:f0:c0,dst=00:25:90:e7:23:94)),set(ipv4(src=
> 10.0.0.0/255.0.0.0,dst=128.0.0.0/128.0.0.0,ttl=62)
> ),ct(commit,zone=3,nat(src=10.250.111.40)),recirc(0xb3)
>
> ct_state(+new-est-rel-inv+trk),recirc_id(0xb1),in_port(8),packet_type(ns=0,id=0),eth(src=0a:00:00:00:00:03,dst=00:00:00:6b:83:b1),eth_type(0x0800),ipv4(src=
> 10.0.0.2/255.255.254.0,dst=216.58.215.110/224.0.0.0,ttl=64,frag=no),
> packets:20, bytes:1960, used:0.942s,
> actions:ct_clear,ct_clear,set(eth(src=00:00:00:73:a8:30,dst=00:00:00:da:6b:85)),set(ipv4(src=
> 10.0.0.0/255.255.254.0,dst=192.0.0.0/224.0.0.0,ttl=63)
> ),ct(zone=2,nat),recirc(0xb2)
>
> ct_state(-new+est-rel-inv+trk),recirc_id(0xb3),in_port(8),packet_type(ns=0,id=0),eth(src=00:00:00:61:f0:c0,dst=00:25:90:e7:23:94),eth_type(0x0800),ipv4(frag=no),
> packets:19, bytes:1862, used:0.942s,
> actions:ct_clear,push_vlan(vid=111,pcp=0),hash(l4(0)),recirc(0x1)
>
>         ==================================
>
>         # ovs-appctl dpctl/dump-conntrack
>
> icmp,orig=(src=10.0.0.2,dst=216.58.215.110,id=246,type=8,code=0),reply=(src=216.58.215.110,dst=10.250.111.40,id=246,type=0,code=0),zone=3
>
>
>
> _______________________________________________
> dev mailing list
> dev at openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
>


More information about the dev mailing list