[ovs-discuss] [ovn] OVN asymmetric routing with conntrack

Thu Aug 26 20:50:30 UTC 2021

Hi Han,

thanks for your comment.
You’re right. disabling ct.inv didn’t help in my case.
I’ve mixed allow-stateful with allow-related rules in port_group ACLs. Currently testing...

At first glance it works as expected.

Regards,
Vladislav Odintsov

> On 25 Aug 2021, at 01:18, Han Zhou <hzhou at ovn.org> wrote:
> 
> 
> 
> On Tue, Aug 24, 2021 at 3:02 PM Vladislav Odintsov <odivlad at gmail.com <mailto:odivlad at gmail.com>> wrote:
> 
> 
> Regards,
> Vladislav Odintsov
> 
>> On 25 Aug 2021, at 00:47, Numan Siddique <numans at ovn.org <mailto:numans at ovn.org>> wrote:
>> 
>> On Mon, Aug 23, 2021 at 11:22 AM Vladislav Odintsov <odivlad at gmail.com <mailto:odivlad at gmail.com>> wrote:
>>> 
>>> Hi,
>>> 
>>> we’ve faced an issue where asymmetric-routed traffic is used. Please help understand what options do we have to allow such traffic.
>>> 
>>> Topology is next:
>>> 
>>>                     client lsp (10.0.0.1/24 <http://10.0.0.1/24>)
>>>                                 |
>>>                            ls-external
>>>                            /         \
>>> lsp router vm1 eth0: 10.0.0.2/24 <http://10.0.0.2/24>     lsp router vm2 eth0: 10.0.0.3/24 <http://10.0.0.3/24>
>>> lsp router vm1 eth1: 192.168.0.1/24 <http://192.168.0.1/24>  lsp router vm2 eth1: 192.168.0.2/24 <http://192.168.0.2/24>
>>>                            \         /
>>>                            ls-internal
>>>                                 |
>>>                    server lsp (192.168.0.10/24 <http://192.168.0.10/24>)
>>> 
>>> 
>>> All LSPs have port_security configured with "<mac> 0.0.0.0/0 <http://0.0.0.0/0> ::/0" and belong to port group pg1.
>>> 
>>> There are two ACLs within this PG:
>>> from-lport 0.0.0.0/0 <http://0.0.0.0/0> allow-related
>>> to-lport 0.0.0.0/0 <http://0.0.0.0/0> allow-related
>>> 
>>> The problem is when traffic from client to server goes through router vm1 and returns through router vm2, there is no connectivity. I see reply traffic on the server interface, which is going to router vm2 mac address, but I don't see it on the router vm2 interface.
>>> I guess the reason for this is that conntrack first time sees packet for the connection and ACK+SYN flags are set and treats this packet as invalid, right?
>> 
>> I think so.
>> 
>>> 
>>> If yes, is there any option how to use asymmetric-routed topologies inside OVN with stateful ACLs?
>>> I found there is an ability to replace ct.inv field check: https://github.com/ovn-org/ovn/commit/3bb91366a6b0d60df5ce8f9c7f6427f7d37dfdd4 <https://github.com/ovn-org/ovn/commit/3bb91366a6b0d60df5ce8f9c7f6427f7d37dfdd4>
>>> Is it good idea to use this option to solve the issue or this is intended specifically to use with smart NICs without invalid state support and can be removed in future?
>>> 
> 
> 
> I am afraid disabling ct.inv will not help here. In your use case the connections won't become established in conntrack, which means stateful ACL wouldn't work. This will be the case even with physical stateful FWs. I'd suggest either disable ACLs or use stateless ACLs (i.e. allow/allow-stateless instead of allow-related).
>> 
>> I do not understand your use case completely.  I'm not quite clear
>> from the diagram which all resources are external
>> and which all are part of OVN.  Have you tried using the ECMP routes feature ?
>> 
> Logical Routers are not used in this topology. Only two logical switches - ls-external and ls-internal.
> 
> In OVN we have:
> 1. LS "ls-external" with 3 LSPs: "lsp-router-vm1-eth0" (10.0.0.2/24 <http://10.0.0.2/24>), "lsp-router-vm2-eth0" (10.0.0.3/24 <http://10.0.0.3/24>) and "lsp-client" (10.0.0.1/24 <http://10.0.0.1/24>)
> 2. LS "ls-internal" with 2 LSPs: "lsp-router-vm1-eth1" (192.168.0.1/24 <http://192.168.0.1/24>), "lsp-router-vm2-eth1" (192.168.0.2/24 <http://192.168.0.2/24>) and "lsp-server" (192.168.0.10/24 <http://192.168.0.10/24>)
> 3. Port group pg1 with mentioned above LSPs
> 4. Ingress and egress ACLs 0.0.0.0/0 <http://0.0.0.0/0> with allow-related action.
> 
> External:
> Linux client VM (10.0.0.1), which has static ECMP route:
> 192.168.0.0/24 <http://192.168.0.0/24> via nexthop 10.0.0.2 via nexthop 10.0.0.3
> 
> sends tcp syn from ip 10.0.0.1 to 192.168.0.10 (server).
> Traffic to server can go either through router VM1 (10.0.0.2) or through router VM2 (10.0.0.3).
> 
> Server also has static ECMP route to reverse direction:
> 10.0.0.0/24 <http://10.0.0.0/24> via nexthop 192.168.0.1 via nexthop 192.168.0.2
> 
> So, when traffic in both directions goes through same router VM, it passes normally.
> When traffic goes from client to server through one router VM and returns back through another - SYN-ACK is blocked somewhere in OVS/conntrack. On router VM’s interface we don’t see SYN+ACK.
> 
> So, OVN-based ECMP feature is not relevant for this case since it doesn’t invoke logical routers.
> 
>> Regarding the ct.inv flag, does it work when you disable the usage of ct.inv ?
>> 
> Not yet. First wanted to understand options.
> 
>> Thanks
>> Numan
>> 
>> 
>> If these routes are configured in the logical router, then
>>> Thanks.
>>> 
>>> Regards,
>>> Vladislav Odintsov
>>> 
>>> _______________________________________________
>>> discuss mailing list
>>> discuss at openvswitch.org <mailto:discuss at openvswitch.org>
>>> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss <https://mail.openvswitch.org/mailman/listinfo/ovs-discuss>
> 
> _______________________________________________
> discuss mailing list
> discuss at openvswitch.org <mailto:discuss at openvswitch.org>
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss <https://mail.openvswitch.org/mailman/listinfo/ovs-discuss>
> _______________________________________________
> discuss mailing list
> discuss at openvswitch.org <mailto:discuss at openvswitch.org>
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss <https://mail.openvswitch.org/mailman/listinfo/ovs-discuss>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20210826/9033e3f9/attachment-0001.html>