[ovs-discuss] Issue with connection tracking for packets modified in pipeline

Fri Jun 16 05:20:48 UTC 2017

On Thu, Jun 15, 2017 at 5:06 PM, Aswin S <aswinsuryan at gmail.com> wrote:

>
> Adding some more info here, Thanks Numan! for pointing to this.
>
> The issue I am facing looks similar to the one described in [1] and [2].
> But it seems the issue is not yet fixed.  Is there a plan to fix this soon?
> In Opendaylight security groups is implemented using ovs-conntrack. So the
> flow based router  ping  responder and floating IP translations hits this
> issue.
>
> [1]https://mail.openvswitch.org/pipermail/ovs-dev/2017-March/329542.html
> [2]https://patchwork.ozlabs.org/patch/739796/
>
>
The same issuse is also seen in OVN as pointed by Aswin.

Joe - If you remember, we had a chat about this same issue during the
Openstack Boston summit.

Thanks
Numan

> Thanks
> Aswin
>
> On Fri, Jun 2, 2017 at 5:41 PM, Aswin S <aswinsuryan at gmail.com> wrote:
>
>> Joe,
>>
>> Thanks for the quick response, I apologize for being cryptic. Yes FIP is
>> floating IP and SG is access control list. The issue can be reproduced with
>> two namespace using the steps in [1] in ovs 2.7.
>>
>> With [1]
>>
>> From 10.100.5.8 if I try to ping/ssh 10.100.5.9 it works, but not when I
>> try ping/ssh to 192.168.56.32 from 10.100.5.8.
>>
>> But it seems to work if I track them in two different ct zones as
>> below(in 40,41,251,252)
>>
>> "table=40,priority=61010,ip,dl_src=fa:16:3e:1d:3d:01,nw_src=
>> 10.100.5.8,actions=ct(table=41,zone=5001)"
>> "table=40,priority=61010,ip,dl_src=fa:16:3e:13:85:be,nw_src=
>> 10.100.5.9,actions=ct(table=41,zone=5002)"
>>
>> "table=41,priority=1000,ct_state=+new+trk,ip,dl_src=fa:16:
>> 3e:1d:3d:01,nw_src=10.100.5.8,actions=ct(commit,zone=5001),resubmit(,21)"
>> "table=41,priority=1000,ct_state=+new+trk,ip,dl_src=fa:16:
>> 3e:13:85:be,nw_src=10.100.5.9,actions=ct(commit,zone=5002),resubmit(,21)"
>>
>> [1]https://gist.github.com/aswinsuryan/c22919576ae19e14ed489bf1f6c668cb
>>
>> Aswin
>>
>> On Fri, Jun 2, 2017 at 5:22 AM, Joe Stringer <joe at ovn.org> wrote:
>>
>>> On 1 June 2017 at 05:23, Aswin S <aswinsuryan at gmail.com> wrote:
>>> > Hi,
>>> > When SG is implemented using conntrack rules , TCP connection via FIP
>>> > between vms in the same compute is failing
>>>
>>> What is SG?
>>>
>>> Is FIP "floating IP"?
>>>
>>> > In my topology I have two vm on the same compute both having floating
>>> ip
>>> > associated with it and the fip translation is done using openflow
>>> rules.
>>> >
>>> > When using vm internal network ip it is working fine and I can ssh to
>>> the
>>> > other vm.
>>> >
>>> > The conntrack event logs are as follows (Src ip 10.100.5.5 Dest Ip
>>> > 10.100.5.12
>>> >    [NEW] tcp      6 120 SYN_SENT src=10.100.5.5 dst=10.100.5.12
>>> sport=43724
>>> > dport=22 [UNREPLIED] src=10.100.5.12 dst=10.100.5.5 sport=22
>>> dport=43724
>>> > zone=5001
>>> >  [UPDATE] tcp      6 60 SYN_RECV src=10.100.5.5 dst=10.100.5.12
>>> sport=43724
>>> > dport=22 src=10.100.5.12 dst=10.100.5.5 sport=22 dport=43724 zone=5001
>>> >  [UPDATE] tcp      6 432000 ESTABLISHED src=10.100.5.5 dst=10.100.5.12
>>> > sport=43724 dport=22 src=10.100.5.12 dst=10.100.5.5 sport=22
>>> dport=43724
>>> > [ASSURED] zone=5001
>>> >
>>> >
>>> > But when I use FIP(the TCP packets are marked as Invalid and dropped.
>>> > The SYN reaches the second vm which sends back the SYN ack and the
>>> status of
>>> > conntrack entry is updated at the destination. Though the SYN-ACK
>>> reaches
>>> > vm1 the conntrack state still remain UNREPLIED and the ack packet send
>>> to
>>> > vm2 is marked as invalid and dropped. In the pipeline the packet is
>>> > submitted to the conntrack both at egress and ingress side. The packet
>>> is
>>> > submitted to conntrack after the fip modification.
>>> >
>>> > The conntrack event logs (Vm1 10.100.5.5, 192.168.56.29, Vm2
>>> 10.100.5.12,
>>> > 192.168.56.23)
>>> >
>>> >      [NEW] tcp      6 120 SYN_SENT src=10.100.5.12 dst=192.168.56.29
>>> > sport=58218 dport=22 [UNREPLIED] src=192.168.56.29 dst=10.100.5.12
>>> sport=22
>>> > dport=58218 zone=5001
>>> >     [NEW] tcp      6 120 SYN_SENT src=192.168.56.23 dst=10.100.5.5
>>> > sport=58218 dport=22 [UNREPLIED] src=10.100.5.5 dst=192.168.56.23
>>> sport=22
>>> > dport=58218 zone=5001
>>> >  [UPDATE] tcp      6 60 SYN_RECV src=192.168.56.23 dst=10.100.5.5
>>> > sport=58218 dport=22 src=10.100.5.5 dst=192.168.56.23 sport=22
>>> dport=58218
>>> > zone=5001
>>>
>>> It looks like you're modifying the destination address on traffic from
>>> VM1->VM2 before submitting to conntrack and modifying the source
>>> address on traffic from VM2->VM1 before submitting to conntrack, which
>>> means that conntrack is not seeing bidirection traffic between two
>>> physical IPs, nor is it seeing bidirectional traffic between floating
>>> IPs.. rather, it is seeing two unidirectional connections between
>>> either VM1's physical IP and VM2's FIP, or VM1's FIP and VM2's
>>> physical IP. The Linux connection tracker, when it sees unidirectional
>>> SYNACK, will classify it as invalid, leading to your drop.
>>>
>>> The flows I have mentioned in [1] works if it is two zones, so I think
>> it should not be an issue of netfilter not seeing bidirectional packets.
>>
>>
>>> > The issue is not limited to TCP, if try with ICMP with FIP, the ping
>>> packet
>>> > from vm1 to vm2 will be always new state in the connection tracker.
>>> This
>>> > works(both TCP and ICMP) fine if the vm are two compute nodes. So is
>>> this an
>>> > issue when a modified packet in the pipeline is  submitted to
>>> connection
>>> > tracker? Does netfilter/ovs conntrack check for any other field than
>>> src ip/
>>> > port dest ip/port for marking a packet as reply pakcet?
>>>
>>> It deals with the actual packet contents when you execute the ct()
>>> action.
>>>
>>> > I am using ovs 2.7.0. I have reported an issue a while ago[1] which
>>> still
>>> > exsits and this seems to be related
>>> >
>>> > [1]https://mail.openvswitch.org/pipermail/ovs-discuss/2016-D
>>> ecember/043228.html
>>>
>>> It looks like you're seeing events corresponding to modified packets
>>> according to your output above, so I don't see the relation to this
>>> other thread.
>>>
>> Yes that makes sense, here it showing modified packets, but I just
>> thought the root cause may not be related to modified packets.
>>
>>>
>>> Cheers,
>>> Joe
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20170616/d3140ffc/attachment-0001.html>