[ovs-discuss] packet loop when using in_port=LOCAL

Fri Feb 19 14:11:53 UTC 2021

Hi colleagues,

I'm facing an issue with packet looping(?) when trying to implement the
following topology using OVS:

Here vnet4 and vnet5 are virtual machines, br-ex has an IP address on host:

root at alpha:~# ovs-vsctl show
     Bridge br-ex
         Port vnet5
             Interface vnet5
         Port br-ex
             Interface br-ex
                 type: internal
         Port vnet4
             Interface vnet4
     ovs_version: "2.13.1"

root at alpha:~# ip a
2: enp9s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
     link/ether a8:a1:59:00:32:ed brd ff:ff:ff:ff:ff:ff
     inet S.N.A.T/24 scope global enp9s0
        valid_lft forever preferred_lft forever
5: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
     link/ether 86:d1:8b:b3:d7:4d brd ff:ff:ff:ff:ff:ff
     inet 100.100.0.1/23 brd 100.100.1.255 scope global br-ex
        valid_lft forever preferred_lft forever

and I'm redirecting all specified traffic to/from vnet4 through vnet5
(note: when defining rule (1), I use 'in_port=br-ex' which translated by
OVS to 'in_port=LOCAL') :

root at alpha:~# ovs-ofctl dump-flows br-ex
1) table=0, n_packets=1080, n_bytes=79344, tcp,in_port=LOCAL,nw_src=3.123.231.xxx,tp_src=443 actions=mod_dl_dst:52:54:00:5b:d4:bf,output:vnet5
2) table=0, n_packets=38, n_bytes=2932, tcp,in_port=vnet4,nw_dst=3.123.231.xxx,tp_dst=443 actions=mod_dl_dst:52:54:00:5b:d4:bf,output:vnet5
3) table=0, n_packets=820695, n_bytes=408963731, priority=0 actions=NORMAL

Now, when I send request to 3.12.231.xxx from vnet4, I expect that
packets in both directions of conversation can be processed by VM
'vnet5' while both sides will continue as if there is nothing in the middle.

But in fact it seems that OVS loops packets by rule (1) and they never
reach target:

- while on VM 'vnet4' I see only 'forward' packets to 3.123.231.xxx and
no responses:

15:56:04.472226 ip: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 64)
     172.29.24.53.62507 > 3.123.231.xxx.443: Flags [S], cksum 0x2203 (correct), seq 3019516173, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val 1011439203 ecr 0,sackOK,eol], length 0
15:56:05.478382 ip: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 64)
     172.29.24.53.62507 > 3.123.231.xxx.443: Flags [S], cksum 0x1e1b (correct), seq 3019516173, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val 1011440203 ecr 0,sackOK,eol], length 0
15:56:06.480944 ip: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 64)
     172.29.24.53.62507 > 3.123.231.xxx.443: Flags [S], cksum 0x1a33 (correct), seq 3019516173, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val 1011441203 ecr 0,sackOK,eol], length 0

- and on host's external interface I see SYN/SYN-ACK from/to vnet4:

13:58:15.308422 a8:a1:59:00:32:ed > 00:31:46:0d:43:5a, ethertype IPv4 (0x0800), length 78: (tos 0x0, ttl 61, id 0, offset 0, flags [DF], proto TCP (6), length 64)
     S.N.A.T.62509 > 3.123.231.xxx.443: Flags [S], cksum 0x7ce1 (correct), seq 1602703235, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val 1011569352 ecr 0,sackOK,eol], length 0
13:58:15.313846 00:31:46:0d:43:5a > a8:a1:59:00:32:ed, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 48, id 0, offset 0, flags [DF], proto TCP (6), length 60)
     3.123.231.xxx.443 > S.N.A.T.62509: Flags [S.], cksum 0x7bbb (correct), seq 3550522983, ack 1602703236, win 62643, options [mss 1460,sackOK,TS val 3719740737 ecr 1011569352,nop,wscale 7], length 0
13:58:16.317557 a8:a1:59:00:32:ed > 00:31:46:0d:43:5a, ethertype IPv4 (0x0800), length 78: (tos 0x0, ttl 61, id 0, offset 0, flags [DF], proto TCP (6), length 64)
     S.N.A.T.62509 > 3.123.231.xxx.443: Flags [S], cksum 0x78f9 (correct), seq 1602703235, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val 1011570352 ecr 0,sackOK,eol], length 0
13:58:16.322958 00:31:46:0d:43:5a > a8:a1:59:00:32:ed, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 48, id 0, offset 0, flags [DF], proto TCP (6), length 60)
     3.123.231.xxx.443 > S.N.A.T.62509: Flags [S.], cksum 0xdbdc (correct), seq 3566290300, ack 1602703236, win 62643, options [mss 1460,sackOK,TS val 3719741746 ecr 1011570352,nop,wscale 7], length 0

- VM 'vnet5' see tons of SYN-ACK responses, which, possibly, are looped
inside OVS:

13:58:15.302960 52:54:00:7a:9e:a4 > 52:54:00:5b:d4:bf, ethertype IPv4 (0x0800), length 78: (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 64)
     172.29.24.53.62509 > 3.123.231.xxx.443: Flags [S], cksum 0x5397 (correct), seq 1602703235, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val 1011569352 ecr 0,sackOK,eol], length 0
13:58:15.302992 52:54:00:5b:d4:bf > 86:d1:8b:b3:d7:4d, ethertype IPv4 (0x0800), length 78: (tos 0x0, ttl 62, id 0, offset 0, flags [DF], proto TCP (6), length 64)
     172.29.24.53.62509 > 3.123.231.xxx.443: Flags [S], cksum 0x5397 (correct), seq 1602703235, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val 1011569352 ecr 0,sackOK,eol], length 0
13:58:15.308659 86:d1:8b:b3:d7:4d > 52:54:00:5b:d4:bf, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 47, id 0, offset 0, flags [DF], proto TCP (6), length 60)
     3.123.231.xxx.443 > 172.29.24.53.62509: Flags [S.], cksum 0x5271 (correct), seq 3550522983, ack 1602703236, win 62643, options [mss 1460,sackOK,TS val 3719740737 ecr 1011569352,nop,wscale 7], length 0
13:58:15.308710 52:54:00:5b:d4:bf > 86:d1:8b:b3:d7:4d, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 46, id 0, offset 0, flags [DF], proto TCP (6), length 60)
     3.123.231.xxx.443 > 172.29.24.53.62509: Flags [S.], cksum 0x5271 (correct), seq 3550522983, ack 1602703236, win 62643, options [mss 1460,sackOK,TS val 3719740737 ecr 1011569352,nop,wscale 7], length 0
13:58:15.308764 86:d1:8b:b3:d7:4d > 52:54:00:5b:d4:bf, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 45, id 0, offset 0, flags [DF], proto TCP (6), length 60)
     3.123.231.xxx.443 > 172.29.24.53.62509: Flags [S.], cksum 0x5271 (correct), seq 3550522983, ack 1602703236, win 62643, options [mss 1460,sackOK,TS val 3719740737 ecr 1011569352,nop,wscale 7], length 0
13:58:15.308769 52:54:00:5b:d4:bf > 86:d1:8b:b3:d7:4d, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 44, id 0, offset 0, flags [DF], proto TCP (6), length 60)
     3.123.231.xxx.443 > 172.29.24.53.62509: Flags [S.], cksum 0x5271 (correct), seq 3550522983, ack 1602703236, win 62643, options [mss 1460,sackOK,TS val 3719740737 ecr 1011569352,nop,wscale 7], length 0
13:58:15.308792 86:d1:8b:b3:d7:4d > 52:54:00:5b:d4:bf, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 43, id 0, offset 0, flags [DF], proto TCP (6), length 60)
     3.123.231.xxx.443 > 172.29.24.53.62509: Flags [S.], cksum 0x5271 (correct), seq 3550522983, ack 1602703236, win 62643, options [mss 1460,sackOK,TS val 3719740737 ecr 1011569352,nop,wscale 7], length 0
13:58:15.308794 52:54:00:5b:d4:bf > 86:d1:8b:b3:d7:4d, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 42, id 0, offset 0, flags [DF], proto TCP (6), length 60)
     3.123.231.xxx.443 > 172.29.24.53.62509: Flags [S.], cksum 0x5271 (correct), seq 3550522983, ack 1602703236, win 62643, options [mss 1460,sackOK,TS val 3719740737 ecr 1011569352,nop,wscale 7], length 0
[ ... and so on from 3.123.231.xxx 'S.'s ... ]

So, the question is - whether it's possible to catch and redirect
traffic, which comes from internal port, "landed" in host's system?

Thank you.

--
Volodymyr Litovka
   "Vision without Execution is Hallucination." -- Thomas Edison

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20210219/d61f1f25/attachment.html>