[ovs-discuss] No connectivity due to missing ARP reply

Plato, Michael michael.plato at tu-berlin.de
Sat Mar 21 18:04:00 UTC 2020


Hi all,

we use OVN with Openstack and have a problem with the following setup:


				|					|
	-------		|	  10.176.0.156	|	-------
	| VM1 |-----   |	 192.168.0.1	        |---| VM2 |
	-------		|	   --------		        |	-------
10.176.0.3.123	|------|  R1  |-------------|	192.168.0.201 / GW: 192.168.0.1
GW:10.176.0.1	|	  |(test)|		|	FIP: 10.176.2.19
				|	   --------		        |
			 Outside		                      test
		 (10.176.0.0/16)  	             (192.168.0.0/24)
			 (VLAN)			 	 (GENEVE)


Versions:
- OVN (20.03)
- OVS (2.13)
- networking-ovn (7.1.0)
			 
Problem: 
- no connectivity due to missing ARP reply for FIP 10.176.2.19 from VM1 (if VM1 is not on GW Chassis for R1 -> is_chassis_resident rules not applied)
- after moving VM1 to chassis hosting R1 ARP reply appears (due to local "is_chassis_resident" ARP responder rules)
- temporarily removing priority 75 rules (inserted by commit [0]) restores functionality (even on non gateway chassis), because ARP requests were flooded to complete L2 domain (but this creates a scaling issue)


Analysis:
- according to ovs-detrace the ARP requests were dropped instead of being forwarded to remote chassis hosting R1 (as intended by [0])


Flow: arp,in_port=61,vlan_tci=0x0000,dl_src=fa:16:3e:5e:79:d9,dl_dst=ff:ff:ff:ff:ff:ff,arp_spa=10.176.3.123,arp_tpa=10.176.2.19,arp_op=1,arp_sha=fa:16:3e:5e:79:d9,arp_tha=00:00:00:00:00:00


bridge("br-int")
----------------
0. in_port=61, priority 100, cookie 0x862b95fc
set_field:0x1->reg13
set_field:0x7->reg11
set_field:0x5->reg12
set_field:0x1a->metadata
set_field:0x4->reg14
resubmit(,8)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d)
  *  Port Binding: logical_port "b19ceab1-c7fe-4c3b-8733-d88cabaa0a23", tunnel_key 4, chassis-name "383eb44a-de85-485a-9606-2fc649a9cbb9", chassis-str "os-compute-01"
8. reg14=0x4,metadata=0x1a,dl_src=fa:16:3e:5e:79:d9, priority 50, cookie 0x9a357820
resubmit(,9)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [ingress]
  *  Logical flow: table=0 (ls_in_port_sec_l2), priority=50, match=(inport == "b19ceab1-c7fe-4c3b-8733-d88cabaa0a23" && eth.src == {fa:16:3e:5e:79:d9}), actions=(next;)
   *  Logical Switch Port: b19ceab1-c7fe-4c3b-8733-d88cabaa0a23 type  (addresses ['fa:16:3e:5e:79:d9 10.176.3.123'], dynamic addresses [], security ['fa:16:3e:5e:79:d9 10.176.3.123']
9. metadata=0x1a, priority 0, cookie 0x1a478ee1
resubmit(,10)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [ingress]
  *  Logical flow: table=1 (ls_in_port_sec_ip), priority=0, match=(1), actions=(next;)
10. arp,reg14=0x4,metadata=0x1a,dl_src=fa:16:3e:5e:79:d9,arp_spa=10.176.3.123,arp_sha=fa:16:3e:5e:79:d9, priority 90, cookie 0x8c5af8ff
resubmit(,11)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [ingress]
  *  Logical flow: table=2 (ls_in_port_sec_nd), priority=90, match=(inport == "b19ceab1-c7fe-4c3b-8733-d88cabaa0a23" && eth.src == fa:16:3e:5e:79:d9 && arp.sha == fa:16:3e:5e:79:d9 && arp.spa == {10.176.3.123}), actions=(next;)
   *  Logical Switch Port: b19ceab1-c7fe-4c3b-8733-d88cabaa0a23 type  (addresses ['fa:16:3e:5e:79:d9 10.176.3.123'], dynamic addresses [], security ['fa:16:3e:5e:79:d9 10.176.3.123']
11. metadata=0x1a, priority 0, cookie 0x13f72632
resubmit(,12)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [ingress]
  *  Logical flow: table=3 (ls_in_pre_acl), priority=0, match=(1), actions=(next;)
12. metadata=0x1a, priority 0, cookie 0xe38d6752
resubmit(,13)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [ingress]
  *  Logical flow: table=4 (ls_in_pre_lb), priority=0, match=(1), actions=(next;)
13. metadata=0x1a, priority 0, cookie 0xa9a6ed5
resubmit(,14)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [ingress]
  *  Logical flow: table=5 (ls_in_pre_stateful), priority=0, match=(1), actions=(next;)
14. metadata=0x1a, priority 0, cookie 0xcf9951d4
resubmit(,15)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [ingress]
  *  Logical flow: table=6 (ls_in_acl), priority=0, match=(1), actions=(next;)
15. metadata=0x1a, priority 0, cookie 0xcc08c09e
resubmit(,16)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [ingress]
  *  Logical flow: table=7 (ls_in_qos_mark), priority=0, match=(1), actions=(next;)
16. metadata=0x1a, priority 0, cookie 0x918349d8
resubmit(,17)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [ingress]
  *  Logical flow: table=8 (ls_in_qos_meter), priority=0, match=(1), actions=(next;)
17. metadata=0x1a, priority 0, cookie 0x944ba2a
resubmit(,18)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [ingress]
  *  Logical flow: table=9 (ls_in_lb), priority=0, match=(1), actions=(next;)
18. metadata=0x1a, priority 0, cookie 0xcbae6cab
resubmit(,19)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [ingress]
  *  Logical flow: table=10 (ls_in_stateful), priority=0, match=(1), actions=(next;)
19. metadata=0x1a, priority 0, cookie 0xf96fbcc8
resubmit(,20)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [ingress]
20. metadata=0x1a, priority 0, cookie 0x5b9711bf
resubmit(,21)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [ingress]
  *  Logical flow: table=12 (ls_in_hairpin), priority=0, match=(1), actions=(next;)
21. metadata=0x1a, priority 0, cookie 0x120d1c68
resubmit(,22)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [ingress]
  *  Logical flow: table=13 (ls_in_arp_rsp), priority=0, match=(1), actions=(next;)
22. metadata=0x1a, priority 0, cookie 0xd446226f
resubmit(,23)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [ingress]
  *  Logical flow: table=14 (ls_in_dhcp_options), priority=0, match=(1), actions=(next;)
23. metadata=0x1a, priority 0, cookie 0x31b45717
resubmit(,24)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [ingress]
  *  Logical flow: table=15 (ls_in_dhcp_response), priority=0, match=(1), actions=(next;)
24. metadata=0x1a, priority 0, cookie 0x715db0f1
resubmit(,25)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [ingress]
  *  Logical flow: table=16 (ls_in_dns_lookup), priority=0, match=(1), actions=(next;)
25. metadata=0x1a, priority 0, cookie 0xd12f2910
resubmit(,26)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [ingress]
  *  Logical flow: table=17 (ls_in_dns_response), priority=0, match=(1), actions=(next;)
26. metadata=0x1a, priority 0, cookie 0x97f5a6b7
resubmit(,27)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [ingress]
  *  Logical flow: table=18 (ls_in_external_port), priority=0, match=(1), actions=(next;)
27. arp,reg10=0/0x2,metadata=0x1a,arp_tpa=10.176.2.19,arp_op=1, priority 75, cookie 0x4a641791
set_field:0x5->reg15
resubmit(,32)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [ingress]
  *  Logical flow: table=19 (ls_in_l2_lkup), priority=75, match=(flags[1] == 0 && arp.op == 1 && arp.tpa == { 10.176.2.19, 10.176.0.156}), actions=(outport = "9f1c631b-ea6c-4f50-85f0-a5b0bb6e2e8e"; output;)
   *  Logical Switch Port: 9f1c631b-ea6c-4f50-85f0-a5b0bb6e2e8e type router (addresses ['router'], dynamic addresses [], security []
32. priority 0
resubmit(,33)
33. reg15=0x5,metadata=0x1a, priority 100
set_field:0x7->reg11
set_field:0x5->reg12
resubmit(,34)
34. priority 0
set_field:0->reg0
set_field:0->reg1
set_field:0->reg2
set_field:0->reg3
set_field:0->reg4
set_field:0->reg5
set_field:0->reg6
set_field:0->reg7
set_field:0->reg8
set_field:0->reg9
resubmit(,40)
40. metadata=0x1a, priority 0, cookie 0xf960a9ea
resubmit(,41)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [egress]
  *  Logical flow: table=0 (ls_out_pre_lb), priority=0, match=(1), actions=(next;)
41. metadata=0x1a, priority 0, cookie 0x31e15a58
resubmit(,42)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [egress]
  *  Logical flow: table=1 (ls_out_pre_acl), priority=0, match=(1), actions=(next;)
42. metadata=0x1a, priority 0, cookie 0x7089b16e
resubmit(,43)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [egress]
  *  Logical flow: table=2 (ls_out_pre_stateful), priority=0, match=(1), actions=(next;)
43. metadata=0x1a, priority 0, cookie 0x8ab997b2
resubmit(,44)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [egress]
  *  Logical flow: table=3 (ls_out_lb), priority=0, match=(1), actions=(next;)
44. metadata=0x1a, priority 0, cookie 0x62e08a84
resubmit(,45)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [egress]
  *  Logical flow: table=4 (ls_out_acl), priority=0, match=(1), actions=(next;)
45. metadata=0x1a, priority 0, cookie 0x19ac76fa
resubmit(,46)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [egress]
  *  Logical flow: table=5 (ls_out_qos_mark), priority=0, match=(1), actions=(next;)
46. metadata=0x1a, priority 0, cookie 0x826c6009
resubmit(,47)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [egress]
  *  Logical flow: table=6 (ls_out_qos_meter), priority=0, match=(1), actions=(next;)
47. metadata=0x1a, priority 0, cookie 0xbaa04ed8
resubmit(,48)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [egress]
  *  Logical flow: table=7 (ls_out_stateful), priority=0, match=(1), actions=(next;)
48. metadata=0x1a, priority 0, cookie 0x7c014dc6
resubmit(,49)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [egress]
  *  Logical flow: table=8 (ls_out_port_sec_ip), priority=0, match=(1), actions=(next;)
49. metadata=0x1a,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00, priority 100, cookie 0x3a1562c4
resubmit(,64)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d) [egress]
  *  Logical flow: table=9 (ls_out_port_sec_l2), priority=100, match=(eth.mcast), actions=(output;)
64. priority 0
resubmit(,65)
65. reg15=0x5,metadata=0x1a, priority 100, cookie 0x99c3e9ca
clone(ct_clear,set_field:0->reg11,set_field:0->reg12,set_field:0->reg13,set_field:0x3->reg11,set_field:0x4->reg12,set_field:0x26->metadata,set_field:0x1->reg14,set_field:0->reg10,set_field:0->reg15,set_field:0->reg0,set_field:0->reg1,set_field:0->reg2,set_field:0->reg3,set_field:0->reg4,set_field:0->reg5,set_field:0->reg6,set_field:0->reg7,set_field:0->reg8,set_field:0->reg9,resubmit(,8))
ct_clear
set_field:0->reg11
set_field:0->reg12
set_field:0->reg13
set_field:0x3->reg11
set_field:0x4->reg12
set_field:0x26->metadata
set_field:0x1->reg14
set_field:0->reg10
set_field:0->reg15
set_field:0->reg0
set_field:0->reg1
set_field:0->reg2
set_field:0->reg3
set_field:0->reg4
set_field:0->reg5
set_field:0->reg6
set_field:0->reg7
set_field:0->reg8
set_field:0->reg9
resubmit(,8)
  *  Logical datapath: "neutron-c2a82a31-632b-4d24-8f35-8a79e2a207a7" (d516056b-19a6-4613-9838-8c62452fe31d)
  *  Port Binding: logical_port "9f1c631b-ea6c-4f50-85f0-a5b0bb6e2e8e", tunnel_key 5,
8. reg14=0x1,metadata=0x26,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00, priority 50, cookie 0x12ef5598
resubmit(,9)
  *  Logical datapath: "neutron-673c043b-5c3a-4eae-90d9-bf6c87c1fc37" (f7fad4e5-71bc-42cd-8a1b-048e25137dc0) [ingress]
  *  Logical flow: table=0 (lr_in_admission), priority=50, match=(eth.mcast && inport == "lrp-9f1c631b-ea6c-4f50-85f0-a5b0bb6e2e8e), actions=(next;)
   *  Logical Router Port: lrp-9f1c631b-ea6c-4f50-85f0-a5b0bb6e2e8e mac fa:16:3e:58:84:8c networks ['10.176.0.156/16'] ipv6_ra_configs {}
9. metadata=0x26, priority 0, cookie 0xab1b8863
load:0x1->OXM_OF_PKT_REG4[3]
resubmit(,10)
  *  Logical datapath: "neutron-673c043b-5c3a-4eae-90d9-bf6c87c1fc37" (f7fad4e5-71bc-42cd-8a1b-048e25137dc0) [ingress]
  *  Logical flow: table=1 (lr_in_lookup_neighbor), priority=0, match=(1), actions=(reg9[3] = 1; next;)
10. reg9=0x8/0x8,metadata=0x26, priority 100, cookie 0x742e0523
resubmit(,11)
  *  Logical datapath: "neutron-673c043b-5c3a-4eae-90d9-bf6c87c1fc37" (f7fad4e5-71bc-42cd-8a1b-048e25137dc0) [ingress]
  *  Logical flow: table=2 (lr_in_learn_neighbor), priority=100, match=(reg9[3] == 1 || reg9[2] == 1), actions=(next;)
11. arp,metadata=0x26, priority 85, cookie 0xb1c400fe
drop
  *  Logical datapath: "neutron-673c043b-5c3a-4eae-90d9-bf6c87c1fc37" (f7fad4e5-71bc-42cd-8a1b-048e25137dc0) [ingress]
  *  Logical flow: table=3 (lr_in_ip_input), priority=85, match=(arp || nd), actions=(drop;)


Final flow: arp,reg11=0x7,reg12=0x5,reg13=0x1,reg14=0x4,reg15=0x5,metadata=0x1a,in_port=61,vlan_tci=0x0000,dl_src=fa:16:3e:5e:79:d9,dl_dst=ff:ff:ff:ff:ff:ff,arp_spa=10.176.3.123,arp_tpa=10.176.2.19,arp_op=1,arp_sha=fa:16:3e:5e:79:d9,arp_tha=00:00:00:00:00:00
Megaflow: recirc_id=0,ct_state=-new-est-rel-rpl-inv-trk,ct_label=0/0x1,eth,arp,in_port=61,dl_src=fa:16:3e:5e:79:d9,dl_dst=ff:ff:ff:ff:ff:ff,arp_spa=10.176.3.123,arp_tpa=10.176.2.19,arp_op=1,arp_sha=fa:16:3e:5e:79:d9
Datapath actions: ct_clear


Looks like theres a rule missing for tunneling ARP/ND to remote chassis in case of distributed router?


Thanks a lot,


Michael


[0] https://github.com/ovn-org/ovn/commit/32f5ebb06226e3433e53e05bdc75d16752859a0e​




More information about the discuss mailing list