[ovs-dev] Fwd: tracing ovs flows in br-int

Tue Jan 19 12:14:48 UTC 2021

On 1/18/21 7:38 PM, Brendan Doyle wrote:
> 
> 
> 
> -------- Forwarded Message --------
> Subject:     tracing ovs flows in br-int
> Date:     Mon, 18 Jan 2021 18:31:53 +0000
> From:     Brendan Doyle <brendan.doyle at oracle.com>
> Organization:     Oracle Corporation
> To:     ovs-discuss at openvswitch.org
> 
> 
> 
> Hi Folks,
> 
> I'm trying to trace a flow through br-int but ovs-appctl ofproto/trace br-int is not
> giving me the output I expect to see.
> 
> I'm trying to trace the pkt below through br-int on the remote chassis. The pkt
> does get tunnel and delivered to its destination, which replies, and the reply
> is successfully tunneled back to the initiating host, so the flows are working, but
> I just can't seem to trace them.
> 
> pkt in question (extract from tcpdump):
> ------------------------------------------------------------------------------
> 00:00:00.000000 98:03:9b:59:af:1c > 98:03:9b:3f:d9:1e, ethertype IPv4 (0x0800), length 156: (tos 0x0, ttl 64, id 9804, offset 0, flags [DF], proto UDP (17), length 142)
>     253.255.2.6.47096 > 253.255.0.35.6081: [udp sum ok] Geneve, Flags [C], vni 0x392, proto TEB (0x6558), options [class Open Virtual Networking (OVN) (0x102) type 0x80(C) len 8 data 00010007]
>     52:54:00:be:06:16 > 40:44:00:00:04:10, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 43284, offset 0, flags [DF], proto ICMP (1), length 84)
>     192.16.1.6 > 169.254.169.254: ICMP echo request, id 25237, seq 1, length 64
>     0x0000:  9803 9b3f d91e 9803 9b59 af1c 0800 4500
>     0x0010:  008e 264c 4000 4011 15eb fdff 0206 fdff
>     0x0020:  0023 b7f8 17c1 007a 10f4 0240 6558 0003
>     0x0030:  9200 0102 8001 0001 0007 4044 0000 0410
>     0x0040:  5254 00be 0616 0800 4500 0054 a914 4000
>     0x0050:  4001 7c81 c010 0106 a9fe a9fe 0800 f629
>     0x0060:  6295 0001 f166 0560 0000 0000 e3a5 0600
>     0x0070:  0000 0000 1011 1213 1415 1617 1819 1a1b
>     0x0080:  1c1d 1e1f 2021 2223 2425 2627 2829 2a2b
>     0x0090:  2c2d 2e2f 3031 3233 3435 3637
> ---------------------------------------------------------------------------
> 
> The ovs br-int details on the target chassis:
> 
> ovs-appctl dpif/show
> system at ovs-system: hit:6576964 missed:4469
>   br-int:
>     br-int 65534/2: (internal)
>     ovn-ca-rai-0 2/1: (geneve: csum=true, key=flow, remote_ip=253.255.2.17)
>     ovn-ca-rai-1 11/1: (geneve: csum=true, key=flow, remote_ip=253.255.0.34)
>     ovn-ca-rai-2 5/1: (geneve: csum=true, key=flow, remote_ip=253.255.0.33)
>     ovn-ca-rai-3 3/1: (geneve: csum=true, key=flow, remote_ip=253.255.2.6) <-- tunnel port for VM host (3)
>     ovn-ca-rai-4 4/1: (geneve: csum=true, key=flow, remote_ip=253.255.2.5)
>     vethVn1M 84/3: (system) <-- Veth port (84)
> 
> I thought this command would replicate the pkt that is deviled to the tunnel port
> on the target chassis:
> 
> -------------------------------------------------------------
> ovs-appctl ofproto/trace br-int in_port=3 98039b3fd91e98039b59af1c08004500008e264c4000401115ebfdff0206fdff0023b7f817c1007a10f402406558000392000102800100010007404400000410525400be0616080045000054a914400040017c81c0100106a9fea9fe0800f62962950001f166056000000000e3a5060000000000101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f3031323334353637
> Flow: udp,in_port=3,vlan_tci=0x0000,dl_src=98:03:9b:59:af:1c,dl_dst=98:03:9b:3f:d9:1e,nw_src=253.255.2.6,nw_dst=253.255.0.35,nw_tos=0,nw_ecn=0,nw_ttl=64,tp_src=47096,tp_dst=6081
> 
> bridge("br-int")
> ----------------
>  0. in_port=3, priority 100
>     move:NXM_NX_TUN_ID[0..23]->OXM_OF_METADATA[0..23]
>      -> OXM_OF_METADATA[0..23] is now 0
>     move:NXM_NX_TUN_METADATA0[16..30]->NXM_NX_REG14[0..14]
>      -> NXM_NX_REG14[0..14] is now 0
>     move:NXM_NX_TUN_METADATA0[0..15]->NXM_NX_REG15[0..15]
>      -> NXM_NX_REG15[0..15] is now 0
>     resubmit(,33)
> 33. No match.
>     drop
> 
> Final flow: unchanged
> Megaflow: recirc_id=0,eth,ip,tun_id=0/0xffffff,in_port=3,nw_frag=no
> Datapath actions: drop
> ---------------------------------------------------------
> 
> But it says the packet is dropped, when in actual fact it is output on port 84 as expected.
> So I'm wondering am I inserting the pkt wrong? to the wrong port on the remote chassis?

Hi.  While receiving packets on tunnel ports, kernel fills up tunnel metadata.
You can see that rules from table 0 are trying to extract TUN_ID and TUN_METADATA0
fields, but you're passing pure packet without this information, so flows in
table 33 that are likely matches on extracted metadata doesn't fit.

You need to pass the packet in a different format with tunnel metadata included
in order to have behavior similar to what you have for a real packet, e.g.

  ovs-appctl ofproto/trace br-int in_port=3,tun_id=<tunnel id> <packet>

You may also need to specify other things like tun_src/dst, tun_metadata0, tun_tos
or tun_flags.

Since you have 'key=flow', you need to find out the correct tun_id for your tunnel
in your OF rules.

Best regards, Ilya Maximets.