[ovs-discuss] OVS dropping NSH packets that are correctly encapsulated

[Chang Ge]

Hello, 

I'm trying to use OVS 2.11.9 (without controller) in Ubuntu 14.04 to push NSH and ETH headers to an ICMP packet. The topology is as follows: 

Client ---> (p1p1) Classifier (p1p2) ---> Server

where p1p1 and p1p2 interfaces on the classifier are bound to the OVS bridge br0. 

The problem is that after a packet enters classifier's p1p1, it gets dropped by OVS datapath before it reaches br0. 

dmesg shows error message as
[ 3538.861851] openvswitch: netlink: Flow actions may not be safe on all matching packets.

ovs-vswitchd.log shows 
2019-06-20T13:05:13.942Z|03925|dpif(handler7)|WARN|system at ovs-system: failed to put[create] (Invalid argument) ufid:7f5e3b96-75ba-4a36-8385-fcb4ee4cda93 recirc_id(0),dp_hash(0/0),skb_priority(0/0),in_port(2),skb_mark(0/0),eth(src=a0:36:9f:a3:5e:f0,dst=a0:36:9f:a3:5c:30),eth_type(0x0800),ipv4(src=10.0.0.1/0.0.0.0,dst=13.0.0.1,proto=1/0,tos=0/0,ttl=64/0,frag=no),icmp(type=8/0,code=0/0), actions:push_nsh(flags=2,ttl=63,mdtype=1,np=3,spi=0x1234,si=255,c1=0x11223344,c2=0x11223344,c3=0x11223344,c4=0x11223344),push_eth(src=a0:36:9f:a3:5c:30,dst=a0:36:9f:8e:4b:48),1

Any suggestion please? 

== dump-flows output ==
root at ovs-br-1:/home/chang# ovs-ofctl -O Openflow13 dump-flows br0
 cookie=0x0, duration=669.284s, table=0, n_packets=669, n_bytes=65562, ip,in_port=p1p1,nw_dst=13.0.0.1 actions=encap(nsh(md_type=1)),set_field:2->nsh_flags,set_field:0x1234->nsh_spi,set_field:255->nsh_si,set_field:63->nsh_ttl,set_field:0x11223344->nsh_c1,set_field:0x11223344->nsh_c2,set_field:0x11223344->nsh_c3,set_field:0x11223344->nsh_c4,goto_table:4
 cookie=0x0, duration=669.281s, table=0, n_packets=16, n_bytes=960, reset_counts priority=1000,arp,arp_tpa=10.0.0.10,arp_op=1 actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],set_field:a0:36:9f:a3:5c:30->eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],load:0xa0369fa35c30->NXM_NX_ARP_SHA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],load:0xa00000a->NXM_OF_ARP_SPA[],IN_PORT
 cookie=0x0, duration=669.252s, table=4, n_packets=669, n_bytes=65562, packet_type=(1,0x894f) actions=encap(ethernet),set_field:a0:36:9f:a3:5c:30->eth_src,set_field:a0:36:9f:8e:4b:48->eth_dst,goto_table:10
 cookie=0x0, duration=552.056s, table=10, n_packets=552, n_bytes=54096, dl_type=0x894f actions=output:p1p2

== appctl trace output ==
Flow: ip,in_port=1,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,nw_src=0.0.0.0,nw_dst=13.0.0.1,nw_proto=0,nw_tos=0,nw_ecn=0,nw_ttl=0

bridge("br0")
-------------
 0. ip,in_port=1,nw_dst=13.0.0.1, priority 32768
    encap(nsh(md_type=1))
    set_field:2->nsh_flags
    set_field:0x1234->nsh_spi
    set_field:255->nsh_si
    set_field:63->nsh_ttl
    set_field:0x11223344->nsh_c1
    set_field:0x11223344->nsh_c2
    set_field:0x11223344->nsh_c3
    set_field:0x11223344->nsh_c4
    goto_table:4
 4. packet_type=(1,0x894f), priority 32768
    encap(ethernet)
    set_field:a0:36:9f:a3:5c:30->eth_src
    set_field:a0:36:9f:8e:4b:48->eth_dst
    goto_table:10
10. dl_type=0x894f, priority 32768
    output:2

Final flow: in_port=1,vlan_tci=0x0000,dl_src=a0:36:9f:a3:5c:30,dl_dst=a0:36:9f:8e:4b:48,dl_type=0x894f,nsh_flags=2,nsh_ttl=63,nsh_mdtype=1,nsh_np=3,nsh_spi=0x1234,nsh_si=255,nsh_c1=0x11223344,nsh_c2=0x11223344,nsh_c3=0x11223344,nsh_c4=0x11223344,nw_proto=0,nw_tos=0,nw_ecn=0,nw_ttl=0
Megaflow: recirc_id=0,eth,ip,in_port=1,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,nw_dst=13.0.0.1,nw_frag=no
Datapath actions: push_nsh(flags=2,ttl=63,mdtype=1,np=3,spi=0x1234,si=255,c1=0x11223344,c2=0x11223344,c3=0x11223344,c4=0x11223344),push_eth(src=a0:36:9f:a3:5c:30,dst=a0:36:9f:8e:4b:48),1

== ovs-vsctl output ==
9e1192fd-3f02-497f-90e5-64d24b24932f
    Bridge "br0"
        Port "p1p1"
            Interface "p1p1"
        Port "br0"
            Interface "br0"
                type: internal
        Port "p1p2"
            Interface "p1p2"
    ovs_version: "2.11.90"

Regards, 
Chang