[ovs-discuss] Tunnel Ports and Packet Marks

Sat Oct 14 01:14:39 UTC 2017

I have seen behavior where it appears that for OVS to match incoming GRE
traffic received by the host to a configured GRE port through OVS, the
packet mark must be a specific value.  Otherwise, the packet is silently
discarded or at least I can't find any log if it being discarded.   Is this
expected behavior?

I can't find it documented anywhere and I can not see how/where to see what
value OVS is expecting this to be.  Is there a command to see how packets
are classified / matched to a configured GRE port and what mark value is
required?  Dumping the interface table with ovs-vsctl doesn't show any mark
options.  I can't find an ovs-ofctl or ovs-dpctl command to show this.

We are running into this because our GRE traffic is being marked with a
specific packet mark when it leaves the system.  Conntrack has the GRE
traffic with a specific mark and through experimentation it seems if that
mark value is not in the packet-mark after the iptables filter input stage
and when the host receives the GRE packet, the packet is never recorded as
received on the OVS/OF GRE port.  I am looking at the "ovs-ofctl
dump-ports" output to see when traffic is received on the GRE port.  The RX
counter remains fixed until I add a single iptables command in the filter
INPUT stage and then it starts to increment.

Currently using 2.7.0 and I can provide output if it helps but I'm not sure
what would be helpful.  Here is the iptables trace output for a GRE packet,
you can see that for packet ID 58534 it has a skb_mark of 0x4 up until the
last step and then it's cleared or set to 0.  While in this state, the RX
counter remains at 1 for Port 1 even though many packets arrive.

Oct 14 01:05:08 cd58 kernel: TRACE: raw:PREROUTING:policy:2 IN=lan0 OUT=
MAC=dc:39:79:80:70:00:00:1c:73:23:24:62:08:00 SRC=10.11.176.3
DST=10.129.0.50 LEN=88 TOS=0x00 PREC=0x00 TTL=64 ID=58534 PROTO=47 MARK=0x4
Oct 14 01:05:08 cd58 kernel: TRACE: mangle:PREROUTING:policy:2 IN=lan0 OUT=
MAC=dc:39:79:80:70:00:00:1c:73:23:24:62:08:00 SRC=10.11.176.3
DST=10.129.0.50 LEN=88 TOS=0x00 PREC=0x00 TTL=64 ID=58534 PROTO=47 MARK=0x4
Oct 14 01:05:08 cd58 kernel: TRACE: mangle:INPUT:policy:2 IN=lan0 OUT=
MAC=dc:39:79:80:70:00:00:1c:73:23:24:62:08:00 SRC=10.11.176.3
DST=10.129.0.50 LEN=88 TOS=0x00 PREC=0x00 TTL=64 ID=58534 PROTO=47 MARK=0x4
Oct 14 01:05:08 cd58 kernel: TRACE: filter:INPUT:rule:2 IN=lan0 OUT=
MAC=dc:39:79:80:70:00:00:1c:73:23:24:62:08:00 SRC=10.11.176.3
DST=10.129.0.50 LEN=88 TOS=0x00 PREC=0x00 TTL=64 ID=58534 PROTO=47 MARK=0x4
Oct 14 01:05:08 cd58 kernel: TRACE: filter:acceptOverlay:rule:1 IN=lan0
OUT= MAC=dc:39:79:80:70:00:00:1c:73:23:24:62:08:00 SRC=10.11.176.3
DST=10.129.0.50 LEN=88 TOS=0x00 PREC=0x00 TTL=64 ID=58534 PROTO=47 MARK=0x4
Oct 14 01:05:08 cd58 kernel: TRACE: filter:acceptOverlay:return:2 IN=lan0
OUT= MAC=dc:39:79:80:70:00:00:1c:73:23:24:62:08:00 SRC=10.11.176.3
DST=10.129.0.50 LEN=88 TOS=0x00 PREC=0x00 TTL=64 ID=58534 PROTO=47

root at cd58:~# ovs-ofctl dump-ports iB-MJFTUWEX9CKQ
OFPST_PORT reply (xid=0x2): 3 ports
  port LOCAL: rx pkts=0, bytes=0, drop=4293, errs=0, frame=0, over=0, crc=0
           tx pkts=0, bytes=0, drop=0, errs=0, coll=0
 * port  1: rx pkts=1, bytes=60, drop=?, errs=?, frame=?, over=?, crc=?*
*           tx pkts=4293, bytes=258568, drop=?, errs=?, coll=?*
  port  2: rx pkts=4293, bytes=258568, drop=0, errs=0, frame=0, over=0,
crc=0
           tx pkts=0, bytes=0, drop=0, errs=0, coll=0

Now I change iptables to accept the packet before adjusting the mark so it
remains at 4.

root at cd58:~# iptables -t filter -I acceptOverlay 1 -j ACCEPT

and this matches conntrack:

root at cd58:~# conntrack -L -p 47
gre      47 179 src=10.129.0.50 dst=10.11.176.3 srckey=0x0 dstkey=0x0
packets=8989 bytes=879106 src=10.11.176.3 dst=10.129.0.50 srckey=0x0
dstkey=0x0 packets=11130 bytes=1066726 [ASSURED] mark=4 delta-time=6176
use=1

and now the mark remains:

root at cd58:~# tail /var/log/syslog -n 10

Oct 14 01:06:27 cd58 kernel: TRACE: raw:PREROUTING:policy:2 IN=lan0 OUT=
MAC=dc:39:79:80:70:00:00:1c:73:23:24:62:08:00 SRC=10.11.176.3
DST=10.129.0.50 LEN=126 TOS=0x00 PREC=0x00 TTL=64 ID=58656 PROTO=47
MARK=0x4
Oct 14 01:06:27 cd58 kernel: TRACE: mangle:PREROUTING:policy:2 IN=lan0 OUT=
MAC=dc:39:79:80:70:00:00:1c:73:23:24:62:08:00 SRC=10.11.176.3
DST=10.129.0.50 LEN=126 TOS=0x00 PREC=0x00 TTL=64 ID=58656 PROTO=47
MARK=0x4
Oct 14 01:06:27 cd58 kernel: TRACE: mangle:INPUT:policy:2 IN=lan0 OUT=
MAC=dc:39:79:80:70:00:00:1c:73:23:24:62:08:00 SRC=10.11.176.3
DST=10.129.0.50 LEN=126 TOS=0x00 PREC=0x00 TTL=64 ID=58656 PROTO=47
MARK=0x4
Oct 14 01:06:27 cd58 kernel: TRACE: filter:INPUT:rule:2 IN=lan0 OUT=
MAC=dc:39:79:80:70:00:00:1c:73:23:24:62:08:00 SRC=10.11.176.3
DST=10.129.0.50 LEN=126 TOS=0x00 PREC=0x00 TTL=64 ID=58656 PROTO=47
MARK=0x4
Oct 14 01:06:27 cd58 kernel: TRACE: filter:acceptOverlay:return:1 IN=lan0
OUT= MAC=dc:39:79:80:70:00:00:1c:73:23:24:62:08:00 SRC=10.11.176.3
DST=10.129.0.50 LEN=126 TOS=0x00 PREC=0x00 TTL=64 ID=58656 PROTO=47
MARK=0x4

And then suddenly packets are received:

root at cd58:~# ovs-ofctl dump-ports iB-MJFTUWEX9CKQ
OFPST_PORT reply (xid=0x2): 3 ports
  port LOCAL: rx pkts=0, bytes=0, drop=4359, errs=0, frame=0, over=0, crc=0
           tx pkts=0, bytes=0, drop=0, errs=0, coll=0
*  port  1: rx pkts=13, bytes=1198, drop=?, errs=?, frame=?, over=?, crc=?*
*           tx pkts=4370, bytes=263606, drop=?, errs=?, coll=?*
  port  2: rx pkts=4370, bytes=263606, drop=0, errs=0, frame=0, over=0,
crc=0
           tx pkts=12, bytes=1138, drop=0, errs=0, coll=0

So I kind of know how to work around it, but want to fully understand it.
Can someone help explain?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20171013/b6aa0e14/attachment.html>