[ovs-discuss] why does my packet VLAN ID get changed?

Ronald van der Pol Ronald.vanderPol at rvdp.org
Fri Aug 1 15:42:55 UTC 2014


Please help me debug this problem. I have an OVS with two ports.
A packet enters the bridge with VLAN 103 and leaves with VLAN 3.
I do not understand why. There seems to be no flow rule for this.
It should not be changed.

This is an OpenStack environment, but I'll try to show the
relevant parts. I think it is ovs related.

This is the bridge:

    Bridge br-of
        Port br-of
            Interface br-of
                type: internal
        Port "eth1"
            Interface "eth1"
        Port phy-br-of
            Interface phy-br-of
        Port "eth2"
            Interface "eth2"

rvdp at lithium:~$ sudo ovs-ofctl show br-of
OFPT_FEATURES_REPLY (xid=0x2): dpid:00000002c9ee01d0
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE
 1(eth2): addr:00:02:c9:ee:01:d1
     config:     0
     state:      0
     current:    COPPER
     advertised: 10GB-FD COPPER
     supported:  10GB-FD COPPER
     speed: 0 Mbps now, 10000 Mbps max
 5(phy-br-of): addr:52:92:c1:84:d9:99
     config:     0
     state:      0
     current:    10GB-FD COPPER
     speed: 10000 Mbps now, 0 Mbps max
 7(eth1): addr:00:02:c9:ee:01:d0
     config:     0
     state:      0
     current:    10GB-FD COPPER
     advertised: 10GB-FD COPPER
     supported:  10GB-FD COPPER
     speed: 10000 Mbps now, 10000 Mbps max
 LOCAL(br-of): addr:00:02:c9:ee:01:d0
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0
rvdp at lithium:~$ 


rvdp at lithium:~$ ifconfig eth1
eth1      Link encap:Ethernet  HWaddr 00:02:c9:ee:01:d0  
          inet6 addr: fe80::202:c9ff:feee:1d0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:23874872 errors:0 dropped:0 overruns:0 frame:0
          TX packets:15691781 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1864954722 (1.8 GB)  TX bytes:1223898214 (1.2 GB)

rvdp at lithium:~$ ifconfig eth1.103
eth1.103  Link encap:Ethernet  HWaddr 00:02:c9:ee:01:d0  
          inet addr:10.103.0.1  Bcast:10.103.0.255  Mask:255.255.255.0
          inet6 addr: fe80::202:c9ff:feee:1d0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4208 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:193568 (193.5 KB)  TX bytes:648 (648.0 B)


Packet enters from eth1 with VLAN 103:

--------------------------------------
rvdp at lithium:~$ tshark -V -n -i eth1 
Capturing on 'eth1'
Frame 1: 64 bytes on wire (512 bits), 64 bytes captured (512 bits) on interface 0
    Interface id: 0
    Encapsulation type: Ethernet (1)
    Arrival Time: Aug  1, 2014 17:09:14.102356000 CEST
    [Time shift for this packet: 0.000000000 seconds]
    Epoch Time: 1406905754.102356000 seconds
    [Time delta from previous captured frame: 0.000059000 seconds]
    [Time delta from previous displayed frame: 0.000000000 seconds]
    [Time since reference or first frame: 0.000059000 seconds]
    Frame Number: 2
    Frame Length: 64 bytes (512 bits)
    Capture Length: 64 bytes (512 bits)
    [Frame is marked: False]
    [Frame is ignored: False]
    [Protocols in frame: eth:vlan:arp]
Ethernet II, Src: fa:16:3e:e4:a0:7f (fa:16:3e:e4:a0:7f), Dst: ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff)
    Destination: ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff)
        Address: ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff)
        .... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
        .... ...1 .... .... .... .... = IG bit: Group address (multicast/broadcast)
    Source: fa:16:3e:e4:a0:7f (fa:16:3e:e4:a0:7f)
        Address: fa:16:3e:e4:a0:7f (fa:16:3e:e4:a0:7f)
        .... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Type: 802.1Q Virtual LAN (0x8100)
802.1Q Virtual LAN, PRI: 0, CFI: 0, ID: 103
    000. .... .... .... = Priority: Best Effort (default) (0)
    ...0 .... .... .... = CFI: Canonical (0)
    .... 0000 0110 0111 = ID: 103
    Type: ARP (0x0806)
    Padding: 0000000000000000000000000000
    Trailer: 1d019b7f
Address Resolution Protocol (request)
    Hardware type: Ethernet (1)
    Protocol type: IP (0x0800)
    Hardware size: 6
    Protocol size: 4
    Opcode: request (1)
    Sender MAC address: fa:16:3e:e4:a0:7f (fa:16:3e:e4:a0:7f)
    Sender IP address: 10.101.0.2 (10.101.0.2)
    Target MAC address: 00:00:00:00:00:00 (00:00:00:00:00:00)
    Target IP address: 10.101.0.5 (10.101.0.5)
--------------------------------------


But it leaves phy-br-of with VLAN 3:

--------------------------------------
rvdp at lithium:~$ tshark -V -c1  -n -i phy-br-of 
Capturing on 'phy-br-of'
Frame 1: 46 bytes on wire (368 bits), 46 bytes captured (368 bits) on interface 0
    Interface id: 0
    Encapsulation type: Ethernet (1)
    Arrival Time: Aug  1, 2014 17:11:34.100388000 CEST
    [Time shift for this packet: 0.000000000 seconds]
    Epoch Time: 1406905894.100388000 seconds
    [Time delta from previous captured frame: 0.000000000 seconds]
    [Time delta from previous displayed frame: 0.000000000 seconds]
    [Time since reference or first frame: 0.000000000 seconds]
    Frame Number: 1
    Frame Length: 46 bytes (368 bits)
    Capture Length: 46 bytes (368 bits)
    [Frame is marked: False]
    [Frame is ignored: False]
    [Protocols in frame: eth:vlan:arp]
Ethernet II, Src: fa:16:3e:e4:a0:7f (fa:16:3e:e4:a0:7f), Dst: ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff)
    Destination: ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff)
        Address: ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff)
        .... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
        .... ...1 .... .... .... .... = IG bit: Group address (multicast/broadcast)
    Source: fa:16:3e:e4:a0:7f (fa:16:3e:e4:a0:7f)
        Address: fa:16:3e:e4:a0:7f (fa:16:3e:e4:a0:7f)
        .... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Type: 802.1Q Virtual LAN (0x8100)
802.1Q Virtual LAN, PRI: 0, CFI: 0, ID: 3
    000. .... .... .... = Priority: Best Effort (default) (0)
    ...0 .... .... .... = CFI: Canonical (0)
    .... 0000 0000 0011 = ID: 3
    Type: ARP (0x0806)
Address Resolution Protocol (request)
    Hardware type: Ethernet (1)
    Protocol type: IP (0x0800)
    Hardware size: 6
    Protocol size: 4
    Opcode: request (1)
    Sender MAC address: fa:16:3e:e4:a0:7f (fa:16:3e:e4:a0:7f)
    Sender IP address: 10.101.0.2 (10.101.0.2)
    Target MAC address: 00:00:00:00:00:00 (00:00:00:00:00:00)
    Target IP address: 10.101.0.5 (10.101.0.5)

--------------------------------------

(the other side, int-br-of, shows the same)

These are the flow rules:

rvdp at lithium:~$ sudo ovs-ofctl dump-flows br-of
NXST_FLOW reply (xid=0x4):
 cookie=0x0, duration=19204.821s, table=0, n_packets=14388, n_bytes=604592, idle_age=1, priority=4,in_port=5,dl_vlan=3 actions=mod_vlan_vid:101,NORMAL
 cookie=0x0, duration=19205.755s, table=0, n_packets=6, n_bytes=268, idle_age=11353, priority=4,in_port=5,dl_vlan=2 actions=mod_vlan_vid:103,NORMAL
 cookie=0x0, duration=19210.085s, table=0, n_packets=236, n_bytes=15528, idle_age=162, priority=2,in_port=5 actions=drop
 cookie=0x0, duration=19210.713s, table=0, n_packets=1778506, n_bytes=160065556, idle_age=15259, priority=1 actions=NORMAL
rvdp at lithium:~$

--------------------------------------

Counters on the flow rule "in_port=5,dl_vlan=3" increase.

ofproto/trace gives:


rvdp at lithium:~$ sudo ovs-appctl ofproto/trace br-of in_port=7,dl_vlan=103,dl_src=fa:16:3e:e4:a0:7f,dl_dst=ff:ff:ff:ff:ff:ff,arp,arp_spa=10.101.0.2,arp_tpa=10.101.0.5,arp_sha=fa:16:3e:e4:a0:7f,arp_tha=00:00:00:00:00:00
Flow: arp,metadata=0,in_port=7,dl_vlan=103,dl_vlan_pcp=0,dl_src=fa:16:3e:e4:a0:7f,dl_dst=ff:ff:ff:ff:ff:ff,arp_spa=10.101.0.2,arp_tpa=10.101.0.5,arp_sha=fa:16:3e:e4:a0:7f,arp_tha=00:00:00:00:00:00
Rule: table=0 cookie=0 priority=1
OpenFlow actions=NORMAL
no learned MAC for destination, flooding

Final flow: unchanged
Relevant fields: skb_priority=0,arp,in_port=7,dl_vlan=103,dl_src=fa:16:3e:e4:a0:7f,dl_dst=ff:ff:ff:ff:ff:ff,nw_frag=no
Datapath actions: 4,6,8
rvdp at lithium:~$

This is what should happen. This is what I expect. But the tshark
shows that the vlan id has changed from 103 to 3.

Is in_port=7 the correct port? If I use in_port=5 the packet gets dropped.

rvdp at lithium:~$ ovs-dpctl show
system at ovs-system:
	lookups: hit:88297643 missed:237835 lost:5824
	flows: 23
	port 0: ovs-system (internal)
	port 1: br-p2p2 (internal)
	port 2: p2p2
	port 3: br-int (internal)
	port 4: br-of (internal)
	port 5: eth1
	port 6: eth2
	port 7: int-br-of
	port 8: phy-br-of
	port 9: int-br-p2p2
	port 10: phy-br-p2p2
	port 11: qvoee503507-d4
	port 12: qvoec91ac92-65
	port 13: qvof3111da3-1e
	port 14: qvoc571d286-66
	port 15: qvo90426bbe-76
	port 16: qvo9db2c337-d9
	port 17: qvoc8335d24-bb
	port 18: qvo41c98397-fd
rvdp at lithium:~$ 

So, what is going on? What else can I do to debug this?

	rvdp



More information about the discuss mailing list