[ovs-discuss] Packets not forwarded to queues after hitting flows in OVS QoS

[Ben Pfaff]

On Wed, Dec 07, 2016 at 12:20:24PM +0100, Santhosh R P wrote:
> Hi,
> 
> The physical port (enp0s25) of the host and the tap devices (vnet0 and
> vnet1) of two hosted VMs are added to an OVS bridge named "test".
> 
>    ovs-vsctl show
>        Bridge test
>            Port test
>                Interface test
>                    type: internal
>            Port "vnet0"
>                Interface "vnet0"
>            Port "enp0s25"
>                Interface "enp0s25"
>            Port "vnet1"
>                Interface "vnet1"
>        ovs_version: "2.5.1"
> 
> I want to apply traffic shaping for the traffic leaving from these two VMs
> to an external machine over the internet. So, I created qos for enp0s25
> with two queues using,
> 
> ovs-vsctl  set port enp0s25 qos=@newqos -- --id=@newqos create qos
> type=linux-htb queues:1=@vnet0queue queues:2=@vnet1queue --
> --id=@vnet0queue create queue other-config:max-rate=100000000 --
> --id=@vnet1queue create queue other-config:max-rate=200000000
> 
> 
> QoS and queues are created as expected (both in OVS and TC).
> 
> 
> ovs-vsctl list qos
> 
>    _uuid               : 9e8b295d-13cf-4c18-9df6-f75809bb1184
> 
>    external_ids        : {}
> 
>    other_config        : {}
> 
>    queues              : {1=c9f1e852-9b29-4736-8b83-8ce1562be857,
> 2=3c61b22c-87a4-4380-96b5-822000d2bc94}
> 
>    type                : linux-htb
> 
> 
> ovs-ofctl -O OpenFlow13 queue-stats test
> 
>    OFPST_QUEUE reply (OF1.3) (xid=0x2): 3 queues
> 
>    port 1 queue 0: bytes=369596, pkts=1119, errors=0, duration=4294961079.
> 3744967296s
> 
>    port 1 queue 1: bytes=108, pkts=2, errors=0, duration=4294961079.
> 3744967296s
> 
>    port 1 queue 2: bytes=108, pkts=2, errors=0, duration=4294961079.
> 3744967296s
> 
> Then, as shown in the FAQ, I created flows to direct the traffic from VMs
> to these queues.
> 
> 
> ovs-ofctl add-flow test priority=2000,in_port=2,actions=set_queue:1,normal
> ovs-ofctl add-flow test priority=2000,in_port=3,actions=set_queue:2,normal
> 
> ovs-ofctl add-flow test priority=10,actions=normal
> 
> 
> The ofport of vnet0 and vnet1 are verified to 2 and 3 respectively
> (using: ovs-vsctl
> -- --columns=name,ofport list Interface) and the flows are configured
> correctly,
> 
> When I use iPerf3 from VM1 to an external IP, I could see that packet count
> is increasing in Port 2 (using: ovs-ofctl dump-ports test).
> 
> 
> The flows before the iPerf test looks like this,
> 
> 
> ovs-ofctl dump-flows test
> 
>     NXST_FLOW reply (xid=0x4):
> 
>     cookie=0x0, duration=7.439s, table=0, n_packets=0, n_bytes=0,
> idle_age=7, priority=2000,in_port=2 actions=set_queue:1,NORMAL
> 
>     cookie=0x0, duration=7.433s, table=0, n_packets=0, n_bytes=0,
> idle_age=7, priority=2000,in_port=3 actions=set_queue:2,NORMAL
> 
>     cookie=0x0, duration=6.263s, table=0, n_packets=55, n_bytes=17232,
> idle_age=0, priority=10 actions=NORMAL
> 
> 
> And after the iPerf test, looks like this (with packet counts increased in
> flow 1 and 3),
> 
> 
> ovs-ofctl dump-flows test
> 
>     NXST_FLOW reply (xid=0x4):
> 
>     cookie=0x0, duration=628.046s, table=0, n_packets=26200,
> n_bytes=1104961741, idle_age=2, priority=2000,in_port=2
> actions=set_queue:1,NORMAL
> 
>     cookie=0x0, duration=628.040s, table=0, n_packets=11, n_bytes=594,
> idle_age=14, priority=2000,in_port=3 actions=set_queue:2,NORMAL
> 
>     cookie=0x0, duration=626.870s, table=0, n_packets=203437,
> n_bytes=1118596590, idle_age=0, priority=10 actions=NORMAL (packet count
> here increased along with first flow in same magnitude)
> 
> But, the packets are not reaching the corresponding queues (verified in
> both OVS and TC), and no traffic shaping takes place. The default queue 0
> gets filled instead.
> 
> 
> On the other hand, the flows are hit, queues are filled and traffic shaping
> takes place if I use the dest IP address of iperf test as filter in the
> flow like this,
> 
> 
> ovs-ofctl add-flow test priority=2000,ip,nw_dst=134.60.64.159,
> actions=set_queue:1,normal
> ovs-ofctl add-flow test priority=2000,in_port=3,actions=set_queue:2,normal
> 
> ovs-ofctl add-flow test priority=10,actions=normal
> 
> 
> Using the src IP of VM1 or the port for VM1, hits the flows, but packets
> are not forwarded to the queue.
> 
> If this has something to do with, I used IP Masquerading in the host with
> private IPs for both VMs.
> 
> In ovs-vswitchd.log, there is no information other than added and deleted
> flows.
> 
> 
> What am I doing wrong here? Would be grateful to anyone who can help.

What's showing up in the kernel flows as displayed by "ovs-dpctl
dump-flows" while the test is running?