[ovs-dev] [PATCH v3 0/4] dpcls subtable miniflow optimizations

[Ilya Maximets]

On 21.11.2018 4:39, Harry van Haaren wrote:
> Hi Folks,
> 
> This patchset is a v3, changes from v2 are only a build
> fix, based on a bleep-bloop report :)
> 
> The work contained in this patchset achieves the following;
> 
> Patch 1:
> Refactor dpcls_lookup and the subtable for flexibility.
> In particular, add a function pointer to the subtable
> structure, which enables "plugging-in" a lookup function
> at runtime. This enables a number of optimizations in future.
> 
> Patch 2 & 3:
> With the function pointer in place, we refactor the existing
> dpcls_lookup matching code into its own function, and later its
> own file. To split it to its own file requires making various
> dpcls data-structures available in the dpif-netdev.h header.
> 
> Patch 4:
> Re-implement and optimize dpcls_rule_matches_key() by removing
> the "loopy-branch-ness" of the FOR_EACH() macros used. Instead
> a popcount() approach is used, which is much more CPU performance
> friendly, due to reduced branches/loads-stores and total work done.
> 
> Performance:
> Patches 1, 2 and 3 are performance neutral in testing here. The
> fourth patch provides a significant performance improvement when
> dpcls or SMC are processing packets.
> 
> Feedback, reviews, performance numbers weclomed! -Harry

Hi Harry,
Thanks for your work on this.
I didn't look at the code yet. Just wanted to make some preliminary tests.
By the first impression this patch set improves my usual scenario by 5-8%.
That is good result. But I observed the issue when I tried to enable SMC.

My case is the VM(vhu0) --> balanced_tcp_bond(ens1f0, ens1f1).
Usually, If I'm sending 8 flows from the VM I have following flows installed:

flow-dump from pmd on cpu core: 2
recirc_id(0),in_port(vhu0),packet_type(ns=0,id=0),eth(src=fa:16:3e:24:30:dd,dst=a0:00:00:00:00:00),eth_type(0x0800),ipv4(frag=no), packets:12628146, bytes:6364585584, used:0.000s, actions:push_vlan(vid=2055,pcp=0),hash(l4(0)),recirc(0x1)
recirc_id(0x1),dp_hash(0xd1779987/0xff),in_port(vhu0),packet_type(ns=0,id=0),eth_type(0x8100),vlan(vid=2055,pcp=0),encap(eth_type(0x0800),ipv4(frag=no)), packets:1572193, bytes:798674044, used:0.000s, actions:ens1f0
recirc_id(0x1),dp_hash(0xe46d47f9/0xff),in_port(vhu0),packet_type(ns=0,id=0),eth_type(0x8100),vlan(vid=2055,pcp=0),encap(eth_type(0x0800),ipv4(frag=no)), packets:1574624, bytes:799908992, used:0.000s, actions:ens1f0
recirc_id(0x1),dp_hash(0x19c6ca61/0xff),in_port(vhu0),packet_type(ns=0,id=0),eth_type(0x8100),vlan(vid=2055,pcp=0),encap(eth_type(0x0800),ipv4(frag=no)), packets:1584822, bytes:805089576, used:0.000s, actions:ens1f1
recirc_id(0x1),dp_hash(0x4064fb27/0xff),in_port(vhu0),packet_type(ns=0,id=0),eth_type(0x8100),vlan(vid=2055,pcp=0),encap(eth_type(0x0800),ipv4(frag=no)), packets:1582437, bytes:803877996, used:0.000s, actions:ens1f1
recirc_id(0x1),dp_hash(0x3729ae26/0xff),in_port(vhu0),packet_type(ns=0,id=0),eth_type(0x8100),vlan(vid=2055,pcp=0),encap(eth_type(0x0800),ipv4(frag=no)), packets:1581506, bytes:803405048, used:0.000s, actions:ens1f1
recirc_id(0x1),dp_hash(0x441edc10/0xff),in_port(vhu0),packet_type(ns=0,id=0),eth_type(0x8100),vlan(vid=2055,pcp=0),encap(eth_type(0x0800),ipv4(frag=no)), packets:1575850, bytes:800531800, used:0.000s, actions:ens1f1
recirc_id(0x1),dp_hash(0xea7d83c2/0xff),in_port(vhu0),packet_type(ns=0,id=0),eth_type(0x8100),vlan(vid=2055,pcp=0),encap(eth_type(0x0800),ipv4(frag=no)), packets:1573263, bytes:799217604, used:0.000s, actions:ens1f0
recirc_id(0x1),dp_hash(0x7ea60089/0xff),in_port(vhu0),packet_type(ns=0,id=0),eth_type(0x8100),vlan(vid=2055,pcp=0),encap(eth_type(0x0800),ipv4(frag=no)), packets:1583581, bytes:804459148, used:0.000s, actions:ens1f0

But, from time to time, If I enable SMC with:
# ovs-vsctl set Open_vSwitch . other_config:smc-enable=true

all the packets starts to match with a single installed flow:

flow-dump from pmd on cpu core: 2
recirc_id(0x1),dp_hash(0xe46d47f9/0xff),in_port(vhu0),packet_type(ns=0,id=0),eth_type(0x8100),vlan(vid=2055,pcp=0),encap(eth_type(0x0800),ipv4(frag=no)), packets:215076660, bytes:108510095624, used:0.000s, actions:ens1f0

All other flows expired.

And it's definitely not right thing to happen. i.e. all packets starts matching
with a single rule ignoring the recirculation id, vlan tag and the dp_hash field.
That leads to sending them directly to one of the physical ports without pushing
vlan tag and without recirculation.
And I actually see these packets on the physical network.

When the SMC disabled back, traffic becomes normal again.

Best regards, Ilya Maximets.

> 
> Harry van Haaren (4):
>   dpif-netdev: implement function pointers/subtable
>   dpif-netdev: move dpcls lookup structures to .h
>   dpif-netdev: split out generic lookup function
>   dpif-netdev: optimized dpcls_rule_matches_key()
> 
>  lib/automake.mk                  |   1 +
>  lib/dpif-netdev-lookup-generic.c |  95 +++++++++++++++++++++++
>  lib/dpif-netdev.c                | 124 +++----------------------------
>  lib/dpif-netdev.h                | 118 +++++++++++++++++++++++++++++
>  4 files changed, 226 insertions(+), 112 deletions(-)
>  create mode 100644 lib/dpif-netdev-lookup-generic.c
>