[ovs-discuss] Communicate flow-mod info to kernel module

Madhu Challa challa at noironetworks.com
Tue May 27 00:03:51 UTC 2014


Jesse,

I ran an iperf test (4 client server pairs) with and without the stats and
there was no noticeable difference in the measured bandwidth. However as I
increase the number of connections from 1 to 4 I do see the spin lock
overhead for this call stack (as a percent of total spin lock overhead on
the system) go up from 6% to 16%. I have 2 numa nodes on my system and 32
cores. But yes it is not affecting bandwidth.

         - 16.49% ovs_flow_stats_update

              ovs_dp_process_packet_with_key
              ovs_dp_process_received_packet
              ovs_vport_receive
              netdev_frame_hook
              __netif_receive_skb_core
              __netif_receive_skb

recirc_id(0),skb_priority(0),in_port(2),eth(src=90:e2:ba:59:1b:c4,dst=90:e2:ba:61:e4:54),eth_type(0x0800),ipv4(src=
12.1.1.2/0.0.0.0,dst=12.1.1.1/0.0.0.0,proto=6/0,tos=0/0,ttl=64/0,frag=no/0xff),
packets:0, bytes:0, used:never, actions:3
recirc_id(0),skb_priority(0),in_port(3),eth(src=90:e2:ba:61:e4:54,dst=90:e2:ba:59:1b:c4),eth_type(0x0800),ipv4(src=
12.1.1.1/0.0.0.0,dst=12.1.1.2/0.0.0.0,proto=6/0,tos=0/0,ttl=64/0,frag=no/0xff),
packets:0, bytes:0, used:never, actions:2

[  3]  0.0-300.0 sec  83.5 GBytes  2.39 Gbits/sec
[  3]  0.0-300.0 sec  80.1 GBytes  2.29 Gbits/sec
[  3]  0.0-300.0 sec  82.3 GBytes  2.36 Gbits/sec
[  3]  0.0-300.0 sec  80.3 GBytes  2.30 Gbits/sec

Thanks.


On Sun, May 25, 2014 at 10:34 AM, Jesse Gross <jesse at nicira.com> wrote:

> On Sat, May 24, 2014 at 6:06 PM, Madhu Challa <challa at noironetworks.com>
> wrote:
> > Hi Jesse,
> >
> > Thanks for the clarifications.
> >
> >
> > On Fri, May 23, 2014 at 9:52 AM, Jesse Gross <jesse at nicira.com> wrote:
> >>
> >> On Thu, May 22, 2014 at 5:02 PM, Madhu Challa <challa at noironetworks.com
> >
> >> wrote:
> >> > I am looking at adding support for On-demand flow counters and figured
> >> > one
> >> > of the benefits of doing this is reduced lock contention in
> >> > ovs_flow_stats_update.
> >> >
> >> > If my understanding is correct, this involves processing flags
> >> > OFPFF13_NO_PKT_COUNTS and OFPFF13_NO_BYT_COUNTS and communicating them
> >> > so
> >> > when the kernel flows are created / updated we somehow pass this info
> >> > along
> >> > so the stats do not get updated for such flows.
> >>
> >> It's not clear that this is really the case. OVS still needs to
> >> maintain stats in order to age out idle flows, etc. even if they
> >> aren't used by the controller.
> >
> >
> > I was making this assumption based on the comments in the TODO list.
> >
> > " * On-demand flow counters. I think this might be a real optimization in
> > some cases for the software switch.
> > [optional for OF1.3+]"
> >
> > I would guess the last used field is the one needed to age out the flows,
> > but if we also have to increment the counters we would need the lock and
> > there would not be much savings I can think of.
>
> My guess is that even just maintaining the last used field would be
> about the same. One other data point is that moving from per-CPU to
> per-NUMA stats didn't appear to have much effect on performance even
> for flows on different cores in the same socket, which would indicate
> that maybe there isn't much overhead here after all.
>
> It would be fairly easy to test out the potential performance gain by
> simply not updating the stats in the kernel at all and see what
> happens. I would try that before doing the work to plumb everything
> through.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20140526/6a2734b6/attachment-0002.html>


More information about the discuss mailing list