[ovs-discuss] Netperf generates warnings

Farrukh Aftab Khan Farrukh.Khan at xflowresearch.com
Thu Dec 20 14:00:23 UTC 2012


I only saw this warning once. And I have failed to reproduce it since then.
So I won't be able to share the ovs-dpctl stats. If I ever see the warning
again, I will report the output of "ovs-dpctl show" then.

At the moment, my output is as follows:
system at br0:
        lookups: hit:1695107704 missed:1214613 lost:11312
        flows: 0
        port 0: br0 (internal)
        port 1: pci0
        port 2: xaui0

The lost count stays the same now. Hardly increases.

I am trying to run about 1000 parallel netperf sessions to generate some
high data rate traffic. The main idea is to benchmark OVS-1.7.1 in terms of
throughput when the flow table has a million configured flows.

Looking forward to seeing the new enhancements. Thank you for the timely
reply and all the help.

Regards.

-
Farrukh Aftab Khan
xFlow Research Inc.


On Thu, Dec 20, 2012 at 12:27 PM, Justin Pettit <jpettit at nicira.com> wrote:

> That one is more worrisome.  I believe that's an indication that packets
> are getting dropped on the way to ovs-vswitchd.  If you're using the
> software kernel datapath, do you see the "lost" count going up in
> "ovs-dpctl show"?  Are you only seeing this when you first kick on the
> netperf tests?  What tests are you running?
>
> Ben's working on some additional performance improvements on the userspace
> classifier, which should help.  I'm working on giving each port its own
> kernel-userspace queue (currently there's a statically defined 16), which
> should provide better isolation and less contention when there are many
> ports.
>
> --Justin
>
>
> On Dec 19, 2012, at 11:18 PM, Farrukh Aftab Khan <
> Farrukh.Khan at xflowresearch.com> wrote:
>
> > I was also getting the following warnings:
> >
> > Jan  1 00:31:07 (none) daemon.warn ovs-vswitchd:
> 00017|dpif|WARN|system at br0: recv failed (No buffer space available)
> >
> >
> >
> > I looked up some old posts from OVS discuss (
> http://www.mail-archive.com/discuss@openvswitch.org/msg03385.html), where
> Ben had provided a patch for increasing the buffer space to 1MB to resolve
> this issue. It is already 1 MB in OVS-1.7.1. So should I increase it
> further? Is this something to be concerned about? Any heads up would be
> appreciated.
> >
> > Regards.
> > -
> > Farrukh Aftab Khan
> >
> >
> > On Thu, Dec 20, 2012 at 12:08 PM, Farrukh Aftab Khan <
> Farrukh.Khan at xflowresearch.com> wrote:
> > Thanks for the explanation Justin.
> > I have lowered their log levels and I don't these warnings anymore.
> Thanks for the help.
> >
> > Regards.
> > -
> > Farrukh Aftab Khan
> > xFlow Research Inc.
> >
> >
> > On Wed, Dec 19, 2012 at 10:14 PM, Justin Pettit <jpettit at nicira.com>
> wrote:
> > OVS logs these when the CPU usage spikes.  That message stating the poll
> interval was over 29 times the weighted mean interval triggered the
> coverage counters below it.  It shouldn't print that again unless there's
> another sudden spike in CPU usage that would delay the time through the
> poll loop.  This was all put in to help diagnose performance issues from a
> long time ago.  We've talked about removing them or at least lowering their
> log level.  They're harmless, though.  If you really don't want to see them
> right now, you could lower the appropriate log levels.
> >
> > --Justin
> >
> >
> > On Dec 19, 2012, at 4:17 AM, Farrukh Aftab Khan <
> Farrukh.Khan at xflowresearch.com> wrote:
> >
> > > Hello,
> > >
> > > I am using OVS-1.7.1 on embedded Linux (kernel-2.6.32.27). I am
> generating traffic at high data rate using 'netperf' and I get the
> following warnings:
> > >
> > > 1970-01-01T00:18:54Z|00029|timeval|WARN|51 ms poll interval (0 ms
> user, 50 ms system) is over 29 times the weighted mean interval 2 ms (2424
> samples)
> > > 1970-01-01T00:18:54Z|00030|timeval|WARN|context switches: 0 voluntary,
> 1 involuntary
> > > 1970-01-01T00:18:54Z|00031|coverage|INFO|Event coverage (epoch
> 2424/entire run), hash=eb85f2f9:
> > > 1970-01-01T00:18:54Z|00032|coverage|INFO|ofproto_dpif_xlate
> 1 /      1259
> > > 1970-01-01T00:18:54Z|00033|coverage|INFO|hmap_expand
> 22 /     22399
> > > 1970-01-01T00:18:54Z|00034|coverage|INFO|netdev_get_stats
>  12 /      2616
> > > 1970-01-01T00:18:54Z|00035|coverage|INFO|poll_fd_wait
>  41 /     98932
> > > 1970-01-01T00:18:54Z|00036|coverage|INFO|util_xalloc
> 1157 /    899297
> > > 1970-01-01T00:18:54Z|00037|coverage|INFO|netlink_received
>  17 /      8725
> > > 1970-01-01T00:18:54Z|00038|coverage|INFO|netlink_sent
>  17 /      8529
> > > 1970-01-01T00:18:54Z|00039|coverage|INFO|bridge_reconfigure
> 0 /        13
> > > 1970-01-01T00:18:54Z|00040|coverage|INFO|ofproto_flush
>  0 /         4
> > > 1970-01-01T00:18:54Z|00041|coverage|INFO|ofproto_update_port
>  0 /        32
> > > 1970-01-01T00:18:54Z|00042|coverage|INFO|dpif_port_add
>  0 /         8
> > > 1970-01-01T00:18:54Z|00043|coverage|INFO|dpif_flow_flush
>  0 /         8
> > > 1970-01-01T00:18:54Z|00044|coverage|INFO|dpif_purge
> 0 /         4
> > > 1970-01-01T00:18:54Z|00045|coverage|INFO|flow_extract
> 0 /        79
> > > 1970-01-01T00:18:54Z|00046|coverage|INFO|hmap_pathological
>  0 /         1
> > > 1970-01-01T00:18:54Z|00047|coverage|INFO|mac_learning_learned
> 0 /        14
> > > 1970-01-01T00:18:54Z|00048|coverage|INFO|mac_learning_expired
> 0 /        12
> > > 1970-01-01T00:18:54Z|00049|coverage|INFO|poll_zero_timeout
>  0 /        32
> > > 1970-01-01T00:18:54Z|00050|coverage|INFO|pstream_open
> 0 /         9
> > > 1970-01-01T00:18:54Z|00051|coverage|INFO|stream_open
>  0 /         1
> > > 1970-01-01T00:18:54Z|00052|coverage|INFO|netdev_set_policing
>  0 /        66
> > > 1970-01-01T00:18:54Z|00053|coverage|INFO|netdev_get_ifindex
> 0 /        10
> > > 1970-01-01T00:18:54Z|00054|coverage|INFO|netdev_get_hwaddr
>  0 /         9
> > > 1970-01-01T00:18:54Z|00055|coverage|INFO|netdev_set_hwaddr
>  0 /         5
> > > 1970-01-01T00:18:54Z|00056|coverage|INFO|netdev_ethtool
> 0 /        31
> > > 1970-01-01T00:18:54Z|00057|coverage|INFO|nln_changed
>  0 /        64
> > > 1970-01-01T00:18:54Z|00058|coverage|INFO|52 events never hit
> > >
> > > These warnings are only generated when traffic hits the bridge at a
> high rate. Is it normal for OVS to show these warnings/information messages
> when dealing with high data rate traffic? Is there a way to hide or
> minimize them?
> > >
> > > Regards.
> > > -
> > > Farrukh Aftab Khan
> > >
> > > _______________________________________________
> > > discuss mailing list
> > > discuss at openvswitch.org
> > > http://openvswitch.org/mailman/listinfo/discuss
> >
> >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openvswitch.org/pipermail/ovs-discuss/attachments/20121220/4401bd69/attachment.html>


More information about the discuss mailing list