[ovs-dev] [PATCH V13] Function tracer to trace all function calls

Ryan Moats rmoats at us.ibm.com
Sat Jul 23 12:56:22 UTC 2016


Ben Pfaff <blp at ovn.org> wrote on 07/22/2016 05:54:45 PM:

> From: Ben Pfaff <blp at ovn.org>
> To: Ryan Moats/Omaha/IBM at IBMUS
> Cc: Nirapada Ghosh/San Jose/IBM at IBMUS, dev at openvswitch.org
> Date: 07/22/2016 05:54 PM
> Subject: Re: [ovs-dev] [PATCH V13] Function tracer to trace all function
calls
>
> On Tue, Jul 19, 2016 at 09:26:46PM -0500, Ryan Moats wrote:
> > "dev" <dev-bounces at openvswitch.org> wrote on 07/08/2016 07:04:06 PM:
> >
> > > From: Nirapada Ghosh/San Jose/IBM at IBMUS
> > > To: dev at openvswitch.org
> > > Date: 07/08/2016 07:04 PM
> > > Subject: [ovs-dev] [PATCH V13] Function tracer to trace all function
> > calls
> > > Sent by: "dev" <dev-bounces at openvswitch.org>
> > >
> > > From: Nirapada Ghosh <nghosh at us.ibm.com>
> > >
> > > In some circumstances, we might need to figure out where in
> > > code, the CPU time is being spent most, so as to pinpoint
> > > the bottleneck and thereby resolve it with proper changes.
> > > Using '-finstrument-functions' flag, that can be achieved, and
> > > this patch exactly does that.
> > >
> > > There is a python file [generate_ft_report.py] with the patch,
> > > that may be used to convert this trace output to a human readable
> > > format with symbol names instead of address and their execution
> > > times. This tool uses addr2line that expects the executable to
> > > be built with -g flag.
> > >
> > > To enable this feature, ovs needs needs to be configured with
> > > "--enable-ft" command line argument [i.e. configure --enable-ft]
> > >
> > > This instrumentation logs the tracing output in separate log files
> > > namely func_trace_<pid>.log. It does not use VLOG mechanism for
> > > logging as that will make the patch very complicated to avoid
> > > recursion in the trace routine.
> > >
> > > This feature starts dumping output, only in debug mode, which means
> > > ovs-appctl -t <module> vlog/set any:any:dbg should be used to enable
> > > this logging.
> > >
> > > Currently, only ovn-northd, ovn-controller, vswitchd are
instrumented.
> >
> > After trying this out in some scaling scenarios, it looks like
> > it could use some reworking - even when not actually logging
> > data, it's resulting in what look like significantly slower
> > performance times of the instrumented binaries.
> >
> > If you can provide some data showing that the degradation is not
> > significant, then I'll remove my objection...
>
> Ryan, with your permission, I'm going to leave it to you to do a first
> round of reviews here before I take a closer look.

Sure, I've tagged it at patchworks...




More information about the dev mailing list