[ovs-discuss] ovs-vswitchd memory consumption behavior

[Ben Pfaff]

This is taking more time than I have to invest at the moment, so I'll
plan to try again if the next valgrind run doesn't help enough.

On Tue, Mar 05, 2019 at 04:15:49PM -0800, Ben Pfaff wrote:
> OK, sorry I missed it earlier.
> 
> This explains why the dump-heap scripts don't work.  They assume glibc
> malloc.  I am now reading the MUSL source to learn about how to write an
> equivalent tool for MUSL malloc.
> 
> On Wed, Mar 06, 2019 at 12:13:32AM +0000, Fernando Casas Schössow wrote:
> > Indeed.
> > 
> > I mentioned this in an email from yesterday with more details about the build (configure options, patches applied).
> > 
> > I'm using OVS on Alpine Linux, which uses musl libc, rather than glibc.
> > 
> > From my previous email:
> > 
> > --------------------------------------------------------------------------quote-------------------------------------------------------------------------
> > Alpine Linux is currently using OVS 2.10.1 and the source for the package comes from: http://www.openvswitch.org/releases/openvswitch-2.10.1.tar.gz
> > 
> > BTW I mentioned I'm running OVS on Alpine Linux a few times but just want to give you more details about the build.
> > For what it worth Alpine uses musl libc. The configure command looks like this:
> > 
> >  ./configure --prefix=/usr \
> >   --sysconfdir=/etc \
> >   --mandir=/usr/share/man \
> >   --infodir=/usr/share/info \
> >   --localstatedir=/var \
> >   --enable-ndebug \
> >   --enable-libcapng \
> >   PYTHON=/usr/bin/python2
> > 
> > And besides the ifupdown scripts the only patch it applies seems to be required by musl libc:
> > 
> > musl-if_packet.patch
> > --- openvswitch-2.4.0/lib/netdev-linux.c 2015-08-20 00:33:42.960971996 +0000
> > +++ openvswitch-2.4.0/lib/netdev-linux.c.new 2015-08-22 18:16:10.741115156 +0000
> > @@ -37,10 +37,9 @@
> >  #include <sys/ioctl.h>
> >  #include <sys/socket.h>
> >  #include <sys/utsname.h>
> > -#include <netpacket/packet.h>
> >  #include <net/if.h>
> >  #include <net/if_arp.h>
> > -#include <net/if_packet.h>
> > +#include <linux/if_packet.h>
> >  #include <net/route.h>
> >  #include <netinet/in.h>
> >  #include <poll.h>
> > 
> > I'm mentioning all this in case it can be useful for the investigation since recently I reported a bug in QEMU that after a deep investigation with the devs seems to be mostly affecting Alpine because of the compiler optimizations it uses.
> > ------------------------------------------------------------------------end quote-----------------------------------------------------------------------
> > 
> > On mié, mar 6, 2019 at 1:08 AM, Ben Pfaff <blp at ovn.org> wrote:
> > From the core dump (that you sent me off-list), it looks like you are using MUSL libc, rather than glibc: blp at sigill:~/nicira/ovs/_build(127)$ ldd ./ovs-vswitchd linux-vdso.so.1 (0x00007fff877bd000) libssl.so.1.1 => /lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007f6bc4526000) libcrypto.so.1.1 => /lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007f6bc423e000) libcap-ng.so.0 => /lib/x86_64-linux-gnu/libcap-ng.so.0 (0x00007f6bc4236000) libc.musl-x86_64.so.1 => not found libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f6bc4231000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f6bc4210000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6bc404d000) /lib/ld-musl-x86_64.so.1 => /lib64/ld-linux-x86-64.so.2 (0x00007f6bc47fb000) blp at sigill:~/nicira/ovs/_build(0)$ Is that correct? On Tue, Mar 05, 2019 at 03:08:59PM -0800, Ben Pfaff wrote:
> > That is worth a shot. Thank you. On Tue, Mar 05, 2019 at 11:06:08PM +0000, Fernando Casas Schössow wrote: > Too bad. :( > > Should I start over but leave it running for let's say 5-7 days instead? > Maybe 48 hours was not enough. > > On mié, mar 6, 2019 at 12:03 AM, Ben Pfaff <blp at ovn.org<mailto:blp at ovn.org>> wrote: > On Tue, Mar 05, 2019 at 09:56:30PM +0000, Fernando Casas Schössow wrote: > Please find attached valgrind log. It was running for around 48 hours. Hopefully it was enough otherwise let me know and I can run it again for more time. > Thanks for the log. There are several suspicious warnings to look at in this log, and I will do that. But it does not point out any significant memory leaks--even if I consider all of the "possible leaks" it mentions as actual leaks (usually, they are not), they are much too small by orders of magnitudes to be the problem reported. > >
> > 
> >