[ovs-dev] performance with GCC 4.1

[Ben Pfaff]

Hi Anoob.  Guru noticed (and I confirmed) that recent versions of Open
vSwitch on master suffer a little, performance-wise, on XenServer.  In
particular, the cmap code doesn't perform well.  In the OVS testsuite,
the cmap test takes about a minute to run on XenServer, versus about 2.5
seconds with other distributions.

I think that the difference is the GCC version.  The XenServer DDK has
GCC 4.1, I believe.  On that version of GCC, OVS has to fall back to a
less flexible set of atomic memory access primitives than available on
GCC 4.7 and later.  I think that GCC must be generating extra CPU
synchronization instructions that are not actually needed, making the
generated code very expensive.

One solution would be to use a newer GCC version on XenServer, but I'm
reluctant to do that because it would mean we'd have to install new
software into all the Xen DDK targets we use, which is a large number.
Another solution would be to open-code CPU synchronization instructions
where necessary in the atomic implementation for GCC pre-4.7.  That
would probably be easier, but it's a little tricky (in practice we'd
probably need a new atomic implementation for x86-on-GCC-pre4.7 and
keep the current one for other-CPUs-on-GCC-pre4.7).

Any thoughts?