[ovs-dev] [ovs-dev, v3] lib/ovs-thread: Add Transactional Memory (TM) support.

[William Tu]

Another concern is that once the CPUs are more common with TSX, then
systems with newer version of glibc will have lock elision enabled, which
means from OVS's point of view, we might not need to patch anything because
pthread lock/unlock API will have this feature.

We might need this patch when
1) Legacy glibc is used on CPU with TSX support
2) We find the implementation of lock elision in glibc isn't optimized
enough for OVS's use cases. Currently glibc implements an adaptive retry
approach and might not aggressively use TSX enough.

Regards,
William


On Mon, May 9, 2016 at 8:17 AM, Ben Pfaff <blp at ovn.org> wrote:

> On Mon, May 09, 2016 at 08:02:07AM -0500, Ryan Moats wrote:
> >
> > "dev" <dev-bounces at openvswitch.org> wrote on 05/07/2016 04:05:06 PM:
> >
> > > From: William Tu <u9012063 at gmail.com>
> > > X-Patchwork-Id: 587107
> > > Message-Id: <1456263597-40004-1-git-send-email-u9012063 at gmail.com>
> > > To: dev at openvswitch.org
> > > Date: Tue, 23 Feb 2016 13:39:57 -0800
> > >
> > > v2->v3
> > > - Pass all test cases
> > > - Introduce separate lock elision for rwlock
> > > - Add performance comparison of cmap/hmap search
> > >
> > > v1->v2
> > > - Fix a bug at UNLOCK_ELISION
> > > - Add checking if glibc version >= 2.21 (OVS_CHECK_GLIBC_TSX)
> > > - Add checking of whether cpu has TSX support (OVS_CHECK_RTM)
> > > - Enable LOCK_ELISION only when CPU has TSX and glibc doesn't
> > >   (if glibc version >= 2.21, then using phtread_mutex has lock elision)
> > >   - Add 20% mutation test-cmap testcase
> > >   - List failed testcases below
> > >
> > > The patch shows the preliminary results of enabling RTM
> > > (Restricted Transactional Memory). A successful transactional execution
> > > elides the lock, i.e., the lock is by-passed, and exposes concurrency.
> > > However, transactions might abort due to several reasons
> > > such as data conflicts, I/O operations, syscall, etc. When transaction
> > aborts,
> > > it falls back to the original locking mechanisms. Thus, the performance
> > > improvement depends on the abort rate, and the overhead of speculative
> > execution
> > > and rollback.
> > >
> > > The patch adds ovs_##ELISION_FUNC at LOCK_FUNCTION, TRY_LOCK_FUNCTION,
> > and
> > > UNLOCK_FUNCTION macros, and calls either rwlock_elision or
> mutex_elision
> > > accordingly. Experiments show that for cmap, the TM does not seem to
> have
> > > observable improvements below 5% mutations, while hmap is more obvious.
> > > For cmap search over 5% mutation, the search time of TM shows much
> better
> > > scalability when % of writers increase.
> > >
> > > Results are shown by using test-cmap benchmark with different mutation
> %.
> > > $ tests/ovstest test-cmap benchmark 20000000 4 <mutation> 1
> > >
> > > Unit: ms, number presented as baseline / TM
> > > <mutation>  cmap_search     hmap_search
> > >  0.1%           117/117        312/292
> > >    2%           124/120        688/328
> > >    5%           142/125       1093/403
> > >   10%           237/132       1588/432
> > >   20%           512/160       2516/679
> > >   40%          1010/277       2927/1007
> > >   60%          1278/324       3313/1119
> > >   80%          1614/343       3392/1291
> > >
> > > Signed-off-by: William Tu <u9012063 at gmail.com>
> >
> > In digging back through the patch queue, I came across this and I'm
> > wondering
> > what its status is.  The patch no longer applies cleanly and patch
> > complains
> > that the meat of the changes to lib/ovs-thread.c have already been
> > applied...
> >
> > If this is still a valid patch, can we get a clean update submitted?
>
> I've been dithering on reviewing this one properly.  It does present
> some useful performance improvement, but it also requires new glibc and
> CPUs that are difficult to find in the wild (at least in server
> processor; my laptop does have TSX!).  Tentatively, I was thinking about
> asking William to defer it for now, and then come back in a while after
> the CPUs are more common.
>