[ovs-dev] Threaded userspace datapath

[Ben Pfaff]

On Fri, Aug 10, 2012 at 07:50:40PM +0200, Luigi Rizzo wrote:
> On Fri, Aug 10, 2012 at 10:17:04AM -0700, Ben Pfaff wrote:
> > On Fri, Aug 10, 2012 at 07:17:13PM +0200, Luigi Rizzo wrote:
> > > 1. rebuild the list of file descriptors from scratch at every iteration,
> > >    looking at all possible sources.
> > >    It would be a lot more efficient to keep a list of current fds,
> > >    and add/remove elements in response to specific events.
> > 
> > On Linux, we use epoll for this in the datapath interface.  We
> > maintain an epoll set of the file descriptors that the datapath needs
> > to poll, then add just the single epoll file descriptor to the main
> > poll loop.  That causes the main loop to wake up whenever any of the
> > datapath file descriptors needs attention.
> > 
> > FreeBSD has kqueue, can it do the same thing?
> 
> it can be done, although the netmap file descriptor is not (yet)
> kqueue/epoll aware so we cannot use this solution with netmap.
> 
> Just curious (also seeing your other email to Giuseppe), was epoll
> support added after 1.0 or we missed it ?

I think we added it in 1.4.  It's encapsulated in lib/dpif-linux.c so
unless you've been reading that code then you would have missed it.
(I wouldn't add epoll-specific code, without some alternative, outside
of a Linux-specific source file, since it would limit portability.)

> > > 2. all callbacks were checked regardless the fact that the file descriptor
> > >    was reported as ready or not;
> > 
> > epoll also allows us to avoid this for the datapath file descriptors.
> 
> for what matters, even poll() would let you do that, so it was
> really a programming mistake (and presumably an easy one to fix,
> together with processing multiple pkts per iteration)

It's not a programming mistake, it's an intentional design choice that
favors making it easy to write code that is obviously correct over
writing code that is as fast as possible.  (If there's a design that
does well on both axes, then obviously we should adopt that one, but I
don't know of one yet.)