[ovs-dev] [PATCH] python: Allow specifying the iterations in Idl.run

[Ben Pfaff]

On Sun, Mar 20, 2016 at 03:29:06AM +0000, Jason Kölker wrote:
> Callers to Idl.run() should be able to specify the number of iterations
> to run in one call. Instrumentation revealed that a call to Idl.run() with a
> can block up to a few seconds, with most of the time being spent parsing
> messages.
> 
> Since parsing is a CPU only task eventlet and friends are unable to
> to provide concurrency. Specifying a configurable number of iterations
> will allow calling code to better manage concurrency.
> 
> Signed-off-by: Jason Kölker <jason at koelker.net>

A couple of seconds is really extreme.  I'm actually shocked to hear
that parsing 4 kB * 200 == 800 kB of data can take multiple seconds.  If
it takes that long, then I'd suggest an approach different from this
patch.  First, reduce the iteration count from 50 to something smaller.
Second, figure out why JSON parsing is so slow, and fix it.

stackoverflow points to some JSON push parsers:
http://stackoverflow.com/questions/444380/is-there-a-streaming-api-for-json