[ovs-dev] [PATCH v2 0/2] Python JSON improvements

Ben Pfaff blp at ovn.org
Thu May 19 01:12:42 UTC 2016


On Tue, May 10, 2016 at 03:36:58PM -0500, Terry Wilson wrote:
> Adds support for using the in-tree C JSON parser in Python. It also
> uses the built-in Python JSON parser for serialization. In a test of
> parsing a 100Mb JSON file, this patch improves performance by 70x. This
> version corrects the code formatting issues with the previous version.
> 
> I'm having trouble figuring out the best way to ensure that the PY2/3
> C extensions are built locally for when tests are run. I think I just
> need to run $(PYTHON)/$(PYTHON3) setup.py build_ext --inplace, but I'm
> not exactly where to make the change. The parsing is slightly different
> between the pure-Python and wrapped versions, so theoretically I'd
> need tests to handle both cases. Anyone have any ideas?

Thanks for working on this.  It is overdue.

That's way beyond my knowledge of Python.

These patches apply cleanly but they cause test failures for me, see
below.  Maybe that is because of the issue you mentioned above?

Will we need changes to distro packaging to fully adopt this?

Thanks,

Ben.

## ------------------------ ##
## Summary of the failures. ##
## ------------------------ ##
Failed tests:
openvswitch 2.5.90 test suite test groups:

 NUM: FILE-NAME:LINE     TEST-GROUP-NAME
      KEYWORDS

 626: json.at:203        integers expressed as reals - Python3
      json positive python
 637: json.at:238        negative zero - Python2
      json positive python
 638: json.at:238        negative zero - Python3
      json positive python
 655: json.at:275        1e-9999 underflows to 0 - Python2
      json positive python
 656: json.at:275        1e-9999 underflows to 0 - Python3
      json positive python

## ---------------------- ##
## Detailed failed tests. ##
## ---------------------- ##

#                             -*- compilation -*-
626. json.at:203: testing integers expressed as reals - Python3 ...
../../tests/json.at:203: printf %s "[1.0000000000,
     2.00000000000000000000000000000000000,
     2e5,
     2.1234e4,
     2.1230e3,
     0e-10000,
     0e10000]" > input
../../tests/json.at:203: $PYTHON3 $srcdir/test-json.py  input
stdout:
[1,2,200000,21234,2123.0,0,0]
../../tests/json.at:203: cat stdout
--- -	2016-05-18 18:09:29.276991908 -0700
+++ /home/blp/nicira/ovs/_build/tests/testsuite.dir/at-groups/626/stdout	2016-05-18 18:09:29.272930340 -0700
@@ -1,2 +1,2 @@
-[1,2,200000,21234,2123,0,0]
+[1,2,200000,21234,2123.0,0,0]
 
input:
> [1.0000000000,
>      2.00000000000000000000000000000000000,
>      2e5,
>      2.1234e4,
>      2.1230e3,
>      0e-10000,
>      0e10000]626. json.at:203: 626. integers expressed as reals - Python3 (json.at:203): FAILED (json.at:203)

#                             -*- compilation -*-
637. json.at:238: testing negative zero - Python2 ...
../../tests/json.at:238: printf %s "[-0, -0.0, 1e-9999, -1e-9999]" > input
../../tests/json.at:238: $PYTHON $srcdir/test-json.py  input
stdout:
[0,0,0,0]
../../tests/json.at:238: cat stdout
--- -	2016-05-18 18:09:29.304775538 -0700
+++ /home/blp/nicira/ovs/_build/tests/testsuite.dir/at-groups/637/stdout	2016-05-18 18:09:29.300930341 -0700
@@ -1,2 +1,2 @@
-[0,0,0.0,0.0]
+[0,0,0,0]
 
input:
> [-0, -0.0, 1e-9999, -1e-9999]637. json.at:238: 637. negative zero - Python2 (json.at:238): FAILED (json.at:238)

#                             -*- compilation -*-
638. json.at:238: testing negative zero - Python3 ...
../../tests/json.at:238: printf %s "[-0, -0.0, 1e-9999, -1e-9999]" > input
../../tests/json.at:238: $PYTHON3 $srcdir/test-json.py  input
stdout:
[0,0,0,0]
../../tests/json.at:238: cat stdout
--- -	2016-05-18 18:09:29.351456576 -0700
+++ /home/blp/nicira/ovs/_build/tests/testsuite.dir/at-groups/638/stdout	2016-05-18 18:09:29.348930343 -0700
@@ -1,2 +1,2 @@
-[0,0,0.0,0.0]
+[0,0,0,0]
 
input:
> [-0, -0.0, 1e-9999, -1e-9999]638. json.at:238: 638. negative zero - Python3 (json.at:238): FAILED (json.at:238)

#                             -*- compilation -*-
655. json.at:275: testing 1e-9999 underflows to 0 - Python2 ...
../../tests/json.at:275: printf %s "[1e-9999]" > input
../../tests/json.at:275: $PYTHON $srcdir/test-json.py  input
stdout:
[0]
../../tests/json.at:275: cat stdout
--- -	2016-05-18 18:09:29.379777599 -0700
+++ /home/blp/nicira/ovs/_build/tests/testsuite.dir/at-groups/655/stdout	2016-05-18 18:09:29.376930344 -0700
@@ -1,2 +1,2 @@
-[0.0]
+[0]
 
input:
> [1e-9999]655. json.at:275: 655. 1e-9999 underflows to 0 - Python2 (json.at:275): FAILED (json.at:275)

#                             -*- compilation -*-
656. json.at:275: testing 1e-9999 underflows to 0 - Python3 ...
../../tests/json.at:275: printf %s "[1e-9999]" > input
../../tests/json.at:275: $PYTHON3 $srcdir/test-json.py  input
stdout:
[0]
../../tests/json.at:275: cat stdout
--- -	2016-05-18 18:09:29.426757764 -0700
+++ /home/blp/nicira/ovs/_build/tests/testsuite.dir/at-groups/656/stdout	2016-05-18 18:09:29.424930346 -0700
@@ -1,2 +1,2 @@
-[0.0]
+[0]
 
input:
> [1e-9999]656. json.at:275: 656. 1e-9999 underflows to 0 - Python3 (json.at:275): FAILED (json.at:275)



More information about the dev mailing list