[ovs-dev] [RFC] [PATCH] 0/1 Add multi-column index support for the Python IDL

twilson at redhat.com twilson at redhat.com
Mon Apr 9 20:03:02 UTC 2018


From: Terry Wilson <twilson at redhat.com>

This adds a Python version of the C IDL's multi-column indexes. I tried
to make this as non-invasive as possible by just replacing idl.table.rows
with a custom dict-like object that updates the index when adding or
removing rows, and modifying Row.__setattr__ to update indexes when an
Row has an indexed column modified.

Some additional changes had to be made since Row.__getattr__ will raise
an AssertionError if ._changes == None, which happens when delete() is
called on a row. Also, when a txn is disassembled after a commit,
deleted rows are added back to idl.table.rows with ._changes == None.
Mostly, I just had to reorder when ._changes was set to None.

All tests pass in-tree both before changing the simple3 table tests to
use indexes and after. Also, functional tests in the ovsdbapp project
pass when it is converted to use indexes.

The performance difference on adding 1000 ports via ovsdbapp (with
may_exist=True which forces a lookup by name) takes the time from around
18 seconds to 9 without using the C json extension, and from 10 seconds
to 5 with the extension.

The patch does use an external library, 'sortedcontainers', which is
packaged by Ubuntu and Fedora (but currently isn't packaged in RHEL). It
is a very mature, pure-Ptyhon library that is also licensed Apache 2.0.
I didn't want to write something from scratch for a PoC if something
good existed, and it did. :)

I'd like some comments on whether this is a good direction, whether
anything in the code looks dumb, if we are ok adding an external
dependency (I could always update the patch to work just without the
indexinng feature if sortedcontainers didn't exist, or licensing would
permit us to just take the parts we want in tree--but I'd hate to do
that). If so, I can add some more in-tree testing for indexes with
multiple columns, etc. and fill in some missing (though unused) dict
methods.

Terry Wilson (1):
  Add multi-column index support for the Python IDL

 python/automake.mk            |   1 +
 python/ovs/db/custom_index.py | 151 ++++++++++++++++++++++++++++++++++++++++++
 python/ovs/db/idl.py          |  55 ++++++++++++---
 tests/test-ovsdb.py           |   7 +-
 4 files changed, 199 insertions(+), 15 deletions(-)
 create mode 100644 python/ovs/db/custom_index.py

-- 
1.8.3.1



More information about the dev mailing list