[ovs-dev] [PATCH 00/10] PMD Testsuite.

[Daniele Di Proietto]

I'd be happy to merge the pmd tests with their requirements.

I think we can convert a few tests (~10 or something) to be run with pmds
and non-pmd datapaths (using an approach similar to d13c05eef713, maybe).
I would still prefer avoiding adding 500 tests, even though we found one
bug with them.

I've skimmed through the series and I have a few preliminary comments:

* I think we should always register both 'dummy' and 'dummy_pmd' classes.
  We can imagine a test where the local port 'br0' is a dummy device,
  while the other interfaces are dummy_pmd.

* netdev_dummy_set_multiq can be used even by the non pmd dummy class
  (even though it's probably not that useful)

What do you think?

Thanks,

Daniele

On 17/03/2016 06:58, "Ilya Maximets" <i.maximets at samsung.com> wrote:

>Hi, Daniele.
>
>On 17.03.2016 03:53, Daniele Di Proietto wrote:
>> Hi Ilya,
>> 
>> thanks for the series, I think it's going to ease development of
>> dpif-netdev a lot.
>> 
>> I have been thinking about doing the same for a while and I've also
>> sketched out some code, which is very similar to this :-) (I've merged
>> the first patch).
>
>Thanks.
>
>> 
>> The interesting thing about this tests for me is that we get coverage
>> for the code that handles pmd threads and rxqs in dpif-netdev. I'm
>> talking about the tests in the last patch, to be clear.
>> 
>> I'm still not 100% convinced that we need to introduce another target
>> (check-pmd) and rerun most of the testsuite for these reasons:
>> 
>> * Most of the code and the branches stressed by 'make check-pmd' are
>>   already tested in 'make check'.
>
>I know that many test cases in new target are indifferent to port type,
>but it's a long and boring task to find out which of this ~500 tests
>are really useful. (Many of them are 100% useful. For example:
>'dpctl/flow-get' currently broken in master and fixed in this patch-set)
>Also, this requires a huge refactoring of testsuite to be able to run
>particular tests with different port types.
>
>>   To have coverage for the pmd specific
>>   conditions in dpif-netdev and ofproto-dpif we still need to write
>>   special testcases (e.g. imagine a testcase to cover the bug fixed by
>>   e4e74c3a2b9a)
>
>Yes. This is actually the real purpose of this series: To prepare
>environment for new PMD specific tests. I prepared only 4 tests here,
>but it's easy to write more with this patch set. And I hope we
>will extend 'tests/pmd.at' a lot in the future.
>
>> * The 'make check' test suite is run by developers for most of their
>>   changes.  If we add tests there we make sure that nobody will break
>>   the pmd code when changing something.
>
>That's would be great.
>
>> 
>> What do you think?  Is there a particular reason why you chose to
>> add another test suite?
>
>Actually no. Previously I thought that it will require serious refactoring
>of original testsuite, but now I realized that it's easy to put all
>tests together in 'make check'.
>
>So, I can add all this tests to 'make check'.
>
>What do you think? And would you like me to post v2 with all tests in
>one target 'make check'?
>
>Best regards, Ilya Maximets.
>
>> 
>> Thanks,
>> 
>> Daniele
>> 
>> On 14/03/2016 08:18, "Ilya Maximets" <i.maximets at samsung.com> wrote:
>> 
>>> New 'dummy_pmd' class created in purposes of testing of PMD interfaces.
>>> Added new target 'make check-pmd' and a bunch of PMD specific tests.
>>>
>>> Ilya Maximets (10):
>>>  netdev: New field 'is_pmd' in netdev_class.
>>>  dummy: Add dummy_pmd class.
>>>  dpctl: Implement dpctl/flow-get for dpif-netdev.
>>>  ofproto-dpif.at: Ignore port type while checking result of
>>>    dpctl/dump-flows.
>>>  ofproto-dpif.at: Allow output from all threads in 'patch ports -
>>>    stack'.
>>>  automake.mk: New target 'check-pmd'.
>>>  netdev-dummy: Queue packets only to one rx queue.
>>>  testsuite: Filter out unnecessary info about queues for dummy ports.
>>>  netdev-dummy: Add fake multiqueue support to dummy_pmd.
>>>  testsuite-pmd: Add PMD specific tests.
>>>
>>> NEWS                      |   1 +
>>> lib/dpctl.c               |   3 +-
>>> lib/dpif-netdev.c         |  49 ++++++++---
>>> lib/dummy.c               |  34 ++++---
>>> lib/dummy.h               |   8 +-
>>> lib/netdev-bsd.c          |   1 +
>>> lib/netdev-dpdk.c         |   1 +
>>> lib/netdev-dummy.c        | 220
>>> ++++++++++++++++++++++++++++------------------
>>> lib/netdev-linux.c        |   1 +
>>> lib/netdev-provider.h     |   3 +
>>> lib/netdev-vport.c        |  20 +++--
>>> lib/netdev-windows.c      |   1 +
>>> lib/netdev.c              |   5 +-
>>> tests/automake.mk         |  18 +++-
>>> tests/bridge.at           |   6 +-
>>> tests/completion.at       |   3 +-
>>> tests/dpctl.at            |   8 +-
>>> tests/dummy-macros.at     |   8 ++
>>> tests/dummy-pmd-macros.at |   8 ++
>>> tests/mpls-xlate.at       |   2 +-
>>> tests/ofproto-dpif.at     |  42 ++++-----
>>> tests/ofproto-macros.at   |  12 +--
>>> tests/ovs-ofctl.at        |   2 +-
>>> tests/ovs-vswitchd.at     |   2 +-
>>> tests/pmd.at              | 199
>>>+++++++++++++++++++++++++++++++++++++++++
>>> tests/testsuite-pmd.at    |  47 ++++++++++
>>> tests/testsuite.at        |   1 +
>>> tests/tunnel-push-pop.at  |   2 +-
>>> tests/tunnel.at           |  28 +++---
>>> 29 files changed, 552 insertions(+), 183 deletions(-)
>>> create mode 100644 tests/dummy-macros.at
>>> create mode 100644 tests/dummy-pmd-macros.at
>>> create mode 100644 tests/pmd.at
>>> create mode 100644 tests/testsuite-pmd.at
>>>
>>> -- 
>>> 2.5.0
>>>
>> 
>> 
>>