[ovs-dev] [PATCHv2] netdev-afxdp: Add interrupt mode netdev class.

Ilya Maximets i.maximets at ovn.org
Fri Feb 28 08:11:57 UTC 2020


On 2/27/20 8:52 PM, William Tu wrote:
> On Thu, Feb 27, 2020 at 11:10 AM William Tu <u9012063 at gmail.com> wrote:
>>
>> On Thu, Feb 27, 2020 at 9:54 AM Ilya Maximets <i.maximets at ovn.org> wrote:
>>>
>>> On 2/27/20 6:51 PM, William Tu wrote:
>>>> On Thu, Feb 27, 2020 at 9:42 AM Ilya Maximets <i.maximets at ovn.org> wrote:
>>>>>
>>>>> On 2/27/20 6:30 PM, William Tu wrote:
>>>>>> The patch adds a new netdev class 'afxdp-nonpmd' to enable afxdp
>>>>>> interrupt mode. This is similar to 'type=afxdp', except that the
>>>>>> is_pmd field is set to false. As a result, the packet processing
>>>>>> is handled by main thread, not pmd thread. This avoids burning
>>>>>> the CPU to always 100% when there is no traffic.
>>>>>>
>>>>>> Tested-at: https://travis-ci.org/williamtu/ovs-travis/builds/655885506
>>>>>> Signed-off-by: William Tu <u9012063 at gmail.com>
>>>>>> ---
>>>>>>  NEWS                  |  3 +++
>>>>>>  lib/netdev-afxdp.c    | 20 +++++++++++++++++++-
>>>>>>  lib/netdev-linux.c    | 20 ++++++++++++++++++++
>>>>>>  lib/netdev-provider.h |  1 +
>>>>>>  lib/netdev.c          |  1 +
>>>>>>  tests/system-afxdp.at | 23 +++++++++++++++++++++++
>>>>>>  6 files changed, 67 insertions(+), 1 deletion(-)
>>>>>>
>>>>>> diff --git a/NEWS b/NEWS
>>>>>> index f62ef1f47ea8..594c55dc11d6 100644
>>>>>> --- a/NEWS
>>>>>> +++ b/NEWS
>>>>>> @@ -4,6 +4,9 @@ Post-v2.13.0
>>>>>>     - OpenFlow:
>>>>>>       * The OpenFlow ofp_desc/serial_num may now be configured by setting the
>>>>>>         value of other-config:dp-sn in the Bridge table.
>>>>>> +   - AF_XDP:
>>>>>> +     * New netdev class 'afxdp-nonpmd' for netdev-afxdp to save CPU cycles
>>>>>> +       by enabling interrupt mode.
>>>>>>
>>>>>>
>>>>>>  v2.13.0 - 14 Feb 2020
>>>>>> diff --git a/lib/netdev-afxdp.c b/lib/netdev-afxdp.c
>>>>>> index 482400d8d135..cd2c7c381139 100644
>>>>>> --- a/lib/netdev-afxdp.c
>>>>>> +++ b/lib/netdev-afxdp.c
>>>>>> @@ -169,6 +169,12 @@ struct netdev_afxdp_tx_lock {
>>>>>>      );
>>>>>>  };
>>>>>>
>>>>>> +static int nonpmd_cnt;  /* Number of afxdp netdevs in non-pmd mode. */
>>>>>> +static bool
>>>>>> +netdev_is_afxdp_nonpmd(struct netdev *netdev) {
>>>>>> +    return netdev_get_class(netdev) == &netdev_afxdp_nonpmd_class;
>>>>>> +}
>>>>>> +
>>>>>>  #ifdef HAVE_XDP_NEED_WAKEUP
>>>>>>  static inline void
>>>>>>  xsk_rx_wakeup_if_needed(struct xsk_umem_info *umem,
>>>>>> @@ -1115,7 +1121,10 @@ netdev_afxdp_batch_send(struct netdev *netdev, int qid,
>>>>>>      struct netdev_linux *dev;
>>>>>>      int ret;
>>>>>>
>>>>>> -    if (concurrent_txq) {
>>>>>> +    /* Lock is required when mixing afxdp pmd and nonpmd mode.
>>>>>> +     * ex: one device is created 'afxdp', the other is 'afxdp-nonpmd'.
>>>>>> +     */
>>>>>> +    if (concurrent_txq || (nonpmd_cnt != 0)) {
>>>>>
>>>>> I'm confused about this change.  Are you expecting same device being
>>>>> opened twice with different netdev type?  Database should not allow this.
>>>>
>>>> Not the same device.
>>>>
>>>> For example, dev1 opened as 'afxdp', and dev2 opened as 'afxdp-nonpmd'.
>>>> When dev2 receives a packet and send to dev1, the main thread used by dev2
>>>> is calling the send(), while it is possible that dev2's pmd thread is
>>>> also calling
>>>> send(). So need a lock here.
>>>
>>> But they will send to different tx queues.
>>
>> OK,  I think you are talking about the XPS feature (dynamic txqs).
>> I thought XPS can only work when all between pmd threads.
>> So adding a lock here.
>> The patch currently doesn't work without the lock. Let me investigate more.
>>
> More details.
> On my test using veth (only have 1 rxq 1 txq)
> Somehow the main thread is assigned to use queue id  = 2, which causes segfault.

That is very strange...
Could you, please, share your test setup?  I'll try to reproduce.

Best regards, Ilya Maximets.


More information about the dev mailing list