[ovs-dev] [dpdk patch 5/8] dpif-netdev: Create 'number of dpdk ifaces on cpu socket' pmd threads for each cpu socket.

Pravin Shelar pshelar at nicira.com
Tue Sep 2 16:14:09 UTC 2014


On Mon, Sep 1, 2014 at 11:56 AM, Alex Wang <alexw at nicira.com> wrote:
>> >  static struct dp_netdev_port *dp_netdev_lookup_port(const struct
>> > dp_netdev *dp,
>> > @@ -281,6 +281,15 @@ struct dp_netdev_actions
>> > *dp_netdev_flow_get_actions(
>> >      const struct dp_netdev_flow *);
>> >  static void dp_netdev_actions_free(struct dp_netdev_actions *);
>> >
>> > +/* Represents the PMD configuration on a cpu socket. */
>> > +struct pmd_socket {
>> > +    struct dp_netdev *dp;
>> > +    struct latch exit_latch;
>> > +    struct pmd_thread *pmd_threads;
>> > +    int socket_id;
>> > +    int n_pmd_threads;
>> > +};
>> > +
>> We should keep socket to core mapping in numa module rather than in
>> dpif-netdev.
>
>
>
> In my future patches (for per pmd cls/flowtable), i removed the pmd_socket.
> I'll just move the change forward...
>
>
>
>>
>> I am not sure why exit latch needs to be per socket, it is global
>> today, it should be ok for now, no?
>>
>
>
> I'll make the 'exit latch' per pmd thread, because, for this optimization:
> - when dpdk port is deleted, if it is the last dpdk port on the socket,
>    all pmd threads on the socket will be removed,
> - using global exit latch will cause all pmd threads removed,
>
>
pmd thread addition should be rare operation, so lets keep exit_latch
per netdev, we can optimize it later if required.

>
>>
>> >  static void *
>> >  pmd_thread_main(void *f_)
>> >  {
>> >      struct pmd_thread *f = f_;
>> > -    struct dp_netdev *dp = f->dp;
>> > +    struct dp_netdev *dp = f->socket->dp;
>> >      unsigned int lc = 0;
>> >      struct rxq_poll *poll_list;
>> > +    struct non_local_pmd_dev *dev_list;
>> >      unsigned int port_seq;
>> > -    int poll_cnt;
>> > +    int poll_cnt, dev_cnt;
>> >      int i;
>> >
>> >      poll_cnt = 0;
>> > +    dev_cnt = 0;
>> >      poll_list = NULL;
>> > +    dev_list = NULL;
>> >
>> > -    pmd_thread_setaffinity_cpu(f->id);
>> > +    pmd_thread_setaffinity_cpu(f->core_id);
>> >  reload:
>> >      poll_cnt = pmd_load_queues(f, &poll_list, poll_cnt);
>> > +    dev_cnt = pmd_get_non_local_pmd_dev(f, &dev_list, dev_cnt);
>> >      atomic_read(&f->change_seq, &port_seq);
>> >
>> >      for (;;) {
>> > @@ -1682,6 +1777,10 @@ reload:
>> >              dp_netdev_process_rxq_port(dp,  poll_list[i].port,
>> > poll_list[i].rx);
>> >          }
>> >
>> > +        for (i = 0; i < dev_cnt; i++) {
>> > +            netdev_dpdk_flush_non_local(dev_list[i].dev, f->core_id);
>> > +        }
>> > +
>>
>> In transmit function we can flush if this is remote queue. To optimize
>> remote queue check on every xmit, we can add remote flag to
>> dpdk-netdev queue.
>
>
>
> may i know more about the reason you want to flush for every remote tx pkt?
>
> - is it for packet order concern?
>

I wanted to avoid extra call for dpdk.
> i'm not sure how expensive it is to call the tx function?  but still think
> we
> should batch the remote tx here,
>
I am not proposing extra call, dpdk-send can flush queue if it is remote queue.



More information about the dev mailing list