[ovs-dev] [dpdk patch 5/8] dpif-netdev: Create 'number of dpdk ifaces on cpu socket' pmd threads for each cpu socket.

Alex Wang alexw at nicira.com
Mon Sep 1 18:56:15 UTC 2014


>
> >  static struct dp_netdev_port *dp_netdev_lookup_port(const struct
> dp_netdev *dp,
> > @@ -281,6 +281,15 @@ struct dp_netdev_actions
> *dp_netdev_flow_get_actions(
> >      const struct dp_netdev_flow *);
> >  static void dp_netdev_actions_free(struct dp_netdev_actions *);
> >
> > +/* Represents the PMD configuration on a cpu socket. */
> > +struct pmd_socket {
> > +    struct dp_netdev *dp;
> > +    struct latch exit_latch;
> > +    struct pmd_thread *pmd_threads;
> > +    int socket_id;
> > +    int n_pmd_threads;
> > +};
> > +
> We should keep socket to core mapping in numa module rather than in
> dpif-netdev.
>


In my future patches (for per pmd cls/flowtable), i removed the pmd_socket.
I'll just move the change forward...




> I am not sure why exit latch needs to be per socket, it is global
> today, it should be ok for now, no?
>
>

I'll make the 'exit latch' per pmd thread, because, for this optimization:
- when dpdk port is deleted, if it is the last dpdk port on the socket,
   all pmd threads on the socket will be removed,
- using global exit latch will cause all pmd threads removed,




> >  static void *
> >  pmd_thread_main(void *f_)
> >  {
> >      struct pmd_thread *f = f_;
> > -    struct dp_netdev *dp = f->dp;
> > +    struct dp_netdev *dp = f->socket->dp;
> >      unsigned int lc = 0;
> >      struct rxq_poll *poll_list;
> > +    struct non_local_pmd_dev *dev_list;
> >      unsigned int port_seq;
> > -    int poll_cnt;
> > +    int poll_cnt, dev_cnt;
> >      int i;
> >
> >      poll_cnt = 0;
> > +    dev_cnt = 0;
> >      poll_list = NULL;
> > +    dev_list = NULL;
> >
> > -    pmd_thread_setaffinity_cpu(f->id);
> > +    pmd_thread_setaffinity_cpu(f->core_id);
> >  reload:
> >      poll_cnt = pmd_load_queues(f, &poll_list, poll_cnt);
> > +    dev_cnt = pmd_get_non_local_pmd_dev(f, &dev_list, dev_cnt);
> >      atomic_read(&f->change_seq, &port_seq);
> >
> >      for (;;) {
> > @@ -1682,6 +1777,10 @@ reload:
> >              dp_netdev_process_rxq_port(dp,  poll_list[i].port,
> poll_list[i].rx);
> >          }
> >
> > +        for (i = 0; i < dev_cnt; i++) {
> > +            netdev_dpdk_flush_non_local(dev_list[i].dev, f->core_id);
> > +        }
> > +
>
> In transmit function we can flush if this is remote queue. To optimize
> remote queue check on every xmit, we can add remote flag to
> dpdk-netdev queue.
>


may i know more about the reason you want to flush for every remote tx pkt?

- is it for packet order concern?

i'm not sure how expensive it is to call the tx function?  but still think
we
should batch the remote tx here,



More information about the dev mailing list