[ovs-dev] OVS to set Virtual Function MAC address
Moshe Levi
moshele at mellanox.com
Mon Jun 24 10:41:28 UTC 2019
> -----Original Message-----
> From: ovs-dev-bounces at openvswitch.org <ovs-dev-
> bounces at openvswitch.org> On Behalf Of Ilya Maximets
> Sent: Monday, June 24, 2019 1:19 PM
> To: Hamdy Khader <hamdyk at mellanox.com>; Stokes, Ian
> <ian.stokes at intel.com>; dev at openvswitch.org
> Subject: Re: [ovs-dev] OVS to set Virtual Function MAC address
>
> On 23.06.2019 17:37, Hamdy Khader wrote:
> > We are working on adding the ability to create VM with DPDK
> > Representor port in OpenStack environment, that means as part of port
> > plugging flow, the port's MAC address will be generated on VM spawning,
> and need to be set to the VF.
> >
> > Usually we use python libraries like os-vif or pyroute2 to set the VF's MAC
> address for bifurcated drivers (e.g. Mellanox NIC) but we are facing a problem
> setting the MAC for non-bifurcated drivers (e.g. I40E).
>
> In usual SRIOV configurations with OpenStack Nova sets the VF mac into Libvirt
So we can move it to be done is os-vif also.
> XML and Libvirt administratively sets VF mac while VM launching.
> So, in usual cases, libvirt is responsible for setting VF mac.
> I understand your issue with DPDK controlled PF. Libvirt might not be able to
> set VF mac while PF is attached to DPDK. But, IIRC, libvirt will fail to pass VF to
> VM if it will not find the PF linux network device on host.
> So, you, probably, have much worse issues than just setting the MAC.
So that true if you do <interface type='hostdev' managed='yes'> in Libvirt. We can do pci passthrough without setting MAC (e.g <hostdev mode='subsystem' type='pci' managed='yes'>)
The problem with the first approach is that is will work just for beforecited drivers, and the problem with the second approach is that we don't have a common way to set the mac address for all drivers.
I was thing if we can extend the api to
ovs-vsctl add-port br0 dpdk-rep3 -- set Interface dpdk-rep3 type=dpdk \
options:dpdk-devargs=0000:08:00.0,representor=[3],mac=XX:XX:XX:XX:XX:XX and this will configure the VF mac.
>
> BTW, setting the MAC to dpdk devices could be implemented by something
> like this (not tested):
>
> diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index
> 4856c56aa..228d943be 100644
> --- a/lib/netdev-dpdk.c
> +++ b/lib/netdev-dpdk.c
> @@ -2507,11 +2507,24 @@ static int
> netdev_dpdk_set_etheraddr(struct netdev *netdev, const struct eth_addr
> mac) {
> struct netdev_dpdk *dev = netdev_dpdk_cast(netdev);
> + int err = 0;
>
> ovs_mutex_lock(&dev->mutex);
> if (!eth_addr_equals(dev->hwaddr, mac)) {
> - dev->hwaddr = mac;
> - netdev_change_seq_changed(netdev);
> + if (dev->type == DPDK_DEV_ETH) {
> + struct ether_addr ea;
> +
> + memcpy(ea.addr_bytes, mac.ea, ETH_ADDR_LEN);
> + err = rte_eth_dev_default_mac_addr_set(dev->port_id, &ea);
> + }
> + if (!err) {
> + dev->hwaddr = mac;
> + netdev_change_seq_changed(netdev);
> + } else {
> + VLOG_WARN("%s: Failed to set requested mac("ETH_ADDR_FMT"):
> %s",
> + netdev_get_name(netdev), ETH_ADDR_ARGS(mac),
> + rte_strerror(-err));
> + }
> }
> ovs_mutex_unlock(&dev->mutex);
>
> ---
>
> I'm not sure if it will work (if we able to change default MAC while port is
> started? Or we should request the reconfiguration for this?) and why we
> didn't set the mac to HW previously.
>
> Best regards, Ilya Maximets.
>
> >
> > OVS-DPDK supports both types of drivers for DPDK representor ports and I
> think OVS should set the MAC (if provided) as part of DPDK port addition.
> >
> > Below you can find DPDK representor port info, "attached-mac" is the
> desired value to be used as VF MAC.
> >
> > # ovs-vsctl list Interface vfrff9715cc-ee _uuid :
> > c4b5aa06-3a3c-4ff4-a5a0-eeac028ac2f4
> > admin_state : up
> > bfd : {}
> > bfd_status : {}
> > cfm_fault : []
> > cfm_fault_status : []
> > cfm_flap_count : []
> > cfm_health : []
> > cfm_mpid : []
> > cfm_remote_mpids : []
> > cfm_remote_opstate : []
> > duplex : full
> > error : []
> > external_ids : {attached-mac="fa:16:3e:65:3d:0c",
> > iface-id="ff9715cc-ee22-4f02-b57b-ac2a1707248a", iface-status=active,
> > vm-uuid="e9a2ac9d-663b-4bdb-907a-b50aa3a238a9"}
> > ifindex : 1277170
> > ingress_policing_burst: 0
> > ingress_policing_rate: 0
> > lacp_current : []
> > link_resets : 0
> > link_speed : 10000000000
> > link_state : up
> > lldp : {}
> > mac : []
> > mac_in_use : "a2:e4:6e:d6:ff:86"
> > mtu : 1500
> > mtu_request : []
> > name : "vfrff9715cc-ee"
> > ofport : 15
> > ofport_request : []
> > options : {dpdk-devargs="0000:02:00.1,representor=[15]"}
> > other_config : {}
> > statistics : {rx_bytes=8638, rx_dropped=0, rx_errors=0,
> > rx_mbuf_allocation_errors=0, rx_missed_errors=0, rx_packets=31,
> > tx_bytes=10010, tx_dropped=0, tx_errors=0, tx_packets=143}
> status
> > : {driver_name="net_mlx5", if_descr="DPDK 18.11.0 net_mlx5",
> > if_type="6", link_speed="10Gbps", max_hash_mac_addrs="0",
> > max_mac_addrs="128", max_rx_pktlen="1518", max_rx_queues="65535",
> > max_tx_queues="65535", max_vfs="0", max_vmdq_pools="0",
> > min_rx_bufsize="32", numa_id="0", pci-device_id="0x1015",
> > pci-vendor_id="0x15b3", port_no="1"} type : dpdk
> >
> > ----------------------------------------------------------------------
> > ----------------------------------------------------------------------
> > ----------------------------------------------------------------------
> > ----------------------------------------------------------------------
> > ----------------------------------------------------------------------
> > ----------------------------------------------------------------------
> > ----------------------------------------------------------------------
> > ----------------------------------------------------------------------
> > ----------------------------------------------------------------------
> > ----------------------------------------------------------------------
> > ----------------------------------------------------------------------
> > ----------------------------------------------------------------------
> > ----------------------------------------------------------------------
> > ----------------------------------------------------------------------
> > ----------
> > *From:* Stokes, Ian <ian.stokes at intel.com>
> > *Sent:* Sunday, June 23, 2019 4:06 PM
> > *To:* Hamdy Khader; dev at openvswitch.org
> > *Cc:* Ilya Maximets; Ophir Munk
> > *Subject:* RE: OVS to set Virtual Function MAC address
> >
> >> Hi,
> >>
> >> I'm working with DPDK representor ports to integrate it with
> >> OpenStack and I have to assign the MAC for the VF using "ip" command.
> >>
> >> Are there plans to make OVS to set the VFs MAC when we add dpdk
> >> representor port?
> >
> > Not that I'm aware of, I would think setting MACs etc. would be an
> administrator task during deployment carried out by something like IP (as you
> have done already). Is there a reason why you feel OVS should do this task?
> >
> > Regards
> > Ian
> _______________________________________________
> dev mailing list
> dev at openvswitch.org
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.op
> envswitch.org%2Fmailman%2Flistinfo%2Fovs-
> dev&data=02%7C01%7Cmoshele%40mellanox.com%7Ca07e000652064
> 8ad938408d6f88d72fa%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C
> 0%7C636969683735349267&sdata=l34YLaY66mTMIjKqiaXdPTzJRezAgH
> 1DoJX5WGOl33I%3D&reserved=0
More information about the dev
mailing list