[ovs-dev] OVS to set Virtual Function MAC address

Ilya Maximets i.maximets at samsung.com
Mon Jun 24 11:53:45 UTC 2019


On 24.06.2019 13:41, Moshe Levi wrote:
> 
> 
>> -----Original Message-----
>> From: ovs-dev-bounces at openvswitch.org <ovs-dev-
>> bounces at openvswitch.org> On Behalf Of Ilya Maximets
>> Sent: Monday, June 24, 2019 1:19 PM
>> To: Hamdy Khader <hamdyk at mellanox.com>; Stokes, Ian
>> <ian.stokes at intel.com>; dev at openvswitch.org
>> Subject: Re: [ovs-dev] OVS to set Virtual Function MAC address
>>
>> On 23.06.2019 17:37, Hamdy Khader wrote:
>>> We are working on adding the ability to create VM with DPDK
>>> Representor port in OpenStack environment, that means as part of port
>>> plugging flow, the port's MAC address will be generated on VM spawning,
>> and need to be set to the VF.
>>>
>>> Usually we use python libraries like os-vif or pyroute2 to set the VF's MAC
>> address for bifurcated drivers (e.g. Mellanox NIC) but we are facing a problem
>> setting the MAC for non-bifurcated drivers (e.g. I40E).
>>
>> In usual SRIOV configurations with OpenStack Nova sets the VF mac into Libvirt
> So we can move it to be done is os-vif also. 
>> XML and Libvirt administratively sets VF mac while VM launching.
>> So, in usual cases, libvirt is responsible for setting VF mac.
>> I understand your issue with DPDK controlled PF. Libvirt might not be able to
>> set VF mac while PF is attached to DPDK. But, IIRC, libvirt will fail to pass VF to
>> VM if it will not find the PF linux network device on host.
>> So, you, probably, have much worse issues than just setting the MAC.
> So that true if you do    <interface type='hostdev' managed='yes'> in Libvirt. We can do pci passthrough without setting MAC (e.g <hostdev mode='subsystem' type='pci' managed='yes'>) 
> The problem with the first approach is that is will work just for beforecited drivers, and the problem with the second  approach is that we don't have a common way to set the mac address for all drivers.

I think that the best way will be to extend the Libvirt's '<interface type='hostdev'>'
with port representors support. This will additionally allow libvirt to show
VF network stats. Something like this:

   <interface type='hostdev' managed='yes'>
     <source>
       <address type='pci' domain='0x0' bus='0x00' slot='0x07' function='0x0'/>
     </source>
     <mac address='52:54:00:6d:90:02'>
     <vf_representor type='openvswitch' name='dpdk-rep3'/>
   </interface>

<virtualport>, probably, could be used instead.

One more thing is that OVS doesn't support setting vlan tags to HW that is required
for OpenStack networking too. I guess, it'll be required to extend representor devargs
for this.


> I was thing if we can extend the api to 
> ovs-vsctl add-port br0 dpdk-rep3 -- set Interface dpdk-rep3 type=dpdk \
>    options:dpdk-devargs=0000:08:00.0,representor=[3],mac=XX:XX:XX:XX:XX:XX and this will configure the VF mac.

If you want to extend 'devargs' syntax, than you should ask DPDK community.
OVS doesn't control that.

The code snippet below should allow you to set mac address of any OVS device with:

    ovs-vsctl set Interface dpdk-rep3 mac=XX:XX:XX:XX:XX:XX


>>
>> BTW, setting the MAC to dpdk devices could be implemented by something
>> like this (not tested):
>>
>> diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index
>> 4856c56aa..228d943be 100644
>> --- a/lib/netdev-dpdk.c
>> +++ b/lib/netdev-dpdk.c
>> @@ -2507,11 +2507,24 @@ static int
>>  netdev_dpdk_set_etheraddr(struct netdev *netdev, const struct eth_addr
>> mac)  {
>>      struct netdev_dpdk *dev = netdev_dpdk_cast(netdev);
>> +    int err = 0;
>>
>>      ovs_mutex_lock(&dev->mutex);
>>      if (!eth_addr_equals(dev->hwaddr, mac)) {
>> -        dev->hwaddr = mac;
>> -        netdev_change_seq_changed(netdev);
>> +        if (dev->type == DPDK_DEV_ETH) {
>> +            struct ether_addr ea;
>> +
>> +            memcpy(ea.addr_bytes, mac.ea, ETH_ADDR_LEN);
>> +            err = rte_eth_dev_default_mac_addr_set(dev->port_id, &ea);
>> +        }
>> +        if (!err) {
>> +            dev->hwaddr = mac;
>> +            netdev_change_seq_changed(netdev);
>> +        } else {
>> +            VLOG_WARN("%s: Failed to set requested mac("ETH_ADDR_FMT"):
>> %s",
>> +                      netdev_get_name(netdev), ETH_ADDR_ARGS(mac),
>> +                      rte_strerror(-err));
>> +        }
>>      }
>>      ovs_mutex_unlock(&dev->mutex);
>>
>> ---
>>
>> I'm not sure if it will work (if we able to change default MAC while port is
>> started? Or we should request the reconfiguration for this?) and why we
>> didn't set the mac to HW previously.
>>
>> Best regards, Ilya Maximets.
>>
>>>
>>> OVS-DPDK supports both types of drivers for DPDK representor ports and I
>> think OVS should set the MAC (if provided) as part of DPDK port addition.
>>>
>>> Below you can find DPDK representor port info, "attached-mac" is the
>> desired value to be used as VF MAC.
>>>
>>> # ovs-vsctl list Interface vfrff9715cc-ee _uuid               :
>>> c4b5aa06-3a3c-4ff4-a5a0-eeac028ac2f4
>>> admin_state         : up
>>> bfd                 : {}
>>> bfd_status          : {}
>>> cfm_fault           : []
>>> cfm_fault_status    : []
>>> cfm_flap_count      : []
>>> cfm_health          : []
>>> cfm_mpid            : []
>>> cfm_remote_mpids    : []
>>> cfm_remote_opstate  : []
>>> duplex              : full
>>> error               : []
>>> external_ids        : {attached-mac="fa:16:3e:65:3d:0c",
>>> iface-id="ff9715cc-ee22-4f02-b57b-ac2a1707248a", iface-status=active,
>>> vm-uuid="e9a2ac9d-663b-4bdb-907a-b50aa3a238a9"}
>>> ifindex             : 1277170
>>> ingress_policing_burst: 0
>>> ingress_policing_rate: 0
>>> lacp_current        : []
>>> link_resets         : 0
>>> link_speed          : 10000000000
>>> link_state          : up
>>> lldp                : {}
>>> mac                 : []
>>> mac_in_use          : "a2:e4:6e:d6:ff:86"
>>> mtu                 : 1500
>>> mtu_request         : []
>>> name                : "vfrff9715cc-ee"
>>> ofport              : 15
>>> ofport_request      : []
>>> options             : {dpdk-devargs="0000:02:00.1,representor=[15]"}
>>> other_config        : {}
>>> statistics          : {rx_bytes=8638, rx_dropped=0, rx_errors=0,
>>> rx_mbuf_allocation_errors=0, rx_missed_errors=0, rx_packets=31,
>>> tx_bytes=10010, tx_dropped=0, tx_errors=0, tx_packets=143}
>> status
>>> : {driver_name="net_mlx5", if_descr="DPDK 18.11.0 net_mlx5",
>>> if_type="6", link_speed="10Gbps", max_hash_mac_addrs="0",
>>> max_mac_addrs="128", max_rx_pktlen="1518", max_rx_queues="65535",
>>> max_tx_queues="65535", max_vfs="0", max_vmdq_pools="0",
>>> min_rx_bufsize="32", numa_id="0", pci-device_id="0x1015",
>>> pci-vendor_id="0x15b3", port_no="1"} type                : dpdk
>>>
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------------------------------------------------------------------
>>> ----------
>>> *From:* Stokes, Ian <ian.stokes at intel.com>
>>> *Sent:* Sunday, June 23, 2019 4:06 PM
>>> *To:* Hamdy Khader; dev at openvswitch.org
>>> *Cc:* Ilya Maximets; Ophir Munk
>>> *Subject:* RE: OVS to set Virtual Function MAC address
>>>
>>>> Hi,
>>>>
>>>> I'm working with DPDK representor ports to integrate it with
>>>> OpenStack and I have to assign the MAC for the VF using "ip" command.
>>>>
>>>> Are there plans to make OVS to set the VFs MAC when we add dpdk
>>>> representor port?
>>>
>>> Not that I'm aware of, I would think setting MACs etc. would be an
>> administrator task during deployment carried out by something like IP (as you
>> have done already). Is there a reason why you feel OVS should do this task?
>>>
>>> Regards
>>> Ian
>> _______________________________________________
>> dev mailing list
>> dev at openvswitch.org
>> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.op
>> envswitch.org%2Fmailman%2Flistinfo%2Fovs-
>> dev&amp;data=02%7C01%7Cmoshele%40mellanox.com%7Ca07e000652064
>> 8ad938408d6f88d72fa%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C
>> 0%7C636969683735349267&amp;sdata=l34YLaY66mTMIjKqiaXdPTzJRezAgH
>> 1DoJX5WGOl33I%3D&amp;reserved=0
> 
> 


More information about the dev mailing list