[ovs-discuss] Unable to hotplug interface with DPDK initialized

[Shriroop Joshi (shrirjos)]

Hi Ian

Thank you for confirming.
I tried loading vfio-pci before starting openvswitch and it is working for me.

I did not get a change to try out TESTPMD. I will try it out but that might take some time.

Thanks for your help.
Regards,
Shriroop

On 6/5/19, 3:12 PM, "Ian Stokes" <ian.stokes at intel.com> wrote:

    On 6/4/2019 6:40 PM, Shriroop Joshi (shrirjos) wrote:
    > Hi Ian
    > 
    > I am reaching out to ask if you were able reproduce the issue. Is there anything else I can provide which will help?
    > 
    > Regards,
    > Shriroop
    
    Hi Shiroop, apologies for the delay, I havent been able to spend as much 
    time as I'd like on the ML lately.
    
    I was able to reproduce the issue this evening. It seems specific to 
    i350 devices using VFIO and hotplugging (I was not able to reproduce it 
    with i40e or ixgbe interfaces).
    
    I'll have to look further into what is going wrong.
    
    Currently the only work around I can think of would be to use igb_uio 
    instead of VFIO if that is an option (hotplug seems to work without 
    issue using igb_uio in this setup although typically VFIO is preferred).
    
    Alternatively if VFIO is required, then I would suggest binding the 
    device to VFIO-PCI before starting ovs-dpdk? Do you see the same issue 
    in that case?
    
    This could be a bug in DPDK for the net_e100_igb PMD, have you tried to 
    reproduce the issue with a DPDK sample app such as TESTPMD?
    
    Regards
    Ian
    > 
    > On 5/16/19, 3:29 PM, "Shriroop Joshi (shrirjos)" <shrirjos at cisco.com> wrote:
    > 
    >      Hi Ian
    >      
    >      I am using Intel i350 PCIe NIC. I have attached entire ovs-vswitchd.log which shows both error and success.
    >      After attaching the interface, I restarted openvswitch service and it was able to attach the interface.
    >      
    >      Regards
    >      Shriroop
    >      
    >      On 5/16/19, 7:35 AM, "Ian Stokes" <ian.stokes at intel.com> wrote:
    >      
    >          On 5/15/2019 10:37 PM, Shriroop Joshi (shrirjos) via discuss wrote:
    >          > Hello ovs community
    >          >
    >          > I am trying to attach an interface (eth4-4) to a bridge (svc-br) with netdev datapath_type. Here are the steps which I follow:
    >          > 1. Load vfio-pci driver
    >          > 2. Bind vfio-pci driver to interface eth4-4
    >          > (following steps as mentioned here: http://docs.openvswitch.org/en/latest/topics/dpdk/phy/#dpdk-binding-nics)
    >          > 3. Hotplug this interface to DPDK bridge:
    >          > # ovs-vsctl add-port svc-br eth4-4 -- set interface eth4-4 type=dpdk options:dpdk-devargs=0000:af:00.3
    >          > (following steps as mentioned here: http://docs.openvswitch.org/en/latest/topics/dpdk/phy/#hotplugging)
    >          >
    >          > After completing these steps, I see that it shows 'Error attaching device '0000:af:00.3' to DPDK' error.
    >          > Here are the error logs:
    >          > 2019-05-15T21:11:20.663Z|00227|dpdk|INFO|EAL: PCI device 0000:af:00.3 on NUMA socket 1
    >          > 2019-05-15T21:11:20.663Z|00228|dpdk|INFO|EAL:   probe driver: 8086:1521 net_e1000_igb
    >          > 2019-05-15T21:11:20.663Z|00229|dpdk|ERR|EAL: Driver cannot attach the device (0000:af:00.3)
    >          > 2019-05-15T21:11:20.663Z|00230|netdev_dpdk|WARN|Error attaching device '0000:af:00.3' to DPDK
    >          > 2019-05-15T21:11:20.663Z|00231|netdev|WARN|eth4-4: could not set configuration (Invalid argument)
    >          >
    >          
    >          I tried to reproduce this on my local system but could not.
    >          
    >          What model HW NIC are you attempting to hotplug out of interest?
    >          
    >          Also could you provide the full log for when OVS fails to hotplug as
    >          well as when it succeeds?
    >          
    >          I'd be interested in seeing the OVS configuration commands passed for DPDK.
    >          
    >          Ian
    >          
    >          > I noticed that restarting openvswitch solves this issue, and I can see the interface attached to bridge after restart.
    >          > Here is the snippet from logs during restart:
    >          > 2019-05-15T21:14:54.172Z|00078|dpdk|INFO|EAL: PCI device 0000:af:00.3 on NUMA socket 1
    >          > 2019-05-15T21:14:54.172Z|00079|dpdk|INFO|EAL:   probe driver: 8086:1521 net_e1000_igb
    >          > 2019-05-15T21:14:54.174Z|00080|dpdk|INFO|EAL:   using IOMMU type 1 (Type 1)
    >          >
    >          > Am I missing any step? Openvswitch documentation does not specify anything else.
    >          >
    >          > # ovs-vswitchd --version
    >          > ovs-vswitchd (Open vSwitch) 2.9.3
    >          > DPDK 17.11.0
    >          >
    >          > Regards,
    >          > Shriroop
    >          >
    >          >
    >          >
    >          >
    >          > _______________________________________________
    >          > discuss mailing list
    >          > discuss at openvswitch.org
    >          > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
    >          >
    >          
    >          
    >      
    >      
    >