[ovs-dev] [ovs-discuss] OVS DPDK VFIO error
Kapil Adhikesavalu
kapil20084 at gmail.com
Tue Aug 9 12:14:57 UTC 2016
Hi Bhanu Prakash,
I already check the BIOS, VT-d is enabled by default.
>From the dmesg, how can i find if VT-d is enabled, i see "IOMMU enabled", i
don't understand the rest.
[root at localhost ~]# dmesg | grep -e DMAR -e IOMMU
[ 0.000000] ACPI: DMAR 0x00000000BDDAD200 000558 (v01 HP ProLiant
00000001 \xffffffd2? 0000162E)
[ 0.000000] DMAR: IOMMU enabled
[ 0.069333] DMAR: Host address width 46
[ 0.069335] DMAR: DRHD base: 0x000000fbefe000 flags: 0x0
[ 0.069341] DMAR: dmar0: reg_base_addr fbefe000 ver 1:0 cap
d2078c106f0466 ecap f020de
[ 0.069342] DMAR: DRHD base: 0x000000f4ffe000 flags: 0x1
[ 0.069346] DMAR: dmar1: reg_base_addr f4ffe000 ver 1:0 cap
d2078c106f0466 ecap f020de
[ 0.069347] DMAR: RMRR base: 0x000000bdffd000 end: 0x000000bdffffff
[ 0.069349] DMAR: RMRR base: 0x000000bdff6000 end: 0x000000bdffcfff
[ 0.069349] DMAR: RMRR base: 0x000000bdf83000 end: 0x000000bdf84fff
[ 0.069351] DMAR: RMRR base: 0x000000bdf7f000 end: 0x000000bdf82fff
[ 0.069352] DMAR: RMRR base: 0x000000bdf6f000 end: 0x000000bdf7efff
[ 0.069353] DMAR: RMRR base: 0x000000bdf6e000 end: 0x000000bdf6efff
[ 0.069355] DMAR: RMRR base: 0x000000000f4000 end: 0x000000000f4fff
[ 0.069356] DMAR: RMRR base: 0x000000000e8000 end: 0x000000000e8fff
[ 0.069356] DMAR: RMRR base: 0x000000bddde000 end: 0x000000bdddefff
[ 0.069357] DMAR: ATSR flags: 0x0
[ 0.069360] DMAR-IR: IOAPIC id 10 under DRHD base 0xfbefe000 IOMMU 0
[ 0.069361] DMAR-IR: IOAPIC id 8 under DRHD base 0xf4ffe000 IOMMU 1
[ 0.069362] DMAR-IR: IOAPIC id 0 under DRHD base 0xf4ffe000 IOMMU 1
[ 0.069362] DMAR-IR: HPET id 0 under DRHD base 0xf4ffe000
[ 0.069364] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out
bit.
[ 0.069364] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the
BIOS setting.
[ 0.070293] DMAR-IR: Enabled IRQ remapping in xapic mode
[ 0.996061] DMAR: dmar0: Using Queued invalidation
[ 0.996220] DMAR: dmar1: Using Queued invalidation
[ 0.996477] DMAR: Hardware identity mapping for device 0000:00:00.0
[ 0.996479] DMAR: Hardware identity mapping for device 0000:00:01.0
[ 0.996481] DMAR: Hardware identity mapping for device 0000:00:01.1
[ 0.996483] DMAR: Hardware identity mapping for device 0000:00:02.0
[ 0.996484] DMAR: Hardware identity mapping for device 0000:00:02.1
[ 0.996489] DMAR: Hardware identity mapping for device 0000:00:02.2
[ 0.996490] DMAR: Hardware identity mapping for device 0000:00:02.3
[ 0.996492] DMAR: Hardware identity mapping for device 0000:00:03.0
[ 0.996494] DMAR: Hardware identity mapping for device 0000:00:03.1
[ 0.996495] DMAR: Hardware identity mapping for device 0000:00:03.2
[ 0.996497] DMAR: Hardware identity mapping for device 0000:00:03.3
[ 0.996499] DMAR: Hardware identity mapping for device 0000:00:04.0
[ 0.996501] DMAR: Hardware identity mapping for device 0000:00:04.1
[ 0.996502] DMAR: Hardware identity mapping for device 0000:00:04.2
[ 0.996504] DMAR: Hardware identity mapping for device 0000:00:04.3
[ 0.996505] DMAR: Hardware identity mapping for device 0000:00:04.4
[ 0.996507] DMAR: Hardware identity mapping for device 0000:00:04.5
[ 0.996509] DMAR: Hardware identity mapping for device 0000:00:04.6
[ 0.996510] DMAR: Hardware identity mapping for device 0000:00:04.7
[ 0.996512] DMAR: Hardware identity mapping for device 0000:00:05.0
[ 0.996514] DMAR: Hardware identity mapping for device 0000:00:05.2
[ 0.996515] DMAR: Hardware identity mapping for device 0000:00:05.4
[ 0.996517] DMAR: Hardware identity mapping for device 0000:00:11.0
[ 0.996519] DMAR: Hardware identity mapping for device 0000:00:1a.0
[ 0.996520] DMAR: Hardware identity mapping for device 0000:00:1c.0
[ 0.996522] DMAR: Hardware identity mapping for device 0000:00:1c.7
[ 0.996523] DMAR: Hardware identity mapping for device 0000:00:1d.0
[ 0.996525] DMAR: Hardware identity mapping for device 0000:00:1f.0
[ 0.996534] DMAR: Hardware identity mapping for device 0000:01:00.1
[ 0.996536] DMAR: Hardware identity mapping for device 0000:01:00.4
[ 0.996545] DMAR: Hardware identity mapping for device 0000:1f:08.0
[ 0.996547] DMAR: Hardware identity mapping for device 0000:1f:09.0
[ 0.996548] DMAR: Hardware identity mapping for device 0000:1f:0a.0
[ 0.996550] DMAR: Hardware identity mapping for device 0000:1f:0a.1
[ 0.996552] DMAR: Hardware identity mapping for device 0000:1f:0a.2
[ 0.996553] DMAR: Hardware identity mapping for device 0000:1f:0a.3
[ 0.996555] DMAR: Hardware identity mapping for device 0000:1f:0b.0
[ 0.996556] DMAR: Hardware identity mapping for device 0000:1f:0b.3
[ 0.996558] DMAR: Hardware identity mapping for device 0000:1f:0c.0
[ 0.996559] DMAR: Hardware identity mapping for device 0000:1f:0c.1
[ 0.996561] DMAR: Hardware identity mapping for device 0000:1f:0c.2
[ 0.996563] DMAR: Hardware identity mapping for device 0000:1f:0c.3
[ 0.996564] DMAR: Hardware identity mapping for device 0000:1f:0c.4
[ 0.996566] DMAR: Hardware identity mapping for device 0000:1f:0c.5
[ 0.996567] DMAR: Hardware identity mapping for device 0000:1f:0d.0
[ 0.996569] DMAR: Hardware identity mapping for device 0000:1f:0d.1
[ 0.996570] DMAR: Hardware identity mapping for device 0000:1f:0d.2
[ 0.996572] DMAR: Hardware identity mapping for device 0000:1f:0d.3
[ 0.996573] DMAR: Hardware identity mapping for device 0000:1f:0d.4
[ 0.996575] DMAR: Hardware identity mapping for device 0000:1f:0d.5
[ 0.996576] DMAR: Hardware identity mapping for device 0000:1f:0e.0
[ 0.996578] DMAR: Hardware identity mapping for device 0000:1f:0e.1
[ 0.996579] DMAR: Hardware identity mapping for device 0000:1f:0f.0
[ 0.996581] DMAR: Hardware identity mapping for device 0000:1f:0f.1
[ 0.996582] DMAR: Hardware identity mapping for device 0000:1f:0f.2
[ 0.996584] DMAR: Hardware identity mapping for device 0000:1f:0f.3
[ 0.996585] DMAR: Hardware identity mapping for device 0000:1f:0f.4
[ 0.996587] DMAR: Hardware identity mapping for device 0000:1f:0f.5
[ 0.996588] DMAR: Hardware identity mapping for device 0000:1f:10.0
[ 0.996590] DMAR: Hardware identity mapping for device 0000:1f:10.1
[ 0.996592] DMAR: Hardware identity mapping for device 0000:1f:10.2
[ 0.996593] DMAR: Hardware identity mapping for device 0000:1f:10.3
[ 0.996595] DMAR: Hardware identity mapping for device 0000:1f:10.4
[ 0.996596] DMAR: Hardware identity mapping for device 0000:1f:10.5
[ 0.996598] DMAR: Hardware identity mapping for device 0000:1f:10.6
[ 0.996600] DMAR: Hardware identity mapping for device 0000:1f:10.7
[ 0.996601] DMAR: Hardware identity mapping for device 0000:1f:13.0
[ 0.996603] DMAR: Hardware identity mapping for device 0000:1f:13.1
[ 0.996604] DMAR: Hardware identity mapping for device 0000:1f:13.4
[ 0.996607] DMAR: Hardware identity mapping for device 0000:1f:13.5
[ 0.996609] DMAR: Hardware identity mapping for device 0000:1f:16.0
[ 0.996611] DMAR: Hardware identity mapping for device 0000:1f:16.1
[ 0.996612] DMAR: Hardware identity mapping for device 0000:1f:16.2
[ 0.996614] DMAR: Hardware identity mapping for device 0000:1f:1c.0
[ 0.996616] DMAR: Hardware identity mapping for device 0000:1f:1c.1
[ 0.996617] DMAR: Hardware identity mapping for device 0000:1f:1d.0
[ 0.996619] DMAR: Hardware identity mapping for device 0000:1f:1d.1
[ 0.996621] DMAR: Hardware identity mapping for device 0000:1f:1d.2
[ 0.996622] DMAR: Hardware identity mapping for device 0000:1f:1d.3
[ 0.996624] DMAR: Hardware identity mapping for device 0000:1f:1d.4
[ 0.996625] DMAR: Hardware identity mapping for device 0000:1f:1d.5
[ 0.996627] DMAR: Hardware identity mapping for device 0000:1f:1e.0
[ 0.996629] DMAR: Hardware identity mapping for device 0000:1f:1e.1
[ 0.996631] DMAR: Hardware identity mapping for device 0000:1f:1e.2
[ 0.996632] DMAR: Hardware identity mapping for device 0000:1f:1e.3
[ 0.996634] DMAR: Hardware identity mapping for device 0000:1f:1e.4
[ 0.996635] DMAR: Hardware identity mapping for device 0000:1f:1e.5
[ 0.996637] DMAR: Hardware identity mapping for device 0000:1f:1e.6
[ 0.996639] DMAR: Hardware identity mapping for device 0000:1f:1e.7
[ 0.996647] DMAR: Hardware identity mapping for device 0000:20:00.0
[ 0.996649] DMAR: Hardware identity mapping for device 0000:20:01.0
[ 0.996651] DMAR: Hardware identity mapping for device 0000:20:01.1
[ 0.996652] DMAR: Hardware identity mapping for device 0000:20:02.0
[ 0.996797] DMAR: Setting RMRR:
[ 0.996815] DMAR: Setting identity map for device 0000:01:00.0
[0xbddde000 - 0xbdddefff]
[ 0.996853] DMAR: Setting identity map for device 0000:01:00.2
[0xbddde000 - 0xbdddefff]
[ 0.996875] DMAR: Ignoring identity map for HW passthrough device
0000:01:00.4 [0xbddde000 - 0xbdddefff]
[ 0.996890] DMAR: Setting identity map for device 0000:00:1f.2 [0xe8000
- 0xe8fff]
[ 0.996911] DMAR: Setting identity map for device 0000:01:00.0 [0xe8000
- 0xe8fff]
[ 0.996927] DMAR: Setting identity map for device 0000:01:00.2 [0xe8000
- 0xe8fff]
[ 0.996955] DMAR: Setting identity map for device 0000:02:00.0 [0xe8000
- 0xe8fff]
[ 0.996998] DMAR: Setting identity map for device 0000:03:00.0 [0xe8000
- 0xe8fff]
[ 0.997043] DMAR: Setting identity map for device 0000:03:00.1 [0xe8000
- 0xe8fff]
[ 0.997080] DMAR: Setting identity map for device 0000:03:00.2 [0xe8000
- 0xe8fff]
[ 0.997121] DMAR: Setting identity map for device 0000:03:00.3 [0xe8000
- 0xe8fff]
[ 0.997158] DMAR: Setting identity map for device 0000:04:00.0 [0xe8000
- 0xe8fff]
[ 0.997202] DMAR: Setting identity map for device 0000:04:00.1 [0xe8000
- 0xe8fff]
[ 0.997226] DMAR: Setting identity map for device 0000:00:1f.2 [0xf4000
- 0xf4fff]
[ 0.997228] DMAR: Setting identity map for device 0000:01:00.0 [0xf4000
- 0xf4fff]
[ 0.997231] DMAR: Setting identity map for device 0000:01:00.2 [0xf4000
- 0xf4fff]
[ 0.997233] DMAR: Setting identity map for device 0000:02:00.0 [0xf4000
- 0xf4fff]
[ 0.997235] DMAR: Setting identity map for device 0000:03:00.0 [0xf4000
- 0xf4fff]
[ 0.997237] DMAR: Setting identity map for device 0000:03:00.1 [0xf4000
- 0xf4fff]
[ 0.997239] DMAR: Setting identity map for device 0000:03:00.2 [0xf4000
- 0xf4fff]
[ 0.997241] DMAR: Setting identity map for device 0000:03:00.3 [0xf4000
- 0xf4fff]
[ 0.997243] DMAR: Setting identity map for device 0000:04:00.0 [0xf4000
- 0xf4fff]
[ 0.997245] DMAR: Setting identity map for device 0000:04:00.1 [0xf4000
- 0xf4fff]
[ 0.997246] DMAR: Setting identity map for device 0000:00:1f.2
[0xbdf6e000 - 0xbdf6efff]
[ 0.997262] DMAR: Setting identity map for device 0000:01:00.0
[0xbdf6e000 - 0xbdf6efff]
[ 0.997270] DMAR: Setting identity map for device 0000:01:00.2
[0xbdf6e000 - 0xbdf6efff]
[ 0.997279] DMAR: Setting identity map for device 0000:02:00.0
[0xbdf6e000 - 0xbdf6efff]
[ 0.997293] DMAR: Setting identity map for device 0000:03:00.0
[0xbdf6e000 - 0xbdf6efff]
[ 0.997311] DMAR: Setting identity map for device 0000:03:00.1
[0xbdf6e000 - 0xbdf6efff]
[ 0.997326] DMAR: Setting identity map for device 0000:03:00.2
[0xbdf6e000 - 0xbdf6efff]
[ 0.997340] DMAR: Setting identity map for device 0000:03:00.3
[0xbdf6e000 - 0xbdf6efff]
[ 0.997356] DMAR: Setting identity map for device 0000:04:00.0
[0xbdf6e000 - 0xbdf6efff]
[ 0.997370] DMAR: Setting identity map for device 0000:04:00.1
[0xbdf6e000 - 0xbdf6efff]
[ 0.997385] DMAR: Setting identity map for device 0000:00:1f.2
[0xbdf6f000 - 0xbdf7efff]
[ 0.997388] DMAR: Setting identity map for device 0000:01:00.0
[0xbdf6f000 - 0xbdf7efff]
[ 0.997390] DMAR: Setting identity map for device 0000:01:00.2
[0xbdf6f000 - 0xbdf7efff]
[ 0.997393] DMAR: Setting identity map for device 0000:02:00.0
[0xbdf6f000 - 0xbdf7efff]
[ 0.997395] DMAR: Setting identity map for device 0000:03:00.0
[0xbdf6f000 - 0xbdf7efff]
[ 0.997397] DMAR: Setting identity map for device 0000:03:00.1
[0xbdf6f000 - 0xbdf7efff]
[ 0.997399] DMAR: Setting identity map for device 0000:03:00.2
[0xbdf6f000 - 0xbdf7efff]
[ 0.997402] DMAR: Setting identity map for device 0000:03:00.3
[0xbdf6f000 - 0xbdf7efff]
[ 0.997404] DMAR: Setting identity map for device 0000:04:00.0
[0xbdf6f000 - 0xbdf7efff]
[ 0.997406] DMAR: Setting identity map for device 0000:04:00.1
[0xbdf6f000 - 0xbdf7efff]
[ 0.997409] DMAR: Setting identity map for device 0000:00:1f.2
[0xbdf7f000 - 0xbdf82fff]
[ 0.997411] DMAR: Setting identity map for device 0000:01:00.0
[0xbdf7f000 - 0xbdf82fff]
[ 0.997413] DMAR: Setting identity map for device 0000:01:00.2
[0xbdf7f000 - 0xbdf82fff]
[ 0.997415] DMAR: Setting identity map for device 0000:02:00.0
[0xbdf7f000 - 0xbdf82fff]
[ 0.997417] DMAR: Setting identity map for device 0000:03:00.0
[0xbdf7f000 - 0xbdf82fff]
[ 0.997419] DMAR: Setting identity map for device 0000:03:00.1
[0xbdf7f000 - 0xbdf82fff]
[ 0.997423] DMAR: Setting identity map for device 0000:03:00.2
[0xbdf7f000 - 0xbdf82fff]
[ 0.997425] DMAR: Setting identity map for device 0000:03:00.3
[0xbdf7f000 - 0xbdf82fff]
[ 0.997427] DMAR: Setting identity map for device 0000:04:00.0
[0xbdf7f000 - 0xbdf82fff]
[ 0.997430] DMAR: Setting identity map for device 0000:04:00.1
[0xbdf7f000 - 0xbdf82fff]
[ 0.997432] DMAR: Setting identity map for device 0000:00:1f.2
[0xbdf83000 - 0xbdf84fff]
[ 0.997434] DMAR: Setting identity map for device 0000:01:00.0
[0xbdf83000 - 0xbdf84fff]
[ 0.997436] DMAR: Setting identity map for device 0000:01:00.2
[0xbdf83000 - 0xbdf84fff]
[ 0.997438] DMAR: Setting identity map for device 0000:02:00.0
[0xbdf83000 - 0xbdf84fff]
[ 0.997440] DMAR: Setting identity map for device 0000:03:00.0
[0xbdf83000 - 0xbdf84fff]
[ 0.997442] DMAR: Setting identity map for device 0000:03:00.1
[0xbdf83000 - 0xbdf84fff]
[ 0.997443] DMAR: Setting identity map for device 0000:03:00.2
[0xbdf83000 - 0xbdf84fff]
[ 0.997445] DMAR: Setting identity map for device 0000:03:00.3
[0xbdf83000 - 0xbdf84fff]
[ 0.997447] DMAR: Setting identity map for device 0000:04:00.0
[0xbdf83000 - 0xbdf84fff]
[ 0.997449] DMAR: Setting identity map for device 0000:04:00.1
[0xbdf83000 - 0xbdf84fff]
[ 0.997451] DMAR: Setting identity map for device 0000:01:00.0
[0xbdff6000 - 0xbdffcfff]
[ 0.997454] DMAR: Setting identity map for device 0000:01:00.2
[0xbdff6000 - 0xbdffcfff]
[ 0.997456] DMAR: Ignoring identity map for HW passthrough device
0000:01:00.4 [0xbdff6000 - 0xbdffcfff]
[ 0.997457] DMAR: Ignoring identity map for HW passthrough device
0000:00:1a.0 [0xbdffd000 - 0xbdffffff]
[ 0.997458] DMAR: Ignoring identity map for HW passthrough device
0000:00:1d.0 [0xbdffd000 - 0xbdffffff]
[ 0.997460] DMAR: Prepare 0-16MiB unity mapping for LPC
[ 0.997461] DMAR: Ignoring identity map for HW passthrough device
0000:00:1f.0 [0x0 - 0xffffff]
[ 0.997465] DMAR: Intel(R) Virtualization Technology for Directed I/O
[ 1.351801] DMAR: 32bit 0000:00:1a.0 uses non-identity mapping
[ 1.362623] DMAR: 32bit 0000:00:1d.0 uses non-identity mapping
[ 1.373601] DMAR: 32bit 0000:01:00.4 uses non-identity mapping
[ 297.035504] vfio-pci 0000:04:00.0: Device is ineligible for IOMMU domain
attach due to platform RMRR requirement. Contact your platform vendor.
Regards
kapil.
On Tue, Aug 9, 2016 at 5:38 PM, Bodireddy, Bhanuprakash <
bhanuprakash.bodireddy at intel.com> wrote:
> >-----Original Message-----
> >From: discuss [mailto:discuss-bounces at openvswitch.org] On Behalf Of Kapil
> >Adhikesavalu
> >Sent: Tuesday, August 9, 2016 10:46 AM
> >To: dev at openvswitch.org; discuss at openvswitch.org
> >Subject: [ovs-discuss] OVS DPDK VFIO error
> >
> >Hi,
> >
> >On a Intel xeon E5-2697 chip with iommu turned on with Intel NIC 82599, i
> am
> >getting the following error while doing the NIC binding using VFIO.
> >kernel: 4.23 fedora 23, i haven't tried the latest kernel yet.
> >
> >E5-2697 supports IOMMU VT-d
> I hope you have already enabled VT-d in BIOS, can you check 'dmesg | grep
> -e DMAR -e IOMMU'.
>
> >
> >VFIO NIC binding steps,
> >modprobe vfio-pci
> >sudo /usr/bin/chmod a+x /dev/vfio
> >sudo /usr/bin/chmod 0666 /dev/vfio/*
> >$DPDK_DIR/tools/dpdk_nic_bind.py --bind=vfio-pci 0000:04:00.0
> >$DPDK_DIR/tools/dpdk_nic_bind.py --status
> >Error
> >=====
> >EAL: Detected 48 lcore(s)
> >EAL: Probing VFIO support...
> >EAL: IOMMU type 1 (Type 1) is supported
> >EAL: IOMMU type 8 (No-IOMMU) is not supported
> >EAL: VFIO support initialized
> >
> >EAL: Master lcore 1 is ready (tid=83504bc0;cpuset=[1])
> >EAL: PCI device 0000:04:00.0 on NUMA socket 0
> >EAL: probe driver: 8086:154d rte_ixgbe_pmd
> >EAL: set IOMMU type 1 (Type 1) failed, error 1 (Operation not permitted)
> >EAL: set IOMMU type 8 (No-IOMMU) failed, error 19 (No such device)
> >EAL: 0000:04:00.0 failed to select IOMMU type
> >EAL: Error - exiting with code: 1
> > Cause: Requested device 0000:04:00.0 cannot be used
> >
> >dmesg:
> >======
> >[ 0.997461] DMAR: Ignoring identity map for HW passthrough device
> >0000:00:1f.0 [0x0 - 0xffffff]
> >[ 0.997465] DMAR: Intel(R) Virtualization Technology for Directed I/O
> >[ 1.351801] DMAR: 32bit 0000:00:1a.0 uses non-identity mapping
> >[ 1.362623] DMAR: 32bit 0000:00:1d.0 uses non-identity mapping
> >[ 1.373601] DMAR: 32bit 0000:01:00.4 uses non-identity mapping
> >[ 297.035504] vfio-pci 0000:04:00.0: Device is ineligible for IOMMU
> domain
> >attach due to platform RMRR requirement. Contact your platform vendor.
> >
> >
> >[root at localhost bin]# cat /proc/cmdline
> >BOOT_IMAGE=/vmlinuz-4.2.3-300.fc23.x86_64 root=/dev/mapper/fedora-
> >root ro rd.lvm.lv=fedora/root rd.lvm.lv=fedora/swap rhgb quiet
> >default_hugepagesz=1G hugepagesz=1G hugepages=16 hugepagesz=2M
> >hugepages=2048 iommu=pt intel_iommu=on
>
> I don’t see any problem with your cmdline as iommu=pt and intel_iommu is
> added.
>
> Regards,
> Bhanu Prakash.
>
> >
> >demsg | grep 10G - 82599 controller
> >04:00.0 Ethernet controller: Intel Corporation Ethernet 10G 2P X520
> Adapter
> >(rev 01)
> >04:00.1 Ethernet controller: Intel Corporation Ethernet 10G 2P X520
> Adapter
> >(rev 01)
> >
> >Regards
> >Kapil.
>
More information about the dev
mailing list