[ovs-discuss] Error attaching device to DPDK

Darrell Ball dball at vmware.com
Wed Mar 29 01:36:27 UTC 2017 


From: <ovs-discuss-bounces at openvswitch.org> on behalf of Brad Cowie <brad at cowie.nz>
Date: Tuesday, March 28, 2017 at 5:35 PM
To: Shivaram Mysore <shivaram.mysore at gmail.com>
Cc: "ovs-discuss at openvswitch.org" <ovs-discuss at openvswitch.org>
Subject: Re: [ovs-discuss] Error attaching device to DPDK

Hi folks,
I have been helping Shivaram off-list with this problem and just wanted to update this thread with the solution.
Basically the issue was with how hugepages were being preallocated by OVS for use with DPDK. I believe by default OVS sets other_config:dpdk-socket-mem to "1024,0". Which will give one 1GB hugepage to CPU0 and none to CPU1.
The ports Shivaram was trying to assign to DPDK were connected physically via a PCI-E bus to CPU1 and since there are no hugepages preallocated there for DPDK then we can't add it.
The fix is simple, to assign a hugepage to CPU1:

$ sudo ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem="1024,1024"
In OVS 2.7.0 unfortunately due to an issue (that is now fixed by commit ef6cca6fdc67f3cee21c6bb1c13c4ca7f8241314) you get a segfault rather than an error message telling you the problem.
In OVS 2.7.90 however the issue becomes quite clear as there is a nice log message like such:

2017-03-29T00:19:57Z|00067|netdev_dpdk|ERR|Insufficient memory to create memory pool for netdev dpdk-p0, with MTU 1500 on socket 1
There is documentation regarding these fields, but for case, the documentation could simply state something like
“you need to allocate hugepage memory on each numa node where dpdk ports are bound and or those ports
will not work with dpdk”
The error log could also be a little more succinct like “no memory pool allocated for netdev dpdk-p0,.., usage on socket 1”

Possibly, going even further, when a PMD thread gets auto-allocated to run on a given numa node,
memory allocation is defaulted there as well. If it cannot be allocated, an error log is generated.

Unfortunately to those unfamiliar with DPDK and NUMA architectures this won't be very obvious. Potentially we could add some additional help to the DPDK documentation pages for OVS that explains the other_config options you may need to reconfigure on multi-CPU NUMA machines?
Brad

On 29 March 2017 at 10:02, Shivaram Mysore <shivaram.mysore at gmail.com<mailto:shivaram.mysore at gmail.com>> wrote:
Ok thanks.  Will remember the same.  Right now, I will not specify anything as I have to get this to work first!

On Tue, Mar 28, 2017 at 2:00 PM, Bodireddy, Bhanuprakash <bhanuprakash.bodireddy at intel.com<mailto:bhanuprakash.bodireddy at intel.com>> wrote:
>Question:
>
>I don't have any specific cpu cores associated with DPDK.  Will this make any
>difference?  I also did not see this requirement as a MUST have in the
>documentation.

I assume your question is on PMD threads. If you don’t specify any thing explicitly
PMD pmd thread shall be created and pinned to core 0.  You can explicitly select the no. of pmd threads
and the cores by explicitly setting the affinity mask using 'other_config:pmd-cpu-mask'.

Based on the affinity mask, the PMD threads shall be spawned and the rx queues will be uniformly
distributed among the PMD threads.

Refer http://docs.openvswitch.org/en/latest/howto/dpdk/<https://urldefense.proofpoint.com/v2/url?u=http-3A__docs.openvswitch.org_en_latest_howto_dpdk_&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=iHLnhj_EA9zONSygrDHr4uCJdcbBZIdLxr9Jq-lAzKI&s=-SAVN5D_fKuClOt1yVUxYfC0WCYBG-4KZDiFtI306Nw&e=>.

Also note that 'dpdk-lcore-mask' should be used to set the dpdk lcore threads and vswitchd.

- Bhanuprakash.

>
># cat /proc/cmdline
>BOOT_IMAGE=/boot/vmlinuz-4.10.0-14-generic root=/dev/mapper/intelsdn-
>-vg-root ro quiet intel_iommu=on iommu=pt default_hugepagesz=1G
>hugepagesz=1G hugepages=4
>
>
>Thanks
>
>/Shivaram
>
>On Tue, Mar 28, 2017 at 1:32 PM, Shivaram Mysore
><shivaram.mysore at gmail.com<mailto:shivaram.mysore at gmail.com>> wrote:
>I was advised on this mailing list that VFIO drivers are better than IGB_UIO +
>more forgiving [1].  Hence switched to the same.
>
>[1] http://dpdk-guide.gitlab.io/dpdk-guide/setup/binding.html<https://urldefense.proofpoint.com/v2/url?u=http-3A__dpdk-2Dguide.gitlab.io_dpdk-2Dguide_setup_binding.html&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=iHLnhj_EA9zONSygrDHr4uCJdcbBZIdLxr9Jq-lAzKI&s=7Ig-meGAs5XoGOcikNq3lJWTL5ZK3lFwqShIb7_dao4&e=>
>
>/Shivaram
>
>On Tue, Mar 28, 2017 at 1:12 PM, Bodireddy, Bhanuprakash
><bhanuprakash.bodireddy at intel.com<mailto:bhanuprakash.bodireddy at intel.com>> wrote:
>
>>any hints for me based on the info provided?
>
>Your BIOS options and bootcmd line options are appropriate and no issues
>there. I wanted to check if you tried igb_uio drivers
>but saw your post here https://mail.openvswitch.org/pipermail/ovs-<https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_pipermail_ovs-2D&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=iHLnhj_EA9zONSygrDHr4uCJdcbBZIdLxr9Jq-lAzKI&s=E4BX55XTK5mJ-wH-1RW3837OV747lpxr2zhnC0Am0Dk&e=>
>discuss/2017-March/043993.html, reporting a crash with UIO drivers.
>
>Is that the reason why you moved to VFIO? I cross checked your logs but
>found nothing suspicious there.
>
>-Bhanuprakash.
>
>>
>>Thanks
>>
>>On Tue, Mar 28, 2017 at 8:41 AM, Shivaram Mysore
>><shivaram.mysore at gmail.com<mailto:shivaram.mysore at gmail.com>> wrote:
>>Hi,
>>
>># dmesg | grep -e DMAR -e IOMMU
>>[    0.000000] ACPI: DMAR 0x00000000798AF340 000158 (v01 SUPERM SMCI--
>>MB 00000001 INTL 20091013)
>>[    0.000000] DMAR: IOMMU enabled
>>[    0.109987] DMAR: Host address width 46
>>[    0.109988] DMAR: DRHD base: 0x000000fbffc000 flags: 0x0
>>[    0.109997] DMAR: dmar0: reg_base_addr fbffc000 ver 1:0 cap
>>d2078c106f0466 ecap f020de
>>[    0.109998] DMAR: DRHD base: 0x000000c7ffc000 flags: 0x1
>>[    0.110002] DMAR: dmar1: reg_base_addr c7ffc000 ver 1:0 cap
>>d2078c106f0466 ecap f020de
>>[    0.110003] DMAR: RMRR base: 0x0000007ba75000 end: 0x0000007ba84fff
>>[    0.110004] DMAR: ATSR flags: 0x0
>>[    0.110005] DMAR: RHSA base: 0x000000c7ffc000 proximity domain: 0x0
>>[    0.110006] DMAR: RHSA base: 0x000000fbffc000 proximity domain: 0x1
>>[    0.110008] DMAR-IR: IOAPIC id 3 under DRHD base  0xfbffc000 IOMMU 0
>>[    0.110008] DMAR-IR: IOAPIC id 1 under DRHD base  0xc7ffc000 IOMMU 1
>>[    0.110009] DMAR-IR: IOAPIC id 2 under DRHD base  0xc7ffc000 IOMMU 1
>>[    0.110010] DMAR-IR: HPET id 0 under DRHD base 0xc7ffc000
>>[    0.110011] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out
>bit.
>>[    0.110012] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the
>>BIOS setting.
>>[    0.110902] DMAR-IR: Enabled IRQ remapping in xapic mode
>>[    1.687595] DMAR: dmar0: Using Queued invalidation
>>[    1.687608] DMAR: dmar1: Using Queued invalidation
>>[    1.688473] DMAR: Hardware identity mapping for device 0000:ff:08.0
>>[    1.688475] DMAR: Hardware identity mapping for device 0000:ff:08.2
>>[    1.688477] DMAR: Hardware identity mapping for device 0000:ff:08.3
>>[    1.688478] DMAR: Hardware identity mapping for device 0000:ff:09.0
>>[    1.688480] DMAR: Hardware identity mapping for device 0000:ff:09.2
>>[    1.688482] DMAR: Hardware identity mapping for device 0000:ff:09.3
>>[    1.688483] DMAR: Hardware identity mapping for device 0000:ff:0b.0
>>[    1.688485] DMAR: Hardware identity mapping for device 0000:ff:0b.1
>>[    1.688487] DMAR: Hardware identity mapping for device 0000:ff:0b.2
>>[    1.688488] DMAR: Hardware identity mapping for device 0000:ff:0c.0
>>[    1.688490] DMAR: Hardware identity mapping for device 0000:ff:0c.1
>>[    1.688491] DMAR: Hardware identity mapping for device 0000:ff:0c.2
>>[    1.688493] DMAR: Hardware identity mapping for device 0000:ff:0c.3
>>[    1.688494] DMAR: Hardware identity mapping for device 0000:ff:0c.4
>>[    1.688496] DMAR: Hardware identity mapping for device 0000:ff:0c.5
>>[    1.688498] DMAR: Hardware identity mapping for device 0000:ff:0c.6
>>[    1.688499] DMAR: Hardware identity mapping for device 0000:ff:0c.7
>>[    1.688500] DMAR: Hardware identity mapping for device 0000:ff:0d.0
>>[    1.688502] DMAR: Hardware identity mapping for device 0000:ff:0d.1
>>[    1.688504] DMAR: Hardware identity mapping for device 0000:ff:0d.2
>>[    1.688505] DMAR: Hardware identity mapping for device 0000:ff:0d.3
>>[    1.688506] DMAR: Hardware identity mapping for device 0000:ff:0d.4
>>[    1.688508] DMAR: Hardware identity mapping for device 0000:ff:0d.5
>>[    1.688509] DMAR: Hardware identity mapping for device 0000:ff:0d.6
>>[    1.688510] DMAR: Hardware identity mapping for device 0000:ff:0d.7
>>[    1.688511] DMAR: Hardware identity mapping for device 0000:ff:0e.0
>>[    1.688513] DMAR: Hardware identity mapping for device 0000:ff:0e.1
>>[    1.688514] DMAR: Hardware identity mapping for device 0000:ff:0f.0
>>[    1.688516] DMAR: Hardware identity mapping for device 0000:ff:0f.1
>>[    1.688517] DMAR: Hardware identity mapping for device 0000:ff:0f.2
>>[    1.688518] DMAR: Hardware identity mapping for device 0000:ff:0f.3
>>[    1.688520] DMAR: Hardware identity mapping for device 0000:ff:0f.4
>>[    1.688521] DMAR: Hardware identity mapping for device 0000:ff:0f.5
>>[    1.688523] DMAR: Hardware identity mapping for device 0000:ff:0f.6
>>[    1.688524] DMAR: Hardware identity mapping for device 0000:ff:10.0
>>[    1.688526] DMAR: Hardware identity mapping for device 0000:ff:10.1
>>[    1.688527] DMAR: Hardware identity mapping for device 0000:ff:10.5
>>[    1.688529] DMAR: Hardware identity mapping for device 0000:ff:10.6
>>[    1.688530] DMAR: Hardware identity mapping for device 0000:ff:10.7
>>[    1.688532] DMAR: Hardware identity mapping for device 0000:ff:12.0
>>[    1.688533] DMAR: Hardware identity mapping for device 0000:ff:12.1
>>[    1.688534] DMAR: Hardware identity mapping for device 0000:ff:12.4
>>[    1.688536] DMAR: Hardware identity mapping for device 0000:ff:12.5
>>[    1.688537] DMAR: Hardware identity mapping for device 0000:ff:13.0
>>[    1.688539] DMAR: Hardware identity mapping for device 0000:ff:13.1
>>[    1.688541] DMAR: Hardware identity mapping for device 0000:ff:13.2
>>[    1.688542] DMAR: Hardware identity mapping for device 0000:ff:13.3
>>[    1.688543] DMAR: Hardware identity mapping for device 0000:ff:13.6
>>[    1.688545] DMAR: Hardware identity mapping for device 0000:ff:13.7
>>[    1.688546] DMAR: Hardware identity mapping for device 0000:ff:14.0
>>[    1.688547] DMAR: Hardware identity mapping for device 0000:ff:14.1
>>[    1.688549] DMAR: Hardware identity mapping for device 0000:ff:14.2
>>[    1.688550] DMAR: Hardware identity mapping for device 0000:ff:14.3
>>[    1.688552] DMAR: Hardware identity mapping for device 0000:ff:14.4
>>[    1.688553] DMAR: Hardware identity mapping for device 0000:ff:14.5
>>[    1.688559] DMAR: Hardware identity mapping for device 0000:ff:14.6
>>[    1.688561] DMAR: Hardware identity mapping for device 0000:ff:14.7
>>[    1.688562] DMAR: Hardware identity mapping for device 0000:ff:16.0
>>[    1.688564] DMAR: Hardware identity mapping for device 0000:ff:16.1
>>[    1.688565] DMAR: Hardware identity mapping for device 0000:ff:16.2
>>[    1.688566] DMAR: Hardware identity mapping for device 0000:ff:16.3
>>[    1.688568] DMAR: Hardware identity mapping for device 0000:ff:16.6
>>[    1.688569] DMAR: Hardware identity mapping for device 0000:ff:16.7
>>[    1.688571] DMAR: Hardware identity mapping for device 0000:ff:17.0
>>[    1.688573] DMAR: Hardware identity mapping for device 0000:ff:17.1
>>[    1.688574] DMAR: Hardware identity mapping for device 0000:ff:17.2
>>[    1.688576] DMAR: Hardware identity mapping for device 0000:ff:17.3
>>[    1.688577] DMAR: Hardware identity mapping for device 0000:ff:17.4
>>[    1.688579] DMAR: Hardware identity mapping for device 0000:ff:17.5
>>[    1.688581] DMAR: Hardware identity mapping for device 0000:ff:17.6
>>[    1.688582] DMAR: Hardware identity mapping for device 0000:ff:17.7
>>[    1.688584] DMAR: Hardware identity mapping for device 0000:ff:1e.0
>>[    1.688586] DMAR: Hardware identity mapping for device 0000:ff:1e.1
>>[    1.688588] DMAR: Hardware identity mapping for device 0000:ff:1e.2
>>[    1.688590] DMAR: Hardware identity mapping for device 0000:ff:1e.3
>>[    1.688592] DMAR: Hardware identity mapping for device 0000:ff:1e.4
>>[    1.688594] DMAR: Hardware identity mapping for device 0000:ff:1f.0
>>[    1.688596] DMAR: Hardware identity mapping for device 0000:ff:1f.2
>>[    1.688610] DMAR: Hardware identity mapping for device 0000:7f:08.0
>>[    1.688612] DMAR: Hardware identity mapping for device 0000:7f:08.2
>>[    1.688614] DMAR: Hardware identity mapping for device 0000:7f:08.3
>>[    1.688616] DMAR: Hardware identity mapping for device 0000:7f:09.0
>>[    1.688617] DMAR: Hardware identity mapping for device 0000:7f:09.2
>>[    1.688619] DMAR: Hardware identity mapping for device 0000:7f:09.3
>>[    1.688621] DMAR: Hardware identity mapping for device 0000:7f:0b.0
>>[    1.688623] DMAR: Hardware identity mapping for device 0000:7f:0b.1
>>[    1.688625] DMAR: Hardware identity mapping for device 0000:7f:0b.2
>>[    1.688627] DMAR: Hardware identity mapping for device 0000:7f:0c.0
>>[    1.688629] DMAR: Hardware identity mapping for device 0000:7f:0c.1
>>[    1.688631] DMAR: Hardware identity mapping for device 0000:7f:0c.2
>>[    1.688632] DMAR: Hardware identity mapping for device 0000:7f:0c.3
>>[    1.688634] DMAR: Hardware identity mapping for device 0000:7f:0c.4
>>[    1.688636] DMAR: Hardware identity mapping for device 0000:7f:0c.5
>>[    1.688637] DMAR: Hardware identity mapping for device 0000:7f:0c.6
>>[    1.688639] DMAR: Hardware identity mapping for device 0000:7f:0c.7
>>[    1.688641] DMAR: Hardware identity mapping for device 0000:7f:0d.0
>>[    1.688642] DMAR: Hardware identity mapping for device 0000:7f:0d.1
>>[    1.688644] DMAR: Hardware identity mapping for device 0000:7f:0d.2
>>[    1.688646] DMAR: Hardware identity mapping for device 0000:7f:0d.3
>>[    1.688647] DMAR: Hardware identity mapping for device 0000:7f:0d.4
>>[    1.688649] DMAR: Hardware identity mapping for device 0000:7f:0d.5
>>[    1.688650] DMAR: Hardware identity mapping for device 0000:7f:0d.6
>>[    1.688652] DMAR: Hardware identity mapping for device 0000:7f:0d.7
>>[    1.688653] DMAR: Hardware identity mapping for device 0000:7f:0e.0
>>[    1.688655] DMAR: Hardware identity mapping for device 0000:7f:0e.1
>>[    1.688657] DMAR: Hardware identity mapping for device 0000:7f:0f.0
>>[    1.688658] DMAR: Hardware identity mapping for device 0000:7f:0f.1
>>[    1.688660] DMAR: Hardware identity mapping for device 0000:7f:0f.2
>>[    1.688661] DMAR: Hardware identity mapping for device 0000:7f:0f.3
>>[    1.688663] DMAR: Hardware identity mapping for device 0000:7f:0f.4
>>[    1.688664] DMAR: Hardware identity mapping for device 0000:7f:0f.5
>>[    1.688666] DMAR: Hardware identity mapping for device 0000:7f:0f.6
>>[    1.688668] DMAR: Hardware identity mapping for device 0000:7f:10.0
>>[    1.688669] DMAR: Hardware identity mapping for device 0000:7f:10.1
>>[    1.688671] DMAR: Hardware identity mapping for device 0000:7f:10.5
>>[    1.688673] DMAR: Hardware identity mapping for device 0000:7f:10.6
>>[    1.688674] DMAR: Hardware identity mapping for device 0000:7f:10.7
>>[    1.688676] DMAR: Hardware identity mapping for device 0000:7f:12.0
>>[    1.688678] DMAR: Hardware identity mapping for device 0000:7f:12.1
>>[    1.688684] DMAR: Hardware identity mapping for device 0000:7f:12.4
>>[    1.688686] DMAR: Hardware identity mapping for device 0000:7f:12.5
>>[    1.688688] DMAR: Hardware identity mapping for device 0000:7f:13.0
>>[    1.688689] DMAR: Hardware identity mapping for device 0000:7f:13.1
>>[    1.688691] DMAR: Hardware identity mapping for device 0000:7f:13.2
>>[    1.688692] DMAR: Hardware identity mapping for device 0000:7f:13.3
>>[    1.688694] DMAR: Hardware identity mapping for device 0000:7f:13.6
>>[    1.688696] DMAR: Hardware identity mapping for device 0000:7f:13.7
>>[    1.688697] DMAR: Hardware identity mapping for device 0000:7f:14.0
>>[    1.688699] DMAR: Hardware identity mapping for device 0000:7f:14.1
>>[    1.688701] DMAR: Hardware identity mapping for device 0000:7f:14.2
>>[    1.688703] DMAR: Hardware identity mapping for device 0000:7f:14.3
>>[    1.688704] DMAR: Hardware identity mapping for device 0000:7f:14.4
>>[    1.688706] DMAR: Hardware identity mapping for device 0000:7f:14.5
>>[    1.688708] DMAR: Hardware identity mapping for device 0000:7f:14.6
>>[    1.688710] DMAR: Hardware identity mapping for device 0000:7f:14.7
>>[    1.688712] DMAR: Hardware identity mapping for device 0000:7f:16.0
>>[    1.688714] DMAR: Hardware identity mapping for device 0000:7f:16.1
>>[    1.688716] DMAR: Hardware identity mapping for device 0000:7f:16.2
>>[    1.688718] DMAR: Hardware identity mapping for device 0000:7f:16.3
>>[    1.688720] DMAR: Hardware identity mapping for device 0000:7f:16.6
>>[    1.688722] DMAR: Hardware identity mapping for device 0000:7f:16.7
>>[    1.688724] DMAR: Hardware identity mapping for device 0000:7f:17.0
>>[    1.688726] DMAR: Hardware identity mapping for device 0000:7f:17.1
>>[    1.688729] DMAR: Hardware identity mapping for device 0000:7f:17.2
>>[    1.688731] DMAR: Hardware identity mapping for device 0000:7f:17.3
>>[    1.688733] DMAR: Hardware identity mapping for device 0000:7f:17.4
>>[    1.688735] DMAR: Hardware identity mapping for device 0000:7f:17.5
>>[    1.688737] DMAR: Hardware identity mapping for device 0000:7f:17.6
>>[    1.688739] DMAR: Hardware identity mapping for device 0000:7f:17.7
>>[    1.688741] DMAR: Hardware identity mapping for device 0000:7f:1e.0
>>[    1.688742] DMAR: Hardware identity mapping for device 0000:7f:1e.1
>>[    1.688744] DMAR: Hardware identity mapping for device 0000:7f:1e.2
>>[    1.688746] DMAR: Hardware identity mapping for device 0000:7f:1e.3
>>[    1.688748] DMAR: Hardware identity mapping for device 0000:7f:1e.4
>>[    1.688750] DMAR: Hardware identity mapping for device 0000:7f:1f.0
>>[    1.688752] DMAR: Hardware identity mapping for device 0000:7f:1f.2
>>[    1.688769] DMAR: Hardware identity mapping for device 0000:00:00.0
>>[    1.688774] DMAR: Hardware identity mapping for device 0000:00:01.0
>>[    1.688778] DMAR: Hardware identity mapping for device 0000:00:01.1
>>[    1.688783] DMAR: Hardware identity mapping for device 0000:00:03.0
>>[    1.688785] DMAR: Hardware identity mapping for device 0000:00:04.0
>>[    1.688787] DMAR: Hardware identity mapping for device 0000:00:04.1
>>[    1.688790] DMAR: Hardware identity mapping for device 0000:00:04.2
>>[    1.688792] DMAR: Hardware identity mapping for device 0000:00:04.3
>>[    1.688794] DMAR: Hardware identity mapping for device 0000:00:04.4
>>[    1.688797] DMAR: Hardware identity mapping for device 0000:00:04.5
>>[    1.688799] DMAR: Hardware identity mapping for device 0000:00:04.6
>>[    1.688801] DMAR: Hardware identity mapping for device 0000:00:04.7
>>[    1.688803] DMAR: Hardware identity mapping for device 0000:00:05.0
>>[    1.688807] DMAR: Hardware identity mapping for device 0000:00:05.1
>>[    1.688809] DMAR: Hardware identity mapping for device 0000:00:05.2
>>[    1.688811] DMAR: Hardware identity mapping for device 0000:00:05.4
>>[    1.688814] DMAR: Hardware identity mapping for device 0000:00:11.0
>>[    1.688816] DMAR: Hardware identity mapping for device 0000:00:11.4
>>[    1.688818] DMAR: Hardware identity mapping for device 0000:00:14.0
>>[    1.688821] DMAR: Hardware identity mapping for device 0000:00:16.0
>>[    1.688823] DMAR: Hardware identity mapping for device 0000:00:16.1
>>[    1.688825] DMAR: Hardware identity mapping for device 0000:00:1a.0
>>[    1.688828] DMAR: Hardware identity mapping for device 0000:00:1c.0
>>[    1.688831] DMAR: Hardware identity mapping for device 0000:00:1c.2
>>[    1.688833] DMAR: Hardware identity mapping for device 0000:00:1d.0
>>[    1.688835] DMAR: Hardware identity mapping for device 0000:00:1f.0
>>[    1.688837] DMAR: Hardware identity mapping for device 0000:00:1f.2
>>[    1.688844] DMAR: Hardware identity mapping for device 0000:00:1f.3
>>[    1.688846] DMAR: Hardware identity mapping for device 0000:00:1f.6
>>[    1.688864] DMAR: Hardware identity mapping for device 0000:01:00.0
>>[    1.688870] DMAR: Hardware identity mapping for device 0000:01:00.1
>>[    1.688876] DMAR: Hardware identity mapping for device 0000:01:00.2
>>[    1.688883] DMAR: Hardware identity mapping for device 0000:01:00.3
>>[    1.688895] DMAR: Hardware identity mapping for device 0000:80:01.0
>>[    1.688899] DMAR: Hardware identity mapping for device 0000:80:02.0
>>[    1.688903] DMAR: Hardware identity mapping for device 0000:80:03.0
>>[    1.688906] DMAR: Hardware identity mapping for device 0000:80:04.0
>>[    1.688908] DMAR: Hardware identity mapping for device 0000:80:04.1
>>[    1.688911] DMAR: Hardware identity mapping for device 0000:80:04.2
>>[    1.688913] DMAR: Hardware identity mapping for device 0000:80:04.3
>>[    1.688915] DMAR: Hardware identity mapping for device 0000:80:04.4
>>[    1.688918] DMAR: Hardware identity mapping for device 0000:80:04.5
>>[    1.688920] DMAR: Hardware identity mapping for device 0000:80:04.6
>>[    1.688923] DMAR: Hardware identity mapping for device 0000:80:04.7
>>[    1.688936] DMAR: Hardware identity mapping for device 0000:80:05.0
>>[    1.688940] DMAR: Hardware identity mapping for device 0000:80:05.1
>>[    1.688943] DMAR: Hardware identity mapping for device 0000:80:05.2
>>[    1.688945] DMAR: Hardware identity mapping for device 0000:80:05.4
>>[    1.688956] DMAR: Hardware identity mapping for device 0000:81:00.0
>>[    1.688961] DMAR: Hardware identity mapping for device 0000:81:00.1
>>[    1.688974] DMAR: Hardware identity mapping for device 0000:82:00.0
>>[    1.688980] DMAR: Hardware identity mapping for device 0000:82:00.1
>>[    1.688986] DMAR: Hardware identity mapping for device 0000:82:00.2
>>[    1.688992] DMAR: Hardware identity mapping for device 0000:82:00.3
>>[    1.689006] DMAR: Hardware identity mapping for device 0000:83:00.0
>>[    1.689012] DMAR: Hardware identity mapping for device 0000:83:00.1
>>[    1.689018] DMAR: Hardware identity mapping for device 0000:83:00.2
>>[    1.689024] DMAR: Hardware identity mapping for device 0000:83:00.3
>>[    1.689025] DMAR: Setting RMRR:
>>[    1.689026] DMAR: Ignoring identity map for HW passthrough device
>>0000:00:14.0 [0x7ba75000 - 0x7ba84fff]
>>[    1.689027] DMAR: Ignoring identity map for HW passthrough device
>>0000:00:1a.0 [0x7ba75000 - 0x7ba84fff]
>>[    1.689028] DMAR: Ignoring identity map for HW passthrough device
>>0000:00:1d.0 [0x7ba75000 - 0x7ba84fff]
>>[    1.689036] DMAR: Prepare 0-16MiB unity mapping for LPC
>>[    1.689036] DMAR: Ignoring identity map for HW passthrough device
>>0000:00:1f.0 [0x0 - 0xffffff]
>>[    1.689040] DMAR: Intel(R) Virtualization Technology for Directed I/O
>>[    2.299766] DMAR: 32bit 0000:00:11.4 uses non-identity mapping
>>[    2.301941] DMAR: 32bit 0000:00:1a.0 uses non-identity mapping
>>[    2.302679] DMAR: Setting identity map for device 0000:00:1a.0 [0x7ba75000
>-
>>0x7ba84fff]
>>[    2.320910] DMAR: 32bit 0000:00:1d.0 uses non-identity mapping
>>[    2.321685] DMAR: Setting identity map for device 0000:00:1d.0
>[0x7ba75000
>>- 0x7ba84fff]
>>#
>>
>>
>>On Tue, Mar 28, 2017 at 1:46 AM, Bodireddy, Bhanuprakash
>><bhanuprakash.bodireddy at intel.com<mailto:bhanuprakash.bodireddy at intel.com>> wrote:
>>Hello shivaram,
>>
>>Darrell has already asked if VT-d is enabled?
>>Your cmdline options are appropriate as you have iommu
>>enabled(intel_iommu=on iommu=pt).
>>can you also check the output of below mentioned command to see if VT-d
>is
>>enabled in BIOS?
>>
>>$ dmesg | grep -e DMAR -e IOMMU
>>
>>Regards,
>>Bhanuprakash.
>>
>>From: ovs-discuss-bounces at openvswitch.org<mailto:ovs-discuss-bounces at openvswitch.org> [mailto:ovs-discuss-<mailto:ovs-discuss->
>>bounces at openvswitch.org<mailto:bounces at openvswitch.org>] On Behalf Of Shivaram Mysore
>>Sent: Tuesday, March 28, 2017 6:25 AM
>>To: Darrell Ball <dball at vmware.com<mailto:dball at vmware.com>>
>>Cc: ovs-discuss at openvswitch.org<mailto:ovs-discuss at openvswitch.org>
>>Subject: Re: [ovs-discuss] Error attaching device to DPDK
>>
>># uname -r
>>4.10.0-14-generic
>>
>># cat /proc/cmdline
>>BOOT_IMAGE=/boot/vmlinuz-4.10.0-14-generic
>root=/dev/mapper/intelsdn-
>>-vg-root ro quiet intel_iommu=on iommu=pt default_hugepagesz=1G
>>hugepagesz=1G hugepages=4
>>
>># uname -a
>>Linux intelsdn 4.10.0-14-generic #16-Ubuntu SMP Fri Mar 17 15:19:26 UTC
>2017
>>x86_64 x86_64 x86_64 GNU/Linux
>>
>># cat /etc/*release
>>DISTRIB_ID=Ubuntu
>>DISTRIB_RELEASE=17.04
>>DISTRIB_CODENAME=zesty
>>DISTRIB_DESCRIPTION="Ubuntu Zesty Zapus (development branch)"
>>NAME="Ubuntu"
>>VERSION="17.04 (Zesty Zapus)"
>>ID=ubuntu
>>ID_LIKE=debian
>>PRETTY_NAME="Ubuntu Zesty Zapus (development branch)"
>>VERSION_ID="17.04"
>>
>>
>>Nothing in the syslog related
>>
>>On Mon, Mar 27, 2017 at 10:02 PM, Darrell Ball <dball at vmware.com<mailto:dball at vmware.com>> wrote:
>>I see the pastebins now.
>>
>>What kernel version are you using ?
>>VT-d enabled ?
>>Anything related in syslog ?
>>
>>From: Shivaram Mysore <shivaram.mysore at gmail.com<mailto:shivaram.mysore at gmail.com>>
>>Date: Monday, March 27, 2017 at 9:34 PM
>>To: Darrell Ball <dball at vmware.com<mailto:dball at vmware.com>>
>>Cc: "ovs-discuss at openvswitch.org<mailto:ovs-discuss at openvswitch.org>" <ovs-discuss at openvswitch.org<mailto:ovs-discuss at openvswitch.org>>
>>Subject: Re: [ovs-discuss] Error attaching device to DPDK
>>
>>One more datapoint, please check the paste bin log in URL sent
>earlier.  Dpdk-
>>devbind works and you can see results in the paste bin log
>>
>>/Shivaram
>>::Sent from my mobile device::
>>
>>On Mar 27, 2017, at 8:34 PM, Darrell Ball <dball at vmware.com<mailto:dball at vmware.com>> wrote:
>>see inline below
>>
>>But a few questions:
>>1)      Are you using the recommended version of dpdk with OVS 2.7 – 16.11 ?
>>2)      Are you getting a crash still or is this post crash ?
>>3)      Was the previous crash still happening with 16.11 or only 17.02 ?
>>
>>Thanks
>>Darrell
>>
>>From: <ovs-discuss-bounces at openvswitch.org<mailto:ovs-discuss-bounces at openvswitch.org>> on behalf of Shivaram
>>Mysore <shivaram.mysore at gmail.com<mailto:shivaram.mysore at gmail.com>>
>>Date: Monday, March 27, 2017 at 7:45 PM
>>To: "ovs-discuss at openvswitch.org<mailto:ovs-discuss at openvswitch.org>" <ovs-discuss at openvswitch.org<mailto:ovs-discuss at openvswitch.org>>
>>Subject: [ovs-discuss] Error attaching device to DPDK
>>
>>Hello,
>>
>>I have installed OVS 2.7 running.  When I do:
>>1.      # ovs-vsctl add-port ovs-ip64-br0 dpdk-p0 -- set Interface dpdk-p0
>>type=dpdk options:dpdk-devargs=0000:82:00.0
>>2.      ovs-vsctl: Error detected while setting up 'dpdk-p0': Error attaching
>>device '0000:82:00.0' to DPDK.  See ovs-vswitchd log for details.
>>
>>Possibly, the port is not bound.
>>Check using
>>$DPDK_DIR/tools/dpdk-devbind.py –status
>>and see the documentation for more detail
>>
>>and the log file shows:
>>1.      2017-03-28T02:04:40.004Z|00064|netdev_dpdk|WARN|Error attaching
>>device '0000:82:00.0' to DPDK
>>2.      2017-03-28T02:04:40.004Z|00065|netdev|WARN|dpdk-p0: could
>>not set configuration (Invalid argument)
>>
>>What would cause such a situation?
>>
>>Commands run: https://pastebin.com/0P754x0t<https://urldefense.proofpoint.com/v2/url?u=https-3A__pastebin.com_0P754x0t&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=iHLnhj_EA9zONSygrDHr4uCJdcbBZIdLxr9Jq-lAzKI&s=VfS5KJZ8kK8qwKRTcwiImfMn7rNMnL6wqiF-yabYLrg&e=>
>>Detailed OVS Log: https://pastebin.com/ctBhLuRG<https://urldefense.proofpoint.com/v2/url?u=https-3A__pastebin.com_ctBhLuRG&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=iHLnhj_EA9zONSygrDHr4uCJdcbBZIdLxr9Jq-lAzKI&s=uq1nuXcPv-XCT_O7nazR7HQyqAu-stvMOH_FiIL1AzU&e=>
>>
>>
>


_______________________________________________
discuss mailing list
discuss at openvswitch.org<mailto:discuss at openvswitch.org>
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss<https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddiscuss&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=iHLnhj_EA9zONSygrDHr4uCJdcbBZIdLxr9Jq-lAzKI&s=5st0X7BnSbw4aF87d4Flp_4tjmXgjj5Xl5R-KDj-bP8&e=>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openvswitch.org/pipermail/ovs-discuss/attachments/20170329/4fedbf5c/attachment-0001.html>

More information about the discuss mailing list