[ovs-dev] [dpdk-dev] [dpdk-stable] [PATCH 17.11] mem: fix memory initialization time

[Eelco Chaudron]

On 12 Nov 2018, at 12:26, Eelco Chaudron wrote:

> On 12 Nov 2018, at 12:18, Alejandro Lucero wrote:
>
>> When using large amount of hugepage based memory, doing all the
>> hugepages mapping can take quite significant time.
>>
>> The problem is hugepages being initially mmaped to virtual addresses
>> which will be tried later for the final hugepage mmaping. This causes
>> the final mapping requiring calling mmap with another hint address 
>> which
>> can happen several times, depending on the amount of memory to mmap, 
>> and
>> which each mmmap taking more than a second.
>>
>> This patch changes the hint for the initial hugepage mmaping using
>> a starting address which will not collide with the final mmaping.
>>
>> Fixes: 293c0c4b957f ("mem: use address hint for mapping hugepages")
>>
>> Signed-off-by: Alejandro Lucero <alejandro.lucero at netronome.com>
>
> Thanks Alejandro for sending the patch. This issue was found in an 
> OVS-DPDK environment.
> I verified/tested the patch.
>
> Acked-by: Eelco Chaudron <echaudro at redhat.com>
> Tested-by: Eelco Chaudron <echaudro at redhat.com>
>> ---
>>  lib/librte_eal/linuxapp/eal/eal_memory.c | 15 +++++++++++++++
>>  1 file changed, 15 insertions(+)
>>
>> diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c 
>> b/lib/librte_eal/linuxapp/eal/eal_memory.c
>> index bac969a12..0675809b7 100644
>> --- a/lib/librte_eal/linuxapp/eal/eal_memory.c
>> +++ b/lib/librte_eal/linuxapp/eal/eal_memory.c
>> @@ -421,6 +421,21 @@ map_all_hugepages(struct hugepage_file 
>> *hugepg_tbl, struct hugepage_info *hpi,
>>  	}
>>  #endif
>>
>> +#ifdef RTE_ARCH_64
>> +	/*
>> +	 * Hugepages are first mmaped individually and then re-mmapped to
>> +	 * another region for having contiguous physical pages in 
>> contiguous
>> +	 * virtual addresses. Setting here vma_addr for the first hugepage
>> +	 * mapped to a virtual address which will not collide with the 
>> second
>> +	 * mmaping later. The next hugepages will use increments of this
>> +	 * initial address.
>> +	 *
>> +	 * The final virtual address will be based on baseaddr which is
>> +	 * 0x100000000. We use a hint here starting at 0x200000000, leaving
>> +	 * another 4GB just in case, plus the total available hugepages 
>> memory.
>> +	 */
>> +	vma_addr = (char *)0x200000000 + (hpi->hugepage_sz * 
>> hpi->num_pages[0]);
>> +#endif
>>  	for (i = 0; i < hpi->num_pages[0]; i++) {
>>  		uint64_t hugepage_sz = hpi->hugepage_sz;
>>
>> -- 
>> 2.17.1

Adding OVS dev to this thread, as this issue is introduced in DPDK 
17.11.4.