[dpdk-dev] 回复： About the shared flock for hugepage files

[Burakov, Anatoly]

Hi Bakari,

On 12-Sep-17 4:41 AM, 912873551 at qq.com (白长敏) wrote:
> Hi Jim,
> 
> 
> Thank you for reply, i understand it, but i still have two doubts.
> 
> 
> 1) if i have > 1 DPDK application on a machine, i can use the parameter "--file-prefix" to distinguish, it can work even if have not flock.

That's not entirely correct. DPDK will attempt to clean up *all* 
hugepage files it encounters, not just those that are designated as "its 
own" via the --file-prefix [1]. This is because hugepages are not freed 
back to the OS unless its backing file in hugetlbfs is removed from the 
system, so DPDK needs to clear out any pre-existing hugepage files in 
order to successfully allocate any hugepages for its use.

> 2) If I understand correctly, in the "map_all_hugepages()", after making the shared flock, close(fd), that is to free the flock. Then, in the "clear_hugedir()", it equal to have not flock ?

You are correct that normally, closing the file descriptor would release 
the lock. However, we're also doing an mmap() call on that fd [2], which 
duplicates the descriptor, and therefore we continue to hold a lock to 
that file.

Quote from mmap() man page [3]:

"The mmap() function shall add an extra reference to the file associated 
with the file descriptor fildes which is not removed by a subsequent 
close() on that file descriptor. This reference shall be removed when 
there are no more mappings to the file."

[1] 
http://www.dpdk.org/browse/dpdk/tree/lib/librte_eal/linuxapp/eal/eal_hugepage_info.c#n192
[2] 
http://www.dpdk.org/browse/dpdk/tree/lib/librte_eal/linuxapp/eal/eal_memory.c#n505
[3] https://linux.die.net/man/3/mmap

> 
> 
> ------------------ ???? ------------------
> ???: "Jim Thompson";<jim at netgate.com>;
> ????: 2017?9?12?(???) ??11:10
> ???: "???"<912873551 at qq.com>;
> ??: "dev"<dev at dpdk.org>;
> ??: Re: [dpdk-dev] About the shared flock for hugepage files
> 
> 
> 
> 
> Bakari,
> 
> 
> This is so you can have > 1 DPDK application on a machine and they won't use each other's hugepages.
> 
> 
> See, for example the comments in clear_hugedir()
> 
> 
> http://dpdk.org/browse/dpdk/tree/lib/librte_eal/linuxapp/eal/eal_hugepage_info.c#n198
> 
> 
> Jim
> 
> On Sep 11, 2017, at 9:16 PM, ??? <912873551 at qq.com> wrote:
> 
> 
> Hi, everyone,
> 
> 
>      I'm a developer from china, now i have a problem is about the shared flock for hugepage files in the function of "map_all_hugepages()".
> 
> 
> as follows:
> ---------------------------------------------------------------------------------------------------
> /* set shared flock on the file. */
> 
> 
> if (flock(fd, LOCK_SH | LOCK_NB) == -1) {
> 
> 
>          RTE_LOG(DEBUG, EAL, "%s(): Locking file failed:%s \n", __func__, strerror(errno));
> 
> 
>          close(fd);
> 
> 
>          return i;
> 
> 
> }
> 
> 
> close(fd);
> 
> --------------------------------------------------------------------------------------------------
> there, it gives a shared flock to each hugepage file in /mnt/huge, and close it right away.
> 
> 
> it makes me puzzled. Why? What doing that is for?
> 
> 
> I'm looking forward to your reply, thank you very much.
> 
> 
> best regards,
> Bakari.
> 

-- 
Thanks,
Anatoly