[dpdk-dev] [PATCH 0/4] support for write combining
Rafał Kozik
rk at semihalf.com
Mon Apr 16 13:36:13 CEST 2018
Hello Bruce,
thank you for your advices.
> 1. Why not always have igb_uio support write-combining since it can be
> controlled thereafter via userspace mapping one file or another?
I added parameter to the igb_uio because currently it perform ioremap
and fails if it return NULL.
But performing ioremap makes it impossible to use WC, so I remove it.
ENA driver work well after this change, but I cannot test it on all
drivers and all platforms.
It seems to me that making it configurable prevents form spoiling
other drivers that could use internal_addr returned by ioremap.
> 2. Why not always map both resource and resource_wc files at the PCI level,
> and make them available via different pointers to the driver? Then the
> driver can choose at the per-access level which it wants to use. For
> example, for init of a device, a driver may do all register access via
> uncachable memory, and only use the write-combined support for
> performance-critical parts. [I have a draft patch lying around here
> somewhere that does something similar to that.]
I tried to implement this idea but without good results. I get mapping
with or without WC depending on mapping order.
As I was trying to find solution I come across with this paper:
https://www.kernel.org/doc/ols/2008/ols2008v2-pages-135-144.pdf
In section 5.3 and 5.4 it is discussing access to PCI resources.
According to it:
A request to uncached access can fail if there is already
an existing write-combine mapping for that region. A
request for write-combine access can succeed with un-
cached mapping instead, in the case of already existing
uncached mapping for this region.
We cannot use WC all the time, because it not guaranteed writing order.
On this basis I suppose that better option is to map each resource
only once depending on parameter provided by PMD.
> One last question - if using vfio-pci kernel module, do the resource_wc
> files present the bars as write-combined memory type, or are they
> uncachable?
I tried to use VFIO to map WC memory, but without success.
Best regards,
Rafal Kozik
2018-04-11 16:42 GMT+02:00 Bruce Richardson <bruce.richardson at intel.com>:
> On Wed, Apr 11, 2018 at 04:07:13PM +0200, Rafal Kozik wrote:
>> Support for write combining.
>>
>> Rafal Kozik (4):
>> igb_uio: add wc option
>> bus/pci: reference driver structure
>> eal: enable WC during resources mapping
>> net/ena: enable WC
>>
>> drivers/bus/pci/linux/pci_uio.c | 39 ++++++++++++++++++++++++++++-----------
>> drivers/bus/pci/pci_common.c | 13 ++++++++-----
>> drivers/bus/pci/rte_bus_pci.h | 2 ++
>> drivers/net/ena/ena_ethdev.c | 3 ++-
>> kernel/linux/igb_uio/igb_uio.c | 17 ++++++++++++++---
>> 5 files changed, 54 insertions(+), 20 deletions(-)
>>
> Couple of thoughts on this set.
>
> You add an option to the kernel module to allow wc to be supported on a
> device, but when we go to do PCI mapping, we either map the regular
> resource file or the _wc one. Therefore:
>
> 1. Why not always have igb_uio support write-combining since it can be
> controlled thereafter via userspace mapping one file or another?
>
> 2. Why not always map both resource and resource_wc files at the PCI level,
> and make them available via different pointers to the driver? Then the
> driver can choose at the per-access level which it wants to use. For
> example, for init of a device, a driver may do all register access via
> uncachable memory, and only use the write-combined support for
> performance-critical parts. [I have a draft patch lying around here
> somewhere that does something similar to that.]
>
> One last question - if using vfio-pci kernel module, do the resource_wc
> files present the bars as write-combined memory type, or are they
> uncachable?
>
> Regards,
> /Bruce
More information about the dev
mailing list