diff mbox

[dpdk-dev,23/28] net/ixgbe: use eal I/O device memory read/write API

Message ID 1481680558-4003-24-git-send-email-jerin.jacob@caviumnetworks.com (mailing list archive)
State Superseded, archived
Delegated to: Thomas Monjalon
Headers

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel compilation success Compilation OK

Commit Message

Jerin Jacob Dec. 14, 2016, 1:55 a.m. UTC 
  From: Santosh Shukla <santosh.shukla@caviumnetworks.com>

Replace the raw I/O device memory read/write access with eal
abstraction for I/O device memory read/write access to fix
portability issues across different architectures.

Signed-off-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
CC: Helin Zhang <helin.zhang@intel.com>
CC: Konstantin Ananyev <konstantin.ananyev@intel.com>
---
 drivers/net/ixgbe/base/ixgbe_osdep.h | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

Comments

Jianbo Liu Dec. 15, 2016, 8:37 a.m. UTC | #1 
On 14 December 2016 at 09:55, Jerin Jacob
<jerin.jacob@caviumnetworks.com> wrote:
> From: Santosh Shukla <santosh.shukla@caviumnetworks.com>
>
> Replace the raw I/O device memory read/write access with eal
> abstraction for I/O device memory read/write access to fix
> portability issues across different architectures.
>
> Signed-off-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>
> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> CC: Helin Zhang <helin.zhang@intel.com>
> CC: Konstantin Ananyev <konstantin.ananyev@intel.com>
> ---
>  drivers/net/ixgbe/base/ixgbe_osdep.h | 13 +++++++++----
>  1 file changed, 9 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/net/ixgbe/base/ixgbe_osdep.h b/drivers/net/ixgbe/base/ixgbe_osdep.h
> index 77f0af5..9d16c21 100644
> --- a/drivers/net/ixgbe/base/ixgbe_osdep.h
> +++ b/drivers/net/ixgbe/base/ixgbe_osdep.h
> @@ -44,6 +44,7 @@
>  #include <rte_cycles.h>
>  #include <rte_log.h>
>  #include <rte_byteorder.h>
> +#include <rte_io.h>
>
>  #include "../ixgbe_logs.h"
>  #include "../ixgbe_bypass_defines.h"
> @@ -121,16 +122,20 @@ typedef int               bool;
>
>  #define prefetch(x) rte_prefetch0(x)
>
> -#define IXGBE_PCI_REG(reg) (*((volatile uint32_t *)(reg)))
> +#define IXGBE_PCI_REG(reg) ({                  \
> +       uint32_t __val;                         \
> +       __val = rte_readl(reg);                 \
> +       __val;                                  \
> +})
>
>  static inline uint32_t ixgbe_read_addr(volatile void* addr)
>  {
>         return rte_le_to_cpu_32(IXGBE_PCI_REG(addr));
>  }
>
> -#define IXGBE_PCI_REG_WRITE(reg, value) do { \
> -       IXGBE_PCI_REG((reg)) = (rte_cpu_to_le_32(value)); \
> -} while(0)
> +#define IXGBE_PCI_REG_WRITE(reg, value) ({             \
> +       rte_writel(rte_cpu_to_le_32(value), reg);       \
> +})
>

memory barrier operation is put inside IXGBE_PCI_REG_READ/WRITE in
your change, but I found rte_*mb is called before these macros in some
places.
Can you remove all these redundant calls? And please do the same
checking for other drivers.

>  #define IXGBE_PCI_REG_ADDR(hw, reg) \
>         ((volatile uint32_t *)((char *)(hw)->hw_addr + (reg)))
> --
> 2.5.5
>
Santosh Shukla Dec. 16, 2016, 4:40 a.m. UTC | #2 
On Thu, Dec 15, 2016 at 04:37:12PM +0800, Jianbo Liu wrote:
> On 14 December 2016 at 09:55, Jerin Jacob
> <jerin.jacob@caviumnetworks.com> wrote:
> > From: Santosh Shukla <santosh.shukla@caviumnetworks.com>
> >
> > Replace the raw I/O device memory read/write access with eal
> > abstraction for I/O device memory read/write access to fix
> > portability issues across different architectures.
> >
> > Signed-off-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>
> > Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> > CC: Helin Zhang <helin.zhang@intel.com>
> > CC: Konstantin Ananyev <konstantin.ananyev@intel.com>
> > ---
> >  drivers/net/ixgbe/base/ixgbe_osdep.h | 13 +++++++++----
> >  1 file changed, 9 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/net/ixgbe/base/ixgbe_osdep.h b/drivers/net/ixgbe/base/ixgbe_osdep.h
> > index 77f0af5..9d16c21 100644
> > --- a/drivers/net/ixgbe/base/ixgbe_osdep.h
> > +++ b/drivers/net/ixgbe/base/ixgbe_osdep.h
> > @@ -44,6 +44,7 @@
> >  #include <rte_cycles.h>
> >  #include <rte_log.h>
> >  #include <rte_byteorder.h>
> > +#include <rte_io.h>
> >
> >  #include "../ixgbe_logs.h"
> >  #include "../ixgbe_bypass_defines.h"
> > @@ -121,16 +122,20 @@ typedef int               bool;
> >
> >  #define prefetch(x) rte_prefetch0(x)
> >
> > -#define IXGBE_PCI_REG(reg) (*((volatile uint32_t *)(reg)))
> > +#define IXGBE_PCI_REG(reg) ({                  \
> > +       uint32_t __val;                         \
> > +       __val = rte_readl(reg);                 \
> > +       __val;                                  \
> > +})
> >
> >  static inline uint32_t ixgbe_read_addr(volatile void* addr)
> >  {
> >         return rte_le_to_cpu_32(IXGBE_PCI_REG(addr));
> >  }
> >
> > -#define IXGBE_PCI_REG_WRITE(reg, value) do { \
> > -       IXGBE_PCI_REG((reg)) = (rte_cpu_to_le_32(value)); \
> > -} while(0)
> > +#define IXGBE_PCI_REG_WRITE(reg, value) ({             \
> > +       rte_writel(rte_cpu_to_le_32(value), reg);       \
> > +})
> >
> 
> memory barrier operation is put inside IXGBE_PCI_REG_READ/WRITE in
> your change, but I found rte_*mb is called before these macros in some
> places.
> Can you remove all these redundant calls? And please do the same
> checking for other drivers.
>

Ok.

Thinking of adding _relaxed_rd/wr style macro agnostic to arch for ixgbe case 
in particular. Such that for those code incident:
x86 case> first default barrier + relaxed call.
arm case> first default barrier + relaxed call.

Does that make sense to you? If so then will take care in v2.

Santosh.


> >  #define IXGBE_PCI_REG_ADDR(hw, reg) \
> >         ((volatile uint32_t *)((char *)(hw)->hw_addr + (reg)))
> > --
> > 2.5.5
> >
Santosh Shukla Dec. 22, 2016, 12:36 p.m. UTC | #3 
Hi Jiangbo,

On Thu, Dec 15, 2016 at 08:40:19PM -0800, Santosh Shukla wrote:
> On Thu, Dec 15, 2016 at 04:37:12PM +0800, Jianbo Liu wrote:
> > On 14 December 2016 at 09:55, Jerin Jacob
> > <jerin.jacob@caviumnetworks.com> wrote:
> > > From: Santosh Shukla <santosh.shukla@caviumnetworks.com>
> > >
> > 
> > memory barrier operation is put inside IXGBE_PCI_REG_READ/WRITE in
> > your change, but I found rte_*mb is called before these macros in some
> > places.
> > Can you remove all these redundant calls? And please do the same
> > checking for other drivers.
> >
> 
> Ok.
> 
> Thinking of adding _relaxed_rd/wr style macro agnostic to arch for ixgbe case 
> in particular. Such that for those code incident:
> x86 case> first default barrier + relaxed call.
> arm case> first default barrier + relaxed call.
> 
> Does that make sense to you? If so then will take care in v2.
> 
> Santosh.

We spend time looking at drivers code where double barrier
may happen. Most of them are in driver init path,
configuration/control path code. So keeping double
barrier won't impact performance. 

We plan to replace only fast path code with _relaxed
style API's. That way we won't impact each driver
performance and we'll have the clean port. 

Does it make sense? Thought?

> 
> > >  #define IXGBE_PCI_REG_ADDR(hw, reg) \
> > >         ((volatile uint32_t *)((char *)(hw)->hw_addr + (reg)))
> > > --
> > > 2.5.5
> > >
Jianbo Liu Dec. 23, 2016, 1:42 a.m. UTC | #4 
Hi Santosh,

On 22 December 2016 at 20:36, Santosh Shukla
<santosh.shukla@caviumnetworks.com> wrote:
> Hi Jiangbo,
>
> On Thu, Dec 15, 2016 at 08:40:19PM -0800, Santosh Shukla wrote:
>> On Thu, Dec 15, 2016 at 04:37:12PM +0800, Jianbo Liu wrote:
>> > On 14 December 2016 at 09:55, Jerin Jacob
>> > <jerin.jacob@caviumnetworks.com> wrote:
>> > > From: Santosh Shukla <santosh.shukla@caviumnetworks.com>
>> > >
>> >
>> > memory barrier operation is put inside IXGBE_PCI_REG_READ/WRITE in
>> > your change, but I found rte_*mb is called before these macros in some
>> > places.
>> > Can you remove all these redundant calls? And please do the same
>> > checking for other drivers.
>> >
>>
>> Ok.
>>
>> Thinking of adding _relaxed_rd/wr style macro agnostic to arch for ixgbe case
>> in particular. Such that for those code incident:
>> x86 case> first default barrier + relaxed call.
>> arm case> first default barrier + relaxed call.
>>
>> Does that make sense to you? If so then will take care in v2.
>>
>> Santosh.
>
> We spend time looking at drivers code where double barrier
> may happen. Most of them are in driver init path,
> configuration/control path code. So keeping double
> barrier won't impact performance.
>
> We plan to replace only fast path code with _relaxed
> style API's. That way we won't impact each driver
> performance and we'll have the clean port.
>
> Does it make sense? Thought?
>

Yes, please continue your work.
diff mbox

Patch

diff --git a/drivers/net/ixgbe/base/ixgbe_osdep.h b/drivers/net/ixgbe/base/ixgbe_osdep.h
index 77f0af5..9d16c21 100644
--- a/drivers/net/ixgbe/base/ixgbe_osdep.h
+++ b/drivers/net/ixgbe/base/ixgbe_osdep.h
@@ -44,6 +44,7 @@ 
 #include <rte_cycles.h>
 #include <rte_log.h>
 #include <rte_byteorder.h>
+#include <rte_io.h>
 
 #include "../ixgbe_logs.h"
 #include "../ixgbe_bypass_defines.h"
@@ -121,16 +122,20 @@  typedef int		bool;
 
 #define prefetch(x) rte_prefetch0(x)
 
-#define IXGBE_PCI_REG(reg) (*((volatile uint32_t *)(reg)))
+#define IXGBE_PCI_REG(reg) ({			\
+	uint32_t __val;				\
+	__val = rte_readl(reg);			\
+	__val;					\
+})
 
 static inline uint32_t ixgbe_read_addr(volatile void* addr)
 {
 	return rte_le_to_cpu_32(IXGBE_PCI_REG(addr));
 }
 
-#define IXGBE_PCI_REG_WRITE(reg, value) do { \
-	IXGBE_PCI_REG((reg)) = (rte_cpu_to_le_32(value)); \
-} while(0)
+#define IXGBE_PCI_REG_WRITE(reg, value) ({		\
+	rte_writel(rte_cpu_to_le_32(value), reg);	\
+})
 
 #define IXGBE_PCI_REG_ADDR(hw, reg) \
 	((volatile uint32_t *)((char *)(hw)->hw_addr + (reg)))