[PATCH] app/dma-perf: fix physical address seg-fault

[Varghese, Vipin]

[AMD Official Use Only - General]

Hi Anoob,

> -----Original Message-----
> From: Anoob Joseph <anoobj at marvell.com>
> Sent: Wednesday, August 16, 2023 11:38 AM
> To: Varghese, Vipin <Vipin.Varghese at amd.com>
> Cc: Yigit, Ferruh <Ferruh.Yigit at amd.com>; cheng1.jiang at intel.com;
> stable at dpdk.org; thomas at monjalon.net; dev at dpdk.org; Jerin Jacob
> Kollanukkaran <jerinj at marvell.com>
> Subject: RE: [PATCH] app/dma-perf: fix physical address seg-fault
>
> [AMD Official Use Only - General]
>
> Caution: This message originated from an External Source. Use proper caution
> when opening attachments, clicking links, or responding.
>
>
> Hi Vipin,
>
> Would using 'rte_pktmbuf_mtod' in 'do_cpu_mem_copy' function resolve the
> issue?

https://patchwork.dpdk.org/project/dpdk/patch/20230816071810.1796-1-vipin.varghese@amd.com/

thanks, that is what we tried and tested.

>
> Thanks,
> Anoob
>
> > -----Original Message-----
> > From: Varghese, Vipin <Vipin.Varghese at amd.com>
> > Sent: Tuesday, August 15, 2023 8:45 PM
> > To: Varghese, Vipin <Vipin.Varghese at amd.com>; thomas at monjalon.net;
> > dev at dpdk.org
> > Cc: Yigit, Ferruh <Ferruh.Yigit at amd.com>; cheng1.jiang at intel.com;
> > stable at dpdk.org
> > Subject: [EXT] RE: [PATCH] app/dma-perf: fix physical address
> > seg-fault
> >
> > External Email
> >
> > ----------------------------------------------------------------------
> > [AMD Official Use Only - General]
> >
> > Apologies, marking this as `NA`. After recheck of this logic without
> > use of ` rte_mbuf_data_iova` will result in mbuf meta-data corruption.
> >
> > Need to fix this in a different way.
> >
> > > -----Original Message-----
> > > From: Vipin Varghese <vipin.varghese at amd.com>
> > > Sent: Tuesday, August 15, 2023 8:41 PM
> > > To: thomas at monjalon.net; dev at dpdk.org
> > > Cc: Yigit, Ferruh <Ferruh.Yigit at amd.com>; cheng1.jiang at intel.com;
> > > stable at dpdk.org
> > > Subject: [PATCH] app/dma-perf: fix physical address seg-fault
> > >
> > > Caution: This message originated from an External Source. Use proper
> > > caution when opening attachments, clicking links, or responding.
> > >
> > >
> > > do_cpu_mem_copy uses DPDK API rte_mbuf_data_iova to return the start
> > > of the virtual address for both src and dst.
> > > But in case of iova mode set as PA, this results in seg-fault.
> > > This is because rte_memcpy VA address and not PA.
> > >
> > > this fix checks the iova mode and invokes rte_memcpy with the right
> > > arguments.
> > >
> > > Bugzilla ID: 1269
> > > Fixes: 623dc9364dc6 ("app/dma-perf: introduce DMA performance test")
> > > Cc: cheng1.jiang at intel.com
> > >
> > > Cc: stable at dpdk.org
> > >
> > > Signed-off-by: Vipin Varghese <vipin.varghese at amd.com>
> > > ---
> > >
> > > tested for both va and pa
> > >
> > > CMD:
> > > PA: dpdk-test-dma-perf --iova-mode=pa  -- --config test.ini
> > > VA: dpdk-test-dma-perf --iova-mode=va  -- --config test.ini
> > > DC: dpdk-test-dma-perf --iova-mode=dc  -- --config test.ini
> > >
> > > Log: fails for dc mode `EAL: invalid parameters for --iova-mode`
> > >
> > > test.ini:
> > > ```
> > > [case1]
> > > type=CPU_MEM_COPY
> > > mem_size=10
> > > buf_size=64,8192,2,MUL
> > > src_numa_node=0
> > > dst_numa_node=0
> > > cache_flush=0
> > > test_seconds=2
> > > lcore = 7
> > > eal_args=--in-memory --no-pci
> > > ```
> > > ---
> > >  app/test-dma-perf/benchmark.c | 36
> > > ++++++++++++++++++++++++++---------
> > >  1 file changed, 27 insertions(+), 9 deletions(-)
> > >
> > > diff --git a/app/test-dma-perf/benchmark.c
> > > b/app/test-dma-perf/benchmark.c index 0601e0d171..73200e1935 100644
> > > --- a/app/test-dma-perf/benchmark.c
> > > +++ b/app/test-dma-perf/benchmark.c
> > > @@ -279,6 +279,10 @@ do_cpu_mem_copy(void *p)
> > >         struct rte_mbuf **srcs = para->srcs;
> > >         struct rte_mbuf **dsts = para->dsts;
> > >         uint32_t i;
> > > +       bool isAddrPaMode = false;
> > > +
> > > +       if (rte_eal_iova_mode() == RTE_IOVA_PA)
> > > +               isAddrPaMode = true;
> > >
> > >         worker_info->stop_flag = false;
> > >         worker_info->ready_flag = true; @@ -286,16 +290,30 @@
> > > do_cpu_mem_copy(void *p)
> > >         while (!worker_info->start_flag)
> > >                 ;
> > >
> > > -       while (1) {
> > > -               for (i = 0; i < nr_buf; i++) {
> > > -                       /* copy buffer form src to dst */
> > > -                       rte_memcpy((void *)(uintptr_t)rte_mbuf_data_iova(dsts[i]),
> > > -                               (void *)(uintptr_t)rte_mbuf_data_iova(srcs[i]),
> > > -                               (size_t)buf_size);
> > > -                       worker_info->total_cpl++;
> > > +       if (true == isAddrPaMode) {
> > > +               while (1) {
> > > +                       for (i = 0; i < nr_buf; i++) {
> > > +                               /* copy buffer form src to dst */
> > > +                               rte_memcpy((void *)dsts[i],
> > > +                                       (void *)srcs[i],
> > > +                                       (size_t)buf_size);
> > > +                               worker_info->total_cpl++;
> > > +                       }
> > > +                       if (worker_info->stop_flag)
> > > +                               break;
> > > +               }
> > > +       } else {
> > > +               while (1) {
> > > +                       for (i = 0; i < nr_buf; i++) {
> > > +                               /* copy buffer form src to dst */
> > > +                               rte_memcpy((void
> > > *)(uintptr_t)rte_mbuf_data_iova(dsts[i]),
> > > +                                       (void *)(uintptr_t)rte_mbuf_data_iova(srcs[i]),
> > > +                                       (size_t)buf_size);
> > > +                               worker_info->total_cpl++;
> > > +                       }
> > > +                       if (worker_info->stop_flag)
> > > +                               break;
> > >                 }
> > > -               if (worker_info->stop_flag)
> > > -                       break;
> > >         }
> > >
> > >         return 0;
> > > --
> > > 2.34.1