[dpdk-dev] Future Direction for rte_eth_stats_get()

David Harton (dharton) dharton at cisco.com
Fri Jan 22 15:40:46 CET 2016


Hi Maryam,

I'm not dictating they be re-added (although adding would be nice) but more important I'm trying to express an application view point rather than a driver view point.  

I completely understand how a driver wants to be able to advertise all the stats they want to advertise to help them debug their issues (i.e. xstats).   Yet, I'm very interested in DPDK providing a driver agnostic method of advertising well-defined stats.

For example, what if there was a kind of "stats registry" composed of ID and name.  It would work similar to xtats except instead of advertising strings only the "get" API would return ID/count pairs.  If the application wishes to use the DPDK provided string they can call another API to get the stat string via the ID.  These IDs would be well-defined clearly explaining what the count represent.  This way the strings for counts will be uniform across drivers and also make it clear to the users what the counts truly represent and the application could obtain stats from any driver in a driver agnostic manner.

Just an idea,
Dave

> -----Original Message-----
> From: Tahhan, Maryam [mailto:maryam.tahhan at intel.com]
> Sent: Friday, January 22, 2016 9:18 AM
> To: David Harton (dharton) <dharton at cisco.com>; dev at dpdk.org
> Cc: Olivier MATZ <olivier.matz at 6wind.com>; Van Haaren, Harry
> <harry.van.haaren at intel.com>
> Subject: RE: Future Direction for rte_eth_stats_get()
> 
> Hi David
> 
> + Olivier and Harry as contributors and advisors in the past on the stats
> and stats API.
> 
> So I pulled the rtnl_link_stats64 from http://lxr.free-
> electrons.com/source/include/uapi/linux/if_link.h#L41 and crossed them
> with http://dpdk.org/dev/patchwork/patch/5842/ and
> http://dpdk.org/browse/dpdk/tree/lib/librte_ether/rte_ethdev.h
> 
> 40 /* The main device statistics structure */
>  41 struct rtnl_link_stats64 {
>  42         __u64   rx_packets;             /* total packets received
> */             <MT> is in struct rte_eth_stats
>  43         __u64   tx_packets;             /* total packets transmitted
> */          <MT> is in struct rte_eth_stats
>  44         __u64   rx_bytes;               /* total bytes received
> */                  <MT> is in struct rte_eth_stats
>  45         __u64   tx_bytes;               /* total bytes transmitted
> */               <MT> is in struct rte_eth_stats
>  46         __u64   rx_errors;              /* bad packets received
> */                <MT> I'm working on a patch to distinguish between error
> and drops for struct rte_eth_stats at the moment only drops are reported
> through ierrors and oerrors and we are tied to those field name for
> backward compatibility.
>  47         __u64   tx_errors;              /* packet transmit problems
> */           <MT> Same comment as rx_errors.
>  48         __u64   rx_dropped;             /* no space in linux buffers
> */          <MT> is in struct rte_eth_stats
>  49         __u64   tx_dropped;             /* no space available in linux
> */         <MT> is in struct rte_eth_stats
>  50         __u64   multicast;              /* multicast packets received
> */           <MT> was deprecated in struct rte_eth_stats - but we can
> remove the notice and expose again from drivers we modified
>  51         __u64   collisions; <MT> Not in struct rte_eth_stats - not
> sure it's in xstats either...
>  52
>  53         /* detailed rx_errors: */
>  54         __u64   rx_length_errors;
> <MT> was deprecated in struct rte_eth_stats, is in xstats - but we can
> remove the notice and expose again from drivers we modified
>  55         __u64   rx_over_errors;         /* receiver ring buff overflow
> */    <MT> was never exposed to struct rte_eth_stats  - but is in xstats.
>  56         __u64   rx_crc_errors;          /* recved pkt with crc error
> */         <MT> was deprecated in struct rte_eth_stats, is in xstats - but
> we can remove the notice and expose again from drivers we modified
>  57         __u64   rx_frame_errors;        /* recv'd frame alignment
> error */ <MT> was never exposed to struct rte_eth_stats - but is in
> xstats.
>  58         __u64   rx_fifo_errors;         /* recv'r fifo overrun
> */               <MT> was never in struct rte_eth_stats. Not exposed by
> all drivers
>  59         __u64   rx_missed_errors;       /* receiver missed packet
> */   <MT> was deprecated in struct rte_eth_stats, is in xstats - but we
> can remove the notice
>  60
>  61         /* detailed tx_errors */
>  62         __u64   tx_aborted_errors;
> <MT> was never in struct rte_eth_stats. Not exposed by all drivers
>  63         __u64   tx_carrier_errors;
> <MT> was never in struct rte_eth_stats. Not exposed by all drivers
>  64         __u64   tx_fifo_errors;
> <MT> was never in struct rte_eth_stats. Not exposed by all drivers
>  65         __u64   tx_heartbeat_errors;
> <MT> was never in struct rte_eth_stats. Not exposed by all drivers
>  66         __u64   tx_window_errors;
> <MT> was never in struct rte_eth_stats. Not exposed by all drivers
>  67
>  68         /* for cslip etc */
>  69         __u64   rx_compressed;
> <MT> was never in struct rte_eth_stats. Not exposed by all drivers
>  70         __u64   tx_compressed;
> <MT> was never in struct rte_eth_stats. Not exposed by all drivers
>  71 };
> 
> So what can be enabled again in struct rte_eth_stats  from what was
> already there is the equivalent of:
> * rx_length_errors
> * rx_crc_errors
> * rx_missed_errors - the deprecation notice was removed for this field.
> * multicast
> 
> What should be added in to distinguish between errors and drops. struct
> rte_eth_stats :
>  * rx_errors
>  * tx_errors
> 
> As for the detailed rx errors and tx errors I'm open to feedback from you
> folks as to what should go in and what is too detailed. These weren't in
> struct rte_eth_stats previously, they are available through xstats and are
> uniformly named across the drivers. Oliver + Harry any thoughts?
> 
> David I assume you are looking for all the missing fields to be added?
> 
> Best Regards,
> Maryam
> 
> 
> > -----Original Message-----
> > From: David Harton (dharton) [mailto:dharton at cisco.com]
> > Sent: Friday, January 22, 2016 1:41 PM
> > To: Tahhan, Maryam <maryam.tahhan at intel.com>; dev at dpdk.org
> > Subject: RE: Future Direction for rte_eth_stats_get()
> >
> > Hi Maryam,
> >
> > Thanks for the pointer.  I'll review the convo's.
> >
> > Consider I have an application that uses many(all?) of the DPDK drivers
> > and their netmap counterparts depending on configuration.  The user
> > interface provides a set of I/O stats to help debug I/O issues and that
> > set is the same regardless of driver type.  The set of stats provided
> > matches what linux provides today since netmap existed before dpdk.
> >
> > What I want to avoid is having an application that is driver independent
> > having to become driver dependent interpreting a bunch of strings
> > (from xstats) or worse the layer running at the data plane core that is
> > advertising the API needed by the application parsing those strings
> > because the application cannot change the upper layer.
> >
> > What if instead of passing strings and values a set of stat ids and
> value
> > are returned.  At least this way the application can remain driver
> > agnostic versus having to parse a free form string that likely isn't the
> > same across driver types.
> >
> > Also, why wouldn't rte_eth_stats_get() align with linux which is the
> > defacto standard?  I understand that not every driver may not support
> > every stat but that's ok.  Just return 0 (pause frames, crc errors,
> etc).
> > It's not like the list of linux stats is ever growing or advertising an
> > absurd number of stats that aren't applicable to all drivers.
> >
> > Thanks again,
> > Dave
> >
> > > -----Original Message-----
> > > From: Tahhan, Maryam [mailto:maryam.tahhan at intel.com]
> > > Sent: Friday, January 22, 2016 6:08 AM
> > > To: David Harton (dharton) <dharton at cisco.com>; dev at dpdk.org
> > > Subject: RE: Future Direction for rte_eth_stats_get()
> > >
> > > Hi David
> > > Some of the stats were HW specific rather than generic stats that
> > > should be exposed through rte_eth_stats and were migrated to the
> > xstats API.
> > > http://dpdk.org/ml/archives/dev/2015-June/019915.html. The
> > naming was
> > > also not generic enough to cover some of the drivers and in some
> > cases
> > > what was exposed was only a partial view of the relevant stats.
> > >
> > > They were marked as deprecated to deter people from using them
> > in the
> > > future, but haven't been removed from all the driver
> > implementations yet.
> > > The Registers that remain undeprecated are those considered to be
> > generic.
> > >
> > > Which registers are you particularly interested in that have been
> > > deprecated? Can you elaborate on what you mean by " scenarios
> > where a
> > > static view is expected "
> > >
> > > Thanks in advance.
> > >
> > > Best Regards,
> > > Maryam
> > >
> > >
> > > > -----Original Message-----
> > > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of David
> > Harton
> > > > (dharton)
> > > > Sent: Wednesday, January 20, 2016 5:19 PM
> > > > To: dev at dpdk.org
> > > > Subject: [dpdk-dev] Future Direction for rte_eth_stats_get()
> > > >
> > > > I see that some of the rte_eth_stats have been marked
> > deprecated in
> > > > 2.2 that are returned by rte_eth_stats_get().  Applications that
> > > > utilize any number of device types rely on functions like this one
> > > > to debug I/O issues.
> > > >
> > > > Is there a reason the stats have been deprecated?  Why not keep
> > the
> > > > stats in line with the standard linux practices such as
> > > rtnl_link_stats64?
> > > >
> > > > Note, using rte_eth_xstats_get() does not help for this particular
> > > scenario
> > > > because a common binary API is needed to communicate through
> > various
> > > > layers and also provide a consistent view/meaning to users.  The
> > > > xstats is excellent for debugging device specific scenarios but
> > > > can't
> > > help
> > > > in scenarios where a static view is expected.
> > > >
> > > > Thanks,
> > > > Dave



More information about the dev mailing list