[dpdk-dev] Beyond DPDK 2.0

Neil Horman nhorman at tuxdriver.com
Tue Apr 28 19:26:18 CEST 2015


On Mon, Apr 27, 2015 at 08:46:01AM -0500, Jay Rolette wrote:
> On Sat, Apr 25, 2015 at 7:10 AM, Neil Horman <nhorman at tuxdriver.com> wrote:
> 
> > On Fri, Apr 24, 2015 at 02:55:33PM -0500, Jay Rolette wrote:
> > > On Fri, Apr 24, 2015 at 1:51 PM, Neil Horman <nhorman at tuxdriver.com>
> > wrote:
> > >
> > > > So, I hear your arguments, and its understandable that you might not
> > want
> > > > a GPL
> > > > licensed product, given that the DPDK is a library (though I'm not sure
> > > > what the
> > > > aversion to LGPL would be).  Regardless, I think this conversation is a
> > > > bit more
> > > > about participation than license choice.  While you are correct, in
> > that
> > > > the
> > > > first step to support (by which I presume you mean participation in the
> > > > community) is use, the goal here is to get people contributing patches
> > and
> > > > helping increase the usefulness of DPDK.
> > >
> > >
> > > > Given that DPDK is primarily licensed as BSD now, whats preventing
> > you, or
> > > > what
> > > > would encourage you to participate in the community?  I see emails from
> > > > infiniteio addresss in the archives asking questions and making
> > > > suggestions on
> > > > occasion, but no patches.  What would get you (or others in a simmilar
> > > > situation) to submit those?
> > > >
> > >
> > > 36 hours in the day? :)
> > >
> > > It's not a lot, but we've submitted a couple of small patches. It's
> > mostly
> > > a matter of opportunity. We submit patches as we come across DPDK bugs or
> > > find useful optos.
> > >
> >
> <snip examples>
> 
> >
> > Understand, I'm not trying to single you out here.  I see that you, and
> > others
> > from infiniteio have had some participation in the project, and thats
> > great, and
> > appreciated. I'm more focused on why that level of participation is not
> > higher
> > (not only from you, but from the larger presumed user base in general).  As
> > noted at the start of this thread, one of the goals of this discussion is
> > to
> > find ways to maximize participation in the project, and from you I'm
> > hearing
> > that:
> >
> > 1) you use the dpdk because it lowers maintenence overhead
> > 2) you don't participate more in the project because your product work
> > schedule
> > doesn't allow you to do so.
> >
> > Are those both accurate statements?
> >
> 
> (1) was just my response to Luke about what would motivate a company to
> submit patches if the license didn't require it (GPL vs BSD discussion).
> Maint overhead had little to do with why we decided to use DPDK.
> 
> (2) is certainly true enough, but not really all that big of a factor in
> the reasons why our participation isn't at some higher level. I'd say there
> are two primary factors:
> 
> *New Contributors*
> Dropping a major patch on a project where we have no history would be
> foolish and frustrating. Every open source project has a vibe to it and its
> own way of operating. It normally takes some time to figure that out, but
> for major contributions, it generally involves a level of trust.
> 
> Major code drops or patches aren't generally well received unless it is
> from someone that is known and trusted by the community. Building up that
> trust ("street cred") normally involves participating in smaller, less
> risky ways. Answering questions where you can, small patches to fix bugs,
> possibly reviewing code (although this can be iffy as well), etc.
> 
> This facilitates understanding the process, who the players are and what
> sort of toes you are likely to step on.
> 
> It also gives you time to ramp up on the internals of the code and
> philosophy/goals of the project better. With DPDK, performance is obviously
> more important than portability. Until recently, very little care was given
> to API stability or the impact that has on DPDK apps. Both of those are
> very different approaches than typical and it affects what you might do
> when approaching submitting a patch.
> 
> If you want to build up the number of folks contributing, expect them to
> start small and make sure it actually GOES somewhere.
> 
> The first patch I submitted in mid-December had some quick responses and
> questions back-and-forth, but then stalled on a couple of undocumented
> minor stylistic things (comment style and use of #ifdefs). I never heard
> anything back and 4.5 months later, a simple startup opto is still sitting
> there.
> 
> The second patch I submitted (also mid-December) got no response at all for
> 4 months. I've replied to the feedback that came eventually, but once
> again, no subsequent answers.
> 
> Neither of the patches were important, but the process doesn't exactly
> inspire a strong desire to contribute more.
> 
> *Different Goals*
> I see at least two different sets of people on the mailing list. The heavy
> hitters generally have a product reason for their high level of involvement
> with DPDK. For Intel and 6Wind, the reasons are pretty obvious. Same deal
> with NIC vendors. For large companies, sometimes the reasons are less
> obvious, but supporting certain open source projects can be strategic for
> them for several reasons.
> 
> For this group, improving DPDK itself is important enough to dedicate
> resources to. It's a goal in and of itself, even if it isn't the main goal.
> 
> The other group are what I'd call "DPDK users". They picked DPDK because it
> fit a need they had in their product/project, just like they might pick
> Linux or FreeBSD, Apache, gcc and any number of libraries.
> 
> I'm in that second group. DPDK has been tremendously valuable to me, so I'm
> happy to contribute back where I can, but it isn't my goal. I've got a
> product to build and DPDK is another piece of the puzzle. The same is true
> of Linux and all the various libraries we use.
> 
> The nature of the contributions from those two groups are going to be
> different.
> 


So, thank you for your feedback, I think this is tremendously valuable.  What I
hear you saying here is that while you like the DPDK, you don't feel motivated
to make any major contribution to it, because its not really your focus, is that
about right?  Thinking about that, I would say, thats actually great.  Most
successful projects thrive off of a healthy community of contributors like you.
While we always need people pushing the future of the project forward, we just
as importantly need people making sure all the corner cases are tested via
regular usage, and any rough edges found during that testing is submitted to fix
it.  To that end, let me ask you, do you think that most bugs that you find in
the DPDK you fix, or do you just report them?  Or do you otherwise work around
them?  If its the first answer great.  If its either of the other two, is there
in your mind a way we can increase that conversion rate from finding the bug, to
fixing the bug?


> Can we also assume, for the sake of discussion that you have encountered
> > problems, or desired additions to the DPDK, for which you have implemented
> > your
> > own code in the library that is not contributed back to the DPDK?
> >
> 
> Yes, there are problems we've hit with DPDK that we haven't fed back the
> fixes for. We generally do, but sometimes it isn't clear that what we are
> doing would be considered a general problem. Sometimes we fix things with
> hacks that work for us, but aren't necessarily a good "real" fix that
> should go into DPDK.
> 

So this, this is the very thing I would like to address.  If you were ever
curious, or wondered, let me be clear: If you find a bug, we want to see the
fix you have for it.  Even if its a hack, getting the report and an initial fix
gives us data and a starting point to develop a real fix for the issue.  Thats
mututally beneficial.  You get a more robust, maintained fix, and so does
everyone else.  Plus you get increased recognition as a participant/leader in
the community.  Always, we always want your code, please never be shy about
sharing it.

> On the "desired additions" bit, it's less clear. Is it product specific or
> something that is general purpose DPDK-wise? As a startup company, we don't
> have the luxury yet of being able to spend the time required to refactor
> subsystems/modules so it can be shared. Hopefully we'll succeed to the
> point where those discussions are possible :)
> 
This is an interesting thing to say.  I come from a background in which the
conventional wisdom is that open source is the starting point on the path to
success (in that it allows for leveraging of the minor bug fixes and maintenence
from interested parties, much as you describe yourselves), while the core
developers push larger features forward.  There also exists the equally
prevalent philosophy you describe in which a company has to be successfull,
prior to being able to 'afford' to open source a product so to speak.  I get the
impression that your company belongs more to the latter group.  Do you feel like
that line of thinking is something that is bound to some immutable nature of
your business, or do you feel like discussion, other other education might lead
your, or other companies like yours to see open source as more of an investment?


> In the meantime, we do provide feedback on RFCs that come across the
> mailing list on things where we have experience. Most of that seems to just
> go in the bit bucket unfortunately. There's not a lot of actual *discussion* /
> back-and-forth on things here.
> 
I completely agree here, more conversation the list about proposed new features
would be helpful.

> If that is true, perhaps part of this conversation needs to revolve around
> > the
> > tangible benefits of contributing that code back to the upstream project
> > (the
> > aforementioned reduction of overhead) as well as the intangible (thought
> > leadership as the project develops).  Its been my experience, that these
> > situations often arise because management at a company often places a
> > strong
> > bias on development of their product over participation in the open source
> > portion of it, treating the latter as if they are a customer of it, rather
> > than
> > a participant in it, and it would be my desire to see that bias adjusted
> > so as
> > to allow you greater freedom to participate in the project.
> >
> 
> It should be clear from above that yes, we have a strong bias on
> development of our product. We are a startup with a very finite runway.
> Just the reality that we live in...
> 
Fair enough.

> There are no executives or senior management telling us we can't contribute
> more, so that isn't the case for us. However, it certainly could be at
> other companies.
> 
Agreed, I've worked for several networking startups who all had the mentality
that open source was a 'freebie', only as good as what you can take from it, and
to which we should never give back, for fear of providing others with
competitive advantage.  I'm glad thats not you, though I'm a bit concerned that
there might be others lurking here for whoom it is true, and I would love to
find a way to reach them.


> One thing I'd describe differently as a "DPDK user" is the notion of
> treating it as a customer. Treating it as a customer implies there are
> expectations of (or support contracts for) things being fixed, features
> being added, etc.
> 

Right, bad use of language on my part.  By customer all I meant was that some
users see it as a product for which they have paid $0.  When its broken they
would typically contact their vendor, but since they have none, they understand
that they 'got what they paid for' and work around the various issues.  I only
meant to imply that a shift from that mentality, that of an 'owner', or some
other entitiy that had a reason to help improve the DPDK.

> I believe we treat it like we do any other open source project. If we run
> into problems, we search for answers and if we don't find anything, we ask
> the community in case someone else can help. When others ask for help, we
> help when we can. Our ability to do so grows over time.
> 
> Would you agree to that assessment?  If so, how would you suggest that we,
> > as a
> > community address this, and illustrate the appeal of contribution and
> > participation to your company and the benefits thereof?
> >
> 
> Hopefully my comments above address your questions and what I see as the
> reasons.
> 
They have, thank you for your thoughts!
Neil

> Cheers,
> Jay


More information about the dev mailing list