Message ID | 20180210094220.16201-1-jerin.jacob@caviumnetworks.com (mailing list archive) |
---|---|
State | Superseded, archived |
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7B9F01B71C; Sat, 10 Feb 2018 10:43:14 +0100 (CET) Received: from NAM03-DM3-obe.outbound.protection.outlook.com (mail-dm3nam03on0089.outbound.protection.outlook.com [104.47.41.89]) by dpdk.org (Postfix) with ESMTP id 8620A1B62B for <dev@dpdk.org>; Sat, 10 Feb 2018 10:43:12 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=QhHh37BYBUhqjskJLSdqbUZId8NNSUCm5FiZ1Xsgs3M=; b=YznOa+HW6O87tBXpDdZIh4EOPZwJCyQbeglNKOIJUOPoqs7iZGGSY98z8Yi4R4E9eMNGAgi+/7E3OaZoPp5bnUFi+j/DNmu9IIH+DE4JdLFBSObScirj0NbtB95jL9V0qD6QK6oPjoWHx7xPvt/qOjzzQG3FO3nmNBpV532BmV8= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Jerin.JacobKollanukkaran@cavium.com; Received: from localhost.localdomain (223.227.37.230) by CY1PR07MB2522.namprd07.prod.outlook.com (10.167.16.13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.485.10; Sat, 10 Feb 2018 09:43:06 +0000 From: Jerin Jacob <jerin.jacob@caviumnetworks.com> To: dev@dpdk.org Cc: Jerin Jacob <jerin.jacob@caviumnetworks.com>, Matan Azrad <matan@mellanox.com>, Thomas Monjalon <thomas@monjalon.net>, Konstantin Ananyev <konstantin.ananyev@intel.com>, Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> Date: Sat, 10 Feb 2018 15:12:20 +0530 Message-Id: <20180210094220.16201-1-jerin.jacob@caviumnetworks.com> X-Mailer: git-send-email 2.16.1 MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [223.227.37.230] X-ClientProxiedBy: MAXPR0101CA0072.INDPRD01.PROD.OUTLOOK.COM (10.174.63.34) To CY1PR07MB2522.namprd07.prod.outlook.com (10.167.16.13) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 955ce27c-0c2b-4a32-94da-08d5706ab251 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(4534165)(7168020)(4627221)(201703031133081)(201702281549075)(5600026)(4604075)(2017052603307)(7153060)(7193020); SRVR:CY1PR07MB2522; X-Microsoft-Exchange-Diagnostics: 1; CY1PR07MB2522; 3:g4SwR+SRlzCjUBarW3LYs40yEuHuF0d6zOr2dxgr1tgj6ygIL4Q0t8z4yNtjnu/LktK3VLWKIQjJbIjaKceYI4BRlKsClMjtae8nyteSQho6d474IbmJ9bd7rmrJ0p+zYtlqbnQWL04cud4jcYB4U5gOQ18I161X/Zqp410grJJOi3xgnvXU99rEdvzj8WrsjYZaK6OcrI/eZAZNlNvQO62d+8ZtuhDvaKO+fj5Wd9g9UNrFphNHmRvQos1IsR2O; 25:hsyk5vmY8OLYZwBKW9E01IDi4iCXNsb+awljcsorLZFmpxYSx6m/CpilKhi6S4lcFH/wQvW5zp+wtuNgGTpmigDJBMGUPclQ6wKvRVJwjWXuaAmeNeifus+BbmO9BdyZOot3D8e8PrBDt7v+UdGSpHJHGXpo5qi7sfsJORm4BxQBDuW0Pm5UlLC06V+pgQ/H2/9+yOCA6hKphjOHlhLH2ndD+Vq3DRDRuMtgRUCGGDpCNJbBOWVxbxPa+y1cHiJ18XeQOhnwuhG3Xg5iJh5p4g6prfeIjJDrQ/hVedK+63tkAlFNtgAmU6dKurqYPVGlBOKHmf0Ovt8Wqm0vQVb32w==; 31:p8PerTaABzrZH8nmadAUcY2rKnDeKfhAh+VsX+9wM0EFBnpuiw92YCdLl2rRP9oshvbrLQPYJR4P7TxlCnv2AM3X3JNuz7HRhSfeUVQq10W70nXgV/c5MpSK4CWQ87vXX/xD+1rkvegy6uV90njz8VU5MFZ8bJdwQKAOLY0+VJtJr/oiuOZhQwrJVrbJj6TOZ42CmCx8+C1BBACLdrqfkqGRCALCN5JXd1P+I1LgNnM= X-MS-TrafficTypeDiagnostic: CY1PR07MB2522: X-Microsoft-Exchange-Diagnostics: 1; CY1PR07MB2522; 20:9j/fRjicn0kLU8e0d9LyMh0ZLMTG7z5Y/0nUlgkbAYyB8krq1LyHArisf+v0AhE9PG7ZAWz564Mu13l82neFP/dUnX8BgGlq/Um20VjsXiZjzqAhzPZ5xlXJKmNmCf17bxBftiSERphtQcz/V7uu1l6srcDsizmsn6j49v0Bpk/t0pCa6k1nHxJ1jIy2XFwr9WIxpWcXxQtaooXQKfY6Z2+eMD1+xpZHKB3s+5X9gn3litcRV3CW1HzquUJnkE1Oh5+6pxxRZ3+aPN7437y0qRZuQh242zAs8fMu77qNfzSyCdcx/ZCSJvBxKjIcpXnjmplqL7uIE+MIU7gHnDvS5m5avY371L7wAXZaGTA8UsqgXEqdCNevMKI3ZL2s/PM1KZsypRr+hnXkHrQJDvdAMZqllBPnsnOCTP7lg2r0lNdL4t7lIevexvq3sBQ5LVB3s4R724Zfr5UoIhLJATklFo6vIOzrmtqyR+PWnBdwQ3fU6pIM3dHomMX9OP9xmshIGEHh0CO0Ui23xyk3IXFyjmf52aoG/uCReH8s6s+v4XBIUAkccWcAVRrY9Z1wqQnJMkdFVhFA5xycbjrWKvZFQ0e8M9aBsaOdZA79TuwadhI= X-Microsoft-Antispam-PRVS: <CY1PR07MB2522EC929CE266DDC30B1BC8E3F10@CY1PR07MB2522.namprd07.prod.outlook.com> X-Exchange-Antispam-Report-Test: UriScan:(60795455431006)(131327999870524)(228905959029699); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040501)(2401047)(8121501046)(5005006)(10201501046)(3231101)(2400082)(944501161)(3002001)(93006095)(6041288)(20161123564045)(20161123560045)(20161123562045)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011); SRVR:CY1PR07MB2522; BCL:0; PCL:0; RULEID:; SRVR:CY1PR07MB2522; X-Microsoft-Exchange-Diagnostics: 1; CY1PR07MB2522; 4:6pc5Jrb6CAImCmQqGL/TBuCTsAMflIMtilLvDPqbXnbDhSJuVzIy36ltqDwFDi+rJk9zsldOetGya7JnPJteaRDvlztvFAlttdOpxP/Mfa74SrwEkv2cTwMDhLabPYXsHgP0g9OF3y9BwMdIvOMk11Bj80FuEIIx2LhfnfVmY2sl07gDTXEjFY60rnsINGfhttl95fv/D7bTcMB4anEYbimhaFOsUzymcFwWjMYCcuExcFf1yB69xmddfZGEw62Bg4eZIjyPHyCpVf8eeK1+YuesupE2TKH2yI7c2J4jApQO0QAw0ZKqAFWpO0EQA/+a8EyiSe7V/IEyzDEtrTtGrzDokPZboujO7UkHrXD9mMuV4SBC6QNe7xtVL0fFgmYC X-Forefront-PRVS: 057906460E X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6069001)(396003)(366004)(376002)(346002)(39860400002)(39380400002)(199004)(189003)(1857600001)(478600001)(2906002)(72206003)(48376002)(6486002)(36756003)(107886003)(4326008)(51416003)(53936002)(6512007)(52116002)(50466002)(16586007)(54906003)(6506007)(68736007)(6916009)(6666003)(42882006)(316002)(386003)(59450400001)(25786009)(8676002)(66066001)(5660300001)(81156014)(81166006)(55236004)(105586002)(7736002)(305945005)(26005)(1076002)(186003)(16526019)(50226002)(106356001)(8936002)(2351001)(97736004)(2361001)(47776003)(6116002)(3846002); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR07MB2522; H:localhost.localdomain; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY1PR07MB2522; 23:fKjlfxD/7pGzGBgG0aUU1zOZF+H3O8fZ3ENG1Ugm7?= 4WnqR7bpnaNk+1c55uvPXqzyxz3xPhLFRlUQaReQx+2RwzyAYDqMG3/CJWqvHIbuGl6BZg1wB5nu4Q5oHSvj4E//CDBt5rWJv08S2N5YAP1bRdXZ5IPMlEdL3WtT8+5iPzghgxnoY9bGmo8tzjHiWTiP8uFPpOy/JGZNqh8OwrJT/odr4gFMYwxJ/Fhlc94IuFdDewn3nSWtrewo9rh1Xvh/5VF3tK5hAXDxyJm5+/GPtzoRt+5Vy4t7T/xegFhmcpv12f7rULyPzGaZ0JlRlGbWnS58ef6+M6kA3qFkjOIDRZQ3zMgrnljtPvQZh7yE6FsFcopVzj1Xnpv4m5aeh/nSAtdO/7EgHaEnKang0FJ7rpuQoOLbGYBrW0FsWqCJXx8znhp34cHV5Z3AFVKc5zNewgepxQxebgly1zk7/cuyCxpNfNwxgYkXzHfFG14TV8gRnG13Nn5i6e26VUwn/DNXrU8+gn534Far9DD+xMtvn50FNGfo+3Rs53p60X5cxkFykdVIcindnCI150PFfNOaITeLfHKhjfHwYUKc5nFaeBphxdR86AV3dELX8SrnCppnYJ1tVtlwdTlV7l+oiZocBL813T0RU1sO/dEe1M6XajGc7AROLhiA3kvTbRclrcdcSDTFqcl324gwdc411j7prj0SBtfYsh8IllB+sWwceQ7SEDrBWmhYeDEBEU/sE17QF9rbw202OVZyecj2PQpb5JXTXwKH7M9xZvEhGE88VGuxbUADVfJ0an1P0pMcQ24UPoGB1p5mpLuGNslw8TV1F02eIMlQ3eznQYnYf4L61NA022YySK5ZcwdDyxfJGiXQuP+IUijgVl1yj33o5EN7QSUZhVECFgVCinz2eMXDsJPfH8OhC4zl1i4A35+2Z0b//qkltr+B2HVLHtz2xdqeHwLta1EOcIw8ehQXBpWSjPkn87+E5Mwe1pXqmO4HIJYTQOyn2bQmD0wLUMpvNV9NfcOMcxjHJHS5iLSR6Op0mjOZuwbQt32+0edM/4dprNEGM5GUxfZ9PSz8o7TfApFswWYxSsnIiBvVG1xZS7AAXiS2FZhQBNkG+zPpXd7TsUELiee0KqFknyIoOHs2CAFDMLDUX7HUBXpSsBkbcX8EkrwstXrfPB0BseNtzSY0cXANuLo62pdDspWtYAb5N26a5DROZEyMhqY6pNw2vWu5mFVM0tsrY0hnhh6cP9/6HndsCOJBuvwERE6Ba82UIgN X-Microsoft-Exchange-Diagnostics: 1; CY1PR07MB2522; 6:wFkLeX03hSMPhr/C5M8KtUGCVp2XPrLzPjwdGvLNuGpW6bUXNvpAqzBDepspFUu51Ni4iAVWebgbjD+PZQZPXz+etH3/AKWdkIktGLBJLNNhg5k1UJsnQhcUnKs8c1Fh11r3n1RFrritd/WT5eghBptN5rvjjwMA+F6PdFGyoIOR1Eb6JpfZAFKh3d/cYUgf38lbRWUa0jB0Y3w7vQPsRWF+9pUR6QyfAvetHugE8fy1e8QP8OG3jzZ6zi+GdlPcX3B3OKvxY+9I5KObyDvHrfT0dRDR9/cISkFUgl93teD0bTiPZc019iR9pgqGCh1OT8gVu4grauBQZDDC4ItCwnkXPVqJXkjC0h986pVrWDU=; 5:j4rbqrO4nOHgEs6knQuA0CiO3WmD8eW1+FHwtVdLKQx4O/yqoSFSksS04MDTWedGCjz79A+5simKJ8brVOYI2ThSUGEnmi7jntc0Zi41wP8BGNodjq8i3zs8+40FQiaVuEgjvFcv8DZO2rCtHohsi9ah3+2aJaxsAvGSGy1fVnc=; 24:WOowTUe9UR5n+WSlk234rW43MMP5G6EpC4JjKWre7ta03eGuToUZSib0n8+xu5lCfiUCIbCbkHBdY1SAg0fzACodMomIs1hf4ngceDxOoD8=; 7:gHp0BNYYFYj16fnjj25dRwh8LbCKrhwIqTqju6JUpK6mT97Y9jSo6gw6Vw/7b3N4VGFGy1MzcOX2V3lfsPu8445hghCptC06L2x68GGStt9lMQbanxqu/NXGKPB6BHpcwEubdVLSjBBxStKCJHTs6Y/oBQBegsm0xnUckkIxcvZ3y4pskKBSne9KtJq4lBIgH3YTMNmYjFPiPZgR8GrAPHdO7uDnV5wUJy9xZousrcrWpCRD3+kE68UQ+2zNt1E/ SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Feb 2018 09:43:06.8953 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 955ce27c-0c2b-4a32-94da-08d5706ab251 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR07MB2522 Subject: [dpdk-dev] [PATCH] ethdev: fix a regression due to cache alignment issue X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://dpdk.org/ml/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://dpdk.org/ml/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://dpdk.org/ml/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
Checks
Context | Check | Description |
---|---|---|
ci/checkpatch | success | coding style OK |
ci/Intel-compilation | success | Compilation OK |
Commit Message
Jerin Jacob
Feb. 10, 2018, 9:42 a.m. UTC
Prior to "ethdev: add port ownership" change, the fast path structure struct rte_eth_dev->data was cache aligned due to the fact that eth_dev_data was allocated from rte_malloc with cache aligned attribute. "ethdev: add port ownership" change set introduced a rte_eth_dev_shared_data container for port ownership change, This resulted in rte_eth_dev->data memory as cache unaligned. Added a compiler alignment attribute to make sure rte_eth_dev->data always cache aligned and hence compiler can load/store the elements in struct rte_eth_dev_data as naturally aligned. Some platform like thunderx + l3fwd showed 1% regression in the performance with the offending changeset. Fixes: 5b7ba31148a8 ("ethdev: add port ownership") Cc: Matan Azrad <matan@mellanox.com> Cc: Thomas Monjalon <thomas@monjalon.net> Cc: Konstantin Ananyev <konstantin.ananyev@intel.com> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> --- lib/librte_ether/rte_ethdev_core.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Comments
Hi Jerin From: Jerin Jacob,Sent: Saturday, February 10, 2018 11:42 AM > Prior to "ethdev: add port ownership" change, the fast path structure struct > rte_eth_dev->data was cache aligned due to the fact that eth_dev_data was > allocated from rte_malloc with cache aligned attribute. > "ethdev: add port ownership" change set introduced a > rte_eth_dev_shared_data container for port ownership change, This > resulted in rte_eth_dev->data memory as cache unaligned. > Added a compiler alignment attribute to make sure rte_eth_dev->data > always cache aligned and hence compiler can load/store the elements in > struct rte_eth_dev_data as naturally aligned. > > Some platform like thunderx + l3fwd showed 1% regression in the > performance with the offending changeset. > > Fixes: 5b7ba31148a8 ("ethdev: add port ownership") > Are you sure the previous port data allocation(before port ownership) was cache aligned for all the ports for every system (cache line size 16\32\64\128...) or maybe it was only aligned to the first port (port_id=0)? I think that if the answer is no, this fix fixes early patch, or just improvement. > Cc: Matan Azrad <matan@mellanox.com> > Cc: Thomas Monjalon <thomas@monjalon.net> > Cc: Konstantin Ananyev <konstantin.ananyev@intel.com> > > Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com> > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > --- > lib/librte_ether/rte_ethdev_core.h | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/lib/librte_ether/rte_ethdev_core.h > b/lib/librte_ether/rte_ethdev_core.h > index 315b31723..e5681e466 100644 > --- a/lib/librte_ether/rte_ethdev_core.h > +++ b/lib/librte_ether/rte_ethdev_core.h > @@ -601,7 +601,7 @@ struct rte_eth_dev_data { > struct rte_vlan_filter_conf vlan_filter_conf; > /**< VLAN filter configuration. */ > struct rte_eth_dev_owner owner; /**< The port owner. */ -}; > +} __rte_cache_aligned; > > /** > * @internal > -- > 2.16.1
-----Original Message----- > Date: Sat, 10 Feb 2018 18:23:42 +0000 > From: Matan Azrad <matan@mellanox.com> > To: Jerin Jacob <jerin.jacob@caviumnetworks.com>, "dev@dpdk.org" > <dev@dpdk.org> > CC: Thomas Monjalon <thomas@monjalon.net>, Konstantin Ananyev > <konstantin.ananyev@intel.com>, Pavan Nikhilesh > <pbhagavatula@caviumnetworks.com> > Subject: RE: [dpdk-dev] [PATCH] ethdev: fix a regression due to cache > alignment issue > > Hi Jerin Hi Matan, > > From: Jerin Jacob,Sent: Saturday, February 10, 2018 11:42 AM > > Prior to "ethdev: add port ownership" change, the fast path structure struct > > rte_eth_dev->data was cache aligned due to the fact that eth_dev_data was > > allocated from rte_malloc with cache aligned attribute. > > "ethdev: add port ownership" change set introduced a > > rte_eth_dev_shared_data container for port ownership change, This > > resulted in rte_eth_dev->data memory as cache unaligned. > > Added a compiler alignment attribute to make sure rte_eth_dev->data > > always cache aligned and hence compiler can load/store the elements in > > struct rte_eth_dev_data as naturally aligned. > > > > Some platform like thunderx + l3fwd showed 1% regression in the > > performance with the offending changeset. > > > > Fixes: 5b7ba31148a8 ("ethdev: add port ownership") > > > > Are you sure the previous port data allocation(before port ownership) was cache aligned for all the ports for every system (cache line size 16\32\64\128...) or maybe it was only aligned to the first port (port_id=0)? Yes, It was aligned to only for the first port. > > I think that if the answer is no, this fix fixes early patch, or just improvement. I will send v2 as fixes early patch/improvement. > > > Cc: Matan Azrad <matan@mellanox.com> > > Cc: Thomas Monjalon <thomas@monjalon.net> > > Cc: Konstantin Ananyev <konstantin.ananyev@intel.com> > > > > Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com> > > Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com> > > --- > > lib/librte_ether/rte_ethdev_core.h | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/lib/librte_ether/rte_ethdev_core.h > > b/lib/librte_ether/rte_ethdev_core.h > > index 315b31723..e5681e466 100644 > > --- a/lib/librte_ether/rte_ethdev_core.h > > +++ b/lib/librte_ether/rte_ethdev_core.h > > @@ -601,7 +601,7 @@ struct rte_eth_dev_data { > > struct rte_vlan_filter_conf vlan_filter_conf; > > /**< VLAN filter configuration. */ > > struct rte_eth_dev_owner owner; /**< The port owner. */ -}; > > +} __rte_cache_aligned; > > > > /** > > * @internal > > -- > > 2.16.1 >
diff --git a/lib/librte_ether/rte_ethdev_core.h b/lib/librte_ether/rte_ethdev_core.h index 315b31723..e5681e466 100644 --- a/lib/librte_ether/rte_ethdev_core.h +++ b/lib/librte_ether/rte_ethdev_core.h @@ -601,7 +601,7 @@ struct rte_eth_dev_data { struct rte_vlan_filter_conf vlan_filter_conf; /**< VLAN filter configuration. */ struct rte_eth_dev_owner owner; /**< The port owner. */ -}; +} __rte_cache_aligned; /** * @internal