From patchwork Fri Mar 17 12:36:40 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hemant Agrawal X-Patchwork-Id: 21857 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id 7DB59D018; Fri, 17 Mar 2017 13:52:05 +0100 (CET) Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-bn3nam01on0062.outbound.protection.outlook.com [104.47.33.62]) by dpdk.org (Postfix) with ESMTP id EF43BCF7A for ; Fri, 17 Mar 2017 13:37:56 +0100 (CET) Received: from BN3PR0301CA0011.namprd03.prod.outlook.com (10.160.180.149) by BLUPR03MB181.namprd03.prod.outlook.com (10.255.212.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.977.11; Fri, 17 Mar 2017 12:37:55 +0000 Received: from BY2FFO11FD009.protection.gbl (2a01:111:f400:7c0c::186) by BN3PR0301CA0011.outlook.office365.com (2a01:111:e400:4000::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.977.11 via Frontend Transport; Fri, 17 Mar 2017 12:37:55 +0000 Authentication-Results: spf=fail (sender IP is 192.88.168.50) smtp.mailfrom=nxp.com; caviumnetworks.com; dkim=none (message not signed) header.d=none; caviumnetworks.com; dmarc=fail action=none header.from=nxp.com; Received-SPF: Fail (protection.outlook.com: domain of nxp.com does not designate 192.88.168.50 as permitted sender) receiver=protection.outlook.com; client-ip=192.88.168.50; helo=tx30smr01.am.freescale.net; Received: from tx30smr01.am.freescale.net (192.88.168.50) by BY2FFO11FD009.mail.protection.outlook.com (10.1.14.73) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_RSA_WITH_AES_256_CBC_SHA) id 15.1.977.7 via Frontend Transport; Fri, 17 Mar 2017 12:37:54 +0000 Received: from bf-netperf1.idc ([10.232.134.28]) by tx30smr01.am.freescale.net (8.14.3/8.14.0) with ESMTP id v2HCalI9023358; Fri, 17 Mar 2017 05:37:52 -0700 From: Hemant Agrawal To: CC: , , , , , Date: Fri, 17 Mar 2017 18:06:40 +0530 Message-ID: <1489754201-1027-22-git-send-email-hemant.agrawal@nxp.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1489754201-1027-1-git-send-email-hemant.agrawal@nxp.com> References: <1489754201-1027-1-git-send-email-hemant.agrawal@nxp.com> X-EOPAttributedMessage: 0 X-Matching-Connectors: 131342278750847265; (91ab9b29-cfa4-454e-5278-08d120cd25b8); () X-Forefront-Antispam-Report: CIP:192.88.168.50; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(336005)(39840400002)(39410400002)(39450400003)(39860400002)(39400400002)(39850400002)(39380400002)(2980300002)(1110001)(1109001)(339900001)(189002)(199003)(9170700003)(5003940100001)(38730400002)(4326008)(110136004)(36756003)(8936002)(8676002)(50986999)(50226002)(81166006)(33646002)(2351001)(76176999)(105606002)(106466001)(356003)(305945005)(77096006)(5660300001)(53936002)(81156014)(8656002)(54906002)(47776003)(2906002)(6666003)(2950100002)(575784001)(50466002)(86362001)(48376002)(6916009)(104016004)(85426001)(189998001); DIR:OUT; SFP:1101; SCL:1; SRVR:BLUPR03MB181; H:tx30smr01.am.freescale.net; FPR:; SPF:Fail; MLV:ovrnspm; MX:1; A:1; PTR:InfoDomainNonexistent; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BY2FFO11FD009; 1:oN09yTd0QOTROuA/agodPGhKp1Txea5brfH7vlqNHJhPIQplAeBLy32+osV8kJchqmDNp2fkHJftUC1nhxwm9uaYnNDE+nwLjSDo4r3qdL0QwQybP4kOQnUZhfJoEUhpshnDnSQN14c1f4wRR0anwaSq57Phy2MkUT4Sa5+RN37+s4GQxgKvz78nMXm7dCTKkMzu163+OOUvZxVCgIAeEoGyVpv1yH4HS174QbpjLRLpF0YcBBWaOayBDfLToZ3ihUG7g7aUngyWZaLHL+gGNKYweQohygk4yYHVQGJnZlBbh+SaO35hINapxgZBpFciaxPNyYmsPXM/NYWoRfI6MHgNlt08f07NN/WS5j6siWxYzGqDjylSQLgfRs4McNCd5rlHdoToeVbhGJ0h5HVfh02GQKlab2x/ti6mW7WXxu7SD2iGWGwY3yILA2uH7TCnWMsA5BIKy3F3o6UGfXcD0vO768Wbuc850gLXGnIeX5uXCyhaYvWtYHT2la956i4qMTJlymaw6EnmA406L6hByVZT2TIXm9qk1lI5KSC8Jlqd74R9Oh2++J8YCO2z/BuEDshO9h4Loru/3oQ6NJfqSWfP5YI9CdpgD9jZGuQ9Qv2IeqFeXDCJrxzqmFk5BIu62HwLy6FEg62hcS8UtL6HCA== MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 169e5540-eb8d-4bfc-4121-08d46d326f66 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001); SRVR:BLUPR03MB181; X-Microsoft-Exchange-Diagnostics: 1; BLUPR03MB181; 3:GynfMC4nk9KcawcR04UR2w0BJcRduLWOD+cmsJHy+EhZkoe/g3xpMbVP7J0Lm4CxHX4tXvmI994vdoBch1jkFUHCQQ57GNOYZRSZE6u/cMuESXrI5xAg7fC/3bALve5saXRqFKkTnEsUjsCpMZ+4PRrNkaTDzenrISBBlPggsqhM2RTibK9knDPM5mwwVYDeqJ7KkFU5g2gdfRa+ApfWmklHG15H2YmbfNwayxipMYIrt8bTslu3ZdxJyOrXJwDw0rTQnG8g81xTw7qoLg4Nuj1W4c1GWsplR4HFjtuObsPKY+XdwvaR+x4HeWynb6Ri761brXK8plcF0LlzbZjrwkd+ZAr5XBAV+0pK+aE0uJqFYT+A9NUwHjeLJ5N0SWh7; 25:Ki3fy8xHvSE7wGukcbuy4d8rb6jFAELdAhRFvw8xMR7pJbirNCx9fGP1Udf+wfSHwChV9eprTxMW2JUAhDdDo3ISdjJ0OiycDNMa+Ut0VJXrfkJy8aX6B6rGS1s47cmpKkkFv05POhwn5+/d5fHZLCSXgscFkuuPHXqVBV3vSSUHYLs4EeOOJ+kFWs33t7gIR3lghQqmhczZvBsiq6r2T+r4YCqzAD15GH8z/wMsdyxWifktXFCWc63LePhakqZuh5H3PiriOQ6Xxvd1j2wxTT81fT/hJ96RkUH2qfl3vZIaNpfJfNbW4jGCXEpsBbUWUMxnGC2YgzgoePZT4wIjAFFS7vcxslgEE+9fG1sdSNQ/5pHcwtWiUSvlHu2SnT11FKg5J7UtEHU4Lax6TIku/XmacATN3fcGYqPwwuPD0ZW9Ez3RoHC85jyJb0Uripbhi0bPmyY+2WVbWkmRboiydA== X-Microsoft-Exchange-Diagnostics: 1; BLUPR03MB181; 31:HtT2699Rjcx+VJ49vZjDaVdMAR2YeiY8z/7BfaUPkS2KG0o4J38EcdfaHkv8AxiHVJEo8Bib7OQZeU5oTgXPG3ykdRW3Subk5MzvZ5vylcmEkmseAcKqZm8O1qQm2mGbmtgDzke9Q5JfjWEoi28YiihlbqTbQoUqqRx6xCZLVkLzxb1KN7jvuW8YtHwrWoEqpJVtPA6xr9A9aQ4GSB2hKQhSlbj3xL+S7qCoGKtVULwcx+/lMBf3EOXTVkG/cPYrOxnAMQhGcxmt8JLY5o7x+g== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(185117386973197); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6095060)(601004)(2401047)(13024025)(8121501046)(13018025)(5005006)(13023025)(13017025)(13015025)(3002001)(10201501046)(6055026)(6096035)(20161123563025)(20161123559025)(20161123565025)(20161123556025)(20161123561025); SRVR:BLUPR03MB181; BCL:0; PCL:0; RULEID:(400006); SRVR:BLUPR03MB181; X-Microsoft-Exchange-Diagnostics: 1; BLUPR03MB181; 4:0bKjWoa39j8Phr0OE34GWK2YYRBVcWjJr5DQtsVXQIZOiONK0dQqYbritHJj/R5HiR0R45FvNSWGCoekdUHpEGeCXJIscHb5Hzo2rPsIRV3909ogJjx6PyfBBCqjgSxhSI5iQWY6cEYStRE1N1bdNyVGBwhV6SL+JzWCj92ErvppFa9sKhgXbpg0nTzMdeNVkMEYv1IJiKhhfbeBSnco4PSmannMb7TkMK0xNBlL90KmTUpIIqBxL/Pm3jjh8ja7e1XNPhV5KaIoVvmxOHuOJUccz+X9Tq/Oep2fRJEcLRJKjBWS2NjS9/ZG33Sfc+PT5MRRF8dcWRUnzqACAx99Ug+TwkrhfExxjClhhV6S+p24HLnZpcSAS9W4g2l6tLctYlNBUa81MNNXaC83Qa3tWj69cai3dcZ29xV1EIshH1IU73viURwlh2j3kwwzn1dkMNDanJJxwUP8jrgxXkE64IRVaxCWSXxQS1MTvTi3aj0r/XhhZPvImjwsm0dl9qeJDEOc1wC5vEXvYOHlE0k14oyYkv8HDPMbwjFp5TDf9cIT3D/BWD9aOaOMQMbmLEUYKcIGRxdyy6yI0xV1imdPzPrwiOwfpZcwgcYj6u7F0Hm42qvmTjHikA5HHo6kY3pm8e+W1zwl+fICkNk5habAqcb2ZI1BBRy+dplqqhcbEBwbh92+QpKvxNnC5cdcPh/6oHvEHK2tk8bXTOTOok7qmoWx+VQGi0DZeqkSBp3cDEnTPe55uI3dwRJsneEGfxLn X-Forefront-PRVS: 0249EFCB0B X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BLUPR03MB181; 23:zdWGT6j7v63x17gDxKNdRHl7ljOGud/hw58Y9Quung?= vERj3iqhDmHAz7NNkOp6G8rE8gOhG4gIZTEeUZoaH2fYJK+P81MmO05ocPcnMXivqihKSQDWOJfP61waOFPh3m0sBYo8BXmyou9K4K9raQt3eruPpaqHPxfXfcbF1vMZGR7R0ImsZrcxBqA01WHR8pl198zVTgg9MyRpNabVL1o6015qwc0nYbkgQjS1MPXJB2A+w6ewwt16vTu8eZ4cuevrfNJbHy8baj+VZLnjrYwGLdMWXV3/rlWbCH0ymzSs2pi68wqU3A0ccpruOleL2eKzzcrHPnMzLsONo0epAt0gws3h86EDDWsJjx4eJDAqyeHJ+GMI3Wq8eMYMF49vO0xb/RWBRJtZ8DaCbZvcyPkX8+LR49d2x0m/HzIG7+ABJGjWV8OfcnDeU/wxNbxR+2Z6fZkxwbbKqPTA0AH1cC++RyeucPmBT+oK3lBaXrDwbFXv+u//is+Q42RdYBTI5S8btLQY72iUxSBHmmoCz3REWN8IXEGZj/BHPsoTyQ5CsOt1goWAOLY/BmzzECHjgVsk+b0BURUhIuCTIzVgPgfPS3wYKJMIQiBoLXheKyQ4AYN3nS1v/FjbJ8OmbBo3x3TMR+/XCF+2XSpqOaRE8J/HbSbzBspMQTyPIwn2+7fHIAGjvfsNyBiwA6pzG24tyWdtuIDQ1OgN0wXB21Qv7NPFw3k640d2uo0IF6t3SUjaHkbirtKxS5y3GJLBQ0boh5PF+tWNQ/obVm8P+yoCgO00fCyai1Awl8bH5sjq52JdA7ASxgqXh2YBg8e4afpFyAAr2JbchB+I5/T1JQ+GW+QLx4HnNW4bdtla2UGZTHH9b2yBvJof5P8pVYdvzuB8RYsmPAJcbNH5X/WDBkwEWt/HEFHc40C2bHpuLlFeRFw9w5pehvrztYb6ub3ysf/SJxiAvHTnJmsmQg1eBf8gAhLyrwL4bn3coLc6hY/R0ewGgJf8+PJ4PwMJgATZnoFk5sMQzjLWaJPICffh24pZKbyjxdW9T/SWBqzI6hYNuzUYlsKw1daBsdKBtjDv7XUmxYh3wpFMbTuJIh9Ht9dZ0ELNABcSHw2pFkVeMTaw6tA0SB/aFmXHf5+bZeObW3FiT4JlqOYIfR9sxsY8xmM10oMSyOzbv1uMTaft29i4oYxEZ6/lqWjfx1Ux8Mg6TELjvb8Cl9uz6xDGDOkM9e30k3ZHCiWtaLDU5E+Y10cfk9I/hDarG5Zo5HPMrdG1ETEcLa X-Microsoft-Exchange-Diagnostics: 1; BLUPR03MB181; 6:Z2mYxE1S805anZdu5zKWIvrvYE+XtxlRWBwC4KsBQeUCKvgbQULNX1SxUcYfl0JfC0i99piPZcs1AvsJGMUCEUS/Fy28MY77159/F2XWWeByCNk97ZgDuWK40uwk+jNo7wg4LtA/4ibe6mSRPOpeXWce3ft4WbYRx/BhEpgfiLEiRqThTJ2ZofS40OQJmwvF17lTkmrRbKe5sGkGZQteQEShadVHnPybinZIx1cqCHHcazQtdniRgI7K5vFGxA6Ilv5LXsaEVVvyrMCr3H9W2nko0M8xk8olZqRggetp+hyPXGbQjBo1Tuvwk2hIG2os8a4586ciaTP/Z48WPEBe796wSx0a7rc9qV/5vfbzcOLwXscxfBwiJR22szO88ZsKnFUtwLQriKe755k+5XEBAerh4c66/xehadytZPYJfbE=; 5:ttUniNZ769qFhoWkUGxcnHDNmI0XBIC1vS28V3JRY+ZKl6j/1cYGOpYOwn4gql+0TiGzQZ7Y3B13ameJlVurOZHfoB6d0z2tujSle5qsh2N93EiLwHrQJ+RIqjoO6Yi654Y4gvPeQViXMWR1V8WDg0hYEzXIzCJMk+4giJeiQ39C9l0CQTOt/USY4qj4XnoO; 24:D4sjhFSIttZm3kvBWx9P27pL8hWMytMe+fy/ApH0JjU95Al4e2xbNVM/Uz/jijUjGHV0V7ebHOt7+kKM2wvB2fWuUT7VRcJZzY06PHfwj84= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BLUPR03MB181; 7:1EloMA+JONNattm0ZueTNaZrFl28+Wi2KJPX6pTRsykSjKdUFxmqTw3xu/ImBq+f9/ZYS6rbwiAT6CQ2M9NfY0gx0L/UX282ORg7khee9eT3SuI2+qjHPhn/uHslPR1SFub8OsQd5sTHm1ft1Wm5eRvZO8h/U3/22mV6z6YhL+m3bpGeuHHSa1Vq+gKlkfXlCTiIQoD+0F8+mGMJGuM4oEa7+GFHmklki+TEfvVE8+YDLY+yBL/Ll8io70KUnhMIJCwotCw9vKCVUa/eZXvovyHxjmNO2sBeGv7kEwIHNvtu0jRFRmxBblv8fdTi9fH5cS1a1YuH0FhgsNwo1MQA/A== X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Mar 2017 12:37:54.8975 (UTC) X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e; Ip=[192.88.168.50]; Helo=[tx30smr01.am.freescale.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR03MB181 Subject: [dpdk-dev] [PATCH v1 21/22] bus/fslmc: add support for DMA mapping for ARM SMMU X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Hemant Agrawal --- drivers/bus/fslmc/fslmc_vfio.c | 96 +++++++++++++++++++++++++++++ drivers/bus/fslmc/fslmc_vfio.h | 1 + drivers/bus/fslmc/rte_bus_fslmc_version.map | 1 + 3 files changed, 98 insertions(+) diff --git a/drivers/bus/fslmc/fslmc_vfio.c b/drivers/bus/fslmc/fslmc_vfio.c index fc017fc..5f1d1b7 100644 --- a/drivers/bus/fslmc/fslmc_vfio.c +++ b/drivers/bus/fslmc/fslmc_vfio.c @@ -76,8 +76,10 @@ static struct fslmc_vfio_group vfio_groups[VFIO_MAX_GRP]; static struct fslmc_vfio_container vfio_containers[VFIO_MAX_CONTAINERS]; static int container_device_fd; +static uint32_t *msi_intr_vaddr; void *(*rte_mcp_ptr_list); static uint32_t mcp_id; +static int is_dma_done; static int vfio_connect_container(struct fslmc_vfio_group *vfio_group) { @@ -147,6 +149,35 @@ static int vfio_connect_container(struct fslmc_vfio_group *vfio_group) return 0; } +static int vfio_map_irq_region(struct fslmc_vfio_group *group) +{ + int ret; + unsigned long *vaddr = NULL; + struct vfio_iommu_type1_dma_map map = { + .argsz = sizeof(map), + .flags = VFIO_DMA_MAP_FLAG_READ | VFIO_DMA_MAP_FLAG_WRITE, + .vaddr = 0x6030000, + .iova = 0x6030000, + .size = 0x1000, + }; + + vaddr = (unsigned long *)mmap(NULL, 0x1000, PROT_WRITE | + PROT_READ, MAP_SHARED, container_device_fd, 0x6030000); + if (vaddr == MAP_FAILED) { + FSLMC_VFIO_LOG(ERR, "Unable to map region (errno = %d)", errno); + return -errno; + } + + msi_intr_vaddr = (uint32_t *)((char *)(vaddr) + 64); + map.vaddr = (unsigned long)vaddr; + ret = ioctl(group->container->fd, VFIO_IOMMU_MAP_DMA, &map); + if (ret == 0) + return 0; + + FSLMC_VFIO_LOG(ERR, "VFIO_IOMMU_MAP_DMA fails (errno = %d)", errno); + return -errno; +} + int vfio_dmamap_mem_region(uint64_t vaddr, uint64_t iova, uint64_t size) @@ -170,6 +201,71 @@ int vfio_dmamap_mem_region(uint64_t vaddr, return 0; } +int rte_fslmc_vfio_dmamap(void) +{ + int ret; + struct fslmc_vfio_group *group; + struct vfio_iommu_type1_dma_map dma_map = { + .argsz = sizeof(struct vfio_iommu_type1_dma_map), + .flags = VFIO_DMA_MAP_FLAG_READ | VFIO_DMA_MAP_FLAG_WRITE, + }; + + int i; + const struct rte_memseg *memseg; + + if (is_dma_done) + return 0; + is_dma_done = 1; + + for (i = 0; i < RTE_MAX_MEMSEG; i++) { + memseg = rte_eal_get_physmem_layout(); + if (memseg == NULL) { + FSLMC_VFIO_LOG(ERR, "Cannot get physical layout."); + return -ENODEV; + } + + if (memseg[i].addr == NULL && memseg[i].len == 0) + break; + + dma_map.size = memseg[i].len; + dma_map.vaddr = memseg[i].addr_64; +#ifdef RTE_LIBRTE_DPAA2_USE_PHYS_IOVA + dma_map.iova = memseg[i].phys_addr; +#else + dma_map.iova = dma_map.vaddr; +#endif + + /* SET DMA MAP for IOMMU */ + group = &vfio_groups[0]; + + if (!group->container) { + FSLMC_VFIO_LOG(ERR, "Container is not connected "); + return -1; + } + + FSLMC_VFIO_LOG(DEBUG, "-->Initial SHM Virtual ADDR %llX", + dma_map.vaddr); + FSLMC_VFIO_LOG(DEBUG, "-----> DMA size 0x%llX\n", dma_map.size); + ret = ioctl(group->container->fd, VFIO_IOMMU_MAP_DMA, + &dma_map); + if (ret) { + FSLMC_VFIO_LOG(ERR, "VFIO_IOMMU_MAP_DMA API" + "(errno = %d)", errno); + return ret; + } + FSLMC_VFIO_LOG(DEBUG, "-----> dma_map.vaddr = 0x%llX", + dma_map.vaddr); + } + + /* TODO - This is a W.A. as VFIO currently does not add the mapping of + * the interrupt region to SMMU. This should be removed once the + * support is added in the Kernel. + */ + vfio_map_irq_region(group); + + return 0; +} + static int64_t vfio_map_mcp_obj(struct fslmc_vfio_group *group, char *mcp_obj) { int64_t v_addr = (int64_t)MAP_FAILED; diff --git a/drivers/bus/fslmc/fslmc_vfio.h b/drivers/bus/fslmc/fslmc_vfio.h index 80c6869..53dd0b7 100644 --- a/drivers/bus/fslmc/fslmc_vfio.h +++ b/drivers/bus/fslmc/fslmc_vfio.h @@ -70,6 +70,7 @@ int vfio_dmamap_mem_region( int fslmc_vfio_setup_group(void); int fslmc_vfio_process_group(void); +int rte_fslmc_vfio_dmamap(void); /* create dpio device */ int dpaa2_create_dpio_device(struct fslmc_vfio_device *vdev, diff --git a/drivers/bus/fslmc/rte_bus_fslmc_version.map b/drivers/bus/fslmc/rte_bus_fslmc_version.map index ed445e2..09c4fac 100644 --- a/drivers/bus/fslmc/rte_bus_fslmc_version.map +++ b/drivers/bus/fslmc/rte_bus_fslmc_version.map @@ -41,6 +41,7 @@ DPDK_17.05 { qbman_swp_send_multiple; rte_fslmc_driver_register; rte_fslmc_driver_unregister; + rte_fslmc_vfio_dmamap; rte_mcp_ptr_list; local: *;