[dpdk-dev] [RFC v1 00/11] scapy/python extension

[Xueming(Steven) Li]

Sorry for spamming, something wrong to post patches.
Please refer to https://github.com/steevenlee/dpdk/tree/master_scapy for code detail.

> -----Original Message-----
> From: Xueming Li [mailto:xuemingl at mellanox.com]
> Sent: Tuesday, December 5, 2017 12:55 PM
> To: Wu at dev-r630-06.mtbc.labs.mlnx; Jingjing <jingjing.wu at intel.com>; Harry
> van Haaren <harry.van.haaren at intel.com>
> Cc: Xueming(Steven) Li <xuemingl at mellanox.com>; dev at dpdk.org
> Subject: [RFC v1 00/11] scapy/python extension
> 
> This patch introduced a set of new elements into DPDK:
> 1. python/scapy as a standard dpdk library, disabled by default.
>    Could be used by any application to integrate powerful python features.
> 2. tx, rx and expect, blocking mode testpmd CLI command
>    tx command send packets with scapy syntax template, support range/enum,
> be able to generate multiple flows in one command and DPDK speed.
>    rx command receive packets with timeout, with several choices on how to
> dump packets and mbuf headers.
>    expect command try to send out packet, receive and compare with the
> packet sent, support mbuf header and packet header comparation, show
> detail info or hexdiff if anything different. Expect command is suitable
> for unit test and regression test by saving batch expect scripts into a
> file and use 'load' command to invoke them together.
> 3. pktgen - new testpmd forwarding engine
>   Used to support tx, rx and expect blocking command.
>   Also support rxonly/loopback/forward/macswap non-blocking idle modes
> with several packet dump choice.
> 4. py command - new testpmd CLI
>    Run and evaluate python clause in a global context, samples:
>     - py a = 123; b=0xaaa; hex(a+b)
>     - py eth=Ether();ip=IP();
>     - py hexdump(eth/ip/UDP());
>     - tx 0 eth/ip
>    py shell: sneak into python shell
> 
> The purpose of this scapy/python integration is to help programmers to
> speed up unit test w/o writing complex c code, verify features with batch
> expect scripts. Testpmd already a widely used platform for people to setup
> PMD and verify features with rich CLI command, that's why I choose to make
> new CLI on this familier tool instead of creating a new one.
> 
> I made hundreds of expect test cases when developing rte_flow vxlan and
> GRE features, that makes me pretty confident to touch any code, hope this
> tool help you as well.
> 
> quick guide document:
> 	https://github.com/steevenlee/dpdk/blob/master_scapy/doc/guides/howt
> o/scapy.rst
> github branch:
>         https://github.com/steevenlee/dpdk/tree/master_scapy
> 
> 
> 
> Xueming Li (11):
>   lib/cmdline: support backspace key
>   lib/cmdline: init parse result memeory
>   lib/cmdline: add echo support in batch loading from file
>   app/testpmd: support command echo in CLI batch loading
>   test: update batch loading test
>   lib/python: add embedded python lib
>   app/testpmd: add python command
>   app/testpmd: add pktgen forwarding engine
>   app/testpmd: add pktgen engine scapy commands
>   test/expect: add expect test scripts
>   doc/scapy: add scapy how-to guide
> 
>  app/test-pmd/Makefile                    |    6 +
>  app/test-pmd/cmdline.c                   |   80 ++-
>  app/test-pmd/pktgen.c                    | 1092
> ++++++++++++++++++++++++++++++
>  app/test-pmd/testpmd.c                   |    1 +
>  app/test-pmd/testpmd.h                   |    5 +
>  config/common_base                       |    6 +
>  doc/guides/howto/scapy.rst               |  300 ++++++++
>  lib/Makefile                             |    2 +
>  lib/librte_cmdline/cmdline_parse.c       |    2 +
>  lib/librte_cmdline/cmdline_rdline.c      |    1 +
>  lib/librte_cmdline/cmdline_socket.c      |    5 +-
>  lib/librte_cmdline/cmdline_socket.h      |    3 +-
>  lib/librte_cmdline/cmdline_vt100.c       |    1 +
>  lib/librte_cmdline/cmdline_vt100.h       |    1 +
>  lib/librte_eal/common/include/rte_log.h  |    1 +
>  lib/librte_python/Makefile               |   60 ++
>  lib/librte_python/rte_python.c           |  387 +++++++++++
>  lib/librte_python/rte_python.h           |   71 ++
>  lib/librte_python/rte_python_version.map |   12 +
>  mk/rte.app.mk                            |    1 +
>  test/expect/init.exp                     |   28 +
>  test/expect/rx.exp                       |  134 ++++
>  test/test/test_cmdline_lib.c             |   10 +-
>  23 files changed, 2199 insertions(+), 10 deletions(-)  create mode 100644
> app/test-pmd/pktgen.c  create mode 100644 doc/guides/howto/scapy.rst
> create mode 100644 lib/librte_python/Makefile  create mode 100644
> lib/librte_python/rte_python.c  create mode 100644
> lib/librte_python/rte_python.h  create mode 100644
> lib/librte_python/rte_python_version.map
>  create mode 100644 test/expect/init.exp  create mode 100644
> test/expect/rx.exp
> 
> --
> 2.13.3