[dts] [RFC] Reorganization of DTS

[Owen Hilyard]

Hello everyone,

Since there was no feedback for a week, the DTS working group has decided
to move ahead with this proposal as written.

Owen Hilyard

On Thu, Aug 5, 2021 at 2:49 PM Owen Hilyard <ohilyard at iol.unh.edu> wrote:

> Hello everyone,
>
> The DTS working group has decided that, conditional upon no issues being
> discovered, DTS should be re-organized into a more standard python
> project format. We have a few reasons for this.
>
> How DTS currently is structured is only possible due to modifications of
> the python interpreter search path. The specific modifications are adding
> all of the directories with python code in them, such as framework, nic,
> and tests, to the search path. This enables an import statement like
> "import logging" to work in a test suite, as opposed to the "import
> framework.logging as logging" that would otherwise be needed. However, this
> change is not detected by most python tooling. As a result, features like
> autocompletion and jump-to-definition are broken in many IDEs, and static
> analysis tools can only do surface-level inspections, which also produce a
> lot of false positives due to undefined variables. There are two core
> reasons the DTS working group has decided to move in this direction.
>
> 1. Developer Convenience
>
> There has been a long-running discussion around making DTS submissions for
> DPDK patches that introduce new features mandatory. We feel that making DTS
> look more like a standard python project would help ease the transition
> period. In addition, as mentioned before, IDE features like autocomplete
> and jump-to-definition would be functional, and we feel that those are very
> useful for learning a codebase. This will also enable any future changes to
> be made with the aid of automated refactoring tools, reducing the risk of
> breaking changes.
>
> 2. Automated Tooling
>
> Most automated tooling also cannot process the additions to the python
> path. Some tools, like formatters, are file-oriented and work perfectly
> fine right now, but others, such as flake8, do not. Due to python's nature
> as a duck typed language, not knowing the base type of something (because
> it can't be imported inside of the tool), means that everything it
> interacts with also has an unknown type. This results in some tooling
> choosing to treat it as always the correct type, and some tooling to treat
> it as though it could be any type in the project. Both of these approaches
> reduce the accuracy of the tooling.
>
>
> What will change:
>
> * The main file, currently located at dts/framework/main.py, will be moved
> to dts/main.py.
> * __init__.py files will be added to all modules to allow for exporting
> the types the modules needs to expose.
> * Most import paths will be updated, it is expected that most imports will
> change from "import logging" to "import framework.logging as logging", to
> avoid needing to change any source code during the reorganization.
>
> What we need from the community:
>
> What the DTS working group needs from the community is to let us know if
> there are other changes that should be made, or if this change would cause
> issues for you. We anticipate that there will not be many issues with this
> reorganization unless there is external tooling attached to DTS that is
> incompatible with a standard python module format.
>
> Owen Hilyard
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mails.dpdk.org/archives/dts/attachments/20210811/138cbeef/attachment.htm>