[PATCH v6 1/7] dts: add startup verification and forwarding modes to testpmd shell

[Jeremy Spewock]

On Mon, Jan 8, 2024 at 6:35 AM Juraj Linkeš <juraj.linkes at pantheon.tech>
wrote:

> On Wed, Jan 3, 2024 at 11:33 PM <jspewock at iol.unh.edu> wrote:
> >
> > From: Jeremy Spewock <jspewock at iol.unh.edu>
> >
> > Added commonly used methods in testpmd such as starting and stopping
> > packet forwarding, changing forward modes, and verifying link status of
> > ports so that developers can configure testpmd and start forwarding
> > through the provided class rather than sending commands to the testpmd
> > session directly.
> >
> > Signed-off-by: Jeremy Spewock <jspewock at iol.unh.edu>
> > ---
> >  dts/framework/exception.py                    |   7 +
> >  dts/framework/remote_session/testpmd_shell.py | 149 +++++++++++++++++-
> >  2 files changed, 155 insertions(+), 1 deletion(-)
> >
> > diff --git a/dts/framework/exception.py b/dts/framework/exception.py
> > index 658eee2c38..cce1e0231a 100644
> > --- a/dts/framework/exception.py
> > +++ b/dts/framework/exception.py
> <snip>
> > @@ -65,9 +108,66 @@ class TestPmdShell(InteractiveShell):
> >      _command_extra_chars: ClassVar[str] = "\n"
> >
> >      def _start_application(self, get_privileged_command:
> Callable[[str], str] | None) -> None:
> > -        self._app_args += " -- -i"
> > +        """Overrides :meth:`~.interactive_shell._start_application`.
> > +
> > +        Add flags for starting testpmd in interactive mode and
> disabling messages for link state
> > +        change events before starting the application. Link state is
> verified before starting
> > +        packet forwarding and the messages create unexpected newlines
> in the terminal which
> > +        complicates output collection.
> > +
>
> We should adjust the collection so that it can handle the newlines.
> Also, can you explain exactly why we are disabling the initial link
> state messages?
>

The problem really comes from the newlines causing the prompt to exist in
the buffer before any command is sent. So, what ends up happening is after
starting the application these link state change events happen at some
point, and they cause an empty "testpmd>" line to exist in the buffer and
the next time you send a command it will stop as soon as it encounters that
line. An additional issue with this prompt is it is put in the buffer
before the link state change event occurs, and there is no prompt that
appears after the event messages, just an empty line. This makes it much
more difficult to detect when the link state change event occurs and
consume it because the event isn't captured the next time you collect
output, all that is consumed is a line containing the prompt.. So, this
makes you essentially one command's worth of output behind because the next
time you send a command you will consume what you were supposed to get from
the last command where you stopped early, and this causes false positives
for things like the link state detection method and failures in output
verification.

This puts you in a position where the only way you can really detect that
one of these events happened is either assuming that only getting an empty
prompt means one of these events happened, or trying to consume output
twice and looking ahead to see if one of these events happened. However,
because we wouldn't be doing anything with these events and we verify link
status before starting anyway, it seemed like the less complex but still
functional solution would just be to mask these events.


>
> > +        Also find the number of pci addresses which were allowed on the
> command line when the app
> > +        was started.
> > +        """
> > +        self._app_args += " -- -i --mask-event intr_lsc"
> > +        self.number_of_ports = self._app_args.count("-a ")
> >          super()._start_application(get_privileged_command)
> >
> > +    def start(self, verify: bool = True) -> None:
> > +        """Start packet forwarding with the current configuration.
> > +
> > +        Args:
> > +            verify: If :data:`True` , a second start command will be
> sent in an attempt to verify
> > +                packet forwarding started as expected.
> > +
> > +        Raises:
> > +            InteractiveCommandExecutionError: If `verify` is
> :data:`True` and forwarding fails to
> > +                start or ports fail to come up.
> > +        """
> > +        self.send_command("start")
> > +        if verify:
> > +            # If forwarding was already started, sending "start" again
> should tell us
> > +            start_cmd_output = self.send_command("start")
> > +            if "Packet forwarding already started" not in
> start_cmd_output:
> > +                self._logger.debug(f"Failed to start packet forwarding:
> \n{start_cmd_output}")
> > +                raise InteractiveCommandExecutionError("Testpmd failed
> to start packet forwarding.")
> > +
> > +            for port_id in range(self.number_of_ports):
> > +                if not self.wait_link_status_up(port_id):
> > +                    raise InteractiveCommandExecutionError(
> > +                        "Not all ports came up after starting packet
> forwarding in testpmd."
> > +                    )
> > +
> > +    def stop(self, verify: bool = True) -> None:
> > +        """Stop packet forwarding.
> > +
> > +        Args:
> > +            verify: If :data:`True` , the output of the stop command is
> scanned to verify that
> > +                forwarding was stopped successfully or not started. If
> neither is found, it is
> > +                considered an error.
> > +
> > +        Raises:
> > +            InteractiveCommandExecutionError: If `verify` is
> :data:`True` and the command to stop
> > +                forwarding results in an error.
> > +        """
> > +        stop_cmd_output = self.send_command("stop")
> > +        if verify:
> > +            if (
> > +                "Done." not in stop_cmd_output
> > +                and "Packet forwarding not started" not in
> stop_cmd_output
> > +            ):
>
> I want to make sure I understand this condition. When none of these
> appear, it's an error. When just "Done." appears, we successfully
> stopped ongoing forwarding and when "Packet forwarding not started"
> appears, we're trying to stop forwarding that didn't start (or isn't
> ongoing - it could've stopped in the meantime)?
> I'm thinking about false failures here (Is there a string that would
> indicate a failure even if one of the strings is printed?) - we're
> basically looking at "not success" instead of looking for strings
> telling us about a failure explicitly. Does the stop command not
> produce such output? Or do we not know all of the failure strings or
> is looking for the above two strings sufficient to rule out false
> failures?
>

You are correct that essentially what I am looking for here is if we
succeeded and else, it's a failure. When I looked through some of the
source code for testpmd from the method stop_packet_forwarding, I didn't
see any explicit error messages other than displaying that there was an
error printing statistics. So this was something where I both didn't know
the error messages but it doesn't look like there are any that are
explicitly printed. In the case of false failures however, the strings I am
detecting are always encountered in our two success cases (forwarding not
currently started and successfully stopped). The "Done." message does also
get printed in the case of statistics failing to print for a core as well.


>
> > +                self._logger.debug(f"Failed to stop packet forwarding:
> \n{stop_cmd_output}")
> > +                raise InteractiveCommandExecutionError("Testpmd failed
> to stop packet forwarding.")
> > +
> >      def get_devices(self) -> list[TestPmdDevice]:
> >          """Get a list of device names that are known to testpmd.
> >
> > @@ -82,3 +182,50 @@ def get_devices(self) -> list[TestPmdDevice]:
> >              if "device name:" in line.lower():
> >                  dev_list.append(TestPmdDevice(line))
> >          return dev_list
> > +
> > +    def wait_link_status_up(self, port_id: int,
> timeout=SETTINGS.timeout) -> bool:
> > +        """Wait until the link status on the given port is "up".
> > +
> > +        Arguments:
> > +            port_id: Port to check the link status on.
> > +            timeout: Time to wait for the link to come up. The default
> value for this
> > +                argument is set using the :option:`-t, --timeout`
> command-line argument
> > +                or the :envvar:`DTS_TIMEOUT` environment variable.
> > +
>
> This really should be "may be modified", as it is optional.
>

Good catch, I'll edit this.


>
> > +        Returns:
> > +            Whether the link came up in time or not.
> > +        """
> > +        time_to_stop = time.time() + timeout
> > +        port_info: str = ""
> > +        while time.time() < time_to_stop:
> > +            port_info = self.send_command(f"show port info {port_id}")
> > +            if "Link status: up" in port_info:
> > +                break
> > +            time.sleep(0.5)
> > +        else:
> > +            self._logger.error(f"The link for port {port_id} did not
> come up in the given timeout.")
> > +        return "Link status: up" in port_info
> > +
> > +    def set_forward_mode(self, mode: TestPmdForwardingModes, verify:
> bool = True):
> > +        """Set packet forwarding mode.
> > +
> > +        Args:
> > +            mode: The forwarding mode to use.
> > +            verify: If :data:`True` the output of the command will be
> scanned in an attempt to
> > +                verify that the forwarding mode was set to `mode`
> properly.
> > +
> > +        Raises:
> > +            InteractiveCommandExecutionError: If `verify` is
> :data:`True` and forwarding mode fails
>
> I think there should be a definite article here - the forwarding mode.
>

Good catch, I'll change this too.


>
> > +                to update.
> > +        """
> > +        set_fwd_output = self.send_command(f"set fwd {mode.value}")
> > +        if f"Set {mode.value} packet forwarding mode" not in
> set_fwd_output:
> > +            self._logger.debug(f"Failed to set fwd mode to
> {mode.value}:\n{set_fwd_output}")
> > +            raise InteractiveCommandExecutionError(
> > +                f"Test pmd failed to set fwd mode to {mode.value}"
> > +            )
> > +
> > +    def close(self) -> None:
> > +        """Overrides :meth:`~.interactive_shell.close`."""
> > +        self.send_command("quit", "")
> > +        return super().close()
> > --
> > 2.43.0
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mails.dpdk.org/archives/dev/attachments/20240108/1bf1ac0b/attachment-0001.htm>