Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extensive testing for tool's correctness #20

Open
rhl-bthr opened this issue Sep 12, 2020 · 1 comment
Open

Extensive testing for tool's correctness #20

rhl-bthr opened this issue Sep 12, 2020 · 1 comment

Comments

@rhl-bthr
Copy link
Collaborator

The biggest impediment in formally announcing the tool is the lack of confidence in its correctness. The tool has not been tested extensively for all the backends that it supports with a variety of P4 programs. We can find more public domain P4 programs here for testing.

Possibly also need to craft custom P4 programs to test handling of corner cases.

@rhl-bthr rhl-bthr linked a pull request Sep 29, 2020 that will close this issue
@rhl-bthr
Copy link
Collaborator Author

After #34, I felt all but qpipe were straightforward and are not bringing good coverage. Should we instead write our own test cases as we have done in the past?

To build a good testing module, we need to cover all type of DFA transition combinations on the parser, maybe also keeping in mind the optimizations done by p4c.

The type is dependent on - input state, output state, and transition.
An input state can be one or more of -

  • Start state
  • in-built / non in-built
  • can / can-not do some processing before the transition (we know that can-not is buggy)

An output state can be one or more of -

  • Accept state
  • in-built / non-inbuilt

A transition can be -

  • based on multiple fields from different headers (not supported)
  • one field, different header
  • one field, same header
  • multiple fields, same header
  • always
  • default

I don't think these should be too many test cases to build, but I'm not sure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant