fix(datadog search): Fix improper handling of conjunctions by the datadog query parser

[gwenaskell]

Summary

The Datadog query parser incorrectly interprets queries mixing AND and OR statements.

For example:

• it interprets A OR NOT B as A AND NOT B
• it interprets A AND B OR C as A AND (B OR C)
• it interprets A OR B AND C as B AND C
• it interprets A OR B C as C
• it interprets A B OR C, as well as A AND B OR C as A

This PR fixes this behavior, while simplifying the parser logic.

Change Type

• Bug fix
• New feature
• Non-functional (chore, refactoring, docs)
• Performance

Is this a breaking change?

• Yes
• No

How did you test this PR?

Added and updated unit tests

Does this PR include user facing changes?

• Yes. Please add a changelog fragment based on
  our guidelines.
• No. A maintainer will apply the "no-changelog" label to this PR.

Checklist

• Our CONTRIBUTING.md is a good starting place.
• If this PR introduces changes to LICENSE-3rdparty.csv, please
  run dd-rust-license-tool write and commit the changes. More details here.
• For new VRL functions, please also create a sibling PR in Vector to document the new function.

References

[bruceg]

Before reviewing, I would argue that, as written above, this is a breaking change, as it could make the behavior of existing queries change.

[bruceg]

This does look like a definite fix and good simplification of the logic. It does have me wondering if it would need to be something recursive, however, to fit all possible cases of nested parentheses. Could you comment on that?

[bruceg]

Does this need to toggle? i.e. is_not = !is_not;

[gwenaskell]

Like, a query containing "NOT NOT"? I just checked, the lexer does not allow it (it fails on "NOT -foo" and interprets NOT NOT foo as NOT "NOT" foo)

[bruceg]

That still makes me nervous, given the prevalence of double-negation in other such expression parsers, but it's good to know it's at least not wrong.

[gwenaskell]

It does have me wondering if it would need to be something recursive, however, to fit all possible cases of nested parentheses. Could you comment on that?

Parentheses are interpreted by the lexer as the beginning of a sub-clause. The parser is already recursive (visit_query -> visit_clause -> visit_query)

[gwenaskell]

@pront could you review this as well?

[pront]

Looks good. Thanks for improving the tests.