Skip to content

enhancement(match_datadog_query): add is_phrase flag to equals method #1334

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

enhancement(match_datadog_query): add is_phrase flag to equals method #1334

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0a7999e
enhancement(filter): add is_phrase flag to equals method
PSeitz-dd Mar 11, 2025
0b0a095
introduce fn phrase callback
PSeitz-dd Mar 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions src/datadog/filter/filter.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,13 @@ pub trait Filter<V: Debug + Send + Sync + Clone + 'static>: DynClone {
/// Will return `Err` if the query contains an invalid path.
fn equals(&self, field: Field, to_match: &str) -> Result<Box<dyn Matcher<V>>, PathParseError>;

/// Determine whether a field value equals `phrase`.
///
/// # Errors
///
/// Will return `Err` if the query contains an invalid path.
fn phrase(&self, field: Field, phrase: &str) -> Result<Box<dyn Matcher<V>>, PathParseError>;

/// Determine whether a value starts with a prefix.
///
/// # Errors
Expand Down
14 changes: 11 additions & 3 deletions src/datadog/filter/matcher.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,15 +129,23 @@ where

Ok(all(matchers?))
}
QueryNode::AttributeTerm { attr, value }
| QueryNode::QuotedAttribute {
QueryNode::AttributeTerm { attr, value } => {
let matchers: Result<Vec<_>, _> = filter
.build_fields(attr)
.into_iter()
.map(|field| filter.equals(field, value))
.collect();

Ok(any(matchers?))
}
QueryNode::QuotedAttribute {
attr,
phrase: value,
} => {
let matchers: Result<Vec<_>, _> = filter
.build_fields(attr)
.into_iter()
.map(|field| filter.equals(field, value))
.map(|field| filter.phrase(field, value))
.collect();

Ok(any(matchers?))
Expand Down
23 changes: 23 additions & 0 deletions src/stdlib/match_datadog_query.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,29 @@ impl Filter<Value> for VrlFilter {
})
}

fn phrase(
&self,
field: Field,
phrase: &str,
) -> Result<Box<dyn Matcher<Value>>, PathParseError> {
let buf = lookup_field(&field)?;
match field {
// Default fields are compared by word boundary.
Field::Default(_) => {
let re = word_regex(phrase);
Ok(resolve_value(
buf,
Run::boxed(move |value| match value {
Value::Bytes(val) => re.is_match(&String::from_utf8_lossy(val)),
_ => false,
}),
))
}
Comment on lines +173 to +182
Copy link
Member

@bruceg bruceg Mar 17, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is this different from the condition in equals below? It looks word-for-word identical but maybe I missed something. I'd also like to see some unit test cases that demonstrate the different behavior.

Copy link
Author

@PSeitz PSeitz Mar 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is the same, since I didn't know the impact of the change, so currently it's a non-breaking behavior.
Currently the non-phrase behavior is also behaving incorrectly, they both would need to be adjusted to include tokenization (and probably some adaption on the query parser)

I implemented it partially here https://github.com/DataDog/pomsky/pull/79. For full compatibility I'd need to have a closer look on how percolation tokenizes.

// Everything else is matched by string equality.
_ => self.equals(field, phrase),
}
}

fn equals(
&self,
field: Field,
Expand Down
Loading