Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE]Add ispresent PPL command #518

Open
YANG-DB opened this issue Aug 2, 2024 · 3 comments
Open

[FEATURE]Add ispresent PPL command #518

YANG-DB opened this issue Aug 2, 2024 · 3 comments
Labels
enhancement New feature or request Lang:PPL Pipe Processing Language support

Comments

@YANG-DB
Copy link
Member

YANG-DB commented Aug 2, 2024

Is your feature request related to a problem?
Add ispresent command for PPL spark based execution driver .

Since spark sql doesnt offer such expression a possible option would be:

  • use WHERE spark sql command
    WHERE  "column_name" IS NOT NULL
  • register a custom expression ispresent

Do you have any additional context?

@YANG-DB YANG-DB added enhancement New feature or request untriaged Lang:PPL Pipe Processing Language support and removed untriaged labels Aug 2, 2024
@lukasz-soszynski-eliatra
Copy link
Contributor

I created a working PoC (proof of concept). The source code can be found eliatra#3.

The usage of the commands looks as follows.

search source=people | is_present name;
search source=people | is_present age;
search source=people |  is_present age, name;

Please let me know if such an approach meets your expectations.

BTW, the MR is not yet ready. Documentation and tests need to be created.

@salyh
Copy link
Contributor

salyh commented Sep 10, 2024

We need a final decision wether this should be a command or a function like asked in #512 (comment)

@lukasz-soszynski-eliatra
Copy link
Contributor

ispresent implemented as a function. @YANG-DB the MR is ready for review.

Example: search source=people | eval p=ispresent(age) | where ispresent(age);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Lang:PPL Pipe Processing Language support
Projects
Status: Done
Development

No branches or pull requests

3 participants