Skip to content

add run_json to JqProgram#51

Open
williamcotton wants to merge 1 commit intoonelson:mainfrom
williamcotton:main
Open

add run_json to JqProgram#51
williamcotton wants to merge 1 commit intoonelson:mainfrom
williamcotton:main

Conversation

@williamcotton
Copy link

Description:

This PR introduces a new method run_json to JqProgram that allows executing jq filters directly against serde_json::Value inputs, bypassing the overhead of string serialization and parsing.

Currently, the crate requires users to serialize data to a JSON string before execution and parse the resulting string back into JSON. For high-throughput middleware or hot paths, this double-parsing (Rust -> String -> jq-internal -> String -> Rust) is a significant bottleneck.

Changes:

  • Direct Marshaling: Implemented JV::from_serde and JV::to_serde to map serde_json::Value types directly to/from libjq's jv pointers via FFI.
  • New API: Added JqProgram::run_json(&mut self, data: &serde_json::Value) -> Result<serde_json::Value>.
  • Dependency Update: Promoted serde_json from [dev-dependencies] to [dependencies] to support these public API types.
  • Integer Handling: Added logic to detect "whole number" floats returned by libjq (which treats all numbers as f64) and convert them to serde_json Integers. This ensures the output matches standard jq CLI behavior and prevents strict equality checks from failing in downstream Rust code.

Performance:

By eliminating the intermediate string allocation and parsing steps, this implementation significantly reduces CPU usage and memory churn.

  • Latency: Removes the O(n) cost of string parsing for every request.

Safety:

  • Utilized jv_copy during FFI conversions to correctly manage libjq reference counting and prevent double-free or use-after-free errors.
  • Ensured error checking for jv_invalid states is performed correctly at the end of the iteration loop to prevent assertion failures in jv_invalid_has_msg.

Verification:

  • Added unit tests covering basic types, complex nested structures, and deep recursion.
  • Verified that integer inputs (42) are correctly returned as integers (42) rather than floats (42.0).
  • Existing tests pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant