Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle skeleton decoding internally #1961

Merged
merged 12 commits into from
Sep 25, 2024

Conversation

roomrys
Copy link
Collaborator

@roomrys roomrys commented Sep 12, 2024

Description

It has become apparent that it will be safer for us to handle (de)serialization of our data structures internally rather than relying on external libraries such as cattr and jsonpickle to do this for us. Without constraining these (de)serialization dependencies (which may also cause other upgrade/dependency constraints elsewhere), we cannot expect that the format of (de)serialization will be backwards compatible within (even the same!) version of SLEAP (albeit installed at different times).

Keeping with the theme above, this PR is the first of many that starts migrating (de)serialization to internal SLEAP code. In this PR, we add a function to replace jsonpickle.decode which is intended to be used on jsonpickle.encoded Skeletons.

Types of changes

  • Bugfix
  • New feature
  • Refactor / Code style update (no logical changes)
  • Build / CI changes
  • Documentation Update
  • Other (explain)

Does this address any currently open issues?

Outside contributors checklist

  • Review the guidelines for contributing to this repository
  • Read and sign the CLA and add yourself to the authors list
  • Make sure you are making a pull request against the develop branch (not main). Also you should start your branch off develop
  • Add tests that prove your fix is effective or that your feature works
  • Add necessary documentation (if appropriate)

Thank you for contributing to SLEAP!

❤️

Summary by CodeRabbit

  • New Features

    • Introduced the SkeletonDecoder class for enhanced decoding of skeleton data and preview images.
    • Added a new JSON file representing the skeletal structure of a fly, improving anatomical modeling capabilities.
    • Introduced a new fixture for testing skeleton data in a dictionary format.
  • Bug Fixes

    • Updated methods for decoding preview images to improve maintainability and performance.
  • Tests

    • Added new tests for the SkeletonDecoder and decoding preview images to ensure functionality.
    • Removed redundant test for decoding preview images from the utility tests.

Copy link

coderabbitai bot commented Sep 12, 2024

Walkthrough

The changes encompass the introduction of a new SkeletonDecoder class for improved decoding of skeleton data, including nodes and edges, while replacing previous decoding methods. The DockWidget class in the GUI has been updated to utilize this new decoder for handling preview images. Additionally, a new JSON file representing a fly's skeletal structure has been added, along with corresponding test fixtures and modifications to existing tests to validate the new functionality.

Changes

Files Change Summary
sleap/gui/widgets/docks.py Updated DockWidget to use SkeletonDecoder.decode_preview_image for decoding preview images.
sleap/skeleton.py Introduced SkeletonDecoder class, which replaces jsonpickle.decode with custom methods for decoding skeleton data, including nodes, edges, and preview images.
sleap/util.py Removed unused imports and the decode_preview_image function, streamlining the codebase.
tests/data/skeleton/fly_skeleton_legs_pystate_dict.json Added a JSON file defining a directed multigraph for a fly's skeletal structure, including nodes and links.
tests/fixtures/skeletons.py Added new fixture fly_legs_skeleton_dict_json for testing, providing a path to the new JSON file.
tests/test_skeleton.py Modified tests to use SkeletonDecoder, added tests for decoding preview images and validating the SkeletonDecoder functionality.
tests/test_util.py Removed the test_decode_preview_image function, which was redundant following the refactor.

Assessment against linked issues

Objective Addressed Explanation
Validate new decoding mechanism
Ensure proper handling of preview images

Possibly related issues

  • None identified.

Possibly related PRs

Suggested labels

MultiView Stack

Poem

In the garden where the data flows,
A rabbit hops where the skeleton grows.
With a decoder bright, it shows the way,
To images and nodes, we cheer and play!
Hopping high, our code's a delight,
With every change, we take flight! 🐇✨


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

codecov bot commented Sep 12, 2024

Codecov Report

Attention: Patch coverage is 98.07692% with 2 lines in your changes missing coverage. Please review.

Project coverage is 75.40%. Comparing base (7ed1229) to head (6a4647d).
Report is 52 commits behind head on develop.

Files with missing lines Patch % Lines
sleap/skeleton.py 98.01% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1961      +/-   ##
===========================================
+ Coverage    73.30%   75.40%   +2.09%     
===========================================
  Files          134      133       -1     
  Lines        24087    24552     +465     
===========================================
+ Hits         17658    18514     +856     
+ Misses        6429     6038     -391     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@roomrys roomrys changed the title Handle skeleton deserialization internally Handle skeleton decoding internally Sep 12, 2024
@roomrys roomrys marked this pull request as ready for review September 12, 2024 23:51
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range and nitpick comments (1)
sleap/skeleton.py (1)

310-313: Consider combining the if branches using logical or operator.

The static analysis hint suggests combining the if branches in the _decode_reduce_dict method using the logical or operator. This would improve readability without changing the functionality.

You can apply this diff to combine the if branches:

-            if reduce_item is None:
-                # Sometimes the reduce list has None values, skip them
-                continue
-            if (
-                "py/type" in reduce_item
-                and reduce_item["py/type"] == "sleap.skeleton.EdgeType"
-            ):
+            if reduce_item is None or (
+                "py/type" in reduce_item and 
+                reduce_item["py/type"] == "sleap.skeleton.EdgeType"
+            ):
Tools
Ruff

310-313: Combine if branches using logical or operator

Combine if branches

(SIM114)

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between e4bb444 and 6a4647d.

Files selected for processing (7)
  • sleap/gui/widgets/docks.py (2 hunks)
  • sleap/skeleton.py (3 hunks)
  • sleap/util.py (0 hunks)
  • tests/data/skeleton/fly_skeleton_legs_pystate_dict.json (1 hunks)
  • tests/fixtures/skeletons.py (1 hunks)
  • tests/test_skeleton.py (4 hunks)
  • tests/test_util.py (0 hunks)
Files not reviewed due to no reviewable changes (2)
  • sleap/util.py
  • tests/test_util.py
Additional context used
Ruff
sleap/skeleton.py

310-313: Combine if branches using logical or operator

Combine if branches

(SIM114)

Additional comments not posted (11)
tests/fixtures/skeletons.py (2)

11-14: LGTM!

The updated docstring clarifies that the JSON file contains skeleton data in tuple format, which improves the understanding of the fixture's purpose.


18-24: Great addition!

The new fly_legs_skeleton_dict_json fixture enhances testing capabilities by providing skeleton data in dictionary format. The docstring clearly explains the purpose and format of the data.

tests/data/skeleton/fly_skeleton_legs_pystate_dict.json (1)

1-1: LGTM!

The addition of this test fixture file is a good practice to ensure the robustness of the skeleton decoding functionality. The JSON structure follows a specific schema to represent the skeleton data and serves as a valid input for testing purposes.

tests/test_skeleton.py (5)

196-198: LGTM!

The change in the decoding method from jsonpickle.decode to SkeletonDecoder.decode is consistent with the expected behavior mentioned in the AI-generated summary. The assertion has been updated accordingly to verify the presence of the "nx_graph" key in the decoded JSON dictionary.


210-212: LGTM!

The code segment correctly tests the behavior of the SkeletonDecoder when is_template is set to True. The assertions verify the presence of the expected keys in the decoded JSON dictionary, which is consistent with the changes mentioned in the AI-generated summary.


226-230: LGTM!

The addition of the test_decode_preview_image function is a valuable test case for verifying the functionality of the newly added decode_preview_image function in the SkeletonDecoder class. The assertion ensures that the decoded preview image is in the expected "RGBA" mode, which is consistent with the changes mentioned in the AI-generated summary.


233-243: LGTM!

The addition of the test_skeleton_decoder function is a valuable test case for verifying that the SkeletonDecoder can handle both tuple and dictionary formats for skeleton data. The assertions ensure that the loaded objects are instances of the Skeleton class and that they match each other, which is consistent with the changes mentioned in the AI-generated summary.


2-6: LGTM!

The addition of the os and SkeletonDecoder import statements is necessary for the changes made in the test suite. The os import is required for file-related operations, and the SkeletonDecoder import is needed for using the newly added SkeletonDecoder class in the test functions. These changes are consistent with the AI-generated summary.

sleap/gui/widgets/docks.py (1)

366-366: Approve the usage of SkeletonDecoder.decode_preview_image.

The change from using the decode_preview_image function to the SkeletonDecoder.decode_preview_image method aligns with the PR objective of internalizing the (de)serialization process. This refactoring helps reduce dependency on external libraries and improves the maintainability of the codebase.

Verify that all usages of decode_preview_image have been replaced with SkeletonDecoder.decode_preview_image in the codebase by running the following script:

Verification successful

Refactoring of decode_preview_image to SkeletonDecoder.decode_preview_image is successful.

All instances of decode_preview_image have been correctly replaced with SkeletonDecoder.decode_preview_image across the codebase, aligning with the refactoring objectives. However, the original decode_preview_image function definition remains in sleap/skeleton.py. Consider reviewing its necessity to ensure it is not redundant.

  • sleap/skeleton.py: Definition of decode_preview_image still exists.
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify all usages of `decode_preview_image` have been replaced.

# Test 1: Search for usages of `decode_preview_image`. Expect: No results.
rg --type python $'decode_preview_image'

# Test 2: Search for usages of `SkeletonDecoder.decode_preview_image`. Expect: At least 1 result.
rg --type python $'SkeletonDecoder.decode_preview_image'

Length of output: 157


Script:

#!/bin/bash
# Description: Verify all usages of `decode_preview_image` have been replaced.

# Test 1: Search for usages of `decode_preview_image`. Expect: No results.
rg 'decode_preview_image'

# Test 2: Search for usages of `SkeletonDecoder.decode_preview_image`. Expect: At least 1 result.
rg 'SkeletonDecoder.decode_preview_image'

Length of output: 767

sleap/skeleton.py (2)

89-377: Great work on introducing the SkeletonDecoder class!

The class is well-structured, provides a clear separation of concerns for decoding different components, and includes helpful docstrings. This custom decoding mechanism is a good step towards internalizing the (de)serialization process within the SLEAP framework.

Tools
Ruff

310-313: Combine if branches using logical or operator

Combine if branches

(SIM114)


1365-1365: LGTM!

The change to use SkeletonDecoder.decode instead of jsonpickle.decode in the from_json method aligns well with the objective of internalizing the decoding process. The modification is straightforward and looks good.

@eberrigan eberrigan mentioned this pull request Sep 24, 2024
11 tasks
@talmo talmo merged commit ab93b9e into develop Sep 25, 2024
9 checks passed
@talmo talmo deleted the liezl/handle-skeleton-deserialization-ourselves branch September 25, 2024 17:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants