wip: Image Cropper

[blessedcoolant]

What type of PR is this? (check all applicable)

• Feature

Have you discussed this change with the InvokeAI team?

• Yes

Have you updated all relevant documentation?

• No

Description

This PR adds Crop Image to the UI. Right click on any image anywhere and access the Crop interface.

The Crop UI allows you to crop and save the cropped version. This is especially handy if you only want to keep some parts of the image but more importantly for stuff like IP Adapter which generally require a 512x512 image.

TODO

• Fix the aspect ratio toggle not synced up with the sliders seamlessly.
• Make the cropped image be updated to a particular slot (like IP Adapter) if the crop image was accessed from that area.

Testing

Uses CropperJS so you will need to do pnpm install once before running to install the required libraries.

95% done.

[psychedelicious]

Had a play, some notes:

• The crop box doesn't seem to reset as expected when closing the modal and cropping another image. The box ends up unexpectedly offset. Once time, the crop box ended up being larger than the available space and I had to use the sliders to resize it. Probably the crop state should reset when the modal opens.
• There are a few things that are not supported by the library natively, but which we might be able to work in by using its imperative API:
  • Snap to grid (e.g. 8px), for both resize and move.
  • Snap crop box to the image boundaries.
  • Hold alt to expand the crop outwards from center.
• I ended up in a situation where lock aspect ratio appeared to be disabled, but the ratio was still locked. Not sure how.
• There are a lot of nice features supported by the library, it would be nice to have some of these. The "crop" drag mode is particularly useful. Also resetting the crop to the original image size and position.
• The preview isn't live. It looks like the library lets you provide an element to render a live preview into. If that doesn't work for some reason, we could also debounce the call to generate the preview data to not thrash the browser.
• The whole output image is passed through redux as a b64 string. This will cause issues with devtools (I killed my FF with a big image). Canvas avoids this by rendering canvas images within the listener. Looks like it's possible to access the cropper instance and do this, for example cropperRef.current?.cropper.getCanvasData(). An easy improvement is to use the action sanitizer to not have the whole image in redux.
• The library is more of a resizer+cropper, but the resizing functionality isn't obvious. I zoomed in and out with my mouse to get a better view and unexpectedly that ended up resizing the image. It doesn't look like there is a way to zoom in without actually resizing the image data.
• As we discussed, we have pretty solid aspect ratio logic via context and hook. It's probably worth it to use that in the cropper so we provide the same user experience. We'd need to use the imperative API accessible by the ref to do this.

All said, I think much of this functionality is fairly straightforward to implement with Konva. Check out this demo that is a good amount of the way towards the needed functionality: https://konvajs.org/docs/select_and_transform/Basic_demo.html

We can make the image a transformable shape with a single rect that acts at the crop box. Using konva we can also support zoom without resizing the image output.