Refactor and redesign ImageEditor component

[pngwn]

This is a not working draft of the image editor. I had it half working until i let o3 run wild and then i broke everything.

o3 has done more to set this country back than Margaret Thatcher.

[pngwn]

Anyway you can do two things:

• upload and image and edit it with crop/ resize (its a bit rough, few constraints atm)
• draw

You cannot do both.

My main question is the interplay between crop and resize.

Should crop refocus on the resized image? It probably should since it is an image crop.
How should we handle the menu UIs? I like the contextual menus but I'm not about the alignment, weird atm.

demo/image_editor should give a half working UI.

[pngwn]

The python API won't do anything and I don't expect it to work with gradio right now, just dealing with finalising the UI work.

[gradio-pr-bot]

🪼 branch checks and previews

•	Name	Status	URL
🦄	Changes	detecting...

[gradio-pr-bot]

🦄 change detected

This Pull Request includes changes to the following packages.

Package	Version
@gradio/icons	minor
@gradio/imageeditor	minor
@gradio/upload	minor
gradio	minor

• Maintainers can select this checkbox to manually select packages to update.

With the following changelog entry.

thanks o3

Maintainers or the PR author can modify the PR title to modify this entry.

Something isn't right?

• Maintainers can change the version label to modify the version bump.
• If the bot has failed to detect any changes, or if this pull request needs to update multiple packages to different versions or requires a more comprehensive changelog entry, maintainers can update the changelog file directly.

[abidlabs]

@pngwn I wasn't able to do a lot of testing since the image upload and drawing features are the broken state, even separately for me:

• When uploadn in image, there's a white gap in the top-left corner
• When drawing, eraser icon doesn't work, nor does changing the color of the brush

Should crop refocus on the resized image? It probably should since it is an image crop.

I agree, and just to confirm, cropping can be undo-ed?

How should we handle the menu UIs? I like the contextual menus but I'm not about the alignment, weird atm.

I'm not sure which menu UIs we are referring to? Can you elaborate or share a screenshot?

[pngwn]

Ah, that is slightly more broken than intended but I see the issue locally too.

I'll repost with more clarity shortly.

[hannahblair]

• can the zoom % font colour be the same as the icons? also can it have the | separator on the right?

[pngwn]

@hannahblair I will consider your request.

Jk, yeah I'll change that!

[abidlabs]

Very very cool @pngwn! Just going through and testing, I noticed a few things:

• (1a) In the initial state, the gr.ImageEditor seems to allow both drawing and uploading an image. So for example, when I click on the gr.ImageEditor, I can draw a stroke, and then as soon as I finish that stroke, the upload dialog screen pops open. Drawing should not be enabled in this screen

Screen.Recording.2025-03-07.at.1.56.11.PM.mov

• (1b) I've mentioned this before, but I find the gray upload box inside the the ImageEditor component very stuffy. I understand that it needs to be there after you've uploaded an image or if you are in draw mode, but if in the initial screen, imo it should look stylistically identical to the Image component on the right (no internal boxes):

• (1c) I also don't think the zoom % (or perhaps the entire zoom toolbar) needs to be shown before an image is uploaded. The 32% was a bit confusing to see

• (2a) I'm a little confused by the buttons on the side toolbar. First of all what does this button do?

The alt text says maximize but it seems to have no effect. If it's really maximizing the input image, then I suggest moving it to the zoom toolbar.

• (2b) After I upload an image, the sub-tools change from the various ways to input an image, to the cropping/resizing sub-tools. I found that confusing. I think these two sub-tools should belong to a different tool, so that it's possible to go back and change the background image using any of the original sources.

• (2c) Imo, after you upload an image, it should automatically change to the draw tool since that's the most common workflow: upload an image -> annotate it.

Actually interestingly, after you upload an image right now, the tool doesn't change, but you have the ability to draw on the uploaded image. However, this drawing is very jerky and buggy. Only when you actually click on the draw tool can you draw reliably, see this:

Screen.Recording.2025-03-07.at.2.18.24.PM.mov

(still testing...)

[abidlabs]

(3) nit: initially I was confused when I started drawing because I thought the brush color would be black because of the black dot sub-tool underneath the paint sub-tool. I wonder if we can expose the currently selected mask color in the toolbar?

[abidlabs]

(4) also nit: but the apply icon looks a bit out of place compared to the other icons in the toolbar:

import gradio as gr

with gr.Blocks() as demo:
    with gr.Row():
        ie = gr.ImageEditor()
        io = gr.Image()
    b = gr.Button()
    ie.apply(lambda x: x["composite"], ie, io)

demo.launch()

But I should note that all the events are working great in my tests!

[abidlabs]

Love the erasing UX!

Screen.Recording.2025-03-07.at.2.28.23.PM.mov

[abidlabs]

(5) The cropping and resizing don't apply to the annotations. Is that expected? I would have expected everything to be cropped/resized:

Screen.Recording.2025-03-07.at.2.39.44.PM.mov

(you'll also notice a flaky behavior the first time the cropping tool was used). The code here is just:

import gradio as gr

with gr.Blocks() as demo:
    with gr.Row():
        ie = gr.ImageEditor()
        io = gr.Image()
    b = gr.Button()
    b.click(lambda x: x["composite"], ie, io)

demo.launch()

[abidlabs]

The zooming works great, exactly as expected