feat: Sort regions by area for rectangles, polygons and ellipses

[LouisJULIEN]

Product vision and this feature goal

My use case is during validation/annotation of pre annotated images.
In many cases, I have more than 5 pre-annotated regions of which at least one should be deleted or modified. In my experience, it’s often similar and overlapping regions that require action.

The problem I’m having is selecting the region I want to delete/modify. The two options I have are:

• Using the side panel « regions »
• Clicking directly on the region on the image

Using the side panel is not really useful as the region I want to delete often have many occurrences of the same class. I have no « best guess » better than the class (score and creation date are irrelevant)
Clicking directly on the image is sometime working sometime not (at best working in 50%) as the region I want to select is very often overlapped, be it by a similar region or a bigger region on the image.
Sorting the image by area solves my issue:

• The smallest regions will always be on top of the image when clicking on them. The bigger regions will still be easy to select as they are bigger thus very likely have a not overlapped area.
• It also narrows a lot when using the « region » side panel. When two regions have a similar size, they will be together. Also, the smallest will always come first (or last depending on the sort order)

In a nutshell, sorting by area help selecting region fast, especially when there are overlapping regions or many instances of the same class.

PR fulfills these requirements

• Commit message(s) and PR title follows the format [fix|feat|ci|chore|doc]: TICKET-ID: Short description of change made ex. fix: DEV-XXXX: Removed inconsistent code usage causing intermittent errors
• Tests for the changes have been added/updated (for bug fixes/features)
• Docs have been added/updated (for bug fixes/features)
• Best efforts were made to ensure docs/code are concise and coherent (checked for spelling/grammatical errors, commented out code, debug logs etc.)
• Self-reviewed and ran all changes on a local instance (for bug fixes/features)

Change has impacts in these area(s)

(check all that apply)

• Product design
• Backend (Database)
• Backend (API)
• Frontend

Describe the reason for change

Having surface sorted only by score or date wasn't relevant to me. I wanted to have regions sorted by surface. It maximizes my chances to click on smaller surfaces. Without surface sorting, bigger surfaces tends to be on top of smaller surfaces, making it harder to reach small surface by click

What does this fix?

(if this is a bug fix)

What is the new behavior?

(if this is a breaking or feature change)
In addition to date and score, regions can now be sorted by surface if they are among Rectangle, Polygon or Ellipse.

What is the current behavior?

(if this is a breaking or feature change)
No way to sort regions by surface

What libraries were added/updated?

None

Does this change affect performance?

(if so describe the impacts positive or negative)
It could affect performance when sorting lots of regions by surface. I didn't observe it. It affects performance only when the feature is directly used.

Does this change affect security?

No

What alternative approaches were there?

Click on the region window and hope to find the good one quick

What feature flags were used to cover this change?

None

Does this PR introduce a breaking change?

(check only one)

• Yes, and covered entirely by feature flag(s)
• Yes, and covered partially by feature flag(s)
• No
• Not sure (briefly explain the situation below)

What level of testing was included in the change?

(check all that apply)

• e2e
• integration
• unit

Which logical domain(s) does this change affect?

(for bug fixes/features, be as precise as possible. ex. Authentication, Annotation History, Review Stream etc.)
Regions Sorting

Others

• Icon for Surface sorting isn't up to the level => done
• Surface sorting for Brush hasn't been done, the surface computation is quite an endeavor to implement
• I couldn't add test => I added test content but I can't instantiate classes inside the test. I would appreciate some help here.

[netlify]

👷 Deploy request for label-studio-docs-new-theme pending review.

Visit the deploys page to approve it

Name	Link
🔨 Latest commit	9fa07e6

[netlify]

👷 Deploy request for heartex-docs pending review.

Visit the deploys page to approve it

Name	Link
🔨 Latest commit	9fa07e6

[LouisJULIEN]

(removed changes to web/dist)

[makseq]

Hi, thank you for the PR! I have a question:
What will happen if you work with audio labeling? Will By Surface appear there? It sounds like it's better to rename it to "by type".

[LouisJULIEN]

Thanks for your feedback @makseq
When the surface is not defined, for example for brushes or audio labelling , the surface is considered to be 0. I made this design choice as it seems to keep things simple. It can be changed.

« By Surface » will appear, regardless of the types of the regions.

« By Type » seems misleading to me as regions with a defined surface (e.g. rectangle and ellipses) are sorted regardless of their type but according to their surface. It is illustrated in the screenshot of the initial comment of this PR.

[makseq]

@LouisJULIEN Ah, I misunderstood the surface word. I think it's better to name it as area. What do you think?

[LouisJULIEN]

@makseq fair point, I renamed surface to area

[LouisJULIEN]

Hey @makseq, could you give me an update on the last changes?

[LouisJULIEN]

up @makseq

[LouisJULIEN]

Updates :

1. the area logo is now good enough
2. I created a test but couldn't figure out how to create region object e.g. PolygonRegion. I added the tests data without the instantiation. Could you help me on that one?

[makseq]

I see lots of tests are failed unfortunately.

[LouisJULIEN]

I'm copy pasting a more detailed explanation of this feature. This message was originally posted on Label Studio's Slack

My use case is during validation/annotation of pre annotated images.
In many cases, I have more than 5 pre-annotated regions per image of which at least one should be deleted or modified. In my experience, it’s often similar and overlapping regions that require action.

The problem I’m having is selecting the region I want to delete/modify. The two options I have are:

• using the side panel « regions »
• clicking directly on the region on the image

Using the side panel is not really useful as the region I want to delete often have many occurrences of the same class. I have no « best guess » better than the class (score and creation date are irrelevant).
Clicking directly on the image is sometime working sometime not (at best working in 50%) as the region I want to select is very often overlapped, be it by a similar region or a bigger region on the image.

Sorting the image by area solves my issue:

• the smallest regions will always be on top of the image when clicking on them. The bigger regions will still be easy to select as they are bigger thus very likely have a not overlapped area.
• It also narrows a lot when using the « region » side panel. When two regions have a similar size, they will be together. Also, the smallest will always come first (or last depending on the sort order)

In a nutshell, sorting by area help selecting region fast, especially when there are overlapping regions or many instances of the same class.