Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added support for EXIF orientation transform in read_image for JPEG #8279

Merged
merged 8 commits into from
Mar 5, 2024
Merged

Added support for EXIF orientation transform in read_image for JPEG #8279

merged 8 commits into from
Mar 5, 2024

Conversation

vfdev-5
Copy link
Collaborator

@vfdev-5 vfdev-5 commented Feb 23, 2024 •
edited

Fixes #7977

TODO:

  • fix failing smoke test
  • make public API less BC-breaking as new args seems to be required withtout taking default value.

Notes:

  1. How to get exif data with Pillow:
from PIL import Image
pimg = Image.open("image-with-exif.jpg")

# Binary exif data:
print(pimg.info['exif'])
# b'Exif\x00\x00MM\x00*\x00\x00\x00\x08\x00\n\x01\x0f\x00\x02\x00\x00\x00\x06\x00\x00\x00\x86\x01\x10\x00\x02\x00\x00\x00\n\x00\x00\x00\x8c\x01\x12\x00\x03\x00\x00\x00\x01\x00\x06\x00\x00\x01\x1a\x00\x05\x00\x00\x00\x01\x00\x00\x00\x96\x01\x1b\x00\x05\x00\x00\x00\x01\x00\x00\x0
# 0\x9e\x01(\x00\x03\x00\x00\x00\x01\x00\x02\x00\x00\x011\x00\x02\x00\x00\x00\x07\x00\x00\x00\xa6\x012\x00\x02\x00\x00\x00\x14\x00\x00\x00\xae\x02\x13\x00\x03\x00\x00\x00\x01\x00\x01\x00\x00\x87i\x00\x04\x00\x00\x00\x01\x00\x00\x00\xc2\x00\x00\x07\xc8Apple\x00iPhone XR\x00\x00\
# x00\x00H\x00\x00\x00\x01\x00\x00\x00H\x00\x00\x00\x0113.1.2\x00\x002019:10:23


# Parsed exif data:
exif = pimg.getexif()
for tag, value in exif.items():
    if tag == 274:  # orientation tag
        print(tag, value)

cc @gau-nernst @kero-ly

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Feb 23, 2024 •
edited

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/vision/8279

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit af933cb with merge base bca7a49 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

vfdev-5
vfdev-5 commented Feb 23, 2024
View reviewed changes
torchvision/csrc/io/image/image.cpp Outdated
@@ -23,11 +23,11 @@ static auto registry =
torch::RegisterOperators()
.op("image::decode_png", &decode_png)
.op("image::encode_png", &encode_png)
.op("image::decode_jpeg", &decode_jpeg)
.op("image::decode_jpeg(Tensor data, int mode, bool apply_exif_orientation=False) -> Tensor", &decode_jpeg)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This helps to make the op BC compatible, otherwise apply_exif_orientation becomes required and thus break the old code

NicolasHug
NicolasHug reviewed Mar 4, 2024
View reviewed changes
Copy link
Member

@NicolasHug NicolasHug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a ton for working on this @vfdev-5 ! I made a few comments / questions but this looks great

torchvision/csrc/io/image/cpu/decode_image.cpp Outdated Show resolved Hide resolved
torchvision/csrc/io/image/cpu/exif.h Show resolved Hide resolved
torchvision/csrc/io/image/cpu/decode_jpeg.cpp Outdated Show resolved Hide resolved
torchvision/csrc/io/image/cpu/exif.h Outdated Show resolved Hide resolved
torchvision/csrc/io/image/cpu/decode_jpeg.cpp Outdated Show resolved Hide resolved
torchvision/csrc/io/image/cpu/decode_jpeg.cpp Outdated
size_t num_entry = get_uint16(exif_data_vec, endianness, offset);
offset += 2; // go to start of tag fields
constexpr size_t tiff_field_size = 12;
for (size_t entry = 0; entry < num_entry; entry++) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any risk that this loop can segfault/overflow if the entries are malformed?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get_uint16 can return -1 in case offset + required size for uint16 go outside the actual buffer size, so we can catch it here and return earlier.
I hope the code is safe but I can't say 100% sure. I have to investigate a bit more.

torchvision/csrc/io/image/cpu/decode_jpeg.cpp Outdated Show resolved Hide resolved
torchvision/csrc/io/image/cpu/exif.h Show resolved Hide resolved
NicolasHug
NicolasHug approved these changes Mar 5, 2024
View reviewed changes
Copy link
Member

@NicolasHug NicolasHug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot @vfdev-5 , some minor comments but LGTM

torchvision/csrc/io/image/cpu/exif.h Outdated Show resolved Hide resolved
test/test_image.py Outdated Show resolved Hide resolved
torchvision/csrc/io/image/cpu/exif.h Outdated Show resolved Hide resolved
torchvision/csrc/io/image/cpu/exif.h Show resolved Hide resolved
torchvision/csrc/io/image/cpu/exif.h Outdated Show resolved Hide resolved
@vfdev-5 vfdev-5 merged commit f3298dc into pytorch:main Mar 5, 2024
77 checks passed
@vfdev-5 vfdev-5 deleted the read_image_exif_support branch March 5, 2024 14:14
@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 5, 2024

Hey @vfdev-5!

You merged this PR, but no labels were added.
The list of valid labels is available at https://github.com/pytorch/vision/blob/main/.github/process_commit.py

@vfdev-5 vfdev-5 mentioned this pull request Mar 6, 2024
Merged
facebook-github-bot pushed a commit that referenced this pull request Mar 20, 2024
@NicolasHug @facebook-github-bot
[fbsync] Added support for EXIF orientation transform in read_image f…
7e4fbf7 
…or JPEG (#8279)

Reviewed By: vmoens

Differential Revision: D55062774

fbshipit-source-id: 6a75321d8f86630d54cd15ad7433f71d0735659d
@NicolasHug NicolasHug mentioned this pull request May 31, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@NicolasHug NicolasHug NicolasHug approved these changes

Assignees
No one assigned
Labels
cla signed enhancement module: io
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

torchvision.io.read_image support processing EXIF information in JPEG file
3 participants
@vfdev-5 @NicolasHug @facebook-github-bot