EXIF Orientation Assumptions for Images

📘

Only applies to images with EXIF orientation tags

Introduction

Some images contain "Orientation" tags. For example, rotate the image by 90 degrees. This is common for smartphones. The standards regarding parsing of this data sometimes have hidden assumptions and issues. This helps you understand our assumptions and avoid machine learning processing errors.

Why should I care?

This is an example of data annotated in a tool that did not handle this correctly:

❗️

Raw data orientated differently from annotation data may result in a fatal error

Even if Diffgram processes everything correctly, it's possible your ML library may process it differently, resulting in a fatal error, such as the annotations being 90 degrees rotated to the image.

Given an image may be loaded in any programming language, and some libraries default to respecting EXIF while others don't, you must validate your specific conditions.

Our Policy

Diffgram will attempt to respect the expected default view of the data as per other standard systems.
So for example for portrait images we will aim to render them "right side up" like you expect.

Becuase of the lack of standards in this space, we do not guarantee the metadata will be respected. If you see a bug in Diffgram report it.

Diffgram uses ImageIO. In general ImageIO respects orientation tags. This means a portrait photo will render as expected as a portrait.

Actions to take:

👍

Verify the orientation is correct in Diffgram and your ML processes

If you are using data with EXIF tags, such as images:

1. Validate Diffgram is treating it the way you expect (e.g. visually looks correct)
2. Validate your ML processes are loading it in the way you expect.

Some examples for common libraries and confusions regarding width/height ordering are below.
In the unlikely event there is still issues, you can sending images without orientation tags, then there is no ambiguity.

Understanding the EXIF problem

For example, consider this image EXIF:

We see the image width is 4000 and height is 3000 with an Orientation tag that corresponds to "Rotate 90 CW". Technically this is reading 0x0112 mapping for example to 8. An example of PILs implementation is here..

Most systems handle this correctly. For example here it renders it as expected, doing the rotation "transformation" resulting in width of 3000 and height of 4000.

If you want to try this yourself you can use almost any photo taken from a smartphone.

As you can see this can get confusing, for example in this case the tool rendered the data without the rotation to the annotator, but here for machine learning it presents it with the rotation, resulting in the problem:

Failure Example

In this example, the annotation was shown without the EXIF data respected:

But then other parts of the system did respect it, meaning the annotations didn't match.

If you have this problem, use the Diffgram migration tool to automatically fix it.

Shape vs Size

As an example, here are common orderings for Numpy and PIL.

🚧

Note different libraries order Width and Height differently

Yours may be different so please verify.

Method Example	0th element	1st element
Shape (Numpy)	Height	Width
Size (PIL Image)	Width	Height

Seperate from the use of orientation tags, is that there are different orderings for width and height.

Correctly Reading it with ImageIO (Default)

ImageIO reads it correctly by default, no changes required.

Python

import imageio
im = imageio.imread("20220323_150411.jpg")
print("Correct height, width", im.shape)
import matplotlib.pyplot as plt
imgplot = plt.imshow(im)
plt.show()

One of the ways to verify it is to use matplot lib, eg. to show it:

PIL EXIF Transpose Correction

In PIL, at time of writing, you should likely be using exif_transpose(). This single line fixes it.

Python

from PIL import Image, ImageOps
img = Image.open("your_path")
fixed_image = ImageOps.exif_transpose(img)

You can see why with these examples:

Python

from PIL import Image, ImageOps

literal_path = "20220324_165210.jpg"
img = Image.open(literal_path)
print("Wrong width, height  ", img.size)
img.show()

fixed_image = ImageOps.exif_transpose(img)

print("Correct EXIF width, height ", fixed_image.size)  # width, height 
fixed_image.show()

Wrong width, height (4000, 3000)
Correct EXIF width, height (3000, 4000)

Print General EXIF Example Script

Get a full list of your image metadata