Only applies to images with EXIF orientation tags
Some images contain "Orientation" tags. For example, rotate the image by 90 degrees. This is common for smartphones. The standards regarding parsing of this data sometimes have hidden assumptions and issues. This helps you understand our assumptions and avoid machine learning processing errors.
This is an example of data annotated in a tool that did not handle this correctly:
Raw data orientated differently from annotation data may result in a fatal error
Even if Diffgram processes everything correctly, it's possible your ML library may process it differently, resulting in a fatal error, such as the annotations being 90 degrees rotated to the image.
Given an image may be loaded in any programming language, and some libraries default to respecting EXIF while others don't, you must validate your specific conditions.
Diffgram will attempt to respect the expected default view of the data as per other standard systems.
So for example for portrait images we will aim to render them "right side up" like you expect.
Becuase of the lack of standards in this space, we do not guarantee the metadata will be respected. If you see a bug in Diffgram report it.
Diffgram uses ImageIO. In general ImageIO respects orientation tags. This means a portrait photo will render as expected as a portrait.
Verify the orientation is correct in Diffgram and your ML processes
If you are using data with EXIF tags, such as images:
- Validate Diffgram is treating it the way you expect (e.g. visually looks correct)
- Validate your ML processes are loading it in the way you expect.
Some examples for common libraries and confusions regarding width/height ordering are below.
In the unlikely event there is still issues, you can sending images without orientation tags, then there is no ambiguity.
For example, consider this image EXIF:
We see the image width is 4000 and height is 3000 with an Orientation tag that corresponds to "Rotate 90 CW". Technically this is reading
0x0112 mapping for example to
8. An example of PILs implementation is here..
Most systems handle this correctly. For example here it renders it as expected, doing the rotation "transformation" resulting in width of 3000 and height of 4000.
If you want to try this yourself you can use almost any photo taken from a smartphone.
As you can see this can get confusing, for example in this case the tool rendered the data without the rotation to the annotator, but here for machine learning it presents it with the rotation, resulting in the problem:
In this example, the annotation was shown without the EXIF data respected:
But then other parts of the system did respect it, meaning the annotations didn't match.
If you have this problem, use the Diffgram migration tool to automatically fix it.
As an example, here are common orderings for Numpy and PIL.
Note different libraries order Width and Height differently
Yours may be different so please verify.
|Method Example||0th element||1st element|
|Size (PIL Image)||Width||Height|
Seperate from the use of orientation tags, is that there are different orderings for width and height.
ImageIO reads it correctly by default, no changes required.
import imageio im = imageio.imread("20220323_150411.jpg") print("Correct height, width", im.shape) import matplotlib.pyplot as plt imgplot = plt.imshow(im) plt.show()
One of the ways to verify it is to use matplot lib, eg. to show it:
In PIL, at time of writing, you should likely be using exif_transpose(). This single line fixes it.
from PIL import Image, ImageOps img = Image.open("your_path") fixed_image = ImageOps.exif_transpose(img)
You can see why with these examples:
from PIL import Image, ImageOps literal_path = "20220324_165210.jpg" img = Image.open(literal_path) print("Wrong width, height ", img.size) img.show() fixed_image = ImageOps.exif_transpose(img) print("Correct EXIF width, height ", fixed_image.size) # width, height fixed_image.show()
Wrong width, height (4000, 3000)
Correct EXIF width, height (3000, 4000)
Updated over 1 year ago