How to copy XMP metadata between JPEG images (again)

How to preserve XMP metadata in JPEG images

What is XMP Metadata?

XMP (Extensible Metadata Platform) is a metadata format introduced by Adobe Systems Inc. In a previous article, we described the necessity to copy XMP metadata from one file to another.

How to read the binary XMP metadata

To avoid formatting problems, unexpected changes to the metadata that can be caused by using external Python packages, it is possible to just copy the binary metadata byte string into the output JPEG file. This can be done very simply and reliably. According to JPEG specification, the metadata is contained in APP1 segment of the image, where APP stands for application data. Furthermore, according to the Adobe specification, XMP metadata is located in the same app segment APP1. Despite being contained in the similar segments, XMP metadata segment can be easily distinguished from the Exif one, since it starts with a link to Adobe http://ns.adobe.com/xap/1.0/. Thus, we do not need to parse all JPEG segments, the only action required to do is to find the link in the binary data:

xmp_start = bytes.find(b'http://ns.adobe.com/xap/1.0/\0')
xmp_start = xmp_start - 4

And the metadata segments starts 4 bytes earlier. First two bytes just denote the APP1 segment but the next two bytes are quite important for us, since they denote the length of XMP segment in the HIGH-LOW order, starting from the length byte itself:

length_bytes = bytes[xmp_start+2:xmp_start+4]
length = int.from_bytes(length_bytes, byteorder='big')
xmp_end = xmp_start+length+2

This allows us to safely extract the XMP bytestring:

if xmp_end > xmp_start > 0:
    xmp_str = bytes[xmp_start: xmp_end]

Note that finding a particular image segment without parcing the whole JPEG is an impossible task. The two bytes that denote the start of a new segment are not unique and can easily be encountered in some other binary data chunk. For example, in the image that we use as an example, one can find two starts of quantization tables that are denoted by FF DB. One of them is an actual start of the quantization table, but another is located in the middle of the Exif metadata:

This means that (in this particular case) without parsing the EXIF data first it is impossible to find the correct start of qunatization table. In general, this means that the link inserted by Adobe makes XMP metadata the only segment (known to us) that can be reliably found without parcing the whole image.

How to write the binary XMP metadata

Writing the XMP metadata is trivial. According to the same JPEG specification, it does not really matter where APP segments are located. The only recommendation that they, as any other metadata, should be as close to the start of the file as possible (so it is easy to find without reading the whole, presumably large, binary file). The first two bytes of a JPEG image denote the start of the image, and we can easily paste the whole binary string right after them:

intermediate_container = BytesIO()
#Save PIL image to bytearray:
img.save(intermediate_container, format = 'JPEG', quality=quality, **save_kwargs)
#Insert XMP metadata into the start of the image:
image_begin = 2
first_part = bytes[:image_begin]
second_part = bytes[image_begin:]
saved_bin_data = first_part + xmp + second_part
#Finally, save image into the file:
with open(file, "wb") as f:
    f.write(saved_bin_data)

Conclusion

With this simple trick we avoid relying on external dependencies needed to parse XMP metadata and the resulting metadata is guaranteed to be in the same format as the input after processing the JPEG image with Celantur software.