Ignore metadata bug

English support for the software AllDup
Post Reply
Evds
Posts: 8
Joined: 06 Sep 2015, 16:37

Ignore metadata bug

Post by Evds » 23 Jan 2020, 01:49

Alldup 4.4.14

Download a file from a Sony A6500 (for example: https://3.img-dpreview.com/files/p/samp ... 051969.jpg).
Make a copy of that file in the same directory and add an IPTC keyword to one of the files.

Start Alldup and search for "file content" and "ignore metadata".
Alldup will not see them as duplicates.

therube
Posts: 186
Joined: 07 Nov 2012, 00:28

Re: Ignore metadata bug

Post by therube » 23 Jan 2020, 09:17

Confirmed - with that file.

Other files I looked at (not from your source link, with purposely changed IPTC data) compared as expected.

(I used IrfanView to write IPTC title to the files I modified.)

User avatar
Administrator
Site Admin
Posts: 3425
Joined: 04 Oct 2004, 18:38
Location: Thailand
Contact:

Re: Ignore metadata bug

Post by Administrator » 23 Jan 2020, 09:34

We could reproduce and fix the problem using your example pic.
The problem are the many & unnecessary white spaces at the end of your file.
The current version of AllDup checks only the last 64 bytes of a jpeg file to find the end marker of the picture stream.
We have extended the search for the end marker to the last 4096 bytes.
Please send me an email if you need a new update with the fix.

Evds
Posts: 8
Joined: 06 Sep 2015, 16:37

Re: Ignore metadata bug

Post by Evds » 24 Jan 2020, 15:43

I know, I have no idea why Sony adds those trailing bytes.
I have photos (unedited, straight out of the camera) with more then 22000 bytes after the EOI marker.
In that case extending the search for the EOI marker to 4096 bytes isn't enough.
Is it possible to make that a configurable field (so I can increase it if necessary) ?

Thanks for investigating this problem !

User avatar
Administrator
Site Admin
Posts: 3425
Joined: 04 Oct 2004, 18:38
Location: Thailand
Contact:

Re: Ignore metadata bug

Post by Administrator » 24 Jan 2020, 20:48

We will extend the search range at the end of the file to 64k bytes.

Evds
Posts: 8
Joined: 06 Sep 2015, 16:37

Re: Ignore metadata bug

Post by Evds » 01 Feb 2020, 00:58

I noticed that after the scan data in the jpeg file (after the SOS and EOI marker) there are some trailing bytes and after those trailing bytes a new image starts (the preview image which is larger than the thumbnail) with a new SOI, SOS and EOI marker and then again some trailing bytes.

User avatar
Administrator
Site Admin
Posts: 3425
Joined: 04 Oct 2004, 18:38
Location: Thailand
Contact:

Re: Ignore metadata bug

Post by Administrator » 01 Feb 2020, 10:28

I guess thats the thumbnail data. See "File format structure" at https://en.wikipedia.org/wiki/JPEG_File ... nge_Format

Post Reply