RAW Exif IPTC support

kylehase
kylehase's picture

Joined: 2005-09-15
Posts: 38
Posted: Tue, 2005-10-11 05:20

I must say that I love the exif data extraction from jpeg files. I was wondering if there is any consideration to extend this to RAW image files as well? My site has both jpeg and raw files but only jpeg files are showing up in search results.

Thanks,

Kyle

Win2k IIS5 fastCGI
PHP 4.4
mysql 4.1
Drupal 4.6.3 integration

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2005-10-11 07:51

does dcraw support exif? does it preserve exif data when generating jpeg from a raw file?

 
kylehase
kylehase's picture

Joined: 2005-09-15
Posts: 38
Posted: Tue, 2005-10-11 10:21

According to this site
http://pages.quicksilver.net.nz/pepe/d70/Nikon_D70_on_Linux.html

Quote:
Raw conversion via dcraw discards EXIF tags. neftags2jpg transfers them back to the jpg. neftags2jpg uses exiv2 (get version 0.7 or newer).

Unfortunately that tool is only for linux AND only works on Nikon NEF files. I've also found a similar binary for canon but no universal program for all makers or even a windows binary yet.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2005-10-11 10:40

well, of course one could add support for these binaries in g2, but a separate binary for each brand...
you're welcome to code the necessary g2 module or module changes yourself, but a corresponding g2 feature request wouldn't gather much interest (feature vote) i guess and thus wouldn't get implemented anytime soon.

 
Lapinoo
Lapinoo's picture

Joined: 2004-05-08
Posts: 378
Posted: Tue, 2005-10-11 14:08

The problem with RAW files is that all formats are different. Although all stores the information that when processed creates the EXIF part of the JPEG file, it is not stored as an EXIF header, so the Exifier lib can not process RAW files...

Maybe one day, all vendors will rally to DNG, thus making it easier to manage RAW files in Gallery...

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2005-10-11 14:34

well, dcraw already manages to convert a whole spectrum of raw formats to standardized formats. it could be extended to also handle exif data of all the different raw types too. of course that's a major effort, but it would make sense to incorporate such a functionality into dcraw.

 
Lapinoo
Lapinoo's picture

Joined: 2004-05-08
Posts: 378
Posted: Tue, 2005-10-11 14:55

OK.

By the way, Canon, Nikon and others are used to modify the structure of their RAW files each time a new camera is designed... You can see the Adobe Camera RAW plugin gets updated every two months.
If we want to extract all information from RAW files, we might have to update the DCRAW plugin every 2 months...

Plus, there are some encryption issues: you might know that Nikon has decided to encrypt the white balance in NEF files from the D2x. Adobe started by not supporting this attribute, which starked a lot of flaming against Nikon.
As a result we may not be able to display all informations depending on the camera model...

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2005-10-11 15:16

nope, if the dcraw binary does the abstraction without changing the required parameters, and that's likely, then we don't have to change the dcraw module. g2 users who use a brand new camera just depend on a brand new dcraw version.

what's this discussion about? :)

 
kylehase
kylehase's picture

Joined: 2005-09-15
Posts: 38
Posted: Tue, 2005-10-11 17:48

Thanks so much for all the great feedback. I understand that due to manufacutres specs this can be a very difficult task. The fact that dcraw works with so many various raw files is already a huge achievement.

The suggestion that I code this module myself is understandable however I neglected to mention that I'm not a very strong programmer (just one semester of PHP in college and a few small scripts for fun) but I'll take a swing at it anyway.

 
oyarzun

Joined: 2003-11-26
Posts: 6
Posted: Thu, 2005-10-13 02:31

I have submitted a patch that updates the exif module to support raw files that are based on TIFF's such as Nikon's NEF format. I have tested with Nikon D100 NEF files.

http://sourceforge.net/tracker/index.php?func=detail&aid=1280237&group_id=7130&atid=307130

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2005-10-13 11:04

cool, but...

we don't patch exifer. we'll upgrade to exifer 1.5 soon. but we don't alter its source code.
you should submit exifer patches / changes to the exifer project (which is unrelated to G2).

changes to other files than modules/exif/lib/ are of course related to G2. but the files in the lib/ directory aren't.

 
oyarzun

Joined: 2003-11-26
Posts: 6
Posted: Fri, 2005-10-14 01:45

I realize that exifer is an external project, and already submitted a patch to the author.

The patch I submitted to the G2 tracker is exifer 1.5 including a merge of the changes to 1.4 found in the G2. The two files I did not update and merge where the canon and gps maker files. Both those had changes from exifer 1.4 -> 1.5 and from G2 inital entry to cvs and the latest HEAD.

 
kylehase
kylehase's picture

Joined: 2005-09-15
Posts: 38
Posted: Mon, 2005-10-17 07:57

Thanks oyarzun,

Ran the patch and checked that the files did indeed take the changes but it still does not seem to be importing the keywords into gallery. Wonder if it's because I'm using a D70. Let me know if you need a test file.

 
oyarzun

Joined: 2003-11-26
Posts: 6
Posted: Tue, 2005-10-18 12:52

The IPTC keywords are extracted with the JPEG library, which does not work on raw files... I'm thinking about trying to write another EXIF modules that will use a different EXIF library, http://www.ozhiker.com/electronics/pjmt/ which supports both JPEG and TIFF files (which includes some raw formats like NEF). I'll let you know when I have anything usable.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2005-10-18 13:04

oyarzun
i'd rather add the option to the exif module to use the jhead binary instead of the php based exif library.
why? because the jhead binary is more robust and seems to do a better job in general.

 
oyarzun

Joined: 2003-11-26
Posts: 6
Posted: Wed, 2005-10-19 04:00

The only problem I see with jhead is that it seems to only work on JPEG's. Which the problem I am trying to solve is to be able to get EXIF and IPTC data from raw files. Probably one of the best exif tools I have seen is http://www.sno.phy.queensu.ca/~phil/exiftool/ which is a perl module.

 
mindless
mindless's picture

Joined: 2004-01-04
Posts: 8601
Posted: Wed, 2005-11-16 01:45

oyarzun, as you've probably seen on the patch you submitted, I have updated our exif module to use Exifixer 1.5 and incorporated your changes to support image/x-dcraw. If you (or anyone) can provide us with a small test image for our unit tests that would be great!

Another G2 dev (coloredpixels) also likes that perl module, particularly for its ability to write exif. The current plan is to have an admin option for our exif module about which exif provider to use. The default will be exifixer because it is pure php == no new dependency. If you have perl available you will be able to pick exiftool and get exif write support (and possibly better read support..).

 
oyarzun

Joined: 2003-11-26
Posts: 6
Posted: Fri, 2005-11-18 13:09

Mindless, I saw that you incorporated the patch into gallery. I've been busy so I have not had the time to get you a sample nef image. The smallest one I have is probably about 8 megs ... I'm not sure how to get one smaller than that. Should I just submit a nef as an attachment to the patch on SF.net?

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Fri, 2005-11-18 13:31

if sf.net accepts the attachement...
maybe you could upload the nef somewhere and link to it. or ask mindless for his email address.

 
brette

Joined: 2005-11-20
Posts: 8
Posted: Sun, 2005-11-20 20:28

Now how about supporting Canon RAW files? I have a Canon 20D, and I'd love to have the EXIF data show up with the pictures. I'd be happy to help out with the effort to get this to work, but I don't know enough about the architecture right now to know where the changes should be made.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2005-11-20 20:54

did you try the latest nightly snapshot of g2? we're using exifixer 1.5 now, which might work.

 
brette

Joined: 2005-11-20
Posts: 8
Posted: Sun, 2005-11-20 22:46

I'm running a CVS snapshot I grabbed yesterday and (as far as I can tell) it's not working. I've verified that I have the EXIF block enabled for that album. Some pictures were upload with gallery remote, some were uploaded by selecting Add Item, then adding from a web browser. None of them show the EXIF data. For reference, here's my gallery (it's slow: it's on a PII 233 MHz!).

 
brette

Joined: 2005-11-20
Posts: 8
Posted: Sun, 2005-11-20 22:50

Not sure how to verify that I'm running the propper version exifixer, but my EXIF/IPTC module carries version 1.0.3.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2005-11-20 22:55

- are you sure the canon raw file has exif data?
- we can't do anything about it right now. maybe it will be changed when use another lib for exif (e.g. exiftool)

 
brette

Joined: 2005-11-20
Posts: 8
Posted: Sun, 2005-11-20 23:45

Yes, I'm sure the Canon raw file has EXIF data. If I open files in Picasa it is able to display the EXIF data.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2005-11-20 23:49

and other files you add have exif data in g2? are you using the sizelimit module?

 
brette

Joined: 2005-11-20
Posts: 8
Posted: Sun, 2005-11-20 23:55

I have some jpeg files that I shot with my old camera. Those pictures display the EXIF data. For example, this picture of Optimus Prime in a Hawaiian shirt displays EXIF data http://brettandcourtney.com/gen_gallery/v/Halloween2005/0.jpg.html

 
brette

Joined: 2005-11-20
Posts: 8
Posted: Sun, 2005-11-20 23:58

I forgot to answer the other half of your question. No, I am not using the Size Limit module.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Mon, 2005-11-21 00:02

ok then. then it's definitely a problem with exifixer 1.5, which we're using as our exif library. don't expect an improvement before we use exiftool or another alternative, which is maybe in the first months of 2006.

 
brette

Joined: 2005-11-20
Posts: 8
Posted: Mon, 2005-11-21 00:29

Ok. Thanks for all of your very quick replies! I'll keep an eye out.

 
mindless
mindless's picture

Joined: 2004-01-04
Posts: 8601
Posted: Tue, 2005-11-22 15:30

Confirmed.. I ran read_exif_data_raw directly on one of your CR2 files and here is the entire output:

array(3) {
  ["VerboseOutput"]=>
  bool(true)
  ["Errors"]=>
  int(0)
  ["ValidJpeg"]=>
  int(0)
}
 
pwagland

Joined: 2005-03-06
Posts: 6
Posted: Mon, 2005-11-28 21:19
Quote:
we don't patch exifer. we'll upgrade to exifer 1.5 soon. but we don't alter its source code.
you should submit exifer patches / changes to the exifer project (which is unrelated to G2).

As far as I can see at the exifer web site http://www.offsky.com/software/exif/index.php (please correct me if there is a new maintainer) exifer is now unmaintained. What is the plan for dealing with this? Will you accept patches for exifer now, or is another solution being searched for?

The reason that I ask is that I have found a bug in the "Light Source" exif attribute handling.

 
mindless
mindless's picture

Joined: 2004-01-04
Posts: 8601
Posted: Mon, 2005-11-28 23:30

go ahead and post your patch (in the other topic..)
we'll probably keep exifixer around because it is pure php, but we're looking into adding alternative exif providers.

 
mridgwel

Joined: 2007-01-27
Posts: 215
Posted: Thu, 2009-02-12 00:26

I know this is a somewhat ancient post... but, there is an update (V1.7) of the exifier library, now part of ZenPhoto - G2.3 IIRC uses V1.5:

http://www.zenphoto.org/trac/wiki/ExifixerLibrary

I just updated my version of exifier to this version as a test, and now it seems to get the EXIF information out of the raw image whereas before it only worked with JPEGs for my cameras:

1D:
http://www.markridgwell.co.uk/v/ByDate/2008/2008-11-02+Harlow+Fireworks/Fireworks+11.CR2/

20D:
http://www.markridgwell.co.uk/v/ByDate/2007/2007-11-10+Bath+and+surrounding+area/2007-11-10+Bath+Abbey/Abbey+at+night+01.CR2/

_________
Mark

 
gcrawford

Joined: 2009-02-16
Posts: 5
Posted: Mon, 2009-02-16 23:46

Excuse my naivity, but I don't suppose you could tell us how you updated your library?

Did you have to make any other changes to Gallery to get it to read the EXIF information from RAW files?

 
mridgwel

Joined: 2007-01-27
Posts: 215
Posted: Tue, 2009-02-17 00:07

What I did was:

1. Download the zip file from zenphoto.
2. Rename the modules/Exif/lib/exifer to modules/Exif/lib/exifier_Original.
3. Extract the zip to so that the trunk/zp-core/exif folder is at modules/Exif/lib/exif.
4. Rename modules/Exif/lib/exif to modules/Exif/lib/exifer.
5. Rename modules/Exif/lib/exifer/exif.php to modules/Exif/lib/exifer/exif.inc.
6. (optional) Restarted apache - to clear out my php accelerator's cache.

You may want to look at renaming the modules/Exif/lib/exifier/makers/*.php to modules/Exif/lib/exifer/makers/*.inc but this will need modules/Exif/lib/exifer/exif.php to be modified to reference these changed filenames so it is like the original exifer library.

_________
Mark

 
mridgwel

Joined: 2007-01-27
Posts: 215
Posted: Fri, 2009-03-06 10:03

I've added this to the patch tracker:

https://sourceforge.net/tracker/index.php?func=detail&aid=2667713&group_id=7130&atid=307130

Installation instructions in the tracker.
_________
Mark

 
eliz82

Joined: 2009-11-06
Posts: 71
Posted: Sun, 2011-09-04 14:03
mridgwel wrote:
I've added this to the patch tracker:

https://sourceforge.net/tracker/index.php?func=detail&aid=2667713&group_id=7130&atid=307130

Installation instructions in the tracker.
_________
Mark

mridgwel, there is some problems in 1.7 version.

seems that fractional exposure time is not correctly represented for canon dSLR. i tested with Canon 400D and 1D mark III