Problems processing RAW images

c4actbe

Joined: 2004-09-01
Posts: 83

Posted: Mon, 2005-10-10 11:58

Hi,
I verified that all components are working just fine separetely (Dcraw works, gallery "sees" it), but when I upload raw file I get not jpeg thumnail /no resized pic.
Debug Report:
Here is the debug output with the error. First line is the command it executes, notice the [0] at the end of the file name. I don't think that should be there. At the end it prints the error, Unable to open file. I'm not sure if it's supposed to be calling this convert command in the first place (seems like dcraw should come before it), but it's not calling it right so I can't be sure.

Quote:

2005-10-10 05:20:23 [190fd4d67820ff5fe84b776cb275927b] Executing: ( "/usr/bin/convert" "-quality" "80" "/home/www/danielphoto/gallery/g2data/albums/events/bm2005/test15.NEF[0]" "jpeg:/home/www/danielphoto/gallery/g2data/tmp/imgk_1Z8Y1j" ) 2>/home/www/danielphoto/gallery/g2data/tmp/g2dbgDmQfCu
2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] getParameter exec.expectedStatus for core plugin
2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] file_exists(/home/www/danielphoto/gallery/g2data/tmp/g2dbgDmQfCu)
2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] filesize(/home/www/danielphoto/gallery/g2data/tmp/g2dbgDmQfCu)
2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] fopen(/home/www/danielphoto/gallery/g2data/tmp/g2dbgDmQfCu, r, 0)
2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] feof(Resource id #437)
2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] fgets(Resource id #437, 4096)
2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] feof(Resource id #437)
2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] fgets(Resource id #437, 4096)
2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] feof(Resource id #437)
2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] fgets(Resource id #437, 4096)
2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] feof(Resource id #437)
2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] fclose(Resource id #437)
2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] unlink(/home/www/danielphoto/gallery/g2data/tmp/g2dbgDmQfCu)
2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] Regular Output:
2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] Error Output:
2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] convert: Unable to open file (/home/www/danielphoto/gallery/g2data/albums/events/bm2005/test15.NEF[0]) [No such file or directory].
2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] convert: Missing an image file name.
2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] Status: 154 (expected 0)

Here is the Stack Trace below the error

Quote:

Error (ERROR_TOOLKIT_FAILURE)in modules/imagemagick/classes/ImageMagickToolkit.class at line 622 (gallerystatus::error)
in modules/imagemagick/classes/ImageMagickToolkit.class at line 369 (imagemagicktoolkit::_transformimage)
in modules/core/classes/GalleryDerivative.class at line 484 (imagemagicktoolkit::performoperation)
in modules/core/classes/GalleryDerivative.class at line 313 (galleryderivativeimage::_rebuildcache)
in modules/core/classes/GalleryDerivativeImage.class at line 175 (galleryderivative::rebuildcache)
in modules/core/classes/helpers/GalleryDerivativeHelper_advanced.class at line 703 (galleryderivativeimage::rebuildcache)
in modules/core/classes/GalleryCoreApi.class at line 843 (galleryderivativehelper_advanced::rebuildcache)
in modules/core/classes/helpers/GalleryDerivativeHelper_simple.class at line 66 (gallerycoreapi::rebuildderivativecache)
in modules/core/classes/GalleryCoreApi.class at line 830 (galleryderivativehelper_simple::rebuildcacheifnotcurrent)
in modules/core/classes/helpers/GalleryDerivativeHelper_advanced.class at line 681 (gallerycoreapi::rebuildderivativecacheifnotcurrent)
in modules/core/classes/GalleryCoreApi.class at line 843 (galleryderivativehelper_advanced::rebuildcache)
in modules/core/classes/helpers/GalleryDerivativeHelper_simple.class at line 66 (gallerycoreapi::rebuildderivativecache)
in modules/core/classes/GalleryCoreApi.class at line 830 (galleryderivativehelper_simple::rebuildcacheifnotcurrent)
in modules/core/CreateThumbnailOption.inc at line 76 (gallerycoreapi::rebuildderivativecacheifnotcurrent)
in modules/core/ItemAdd.inc at line 151 (createthumbnailoption::handlerequestafteradd)
in main.php at line 188 (itemaddcontroller::handlerequest)
in main.php at line 87
in main.php at line 80

Gallery version = 2.0 core 1.0.0
PHP version = 4.3.11 apache
Webserver = Apache/1.3.33 (Unix) PHP/4.3.11 FrontPage/5.0.2.2510
Database = mysqlt 3.23.58
Toolkits = ArchiveUpload, Gd, Dcraw, Thumbnail, ImageMagick, NetPBM, Exif
Operating system = Linux danielphoto.propagation.net 2.4.31 #2 SMP Mon Jun 6 08:53:40 CDT 2005 i686
Browser = Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; en) Opera 8.50

valiant

Joined: 2003-01-04
Posts: 32509

Posted: Mon, 2005-10-10 12:18

/usr/bin/convert" "-quality" "80" "/home/www/danielphoto/gallery/g2data/albums/events/bm2005/test15.NEF[0]" "jpeg:/home/www/danielphoto/gallery/g2data/tmp/imgk_1Z8Y1j" )

is a little weird.
what's the filename you uploaded? what extension does it have?
imo it should first create a webviewable (jpg) derivative with dcraw and then use convert.

besides, have you tried using dcraw in the command line with the image you uploaded?

c4actbe

Joined: 2004-09-01
Posts: 83

Posted: Mon, 2005-10-10 14:00

filename is "test15.NEF" 'NEF' is Nikon d70raw format, dcraw supports it. I believe i saw a post few month ago that someone had successfully processed 'nef' files with G2.
Here is a link to download test file

dcraw in the command line processed the file and created 'ppm' (maybe i am wrong)

bharat

Joined: 2002-05-21
Posts: 7985

Posted: Tue, 2005-10-11 03:28

The [0] in the argument to convert tells it to use the first "scene" in a "multi-scene" image. In short, if the image is composed of multiple sub-images (like an animated gif) it forces us to take only the first one.

But if you're seeing this, then it means that something has gone wrong elsewhere. Imagemagick shouldn't be getting called upon to operate directly on this image; dcraw should be involved first to convert the .nef file to a JPEG. My guess is that something is misconfigured (or there's a bug) which causes ImageMagick to supercede dcraw in this case. Let's figure out what.

I'd start by going to your Site Admin -> Mime Types page and checking to make sure that "nef" is a registered mime type for image/x-dcraw. If that's the case, then I'd enable buffered debug mode and add just this one image and capture the entire debug output. You're looking for a line like this:

Executing: ( "/usr/bin/dcraw"  "-c"  "-a"  "-q" 
"/g2data-psql/albums/test.nef" >"/g2data-psql/tmp/dcraw_HHsaVT" )
2>/g2data-psql/tmp/g2dbgNtbf4W

Note that dcraw is operating on the original image. After that, ImageMagick should only be operating on the result from dcraw since dcraw is going to convert it into a JPEG. If you don't see that (and I'm pretty sure you won't) then please post a link to your entire debug output so that we can pick through it and see what's up.

c4actbe

Joined: 2004-09-01
Posts: 83

Posted: Tue, 2005-10-11 08:46

Thanks for the reply. I agree that this is probably what's going on. But how do I make sure dcraw is called first? It seems to be calling ImageMagick "convert" and bailing after the error. Here is a link to the full log (very long) for uploading just the one NEF file:

http://danielphoto.com/bug.log

dcraw never gets executed...

bharat

Joined: 2002-05-21
Posts: 7985

Posted: Tue, 2005-10-11 18:35

Did you verify the mime types? (see my last post).

c4actbe

Joined: 2004-09-01
Posts: 83

Posted: Tue, 2005-10-11 19:35

Yes, "nef" is listed under image/x-dcraw. Should "visible" be checked for the mime type or not?

valiant

Joined: 2003-01-04
Posts: 32509

Posted: Wed, 2005-10-12 14:46

no, it should not be checked. that's fine.

i've tested your file with my g2 and it worked of course.

what's the history of your G2?
what was the original version, what upgrades did you make etc?

c4actbe

Joined: 2004-09-01
Posts: 83

Posted: Wed, 2005-10-12 15:23

original version was alpha and then upgraded every 2-3 weeks. all 'upgrades' (send as postcard, dynamic dropdown navigation, possibility to sell stock/print images) are in the separate module.. no mods to any of gallery's files.
I have few other domains on the same server. I will try to do a clean install and will post results..
thanks

valiant

Joined: 2003-01-04
Posts: 32509

Posted: Wed, 2005-10-12 16:00

c4actbe
clean install? but just in parallel, right? i don't think a clean install is necessary.

c4actbe

Joined: 2004-09-01
Posts: 83

Posted: Wed, 2005-10-12 18:08

yeah, a parallel. here is my failed attempt...
all 4 pics are the same file. First one was uploaded with Imagemagick, GD amd dcraw modules activated and configured. Debug tree shows "mimeType image/x-dcraw"
Second image was uploaded right after i tried to activate NetPBM, which failed to get activated because it it didn't find jhead. (debug tree: mimeType image/tiff )
3rd image was uploaded with imagemagick deactivated.(debug tree: mimeType image/tiff )
4th one - with GD off. (debug tree: mimeType image/tiff )

My second test was in uploading bpm,png24 tiff, ,psd, and photoshop raw file...
last 3 failed ..

valiant

Joined: 2003-01-04
Posts: 32509

Posted: Wed, 2005-10-12 21:48

please activate all toolkits and dcraw, then query your database.

select * from g2_TkOperatnMimeTypeMap;

and post here the results.
thanks.

c4actbe

Joined: 2004-09-01
Posts: 83

Posted: Thu, 2005-10-13 07:09

Here's the output. I apologize for the formatting, it's copied from an HTML table.

g_operationName  	 g_toolkitId  	 g_mimeType  	 g_priority
extract 	ArchiveUpload 	application/zip 	5
rotate 	Gd 	image/png 	28
rotate 	Gd 	image/jpeg 	28
rotate 	Gd 	image/gif 	28
resize 	Gd 	image/vnd.wap.wbmp 	28
resize 	Gd 	image/png 	28
resize 	Gd 	image/jpeg 	28
resize 	Gd 	image/gif 	28
thumbnail 	Gd 	image/vnd.wap.wbmp 	28
thumbnail 	Gd 	image/png 	28
thumbnail 	Gd 	image/jpeg 	28
thumbnail 	Gd 	image/gif 	28
scale 	Gd 	image/vnd.wap.wbmp 	28
scale 	Gd 	image/png 	28
scale 	Gd 	image/jpeg 	28
scale 	Gd 	image/gif 	28
convert-to-image/jpeg 	Gd 	image/vnd.wap.wbmp 	28
convert-to-image/jpeg 	Gd 	image/png 	28
convert-to-image/jpeg 	Gd 	image/jpeg 	28
convert-to-image/jpeg 	Gd 	image/gif 	28
composite 	ImageMagick 	application/photoshop 	30
composite 	ImageMagick 	image/bmp 	30
composite 	ImageMagick 	image/tiff 	30
composite 	ImageMagick 	image/pjpeg 	30
composite 	ImageMagick 	image/jpeg 	30
composite 	ImageMagick 	image/gif 	30
crop 	ImageMagick 	application/photoshop 	30
crop 	ImageMagick 	image/bmp 	30
crop 	ImageMagick 	image/tiff 	30
crop 	ImageMagick 	image/pjpeg 	30
crop 	ImageMagick 	image/jpeg 	30
crop 	ImageMagick 	image/gif 	30
rotate 	ImageMagick 	application/photoshop 	30
rotate 	ImageMagick 	image/bmp 	30
rotate 	ImageMagick 	image/tiff 	30
rotate 	ImageMagick 	image/pjpeg 	30
rotate 	ImageMagick 	image/jpeg 	30
rotate 	ImageMagick 	image/gif 	30
resize 	ImageMagick 	application/photoshop 	30
resize 	ImageMagick 	image/bmp 	30
resize 	ImageMagick 	image/tiff 	30
resize 	ImageMagick 	image/pjpeg 	30
resize 	ImageMagick 	image/jpeg 	30
resize 	ImageMagick 	image/gif 	30
thumbnail 	ImageMagick 	application/photoshop 	30
thumbnail 	ImageMagick 	image/bmp 	30
thumbnail 	ImageMagick 	image/tiff 	30
thumbnail 	ImageMagick 	image/pjpeg 	30
thumbnail 	ImageMagick 	image/jpeg 	30
thumbnail 	ImageMagick 	image/gif 	30
scale 	ImageMagick 	application/photoshop 	30
scale 	ImageMagick 	image/bmp 	30
scale 	ImageMagick 	image/tiff 	30
scale 	ImageMagick 	image/pjpeg 	30
scale 	ImageMagick 	image/jpeg 	30
scale 	ImageMagick 	image/gif 	30
convert-to-image/tiff 	ImageMagick 	image/x-portable-pixmap 	30
convert-to-image/tiff 	ImageMagick 	application/photoshop 	30
convert-to-image/tiff 	ImageMagick 	image/bmp 	30
convert-to-image/tiff 	ImageMagick 	image/pjpeg 	30
convert-to-image/tiff 	ImageMagick 	image/jpeg 	30
convert-to-image/tiff 	ImageMagick 	image/gif 	30
convert-to-image/gif 	ImageMagick 	image/x-portable-pixmap 	30
convert-to-image/gif 	ImageMagick 	application/photoshop 	30
convert-to-image/gif 	ImageMagick 	image/bmp 	30
convert-to-image/gif 	ImageMagick 	image/tiff 	30
convert-to-image/gif 	ImageMagick 	image/pjpeg 	30
convert-to-image/gif 	ImageMagick 	image/jpeg 	30
convert-to-image/jpeg 	ImageMagick 	image/x-portable-pixmap 	30
convert-to-image/jpeg 	ImageMagick 	application/photoshop 	30
convert-to-image/jpeg 	ImageMagick 	image/bmp 	30
convert-to-image/jpeg 	Thumbnail 	video/x-msvideo 	50
convert-to-image/jpeg 	Thumbnail 	video/quicktime 	50
convert-to-image/jpeg 	Thumbnail 	video/mpeg 	50
convert-to-image/jpeg 	Thumbnail 	audio/x-ms-wma 	50
convert-to-image/jpeg 	Thumbnail 	audio/basic 	50
convert-to-image/jpeg 	Thumbnail 	audio/midi 	50
convert-to-image/jpeg 	Thumbnail 	audio/x-aiff 	50
convert-to-image/jpeg 	Thumbnail 	audio/x-wav 	50
convert-to-image/jpeg 	Thumbnail 	audio/mpeg 	50
convert-to-image/jpeg 	ImageMagick 	image/tiff 	30
convert-to-image/jpeg 	Thumbnail 	video/x-ms-asf 	50
convert-to-image/jpeg 	Thumbnail 	video/x-ms-wmv 	50
convert-to-image/jpeg 	ImageMagick 	image/pjpeg 	30
convert-to-image/jpeg 	ImageMagick 	image/gif 	30
rotate 	Gd 	image/vnd.wap.wbmp 	28
crop 	Gd 	image/gif 	28
crop 	Gd 	image/jpeg 	28
crop 	Gd 	image/png 	28
crop 	Gd 	image/vnd.wap.wbmp 	28
composite 	Gd 	image/gif 	28
composite 	Gd 	image/jpeg 	28
composite 	Gd 	image/png 	28
composite 	Gd 	image/vnd.wap.wbmp 	28
compress 	Gd 	image/jpeg 	28
compress 	ImageMagick 	image/jpeg 	30
select-page 	ImageMagick 	application/photoshop 	30
select-page 	ImageMagick 	image/tiff 	30
convert-to-image/x-portable-pixmap 	Dcraw 	image/x-dcraw 	29

c4actbe

Joined: 2004-09-01
Posts: 83

Posted: Sat, 2005-10-15 08:58

Valiant? do you have any idea?

valiant

Joined: 2003-01-04
Posts: 32509

Posted: Sat, 2005-10-15 16:26

nope, no idea. plus bharat said he'd cover this issue (and he's very busy).

i assume your G2 file integrity is ok.
http://codex.gallery2.org/index.php/Gallery2:faq#How_can_I_make_sure_that_my_installation_files_are_all_intact.3F

bharat

Joined: 2002-05-21
Posts: 7985

Posted: Sat, 2005-10-15 18:00

Sorry I haven't been responding. I'm not sure what's going wrong here, but whatever it is it's not obvious. Is there any chance that I could ssh to your box and look at the code/database directly? That would probably let me resolve this pretty quickly. Private message me with the details if that's acceptable. Thanks!

bharat

Joined: 2002-05-21
Posts: 7985

Posted: Sun, 2005-10-16 03:50

Thanks for giving me access! I've fixed the problem. It turns out that this image was marked as an "image/tiff" in the database. Is it possible that you added it *before* enabling the dcraw module? My guess is that you did, and G2 didn't realize that it was a raw image. I changed the mime type and fixed the derivative (in the database) and it started working. Alternatively, you could have deleted and re-added the image and it would have worked. I added a couple of more copies of the image to the same album to verify that it works. I'm not sure how we could prevent this from happening in the future -- once an image is miscategorized it's a little tricky. Perhaps we should add the capability for overriding the mime type via the web interface?

valiant

Joined: 2003-01-04
Posts: 32509

Posted: Sun, 2005-10-16 11:14

Quote:

Perhaps we should add the capability for overriding the mime type via the web interface?

and / or add the option to recategorize existing images, e.g. as an admin task.
e.g. when you add ffmpeg to your g2, you'll want to get the type and dimensions of all existing movies in your g2.

c4actbe

Joined: 2004-09-01
Posts: 83

Posted: Sun, 2005-10-16 13:17

Bharat, thank you very much for looking at the site... but this problem still exists...

Quote:

Is it possible that you added it *before* enabling the dcraw module?

No it was added after

Quote:

G2 didn't realize that it was a raw image

G2 on this server still doesn't recognize NEF images as RAW and interpets them as TIFF. I tried to upload 3 files and all of them "failed". First 2 were uploaded using web worm, 3rd - using applet.
Thanks!

valiant

Joined: 2003-01-04
Posts: 32509

Posted: Sun, 2005-10-16 16:23

bharat wrote:

Quote:

I added a couple of more copies of the image to the same album to verify that it works.

c4actbe wrote:

Quote:

I tried to upload 3 files and all of them "failed".

what now? did you both upload to the same G2 (since you have now 2 parallel G2s on this server)?

c4actbe

Joined: 2004-09-01
Posts: 83

Posted: Sun, 2005-10-16 18:35

yes we did. the same folder, which has failed images, contains processed NEF files. I never succeeded when tried to process raw files.

bharat

Joined: 2002-05-21
Posts: 7985

Posted: Thu, 2005-10-20 05:14

Sorry to take so long to get back to you. I think I have an idea what's going on here. When I did the upload, I uploaded the images from /home/www/danielphoto/temp_pictures where I put a copy of test15.NEF and totallynewname.NEF -- and both of those worked. I just logged on again and uploaded them.

How are you doing the upload. Are you uploading from your browser? And if so, what operating system is your browser running on? I'm wondering if your browser is telling Gallery that the file is an image/tiff .. if it tells us the mime type we believe it, and it could be misleading us. If that's the problem then I think we may have to introduce a way to allow you to override the browser's suggestion.

c4actbe

Joined: 2004-09-01
Posts: 83

Posted: Thu, 2005-10-20 09:41

You are correct. I uploaded the same file from my win xp sp2 machine using 3 browsers:

Opera 8.5: failure
IE: failure
Firefox: processed correctly ( thumbnail wasn't created, but it might be a separate issue).

Thanks for helping me out to narrow this issue.
I tried to submit bug to sourceforge, but i am not sure if it went through: i attached a test file, but after upload i got "Invalid file type" screen, Back button just cleared everything i wrote...

valiant

Joined: 2003-01-04
Posts: 32509

Posted: Thu, 2005-10-20 10:15

@sourceforge:
you see your bug listed immediately afterwards at the top, if it worked. given the error you received, it didn't work i guess.
maybe zip the nef file if you want to attach it.

c4actbe Joined: 2004-09-01 Posts: 83	Posted: Mon, 2005-10-10 11:58
	Hi, I verified that all components are working just fine separetely (Dcraw works, gallery "sees" it), but when I upload raw file I get not jpeg thumnail /no resized pic. Debug Report: Here is the debug output with the error. First line is the command it executes, notice the [0] at the end of the file name. I don't think that should be there. At the end it prints the error, Unable to open file. I'm not sure if it's supposed to be calling this convert command in the first place (seems like dcraw should come before it), but it's not calling it right so I can't be sure. Quote: 2005-10-10 05:20:23 [190fd4d67820ff5fe84b776cb275927b] Executing: ( "/usr/bin/convert" "-quality" "80" "/home/www/danielphoto/gallery/g2data/albums/events/bm2005/test15.NEF[0]" "jpeg:/home/www/danielphoto/gallery/g2data/tmp/imgk_1Z8Y1j" ) 2>/home/www/danielphoto/gallery/g2data/tmp/g2dbgDmQfCu 2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] getParameter exec.expectedStatus for core plugin 2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] file_exists(/home/www/danielphoto/gallery/g2data/tmp/g2dbgDmQfCu) 2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] filesize(/home/www/danielphoto/gallery/g2data/tmp/g2dbgDmQfCu) 2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] fopen(/home/www/danielphoto/gallery/g2data/tmp/g2dbgDmQfCu, r, 0) 2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] feof(Resource id #437) 2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] fgets(Resource id #437, 4096) 2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] feof(Resource id #437) 2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] fgets(Resource id #437, 4096) 2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] feof(Resource id #437) 2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] fgets(Resource id #437, 4096) 2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] feof(Resource id #437) 2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] fclose(Resource id #437) 2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] unlink(/home/www/danielphoto/gallery/g2data/tmp/g2dbgDmQfCu) 2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] Regular Output: 2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] Error Output: 2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] convert: Unable to open file (/home/www/danielphoto/gallery/g2data/albums/events/bm2005/test15.NEF[0]) [No such file or directory]. 2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] convert: Missing an image file name. 2005-10-10 05:20:24 [190fd4d67820ff5fe84b776cb275927b] Status: 154 (expected 0) Here is the Stack Trace below the error Quote: Error (ERROR_TOOLKIT_FAILURE)<b>in</b> modules/imagemagick/classes/ImageMagickToolkit.class <b>at line</b> 622 (gallerystatus::error) <b>in</b> modules/imagemagick/classes/ImageMagickToolkit.class <b>at line</b> 369 (imagemagicktoolkit::_transformimage) <b>in</b> modules/core/classes/GalleryDerivative.class <b>at line</b> 484 (imagemagicktoolkit::performoperation) <b>in</b> modules/core/classes/GalleryDerivative.class <b>at line</b> 313 (galleryderivativeimage::_rebuildcache) <b>in</b> modules/core/classes/GalleryDerivativeImage.class <b>at line</b> 175 (galleryderivative::rebuildcache) <b>in</b> modules/core/classes/helpers/GalleryDerivativeHelper_advanced.class <b>at line</b> 703 (galleryderivativeimage::rebuildcache) <b>in</b> modules/core/classes/GalleryCoreApi.class <b>at line</b> 843 (galleryderivativehelper_advanced::rebuildcache) <b>in</b> modules/core/classes/helpers/GalleryDerivativeHelper_simple.class <b>at line</b> 66 (gallerycoreapi::rebuildderivativecache) <b>in</b> modules/core/classes/GalleryCoreApi.class <b>at line</b> 830 (galleryderivativehelper_simple::rebuildcacheifnotcurrent) <b>in</b> modules/core/classes/helpers/GalleryDerivativeHelper_advanced.class <b>at line</b> 681 (gallerycoreapi::rebuildderivativecacheifnotcurrent) <b>in</b> modules/core/classes/GalleryCoreApi.class <b>at line</b> 843 (galleryderivativehelper_advanced::rebuildcache) <b>in</b> modules/core/classes/helpers/GalleryDerivativeHelper_simple.class <b>at line</b> 66 (gallerycoreapi::rebuildderivativecache) <b>in</b> modules/core/classes/GalleryCoreApi.class <b>at line</b> 830 (galleryderivativehelper_simple::rebuildcacheifnotcurrent) <b>in</b> modules/core/CreateThumbnailOption.inc <b>at line</b> 76 (gallerycoreapi::rebuildderivativecacheifnotcurrent) <b>in</b> modules/core/ItemAdd.inc <b>at line</b> 151 (createthumbnailoption::handlerequestafteradd) <b>in</b> main.php <b>at line</b> 188 (itemaddcontroller::handlerequest) <b>in</b> main.php <b>at line</b> 87 <b>in</b> main.php <b>at line</b> 80 Gallery version = 2.0 core 1.0.0 PHP version = 4.3.11 apache Webserver = Apache/1.3.33 (Unix) PHP/4.3.11 FrontPage/5.0.2.2510 Database = mysqlt 3.23.58 Toolkits = ArchiveUpload, Gd, Dcraw, Thumbnail, ImageMagick, NetPBM, Exif Operating system = Linux danielphoto.propagation.net 2.4.31 #2 SMP Mon Jun 6 08:53:40 CDT 2005 i686 Browser = Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; en) Opera 8.50
	XUser login Username: * Password: * Create new account Request new password
valiant Joined: 2003-01-04 Posts: 32509	Posted: Mon, 2005-10-10 12:18
	/usr/bin/convert" "-quality" "80" "/home/www/danielphoto/gallery/g2data/albums/events/bm2005/test15.NEF[0]" "jpeg:/home/www/danielphoto/gallery/g2data/tmp/imgk_1Z8Y1j" ) is a little weird. what's the filename you uploaded? what extension does it have? imo it should first create a webviewable (jpg) derivative with dcraw and then use convert. besides, have you tried using dcraw in the command line with the image you uploaded?

c4actbe Joined: 2004-09-01 Posts: 83	Posted: Mon, 2005-10-10 14:00
	filename is "test15.NEF" 'NEF' is Nikon d70raw format, dcraw supports it. I believe i saw a post few month ago that someone had successfully processed 'nef' files with G2. Here is a link to download test file dcraw in the command line processed the file and created 'ppm' (maybe i am wrong)

bharat Joined: 2002-05-21 Posts: 7985	Posted: Tue, 2005-10-11 03:28
	The [0] in the argument to convert tells it to use the first "scene" in a "multi-scene" image. In short, if the image is composed of multiple sub-images (like an animated gif) it forces us to take only the first one. But if you're seeing this, then it means that something has gone wrong elsewhere. Imagemagick shouldn't be getting called upon to operate directly on this image; dcraw should be involved first to convert the .nef file to a JPEG. My guess is that something is misconfigured (or there's a bug) which causes ImageMagick to supercede dcraw in this case. Let's figure out what. I'd start by going to your Site Admin -> Mime Types page and checking to make sure that "nef" is a registered mime type for image/x-dcraw. If that's the case, then I'd enable buffered debug mode and add just this one image and capture the entire debug output. You're looking for a line like this: Executing: ( "/usr/bin/dcraw" "-c" "-a" "-q" "/g2data-psql/albums/test.nef" >"/g2data-psql/tmp/dcraw_HHsaVT" ) 2>/g2data-psql/tmp/g2dbgNtbf4W Note that dcraw is operating on the original image. After that, ImageMagick should only be operating on the result from dcraw since dcraw is going to convert it into a JPEG. If you don't see that (and I'm pretty sure you won't) then please post a link to your entire debug output so that we can pick through it and see what's up.

c4actbe Joined: 2004-09-01 Posts: 83	Posted: Tue, 2005-10-11 08:46
	Thanks for the reply. I agree that this is probably what's going on. But how do I make sure dcraw is called first? It seems to be calling ImageMagick "convert" and bailing after the error. Here is a link to the full log (very long) for uploading just the one NEF file: http://danielphoto.com/bug.log dcraw never gets executed...

bharat Joined: 2002-05-21 Posts: 7985	Posted: Tue, 2005-10-11 18:35
	Did you verify the mime types? (see my last post).

c4actbe Joined: 2004-09-01 Posts: 83	Posted: Tue, 2005-10-11 19:35
	Yes, "nef" is listed under image/x-dcraw. Should "visible" be checked for the mime type or not?

valiant Joined: 2003-01-04 Posts: 32509	Posted: Wed, 2005-10-12 14:46
	no, it should not be checked. that's fine. i've tested your file with my g2 and it worked of course. what's the history of your G2? what was the original version, what upgrades did you make etc?

c4actbe Joined: 2004-09-01 Posts: 83	Posted: Wed, 2005-10-12 15:23
	original version was alpha and then upgraded every 2-3 weeks. all 'upgrades' (send as postcard, dynamic dropdown navigation, possibility to sell stock/print images) are in the separate module.. no mods to any of gallery's files. I have few other domains on the same server. I will try to do a clean install and will post results.. thanks

valiant Joined: 2003-01-04 Posts: 32509	Posted: Wed, 2005-10-12 16:00
	c4actbe clean install? but just in parallel, right? i don't think a clean install is necessary.

c4actbe Joined: 2004-09-01 Posts: 83	Posted: Wed, 2005-10-12 18:08
	yeah, a parallel. here is my failed attempt... all 4 pics are the same file. First one was uploaded with Imagemagick, GD amd dcraw modules activated and configured. Debug tree shows "mimeType image/x-dcraw" Second image was uploaded right after i tried to activate NetPBM, which failed to get activated because it it didn't find jhead. (debug tree: mimeType image/tiff ) 3rd image was uploaded with imagemagick deactivated.(debug tree: mimeType image/tiff ) 4th one - with GD off. (debug tree: mimeType image/tiff ) My second test was in uploading bpm,png24 tiff, ,psd, and photoshop raw file... last 3 failed ..

valiant Joined: 2003-01-04 Posts: 32509	Posted: Wed, 2005-10-12 21:48
	please activate all toolkits and dcraw, then query your database. select * from g2_TkOperatnMimeTypeMap; and post here the results. thanks.

c4actbe Joined: 2004-09-01 Posts: 83	Posted: Thu, 2005-10-13 07:09
	Here's the output. I apologize for the formatting, it's copied from an HTML table. g_operationName g_toolkitId g_mimeType g_priority extract ArchiveUpload application/zip 5 rotate Gd image/png 28 rotate Gd image/jpeg 28 rotate Gd image/gif 28 resize Gd image/vnd.wap.wbmp 28 resize Gd image/png 28 resize Gd image/jpeg 28 resize Gd image/gif 28 thumbnail Gd image/vnd.wap.wbmp 28 thumbnail Gd image/png 28 thumbnail Gd image/jpeg 28 thumbnail Gd image/gif 28 scale Gd image/vnd.wap.wbmp 28 scale Gd image/png 28 scale Gd image/jpeg 28 scale Gd image/gif 28 convert-to-image/jpeg Gd image/vnd.wap.wbmp 28 convert-to-image/jpeg Gd image/png 28 convert-to-image/jpeg Gd image/jpeg 28 convert-to-image/jpeg Gd image/gif 28 composite ImageMagick application/photoshop 30 composite ImageMagick image/bmp 30 composite ImageMagick image/tiff 30 composite ImageMagick image/pjpeg 30 composite ImageMagick image/jpeg 30 composite ImageMagick image/gif 30 crop ImageMagick application/photoshop 30 crop ImageMagick image/bmp 30 crop ImageMagick image/tiff 30 crop ImageMagick image/pjpeg 30 crop ImageMagick image/jpeg 30 crop ImageMagick image/gif 30 rotate ImageMagick application/photoshop 30 rotate ImageMagick image/bmp 30 rotate ImageMagick image/tiff 30 rotate ImageMagick image/pjpeg 30 rotate ImageMagick image/jpeg 30 rotate ImageMagick image/gif 30 resize ImageMagick application/photoshop 30 resize ImageMagick image/bmp 30 resize ImageMagick image/tiff 30 resize ImageMagick image/pjpeg 30 resize ImageMagick image/jpeg 30 resize ImageMagick image/gif 30 thumbnail ImageMagick application/photoshop 30 thumbnail ImageMagick image/bmp 30 thumbnail ImageMagick image/tiff 30 thumbnail ImageMagick image/pjpeg 30 thumbnail ImageMagick image/jpeg 30 thumbnail ImageMagick image/gif 30 scale ImageMagick application/photoshop 30 scale ImageMagick image/bmp 30 scale ImageMagick image/tiff 30 scale ImageMagick image/pjpeg 30 scale ImageMagick image/jpeg 30 scale ImageMagick image/gif 30 convert-to-image/tiff ImageMagick image/x-portable-pixmap 30 convert-to-image/tiff ImageMagick application/photoshop 30 convert-to-image/tiff ImageMagick image/bmp 30 convert-to-image/tiff ImageMagick image/pjpeg 30 convert-to-image/tiff ImageMagick image/jpeg 30 convert-to-image/tiff ImageMagick image/gif 30 convert-to-image/gif ImageMagick image/x-portable-pixmap 30 convert-to-image/gif ImageMagick application/photoshop 30 convert-to-image/gif ImageMagick image/bmp 30 convert-to-image/gif ImageMagick image/tiff 30 convert-to-image/gif ImageMagick image/pjpeg 30 convert-to-image/gif ImageMagick image/jpeg 30 convert-to-image/jpeg ImageMagick image/x-portable-pixmap 30 convert-to-image/jpeg ImageMagick application/photoshop 30 convert-to-image/jpeg ImageMagick image/bmp 30 convert-to-image/jpeg Thumbnail video/x-msvideo 50 convert-to-image/jpeg Thumbnail video/quicktime 50 convert-to-image/jpeg Thumbnail video/mpeg 50 convert-to-image/jpeg Thumbnail audio/x-ms-wma 50 convert-to-image/jpeg Thumbnail audio/basic 50 convert-to-image/jpeg Thumbnail audio/midi 50 convert-to-image/jpeg Thumbnail audio/x-aiff 50 convert-to-image/jpeg Thumbnail audio/x-wav 50 convert-to-image/jpeg Thumbnail audio/mpeg 50 convert-to-image/jpeg ImageMagick image/tiff 30 convert-to-image/jpeg Thumbnail video/x-ms-asf 50 convert-to-image/jpeg Thumbnail video/x-ms-wmv 50 convert-to-image/jpeg ImageMagick image/pjpeg 30 convert-to-image/jpeg ImageMagick image/gif 30 rotate Gd image/vnd.wap.wbmp 28 crop Gd image/gif 28 crop Gd image/jpeg 28 crop Gd image/png 28 crop Gd image/vnd.wap.wbmp 28 composite Gd image/gif 28 composite Gd image/jpeg 28 composite Gd image/png 28 composite Gd image/vnd.wap.wbmp 28 compress Gd image/jpeg 28 compress ImageMagick image/jpeg 30 select-page ImageMagick application/photoshop 30 select-page ImageMagick image/tiff 30 convert-to-image/x-portable-pixmap Dcraw image/x-dcraw 29

c4actbe Joined: 2004-09-01 Posts: 83	Posted: Sat, 2005-10-15 08:58
	Valiant? do you have any idea?

valiant Joined: 2003-01-04 Posts: 32509	Posted: Sat, 2005-10-15 16:26
	nope, no idea. plus bharat said he'd cover this issue (and he's very busy). i assume your G2 file integrity is ok. http://codex.gallery2.org/index.php/Gallery2:faq#How_can_I_make_sure_that_my_installation_files_are_all_intact.3F

bharat Joined: 2002-05-21 Posts: 7985	Posted: Sat, 2005-10-15 18:00
	Sorry I haven't been responding. I'm not sure what's going wrong here, but whatever it is it's not obvious. Is there any chance that I could ssh to your box and look at the code/database directly? That would probably let me resolve this pretty quickly. Private message me with the details if that's acceptable. Thanks!

bharat Joined: 2002-05-21 Posts: 7985	Posted: Sun, 2005-10-16 03:50
	Thanks for giving me access! I've fixed the problem. It turns out that this image was marked as an "image/tiff" in the database. Is it possible that you added it before enabling the dcraw module? My guess is that you did, and G2 didn't realize that it was a raw image. I changed the mime type and fixed the derivative (in the database) and it started working. Alternatively, you could have deleted and re-added the image and it would have worked. I added a couple of more copies of the image to the same album to verify that it works. I'm not sure how we could prevent this from happening in the future -- once an image is miscategorized it's a little tricky. Perhaps we should add the capability for overriding the mime type via the web interface?

valiant Joined: 2003-01-04 Posts: 32509	Posted: Sun, 2005-10-16 11:14
	Quote: Perhaps we should add the capability for overriding the mime type via the web interface? and / or add the option to recategorize existing images, e.g. as an admin task. e.g. when you add ffmpeg to your g2, you'll want to get the type and dimensions of all existing movies in your g2.

c4actbe Joined: 2004-09-01 Posts: 83	Posted: Sun, 2005-10-16 13:17
	Bharat, thank you very much for looking at the site... but this problem still exists... Quote: Is it possible that you added it before enabling the dcraw module? No it was added after Quote: G2 didn't realize that it was a raw image G2 on this server still doesn't recognize NEF images as RAW and interpets them as TIFF. I tried to upload 3 files and all of them "failed". First 2 were uploaded using web worm, 3rd - using applet. Thanks!

valiant Joined: 2003-01-04 Posts: 32509	Posted: Sun, 2005-10-16 16:23
	bharat wrote: Quote: I added a couple of more copies of the image to the same album to verify that it works. c4actbe wrote: Quote: I tried to upload 3 files and all of them "failed". what now? did you both upload to the same G2 (since you have now 2 parallel G2s on this server)?

c4actbe Joined: 2004-09-01 Posts: 83	Posted: Sun, 2005-10-16 18:35
	yes we did. the same folder, which has failed images, contains processed NEF files. I never succeeded when tried to process raw files.

bharat Joined: 2002-05-21 Posts: 7985	Posted: Thu, 2005-10-20 05:14
	Sorry to take so long to get back to you. I think I have an idea what's going on here. When I did the upload, I uploaded the images from /home/www/danielphoto/temp_pictures where I put a copy of test15.NEF and totallynewname.NEF -- and both of those worked. I just logged on again and uploaded them. How are you doing the upload. Are you uploading from your browser? And if so, what operating system is your browser running on? I'm wondering if your browser is telling Gallery that the file is an image/tiff .. if it tells us the mime type we believe it, and it could be misleading us. If that's the problem then I think we may have to introduce a way to allow you to override the browser's suggestion.

c4actbe Joined: 2004-09-01 Posts: 83	Posted: Thu, 2005-10-20 09:41
	You are correct. I uploaded the same file from my win xp sp2 machine using 3 browsers: Opera 8.5: failure IE: failure Firefox: processed correctly ( thumbnail wasn't created, but it might be a separate issue). Thanks for helping me out to narrow this issue. I tried to submit bug to sourceforge, but i am not sure if it went through: i attached a test file, but after upload i got "Invalid file type" screen, Back button just cleared everything i wrote...

valiant Joined: 2003-01-04 Posts: 32509	Posted: Thu, 2005-10-20 10:15
	@sourceforge: you see your bug listed immediately afterwards at the top, if it worked. given the error you received, it didn't work i guess. maybe zip the nef file if you want to attach it.

Problems processing RAW images

XUser login