No thumnbails and original goes to browser after upgrade

Netter

Joined: 2007-03-26
Posts: 39
Posted: Mon, 2007-03-26 10:25

I upgraded Gallery2 from 2.1.x to the latest 2.2.1.
Everything made by instruction (uploaded new files over old ones by ftp, run upgrade, removed unnessessary files by cleanup.sh and did upgrade).
All works, album are still here, users too, except for displaying images. No thumbnails, no middle size, no original size diplayed. But files are there, I cleared cache several times and I can see that thumbnails and middle size images are being created in 'g2data/cache/derivative/...'. And these files are correct (I downloaded them, renamed .dat to .jpg and I can see the content).

But it looks like the problem is with DownloadItem part of the core. It just does not pass thru data from .dat files to my browser. Also, when I try to do 'Maintenance -> Build all thumbnails/resizes' I just get empty not-found page (and the same empty page with 'Refresh capture dates'). So from that you can understand that NetPBM/GD is working and creating cache files every first time they are called by broeser, but data of these files is not sent to browser.

I also tried to deactivate/uninstall URL rewrite. No effect at all.
I also edited 'modules/core/DownloadItem.inc' and added print('ok'); at the very beginning of php code... But nothing in browser when I try to open separate photo url.

What can be the problem?


Gallery version = 2.2.1 core 1.2.0.1
PHP version = 4.3.4 apache2handler
Webserver = Apache/2.0.49 (Linux/SuSE)
Database = mysqlt 4.0.18, lock.system=flock
Toolkits = Getid3, Gd, SquareThumb, Thumbnail, NetPBM, Ffmpeg, Exif
Acceleration = none, none
Operating system = Linux web-server 2.6.5-7.244-smp #1 SMP Mon Dec 12 18:32:25 UTC 2005 x86_64
Default theme = classic
gettext = enabled
Locale = en_US
Browser = Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506)
Rows in GalleryAccessMap table = 511
Rows in GalleryAccessSubscriberMap table = 3357
Rows in GalleryUser table = 154
Rows in GalleryItem table = 3335
Rows in GalleryAlbumItem table = 189
Rows in GalleryCacheMap table = 0

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Mon, 2007-03-26 13:46

- no warnings in the system check of the upgrader?
- please revert your changes to the code (downloaditem.inc, ...)
- please post the address/url of your gallery
- please enable buffered debug mode FAQ: How to set/use Gallery in debug mode?
- please post a phpinfo link (FAQ: What information is required when I ask for help in the forums?)

--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage

 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Tue, 2007-03-27 21:00

- no warnings, upgrade went fine
- reverted original inc file
- http://photo2.pyza.ru/main.php
- ok, enabled
- http://photo2.pyza.ru/phpinfo.php

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2007-03-27 21:18

> - no warnings, upgrade went fine

you can visit the upgrade wizard again. does is still show no warning?

- please deactivate the url rewrite module
- please deactivate the gd module as well

this is an example url:
http://photo2.pyza.ru/d/3217-1/DSCN0634.JPG (fullsize image)
-> returns nothing.

maybe it works once the above features are disabled.

--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage

 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Tue, 2007-03-27 21:24

deactivated now

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2007-03-27 21:37

still the same.

please check your server's error log. there should be a lot of error entries.
FAQ: What is the webserver error log?

--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage

 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Tue, 2007-03-27 21:56

this may sound strange but there are no errors in error_log (server is on plesk) involved with gallery2. only like not found robots.txt, not found favicon.ico...

btw, photo2 subdomain is a mirror for original photo. I copied all files exactly and exported/inported mysql dump and as a difference at least on photo2 previews are being recreated (I can see some when I go to different albums). On original there's no way any picture is shown. Very strange and makes me want to do the same thing to original website (remove mysql data, import form dump).

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2007-03-27 22:06
 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Tue, 2007-03-27 22:14

on photo2 subdomain? it is set to buffered now.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2007-03-27 22:24

still nada.

please enable "immediate" debug mode. that will usually corrupt the transfer, but it could show some helpful stuff.

--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage

 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Tue, 2007-03-27 22:28

I also sent you a private message.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2007-03-27 22:34

arg, nothing.
please disable the debug mode again.

you say you cleared your caches already. all of them?

maybe just delete g2data/cache/* .

i'd like to ensure that the entity and its fast-download caches are gone as well.

running out of ideas here...

--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage

 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Tue, 2007-03-27 22:38

yes. I cleared cache many times. right now I removed cache directory completely
how about I give you admin access so you can see some other strange staff like blank (not found) pages when I click Maintenance -> Build thumbnails for example.

 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Tue, 2007-03-27 22:40

um.. now it seems like working better (at least thumbs for albums are also being created)

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2007-03-27 22:45

if you get other blank pages, then error reporting / logging of your php is clearly not working.

i suggest you ensure that errors are properly logged, then we can look further into it.

e.g. intentioanlly trigger a fatal php error and check if it's logged to the webserver's error log.

--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage

 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Tue, 2007-03-27 23:00

logging is working great. because when I type url that is not found it's in error_log
it's just that in firefox I see blank pages, IE7 shows it like not-found page. so maybe it's not exactly 404 error, but php exit function.

after complete cache remove it works better (though not all albums pictures are being recreated once I load it but most of them yes)

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2007-03-27 23:22

> logging is working great. because when I type url that is not found it's in error_log

that's an apache error.
we want to ensure that PHP errors are logged too.

--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage

 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Wed, 2007-03-28 10:23

I set
ini_set('display_errors', 1);
ini_set('log_errors', 1);
ini_set('error_log', dirname(__FILE__) . '/error_log.txt');
error_reporting(E_ALL);
in config.php.

btw, from yesterday some things started to work (I guess after disabling gd and mod_rewrite. So now gallery2 creates thumbnail and middle-size previews. But with original size still no luck, the same empty page.

 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Wed, 2007-03-28 11:00

btw, could it be like some kind of size limit in admin area set by default?

So it looks like:
If original image is too large only middle-size is shown.
If original image is large - only thumb and middle-size (like 640x480)is created.
If original image is small (or I guess original size was not kept by user setting) then no thumb middle (that is also an original sice you have no option to see different size)

But original sizes are not shown in any case.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Wed, 2007-03-28 11:32

@error logging:

can you edit your php.ini file?
please edit log_errors, error_log, display_errors and error_reporting in there and restart the webserver.
sometimes, setting these values during runtime is too late.

once we see the errors in the log, it should be easier to diagnose your case.

--------------
Doumentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage

 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Wed, 2007-03-28 23:07

it's me again. I checked my php.ini settings. all error reporting/display is enabled and working. but gallery2 produces no errors in case of these previews or middle-size original pictures. I run some scripts with errors intentionally in the same dir and these errors were shown in browser and logged in apache log.

I suggest another thing. This photo gallery is important and people are waiting for it to be back. I have this f**cked up database (as I believe). How about making a clear install and then:
- copy g2data/album folder to new clean g2
- replace/import only tables with users accounts and items/albums.

I believe the problem is somewhere in tables with plugins or g2 settings and it happened during the upgrade. Please tell me what only tables are important for user accounts, user albums and images in gallery so that I could try to export them now and import after clean install.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2007-03-29 00:10

please change error_reporting to E_ALL in your php.ini.

> I checked my php.ini settings. all error reporting/display is enabled and working.

how did you check whether it's working?

i suggest you put this into main.php of g2:

after <?php
add this on the next line:

calling_a_method_that_does_not_exist();

save the file.

browse to your gallery.

if you don't see an error in your webserver's error log, php error logging isn't working.

--------------
Documentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage

 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Thu, 2007-03-29 09:44

error reporting is working.
you can check now
http://photo2.pyza.ru/main.php

also there's a line in error_log
[client xx.xx.xx.xx] PHP Fatal error: Call to undefined function: calling_a_method_that_does_not_exist() in /var/www/vhosts/pyza.ru/subdomains/photo2/http
docs/main.php on line 2

so as I reported before the problem is not in php, but in corrupted databse. read my previous message and help me to migrate users/albums/pictures only to a new clean install.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2007-03-29 15:30

good.

> read my previous message and help me to migrate users/albums/pictures only to a new clean install.

no pretty please? :)

> Please tell me what only tables are important for user accounts, user albums and images in gallery so that I could try to export them now and import after clean install.

that's very hard to do. g2's database schema has a lot of inter-dependencies.
it's sure easier to fix your db.

--------------
Documentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage

 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Thu, 2007-03-29 15:45
Quote:
no pretty please?

I am really gratefull for your help.

Quote:
that's very hard to do. g2's database schema has a lot of inter-dependencies.
it's sure easier to fix your db.

As I sent you in PM before, I can send you access to mysql and admin access to gallery so you can see it from the inside and maybe find a problem. I sure understand that it's not just copying 2-3 tables, but maybe all except a few that contain plugins/core settings.

 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Thu, 2007-03-29 16:08

right now I am at the state where I managed to combine old data tables with new core/plugins tables and thimbnails and middle size pictures are being generated (removed cache completely) and shown, but with full size pictures the problem persists.

 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Thu, 2007-03-29 16:21

ok. new update:
it looks like in cache new thumbs and middle size images are being created but the biggest original file - not.
For example where I upload an image or check for an existing one for the first time:

main.php?g2_view=core.DownloadItem&g2_itemId=12831&g2_serialNumber=1 (files 12831.dat, 12831-meta.inc are not in cache)
main.php?g2_view=core.DownloadItem&g2_itemId=12832&g2_serialNumber=2 (files 12832.dat, 12832-meta.inc and newly created)
main.php?g2_view=core.DownloadItem&g2_itemId=12833&g2_serialNumber=2 (files 12833.dat, 12833-meta.inc and newly created)

and
31 is the biggest original one,
32 is thumbnail size
33 is middle size

does gallery2 also creates/copies original size files to cache or it just sends it to browser with fread() function from original location?

 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Thu, 2007-03-29 17:17

ok. next step.
I checked on another gallery2 album (thank god, not upgraded) and found out that when original files are not recreated in cache, but they are just fread() from the source album/username/filename.jpg

After that I intentionally cleared cache and chaned original file name of one image in g2_FileSystemEntity from abc.jpg to errorabc.jpg and refreshed the page - all new thumbs and middle size images were recreted but for image abc.jpg I have logged php error (it failed to do copy() to recreate thumb and resized images). So? So now we know that the problem occures only when we try to see original size image. And php script doesn't even tried to fopen() or copy() the file for that. It just ends before that with no error (skips the statement?). No php errors and no page when I open full-size image (with good and bad filename in mysql) So I guess the problem should be in some statements of DownloadItem.inc file. Can you please give me advise what to change so to be sure?

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2007-03-29 17:54

please don't edit your older posts. this forum shows the edited posts out of order.

edit: i was confused. seems to be all in order.

--------------
Documentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage

 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Thu, 2007-03-29 18:46

valiant, do you still read this thread? I need your help.
I think the problem is somewhere here:

	/* Rebuild derivative cache, if necessary */
	$itemForPermission = $itemId;
	$derivativeType = null;
	if (GalleryUtilities::isA($item, 'GalleryDerivative')) {
	    list ($ret, $item) = GalleryCoreApi::rebuildDerivativeCacheIfNotCurrent($itemId);
	    if ($ret) {
		return $ret;
	    }

	    $itemForPermission = $item->getParentId();
	    $derivativeType = $item->getDerivativeType();
	}

	$ret = $this->_sendFile(array('derivativePath' => $path,
				      'derivativeType' => $derivativeType,
				      'mimeType' => $item->getMimeType(),
				      'pseudoFileName' => $pseudoFileName,
				      'parentId' => $itemForPermission,
				      'id' => $itemId));
	if ($ret) {
	    return $ret;
	}

for showing original file it does nothing after that. Especially I guess _sendFile ends the script or return($ret) does it.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2007-03-29 20:17

you'll have to dug deeper there.
does it print the file in sendFile? does it suddenly stop the execution?
if php was logging errors correctly (and it should), there should be an entry in your error logs.

FYI: your error_reporting value is still not set to E_ALL in php.ini (see phpinfo).

--------------
Documentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage

 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Thu, 2007-03-29 21:41

the last code (sql queries and my point-tracks) with immediate debug show that script stops at line 62 of DownloadItem.inc:
list ($ret, $item) = GalleryCoreApi::loadEntitiesById($itemId);

so GalleryCoreApi::loadEntitiesById($itemId) does something wrong when loading original file info. Right after that line no code execution.

 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Thu, 2007-03-29 21:42

also, error_reporting is set to E_ALL & ~E_NOTICE (that's what code 2039 means)

 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Thu, 2007-03-29 21:47

I changed it to E_ALL only (code 2047) and it's just the same. so I am changing it back.

 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Thu, 2007-03-29 22:25

oh sh**t. I just tried to make a comlete clean installation an another subdomain and upload some big files to my album.
The same problem with full size images. Thumbs and middle-size are created. I spent 3 days finding a problem after upgrade and the problem is stupid version 2.2.1!!! On the same server I have a few other installations of 2.1.2 and they are just doing great. damn this 2.2 version! no way I upgrade other versions.

Is it possible to get an old 2.1.2 from somewhere? I cannot find it here.

 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Thu, 2007-03-29 22:58

found 2.1.2, installed. works fine. now I need to figure out how to manage to work upgraded mysql data to old 2.1.2 engine. damn I wish I didn't do this upgrade.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2007-03-29 23:05

since you're so polite and curse about a year of work that went into g2.2, i'm very glad to help you.

sure you can still get g2.1.2.
it's available on our download page on sourceforge and there's a link to it on our 2.2 download page.

--------------
Documentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage

 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Thu, 2007-03-29 23:16

man, I love gallery2, I really do and I use it wherever I can.
it's just version 2.2 now seems like an alpha for me. looks like incompatible with something that 2.1 was working good with. and I have standard hosting many users have. that means not only me have this problem and the best way to avoid problem with upgrade is to make a clean install on another subdomain first.

I hope those bugs will be fixed soon and I will be able to upgrade it flawlessly. until then 2.1 rules for me.
keep the good job.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2007-03-29 23:29

> I hope those bugs will be fixed soon and I will be able to upgrade it flawlessly. until then 2.1 rules for me.

we don't see any issues with it.
if there's a case that we can reproduce on our development machines, we can try to fix it.

--------------
Documentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage

 
kashent

Joined: 2007-03-31
Posts: 6
Posted: Sat, 2007-03-31 18:44

I've found your bug.

I was having this same problem and tracked it down, fortunately it's easy to sidestep, but I'm not sure what all the reprecussions are.

Change line 123 of modules/core/DownloadItem.inc to read as follows:

    123 /* FIXME - commented out by user -
    124         if (function_exists('apache_setenv')) {
    125             @apache_setenv('no-gzip', '1');
    126         }
    127 */

For some reason this call forces the entire thread to prematurely exit. It may be specific to the apache version Netter and I are using.
I'm running apache 2.0.47 on SuSE 9.0.

Can one of the more experienced gallery coders chime in and let us know if this is likely to break anything else?

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sat, 2007-03-31 19:28

kashent

wow. nothing logged in your apache error log?
what if you use
apache_setenv('no-gzip', '1');
instead of
@apache_setenv('no-gzip', '1');
(no trailing @)

do you see anything in the logs then?

--------------
Documentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage

 
kashent

Joined: 2007-03-31
Posts: 6
Posted: Sat, 2007-03-31 19:49

Sorry, I didn't think about including that in my last post-
Everytime the function fails there, you get a seg fault for that apache thread, like so:

[Sat Mar 31 12:45:54 2007] [notice] child pid 11279 exit signal Segmentation fault (11)

Removing the @ made absolutely no difference.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sat, 2007-03-31 19:53

what's your php version and API (php-cgi or mod_php?)?

--------------
Documentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage

 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Sat, 2007-03-31 20:01

looks like it fixed problem with images. I made a fresh installation. All pictures are generated and shown (including an original one).
But there's some other things to fix.
If you were able to recreate this problem, then in Admin section go to Maintenance and try any of these:
- Build all thumbnails/resizes
- Refresh capture dates
- Delete user sessions

When you click any of these you get the same blank page with nothing. I suppose some other .inc files contain similar code.
I will try to update my previous installation 2.1.2 to 2.2.1 with data and tell you the result with thumbs/images.

 
kashent

Joined: 2007-03-31
Posts: 6
Posted: Sat, 2007-03-31 20:08

Php version is 4.3.3, API is 20020918

and fwiw:
PHP Extension 20020429
Zend Extension 20021010
Debug Build no
Thread Safety disabled
Registered PHP Streams php, http, ftp, https, ftps, compress.bzip2, compress.zlib

And related bits-

Apache Environment:
HTTP_ACCEPT_ENCODING gzip,deflate

Http headers information:
Accept-Encoding gzip,deflate

Environment:
GZIP -9

PHP Variables:
_SERVER["HTTP_ACCEPT_ENCODING"] gzip,deflate
_ENV["GZIP"] -9

Hopefully something in there is useful.

I suppose for now I should be safe to run as long as I have mod_gzip turned off, right?

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sat, 2007-03-31 20:08

Netter

> looks like it fixed problem with images. I made a fresh installation. All pictures are generated and shown (including an original one).

are you referring to kashent suggestion? or are you using an unchanged g2.2.1 installation?

--------------
Documentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage

 
kashent

Joined: 2007-03-31
Posts: 6
Posted: Sat, 2007-03-31 20:13
Quote:
But there's some other things to fix.
If you were able to recreate this problem, then in Admin section go to Maintenance and try any of these:
- Build all thumbnails/resizes
- Refresh capture dates
- Delete user sessions

You were right, two other places to make this change as well:

main.php: @apache_setenv('no-gzip', '1');
modules/zipcart/Download.inc: @apache_setenv('no-gzip', '1');

Making the change in main.php will allow you to do all those Maintenance things you couldn't do before.

 
kashent

Joined: 2007-03-31
Posts: 6
Posted: Sat, 2007-03-31 20:54

One more bit of information, setting no-gzip to '0' also fails in exactly the same way. It appears that just calling apache_setenv on no-gzip is not feasable for our installations.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sat, 2007-03-31 21:02
 
Netter

Joined: 2007-03-26
Posts: 39
Posted: Sat, 2007-03-31 23:21

yes. kashent was absolutely right about those apache_setenv lines. commented them and everything works like a charm.
I managed to make a clean installation and to migrate some user/album/item tables into it, so now everything works great.

thank you, kashent. you have made my day!

I wish there was some good export/import plugin that allows you to export user accounts/albums/items only because those internal dependences in mysql gave me a headache.

btw, does anybody know how to put dynamic "last additions" link into left menu without editing files?

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2007-04-01 00:19