Large error log when attempting to rebuild thumbnails

chadeldridge

Joined: 2008-05-12
Posts: 19
Posted: Sun, 2011-01-23 04:02

Gallery URL = http://chadeldridge.com/gallery2/main.php
Gallery version = 2.3 core 1.3.0
API = Core 7.54, Module 3.9, Theme 2.6, Embed 1.5
PHP version = 4.3.11 cgi-fcgi
Webserver = Apache
Database = mysqlt 5.0.67.d7-ourdelta-log, lock.system=flock
Toolkits = ArchiveUpload, Exif, Getid3, LinkItemToolkit, Thumbnail, Gd, ImageMagick
Acceleration = none, none
Operating system = Linux linhostcf01.prod.mesa1.secureserver.net 2.4.21-53.ELsmp #1 SMP Wed Nov 14 03:54:12 EST 2007 i686
Default theme = matrix
gettext = enabled
Locale = en_US
Browser = Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 (.NET CLR 3.5.30729)
Rows in GalleryAccessMap table = 4
Rows in GalleryAccessSubscriberMap table = 3058
Rows in GalleryUser table = 6
Rows in GalleryItem table = 3056
Rows in GalleryAlbumItem table = 16
Rows in GalleryCacheMap table = 0

I have lots of broken thumbnails which i am trying to run the "Build all Thumbnails/resizes" which runs quickly through 9000 some odd pictures and then throws the following error.

AttachmentSize
errorlog.txt135.59 KB
 
Dayo

Joined: 2005-11-04
Posts: 1642
Posted: Sun, 2011-01-23 04:17

ImageMagick is rejecting some of the images.

For example:

...
Error Output:
/usr/bin/convert: Corrupt JPEG data: premature end of data segment
...

Looks to me that something has mangled the images. Any ideas what you may have run on them previously?

Alternatively, you can try moving GD up your toolkit priority list although I'll expect it would face the same issues but nothing to lose in trying.
You might need to increase your php memory allocation for GD to run through or perhaps restart it a few times.
--
dakanji.com

 
chadeldridge

Joined: 2008-05-12
Posts: 19
Posted: Sun, 2011-01-23 04:48

I moved GD up in priority and now the process just stops at image 2905, and then hangs there. Going back to original settings still throws the same error obviously. I haven't ran anything special on the files, most of the time just rename them based on time/date stamp and upload. Since this is shared hosting with GoDaddy i have no option to increase the amount of memory given to the operation, although it seems to be allocated 32mb of which the memory counter never gets close to.

 
Dayo

Joined: 2005-11-04
Posts: 1642
Posted: Sun, 2011-01-23 16:03

Looking at this further, there is basically no cure for this open to you.

You will need to delete the corrupted images.

If you can access the gallery, it will be better to delete from within gallery2. Otherwise, delete them from your picdata album directly although that will create the missing object issue but there is a fix for that.
Here are the picks from your error log ...

gallery2/picdata/albums/Aiden/Birthdays/2bday/2009_01_25_02_00_EASTMAN KODAK COMPANY-001.jpg
gallery2/picdata/albums/obxwithmom/2004_09_21_19_17.jpg
gallery2/picdata/albums/obxwithmom/2004_09_20_15_50.jpg
gallery2/picdata/albums/Aiden/Christmas/Christmas_08/IMG_1508.JPG
gallery2/picdata/albums/family_pics/071103_FH000185.jpg
gallery2/picdata/albums/Aiden/080503_IMG_0569_001.JPG

... basically files with a .jpg extension in the log.

So you delete them, run the process, check the logs, delete any new ones and repeat until the process runs to the end without crashing.
--
dakanji.com

 
suprsidr
suprsidr's picture

Joined: 2005-04-17
Posts: 8339
Posted: Sun, 2011-01-23 16:22
Quote:
You will need to delete the corrupted images.

Gallery allows you to replace images, so if you have the originals you could try a replace first.

Also you can rebuild resizes and thumbs per album so you won't run into the memory limits.

-s
FlashYourWeb and Your Gallery with The E2 XML Media Player for Gallery2

 
Dayo

Joined: 2005-11-04
Posts: 1642
Posted: Sun, 2011-01-23 16:29

Good idea ... hopefully they were not corrupted before uploading them.
--
dakanji.com

 
chadeldridge

Joined: 2008-05-12
Posts: 19
Posted: Sun, 2011-01-23 16:40

Can i just overwrite the full size pictures with the originals or do i have to delete them all and readd them?

 
Dayo

Joined: 2005-11-04
Posts: 1642
Posted: Sun, 2011-01-23 16:47

Two options.

1. Replace from within G2. (needs reupload module)
2. Replace directly in the picdata folder.

I'll go for Option 1 first.

PS. You may want to change your toolkit back to IM as a 32m limit with 9000 images is not likely to fly.

--
dakanji.com

 
chadeldridge

Joined: 2008-05-12
Posts: 19
Posted: Sun, 2011-01-23 16:52

Well i just deleted the files and now running the rebuild script gives a more ominous error:

Debug output for failed items:

file_exists(/home/content/c/h/a/chadeldridge/html/gallery2/picdata/cache/entity/8/5/8594.inc)
file_exists(/home/content/c/h/a/chadeldridge/html/gallery2/picdata/cache/derivative/8/5/8594.dat)
file_exists(/home/content/c/h/a/chadeldridge/html/gallery2/picdata/cache/entity/8/5/8592.inc)
getParameter lock.system for core plugin
file_exists(/home/content/c/h/a/chadeldridge/html/gallery2/picdata/cache/module/core/0/0/0.inc)
fopen(/home/content/c/h/a/chadeldridge/html/gallery2/picdata/locks/8594,
wb+, 0)
flock(Resource id #4077, 6, )
mysqlt error: [2013: Lost connection to MySQL server during query] in
EXECUTE("SELECT g_id, g_entityType FROM g2_Entity WHERE g_id IN (8594)")

(mysqlt): SELECT g_id, g_entityType FROM g2_Entity WHERE g_id IN (8594)

2013: Lost connection to MySQL server during query

ADOConnection._Execute(SELECT g_id, g_entityType FROM
g2_Entity WHERE g_id IN (8594)) % line
872, file: adodb.inc.php

ADOConnection.Execute(SELECT g_id,
g_entityType FROM g2_Entity WHERE g_id IN (?), Array[1]) % line 2020, file: GalleryStorageExtras.class

GalleryStorageExtras._identifyEntities(Array[1]) % line 98, file: GalleryStorageExtras.class

GalleryStorageExtras.loadEntities(Array[1]) % line 313, file: GalleryStorage.class

GalleryStorage.loadEntities(Array[1]) % line 520, file: GalleryStorageExtras.class

Error: building 8594: Error (ERROR_STORAGE_FAILURE)

 
Dayo

Joined: 2005-11-04
Posts: 1642
Posted: Sun, 2011-01-23 16:58

I thought you were going to replace the items.

--
dakanji.com

 
chadeldridge

Joined: 2008-05-12
Posts: 19
Posted: Sun, 2011-01-23 17:03

just attempted to delete them from inside album2 and then rebuild .. i have all the originals so its not a problem to put them back. The deletes went fine, however the rebuilds seem to still be broken.

 
Dayo

Joined: 2005-11-04
Posts: 1642
Posted: Sun, 2011-01-23 17:04

Anyway, a quick Google search shows that MySQL Error 2013: "Lost connection to MySQL server during query" is a network connectivity issue. Try again. Maybe the server you are hosted on was having a blip.

--
dakanji.com

 
Dayo

Joined: 2005-11-04
Posts: 1642
Posted: Sun, 2011-01-23 17:07
chadeldridge wrote:
The deletes went fine, however the rebuilds seem to still be broken.

Which rebuilds? If you have deleted the file, it shouldn't have a rebuild version.

If you mean other rebuilds, then you need to go back to the error log and work your way through every .jpg file listed. My guess is that he process stops after so many errors so that if you fix / remove the causes, it will move on to the next ones down the line which you then need to similarly address.

--
dakanji.com

 
chadeldridge

Joined: 2008-05-12
Posts: 19
Posted: Sun, 2011-01-23 17:14

Thanks for your help seems the re builds are working fine now howecver it gets about 50% done and then stops .. my guess is its running out of memory space even using IM and not GD ... so i will call the hosting company and see if they can allocate more memory.

 
Dayo

Joined: 2005-11-04
Posts: 1642
Posted: Sun, 2011-01-23 17:17

IM does that on my setup as well and I only have a few images.

Just keep restarting the process as it should pick up where it stopped in effect and proceed until it either falls over again or gets to the end.
--
dakanji.com

 
chadeldridge

Joined: 2008-05-12
Posts: 19
Posted: Sun, 2011-01-23 17:20

Thanks a ton ... i really do appreciate it. I really did not want to have to redo all this huge amount of work setting up 9k pictures.

 
suprsidr
suprsidr's picture

Joined: 2005-04-17
Posts: 8339
Posted: Sun, 2011-01-23 18:14
suprsidr wrote:
Also you can rebuild resizes and thumbs per album so you won't run into the memory limits.

-s
FlashYourWeb and Your Gallery with The E2 XML Media Player for Gallery2