album.dat and photos.dat got zeroed out resulting in invalid albums

kenalker
kenalker's picture

Joined: 2008-11-30
Posts: 1
Posted: Sun, 2008-11-30 06:09
Quote:
The following information is required to get an answer:
Get this information from the PHP diagnostic (in the configuration wizard).
Gallery URL (optional but very useful): www.alker.net/gallery
Gallery version: 1.4.4-pl6
Apache version: 1.3.31
PHP version (don't just say PHP 4, please): PHP v4.3.8
Graphics Toolkit: ImageMagick-6.2.2 (I think, but probably does not matter)
Operating system: FreeBSD 4.11-RELEASE-p16
Web browser/version (if applicable): Firefox 3.0.4 (or 3.0.2/3 at the time; unsure, but probably does not matter)

Last month I viewed some pictures in an album and they looked just fine. I left these open in Firefox on my desktop computer (and they remain open to this day). A few days ago I tried to view these same pictures via my iPhone and noticed the albums in which those picture should reside were no longer showing up in my Gallery home page. I got home later in the day and logged into Gallery as admin and got a window with this error:

Quote:
Attention Gallery Administrator!

Gallery has detected the following 8 invalid album(s) in your albums directory
(/home/users/alker/public_html//albums):

Anapamu-speed-sensor
Easter_2006
data-center
CarPosters
Tiger-Lily-the-cat
Lompoc-radio-upgrade
data-center-ATS
album01

Please move it/them out of the albums directory.

My gallery install is old, simple, has only a handful of albums, and is very rarely used. I'm the only one with admin access to it and few people view the contents, although it is publicly available to anyone on the Internet.

Upon browsing the directory structure of "public_html/albums" I found that every album reported as invalid (see above quoted error) has zero byte album.dat, album.dat.bak, photos.dat, and photos.dat.bak files. I also found that several other albums have from 1-15 photos.dat.x (x=integer) files, every one of which is zero bytes. All zero byte files have date stamps between 10/10/08 and 10/14/08. (See quoted list at end of this post). I've not modified any of these albums in nearly a year and the only album I added this year was in 9/08.

I inspected a backup of my "public_html/albums" from 10/8/08 and found that there are no zero byte album* or photos* files and no photos.dat.x files at all. Thus, all the files that became zero bytes did so between 10/10/08 and 10/14/08 (see above) and all the photos.dat.x files were created in this same time frame.

It is very odd that I've had gallery installed and running for many years (not using it very actively) and over a 5-day window it suddenly had problems. Although I was not updating or writing to my albums during this time, I have determined through experimentation that just viewing the album contents can cause the album.dat, album.dat.bak, photos.dat, and photos.dat.bak files to be updated (obvious to experienced users, I'm sure). I was most likely very actively viewing some of these albums during that 5-day window, however. I was using my Windows desktop my MacBook Pro to view the files.

I find it noteworthy that the Gallery software looks to see if photos.dat.x (meant to be a temporary file) exists before overwriting it and loops, incrementing x until it comes up with a filename that does not exist. I wonder why this would be and if it should be telling me something about the fact that I suddenly have a bunch of these files floating around (all zero-bytes and many with identical time-stamps). It is almost as if Gallery expects these to be floating around and is avoiding overwriting them for some reason (maybe resolution for some previous bug?).

I should also mention that we migrated all data for the server upon which Gallery runs from a directly attached JBOD device to an external NAS box in 9/08. I assume this is irrelevant, but mention it for completeness. In inspecting the files, I found that permissions and ownership all appear to be fine. Based on my dates, you'll see that the problem occurred after migrating to the NAS, but about a month went by before the problem occurred.

Here is my question:
Has anyone seen this behavior in the past? If so, do you know if it has been cured in a current release (I can't find anything in ChangeLog indicating such) and can you please help me understand what would cause this? It not, should I report it as a bug? I have not been able to reproduce it. I know I'm quite behind and need to upgrade.

My best-guess (after hours of research into code, forums, etc.) is that either a) there was a 5-day problem with writing to my NAS (far-fetched, and no other users reported problems), or b) there is a bug in the version of Gallery I am running (maybe a locking problem on the files that are being re-written), or perhaps c) there is a bug in the "rename" PHP function that is called when renaming photos.dat to photos.dat.bak and then photos.dat.x to photos.dat, or d) something else I'm not considering. With considerable research I found that there were problems with the serialize function in Gallery that could lead to album.dat file corruption, but the only references to these bugs in the ChangeLog appear to be cured in versions previous to the version I'm running. Elsewhere, I found people complaining about ending up with zero-byte files due to the rename function under some pretty specific conditions (both in ANSI C and in use with PHP), but I'm not sure those conditions apply here.

For completeness, following is a list of all zero-byte files in my "public_html/albums" directory (excluding all the .lock files, which I presume are supposed to be in every album and are all supposed to be zero bytes):

Quote:
1070808 0 -rw-r--r-- 1 www www 0 Oct 11 12:53 ./Anapamu-speed-sensor/album.dat
657883 0 -rw-r--r-- 1 www www 0 Oct 11 09:33 ./Anapamu-speed-sensor/album.dat.bak
495803 0 -rw-r--r-- 1 www www 0 Oct 11 09:33 ./Anapamu-speed-sensor/photos.dat.bak
829989 0 -rw-r--r-- 1 www www 0 Oct 11 12:53 ./Anapamu-speed-sensor/photos.dat
434242 0 -rw-r--r-- 1 www www 0 Oct 11 09:42 ./CarPosters/album.dat.bak
782918 0 -rw-r--r-- 1 www www 0 Oct 11 09:42 ./CarPosters/photos.dat.bak
882560 0 -rw-r--r-- 1 www www 0 Oct 11 09:55 ./CarPosters/photos.dat
882565 0 -rw-r--r-- 1 www www 0 Oct 11 09:55 ./CarPosters/album.dat
760560 0 -rw-r--r-- 1 www www 0 Oct 11 09:40 ./DavidLoveAntennas/photos.dat.0
882578 0 -rw-r--r-- 1 www www 0 Oct 12 18:57 ./DavidLoveAntennas/photos.dat.1
181885 0 -rw-r--r-- 1 www www 0 Oct 13 18:40 ./Easter_2006/album.dat
463147 0 -rw-r--r-- 1 www www 0 Oct 13 16:27 ./Easter_2006/album.dat.bak
834984 0 -rw-r--r-- 1 www www 0 Oct 13 16:27 ./Easter_2006/photos.dat.bak
696330 0 -rw-r--r-- 1 www www 0 Oct 13 18:40 ./Easter_2006/photos.dat
629755 0 -rw-r--r-- 1 www www 0 Oct 14 07:03 ./January-2005-rains/photos.dat.0
623819 0 -rw-r--r-- 1 www www 0 Oct 14 07:52 ./January-2005-rains/photos.dat.1
643292 0 -rw-r--r-- 1 www www 0 Oct 12 17:21 ./Lompoc-radio-upgrade/album.dat.bak
181892 0 -rw-r--r-- 1 www www 0 Oct 13 20:45 ./Lompoc-radio-upgrade/album.dat
992016 0 -rw-r--r-- 1 www www 0 Oct 12 17:21 ./Lompoc-radio-upgrade/photos.dat.bak
850463 0 -rw-r--r-- 1 www www 0 Oct 13 20:45 ./Lompoc-radio-upgrade/photos.dat
623881 0 -rw-r--r-- 1 www www 0 Oct 13 18:25 ./Noahs_first_birthday/photos.dat.0
637505 0 -rw-r--r-- 1 www www 0 Oct 11 12:54 ./Rio-Chesluk/photos.dat.0
629761 0 -rw-r--r-- 1 www www 0 Oct 11 12:54 ./Rio-Chesluk/photos.dat.1

873840 0 -rw-r--r-- 1 www www 0 Oct 11 12:54 ./Rio-Chesluk/photos.dat.2
873842 0 -rw-r--r-- 1 www www 0 Oct 11 12:54 ./Rio-Chesluk/photos.dat.3
873843 0 -rw-r--r-- 1 www www 0 Oct 11 12:54 ./Rio-Chesluk/photos.dat.4
873844 0 -rw-r--r-- 1 www www 0 Oct 11 12:54 ./Rio-Chesluk/photos.dat.5
873845 0 -rw-r--r-- 1 www www 0 Oct 11 12:54 ./Rio-Chesluk/photos.dat.6
873846 0 -rw-r--r-- 1 www www 0 Oct 11 12:54 ./Rio-Chesluk/photos.dat.7
873847 0 -rw-r--r-- 1 www www 0 Oct 11 12:54 ./Rio-Chesluk/photos.dat.8
873848 0 -rw-r--r-- 1 www www 0 Oct 11 12:54 ./Rio-Chesluk/photos.dat.9
763676 0 -rw-r--r-- 1 www www 0 Oct 11 10:42 ./Santa-Ynez-Peak-antenna-change/photos.dat.0
623877 0 -rw-r--r-- 1 www www 0 Oct 13 18:02 ./Santa-Ynez-Peak-installation/photos.dat.0
557461 0 -rw-r--r-- 1 www www 0 Oct 12 06:17 ./Sierra-Brooklyn-Chesluk/photos.dat.0
696282 0 -rw-r--r-- 1 www www 0 Oct 13 15:56 ./Taj-Chesluk/photos.dat.0
873841 0 -rw-r--r-- 1 www www 0 Oct 10 15:02 ./album01/album.dat.bak
829947 0 -rw-r--r-- 1 www www 0 Oct 10 15:02 ./album01/photos.dat.bak
882540 0 -rw-r--r-- 1 www www 0 Oct 10 17:31 ./album01/album.dat
772702 0 -rw-r--r-- 1 www www 0 Oct 10 17:31 ./album01/photos.dat
623882 0 -rw-r--r-- 1 www www 0 Oct 13 18:29 ./Tiger-Lily-the-cat/photos.dat
643534 0 -rw-r--r-- 1 www www 0 Oct 11 11:26 ./Tiger-Lily-the-cat/album.dat.bak
834979 0 -rw-r--r-- 1 www www 0 Oct 11 11:26 ./Tiger-Lily-the-cat/photos.dat.bak
643845 0 -rw-r--r-- 1 www www 0 Oct 13 18:29 ./Tiger-Lily-the-cat/album.dat
643541 0 -rw-r--r-- 1 www www 0 Sep 14 23:44 ./Willow-Springs-VOI/CP3_1480_001.thumb.jpg
643546 0 -rw-r--r-- 1 www www 0 Sep 14 23:48 ./Willow-Springs-VOI/CP3_1578_001.thumb.jpg
643545 0 -rw-r--r-- 1 www www 0 Sep 14 23:44 ./Willow-Springs-VOI/CP3_1578.thumb.jpg
623828 0 -rw-r--r-- 1 www www 0 Oct 12 15:53 ./data-center-AChandlers/photos.dat.0
992012 0 -rw-r--r-- 1 www www 0 Oct 12 17:03 ./data-center-AChandlers/photos.dat.1
882573 0 -rw-r--r-- 1 www www 0 Oct 12 18:08 ./data-center-AChandlers/photos.dat.2
882574 0 -rw-r--r-- 1 www www 0 Oct 12 18:12 ./data-center-AChandlers/photos.dat.3
882577 0 -rw-r--r-- 1 www www 0 Oct 12 18:54 ./data-center-AChandlers/photos.dat.4
623879 0 -rw-r--r-- 1 www www 0 Oct 13 18:18 ./data-center-AChandlers/photos.dat.5
829933 0 -rw-r--r-- 1 www www 0 Oct 10 17:26 ./data-center-ATS/album.dat.bak
829954 0 -rw-r--r-- 1 www www 0 Oct 10 17:26 ./data-center-ATS/photos.dat.bak
829960 0 -rw-r--r-- 1 www www 0 Oct 10 18:25 ./data-center-ATS/photos.dat
390659 0 -rw-r--r-- 1 www www 0 Oct 10 18:25 ./data-center-ATS/album.dat
873849 0 -rw-r--r-- 1 www www 0 Oct 11 12:55 ./data-center-DCsupplies/photos.dat.0
873850 0 -rw-r--r-- 1 www www 0 Oct 11 12:55 ./data-center-DCsupplies/photos.dat.1
873851 0 -rw-r--r-- 1 www www 0 Oct 11 12:55 ./data-center-DCsupplies/photos.dat.2
873852 0 -rw-r--r-- 1 www www 0 Oct 11 12:55 ./data-center-DCsupplies/photos.dat.3
873853 0 -rw-r--r-- 1 www www 0 Oct 11 12:55 ./data-center-DCsupplies/photos.dat.4
873854 0 -rw-r--r-- 1 www www 0 Oct 11 12:55 ./data-center-DCsupplies/photos.dat.5
873855 0 -rw-r--r-- 1 www www 0 Oct 11 12:55 ./data-center-DCsupplies/photos.dat.6
873856 0 -rw-r--r-- 1 www www 0 Oct 11 12:55 ./data-center-DCsupplies/photos.dat.7
873857 0 -rw-r--r-- 1 www www 0 Oct 11 12:55 ./data-center-DCsupplies/photos.dat.8
873858 0 -rw-r--r-- 1 www www 0 Oct 11 12:55 ./data-center-DCsupplies/photos.dat.9
873859 0 -rw-r--r-- 1 www www 0 Oct 11 12:55 ./data-center-DCsupplies/photos.dat.10
873860 0 -rw-r--r-- 1 www www 0 Oct 11 12:55 ./data-center-DCsupplies/photos.dat.11
278005 0 -rw-r--r-- 1 www www 0 Oct 14 02:14 ./data-center-DCsupplies/photos.dat.12
278006 0 -rw-r--r-- 1 www www 0 Oct 14 02:14 ./data-center-DCsupplies/photos.dat.13
278007 0 -rw-r--r-- 1 www www 0 Oct 14 02:14 ./data-center-DCsupplies/photos.dat.14
278008 0 -rw-r--r-- 1 www www 0 Oct 14 02:15 ./data-center-DCsupplies/photos.dat.15
278009 0 -rw-r--r-- 1 www www 0 Oct 14 02:15 ./data-center-DCsupplies/photos.dat.16
278010 0 -rw-r--r-- 1 www www 0 Oct 14 02:15 ./data-center-DCsupplies/photos.dat.17
278011 0 -rw-r--r-- 1 www www 0 Oct 14 02:15 ./data-center-DCsupplies/photos.dat.18
278012 0 -rw-r--r-- 1 www www 0 Oct 14 02:15 ./data-center-DCsupplies/photos.dat.19
278013 0 -rw-r--r-- 1 www www 0 Oct 14 02:15 ./data-center-DCsupplies/photos.dat.20
278014 0 -rw-r--r-- 1 www www 0 Oct 14 02:15 ./data-center-DCsupplies/photos.dat.21
278015 0 -rw-r--r-- 1 www www 0 Oct 14 02:15 ./data-center-DCsupplies/photos.dat.22
278016 0 -rw-r--r-- 1 www www 0 Oct 14 02:15 ./data-center-DCsupplies/photos.dat.23
278017 0 -rw-r--r-- 1 www www 0 Oct 14 02:15 ./data-center-DCsupplies/photos.dat.24
278018 0 -rw-r--r-- 1 www www 0 Oct 14 02:15 ./data-center-DCsupplies/photos.dat.25
278019 0 -rw-r--r-- 1 www www 0 Oct 14 02:15 ./data-center-DCsupplies/photos.dat.26
882683 0 -rw-r--r-- 1 www www 0 Oct 13 02:00 ./data-center-UPS/photos.dat.0
772697 0 -rw-r--r-- 1 www www 0 Oct 10 16:43 ./data-center-genset/photos.dat.0
696281 0 -rw-r--r-- 1 www www 0 Oct 13 15:53 ./data-center-genset/photos.dat.1
277278 0 -rw-r--r-- 1 www www 0 Oct 14 02:13 ./data-center-genset/photos.dat.2
277994 0 -rw-r--r-- 1 www www 0 Oct 14 02:13 ./data-center-genset/photos.dat.3
277995 0 -rw-r--r-- 1 www www 0 Oct 14 02:13 ./data-center-genset/photos.dat.4
277996 0 -rw-r--r-- 1 www www 0 Oct 14 02:13 ./data-center-genset/photos.dat.5
277997 0 -rw-r--r-- 1 www www 0 Oct 14 02:13 ./data-center-genset/photos.dat.6

277998 0 -rw-r--r-- 1 www www 0 Oct 14 02:13 ./data-center-genset/photos.dat.7
277999 0 -rw-r--r-- 1 www www 0 Oct 14 02:13 ./data-center-genset/photos.dat.8
278000 0 -rw-r--r-- 1 www www 0 Oct 14 02:14 ./data-center-genset/photos.dat.9
278002 0 -rw-r--r-- 1 www www 0 Oct 14 02:14 ./data-center-genset/photos.dat.10
278004 0 -rw-r--r-- 1 www www 0 Oct 14 02:14 ./data-center-genset/photos.dat.11
184653 0 -rw-r--r-- 1 www www 0 Oct 14 04:24 ./data-center-genset/photos.dat.12
97252 0 -rw-r--r-- 1 www www 0 Oct 11 21:36 ./data-center/album.dat.bak
882260 0 -rw-r--r-- 1 www www 0 Oct 14 02:13 ./data-center/album.dat
1031343 0 -rw-r--r-- 1 www www 0 Oct 11 21:36 ./data-center/photos.dat.bak
696344 0 -rw-r--r-- 1 www www 0 Oct 14 02:13 ./data-center/photos.dat
992013 0 -rw-r--r-- 1 www www 0 Oct 12 17:06 ./dpengineering-in-phoenix/photos.dat.0
992014 0 -rw-r--r-- 1 www www 0 Oct 12 17:14 ./dpengineering-in-phoenix/photos.dat.1
887181 0 -rw-r--r-- 1 www www 0 Oct 12 17:54 ./dpengineering-in-phoenix/photos.dat.2
623876 0 -rw-r--r-- 1 www www 0 Oct 13 17:56 ./fix-shower/photos.dat.0
623878 0 -rw-r--r-- 1 www www 0 Oct 13 18:16 ./fix-shower/photos.dat.1
1322702 0 -rw-r--r-- 1 www www 0 Oct 13 19:13 ./fix-shower/photos.dat.2
623875 0 -rw-r--r-- 1 www www 0 Oct 13 17:10 ./offices-for-rent/photos.dat.0
830046 0 -rw-r--r-- 1 www www 0 Oct 13 20:03 ./offices-for-rent/photos.dat.1
623787 0 -rw-r--r-- 1 www www 0 Oct 11 08:37 ./papacar/photos.dat.0
873861 0 -rw-r--r-- 1 www www 0 Oct 11 12:55 ./papacar/photos.dat.1
873862 0 -rw-r--r-- 1 www www 0 Oct 11 12:55 ./papacar/photos.dat.2
873863 0 -rw-r--r-- 1 www www 0 Oct 11 12:55 ./papacar/photos.dat.3
873864 0 -rw-r--r-- 1 www www 0 Oct 11 12:55 ./papacar/photos.dat.4
873865 0 -rw-r--r-- 1 www www 0 Oct 11 12:55 ./papacar/photos.dat.5
873866 0 -rw-r--r-- 1 www www 0 Oct 11 12:55 ./papacar/photos.dat.6
873867 0 -rw-r--r-- 1 www www 0 Oct 11 12:55 ./papacar/photos.dat.7
873868 0 -rw-r--r-- 1 www www 0 Oct 11 12:55 ./papacar/photos.dat.8
696287 0 -rw-r--r-- 1 www www 0 Oct 13 17:28 ./papacar/photos.dat.9
696327 0 -rw-r--r-- 1 www www 0 Oct 13 17:34 ./papacar/photos.dat.10
623880 0 -rw-r--r-- 1 www www 0 Oct 13 18:24 ./papacar/photos.dat.11
1070889 0 -rw-r--r-- 1 www www 0 Oct 13 19:32 ./papacar/photos.dat.12
882572 0 -rw-r--r-- 1 www www 0 Oct 13 19:48 ./papacar/photos.dat.13
830004 0 -rw-r--r-- 1 www www 0 Oct 13 20:28 ./papacar/photos.dat.14