Timeouts when using Gallery2 Remote API

callingshotgun

Joined: 2007-12-29
Posts: 8

Posted: Sun, 2008-02-24 16:55

I'm not sure if this is a bug on my end, but I'm fairly confident that something is happening server-side to cause this:

Background: The context for this bug is an application I develop called Migratr, which bulk migrates photos between different photo sharing services. Pretty much what this means is a lot of bulk uploading/downloading. The bug I'm encountering has to do with the Gallery Remote API, as implemented in Migratr.

Essentially, when I iterate through all the albums in a gallery, calling "fetch-album-images" on each album, at some point I stop getting information and start timing out on my requests.

To give specific numbers: The gallery I'm working with (Unfortunately I can't give out login information: I just develop the application, and one of my users filed this bug) has 123 albums. I successfully pull the photo information for the first 101 galleries. Once it gets to gallery #102, the request times out.

I do not believe this to be a quirk of the individual gallery, because I tried changing the code to only pull the second half of all the albums (so the number would be smaller) and it worked fine.

To confirm that the timeout was number-of-requests related, I also tried making the code pull all albums EXCEPT #102, at which point it began the timeouts on album #103.

I also attempted setting delays in case the server thought I was DOSing it or something: Tried 2 and 10 second delays, and in both cases, the 102nd album data request failed.

Has anyone else encountered this? Is this something I'm doing wrong, or some sort of "brute force" prevention setting that needs tweaking?

The relevant server information:

Gallery version = 2.2.4 core 1.2.0.6
PHP version = 5.2.4 apache
Webserver = Apache
Database = mysql 5.0.45-community, lock.system=database
Toolkits = ArchiveUpload, Exif, ImageMagick
Acceleration = none, none
Operating system = Linux luke.tchmachines.com 2.6.9-55.0.12.ELsmp #1 SMP Fri Nov 2 11:19:08 EDT 2007 i686
Default theme = matrix
gettext = enabled
Locale = en_US
Browser = Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/523.12.2 (KHTML, like Gecko) Version/3.0.4 Safari/523.12.2
Rows in GalleryAccessMap table = 451
Rows in GalleryAccessSubscriberMap table = 2550
Rows in GalleryUser table = 2
Rows in GalleryItem table = 2549
Rows in GalleryAlbumItem table = 123
Rows in GalleryCacheMap table = 0

Much appreciated.
-Alex

paour

Joined: 2002-08-14
Posts: 1479

Posted: Mon, 2008-02-25 09:53

Hi Alex,

There's no mechanism in the Remote module itself that would cause this. I'm not sure about G2 itself, but there might be. Also, any network components (firewall) between the machine running Migratr and the G2 server?

Did you try to add a delay loop, or reseting the cookies, after 50 requests for example?

Pierre

--
Gallery Remote developer

callingshotgun

Joined: 2007-12-29
Posts: 8

Posted: Mon, 2008-02-25 15:13

As far as Delay loops, I did attempt setting 2 second (and then later, 10 second) delays between requests to the server, with no luck. My next attempt was going to be re-authenticating after a safety of 50 requests or so (I assume that's what you mean by resetting the cookies?) I'm also actively keeping track of the last auth_token returned, and logs show that I'm always sending out the last one received, so that's not the issue (Though I wish it was- that'd be such an easy fix)

I tried turning my firewall off, just in case it was the source of the issue, but again, no luck. I'm starting to wonder if there's some sort of DOS-protection on the webhost being used (http://www.totalchoicehosting.com/) that's simply overreacting- Though I would think that were that the case, the 10-second delay between requests would mitigate the problem.

callingshotgun Joined: 2007-12-29 Posts: 8	Posted: Sun, 2008-02-24 16:55
	I'm not sure if this is a bug on my end, but I'm fairly confident that something is happening server-side to cause this: Background: The context for this bug is an application I develop called Migratr, which bulk migrates photos between different photo sharing services. Pretty much what this means is a lot of bulk uploading/downloading. The bug I'm encountering has to do with the Gallery Remote API, as implemented in Migratr. Essentially, when I iterate through all the albums in a gallery, calling "fetch-album-images" on each album, at some point I stop getting information and start timing out on my requests. To give specific numbers: The gallery I'm working with (Unfortunately I can't give out login information: I just develop the application, and one of my users filed this bug) has 123 albums. I successfully pull the photo information for the first 101 galleries. Once it gets to gallery #102, the request times out. I do not believe this to be a quirk of the individual gallery, because I tried changing the code to only pull the second half of all the albums (so the number would be smaller) and it worked fine. To confirm that the timeout was number-of-requests related, I also tried making the code pull all albums EXCEPT #102, at which point it began the timeouts on album #103. I also attempted setting delays in case the server thought I was DOSing it or something: Tried 2 and 10 second delays, and in both cases, the 102nd album data request failed. Has anyone else encountered this? Is this something I'm doing wrong, or some sort of "brute force" prevention setting that needs tweaking? The relevant server information: Gallery version = 2.2.4 core 1.2.0.6 PHP version = 5.2.4 apache Webserver = Apache Database = mysql 5.0.45-community, lock.system=database Toolkits = ArchiveUpload, Exif, ImageMagick Acceleration = none, none Operating system = Linux luke.tchmachines.com 2.6.9-55.0.12.ELsmp #1 SMP Fri Nov 2 11:19:08 EDT 2007 i686 Default theme = matrix gettext = enabled Locale = en_US Browser = Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/523.12.2 (KHTML, like Gecko) Version/3.0.4 Safari/523.12.2 Rows in GalleryAccessMap table = 451 Rows in GalleryAccessSubscriberMap table = 2550 Rows in GalleryUser table = 2 Rows in GalleryItem table = 2549 Rows in GalleryAlbumItem table = 123 Rows in GalleryCacheMap table = 0 Much appreciated. -Alex
	XUser login Username: * Password: * Create new account Request new password
paour Joined: 2002-08-14 Posts: 1479	Posted: Mon, 2008-02-25 09:53
	Hi Alex, There's no mechanism in the Remote module itself that would cause this. I'm not sure about G2 itself, but there might be. Also, any network components (firewall) between the machine running Migratr and the G2 server? Did you try to add a delay loop, or reseting the cookies, after 50 requests for example? Pierre -- Gallery Remote developer

callingshotgun Joined: 2007-12-29 Posts: 8	Posted: Mon, 2008-02-25 15:13
	As far as Delay loops, I did attempt setting 2 second (and then later, 10 second) delays between requests to the server, with no luck. My next attempt was going to be re-authenticating after a safety of 50 requests or so (I assume that's what you mean by resetting the cookies?) I'm also actively keeping track of the last auth_token returned, and logs show that I'm always sending out the last one received, so that's not the issue (Though I wish it was- that'd be such an easy fix) I tried turning my firewall off, just in case it was the source of the issue, but again, no luck. I'm starting to wonder if there's some sort of DOS-protection on the webhost being used (http://www.totalchoicehosting.com/) that's simply overreacting- Though I would think that were that the case, the 10-second delay between requests would mitigate the problem.

Timeouts when using Gallery2 Remote API

XUser login