Database errors stemming from bloated g2_cache table

phidelity

Joined: 2004-09-21
Posts: 5
Posted: Tue, 2010-01-26 16:49

Hello there,
I have been having issues with my webhost (Mediatemple) and the way it interacts with my databases. Mediatemple offers this service called SQL Bursting - where they will put you in a temporary ' burst container' if your resource demands exceed the amount they have alloted for you. However the past two times this has occurred 50 or so of the tables in my databases simply go missing. After 4 phone calls and over two hours of hold time Ive finally some explanation as to why - which is below

Quote:
After much work, your db14349_phidelityPhotos2 database is back online. The crux of the problem was an overbloated "g2_CacheMap" table for both db14349_phidelityPhotos2 and db14349_phidelityPhotos3 databases. This is what ultimately caused the BURST process to fail. While attempting to fix your issue, I tried re-running our automated bursting process to sync your data and it failed. Eventually this was done manually.

The reason why you're missing tables for that database is due to the following sequence of events:

1. 2010-01-10 04:04 am - Moved from SmartPool to BURST Container.
2. 2010-01-14 04:09 am - Moved back from BURST Container to a different SmartPool server
3. 2010-01-23 11:50 pm - Moved from 2nd SmartPool server back into a BURST Container (where you currently reside)

From what we can see, the initial move on 2010-01-10 is where the database did not carry over properly. You then subsequently got moved back to a different pool server with the incomplete data and back to your current container with the same incomplete data set.

I went back to the last-known-good SmartPool server where your database has 56 rows and I performed a MySQL dump manually. I have provided you two versions of this file in your /home/14349/data directory - one with the '--skip-extended-insert' option and one without. These are provided to you as a "hard copy" in case you need a complete and recent dump for the future.

At this time the db14349_phidelityPhotos2 database is showing that it has 56 rows. I strongly suggest looking into the g2_CacheMap table for stale data. This is ultimately what caused these issues to manifest.

My question - How exactly would I look into g2_cacheMap for stale data?
thanks kris

 
alecmyers

Joined: 2006-08-01
Posts: 4342
Posted: Tue, 2010-01-26 19:04

So your hosting provider has a crappy and bug-ridden "procedure" if your db gets too big for their oversold and overloaded server - and when it doesn't work it's your fault? niiiiiice.....

I just did some checking, and I think (think!) that if you turn off the acceleration in site admin -> performance, the cache map table isn't used.

That being the case, of course, your cpu utilisation may go up.

 
nivekiam
nivekiam's picture

Joined: 2002-12-10
Posts: 16504
Posted: Tue, 2010-01-26 19:38

I'm looking at and seeing the same thing.

Also, Gallery does appear to clear the cache out properly based on what ever time limits you set there in my testing. I set the cache to 15 minutes and on a properly configured and functioning server, after 15 minutes (more like 75 minutes) I refreshed a page and the cache map table was reduced in size and cleaned up.

I'm also seeing that clearing the cache from Site Admin > Maintenance (both DB and template) doesn't clean up the cache map table. To do that, you need to go to Site Admin > Performance and click on Clear Saved Pages and that seems to clear out the cache map table, forcing Gallery to rebuild that data as people browse your site.
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here

 
nivekiam
nivekiam's picture

Joined: 2002-12-10
Posts: 16504
Posted: Tue, 2010-01-26 22:00

I've just been digging into this a bit more. It looks like if for some reason your cache isn't getting cleared out based on your settings (stale data), then perhaps you could run the command to clear the cache through the API and run that from a cron job? I'm not sure how to do it at all though. Alec may have a better idea.

See line 91 of /modules/core/AdminPerformance.inc and you'll see there how that goes about clearing the cache, of course if you did that through an external script you'd need to do a bit more than simply calling that function.

Then you can take advantage of Gallery's caching feature and make sure (hopefully) that the cache map table doesn't get too big.

Another thing to take a look at is maybe make sure you're blocking robots from browsing your gallery if you don't care about it being listed on search engines, so they are not hitting pages and increasing the cache map table unnecessarily.

What settings do you have set under Site Admin > Performance?
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here