[SOLVED] Gallery 3 Performance Problem

softcat

Joined: 2010-07-04
Posts: 16
Posted: Sun, 2010-07-04 17:36

Hello,

i've had a smoothely running Gallery2 installation and decided to try out Gallery3.

But it's performance is unusable on my machine. It takes 6-8 seconds for a page to generate, while Gallery2 and lots of other things run fine including Wordpress etc.

I've enabled debugging but I'm not able to identity the problem.

Benchmarks	        Count	Time	Memory
Kohana Loading	        1	0.048	0.27MB
Environment Setup	1	0.048	0.20MB
System Initialization	1	1.717	0.55MB
Controller Setup	1	0.138	0.01MB
Controller Execution	1	4.342	0.94MB
Total Execution		        6.245	1.77MB

Full debugging can be found under http://gallery3.softcat.org, this is a fresh install.

Hosting enviroment: Debian Etch in a Virtual VServer, MySQL 5.0.51a-24+lenny4-log, Apache/2.2.9 (Debian) DAV/2 PHP/5.2.6-1+lenny8 with Suhosin-Patch mod_ssl/2.2.9 OpenSSL/0.9.8g

# free
             total       used       free     shared    buffers     cached
Mem:       1310720     125696    1185024          0          0          0
-/+ buffers/cache:     125696    1185024
Swap:            0          0          0
# top
18054 www-data  18   0 82700  34m 6280 S    2  2.7   0:00.47 apache2
    1 root      15   0  1984  684  588 S    0  0.1   0:10.57 init
 7360 root      18   0  1696  600  496 S    0  0.0   0:04.24 syslogd
 7372 root      15   0  5276  740  636 S    0  0.1   0:00.03 sshd
 7694 root      18   0 46680 5640 2640 S    0  0.4   1:27.74 spamd
 7716 root      18   0  1836  328  324 S    0  0.0   0:00.00 courierlogger
 7718 root      18   0  4352  616  592 S    0  0.0   0:00.06 authdaemond
 7727 root      20   0  1836  264  260 S    0  0.0   0:00.00 courierlogger
 7728 root      18   0  1940  432  428 S    0  0.0   0:00.00 couriertcpd
 7751 root      18   0  1836  368  336 S    0  0.0   0:00.00 courierlogger
 7752 root      16   0  1940  468  464 S    0  0.0   0:00.00 couriertcpd
 7760 root      18   0  4352  152  128 S    0  0.0   0:00.07 authdaemond
 7761 root      18   0  4564  808  784 S    0  0.1   0:00.07 authdaemond
 7763 root      18   0  4352  152  128 S    0  0.0   0:00.09 authdaemond
 7764 root      15   0  4564  808  784 S    0  0.1   0:00.08 authdaemond
 7765 root      18   0  4352  152  128 S    0  0.0   0:00.07 authdaemond
 7812 Debian-e  15   0 13188  708  628 S    0  0.1   0:02.73 exim4
 7837 root      21   0  1980  336  332 S    0  0.0   0:00.00 pptpd
 7863 root      18   0  2916  732  640 S    0  0.1   0:00.00 xinetd
 7988 daemon    18   0  2128  320  304 S    0  0.0   0:00.01 atd
 8008 root      18   0  3812  648  580 S    0  0.0   0:03.12 cron
 8114 spamd     15   0 46680 3328  900 S    0  0.3   0:01.70 spamd
 9565 root      16   0  8424 2856 2356 S    0  0.2   0:00.05 sshd
 9591 root      15   0  2652 1112  972 S    0  0.1   0:00.02 bash
 9741 root      16   0  8428 2556 2096 S    0  0.2   0:00.13 sshd
 9746 root      18   0  2832 1512 1172 S    0  0.1   0:00.03 bash
13618 nobody    18   0  4960 2268 1556 S    0  0.2   0:07.84 openvpn
18044 root      18   0 79364  35m 8496 S    0  2.8   0:00.25 apache2
18053 www-data  21   0 83048  37m 8092 S    0  2.9   0:02.02 apache2
18055 www-data  18   0 79364  27m  584 S    0  2.2   0:00.00 apache2
18057 www-data  18   0 79364  27m  580 S    0  2.2   0:00.00 apache2
18058 www-data  22   0 79364  27m  580 S    0  2.2   0:00.00 apache2
18059 www-data  22   0 79364  27m  580 S    0  2.2   0:00.00 apache2
18060 www-data  22   0 79364  27m  580 S    0  2.2   0:00.00 apache2
18061 www-data  22   0 79364  27m  580 S    0  2.2   0:00.00 apache2
19623 root      16   0  8284 2720 2276 S    0  0.2   0:00.04 sshd
19629 root      16   0  2652 1120  972 S    0  0.1   0:00.00 bash
20210 root      18   0  2256  872  704 R    0  0.1   0:00.00 top
22386 root      25   0  2476  952  948 S    0  0.1   0:00.00 mysqld_safe
22400 mysql     15   0  141m  26m 4232 S    0  2.1   0:40.14 mysqld
22401 root      16   0  1632  496  464 S    0  0.0   0:00.00 logger

Can I help debugging with any additional information?

Many thanks and regards,
Nils

 
floridave
floridave's picture

Joined: 2003-12-22
Posts: 27300
Posted: Sun, 2010-07-04 19:36

check this out:
http://gallery.menalto.com/node/95109

Dave
_____________________________________________
Blog & G2 || floridave - Gallery Team

 
softcat

Joined: 2010-07-04
Posts: 16
Posted: Mon, 2010-07-05 11:07

Hi Dave,

thank you, I checked the link but this didn't solve the problem because my gallery3 installation is not residing an a network share. Database ist hosted locally either.

Would a webgind output help?

Regards,
Nils

 
floridave
floridave's picture

Joined: 2003-12-22
Posts: 27300
Posted: Mon, 2010-07-05 13:32
Quote:
Would a webgind output help?

Wouldn't hurt.

Dave
_____________________________________________
Blog & G2 || floridave - Gallery Team

 
softcat

Joined: 2010-07-04
Posts: 16
Posted: Tue, 2010-07-06 19:28

My Webgrind output can be found here:

http://gallery3.softcat.org/webgrind/ (please click "update")

Seems like the problem "php::is_file". The files ar not located on a network share like in the link provided.

Would anybody be so kind and would look into it?

Many thanks!

Regards,
Nils

 
bharat
bharat's picture

Joined: 2002-05-21
Posts: 7993
Posted: Wed, 2010-07-07 04:59

Wow. That's crazy -- we're talking about 2ms for each is_file() call. I haven't seen performance that bad where it wasn't on a network share. Are you, by any chance, running any kind of special filesystem? Those is_file() calls should be cached in RAM and be relatively instantaneous.

I'd love to investigate this more closely. Any chance I can ssh in and take a more in-depth look? You seem pretty savvy based on the info you provided, so perhaps you could run some basic benchmark tests on the is_file() call? eg:

<pre>
<?
echo microtime(true), "\n";
for ($i = 0; $i < 5000; $i++) {
  is_file("/var/tmp/some_file_that_exists");
  is_file("/var/tmp/some_file_that_doesnt_exist");
}
echo microtime(true), "\n";
?>

Fix the paths above to be files that do and don't exist, respectively. Try using the same partition as your Gallery3 install is on. On my system I get output like this:

1278478646.8148
1278478646.9079

That's about 93 ms for me and it's doing twice as many is_file() calls as you're seeing (but it's for the same file over and over again so it should be super fast).

---
Problems? Check gallery3/var/logs
bugs/feature req's | upgrade to the latest code | use git

 
softcat

Joined: 2010-07-04
Posts: 16
Posted: Wed, 2010-07-07 16:53

Hi bharat,

I really appreciate your help. This are the results with your benchmark:

1278521357.25 
1278521364.51 

The machine is a Virtuozzo vServer, standard file system.

I'l pn you ssh access...

Again, many thanks.

Regards,
Nils

 
floridave
floridave's picture

Joined: 2003-12-22
Posts: 27300
Posted: Wed, 2010-07-07 23:36

just for comparison this is what I get:

1278545736.01
1278545736.04

Dave
_____________________________________________
Blog & G2 || floridave - Gallery Team

 
bharat
bharat's picture

Joined: 2002-05-21
Posts: 7993
Posted: Thu, 2010-07-08 01:52

@softcat -- thanks for sharing your info via private message. I checked out your system but I see no obvious reason why it would be acting this way. The only clue I have to go on is the fact that you're using vzfs which is because of your virtual host setup. But -- the code snippet that I posted should take milliseconds and it's taking 6-7 seconds so it's 2 orders of magnitude too slow. I'd show that script and its performance (and maybe this thread) to your sysadmin and ask them what's going on because there's probably a configuration change they could make to greatly improve your performance. As far as I can tell, this isn't a Gallery3 thing (though maybe we can figure out a workaround).
---
Problems? Check gallery3/var/logs
bugs/feature req's | upgrade to the latest code | use git

 
softcat

Joined: 2010-07-04
Posts: 16
Posted: Sat, 2010-07-10 18:40

Hi bharat,

thanks a lot. There is definetly something wrong with my apache and php configuration. I've done a test install with lighttp and have run the benchmark again:

1278786681.23 
1278786681.37 

I've no idea whats going wrong with apache...

Regards,
Nils

 
bharat
bharat's picture

Joined: 2002-05-21
Posts: 7993
Posted: Sat, 2010-07-10 19:04

Best to ask your system administrator -- not sure that there's much of anything we can do on our side.

I'm going to mark this topic as solved for now since I don't want people to think we have rampant performance issues in G3. But please let us know what your sysadmin says!
---
Problems? Check gallery3/var/logs
bugs/feature req's | upgrade to the latest code | use git

 
softcat

Joined: 2010-07-04
Posts: 16
Posted: Tue, 2010-07-13 17:27

After hours of debugging I've found which causend my performance problem:

open_basedir in PHP

Disabling it resolves any performance issues I had. I'm still investigating the reason, it shouldn't have that impact.

Regards,
Nils

 
nivekiam
nivekiam's picture

Joined: 2002-12-10
Posts: 16504
Posted: Tue, 2010-07-13 17:39

Please post anything you discover. Some of us can try duplicating or at the very least if we run across this again, we'll have some hint as to why.
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here

 
Q1963

Joined: 2010-09-12
Posts: 5
Posted: Sun, 2010-09-12 21:33

Hi, is this issue really solved?

With RC1+2 i see the same problem on my web-server, a Synology Diskstation 408. These box uses open_basedir to restrict the dirs which can be accessed by php. It need more than 10 seconds to display the next page. I can see 2 httpd processes which each runs at 50% cpu time during this period. As far as i can see usage of open_basedir seems to be an security feature to secure a public web server.
I have another smaller nas, an QNap TS119, which is normally 2-3 times slower than the Synology DS408. This box runs well with gallery3 RC2 and dont use open_basedir.

I offer to test a current build on both boxes to check these performance issue. I didnt found a newer build on gallery.menalto.com.

If you have any question, don't hesitate to contact me. I would really like to run gallery3 on my box.

Frank

 
nivekiam
nivekiam's picture

Joined: 2002-12-10
Posts: 16504
Posted: Sun, 2010-09-12 22:41

Did you try disabling open_basedir on the Diskstation?

Also, is this running Apache 2.2.x, MySQL 5.x, PHP 5.x?

Make sure you're running the latest experimental code, here it is: http://github.com/gallery/gallery3/zipball/master

Upgrade instructions here: http://codex.gallery2.org/Gallery3:Upgrading
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here

 
Q1963

Joined: 2010-09-12
Posts: 5
Posted: Mon, 2010-09-13 09:30

Hi,
another guy on the synology forum tried to disable open_basedir, which solved the issue. But unfortunately this isn't the manufacturer/recommended configuration for this box, so i had to enable open_basedir for a production / internet environment.
Yes, the box matched the version requirements (Apache 2.2.13,PHP 5.2.10, MySQL 5.1.34).
Thanks for the latest experimental code. I will try this build this evening, when i'm back at home.

Thx
Frank

 
Q1963

Joined: 2010-09-12
Posts: 5
Posted: Mon, 2010-09-13 21:20

Hi nivekiam,

i tried the latest experimental build (gallery3-b08bf26). The performance problem still ocurs. To call the next page on a fresh installed systems need ~ 10 seconds. Most of this period two httpd daemons are running with each 50% cpu usage.
When open_basedir in php.ini isn't used the performance is fine. But unfortunately i need open_basedir for security reasons and php.ini will be created by an shell script every time the box is booting.
How can i help to analyse this problem, any logs ???

Thx.
Frank

 
bharat
bharat's picture

Joined: 2002-05-21
Posts: 7993
Posted: Mon, 2010-09-13 23:36

@Q1963: Try this. Edit application/config/config.php and find this block:

    64  /**                                                                                        
    65   * Length of time of the internal cache in seconds. 0 or FALSE means no caching.           
    66   * The internal cache stores file paths and config entries across requests and             
    67   * can give significant speed improvements at the expense of delayed updating.             
    68   */                                                                                        
    69  $config["internal_cache"] = FALSE;                                                         
    70  $config["internal_cache_path"] = VARPATH . "tmp/";                                         

Change it to:

    64  /**                                                                                        
    65   * Length of time of the internal cache in seconds. 0 or FALSE means no caching.           
    66   * The internal cache stores file paths and config entries across requests and             
    67   * can give significant speed improvements at the expense of delayed updating.             
    68   */                                                                                        
    69  $config["internal_cache"] = 3600;                                                         
    70  $config["internal_cache_path"] = VARPATH . "tmp/";                                         

See if that makes things faster for you. It should greatly cut down on the filesystem access *but* it does so at the expense of any module changes (I'll work on making that better).

---
Problems? Check gallery3/var/logs
bugs/feature req's | upgrade to the latest code | use git

 
Q1963

Joined: 2010-09-12
Posts: 5
Posted: Tue, 2010-09-14 01:35

Hi bharat,
it definitly improves performance. New admin pages will be shown within 2-3 seconds. Browsing within an album (50 jpgs) needs 2-4 seconds per page.
I will play around with "internal_cache" to balance performance and changes. Additionally i will upload much more pictures and try to make more accurate measurement with more images and real world access. I will post the result here.
Thx.
Frank

 
Q1963

Joined: 2010-09-12
Posts: 5
Posted: Wed, 2010-09-15 11:51

Hi bharat,

i browsed gallery3 with 1000 jpgs and performance was fine with internal_cache=3600, all user requests were answered within 2-3 seconds. Additionally, after a break i waited 3600 seconds to continue browsing to make sure that the cache is out of date. The response time was similar, i didn't expected this behaviour.
Until now i only installed the default modules, today i will add all other modules which i want to use and will retest.

The picture upload "from a server path" crashed. It starts to scann the upload directory and found none of the 10 directories with 1000 jpgs. I found some crash/debug info in var/logs and will attach this logs when i'm back at home. So i added the jpgs with the regular "add picture" and "organize albums" function, which works fine.

BestRegards,
Frank

 
undagiga

Joined: 2010-11-26
Posts: 693
Posted: Fri, 2011-02-04 00:57
bharat wrote:
.... *but* it does so at the expense of any module changes (I'll work on making that better).

Can you please explain what you mean by "at the expense of any module changes"? Can I assume that this means that if you change a module then you won't see any impact for at least an hour? But if you do a reload then surely it will?

Were you successful in "making it better" in 3.0.1?

U-G