[SOLVED] External random image (imageblock) takes too long to load. Possibility to pre-load?

peter_k

Joined: 2005-09-15
Posts: 134
Posted: Wed, 2005-10-05 13:41

I use the @readfile(...) method to place a random image on my website's homepage. However, it makes the page very slow (2-5 sec) to load (test yourself) (and since I have the image in the sidemenu -by design-, it shows on every page, which slows down considerably the navigation of the entire site).

Hence my question: is it possible to pre-load this image to reduce load time?

I am not really sure whether this would be possible (from reading previous post on this topic, I understood that the @readfile() command loads the entire G2 framework, which takes time)

But I was thinking about a workaround: the external imageblock php command would not display the image, but would save the image in a specific (fixed) location (e.g. \images\g2randomimg.jpg), and the link to a corresponding .txt file (so you can put that command at the end of the page)

So the way it works is:
- in the html of the page, you load the existing saved jpg, and build the <a> link using the txt (both were saved when a previous page was loaded, possibly even by another visitor)
- at the end of the page, the @readfile() command will replace the existing saved jpg and txt with new ones (which then will be used when another page is loaded)

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Wed, 2005-10-05 14:17

your talking about another caching mechanism. you can do that for your own website since your server is very slow. but g2 itself already has some levels of caching, so usually the imageblock works quite fast and i don't see the necessity for a change.
and your approach doesn't sound very convincing...

have you compared the page loading time of the external imageblock vs. a g2 album page vs. a website page with embedded external imageblock? what's the outcome?

 
peter_k

Joined: 2005-09-15
Posts: 134
Posted: Wed, 2005-10-05 16:15

My idea is indeed a kind of caching mechanism, but for a very specific objective, i.e. getting the external imageblock working at an acceptable speed (which isn't at all the case at the moment). I don't think it would "compete" with G2's caching mechanism.

Well, I compared the loading time of my (non-G2) pages before and after adding the imageblock, and the differences are huge (2-5 seconds). I don't have imageblock on my album pages, but loading time of pages is acceptable (when I'm loading images and photos I accept that there is a small delay, not so on the main site pages).

I read the embedding docs, but I don't think embedding G2 in the rest of the site (which isn't really a database driven application, it's just a bunch of dynamic pages) would increase performance.

But I don't want to argue that my solution is great or should be adopted by the standard Imageblock module, I am just looking for a bit of help on implementing this caching mechanism.

I looked into the Imageblock.tpl file, and I'll need to make two changes:

change

  <a href="{g->url arg1="view=core.ShowItem" arg2="itemId=`$block.id`"}" {strip}
    {if isset($ImageBlockData.linkTarget)}
      target="{$ImageBlockData.linkTarget}"
    {/if}{/strip}>

into something like this

  {$handle=fopen("\tmp\g2randimg.txt", "w")}
  {fwrite($handle, "g->url arg1="view=core.ShowItem" arg2="itemId=`$block.id`"")}
  {fclose($handle)}

and also change

    {$smarty.capture.link}
      {g->image item=$imageItem image=$block.thumb class="giThumbnail" maxSize=$maxSize}

into something like this

    {copy('src of g->image item=$imageItem image=$block.thumb 
class="giThumbnail" maxSize=$maxSize', '\tmp\g2randimg.jpg')

My PHP and Smarty experience doesn't go much further than hacking some existing code, so here I'm kind of stuck...
- the first change doesn't seem too complicated to me, but it just doesn't work;
- for the second one, I have no idea how to get the image returned by the g->image call into a file.

All help appreciated.


PS: I know that hacking the core imageblock.tpl file like that is not really good practice, but I use the imageblock module only and alone for this. Next step is to write a specific .inc & .tpl for it ...

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Wed, 2005-10-05 17:10

if you say the page with the external imageblock is slower than g2 itself, then there's another slow component involved. you do a readfile, which is a second http request.
rather you should use
GalleryEmbed::init(array('fullInit' => true, 'activeUserId' => ''));
GalleryEmbed::getImageBlock(array(some arguments, ...));

rather than readfile. could be much faster in your case.
of course this works only if the page and G2 are on the same server.

see docs/EMBEDDING for more information / details on GalleryEmbed

 
peter_k

Joined: 2005-09-15
Posts: 134
Posted: Wed, 2005-10-05 23:16

I tried it but couldn't make it work.

In the file where I want to display the random image I put this:

    <?php
    	include('g2embed.php');
    ?>

and the g2embed.php file contains this:

<?php
/* entry file for embedding G2  */
require_once(dirname(__FILE__) . './photos/embed.php');

$ret = GalleryEmbed::init(array('embedUri'=>'g2embed.php',
'embedPath'=>'/mw1',
'relativeG2Path'=>'photos',
'activeUserId' => '', 
'fullInit' => true ) ); 

print $ret->getAsHtml();

GalleryEmbed::getImageBlock(array('blocks' => 'randomImage', 
'show' => 'none', 
'activeUserId' => '', 
'maxSize' => '120'));
?>

The output of the print $ret->getAsHtml() is

error message wrote:
Error (GALLERY_SUCCESS) in at line 0

The message is a bit confusing (error or success?), but no image is shown, so one (or both?) of the GalleryEmbed must be wrong ...

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2005-10-06 08:52

'embedUri'=>'g2embed.php', should be main.php since you want it to point to your actual g2.
embedPath should be set to photos too i guess.

of course you shouldn't print the the return status, so change
print $ret->getAsHtml();
to

if ($ret->isError()) {
print $ret->getAsHtml();
exit;
}

and in getImageBlock, leave 'activeUserId' => '', away.

plus you don't actually do something with the results:
it should be

list ($ret, $data) = GalleryEmbed::getImageBlock(array(....));
if ($ret->isError()) {
print $ret->getAsHtml();
exit;
}
// do something with data, see modules/core/classes/GalleryEmbed.class function getImageBlock for the structure of data,
 
peter_k

Joined: 2005-09-15
Posts: 134
Posted: Thu, 2005-10-06 14:38

thanks valiant. I don't really understand all the changes, but it worked :-)

So now I use the embed method AND I developed my own caching/buffering mechanism (as described in my first post), and it works great! Virtually no loading-time difference with the situation before (without the randomimage in the sidemenu), and a lot faster than using the plain external or embed method.

if anybody is interested, I can post the .php and .tpl files.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2005-10-06 15:21

cool.
please post your solution, it's always good to have complete solutions in a forum topic. else someone finds this discussion in a year or two and wonders about the details ;)
plus, i'm also curious how you did it.

 
fryfrog

Joined: 2002-10-30
Posts: 3236
Posted: Fri, 2005-10-07 06:45

Putting the image block in an iFrame would have probably fixed it also.

Signature: Like Gallery? Appreciate my help? Think I'm an asshole? Make your point by donating to the Gallery project! Or maybe just visit my website.

 
peter_k

Joined: 2005-09-15
Posts: 134
Posted: Fri, 2005-10-07 07:58

Why would that speed up the time of displaying the image?

 
fryfrog

Joined: 2002-10-30
Posts: 3236
Posted: Fri, 2005-10-07 08:40

An iFrame is created with the specified dimensions and then the rest of the page rendering just keeps going. The iFrame will populate its data as quickly as it can, but it doesn't actually hold up the page draw. Does that make sense? I've seen it used for RSS feeds that might not be super fast but you don't want your page to stall while the RSS feed is queried.

In G1, you could put the album listing in an iFrame to speed up your main page load times and its usability. The sub-album listing in G1 was one of the slowest things you could turn on besides "accurate photo count" :)

Signature: Like Gallery? Appreciate my help? Think I'm an asshole? Make your point by donating to the Gallery project! Or maybe just visit my website.

 
peter_k

Joined: 2005-09-15
Posts: 134
Posted: Fri, 2005-10-07 08:59

Thanks, I understand. That certainly improves usability of the site.
The only inconvenience with the iFrame is that the photo will not show until after finishing the script. The caching method I use allows for instanteneous rendering of the page, including the random image. You can check for yourself here.

As valiant suggested, I'll post a more detailed description and the code when I've some more time.

 
peter_k

Joined: 2005-09-15
Posts: 134
Posted: Mon, 2005-10-10 11:04

As promised, the code of the script that I use for buffering the imageblock (random image) output. It basically implements what I outlined in the first post: the random image displayed is a cached image, and the imageblock module is called after the page has loaded to prepare the random image for the next (or re-freshed) page.

Here's the detail:

- you have to create a directory called "randimgbuffer", where three files will be stored: two buffered images, and a small php file that contains the filename of the thumbnail and the associated link to display the photo page;

- the g2embed.php (which I put in the web root) script does the following:
* initiates the imageblock embedding;
* processes the output : from the html (standard output of the imageblock module), two variables are extracted: the path of the thumbnail, and the associated link;
* the thumbnail is copied into the randimgbuffer directory;
* the php file "g2randimg.php" is updated with three variables: the link, the filename of the copy of the thumbail, and the filename of the previous thumbnail;
* the previous thumbnail is deleted (we store two images, to prevent that the cached thumbnail is deleted before it is displayed, which could happen if some other graphics on the page are slow to load);

<?php
/* make same php file useable on my local server and on hosted server  */
if ($_SERVER['SERVER_NAME'] == "localhost") { $mid="mw1/"; }
else { $mid=""; }

/* configure  G2 embedding  */
require_once(dirname(__FILE__) . '/photos/embed.php'); 
$ret = GalleryEmbed::init(array('embedUri'=>'main.php',
'embedPath'=>$mid.'photos',
'relativeG2Path'=>'photos',
'activeUserId' => '', 
'fullInit' => true ) );

if ($ret->isError()) {
print $ret->getAsHtml();
exit;
}

list ($ret, $data) = GalleryEmbed::getImageBlock(array('blocks' => 'randomImage', 
'show' => 'none', 
'maxSize' => '120'));
if ($ret->isError()) {
print $ret->getAsHtml();
exit;
}

/* extract link & thumbnail path from html block $data */
$cut = "http:";
$suflink = ".html";
$sufthumb = ".jpg";
$results = explode ($cut , strtolower(trim($data)));
$link = $cut.array_shift(explode($suflink, $results[1])).$suflink;
$thumb = $cut.array_shift(explode($sufthumb, $results[2])).$sufthumb;

/* isolate filename of the thumbnail path */
$thumbname = array_pop(explode("/", $thumb));

/* copy the thumbnail to the randimgbuffer directory */
$bufferedthumb = "randimgbuffer/".$thumbname;
if(!copy($thumb, $bufferedthumb)) {echo "failed";}

/* write link & buffered thumb path & old thumb path to php file */
$fname = "randimgbuffer/g2randimg.php";
include ($fname);
$oldthumb = $rithumb;

$content = sprintf('<?php $rilink="%s"; $rithumb="%s"; $rioldthumb="%s"; ?>', 
$link, $bufferedthumb, $oldthumb);

$fhandle = fopen($fname, 'w') or die("can't open file");
fwrite($fhandle, $content);
fclose($fhandle);
unlink ($rioldthumb);
?>

- on the page where you want to put the random image, you put this code (you'll have to change the width & height settings of the image for your default thumbnail sizes):

<?php
	include('./randimgbuffer/g2randimg.php');
	echo '<a href="'.$rilink.'"><img src="'. $rithumb.'" width=120 height=120 /></a>';
?>

and on the bottom of the page (as you can see, I put it outside the actual html), the main script is called:

</body>
</html>

<?php
	include('g2embed.php');
?>

That's it. To see it in action: http://www.mir-w-art.org/accueil.html (the random-image is the thumbnail in the side-menu).

3 comments:
# you have to create the "/randimgbuffer/" directory yourself;
# the script is based on interpreting G2 URLs with the mod_rewrite module enabled, so it will *not* work without mod_rewrite (you'll have to change the part of the script that extracts the link & thumbnail path from the html block to make it work without mod_rewrite);
# when you run the script for the first time on your server, you'll have to initialize it by refreshing your page a couple of times. The first time, you'll get off course no image on your page (no buffered thumbnail has been created yet). Refresh the page: now you get your image, and normally also a warning because the script tries to delete (unlink) an non-existing file. Refresh again, and now the script is fully initialized.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Mon, 2005-10-10 11:07

thanks :)
maybe i was confused before, but now i understand what you did there :)

 
peter_k

Joined: 2005-09-15
Posts: 134
Posted: Tue, 2005-10-11 20:54

ok. hope you liked it ... :-)

 
peter_k

Joined: 2005-09-15
Posts: 134
Posted: Fri, 2005-10-21 13:12

I made changes to the php script (I edited the original post and updated the script) for stability reasons.
Parsing the html output is not based anymore on the presence of the "?" character, because, based on my understanding of G2, the "?" (preceding the "session_ID") are only present if user does not accept cookies.

 
jshore

Joined: 2004-06-30
Posts: 7
Posted: Sat, 2006-06-10 01:41

I was having the same problem with a 500px random image block on the index page of my site. It was darn slow. At first I tried this solution, but unfortunately url rewrite on IIS does not address image tags and I could not find a work around. (Plus the URL rewrite for IIS seems to have a few other bugs/issues that made me decide to stop using it.) So instead I tried to enhance the ImageBlockHelper::_getBlockData function to address the issue My intension was to add logic such that any time a full-size image was requested, rather than just returning the full size block, the code would check the block's maxSize parameter and find the smallest resize that was greater than or equal to the maxSize and return the resize. At the very least I figured that the performance should benefit from returning a smaller image; however, I did not see much of a change from that. Where I saw the greatest improvement was when I could find a resize that was the exact match for the image block. Ensuring that the browser does not have to resize the image made a huge improvement. So even with this logic, I still had to go an create a specific resize to match the image block size that I wanted. Anyway it seems to be working great for me. Here is a copy of the code that I added, its my first attempt at php so any feedback would be welcome.

I added this code just after it attempts to find a preferred image (btw - what is a preferred image vs a resize?) around line 269. I am using the Gallery 2.1 release code - not the svn.

        $image = isset($preferred[$id]) ? $preferred[$id] : $item; 
/* start new code here */
        if (!$preferred) {
            list ($ret, $resizes) = GalleryCoreApi::fetchResizesByItemIds(array($id));
		    if ($ret) {
		        return array($ret->wrap(__FILE__, __LINE__), null);
		    }
 
            function parseMaxDimension($resize){
                $dims = explode(",",substr($resize->getDerivativeOperations(),strpos($resize->getDerivativeOperations(),"|")+1));
                return max($dims[1],$dims[2]);
            }
            function cmp($a, $b) {               
                return strcmp(parseMaxDimension($a),parseMaxDimension($b));
            }

            if (isset($resizes[$id])){
                usort($resizes[$id], "cmp");
                $i=0;
                   while ( $i < sizeof($resizes[$id])){
                    if (parseMaxDimension($resizes[$id][$i]) >= $maxSize){
                        $image= $resizes[$id][$i];
                        break;  
                    }
                    $i++;    
                }
            } elseif (parseMaxDimension($resizes) >= $maxSize){
                $image = $resizes;
            } 
/* end new code */
	    } else {
		list ($ret, $thumbnail) = GalleryCoreApi::fetchThumbnailsByItemIds(array($id));

I also had to change the signature of the _getBlockData function to include the maxSize. I made it optional for backwards compatibility, but since its a private function I assume I have control of the interface anyway.

Here is thew new interface:

function _getBlockData($title, $itemType, $order, $count,
			   $parentId=null, $fullSize=false, $userId=null, $maxSize=null)

Here is the corresponding updated function call:

	    list ($ret, $blockData[$block]) = ImageBlockHelper::_getBlockData(
		($heading && isset($headings[$block])) ? $module->translate($headings[$block]) : '',
		$itemType, $order, $count, $parentId, $fullSize,
		isset($params['userId']) ? $params['userId'] : null, $params['maxSize']);

BTW - I also ran into problems using the Recreate All Resizes option on my top level album. It needed to get through about 1700 picts and I was never able to get it to finish. I had to do it album by album.

Thanks,
J

 
popoutdoor

Joined: 2006-10-20
Posts: 5
Posted: Fri, 2006-10-20 18:16

oops! I post to the wrong thread, sorry all!

 
kendawes

Joined: 2008-03-08
Posts: 10
Posted: Sat, 2008-03-29 17:26

Hello,
I am trying peter_k's code (above) and am having a problem making it work. I am no coder at all, but am I correct in assuming that in the code where Peter has "photos" in 3 places, I should put my gallery2 directory? Any other changes I need to make? With those 3 "photos" changes when I refresh I get a brief empty block (the 120px x 120px) and a small bit of the php line in the page ie '; ?>
Here is the URL to my test page http://www.kendawesphotography.com/fred.html

Any help would be most appreciated!

Thanks
Ken

 
kendawes

Joined: 2008-03-08
Posts: 10
Posted: Fri, 2008-04-04 22:07

Hello Again,
In trying peter_k's code again, I decided to do step one and just make a php page with only his script (and the g2embed.php in the web root) I get a border but no image. http://www.kendawesphotography.com/randomimage.phpI know that I likely blew it in putting in my paths into the g2embed.php. I am enclosing the info I have in hopes that someone can hit me with a clue stick. (php is hardly a forte of mine)

My adaption of peter_k's g3embed.php...

Quote:
<?php
/* make same php file useable on my local server and on hosted server */
/*if ($_SERVER['SERVER_NAME'] == "localhost") { $mid="mw1/"; }
else { $mid=""; }
*/
/* configure G2 embedding */
require_once(dirname(__FILE__) . '/gallery_db/embed.php');
$ret = GalleryEmbed::init(array('embedUri'=>'main.php',
'embedPath'=>$mid.'gallery_db/',
'relativeG2Path'=>'gallery_db/',
'activeUserId' => '',
'fullInit' => true ) );

if ($ret->isError()) {
print $ret->getAsHtml();
exit;
}

list ($ret, $data) = GalleryEmbed::getImageBlock(array('blocks' => 'randomImage',
'show' => 'none',
'maxSize' => '120'));
if ($ret->isError()) {
print $ret->getAsHtml();
exit;
}

/* extract link & thumbnail path from html block $data */
$cut = "http:";
$suflink = ".html";
$sufthumb = ".jpg";
$results = explode ($cut , strtolower(trim($data)));
$link = $cut.array_shift(explode($suflink, $results[1])).$suflink;
$thumb = $cut.array_shift(explode($sufthumb, $results[2])).$sufthumb;

/* isolate filename of the thumbnail path */
$thumbname = array_pop(explode("/", $thumb));

/* copy the thumbnail to the randimgbuffer directory */
$bufferedthumb = "randimgbuffer/".$thumbname;
if(!copy($thumb, $bufferedthumb)) {echo "failed";}

/* write link & buffered thumb path & old thumb path to php file */
$fname = "randimgbuffer/g2randimg.php";
include ($fname);
$oldthumb = $rithumb;

$content = sprintf('<?php $rilink="%s"; $rithumb="%s"; $rioldthumb="%s"; ?>',
$link, $bufferedthumb, $oldthumb);

$fhandle = fopen($fname, 'w') or die("can't open file");
fwrite($fhandle, $content);
fclose($fhandle);
unlink ($rioldthumb);
?>

The php errors I get from my server...

Quote:
PHP Warning: main() [<a href='function.include'>function.include</a>]: Failed opening './randimgbuffer/g2randimg.php' for inclusion (include_path='.:/usr/local/lib/php-4.4.7/lib/php') in /hermes/web04/b1627/blu.bdweb317285/randomimage.php on line 1
PHP Warning: main(./randimgbuffer/g2randimg.php) [<a href='function.main'>function.main</a>]: failed to open stream: No such file or directory in /hermes/web04/b1627/blu.bdweb317285/randomimage.php on line 1
PHP Warning: main(./randimgbuffer/g2randimg.php) [<a href='function.main'>function.main</a>]: failed to open stream: No such file or directory in /hermes/web04/b1627/blu.bdweb317285/randomimage.php on line 1
PHP Warning: main() [<a href='function.include'>function.include</a>]: Failed opening './randimgbuffer/g2randimg.php' for inclusion (include_path='.:/usr/local/lib/php-4.4.7/lib/php') in /hermes/web04/b1627/blu.bdweb317285/randomimage.php on line 1
PHP Warning: main(./randimgbuffer/g2randimg.php) [<a href='function.main'>function.main</a>]: failed to open stream: No such file or directory in /hermes/web04/b1627/blu.bdweb317285/randomimage.php on line 1
PHP Warning: main(./randimgbuffer/g2randimg.php) [<a href='function.main'>function.main</a>]: failed to open stream: No such file or directory in /hermes/web04/b1627/blu.bdweb317285/randomimage.php on line 1

I sure hope someone can give me a clue!

Thanks!
Ken