Module: Coralialze - generates image urls that use Coral CDN
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Wed, 2006-01-04 19:51
i think other mirror methods should be implemented in other modules. keep it simple / maintainable.
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2006-01-10 14:23
sandymac
the API changes / rewrite module changes are now in the official CVS / nightly snapshots.
it would be great if you could update your module to work with it.
So I have finally gotten around to installing your module, only to find one minor issue... if I add your coral check to the side bar it makes the side bar exist. What I mean is, I don't have anything else in the side bar so I have none (on purpose). When I add yours, it suddenly exists (reasonable), but with nothing in it (sucky). I spose the solution is to turn the check off?
_________________________________ Support & Documentation || Donate to Gallery || My Website
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2006-01-10 23:54
fryfrog:
is that check in a block? can't you add it to album and photo block lists instead of the sidebar block list?
So I've installed and activated it, unchecked the "check if its working before using" box... but it still doesn't yield any coralized image urls. I've also tried loading my site manually coralized, and it works... investigating!
_________________________________ Support & Documentation || Donate to Gallery || My Website
Robrob
Joined: 2006-01-11
Posts: 6
Posted: Wed, 2006-01-11 19:08
I have this module installed, and it seems to work well.
Should it include a Vary: header in the request, so proxies don't serve the wrong page if the ability of clients to get to Coral changes? I see "coralCheck=passed" appended to the end of the Cookie: header. Is this field relevant to proxies for deciding to serve a cached page or retrieve from the server?
Here's one scenario: Client A is configured to use a proxy and visits a Gallery page, and is able to access Coral cache. The proxy caches the page and the images. After some time, the proxy purges the images from it's cache, but the pages remain. Client B uses the proxy to visit the same Gallery page, but Coral is unavailable/unreachable. Client B gets the cached pages that contain links to the Coralized objects. Incorrect behavior results.
If we make a new header, X-CoralCheck, and include "Vary: X-CoralCheck", we could avoid this issue. See http://www.ietf.org/rfc/rfc2616.txt section 14.44
sandymac
Joined: 2005-09-15
Posts: 31
Posted: Wed, 2006-01-11 21:49
Robrob wrote:
Should it include a Vary: header in the request, [...] I see "coralCheck=passed" appended to the end of the Cookie: header. Is this field relevant to proxies for deciding to serve a cached page [...]
Nope, the cookie is used by the Coralize module only if you have the coral check enabled in the module. What it does is request some JavaScript via the Coral CDN and that JavaScript sets the coralCheck cookie. If the user cannot connect to Coral CDN, the cookie doesn't get set and the Coralize module doesn't ever coralize images.
Because the URLs between a normal image request and a coralized image request are different adding a Vary header doesn't do anything useful.
Robrob
Joined: 2006-01-11
Posts: 6
Posted: Wed, 2006-01-11 21:57
Quote:
Because the URLs between a normal image request and a coralized image request are different
Right... that's why we don't want a page cached by the proxy with coralized image urls being served to a client that isn't able to reach the Coral network. I don't see anything in the current implementation that would prevent this.
sandymac
Joined: 2005-09-15
Posts: 31
Posted: Thu, 2006-01-12 05:01
Robrob wrote:
that's why we don't want a page cached by the proxy with coralized image urls being served to a client that isn't able to reach the Coral network.
Section 13.4 of the HTTP/1.1 spec states that:
HTTP/1.1 Spec wrote:
If there is neither a cache validator nor an explicit expiration time associated with a response, we do not expect it to be cached
I did some packet sniffing and verified my gallery install returns neither a cache validator header nor an explicit expiration header for the xhtml pages. If you find a cache that is violating the spec I encourage you inform the product's creator of their non-compliance.
fryfrog
Joined: 2002-10-30
Posts: 3236
Posted: Mon, 2006-01-23 14:12
I still can't figure out why the module installs and can be configured, but totally fails to coralize any images. I've put my G2 into buffered and immediate debug mode and the only references I can find to coralize are at the points when it loads up the "check". I am thinking that it must be due to the CVS version of G2 that I am using... any plans for moving to current nightly/cvs?
@fryfrog: It could be from having your gallery mixed in with wordpress. Maybe wordpress doesn't use a standard URL Generator class and thus Coralize cannot extend it.
No I'm not using CVS. I only use release versions. I have better things to do than track every Gallery change.
Here is my development gallery: http://dargo.us/gallery2/main.php and it's running the latests Coralize module. You could check to see if images are coralized for you.
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Mon, 2006-01-23 18:42
@sandymac:
Quote:
No I'm not using CVS. I only use release versions. I have better things to do than track every Gallery change.
we're releasing G2.1 on March 1st. and G2.1 is to the most part what we have currently in CVS. the big url generator changes were committed a few weeks ago and there shouldn't be any relevant changes for your module anymore.
it would be great if you could begin on a solution for G2.1 before G2.1 is released, you'll make a lot of users happy. but it's up to you of course
sandymac
Joined: 2005-09-15
Posts: 31
Posted: Mon, 2006-01-23 19:23
valiant wrote:
it would be great if you could begin on a solution for G2.1 before G2.1 is released, you'll make a lot of users happy. but it's up to you of course
Well, I encourage someone else to send me a patch or take over the module.
But we're not talking about a large userbase. Since last November there have been less than 60 downloads, about 15 of those being users that have downloaded more than one version.
fryfrog
Joined: 2002-10-30
Posts: 3236
Posted: Tue, 2006-01-24 16:31
sandymac, I'm sorry if you got the impression that I was implying that your module doesn't work. I know its on my end, but I am having trouble figuring out *why*. In debug mode, I am seeing portions of your module load up. Since the check blocks are in, I am seeing them be initialized just fine. I've also installed/activated/etc the module in debug mode, so I am seeing it get that right too.
What I am having trouble with is debuging. I am quite unfamiliar with both G2's code and your own module, so what I was hoping for was some pointers on figuring out where it isn't working.
I suppose it *could* be the WP integration, but afaik the "wp-gallery2.php" file is *nearly* the same as the main.php in the /gallery2/ dir. The only difference would be the theme, which of course is quite different. I don't *think* your module is very theme dependant, but I could be wrong? I'd really like to get this working because image downloads are actually starting to affect my WoW game play!
As for the number of users... I suspect when the "downloadable modules" portion of G2 goes live, you will end up with a *lot* more customers. For anyone hosting their own images on their own connection, your coralize module is neigh perfect! Just gotta get the word out ;)
I know the devs are pretty friendly to module makers, so if you were to visit them on irc.freenode.net in #gallery, I am fairly confidant you would get any help you needed concerning the API changes. If you are even able to get it up to G2's "strict" code standards, there is a chance it could be included in the official next release, though being quite low in the project I'm not one to make that kind of call. My personal opinion is that it'd be good! Also, once its in the official release, some poor G2 sap will end up making sure it works ;)
Hopefully now that I am working nights, I will have more time to spend working with my (and others) gallerys. Maybe I'll even be the one to work on your module for G2.1!
Edit: Oh yeah, of course I still have gallery located at /gallery2/ on my site, so during testing/debugging I am visiting that directly, bypassing WP (except of course the WP theme). I can switch to another theme for testing, which I will try later... but I doubt its the theme. I'm running a fairly old CVS version, I stopped updating a few weeks before the big API changes so my G2 would continue to work w/o a ton of work on my part ;)
_________________________________ Support & Documentation || Donate to Gallery || My Website
sandymac
Joined: 2005-09-15
Posts: 31
Posted: Tue, 2006-01-24 17:12
@fryfrog: k, first, limit the number of variables: turn off the coral check feature. This should make it always coralize so long as you are connecting to the gallery from a public IP address (aka, not your machine and on the internet) Also edit .../modules/coralize/classes/CoralizeUrl.class and add "return true;" to the first line of the requestFromInternet function.
If it does work as expected the I guess wordpress is doing a poor job of telling Coralize what the remote IP address is.
Let me know what happens.
fryfrog
Joined: 2002-10-30
Posts: 3236
Posted: Tue, 2006-01-24 17:52
function requestFromInternet($ip=null) {
return true;
if ($ip == null) {
$ip=$_SERVER['REMOTE_ADDR'];
}
Okay, I made the change. I assume this *should* cause it to coralize anything no matter what? When I first installed the module, I couldn't test due to being behind my own firewall and I knew your module doesn't coralize for RFC addresses. I was finally able to test from work, but that didn't coralize either. I also tried visiting the coralized url manually for both the images and the pages and that does work. The "check if it works" setting has been off for testing since I first installed.
Okay, I made the change. I assume this *should* cause it to coralize anything no matter what?
It changes the behavior so that the coralize module it always thinks requests are coming from the internet.
Try this breifly: edit the same file and add the following line to teh start of the coralize function:
echo "<b>in coralize($url)</b>";
Then reload the page. If that method is called then you'll see that text randomly in the page at weird places. If you don't see it on reloaded pages the the Coralize module isn't being called. No link gets coralized without going through that method.
fryfrog
Joined: 2002-10-30
Posts: 3236
Posted: Wed, 2006-01-25 01:18
Okay, upon adding it I find it only *once* per page. I threw it in as "?> <!-- in coralize($url) --> <?php" so you can see it on a page by view source but it won't make my site look tooo crazy.
It comes out as "in coralize($url)" with nothing actually in $url! So I'm guessing that is part of the issue, eh? I'll dig deeper, thanks for the pointers.
Edit: Blech, had to take it out cause it was causing headers to be sent when they had already been sent. But no matter, I can see there is an issue.
_________________________________ Support & Documentation || Donate to Gallery || My Website
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Wed, 2006-01-25 02:31
maybe wp is parsing the g2 html output and adding / removing newlines.
i can remember of a related issue, can't remember whether it was wp, joomla or drupal. i guess it was wp.
html formatting made the embedding application add a <br> or newlone or something like that.
already tested the whole thing in standalone?
fryfrog
Joined: 2002-10-30
Posts: 3236
Posted: Wed, 2006-01-25 19:54
Yar, 98% of my testing is via visiting the un-embeded URL, you can see it 2 or 3 posts back. It includes a link to an album that is even setup using another theme.
_________________________________ Support & Documentation || Donate to Gallery || My Website
doomdead
Joined: 2003-04-06
Posts: 174
Posted: Mon, 2006-02-13 21:03
Currently, if coralize is active, and you upgrade to G2.1 RC1, the upgrade will complete, but gallery will fail to load. It took me nearly 2 hours to fix/repair my gallery after I spotted the problem. After upgrading, coralize was disabled (due to the code not being up to par with current changes), however the getCoralizeURL (or whatever) class is still being called. I had to go into the gallery table that has that map in it (I can't remember and im at work), and remove that record. Then gallery would load fine.
Thats a big problem. Anyone with coralize loaded, will have the same problem during the upgrade.
fryfrog
Joined: 2002-10-30
Posts: 3236
Posted: Fri, 2006-03-03 08:56
Hey sandymac, G2 2.1 is at RC1, progressing quickly to RC2 then final! You know you want the ratings module, so you may as well update coralize to work with 2.1!
_________________________________ Support & Documentation || Donate to Gallery || My Website
mindless
Joined: 2004-01-04
Posts: 8601
Posted: Tue, 2006-05-30 17:04
Attached is an (untested) version of the coralize module migrated to G2.1
mindless
Joined: 2004-01-04
Posts: 8601
Posted: Mon, 2006-06-05 16:36
valiant pointed out there is some discussion above regarding rewrite module changes in G2.1.. i didn't make any fixes for this in the attachment above, just a simple conversion for other 2.1 updates.
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Mon, 2006-06-05 16:55
thanks mindless!
sandymac, I really hope you consider one of my suggestions to make it work with g2.1's rewrite module! it's one of the coolest module's because it achieves a lot in an elegant way. i hope it will work with g2.1.
sandymac
Joined: 2005-09-15
Posts: 31
Posted: Mon, 2006-06-05 17:14
I'm not against updating it. I just am busy. At the next point in time where I have many pictures to share to a bunch of friends I'll probably make time to update it as I've personally upgraded my sites to the latests gallery already. Until then I'd be very happy if someone else has the time to carry the code forward.
mariposaHD
Joined: 2006-07-01
Posts: 1
Posted: Sat, 2006-07-01 10:07
Thanks a lot for this great module. We just installed it and started testing it on our G2 installation: http://www.mariposaHD.tv/gallery
It seems to be working perfectly so far.
We have nearly 2000 high-res images in our gallery (some of which seem to be very popular) and we've run the risk of getting our server swamped when we've been linked to by other popular web sites. Hopefully this might help.
Thanks again for putting together this module.
IanWorthington
Joined: 2007-01-12
Posts: 18
Posted: Thu, 2007-01-25 04:42
Anyone know if this was ever updated for 2.1.2?
Trying to install Sandy's code gets me:
Incompatible module!
Core API Required: 6.0 (available: 7.3)
Module API Required: 2.0 (available: 3.0)
ian
...
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2007-01-25 09:13
there is a g2.1 compatible version, but it won't work with the url rewrite module.
Posts: 32509
i think other mirror methods should be implemented in other modules. keep it simple / maintainable.
Posts: 32509
sandymac
the API changes / rewrite module changes are now in the official CVS / nightly snapshots.
it would be great if you could update your module to work with it.
see:
http://codex.gallery2.org/index.php/Gallery2:API_Changes_Since_The_Last_Release#6.10_.3D.3E_7.0
http://codex.gallery2.org/index.php/Gallery2:API_Changes_Since_The_Last_Release#2.4_.3D.3E_3.0
http://gallery.menalto.com/node/36446#comment-157101
Posts: 389
and see:
http://codex.gallery2.org/index.php/Gallery2:URL_Rewrite
Posts: 3236
So I have finally gotten around to installing your module, only to find one minor issue... if I add your coral check to the side bar it makes the side bar exist. What I mean is, I don't have anything else in the side bar so I have none (on purpose). When I add yours, it suddenly exists (reasonable), but with nothing in it (sucky). I spose the solution is to turn the check off?
_________________________________
Support & Documentation || Donate to Gallery || My Website
Posts: 32509
fryfrog:
is that check in a block? can't you add it to album and photo block lists instead of the sidebar block list?
Posts: 3236
I did add to both, but wasn't able to remotly test yet
_________________________________
Support & Documentation || Donate to Gallery || My Website
Posts: 3236
So I've installed and activated it, unchecked the "check if its working before using" box... but it still doesn't yield any coralized image urls. I've also tried loading my site manually coralized, and it works... investigating!
_________________________________
Support & Documentation || Donate to Gallery || My Website
Posts: 6
I have this module installed, and it seems to work well.
Should it include a Vary: header in the request, so proxies don't serve the wrong page if the ability of clients to get to Coral changes? I see "coralCheck=passed" appended to the end of the Cookie: header. Is this field relevant to proxies for deciding to serve a cached page or retrieve from the server?
Here's one scenario: Client A is configured to use a proxy and visits a Gallery page, and is able to access Coral cache. The proxy caches the page and the images. After some time, the proxy purges the images from it's cache, but the pages remain. Client B uses the proxy to visit the same Gallery page, but Coral is unavailable/unreachable. Client B gets the cached pages that contain links to the Coralized objects. Incorrect behavior results.
If we make a new header, X-CoralCheck, and include "Vary: X-CoralCheck", we could avoid this issue. See http://www.ietf.org/rfc/rfc2616.txt section 14.44
Posts: 31
Nope, the cookie is used by the Coralize module only if you have the coral check enabled in the module. What it does is request some JavaScript via the Coral CDN and that JavaScript sets the coralCheck cookie. If the user cannot connect to Coral CDN, the cookie doesn't get set and the Coralize module doesn't ever coralize images.
Because the URLs between a normal image request and a coralized image request are different adding a Vary header doesn't do anything useful.
Posts: 6
Right... that's why we don't want a page cached by the proxy with coralized image urls being served to a client that isn't able to reach the Coral network. I don't see anything in the current implementation that would prevent this.
Posts: 31
Section 13.4 of the HTTP/1.1 spec states that:
I did some packet sniffing and verified my gallery install returns neither a cache validator header nor an explicit expiration header for the xhtml pages. If you find a cache that is violating the spec I encourage you inform the product's creator of their non-compliance.
Posts: 3236
I still can't figure out why the module installs and can be configured, but totally fails to coralize any images. I've put my G2 into buffered and immediate debug mode and the only references I can find to coralize are at the points when it loads up the "check". I am thinking that it must be due to the CVS version of G2 that I am using... any plans for moving to current nightly/cvs?
_________________________________
Support & Documentation || Donate to Gallery || My Website
Posts: 31
@fryfrog: It could be from having your gallery mixed in with wordpress. Maybe wordpress doesn't use a standard URL Generator class and thus Coralize cannot extend it.
No I'm not using CVS. I only use release versions. I have better things to do than track every Gallery change.
Here is my development gallery: http://dargo.us/gallery2/main.php and it's running the latests Coralize module. You could check to see if images are coralized for you.
Posts: 32509
@sandymac:
we're releasing G2.1 on March 1st. and G2.1 is to the most part what we have currently in CVS. the big url generator changes were committed a few weeks ago and there shouldn't be any relevant changes for your module anymore.
it would be great if you could begin on a solution for G2.1 before G2.1 is released, you'll make a lot of users happy. but it's up to you of course
Posts: 31
Well, I encourage someone else to send me a patch or take over the module.
But we're not talking about a large userbase. Since last November there have been less than 60 downloads, about 15 of those being users that have downloaded more than one version.
Posts: 3236
sandymac, I'm sorry if you got the impression that I was implying that your module doesn't work. I know its on my end, but I am having trouble figuring out *why*. In debug mode, I am seeing portions of your module load up. Since the check blocks are in, I am seeing them be initialized just fine. I've also installed/activated/etc the module in debug mode, so I am seeing it get that right too.
What I am having trouble with is debuging. I am quite unfamiliar with both G2's code and your own module, so what I was hoping for was some pointers on figuring out where it isn't working.
I suppose it *could* be the WP integration, but afaik the "wp-gallery2.php" file is *nearly* the same as the main.php in the /gallery2/ dir. The only difference would be the theme, which of course is quite different. I don't *think* your module is very theme dependant, but I could be wrong? I'd really like to get this working because image downloads are actually starting to affect my WoW game play!
As for the number of users... I suspect when the "downloadable modules" portion of G2 goes live, you will end up with a *lot* more customers. For anyone hosting their own images on their own connection, your coralize module is neigh perfect! Just gotta get the word out ;)
I know the devs are pretty friendly to module makers, so if you were to visit them on irc.freenode.net in #gallery, I am fairly confidant you would get any help you needed concerning the API changes. If you are even able to get it up to G2's "strict" code standards, there is a chance it could be included in the official next release, though being quite low in the project I'm not one to make that kind of call. My personal opinion is that it'd be good! Also, once its in the official release, some poor G2 sap will end up making sure it works ;)
Hopefully now that I am working nights, I will have more time to spend working with my (and others) gallerys. Maybe I'll even be the one to work on your module for G2.1!
Edit: Oh yeah, of course I still have gallery located at /gallery2/ on my site, so during testing/debugging I am visiting that directly, bypassing WP (except of course the WP theme). I can switch to another theme for testing, which I will try later... but I doubt its the theme. I'm running a fairly old CVS version, I stopped updating a few weeks before the big API changes so my G2 would continue to work w/o a ton of work on my part ;)
_________________________________
Support & Documentation || Donate to Gallery || My Website
Posts: 31
@fryfrog: k, first, limit the number of variables: turn off the coral check feature. This should make it always coralize so long as you are connecting to the gallery from a public IP address (aka, not your machine and on the internet) Also edit .../modules/coralize/classes/CoralizeUrl.class and add "return true;" to the first line of the requestFromInternet function.
If it does work as expected the I guess wordpress is doing a poor job of telling Coralize what the remote IP address is.
Let me know what happens.
Posts: 3236
function requestFromInternet($ip=null) { return true; if ($ip == null) { $ip=$_SERVER['REMOTE_ADDR']; }Okay, I made the change. I assume this *should* cause it to coralize anything no matter what? When I first installed the module, I couldn't test due to being behind my own firewall and I knew your module doesn't coralize for RFC addresses. I was finally able to test from work, but that didn't coralize either. I also tried visiting the coralized url manually for both the images and the pages and that does work. The "check if it works" setting has been off for testing since I first installed.
http://fryfrog.com/gallery2/ bypasses anything wordpress (cept the theme), so I doubt its wordpress specifically. For giggles and poop, I have set http://fryfrog.com/gallery2/v/MaryAnn-Donnie/Certs/ to the Matrix theme. It isn't coralizing from an *internal* browser, and I am pretty sure that it *should* be doing so with that "return true;" I added, correct?
_________________________________
Support & Documentation || Donate to Gallery || My Website
Posts: 31
It changes the behavior so that the coralize module it always thinks requests are coming from the internet.
Try this breifly: edit the same file and add the following line to teh start of the coralize function:
echo "<b>in coralize($url)</b>";
Then reload the page. If that method is called then you'll see that text randomly in the page at weird places. If you don't see it on reloaded pages the the Coralize module isn't being called. No link gets coralized without going through that method.
Posts: 3236
Okay, upon adding it I find it only *once* per page. I threw it in as "?> <!-- in coralize($url) --> <?php" so you can see it on a page by view source but it won't make my site look tooo crazy.
It comes out as "in coralize($url)" with nothing actually in $url! So I'm guessing that is part of the issue, eh? I'll dig deeper, thanks for the pointers.
Edit: Blech, had to take it out cause it was causing headers to be sent when they had already been sent. But no matter, I can see there is an issue.
_________________________________
Support & Documentation || Donate to Gallery || My Website
Posts: 32509
maybe wp is parsing the g2 html output and adding / removing newlines.
i can remember of a related issue, can't remember whether it was wp, joomla or drupal. i guess it was wp.
html formatting made the embedding application add a <br> or newlone or something like that.
already tested the whole thing in standalone?
Posts: 3236
Yar, 98% of my testing is via visiting the un-embeded URL, you can see it 2 or 3 posts back. It includes a link to an album that is even setup using another theme.
_________________________________
Support & Documentation || Donate to Gallery || My Website
Posts: 174
Currently, if coralize is active, and you upgrade to G2.1 RC1, the upgrade will complete, but gallery will fail to load. It took me nearly 2 hours to fix/repair my gallery after I spotted the problem. After upgrading, coralize was disabled (due to the code not being up to par with current changes), however the getCoralizeURL (or whatever) class is still being called. I had to go into the gallery table that has that map in it (I can't remember and im at work), and remove that record. Then gallery would load fine.
Thats a big problem. Anyone with coralize loaded, will have the same problem during the upgrade.
Posts: 3236
Hey sandymac, G2 2.1 is at RC1, progressing quickly to RC2 then final! You know you want the ratings module, so you may as well update coralize to work with 2.1!
_________________________________
Support & Documentation || Donate to Gallery || My Website
Posts: 8601
Attached is an (untested) version of the coralize module migrated to G2.1
Posts: 8601
valiant pointed out there is some discussion above regarding rewrite module changes in G2.1.. i didn't make any fixes for this in the attachment above, just a simple conversion for other 2.1 updates.
Posts: 32509
thanks mindless!
sandymac, I really hope you consider one of my suggestions to make it work with g2.1's rewrite module! it's one of the coolest module's because it achieves a lot in an elegant way. i hope it will work with g2.1.
Posts: 31
I'm not against updating it. I just am busy. At the next point in time where I have many pictures to share to a bunch of friends I'll probably make time to update it as I've personally upgraded my sites to the latests gallery already. Until then I'd be very happy if someone else has the time to carry the code forward.
Posts: 1
Thanks a lot for this great module. We just installed it and started testing it on our G2 installation: http://www.mariposaHD.tv/gallery
It seems to be working perfectly so far.
We have nearly 2000 high-res images in our gallery (some of which seem to be very popular) and we've run the risk of getting our server swamped when we've been linked to by other popular web sites. Hopefully this might help.
Thanks again for putting together this module.
Posts: 18
Anyone know if this was ever updated for 2.1.2?
Trying to install Sandy's code gets me:
Incompatible module!
Core API Required: 6.0 (available: 7.3)
Module API Required: 2.0 (available: 3.0)
ian
...
Posts: 32509
there is a g2.1 compatible version, but it won't work with the url rewrite module.
--------------
Enter the Gallery 2 Theme Contest today!
Posts: 18
Thanks valiant.