New Client: Gallery 3 Publish Service for Lightroom 3

Outstanding.. great work!
---
Problems? Check gallery3/var/logs
bugs/feature req's | upgrade to the latest code | use git

Hey Pi,

Really nice of you to put this together. Looking forward to giving it a whirl.

When I load the plugin, I get the following error:

Plug-in error log for plug-in at: /Volumes/data/d2/software/mac/plugins/lightroom3/G3_publish_service_20100905/G3PublishService.lrplugin

**** Error 1

An error occurred while attempting to load this plug-in.
The plug-in description script (Info.lua) is missing.

The status in Plug-in Manager is listed as 'Installed but not working'.

Any thoughts?

OS: MacOS 10.6.4
LR: 3.2

-Avery

Well, after I uploaded the new plugin (removed prior version and rebooted LR first), I received the status "Installed but not working". I rebooted LR, and the plugin was then listed as disabled. After enabling, it works.

Anyhow, I was able to upload to my dev site, just fine (only basic testing thus far). That's just awesome. Great work, and thanks! I'd buy you a beer if you were local. Now, I just need a G3-WordPress3 SSO solution that works!

One question about auth (since I really don't know how G3 3rd party auth works)... if you indicate an httpS url (I don't have SSL configured on my dev env yet), does it pass the credentials over SSL? Also, with the auth key, does it pass the credentials again, or does it exclusively rely on a paired key to authenticate my workstation for future requests (and only pass the password when retrieving the initial key)?

Thanks!
Avery

Cool on the update.

I initially asked the question, because I was lazy and didn't want to order a cert, setup ssl to test and do a packet capture to test, if someone already knew this answer. Well, here are the results of the test:

- LR 3.2 *does* support http traffic over SSL (https)
- Without SSL, the auth key and initial passing of the password is easily seen with a sniffer (as expected)

I think it probably makes sense to highlight somewhere in the documentation that there is a vulnerability if you are not using https (Gallery should point this out to folks, as well). Then again, sites like facebook do insecure submits all day long... that's just poor.

################################
Clicking 'Request Key' > 'OK'
(insecure http url)

X-Gallery-Request-Method: post
Content-Type: application/x-www-form-urlencoded
Content-Length: 28
Accept: */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: keep-alive
user=<MY USER NAME CLEAR-TEXT>&password=<MY PASSWORD CLEAR-TEXT>

22
"<NEW KEY CLEAR-TEXT>"
0
################################
Clicking Publish
(insecure http URL)

X-Gallery-Request-Method: post
X-Gallery-Request-Key: <MY KEY CLEAR-TEXT HERE>
Content-Type: multipart/form-data; boundary=_846843213219879654635132_
Accept: */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Content-Length: 687970
Connection: keep-alive
################################
Clicking 'Request Key' > 'OK'
(secure httpS url)

+.KGSa.......gp.Y........}.|-.y....~.U.t....O..-...hEm.....Xw"Kg...R@....?..toa.1!....@.]L..S6..E... ?,..K....hh..a>.H#.N..G.5e..... N`x..)...4f.%..Y/.
Wl.......V>......
\.....g....g9!.G..s...%..I*........*...q.....{|..5P./.........i.1./...:..H...Z.....
u..!m.A.|.(.`..s
.O..3ux..y..i.D.)].j.s....+..~.\.G......
...R....`.. m..f.a..k......m.PT..O.._.R....%.._...*. 6..0..P.f.].E.....G...-;..(...]...s..[..a..:lM....'.....
@...K.(..)../I...t.*..@.!i....r...O$4p.<..._.Y......
################################
Clicking Publish
(secure httpS URL)

..*.H..
..........K..L%.......K...#B.U,.....!T!._.Od7..!.2...SJ%.-..l..?.E+.....}.\8..W.G.....$.....8F.m..p|.#D.u.~.*k..^..U.e41)........
P..'.7n7Q.V'... \Mh..K...M........G.M..qk.......D.....N..C...M.C....3....E.......q<B&....5Su....<.T...."D....F.H9*..
j.. ??Z...#z.....;x.&D........d...............4..)\.......H#]..~...u....3."H...C......n.u.......y{.Sk..2.'.!...5.B.Y.D.....S
%.*qf..3.|.....K..jf`..T.%S.]..."..<T...&R..F8..V.......d....0...."..do.qV.W.C..M.m!.....fgy...i."...L..T..y..y..........0
.ApA..j.U.4........9....v.?t....L..O...."....dY`......._ -<........ot...G..........?}..0r....d..2.-.'.u...s4....T... 8^Y.....4b..bv
################################

Fairly big update (0.3):
* Merged with Gloopers Lightroom 3 Export code. This can now do both publish service-ing and exporting.
* Lots of bugs fixed
* Lots of new bugs introduced, since I've restructured and rewritten a lot of the code - I ran my testing regimen and couldn't find any, but there must be lots of them in there. When you put your ear really close to the CPU, you can actually hear them crawl around (AMD only)
* New (hopefully better) GUIs
* Collection names are now being validated
* A few error messages make more sense now (or appear at all)
* Export should be preset-friendly (seems to work for me, at least)
* More than one publish service to different gallery installations and using different user accounts are (should be) now supported too
* "Go to published photo" etc. won't work with RC2, only with recent code. This is a known issue.
* Lots of other stuff that I can't remember right now.

Besides, if anyone in the know is reading this, are there any official repositories or web sites or any kind of central registry for Gallery 3 clients? I don't think too many people will find this forum thread, so once this is stable enough for everyday use something more exposed would be good.

Just tried the integrated export functionality. Works well. I like the interface, it's a bit more streamline than the original. Not so keen on the pop-up message boxes that appear after things happen, but once logged in you don't see them again anyway so I'm fine with it. Excellent stuff. .

The only bug I've noticed is that it doesn't remember the last album you've chosen in the drop down list, which it did before, was useful when you keep uploading to the same album.

Cheers
Ben

Benjamin Albert Smith - Photography
Pikitia - New Zealand Photography

This is awesome!
But, I get an error when publishing pictures. It reads:

[Window Title]
Warning

[Main Instruction]
Can’t update this collection.

[Content]
An internal error has occurred: [string "G3PublishServiceProvider.lua"]:207: attempt to index global 'collectionInfo' (a nil value)

Keep up the good work!

Åsmund

I ran into the attempt to index global 'collectionInfo' problem as well (Lightroom 3.2, OS X 10.6.4, Gallery git pull 3936183446375e039cc3).

It turnes out that
local collectionInfo = exportContext.publishedCollectionInfo
needed to be pulled out above the
if exportSettings.LR_isExportForPublish then
line.

Once I did that it seams to be working well.

Thanks very much for publishing this. I have been looking for a good way to manage my high res images locally and lower res ones on gallery.

chris.

Hi,

I always get "result is nil" and "failed to retrieve auth key" when I want to add an account.

What can be wrong?

[IMG]http://cindyenguy.be/ico_www.gif[/img] My Blog [IMG]http://cindyenguy.be/ico_www.gif[/img] My Gallery

3.14159: It's getting there
I now get another error

[Window Title]
Warning

[Main Instruction]
Can’t update this collection.

[Content]
Error uploading image. (Code: 400)

with the previous error it did upload the picture, now it does not.

GuyVerschuere: The http:// is necessary, you can't omit that. So .. are you really sure your credentials are valid? i've checked in the code, the only way this error message can possibly show up is when gallery returns a "403 Forbidden" header to your authentication request. That's also the only way I can reproduce this. So either something is seriously broken or it's just that .. before i start digging deeper into the code please check again if your user name and passwort really are correct.

aalang: Code 400, that's odd. What gallery version are you on, and what exactly were you trying to do? I'll send you a private message with another thing you could try.

Guy: Thanks! I'm taking a look right now, I can reproduce the issue.
OK, it seems your url rewriting is to blame; I'm currently guessing the rest base url, and i'm using gallery_url/index.php/rest/. For you, it's just /rest/ (which returns a 404 file not found on other gallery installations). When I use just the / When I open the URL in a browser I'm being redirected just fine, but that doesn't seem to work for POST requests. You could try using the default .htaccess file, if that solves your problem we know for sure. If the .htaccess is the culprit, then I guess you could fix that if you excluded all /index.php/rest/... urls from being rewritten. These aren't supposed to be opened in a browser anyway, so your users wouldnt notice that. I'll see that I include an option to specify the relative REST path in the next version, but that will take some time. For now that htaccess tweak should be your best bet, more so since other clients will probably have problems with that, too.

So long
Three

It isn't easy

this doesn't work:

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^rest/(.*)$ rest/$1 [L]
RewriteRule ^(.*)$ index.php?kohana_uri=$1 [QSA,PT,L]
RewriteRule ^$ index.php?kohana_uri=$1 [QSA,PT,L]
RewriteRule ^index.php/(.*) $1 [QSA,R,L]
</IfModule>

and this also doesn't work:

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_URI} ^/rest/.*$
RewriteRule ^.*$ - [L]
RewriteCond %{REQUEST_URI} ^/REST/.*$
RewriteRule ^.*$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.php?kohana_uri=$1 [QSA,PT,L]
RewriteRule ^$ index.php?kohana_uri=$1 [QSA,PT,L]
RewriteRule ^index.php/(.*) $1 [QSA,R,L]
</IfModule>

Anyone who knows how to set.htaccess to exclude the rest module?

[IMG]http://cindyenguy.be/ico_www.gif[/img] My Blog [IMG]http://cindyenguy.be/ico_www.gif[/img] My Gallery

THANK YOU!

it works!

[IMG]http://cindyenguy.be/ico_www.gif[/img] My Blog [IMG]http://cindyenguy.be/ico_www.gif[/img] My Gallery

3.14159: Thanks! I'll set up an account for you. Sending you an email with the details.

aalang was able to solve the mystery; if anyone else experiences code 400 errors, deleting and republishing the responsible images should help.

pyjamasam: thanks for the praise! glad to hear this works for you, and for your pretty specialized requirements, too. i agree the behaviour on missing albums (and when updating missing images, too!) can't stay that way. I have an idea how to fix that, but please share what you have, yours might be better.
as to the rest of your modifications, they concern the gallery software, don't they? in that case you could package them into modules if you havent already done so, especially the "sort by filename" might be useful to a lot of people.
concerning your wish list, i'd like to have nested collections, too, but lightroom simply doesn't support that. you can nest collection sets, but these can't hold any photos; you could have a "Photos in this album" collection inside each collection set, but some users would inevitably delete them and there's currently no way to prevent that, as far as I know. Also, it would probably be a lot more visual clutter than the current solution, crude as it may be. I do hope Adobe includes nested collections in a future release, but for now I think the way I'm doing it is the best of the available options. I'm always open to suggestions, though.
As to uploading video, I'm currently not supporting this because I have absolutely no idea how this video stuff works, both in gallery and LR. I believe enabling it in LR isn't hard, one line of config and another two or so lines to make sure there's "type:video" instead of "type:photo" in the upload requests. However, if I understand correctly, then somewhere somehow there must be a transcoding step to get something that flash-based video players or browsers can play. Where does that happen? It had better not be on gallery's end, since that would surely not fit into any sensible php execution time limit, and would probably not be tolerated by many shared hosters either, since that kind of thing eats up loads of resources. If it has to be done on LR's end, though, and if it isn't already being done by LR (which I doubt), then I'd have to hand it off to an external application to do this and cope with a whole host of ways for things to go awry, on two different platforms (win+mac) no less. What is more, since I don't even have any videos in my LR catalogue (and my SLR doesn't do video) I could hardly help anyone for whom it doesn't work. And from a user experience perspective, if I were to do the transcoding on the client side, having to wait half an hour per file with 100% system load might not appeal to people used to fast connections and the youtube/flickr experience. I'd like to include that functionality, but for now I suppose all I could do would be to simply push the unprocessed video file over the wire. I think i'll stick to both lr's and gallery's core competence for now, photos.

so long
three

Hello (just registered to post this comment to improve an already fantastic plugin! )

I seem to be experiencing Code 400 errors as well. What I found out from looking around a bit is as follows:

I'm using the modified plugin with the "Path to REST base" option field. Works basically OK (i.e. Export to Gallery3 is perfect), but... When trying to create an album as a target for Publishing, the following happens.

1) The Create album asks for the album name and accepts whatever the user gives OK.
2) The album accepts pictures for import
3) When clicking Publish, the plugin notices that the desired album is not (yet) present in Gallery3 and asks for a title.
4) The update then fails with a 400 error.

After looking at the error log in Gallery, the following clue is given:

2010-09-13 09:28:21 -07:00 --- error: Rest_Exception [ 400 ]: Bad Request /var/www/gallery/modules/gallery/helpers/item_rest.php [ 184 ]
#0 [internal function]: item_rest_Core::post(Object(stdClass))

Looking at item_rest.php, line 184, indicates that a Rest_Exception is thrown whenever the $entity->type is not any of album, photo or movie. What gives? It should be creating an album, shouldn't it?

Now here is where I think the "REST Base URL" option comes into play and isn't properly handled. I ran a Wireshark trace on the Publish communication, and the HTTP POST request is "POST /index.php/rest/item/1". This isn't really a problem from Gallery's (or Apache's, actually) point of view, and it gives a 302 Found redirect to the proper path (/rest/item/1). However, for some reason unknown to me, the LR Gallery plugin then tries a HTTP GET request on the redirect URL. This, of course, strips the POST contents from the request, leading to the exception being thrown by item_rest.php.

Could somebody please fix this? I'm a bit afraid to touch the plugin myself..

Best regards,
Sam

sounds good.
as to the wish list, these things happen to be quite high on mine, too, especially the nested collections issue. but there's nothing that can be done about that, sadly.

samwise_s: thanks for helping! these 400s are quite annoying, even more so since I fail to reproduce them. What is the exact error message you're getting?
Also, there's another source of information you could use; I'm using LR's LrLogger facility to debug these early builds; you should have a file called g3.log in your My Documents (win) or ~/Documents (mac) folder. I'm keeping a more detailed log of the steps the plugin takes there, nothing sensitive (no credentials or the like). Please repeat what you did to get the error and send me the last couple of lines of this debug file; maybe that information helps. The get request in your wireshark trace could be alright; i'm sending one immediately after posting the new album to fetch that album's id (the post request returns only a rest url.) This does not strip anything off the post request, since the requests are processed sequentially on the server; once the reply to the post request is out the next one (the get) can be processed. To me this indicates that creating the album probably works as intended; the 400 error might occur someplace inside the image upload part. I suspect that something breaks on updating from earlier builds to more current ones, I just don't have any idea what that might be. Any chance I could get my hands on an actual wireshark iptrace file?
As to the propertyTables, each publish service and each export has its own independent propertyTable; I'm saving the PATH_TO_REST in the global plugin preferences, though, and the property tables only store which of the accounts the current task uses. And in any way, unless you change the PATH_TO_REST setting, everything works just the way it did before (at least it should.)
I'll keep your hypothesis in mind at any rate, I'd be glad if you were right since then I had a starting point for a fix at last.

so long
Three

Wait, did you change the rewrite settings in the .htaccess? I can't reproduce any 302 responses on none of my test setups (gallery3 latest code running on apache on win7 and apache on debian linux) and /rest/ without index.php doesn't even work in the browser either (404 from the web server, with /index.php/ it works.) if that's the case, then you'd enter /rest/ or /my_new_rest_path/ in the "path to REST base" field in the add account dialog, and then the plugin wouldn't send anything to /index.php/rest/ at all, since that would be expected to cause errors. If you didn't see that text field when you added your account, maybe you created it with a previous version of the plugin that didn't have this option yet. Just add the account again (the old one will be replaced) and enter /rest/ in the "path to rest" field; that should do the same as your workaround. if you did/do enter "/rest/" instead of the default "/index.php/rest/" and the default value shows up in any urls, that would be a good starting point for a fix.

Thanks!

so long
Three

You're welcome!

Change the resolution? You're talking about the dpi, aren't you? It's trivial to add indeed, but I disabled that on purpose; is there any sense in having that kind of setting when you export to a web gallery? I can't quite think of a use case for that, what's yours?

Interesting suggestion; I'll put that onto my todo list. I'll have to do all of this organizating stuff sometime soon anyway, for the moment I try to spend as much of the time I can devote to this project progamming and testing, that's why it's all a bit chaotic.

To everybody: If you find that deleting collections doesn't work, it's totally broken. I've fixed it and it'll work again in the next version, but for now it's a know issue.

Glad to hear you find it useful!

No, in your case that's definitely not a solution. I thought this was happening only when you had photos from a really early version of this plugin and updated to the latest one. But in your case that won't do, what's needed is a proper fix. You'll have to help me somewhat, since I couldn't yet reproduce this issue, though.
First, can you pinpoint anything you did that could have made the photos behave like that? did you update the plugin, perhaps? did you cancel any uploads? did you make any changes in the gallery?
Second, there's a log file in your My Documents (win) or ~/Documents (mac) folder called g3.log. I'd like to take a look at it, if possible. I'll send you my email address via PM.
Third, any chances you could make a wireshark trace of an upload that throws an error and send me the wireshark iptrace file? If you happen to be familiar with wireshark, that is. I'll be glad to help if there are any questions.

I do hope we'll find one!

So long,
Felix

September 16, that's as up-to-date as can be. I assume you did run the update script after upgrading your Gallery installation (/index.php/upgrader), so you should be up-to-date there, too. That's good because it rules out a lot of possible causes for this error, should make locating the issue easier. Thanks for helping! i'll wait for your email.

so long
Three

Hmm. I've experimented a bit and I have a working prototype for a possible new feature. The functionality would be this: On adding a remote album to the publish service, it downloads all existing images and adds them to the catalogue; they're then being treated like any other photo you add to the published collection, you can edit them, republish them, use the comment integration (that will be in the next version), anything. I also have the parts to make a working synchronization, that is, on publishing the plugin could update the local copies of those downloaded if their counterparts in the gallery get updated; I didn't actually write this yet, but im 99% convinced it's can be done. Just loads of work to get all the critical details right.
I'm not sure if it makes sense to pursue this further, or if this is the kind of pointless feature that no one really needs anyway. Opinions?

Great work!
I'm an old G2 user and having upgraded to the G3 most of my pictures sits both in G3 and in Lightroom. I like the idea of the new feature but i would like it to first attempt to locate the image in the Lightroom catalog. If found, then don't download. Just add to the collection. Else download.

The sync function sounds great. G3 could be used as an online backup.

Keep up the good work!

Pi -- Just wanted to voice that I REALLY like the sounds of that... especially since I'm planning on using GPS tagging and extensive meta tagging. And, yeah -- it would be really cool to maintain the updated title/description with a bidirectional sync!!!

Anyhow, thanks for all your work.