Lightroom <-> Gallery

Yo.

I have installed Gallery. It looks like it's the only PHP web gallery that works ;) and I like it.

So, here is the deal. I have lots of RAW photos in Lightroom, organized in folders, tagged with keywords, and rated. So, I export all this as JPEG. Lightroom, being stupid, puts everything in a flat folder. So, I write a 5 minute Python script to rebuild my folder organization.

Then, I use a Photoshop script to resize, sharpen and Save For Web (TM) all this. I generate 1680x1050 for display on my LCD, and 1024x768 for web use, both have different sharpen settings. Unfortunately I can't do this directly in Lightroom since it only supports one sharpen setting.

I re-run the little script to rebuild the folders, and hooo ! everything imports cleanly in Gallery2, I get one album per folder, it's really neat.

But, the tags and keywords and EXIF crap got lost on the way (in Save For Web) and the ratings were not exported by Lightroom anyway. Also, all the dates are from 2 hours ago instead of being the real photo dates.

So, I hit the Lightroom SQLite database with a few queries (so cool of them to use an opensource database) and I can get my data back without a problem : filename (which is unique, ie. usable as a primary key), tags, ratings, dates, titles, captions, etc... But, how do I tell this to Gallery ?

I would really like if someone pointed me to the queries to perform on the Gallery database (or the API if it supports that, it doesn't seem to) in order to :

- get the ID of the image in Gallery's database from the filename.
- insert the tags, ratings, dates, titles into gallery's database.

Thanks ;)

OK, so I have developed the following lightroom / gallery2 workflow :

- import stuff into lightroom, check, sort, move into folders, etc
- check that all filenames are globally unique (using Lightroom's auto-rename works well for that)
- develop photos, crop, etc, mark the good & bad ones
- tag, put keywords and titles (still in lightroom)

- export the good ones at full resolution to a separate directory
- run a photoshop script to resize / sharpen / save for web

- Run python script to copy EXIF from original pics to saved pics
- Run python script to rebuild the original directory tree in the export dir
- upload to server
- Gallery automagically scans the dirs and adds the pics
- Recheck that albums have correct names, set correct dates on new albums, add some comments to albums

- Run python+php lightroom/gallery2 script which transfers ratings, keywords, titles, captions etc, from lightroom's database, to gallery2

Thanks for the help, lol.

I still have a little problem with Unicode accents getting transformed into crap at some point. I guess this is due to PHP's ignorance of Unicode... Have to investigate.

Am surprised no one responded to this. This sounds like a very cool application of transferring metadata...

I don't like to release undocumented (and therefore rather unusable by others) code. So I will take the time to document it and make a webpage for it, if people are interested... Are you interested ?

Hi, I just got gallery setup again last night. Trying again to tame the beast of choas and duplicates in my photo collection. I am hoping to make gallery my central repository. However, I have lots of data in lightroom I'd like to import/sync. Even if the code is not ready for wide distribution, I'd be interested in at least reading thru it to get it working, or get ideas how I might get something similar working
+1 interested
~Michael

Yo.
Here is the code, with a PDF manual covering installation instructions for the required tools, and how to use it.
I submitted a project on SourceForge, but it needs 2 days to be approved. In the meantime, you can get the code there :

http://home.peufeu.com/temp/gallery2lightroom.zip

This is beta (works for me...) so please tell me about your experience, if it works, what I can improve, etc.
Please don't ask for a GUI, though ;)

See ya

PS : the code that reads metadata from Lightroom is easily replaceable by some other code which would read its metadata from another source (Aperture, whatever) if someone wants to write it.

I think Save for Web results in smaller files... if your website resizes or otherwise processes the images (like Flickr, Picasa or Gallery2 if you use the Resize feature), it won't make a difference, though.

After looking at the packages it's clear that the two are pretty different.

@Peufeu:
I'd highly recommend to use the Gallery API on the server component. By using the API, your integration will be much more robust and will have less limitations.

I see that you can't use the webdav or GR protocol for now since neither of them expose item ratings. GR doesn't expose the originationdate as far as I know. webdav exposes everything you need, but ratings. If you could submit a patch for the webdav module to handle ratings in proppatch requests as well, you could remove your server component and just talk webdav with the g2 server.
And we're working on a new web services / RPC API which will be easier and more powerful than the current GR protocol.

So I understand that you can't use GR/webdav for now. So you have to write your own server component.
But why not use the Gallery API (GalleryCoreApi, ...)? Your current code is limited to a table prefix of "g2_" and only works with MySQL.
Also, the likelihood for bugs/security issues with your server component is much larger than if you'd use the Gallery API. The reason is simple: It's a lot of new code and it's bypassing all checks / layers of G2. Sure, you can try to account for all that, but bug-free code is a rarity. It doesn't exist in practice.

Don't get me wrong. I'm very excited to see your Lightroom <-> G2 meta-data integration! I just want to avoid potential data-corruption or even security problems and adding a new server-connector (your server-side script) introduces such a risk, largely unnecessarily.
I'd be glad to guide you in either transforming your server-connector to a GalleryEmbed API / GalleryCoreApi script making as much use of safe Gallery API methods as possible. Or transforming the client to use the GR or the webdav protocol and patching the server's protocol implementation to handle ratings too.

Maybe there's also potential in joining forces with the other Lightroom integration.

--------------
Documentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage

I finally published the new release for my plugin.
New features:
* Fully compliant with Lightroom 2.x SDK.
* Support for Gallery 1.x, Gallery 2.x and Jallery 1.x
* Available in English, French & German.
* Still user defined export of Lightroom metadata to Gallery.

You can find it there: http://www.starway.org/blogs/Photographie/lightroom-gallery/

By the way, I hope Gallery 3 will provide a better API to manipulate the Gallery from an external application ;)

This is interesting.

Regarding your problem, I was expecting the solution on the other way arourd : Flickr assings an UID to every picture, and this UIS is stored in the catalog when the pictured has been published by Jeff's plugin. I am expecting the same feature in Gallery. However, the Gallery Remote API does not provide any such information. So this is not doable in G1/G2. I still hope that G3 will expose some kind of UID of the picture.

This way, pictures can be reuploaded (bigger, better treatment, etc.) or updated if the metadata has been changed.

Lightroom 3 also provides some new publishing service, however, I am not very sure this will be very convenient in G2/G3 if several (10+) albums are used.

By the way, this method might not work for me : the files generated for Gallery do not have the same filename as the original file...

Interesting. Let's consider that :

- all the tagging, keywording, assigning titles and descriptions etc, should be done in Lightroom (and not repeated in Gallery)
- this information can change (maybe you come back later to fix/add keywords or edit a title)

=> need for an automated metadata update for existing images

- you can also modify a bunch of images a bit, change the develop settings, and want to upload the new versions to Gallery without having to manually find the old ones and replace them
- maybe next year you get less paranoid about copyright, or you get a cheap new web hosting with plenty of space, and all your friends with HD screens complain about your tiny photos, so you'd like to re-upload larger versions of all your images,
- or the new ACR engine in LR 3.0 makes your photos look a lot better

=> need for a way to re-upload a batch of images to replace existing ones

Since this would be an extremely useful, why not do it ?

I think we need a mix of Lapinou's export plugin and my stuff, which would do the following :

1) Export to Gallery and create a link between Lightroom's images and G2's images during the export

This "link" is the essential part : as seen above, the filename is a pretty crummy way of doing it ; I used that only because I had no other choice, but frankly, it's ugly, doesn't work if your filenames are not unique or differ between Gallery and Lightroom, etc.

We need something like a GUID stored inside the EXIF, IPTC or XMP data, at export, inside the photo itself.

If you dug a bit in Lightroom's database, you may have noticed that Lightroom ALREADY generates GUIDs for images and stores them inside its sqlite database ! Unfortunately, it doesn't include it inside the exported files' metadata. Could Lapinou's export plugin grab those ?

The GUID could also be generated by Gallery, and added to Lightroom by Lapinou's plugin.

Then there is the problem of taking an existing Lightroom catalog, an existing Gallery, and linking all the existing photos. Using capture times, filenames, and/or some image matching, this should not be too difficult.

2) Synchronize Gallery

Once each image has a GUID on both sides of the fence, updating meta-data and re-uploading new versions can be made push-button.

For metadata updates, though, a bulk API is really needed : here, I have a bit of PHP that dumps all the photo information in JSON format. Python parses this easily. Then a list of updates is sent (basically a list of image_id, fieldname, new_value). It is simple and extremely fast.

I think we should wait for LR 3.0 since this will have a more interesting plugin framework to work with.

Considering the export plugin, is it possible to add the Lightroom GUID to any of the embedded fields inside the JPEG file ?

The current plugin works with LR3 and G1/G2.

I'll wait to see how the REST interface of G3 progresses before doing any work.

However, don't have huge expectations : neither G1, G2 nor G3 assigns unique IDs to the pictures, so having a publishing service working as the Flickr one is impossible. If for instance you move a picture/rename a picture in Gallery, there is no way to link it back to an element in LR. That's a shame.

Well, the real problem is that the Gallery Remote API does not return nor manipulate G2 elements with IDs.

The id would be useful :
- to reupload an element (even if it has been moved or renamed in Gallery)
- to track comments on a picture (new LR3 feature)

Well, I tend to prefer to reuse the work already done.

If everyone who needs to interface Gallery with an external application (Gallery Remote, Lightroom, Aperture, ACDSee...) had to write a Gallery module, then we would end up with a lot of different modules, all probably incomplete and having different bugs.

Moreover, if was not a huge investment to understand LR API and the LUA language. However, if I had to study as well the Gallery model and API in order to have a working solution, I don't think I would have started the work.

So from my Lightroom developer point of view, Gallery 1 & 2 come with an API to manipulate content : it's called Gallery Remote, and I try to do as much as I can with it.

Yes, in the ends it depends from which point of view you look at things.

Maybe the core is perfect, I honnestly don't know. However, From my perspective, this is something that is note easily done.

For instance, I saw that Übermind is featured on the Gallery site. Übermind used to make a plugin for Lightroom too. They used the Gallery Remote interface, so I guess they're doing the same for the Aperture and iPhoto plugin (the feature list matches was is possible with the GR interface).
This means that the official API to access Gallery content is important: whatever are the core feature, in the end Gallery is limited by the features provided by its official API.

I strongly disagree with GR not being an official API:

• It is a core module (not a contrib one).
• It is shipped with Gallery 2, no need to get it from somewhere else.
• It is documented on the Codex : http://codex.gallery2.org/Gallery_Remote:Protocol. If it was meant to be only used by Gallery Remote, then there was no need to fully document the protocol.
• Gallery Remote is featured as an official Gallery "product" since it appears at the top of the Gallery page.
• Many other developpers have used this module for the same purposes. Übermind is one of the very last examples.

Then, why I won't write a new module for Gallery:

• For a long time (maybe it's not true anymore), G2 had a stip learning curve.
• I work 50 to 60 hours a week so I tend to concentrate my effort on things I can make a difference. I was using Gallery, I was using Lightroom, making existing LR to gallery plugins work right was much easier than starting everything from scratch. If Adobe did not provide a good SDK and if I had to write a Gallery module, I would have ditched Gallery and migrate everything to Flickr or Zenfolio.
• Writing a module for Gallery means explaining people how to install it. Since some LR users have questions regarding activating a plugin within LR, I don't wan't to spend ages explaining people how to install another module within Gallery. I did the plugin for me in the first place and offered to everybody else. I'm not willing to spend hours doing technical support.
• Adding another module to G2 might add some bugs and some vulnerabilities. No code is 100% safe.
• Gallery 2 is "dead" as far a development is concerned.

And finaly, even if G2 could have supported external IDs to external tools, G3 doesn't seam to support this. Let me quote Tim:

You can check the whole discussion at http://old.nabble.com/REST-docs-to27351326.html

So, to sum up:

• Gallery Remote is the only simple protocol to use to manipulate G2's content, and it's not as advanced as it could be to have a very nice integration of Gallery with Lightroom (and other tools).
• Gallery 2 is end of life, so nobody should expect a better official protocol on it.
• Gallery 3 REST interface will fix some shortcomings of G2 GR protocol, but it won't be powerful enough to allow a complete interface between Lightroom and Gallery (no IDs provided).

Yes, I tried, however my subscription to the mailing list was with my old address that is now redirected to my new one, so when I replied the mail got bounced back... I never took the time after to try to fix this, especially since the discussion showed that my position was not the position the majority was thinking for G3.

I would have loved to have a better protocol for accessing G2. But I think this module should not be aimed to any tool in particular. If well done, Gallery Remote, Lightroom, Aperture, iPhoto would all use it. I think Pierre-Luc (Gallery Remote maintainer) would have loved it too!
Note to G2 developpers and LR users : if someone wants to improve my module and plug it in G2 with a specific G2 module, please do!

G3 is a nice step forward with its REST interface, but in the end it is still not the best solution for LR / Aperture users who manage a photo collection and want to display it online. G3 would have needed to be more like a Flickr on your website (with themes) to be perfect for this purpose.

I was wondering if it might be possible to get a copy of your most recent plugin. I just purchased a copy of Lightroom 3 and I'd like to extend it to support a few additional metadata fields (for example, allowing for Photo Captions to uploaded to the extended description field).

I have a very old version of the plugin that works, but it lacks these additional features. I'm wondering if the version you have been developing might include them.