Using Symlinks or Null Mount to Image Data

chaoticevidence

Joined: 2003-11-20
Posts: 5

Posted: Mon, 2003-12-22 15:36

Background
I keep a copy of all my images on my co-located unix box (with the originals on my local machine at home and burned to <insert favorite media here> yadda yadda yadda). I'm somewhat retentive when it comes to my structure and hard drive neatness, but that's not relevant information.

A typical picture hierarchy might resemble the following:

The Build Up to the Request
When Gallery2 adds images, you need to specify the local directory(ies) (I specify /home/username/pictures as the top-level and gallery is smart enough to know to allow me into the subdirectories) on the server that holds said images for adding to albums. I'm ok with that, it's a good security policy.

Gallery2 happily imports the images and copies the image from the source into the directory I specified that is to be the album directory. Well that's just fine and dandy everything works wonderful. Kudos to you developers for doing a fine job.

I've attempted to use a symlink'ed directory within the Gallery2 image store as the source of the images and then hoping it'd copy the symlinks into the Gallery2 album diretory (this creates an evil web of linking, but only 1 image). It didn't work so well (translated, not at all).

The Actual Request! (Finally!)
I'm sure this has been asked before and I've forgotten the answer, but, is it possible to not copy the images but merely link (symbolic) to them? If necessary, I'm willing to symlink or null mount (mount -t null) my /home/username/pictures directory(ies) under the Gallery2 albums directory store (to use as either the source for copying or the actual displayed full-size image).

Additonal Data
Hard drive space is relatively cheap but co-lo RUs are not (more drives == more space == more money, etc. etc.). I use a Canon 10d which produces large images (and as technology marches dutifully forward, the images will get larger). I prefer to allow my adoring gallery visitors to imbibe on high-resolution full-size images, so I don't tend to scale them down or save them with a JPEG quality of 20, and this starts to consume disk space (which I'm okay with, for the first image). What I'm not-so-okay-is-there-a-possible-solution is having a duplicate copy of the same image for the use of Gallery2.

Conclusion
Anyway, that's a lot of ramble and drivel, but if there's any possibility that a single stored image (regardless of the location of the image) can serve as the source for subsequent Gallery2 needs, that'd be just awesome.

Thanks!

/aaron

Additonal Data Part Duex
I just conducted a quick test ...

I used the normal procedures to create a new album and Gallery2 did it's thing and copied the images to the new album directory. I deleted one of the images and then tried to view the re-sized image, which of course Gallery2 complained loudly about (since the source file was deleted).

I created a symlink to the image using the same name and Gallery2 responded with a positive answer by displaying the re-sized image. This somewhat proves that the symlink concept is possible.

A concern with symlinks is that Gallery2 doesn't actually try to modify the original file. To get around this, if Gallery2 detects it's a symlink and not an image file, ensure none of the image editing techniques work. Even more "user friendly" would be to ask the user if they wanted the symlink converted to a copied file for that particular image so editing is possible and it's a copy being edited.[/]

chaoticevidence

Joined: 2003-11-20
Posts: 5

Posted: Mon, 2003-12-22 16:02

Problems
I've thought about it some more, and the problem of source image mobility reared it's ugly head in my thought process. For me, this isn't so much an issue, I have a defined method for my directory hierarchy and it isn't changing anytime soon. However, I might rename/delete an image or album.

In that case, I'd suggest a token system-default message in place of the image that indicates the source image file is no longer available. This keeps the Gallery2 database information intact, ensuring the integrity of the database. A solution could be devised to allow for a user to re-symlink the individual image or album.

bharat

Joined: 2002-05-21
Posts: 7985

Posted: Tue, 2003-12-23 01:37

The bad news is that we can't use symlinks. They're just not portable to all platforms and we really want G2 to be as portable as possible.

The good news is that I implemented a linking feature about a week and a half ago that sounds like it will do exactly what you want. To use it, just select the "Create link" action on any item that you want to link (except albums -- can't do that yet). This will allow you to create a lightweight link of the item in any album where you have permission to add items.

All links share the original file from the original data item. Once a data item is linked, you can no longer modify the original image (because doing so would affect the data source for all links) so any rotate/resize/scale operations would affect the preferred derivative (this is a copy we make of the original when you modify an image but check the "preserve original" checkbox). If you delete the original, then one of the links becomes the new "original" and all other links point to it instead.

Each link can have its own title, description, keywords, permissions, etc and can be treated as an independent entity so creating a link is just like uploading the same image, except you save the disk space.

chaoticevidence

Joined: 2003-11-20
Posts: 5

Posted: Tue, 2003-12-23 14:26

I read the release notes and saw the Create link action. That's what prompted me to start this line of thinking ... about linking. I completely understand your portability requirements.

If I'm understanding you correctly, the image (or data item) first has to be inserted (copied) into the Gallery2 file system ($gallery->setConfig('data.gallery.base') which defeats the purpose of what I'm attempting to do. Translated, the Create link is only available once a data item has been inserted into the Gallery, which insinuates that is has been copied from the somewhere (directory, Gallery Remote, HTTP source, etc). I'm trying to establish a link to the original item in the first place, without copying it.

I know what I'm asking is out of the normal scope for a lot of users, and since I've come up with a "work around", I'll see if I can't flesh that out and make it a little neater (html/php magic). It'll be a kludge, but a working kludge is better than nothing.

jmullan

Joined: 2002-07-28
Posts: 974

Posted: Tue, 2003-12-23 17:06

chaoticevidence, why not simply use gallery2 to maintain your image hierarchy, and symlink to its hierarchy for your other purposes? Once you have imported everything you could back up the gallery data tree and delete the original tree. Or leave it.

chaoticevidence

Joined: 2003-11-20
Posts: 5

Posted: Tue, 2003-12-23 20:17

hahahaha, that's what i call thinking outside the box! i like it!

not all of my photos make it into a gallery and some photos are in different galleries. i wonder what type of havoc would be created with pointing two different instances of gallery at the same $data.gallery.base.

from a purity standpoint, i'd prefer to ensure my photo repository is outside of any application influence (within reason of course).

but your idea is a good one and i shall consider how i could change my workflow.

jmullan

Joined: 2002-07-28
Posts: 974

Posted: Tue, 2003-12-23 21:35

There's a box? Does it make thinking easier?

I've been toying with the notion of doing just what I suggested. Not all images need to be visible to the public in your gallery. I was thinking about having G2 be the first (or second) step in my workflow, and adding all of my images to hidden albums. I really want to set up keywords (categories if you will) and some sort of virtual albums built from whichever images meet certain criteria. That way I could keep all of my images in a central place but have some of them show up here and there throughout my gallery as appropriate. Or something.

That is neither here nor there, but is a long way off. ;)

2pin

Joined: 2004-10-15
Posts: 2

Posted: Fri, 2004-10-15 12:15

Hi Chaoticevidence (you must hate typing that user name in all the time, thank goodness for cookies).

Am I right in thinking that you would be happy with an option to use symbolic links instead of a file copy when using the Add Items - From Local Server gallery action to import/add files from your local files system (mounted else where maybe)? This would make me happy I think? To get around the cross platform problems I guess the option would only be available to *nix platforms.

I will anyway have a play around and hack the source a bit.

Regards.

2pin.

baschny

Joined: 2003-01-04
Posts: 328

Posted: Fri, 2004-10-15 12:47

2pin, this exact feature is already being implemented as a module right now by our new developer. It's almost finished, we just have to review his code and commit. Should be there in the next alpha/beta.

Cheers,
Ernesto

2pin

Joined: 2004-10-15
Posts: 2

Posted: Tue, 2004-10-19 16:22

Hi Baschny,

Thanks for your reply I guess I should spend my time doing something else for a bit then!

I have another request which may also have already been addressed, I would like to have an option in the add items page to search all sub- directories adding any known (e.g. .jpg) files found along the way. If this feature is not going to be in the import module I would be happy to develop it for you.

Thanks.

2pin.

valiant

Joined: 2003-01-04
Posts: 32509

Posted: Tue, 2004-10-19 16:26

a wget interface

baschny

Joined: 2003-01-04
Posts: 328

Posted: Tue, 2004-10-19 17:02

2pin, the ItemAddFromWeb core view can handle media in a single file. You could create a new ItemAdd-view in a new module based on this one and have it recursively scan through the requested website.

Alternatively your module can use the external wget, as valiant suggested, to do the "dirty job".

Either way, sounds like a cool module.

mikgus

Joined: 2002-12-01
Posts: 13

Posted: Sun, 2004-10-31 15:19

Is there a way where you can search for all images that for example has custom fields ="soccer" and then have them linked to a album? That is; I would like to create special "subject-albums" that gets its images from the other "normal albums".

Another exampel. I got hundreds of images of birds, in albums orderd after the different trips that I made to take the pictures. But now I also want an album named: Eagles, where I link all images withthe word eagle in a custom field.

valiant

Joined: 2003-01-04
Posts: 32509

Posted: Sun, 2004-10-31 15:49

mikgus, your feature request is not related to this topic.

Please add your comments to this thread:
http://gallery.menalto.com/index.php?name=PNphpBB2&file=viewtopic&t=21145&highlight=categories#96096

This topic here is about symlinks which has something to do with filesystems and where to physically (not logically) store the images.

altitude

Joined: 2004-10-07
Posts: 2

Posted: Wed, 2004-12-01 18:18

Just as an FYI, the add item from local server using symlink has been added to the CVS tree (it went in after G2A4 came out, so for now you'll need to pull the CVS source).

See http://gallery.menalto.com/index.php?name=PNphpBB2&file=viewtopic&t=23440 for more information.

Good Luck.

...alex...

jamzyamz

Joined: 2004-01-12
Posts: 154

Posted: Thu, 2004-12-02 01:32

I absolutely love this feature! now if we could just copy albums or symlink albums I'd be very happy indeed.

Altitude are you the one that did the symlink feature? If so Kudos works very well. I love the fact that all the thums are listed on one page too. With regular gallery stuff I have to next page forever eveytime I want to do things like delete all pictures.

If you are the Alex that submitted the symlink feature, any other stuff you are working on with it?

chaoticevidence Joined: 2003-11-20 Posts: 5	Posted: Mon, 2003-12-22 15:36
	Background I keep a copy of all my images on my co-located unix box (with the originals on my local machine at home and burned to <insert favorite media here> yadda yadda yadda). I'm somewhat retentive when it comes to my structure and hard drive neatness, but that's not relevant information. A typical picture hierarchy might resemble the following: /home/username/pictures/2002 /home/username/pictures/2002/January1-2002 /home/username/pictures/2002/April4-2002 /home/username/pictures/2003 /home/username/pictures/2003/LifeInTheFastLaneJun1-2003 /home/username/pictures/2003/HelterSkelterOct31-2003 /home/username/pictures/Misc/BrokenWindowAug8-2003 /home/username/pictures/Misc/LateExGirlfriendsSep5-2003 The Build Up to the Request When Gallery2 adds images, you need to specify the local directory(ies) (I specify /home/username/pictures as the top-level and gallery is smart enough to know to allow me into the subdirectories) on the server that holds said images for adding to albums. I'm ok with that, it's a good security policy. Gallery2 happily imports the images and copies the image from the source into the directory I specified that is to be the album directory. Well that's just fine and dandy everything works wonderful. Kudos to you developers for doing a fine job. I've attempted to use a symlink'ed directory within the Gallery2 image store as the source of the images and then hoping it'd copy the symlinks into the Gallery2 album diretory (this creates an evil web of linking, but only 1 image). It didn't work so well (translated, not at all). The Actual Request! (Finally!) I'm sure this has been asked before and I've forgotten the answer, but, is it possible to not copy the images but merely link (symbolic) to them? If necessary, I'm willing to symlink or null mount (mount -t null) my /home/username/pictures directory(ies) under the Gallery2 albums directory store (to use as either the source for copying or the actual displayed full-size image). Additonal Data Hard drive space is relatively cheap but co-lo RUs are not (more drives == more space == more money, etc. etc.). I use a Canon 10d which produces large images (and as technology marches dutifully forward, the images will get larger). I prefer to allow my adoring gallery visitors to imbibe on high-resolution full-size images, so I don't tend to scale them down or save them with a JPEG quality of 20, and this starts to consume disk space (which I'm okay with, for the first image). What I'm not-so-okay-is-there-a-possible-solution is having a duplicate copy of the same image for the use of Gallery2. Conclusion Anyway, that's a lot of ramble and drivel, but if there's any possibility that a single stored image (regardless of the location of the image) can serve as the source for subsequent Gallery2 needs, that'd be just awesome. Thanks! /aaron Additonal Data Part Duex I just conducted a quick test ... I used the normal procedures to create a new album and Gallery2 did it's thing and copied the images to the new album directory. I deleted one of the images and then tried to view the re-sized image, which of course Gallery2 complained loudly about (since the source file was deleted). I created a symlink to the image using the same name and Gallery2 responded with a positive answer by displaying the re-sized image. This somewhat proves that the symlink concept is possible. A concern with symlinks is that Gallery2 doesn't actually try to modify the original file. To get around this, if Gallery2 detects it's a symlink and not an image file, ensure none of the image editing techniques work. Even more "user friendly" would be to ask the user if they wanted the symlink converted to a copied file for that particular image so editing is possible and it's a copy being edited.[/]
	XUser login Username: * Password: * Create new account Request new password
chaoticevidence Joined: 2003-11-20 Posts: 5	Posted: Mon, 2003-12-22 16:02
	Problems I've thought about it some more, and the problem of source image mobility reared it's ugly head in my thought process. For me, this isn't so much an issue, I have a defined method for my directory hierarchy and it isn't changing anytime soon. However, I might rename/delete an image or album. In that case, I'd suggest a token system-default message in place of the image that indicates the source image file is no longer available. This keeps the Gallery2 database information intact, ensuring the integrity of the database. A solution could be devised to allow for a user to re-symlink the individual image or album.

bharat Joined: 2002-05-21 Posts: 7985	Posted: Tue, 2003-12-23 01:37
	The bad news is that we can't use symlinks. They're just not portable to all platforms and we really want G2 to be as portable as possible. The good news is that I implemented a linking feature about a week and a half ago that sounds like it will do exactly what you want. To use it, just select the "Create link" action on any item that you want to link (except albums -- can't do that yet). This will allow you to create a lightweight link of the item in any album where you have permission to add items. All links share the original file from the original data item. Once a data item is linked, you can no longer modify the original image (because doing so would affect the data source for all links) so any rotate/resize/scale operations would affect the preferred derivative (this is a copy we make of the original when you modify an image but check the "preserve original" checkbox). If you delete the original, then one of the links becomes the new "original" and all other links point to it instead. Each link can have its own title, description, keywords, permissions, etc and can be treated as an independent entity so creating a link is just like uploading the same image, except you save the disk space.

chaoticevidence Joined: 2003-11-20 Posts: 5	Posted: Tue, 2003-12-23 14:26
	I read the release notes and saw the Create link action. That's what prompted me to start this line of thinking ... about linking. I completely understand your portability requirements. If I'm understanding you correctly, the image (or data item) first has to be inserted (copied) into the Gallery2 file system ($gallery->setConfig('data.gallery.base') which defeats the purpose of what I'm attempting to do. Translated, the Create link is only available once a data item has been inserted into the Gallery, which insinuates that is has been copied from the somewhere (directory, Gallery Remote, HTTP source, etc). I'm trying to establish a link to the original item in the first place, without copying it. I know what I'm asking is out of the normal scope for a lot of users, and since I've come up with a "work around", I'll see if I can't flesh that out and make it a little neater (html/php magic). It'll be a kludge, but a working kludge is better than nothing.

jmullan Joined: 2002-07-28 Posts: 974	Posted: Tue, 2003-12-23 17:06
	chaoticevidence, why not simply use gallery2 to maintain your image hierarchy, and symlink to its hierarchy for your other purposes? Once you have imported everything you could back up the gallery data tree and delete the original tree. Or leave it.

chaoticevidence Joined: 2003-11-20 Posts: 5	Posted: Tue, 2003-12-23 20:17
	hahahaha, that's what i call thinking outside the box! i like it! not all of my photos make it into a gallery and some photos are in different galleries. i wonder what type of havoc would be created with pointing two different instances of gallery at the same $data.gallery.base. from a purity standpoint, i'd prefer to ensure my photo repository is outside of any application influence (within reason of course). but your idea is a good one and i shall consider how i could change my workflow.

jmullan Joined: 2002-07-28 Posts: 974	Posted: Tue, 2003-12-23 21:35
	There's a box? Does it make thinking easier? I've been toying with the notion of doing just what I suggested. Not all images need to be visible to the public in your gallery. I was thinking about having G2 be the first (or second) step in my workflow, and adding all of my images to hidden albums. I really want to set up keywords (categories if you will) and some sort of virtual albums built from whichever images meet certain criteria. That way I could keep all of my images in a central place but have some of them show up here and there throughout my gallery as appropriate. Or something. That is neither here nor there, but is a long way off. ;)

2pin Joined: 2004-10-15 Posts: 2	Posted: Fri, 2004-10-15 12:15
	Hi Chaoticevidence (you must hate typing that user name in all the time, thank goodness for cookies). Am I right in thinking that you would be happy with an option to use symbolic links instead of a file copy when using the Add Items - From Local Server gallery action to import/add files from your local files system (mounted else where maybe)? This would make me happy I think? To get around the cross platform problems I guess the option would only be available to *nix platforms. I will anyway have a play around and hack the source a bit. Regards. 2pin.

baschny Joined: 2003-01-04 Posts: 328	Posted: Fri, 2004-10-15 12:47
	2pin, this exact feature is already being implemented as a module right now by our new developer. It's almost finished, we just have to review his code and commit. Should be there in the next alpha/beta. Cheers, Ernesto

2pin Joined: 2004-10-15 Posts: 2	Posted: Tue, 2004-10-19 16:22
	Hi Baschny, Thanks for your reply I guess I should spend my time doing something else for a bit then! I have another request which may also have already been addressed, I would like to have an option in the add items page to search all sub- directories adding any known (e.g. .jpg) files found along the way. If this feature is not going to be in the import module I would be happy to develop it for you. Thanks. 2pin.

valiant Joined: 2003-01-04 Posts: 32509	Posted: Tue, 2004-10-19 16:26
	a wget interface

baschny Joined: 2003-01-04 Posts: 328	Posted: Tue, 2004-10-19 17:02
	2pin, the ItemAddFromWeb core view can handle media in a single file. You could create a new ItemAdd-view in a new module based on this one and have it recursively scan through the requested website. Alternatively your module can use the external wget, as valiant suggested, to do the "dirty job". Either way, sounds like a cool module.

mikgus Joined: 2002-12-01 Posts: 13	Posted: Sun, 2004-10-31 15:19
	Is there a way where you can search for all images that for example has custom fields ="soccer" and then have them linked to a album? That is; I would like to create special "subject-albums" that gets its images from the other "normal albums". Another exampel. I got hundreds of images of birds, in albums orderd after the different trips that I made to take the pictures. But now I also want an album named: Eagles, where I link all images withthe word eagle in a custom field.

valiant Joined: 2003-01-04 Posts: 32509	Posted: Sun, 2004-10-31 15:49
	mikgus, your feature request is not related to this topic. Please add your comments to this thread: http://gallery.menalto.com/index.php?name=PNphpBB2&file=viewtopic&t=21145&highlight=categories#96096 This topic here is about symlinks which has something to do with filesystems and where to physically (not logically) store the images.

altitude Joined: 2004-10-07 Posts: 2	Posted: Wed, 2004-12-01 18:18
	Just as an FYI, the add item from local server using symlink has been added to the CVS tree (it went in after G2A4 came out, so for now you'll need to pull the CVS source). See http://gallery.menalto.com/index.php?name=PNphpBB2&file=viewtopic&t=23440 for more information. Good Luck. ...alex...

jamzyamz Joined: 2004-01-12 Posts: 154	Posted: Thu, 2004-12-02 01:32
	I absolutely love this feature! now if we could just copy albums or symlink albums I'd be very happy indeed. Altitude are you the one that did the symlink feature? If so Kudos works very well. I love the fact that all the thums are listed on one page too. With regular gallery stuff I have to next page forever eveytime I want to do things like delete all pictures. If you are the Alex that submitted the symlink feature, any other stuff you are working on with it?

Using Symlinks or Null Mount to Image Data

XUser login