Backing up Gallery2

nbc

Joined: 2003-10-12
Posts: 7
Posted: Thu, 2005-08-18 14:15

I've used Gallery 1 for a long time. Installed Gallery 2 this week. Turned on MySQL, and imported my Gallery 1 data. No problems - thanks to all the developers!

Question - backup. I can see the .jpg files in the data directory I set up. But what do I need to do to back up the MySQL database? Where should it be located and what is the best way to back it up?

Can I back up all of gallery with one set of commands? Data, s/w and mysql?

Thanks,

nbc

 
paulbearne

Joined: 2005-08-01
Posts: 87
Posted: Fri, 2005-08-19 11:11

I am about to put a large Gallery(2-3g) online and a backup tool would be very nice.

 
virshu
virshu's picture

Joined: 2003-09-13
Posts: 314
Posted: Sat, 2005-08-20 03:15

well, you don't really want to back up software together with data... Currently, the easiest way to backup jpg is through file system, and the database - through your database tools. I'll put a (somewhat limited) album export/import function to the user contribution page. I'll put a note here when I do that...

Update: here is the module. Note, that it only makes an extract from the database; it doesn't move files. If you want to restore (import) from the same place as you exported - you need to copy album directory tree. I am usually importing these pictures on a different box; so copying a gig of data wouldn't work for me.

Also, zipping (compression) may or may not work. If it doesn't work - I can't fix it at the moment (since it requires a change in core apis). I will address it after go-live.

Any other problems or comments - please let me know
Felix

 
thecaretaker

Joined: 2005-09-22
Posts: 1
Posted: Thu, 2005-09-22 15:22
virshu wrote:
Update: here is the module.

Hi, I'm a newbie to the Gallery, installed it 3 days ago, but can't seem to find the link to this module you speak about. This sound what I could be looking for to have some kind of back-up of my photos.

I have read 100s of posts about ways to backup your album folders in G2, the most promising was a script that would run on the server and tempererilly change the permissions to let you copy via ftp, then you could re-set the permissions for security and remove the script. But alas, it assumed the reader had the knowledge how to write such a script. I don't have those skills.

I have been moved 4 times now by my server host to another server. My fear is without locally being able to back-up my pictures, they would all be lost forever. So, not totally sure if your script will help, but can't seem to find it to try it. Is anyone able to help me out?

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2005-09-22 15:25

it's attached to the first post in this topic.

 
c4actbe
c4actbe's picture

Joined: 2004-09-01
Posts: 83
Posted: Fri, 2005-09-23 17:55

The content of archive looks suspicious....
there is only one file " importexport_561" without any extension... is it correct?

HostingCompared.com

 
c4actbe
c4actbe's picture

Joined: 2004-09-01
Posts: 83
Posted: Fri, 2005-09-23 17:58

The content of archive looks suspicious....
there is only one file " importexport_561" without any extension... is it correct?

HostingCompared.com

 
c4actbe
c4actbe's picture

Joined: 2004-09-01
Posts: 83
Posted: Fri, 2005-09-23 18:00

My appologies for multipost... i was waiting at least a min and progress bar was at zero

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Fri, 2005-09-23 18:29

no, for me the archive looks ok, lots of files in it.
it's a .tar archive, open it with winzip or tar.

 
swordfish
swordfish's picture

Joined: 2004-10-01
Posts: 388
Posted: Sat, 2005-09-24 06:40

I could not open it either at first.
Download the file and rename to importexport_561.tar.gz
Winzip or similar windoze tool to open or tar -xvzf on Linux.

 
dracula

Joined: 2005-10-14
Posts: 61
Posted: Tue, 2005-11-15 15:41

thanks, this is exactly what I am looking for !
the export works great, but the Import doesn't

Fatal error: Call to undefined function: getlayout() in /var/www_livadaru/html/cristian/gallery2/modules/importexport/classes/ImportHelper.class on line 429

any idea what's wrong here?

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2005-11-15 15:44

the module is very old and getLayout was removed about 5 months ago.

 
dracula

Joined: 2005-10-14
Posts: 61
Posted: Tue, 2005-11-15 15:52

this is just not my day ... any workaround for this ? a export feature is useless without import. The export does great job and beofre starting the import I can see the albums and the descriptions o it has all it needs.
with what could I replace getLayout ? I need to get this thing running, or any other way to move an album from gallery a to gallery b

 
dracula

Joined: 2005-10-14
Posts: 61
Posted: Tue, 2005-11-15 16:34

I just found a workaround, I comented out the getLayout line and the setLayout and it works.

 
id01

Joined: 2005-11-07
Posts: 62
Posted: Fri, 2006-01-06 20:02

This module definitely need some attention. With a relible back up and restore method the project will feel a lot more complete.
Those other way to back up does seem a little intimating, since you have to acess the database directly at the same time have to have root access to the server to chowns the files. Neither of those are very userfriendly, sometime downright impossible when you are dealing with a less then responsive host in a shared hosting enviroment.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Fri, 2006-01-06 21:12

on shared webhosting, you've got backup / download solutions usually anyway. i've got a cheap shared webhosting plan and i can click in cpanel in 2 things to get a db backup and a data backup both as a zip.

 
id01

Joined: 2005-11-07
Posts: 62
Posted: Sat, 2006-01-07 23:09

But shared hosting doesn't let me upload and set the directory back to owned by the server. Everytime I tried that method It keep giving me major grift about the directory right when I do any server move or directory structure changes. Given, I understand it is not a major issue. But a fail safe and easy to use backup/restore without needing root access to the server to complete the cycle must have some value.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2006-01-08 03:32

then just vote for the backup feature request on http://gallery.menalto.com/sfvote/all :)

 
borkgallery

Joined: 2006-02-05
Posts: 1
Posted: Sun, 2006-02-05 13:40

I am new to gallery and trying to decide if I want to use Gallery 1 or 2. My hosting provider allows for high performance MySql database up to 100MB in size. Does G2 include the image data in the DB, or only the metadata? If it is only the metadata then G2 look slike a good choice for me. Thanks in advance!
--Bork!

 
mindless
mindless's picture

Joined: 2004-01-04
Posts: 8601
Posted: Mon, 2006-02-06 03:53

metadata in the db, images on filesystem.

 
vahan1

Joined: 2002-11-01
Posts: 27
Posted: Wed, 2006-02-15 04:54

this module's been such a helpful tool! Saved and will save me tons of work :) Thank You!!!!

Would be awesome if someone updated it. Few notes..

* Needs to have the getlayout() function updated - I assume because I commented it out, the settings from original album did not import in.

* Better documentation as far as which part of album tree needs to go where. I spent good couple of hours playing with it before I figured it out.

* How to turn off Export Album link under albums when no user's logged in? My current solution is to deactivate the module when I don't use it.

* After importing in over 1000 items and about 10 albums (at once), I had a strange behavior with rearranging/reordering the albums on the main page of the import. Rearrange wouldn't work, the settings wouldn't update. Reorder worked better but I still saw some sporradic hicups where the album order didn't update.

Thanks again!
-vahan

 
afatac

Joined: 2006-02-05
Posts: 52
Posted: Sun, 2006-03-05 08:59

I am trying out this module but looks like it cannot be used with G2 2.1 RC1. Error message at Site Admin-->Module:

Incompatible module!
Core API Required: 6.7 (available: 7.1)
Module API Required: 2.0 (available: 3.0)

Any solution?

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2006-03-05 17:54

right. the module is experimental and hasn't been updated for g2.1 yet.

 
afatac

Joined: 2006-02-05
Posts: 52
Posted: Mon, 2006-03-06 03:02

In that case, how can we export data that we have already entered to thousands of images?

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Mon, 2006-03-06 04:33

- there's no built-in export mechanism.
- backup instructions can be found on codex.gallery2.org -> howto
- you can adopt this module for g2.1 yourself

 
dracula

Joined: 2005-10-14
Posts: 61
Posted: Mon, 2006-03-13 09:37

ok no a complete different question
How do I uninstall the module?
I uninstalled it over the Admin interface it now show's me that it's unisntalled, but when adding an Item I still have the Backup tab and if I delete the directory I can't add any items.
With this module I can't upgrade to 2.1 ! Please give me a tip how to uninstall it cleanly.

 
vahan1

Joined: 2002-11-01
Posts: 27
Posted: Thu, 2006-06-01 08:24

hey guys,

are there any plans to update this module for new version of G2? I've been using it a lot to copy albums+metadata between different galleries..

Thanks a lot!
-vahan

 
virshu
virshu's picture

Joined: 2003-09-13
Posts: 314
Posted: Sat, 2006-06-03 07:41

by popular demand - I'll update (and probably enhance) this module in the next week. Stay tune

 
virshu
virshu's picture

Joined: 2003-09-13
Posts: 314
Posted: Mon, 2006-06-05 07:06

OK, attached is the new version. I tested it with Gallery 2.2, and it seems to be working. Please note the following:
1. It's still beta; it may corrupt your data (esp. on the import side), burn your hard drive, report your pictures to thought police, etc. Please report any problems in this thread!
2. This module only exports and imports settings from core module. That is, non-core settings (watermark, custom fields, multi-language) are not preserved. Also, import ignores links in most cases. If it is important - vote for this enhancement at sf.net
3. This module isn't well-documented. While in some use cases it is quite natural (export the metadata, copy the resulting files or zip to another server, unzip it there, copy the data files and run import) - I am sure there are other situationa when it is not so obvious. Again - please comment here.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Mon, 2006-06-05 07:39

awesome, thanks virshu!

can you tell us a little about the format in which you keep the exported data?
- is it ascii, binary, ...? if it's binary, is there an issue between mac and x86 (bit-order, MSB to LSB). if it's ascii, is it utf8?
- is it 100% platform independent? can i import DB2 exported data into a MySQL db? does it only rely on the G2 API?
- do I have to worry about the base installation? the root album, guest user, admin user, admin, everybody and users groups are exported as well or just their names?

even if it's not perfect yet, i think this is awesome. we absolutely need this module!

 
virshu
virshu's picture

Joined: 2003-09-13
Posts: 314
Posted: Mon, 2006-06-05 17:33

valiant, thank you for nice words :)
to answer your questions:
- the format is ascii. it's simply serialized PHP objects. Don't know about Mac, but I was moving them between Linux and windows (including Russian characters in the strings) without any problem. I am using PHP's serialize/unserialize functions and php docs are numb on the utf8 issue.
- yes, it is only using G2 APIs. I just looked for SQL calls and noticed that there is one place where I am making such. While it is db-independent (I am using $gallery->search($query, $data) rather than direct SQL statements) - I think that the whole function - fetchDescendantAlbums($parentAlbum) - is now obsolete. IIRC, there is a core function that does same functionality
-- users and groups are not exported; only albums and pictures
-- import tries to match owner by name; if it can't find - current user becomes the owner
-- I am sure permissions need major improvements. In fact, here are my comments in Import:
/*
* TODO Set album owner permissions
* it isn't that trivial, since we are only backing up user IDs
* there is no reason to think that the albums were backed up
* from *this* gallery, and the IDs match between the album and DB
*So, for now I'll take minimalistic approach -
* - admins group can have it all
* - Gallery Administrator as well
* - everybody can view
*/

During export you specify the album to export (and optionally descendants). So, the file structure is the following:
export.dat - points to the id of the "root" album, say 123.
123.dat - has the metadata of the album, including children
album metadata is in the format:
/*
* metadata is an array of at least 7 members:
* 0. album metadata
* 1. Owner name
* 2. Highlight item ID
* 3. view count for the album
* 4. an array of children in the form of id=>viewcount
* 5. Permissions.
* 6. Derivative preferences
* 7. Theme customizations
*/
And then child albums have the same structure, and child photos only have metadata - serialize(GalleryPhotoItem).

I haven't changed the way I serialize photo metadata since I originally wrote the module a year ago; so it may need some improvements; however in my testing it seems to be working fine.

Also, in order to export non-core metadata, I would need to develop a notification that other modules can plug into. Bharat said that it's not that difficult; however (a) I didn't research it and (b) more importantly, it will require modifications in other modules. This way, I can package it in a standalone module that a user can install/uninstall without impacting the rest of the gallery.

HTH... Please let me know if additional clarifications are needed. valiant, again - you are closer to the pulse of the community than I am; so you know better how much interest would there be in such module. If there is, then I would need to have an event (Gallery::export) and other modules (Watermark, colorpack, etc) could export their data as well.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2006-06-06 08:53

ok, sounds good. i guess the character encoding doesn't matter when using php serialization. and both, reader and writer use the same encoding in our case anyway.
i guess we could use reflection or our db map / entity info to do a lot without hardcoding what we actually export.

we don't have foreign keys yet, so we'll still have to hardcode the dependencies.

i think this module / feature would be pretty popular. first as a backup tool, second as a way to import / export / exchange stuff.

why do you need an export event?

 
virshu
virshu's picture

Joined: 2003-09-13
Posts: 314
Posted: Tue, 2006-06-06 18:57
valiant wrote:
why do you need an export event?

I need a hook to allow individual modules to be responsible for serialization (more precisely, to provide data that my module will store). For example, sizelimit would provide dimension and/or filesize information, colorpack will provide colors, etc. My module shouldn't be aware of all the other modules that have persistant data. Currently, it is aware of core module data (which became obvious, since the old version had layout, and new one doesn't); but with the hook even the core could serialize its own data. And I counted almost a dozen other modules that could hook into such event...

 
vahan1

Joined: 2002-11-01
Posts: 27
Posted: Tue, 2006-06-06 20:56

virshu,

Thanks so much for updating the module!! Ochen' Blagodaren :)

I'm running 2.1 at the moment and will wait a bit before upgrading to 2.2. Or until I *need* to use the module.

Thanks again

-vahan

 
virshu
virshu's picture

Joined: 2003-09-13
Posts: 314
Posted: Wed, 2006-06-07 19:13

pameche :)
you can change the version in module.inc - line 45 to: $this->setRequiredCoreApi(array(7, 0)); and line 46, I think, to: $this->setRequiredModuleApi(array(3, 0)); I am not using any 2.2-specific features. Let me know if you run into any trouble...

 
Pierre Palatin

Joined: 2006-07-23
Posts: 1
Posted: Sun, 2006-07-23 19:45

I've just tested it with gallery 2.1.1, works almost fine. There are some needed changes :
* Required API versions, as said in previous posts; "line 45 to: $this->setRequiredCoreApi(array(7, 0)); and line 46, to: $this->setRequiredModuleApi(array(3, 0))"
* Fetch of local server upload directories seems to have changed in g2.2. So you need to change line 86 in ItemAddFromBackup.in to: list ($ret, $param) = GalleryCoreApi::fetchAllPluginParameters('module', 'core'); // ('core' instead of 'itemadd').

And beware of date format mismatch between source gallery and destination gallery, it may affect photo order - otherwise, with same date format, sorting is kept.

With those changes, it works fine. There's no message when data files aren't present (wrong directory as example :) ), so it's a bit hard to guess what's going wrong. However, it's a nice and useful module, thanks a lot :) (it would be really great to have it in mainline gallery in fact)

 
vahan1

Joined: 2002-11-01
Posts: 27
Posted: Mon, 2006-11-27 00:45

I have too tested it with 2.1.1 and it worked beautifully. Thanks for the tips Pierre!

-vahan

 
photo-web.net
photo-web.net's picture

Joined: 2004-12-05
Posts: 14
Posted: Sat, 2006-12-30 23:17

For exporting, it's OK. Can you gives me the procedure to Import metadata ?
Can I use this module to import metadata on a mirror ? (Picture id of are not the same on the master and the mirror but the structure is identically)

 
barleywine

Joined: 2007-04-10
Posts: 24
Posted: Fri, 2007-05-18 04:21

Hello - I need help!

I've run the export - no problem - but just like the question above, I can't find the way to import. could you give a quick, step by step rundown of how to import? I've uploaded the album files to the album directory and that didn't work either; I just get an error message when I try to add the same album. Nowhere do I see a command to run import.
I'm having to recreate my gallery in a different folder on the same server because of an different unanswered question regarding a destroyed database checkout module and would love to use this module to do that. So...where exactly is the command to import? What is the procedure?

Thanks so much,

 
chamb1

Joined: 2004-12-27
Posts: 32
Posted: Wed, 2007-06-20 15:51

Maybe just a quick step-by-step on the import side o things would help - I am having trouble with this part too. it imports the .dir files but cannot find the photos (even though they are under the g2data/albums where they should be)...

 
mindless
mindless's picture

Joined: 2004-01-04
Posts: 8601
Posted: Thu, 2007-07-05 18:10

FYI, current svn / nightly snapshots have a new database backup feature. You can backup while running through the Upgrader and also from Site Admin / Maintenance. You can import a backup from lib/support/index.php

 
Trouble

Joined: 2002-09-11
Posts: 21
Posted: Thu, 2007-08-09 05:45
mindless wrote:
FYI, current svn / nightly snapshots have a new database backup feature. You can backup while running through the Upgrader and also from Site Admin / Maintenance. You can import a backup from lib/support/index.php

Hi!
Would this allow the import of a backup into an existing gallery? (Effectively merging the two galleries and their metadata (including comments etc.))

Many thanks.

 
mindless
mindless's picture

Joined: 2004-01-04
Posts: 8601
Posted: Thu, 2007-08-09 05:53

No, an import cannot merge with existing data.

 
pdeg

Joined: 2007-02-07
Posts: 8
Posted: Tue, 2007-09-11 05:37

Are these backups at the album level.

I have some 25 Gig of photos and not being able to backup to DVD is
a real show stopper....

I want to be able to backup an album (recursively) including the photos and database metadata into a single archive.
And then be able to restore...

The size of my installation precludes a backup of the whole lot...

I need the granularity.

TIA

Peter

 
desiredbard

Joined: 2006-07-05
Posts: 80
Posted: Wed, 2007-11-14 10:08
Quote:
Export and import album metadata
Incompatible plugin!
Core API Required: 6.7 (available: 7.18)
Plugin API Required: 2.0 (available: 3.5)

Any suggestions

 
pdeg

Joined: 2007-02-07
Posts: 8
Posted: Thu, 2007-11-15 03:38

Apologies for my last post .. I did not read the thread properly.

I have installed the import and export module in the latest nightly build ..

It appears to run, but does not seem to include the actual photos in the zip file ?

Any suggests ? I notice some changes required for each rev.

Build specs below

TIA
Pter

Gallery version = 2.2-svn core 1.1.20
PHP version = 5.1.6 apache2handler
Webserver = Apache/2.2.2 (Fedora)
Database = mysql 5.0.27, lock.system=flock
Toolkits = ArchiveUpload, Dcraw, Exif, Getid3, ImageMagick, NetPBM, Thumbnail, SquareThumb, Gd, LinkItemToolkit
Acceleration = full/2700, full/2700
Operating system = Linux curric4182-05 2.6.18-1.2239.fc5 #1 Fri Nov 10 13:04:06 EST 2006 i686
Default theme = matrix
gettext = enabled
Locale = en_GB
Browser = Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9
Rows in GalleryAccessMap table = 113
Rows in GalleryAccessSubscriberMap table = 8794
Rows in GalleryUser table = 10
Rows in GalleryItem table = 8782
Rows in GalleryAlbumItem table = 113
Rows in GalleryCacheMap table = 8674

 
Ka1N

Joined: 2005-11-01
Posts: 24
Posted: Fri, 2008-06-20 23:12

can anyone tell me if this module works with Gallery version = 2.2.4 core 1.2.0.6 ?

I need to move an album from one install to another... any ideas?