Boost the import of a large G2 gallery ?

bvossey

Joined: 2007-08-19
Posts: 17
Posted: Fri, 2009-07-10 12:35

Hi,

I just want to try the new G3 bêta with the datas of my G2 gallery...
Install very easy ok.
Configure the G2 import module ok.

Launch the import of my G2... several hours !
Indeed, my G2 gallery is quite impressive :

37 users
1472 albums
25497 photos

The import is in progress right now since 3hours... and it's at 3509 / 25497 !

Is it possible to add an import in command line ? (execute by an admin on shell script for huge gallery ?)

Thanks for your product,
I will post my G3 feedback as soon the import is finish ;)

Regards,

Benjamin
 
floridave
floridave's picture

Joined: 2003-12-22
Posts: 27300
Posted: Fri, 2009-07-10 13:36

You can stop the import and you can resume the import from the admin pages.

Dave
_____________________________________________
Blog & G2 || floridave - Gallery Team
 
bvossey

Joined: 2007-08-19
Posts: 17
Posted: Fri, 2009-07-10 13:48

Indeed, but the import duration will be the same no ?
 
floridave
floridave's picture

Joined: 2003-12-22
Posts: 27300
Posted: Fri, 2009-07-10 22:30

Yes it will. There is a lot going on in the background and we have to make it work.

Dave
_____________________________________________
Blog & G2 || floridave - Gallery Team
 
bharat
bharat's picture

Joined: 2002-05-21
Posts: 7994
Posted: Sun, 2009-07-12 16:18

Unfortunately, there's no magic bullet here (or at least none that I can see so far). Gallery 3 is wholly different (so that it can be wholly better) and that requires us to go through a conversion process. The thing that makes it the slowest is running commands to regenerate your images.. if you use the same thumb and intermediate image sizes, that'll speed things up tremendously. Also, turn off the search and exif modules temporarily, that'll make things faster
---
Problems? Check gallery3/var/logs
bugs/feature req's | upgrade to the latest code | use git | help! vote!
 
bvossey

Joined: 2007-08-19
Posts: 17
Posted: Sun, 2009-07-12 16:51

Thanks for your tips bharat.
I understand that you have a lot of operations to do in the import process. Maybe the import via an admin script (.sh) and launch direclty on the server with an SSH console could be quicker no ?

Thanks again for your product and for the time you read (and reply!) all messages.
 
bharat
bharat's picture

Joined: 2002-05-21
Posts: 7994
Posted: Mon, 2009-07-13 02:10

That'll be a little quicker, but not that much. The only difference would be in removing the overhead of communicating between the browser and server to update on status and make the next request. You can simulate what that's like by editing modules/g2_import/helpers/g2_import_task.php and changing the timeout value on line 77 from 1.5 seconds to 20 seconds. You can set it to a higher value if you want, but you may run into timeout issues on your server.
---
Problems? Check gallery3/var/logs
bugs/feature req's | upgrade to the latest code | use git | help! vote!
 
bvossey

Joined: 2007-08-19
Posts: 17
Posted: Tue, 2009-07-21 20:22

I will change the value of the timeout.
One another very good point is that with command line, i'm not longer concerned by browser problem, crash, logout or anything else ; I just launch the script on the server and several hours it's done.
 
bharat
bharat's picture

Joined: 2002-05-21
Posts: 7994
Posted: Thu, 2009-07-23 17:13

Sounds good to me. Want to write the code for it?
---
Problems? Check gallery3/var/logs
bugs/feature req's | upgrade to the latest code | use git | help! vote!
 
bvossey

Joined: 2007-08-19
Posts: 17
Posted: Thu, 2009-07-23 22:30

I'm sorry but I don't have the skills to do a such script.
I just wrote the idea...