Improving the "Add Items" from local server
|
djarnis
Joined: 2007-01-02
Posts: 9 |
Posted: Tue, 2007-01-02 10:56
|
|
Hello I'm am desperately looking into migrating from Coppermine to Gallery2 however I run in to some troubles no matter how I try to import my pictures. I have just over 10.000 pictures distributed over 52 albums, so importing all the pictures manually is kinda out of the question I have tried to use the new "2.2 RC1"-release along with the "cmmigrate v0.5.0"-module, but I get some kind of time-out, which I assume is the same as the one I am about to describe. I just wanted to point out, that I have tried to migrate the pictures that way. When I try to add items from "local server" from a Coppermine album which contains many pictures (which I assume is the fastest and best way), then the page loads for about 60 seconds and then stops with a blank screen. I assume (lots of assuming here So my main question (or improvement suggestion) is: Quote:
Wouldn't it be possible to improve the import part of G2, so you can choose to automatically reload the import process each xx picture? To me it looks like the import process, tries to import all xx pictures in the same batch, but if you can choose to import i.e. 20 pictures at a time, so it imports picture 1-20, then reloads and import picture 21-40, etc. etc., then the max execution timeout shouldn't occure. |
|
.
) that is because of an execution time out. It is possible for me to import fewer pictures at a time, but again... breaking 10K+ pictures into batches of 20ish pictures at a time will still result in tons of manual work. I have tried to move around with NetPMB, GD and ImageMagick, but when I deal with many pictures, then I still get the same blank screen 
Posts: 32509
the cmmigrate hasn't been officially released yet. it needs some polishing and we need feedback from users like you to ensure that it's reliable enough.
we'd need to know where exactly it stopped.
probably, it would be necessary to add more debugging information to the module and then we could just enable logged debugging to get the information we need.
the problem with "add items from the local server" is that it doesn't have a progress-bar yet. and while we do some magic in the background to prevent PHP to stop processing, there's still the webserver and the connection to your browser that could be aborted. maybe your webserver has a very low timeout.
personally, i've successfully added about 5000 items with "from local server" at once. it really depends on the server and we still need to add this progress bar to make it a little more reliable.
@blank screens:
could also be memory related. you should definitely look at the weberver error logs:
FAQ: I get only a blank screen when trying to view my Gallery, what's wrong?
Some few web-applications do this, reloading a page for long-running tasks. In Gallery, we're doing something else. We're dynamically extending the timeouts and we're trying to keep the connection to the client (browser) alive by sending periodically some updates to the browser (using a progress bar).
There's no progress bar yet, but it's on our todo list.
Posts: 9
The cmmigrate module was an additional method I tried, just to see if it was possible. This thread wasn't ment to be focusing on that exact module, but I will start another thread with some debugging info from that module.
I will try to use cmmigrate to migrate all my pictures and then report all errors I run into.
I wasn't aware of the fact that you already did some "magic" in the background, so I guess that batch-method isn't gonna help me out here
Now I have tried to time some imports and it does indeed seem to go further than me previously mentioned 60 seconds.
I am using a shared host, so I don't really think there is much I can do about this
.
It's quite fancy and I love it bigtime... and now even bigger, since it's doing the "magic" thing to avoid timeouts
.
Posts: 24
I'm on a shared host too, and I can get access to my logs. You can even set up your log files and all in .htaccess and all. I suggest contacting the host and asking if you can have access to your PHP logs and all. It's worth a shot...
Posts: 9
My host told me, that they wont be able to give me access to the log files because they run a clustered server structure.
Are you saying, that I can add some lines to .htaccess and then the error logs can be saved where I want?
Posts: 32509
Update: We've just added a progress-bar and many improvements under the hood to make batch-adding items more reliable and user-friendly.
Upgrade to the next nightly snapshot of G2 including the itemadd module or wait for G2.3 (still months away) to get the improvements.
--------------
Documentation: Support / Troubleshooting | Installation, Upgrade, Configuration and Usage
Posts: 22
i had some trouble adding items via local server, had to modify the (database timeout and default socket timeout) (possibly only one but i did both, just for initial upload, then change it back, unless you upload a lot of pix at once a lot) in php.ini and the timeout on my iis6 server. that seemed to have fixed my problem but after some more tests, it didn't, then i looked up and searched for other timeouts, found cgitimeout, but you must edit metabase.xml (after enabling direct editing via properties of iis) changed the default 300 to a higher number to test and that fixed it for sure ...unless i post again.
Posts: 22
one last note, you can optionally uncheck create thumbnails now, and that speeds up the upload script by a WHOLE LOT, if you uploading a couple hundred photos.
Posts: 16504
This thread is over 2 years old and is discussing an old version of G2. All new development efforts are being focused on G3. Improvements for G2 most likely are not going to happen. It will still be supported for awhile and you can continue to use it for as long as you wish, but new features and enhancements are not going to happen unless someone wants to keep coding this beast.
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here