Problem with uplading a large number of files

mhoey
mhoey's picture

Joined: 2003-11-16
Posts: 5
Posted: Sun, 2003-11-16 20:10

I've tried to read all of the forums to find an answer to this without luck.

When I am uploading files with the Java client -- I can get some files to go up succcessfully, usually about 15 with an average file size of 350K -- then I always get the message:
----------
You may need to set LimitRequestBody 16777216 and memory_limit = 64M in /etc/php.ini and /etc/httpd/conf.d/php.cond on your Gallery server.
-------------

After the message, I can hit OK -- then hit upload and things start again, but I always get about the same number of files up.

If I have larger files, I can only get a small number updoaded before getting this -- usually 3 with 1.5M files.

I am pretty sure I have made the appropriate changes to the files:

c:\winnt\php.ini:

max_execution_time = 300 ; Maximum execution time of each script, in seconds
max_input_time = 60 ; Maximum amount of time each script may spend parsing

request data
memory_limit = 64M ; Maximum amount of memory a script may consume (8MB)
LimitRequestBody = 16777216

and in my httpd.conf:
LimitRequestBody 16777216

How can I tell if I have made these changes right, or is this message a red herring?

Thanks for any help

Running Gallery on Win2K Server / Apache with further details below.
----
Give us your Gallery/webserver information to get a faster answer.
Get this information from the PHP diagnostic (in the configuration wizard).
Gallery URL (optional but very useful): www.hoeyweb.com:8080/gallery
Gallery version: release on 11/10/2003
Apache version: apache_1.3.29-win32-x86-no_src.exe
PHP version (don't just say PHP 4, please): PHP v4.3.4
Graphics Toolkit: netpbm1.1-gallery1.0-win32.gz and ImageMagick-5.5.7-Q8-windows-dll.exe
Operating system: Win 2K Server
Web browser/version (if applicable): n/a

 
joan
joan's picture

Joined: 2002-10-21
Posts: 3473
Posted: Sun, 2003-11-16 20:38

Moved to GR forum

 
Borsti1

Joined: 2003-11-03
Posts: 5
Posted: Sun, 2003-11-30 19:03

Hi,
did you get help for your problem?
I have exactly the same error message, but I am running gallery on a service provider and cannot change setting in the php.ini.
Do you know how to solve this problem?
Thx
Borsti

 
mhoey
mhoey's picture

Joined: 2003-11-16
Posts: 5
Posted: Sun, 2003-11-30 23:10

Not yet -- I was about to put int the final of 1.41 and see if it went away. I'll post back in the next couple of days to let you know how it went.

 
joel558
joel558's picture

Joined: 2003-04-10
Posts: 93
Posted: Sun, 2003-11-30 23:31

can you upload all those files that you are having a problem with using the web interface? if not, its likely a gallery problem, not gallery remote.

borsti, you can add those values to your .htaccess file, similar to here FAQ Gallery:c.16

 
richardofstvictor

Joined: 2003-12-31
Posts: 2
Posted: Wed, 2003-12-31 02:55

I'm get the exact same error message when uploading multiple images using Gallery Remote.
I'm using Gallery 1.41 on my Dixiesys website. I tried to follow some suggestions I found on this and various galleries.
I can't find /etc/php.ini or /etc/httpd/conf.d/php.conf on my website so I added the following values to /var/www/html/.htaccess and /var/www/html/gallery/.htaccess of my website since I'm not sure which .htaccess I'm supposed to use:
php_value memory_limit 67108864
php_value post_max_size 67108864
php_value upload_max_filesize 67108864

i still get the error message. what am i doing wrong?
thanks for any help anybody can give.

 
Warren
Warren's picture

Joined: 2002-07-24
Posts: 794
Posted: Wed, 2003-12-31 15:37

richardofstvictor, what version of apache are you running if at all?

I'm sure the .htaccess file in question is in your Gallery directory root.

 
PhilippeE

Joined: 2004-01-02
Posts: 4
Posted: Fri, 2004-01-02 04:27

I believe I am having a similar problem.

I can upload no problem via the web interface. I can ftp the my server
and upload no problem big and small pictures.

When I use the remote interface I can upload pictures that are half a meg
or smaller. Tried to increase my .htaccess configuration to accept bigger files

Here is my server configuration

Linux redhat v9 kernel 2.4.21-1.1931.2.274.entsmp #1 SMP Tue Jun 24
Apache 2.0
PHP v4.2.2
Gallery version 1.4.1
Gallery Remote 1.2 Java version 1.4.2_02

...

# apache httpd.conf configuration
# Inserted by cc-gallery to override
<Directory /var/www/html/gallery>
AllowOverride All
</Directory>

...

# gallery .htaccess configuration
php_value post_max_size 5M
php_value upload_max_filesize 5M
php_value magic_quotes_gpc off
php_flag session.bug_compat_warn off

# error log from gallery remote

...

1717764|TRACE|PreviewFra|Starting Z:\IMG_5481.JPG
1718535|TRACE|GalComm2 |Url: /gallery/gallery_remote2.php
1718535|TRACE|GalComm |Sending cookie: PHPSESSID=e26e955e046d24313d9ba52eae51946b; path=/; domain=www.cognoquest.com
1720067|ERROR|GalComm2 |com.gallery.GalleryRemote.GR2Exception: HTTP POST failed (HTTP 500 Internal Server Error )
at com.gallery.GalleryRemote.GalleryComm2$GalleryTask.requestResponse(GalleryComm2.java:313)
at com.gallery.GalleryRemote.GalleryComm2$UploadTask.uploadPicture(GalleryComm2.java:592)
at com.gallery.GalleryRemote.GalleryComm2$UploadTask.runTask(GalleryComm2.java:510)
at com.gallery.GalleryRemote.GalleryComm2$GalleryTask.run(GalleryComm2.java:237)
at java.lang.Thread.run(Thread.java:534)

1720067|INFO |GalComm2 |Error: HTTP POST failed (HTTP 500 Internal Server Error )
1720067|ERROR|UploadProg|Bad level
1720067|ERROR|UploadProg|java.lang.Exception: Dump stack
at com.gallery.GalleryRemote.Log.logStack(Log.java:129)
at com.gallery.GalleryRemote.UploadProgress.checkLevel(UploadProgress.java:194)
at com.gallery.GalleryRemote.UploadProgress.updateProgressStatus(UploadProgress.java:122)
at com.gallery.GalleryRemote.GalleryComm2.status(GalleryComm2.java:185)
at com.gallery.GalleryRemote.GalleryComm2.error(GalleryComm2.java:189)
at com.gallery.GalleryRemote.GalleryComm2$UploadTask.uploadPicture(GalleryComm2.java:605)
at com.gallery.GalleryRemote.GalleryComm2$UploadTask.runTask(GalleryComm2.java:510)
at com.gallery.GalleryRemote.GalleryComm2$GalleryTask.run(GalleryComm2.java:237)
at java.lang.Thread.run(Thread.java:534)

1722020|TRACE|ImageUtils|Time: 4246 - Avg: 2972
1722020|TRACE|PreviewFra|Adding to cache: Z:\IMG_5481.JPG
1722020|TRACE|PreviewFra|touch Z:\IMG_5481.JPG
1722020|TRACE|PreviewFra|2694280 - 10899456
1722020|TRACE|PreviewFra|2693432 - 10899456
1722020|TRACE|PreviewFra|Ending
9103331|INFO |MainFrame |Shutting down GR
9103401|INFO |MainFrame |Shutting down log

Thank you for a great product and the help
PhilippeE

 
richardofstvictor

Joined: 2003-12-31
Posts: 2
Posted: Fri, 2004-01-02 09:12
Quote:
richardofstvictor , what version of apache are you running if at all?

warren,

My version of Apache is Apache/1.3.22
PHP 4.2.2
Gallery ver. 1.4.1
Gallery Remote ver. 1.2.1-b6

Thanks for any help.

 
PhilippeE

Joined: 2004-01-02
Posts: 4
Posted: Sat, 2004-01-03 04:35

Sorry richardofstvictor. I was not very precise with my version of apache

Apache: httpd-2.0.40-21.5

Also I tried:
gallery_remote_1.2.1-b6 new jar file on both NT4 and WINXP, same problem.

Thanks

 
paour
paour's picture

Joined: 2002-08-14
Posts: 1479
Posted: Tue, 2004-01-06 16:11

PhilippeE, are you sure you can upload a ZIP of many images via the web interface? This error sounds like a Web server throttling one way or another.

 
diomark
diomark's picture

Joined: 2003-03-14
Posts: 90
Posted: Wed, 2004-01-07 10:52

For what it's worth, I've seen the same error when uploading large files.. seems to be random for me though (sometimes after 30 images, sometimes after 50 images, sometimes never..) My php memory setting and max upload size are actually set to 128mb mem/100mb file size..
-m

 
PhilippeE

Joined: 2004-01-02
Posts: 4
Posted: Thu, 2004-01-08 03:01

[paour]
PhilippeE , are you sure you can upload a ZIP of many images via the web interface? This error sounds like a Web server throttling one way or another.

[PhilippeE]
This is not an upload of a zip file. I copied the same files to my server and I used the following upload directory feature:
‘Or, upload any images found at this location.The location can either be a URL or a directory on the server.’

By the way this the only way I can upload my files: by using an URL. If I try to upload one photo at a time from a local drive it does not work small or large photos.

(small message in French not related to this problem)
Paour je viens de remarquer que tu m’envoies ce message de la France. Je n’ai pas souvent l’occasion de pouvoir écrire en français alors j’en profite. Bonne et heureuse année. Hélas il faudra continuer en anglais pour le bien de la communauté.

I agree it is probably some kind of web server configuration problem. Any suggestion on what I should try next. For sure I will try [diomark] configuration suggestions.

 
paour
paour's picture

Joined: 2002-08-14
Posts: 1479
Posted: Thu, 2004-01-08 08:42

PhilippeE, merci pour tes voeux. Bonne annee a toi aussi.

Because most upload problems reported in the forums are actually problems on the server side of things, we can only support in the Gallery Remote forum people who have tried and succeeded in uploading with the form where you can either upload a bunch of local images or a local zip. Uploading through a URL doesn't use the same mechnisms, and as such it doesn't tell us much in terms of what GR needs from the server.

If diomark's suggestions don't work, I could move the thread to the Gallery Troubleshooting forum.

 
mhoey
mhoey's picture

Joined: 2003-11-16
Posts: 5
Posted: Fri, 2004-01-09 19:50

just thought i would put an update in from the original post -- I upgraded to the latest remote today -- I have similar behavior, but a new error message. I get about 50 large image files up ( 1.5mb each ) then I get the following error:

Error: HTTP POST failed (HTTP 413 Request Entity Too large )

The other thing that I noticed is that the first time it may happen at 50, then it will happen at 20 and so on down. Its almost as if havimg more images in a single directory is exhasiting some internal resource on the webserver. Not sure if that helps any. I suspect this is really a server problem that is bubbling up from GR, so this may be the wrong forum.

 
mhoey
mhoey's picture

Joined: 2003-11-16
Posts: 5
Posted: Sat, 2004-01-10 14:26

Dug a little deeper and realized that the message I was getting was in fact correct. I had dropped a quicktime file in the mix which was 18Mg -- I am happy to report that since upgrading to the latest GR client -- everything is working smoothly. I can now upload hundreds of 1.5mb images with no issues!

Thanks to all that helped me get it working -- I've been looking for a product like this for year.

Mike

 
PhilippeE

Joined: 2004-01-02
Posts: 4
Posted: Sun, 2004-01-11 19:18

My turn for an update.
If can now upload photo via the browser and GR (moved the albums directory back to its default/original position).
I was still limited to less than half a Meg of upload (same error HTTP POST failed (HTTP 413 Request Entity Too large).
I had to do a few configurations changes:

Configuration for: /etc/httpd/conf.d/php.conf
LimitRequestBody 0 ; unlimited size

Configuration for: /etc/php.ini
max_execution_time = 240 ; maximum time of execution of each script, in seconds
max_amount_memory = 64M ; maximum amount of memory a script may consume (64 Meg)
post_max_size = 8M ; maximum size of POST data that php will accept (8 Meg)
upload_max_filesize = 5M ; maximum size for uploaded files (5 Meg)

The .htaccess will override post_max_size and upload_max_filesize if you have the following configuration in your etc/httpd/conf/httpd.conf file:
<Directory /var/www/html/gallery>
AllowOverride All
</Directory>

Configuration for: /var/www/html/gallery/.htaccess
php_value post_max_size 67108864 ; maximum size of POST data that php will accept (does not work with 64M had to put in bytes)
php_value upload_max_size 41943040 ; maximum size for uploaded files (does not work with 40M had to put in bytes)

Gallery following file may have to be modified for larger uploads:
/var/www/html/gallery/add_photos.php
<input type="hidden" name="max_file_size" value="10000000">

Here are some of the links I found to trace this one:
http://www.clarkconnect.com/forums/showflat.php?Cat=&Board=fun&Number=47579&Forum=All_Forums&Words=gallery%20&Match=And&Searchpage=0&Limit=25&Old=6months&Main=47579&Search=true#Post47579

http://gallery.menalto.com/index.php?name=PNphpBB2&file=viewtopic&t=7419

Big Thanks to ALL for your help,
Philippe