Module: ARTFU: Auto rename filename, title, url - automatically on upload
|
jnash
Joined: 2004-08-02
Posts: 814 |
Posted: Mon, 2012-05-14 18:31
|
|
RELEASE EDIT: The ARTFU Module is now up at the codex page: http://codex.gallery2.org/Gallery3:Modules:ARTFU So, I was working on a new module to automatically rename the filename, url, and title (selectively of course) when uploaded, however, I find a problem when sending up multiple images at once... Here is the thought process on this... 1) Enable user to select what to automatically rename (title, url, filename) 2) Give option of rename to: 3) Also allow a prefix and/or suffix * IE: prefix1, prefix2, prefix3, etc... -OR- 1suffix, 2suffix, 3suffix, etc. -OR- pre1suf pre2suf pre3suf... I've got everything working (via the item_created event), except when uploading multiple images via the 'add photos' option I'm not sure if there is a way around this or not... any suggestions? [hopefully this is clear enough] Also, I've coded in renames in the event of duplicates for the increment option (say a user deletes an image later, then on next addition, the incremented count could duplicate a previous uploaded (last) image...) as well (like item->slug in models/item.php) This is not a pretty method, as then the items show as: item1 - item2 - item3 - item3.1234115 - item4 - item5 etc. So, with all this being said, I'm not sure if this will work... |
|
Posts: 76
Cool idea
Posts: 27300
A race condition like this would best be answered by Bharat.
Ping him on IRC.
Dave
_____________________________________________
Blog & G2 || floridave - Gallery Team
Posts: 814
Okay, figured out this race condition only exists when simultaneous_upload_limit is set above 1.
I added some debugs into the function item_created($item) to log each $item->id and I can see that it is missing some during the simultaneous uploads... also, in some instances, I can see it generate an IO 2038 error... This suggests to me that trying to do too much with the DB during upload is a no-go.
setting the simultaneous_upload_limit to 1 seems to avoid this race...
So I think I'll continue, but code in that this var must be set to 1 for the module to function appropriately.
Posts: 27300
Your module could do a check to see if that is set and put up a warning.
error #2038 is a generic error and there may be several reasons, 2038 errors are the default error from the up-loader. I get it occasionally due to a poor internet connection.
What we need now is a cool html5 uploader. Next project?
Dave
_____________________________________________
Blog & G2 || floridave - Gallery Team
Posts: 814
Yes, I got the code to generate an error actually if not set to 1... as a warning can be ignored
It appears as if there is a lock or something placed on the DB during multi-uploads that can cause an occasional failure if multiple read/writes are encountered.
I think I have things working well though... now just to build the admin side of the house.
RE: HTML5 uploader - wow how I wish!
Posts: 814
I really can't think of a way to do auto-increment within an album. There are way too many potentials for naming conflicts, as one can delete, rename or move images. It'd be near to impossible to account for this automatically. I
methods thought of so far that fail
1) count the images within and album and increment - fails because of potentially deleted items... Image1 Image2 Image4 Image4 Image5 (note that image3 was previously deleted by user)
2) account for the incremented value in another table lookup - IE: album has a value that is incremented everytime an item is added to it. - fails again if someone happens to rename an item to a higher value - I know stupid, but it can happen - and to account for this is resource intensive with DB queries...
3) query the database for count by getting the highest numeric value, and then incrementing... labor intensive? Not sure, I could try this one... (but what if previously added images have digits in the name before activating the module.. IE: DSC_0134.JPG would generate the next count as '135' which might be okay... but querying this and then stripping to digits, then ordering by value, then incrementing may be greedy as well
4) ?
5) Profit?
Suggestions are welcome... For now, I think using the item->id value is best, but unfortunately, the numbering method may not be sequential if you have more than one album, as the ID is auto-incremented globally across the entire gallery.
Posts: 814
Well, I've made some changes that make this module somewhat useful and safer. I don't think I'll be releasing it, as things become ugly (just visually in the naming) if folks move/delete/rename items on their own.
What I've done for the incremental setting is on initialization, count the items in the album, and store this in a DB table... the each subsequent item uploaded increments from this count.
So, it works, and it's nice. But it is unique enough not to really release at this point.
If anyone wants to test it out, let me know, and I'll link it for you.
James
Posts: 814
Screeny: (and latest zip attached)
[img]http://codex.gallery2.org/images/d/d2/Artfu.jpg[/img]
Posts: 76
Cool stuff! That's awesome, and btw: really helpful for many users!!! I hope you get the help to fix the last issues.
Posts: 814
Well, feedback thus far has been good, nothing standing in the way of running the module. With the fix up's in place on the incremental functions (IE: count up when added, for lack of a better explanation, mysql's auto-increment function)
So, with this being said, I'm taking the development tag off this thread...
DAVE: If you want, go ahead and create a codex blank for me.
Posts: 27300
Done: http://codex.gallery2.org/Gallery3:Modules:ARTFU
Dave
_____________________________________________
Blog & G2 || floridave - Gallery Team
Posts: 814
codex page updated... thanks!
Posts: 76
I don't understand this quite right, I think... If I'd like to rename (e.g with mrtfu), to move or delete a picture or an album I have to deactivate this plugin first?
Posts: 76
Hmm... I have trouble uploading a picture... after activating the module and activate the automatic-renaming to "id" and "also url" there appears an error while uploading (and shows me an link to http://codex.gallery2.org/Gallery3:Troubleshooting:Uploading ). After pressing the cancel button, and refreshing the page, the new pic appears, but (without renaming).
Posts: 814
No, you're fine to do that. The fixes in place prevent future issues. You don't need to disable.
Posts: 814
I need a better description, or log file, or full error.
Posts: 814
I've tried to reproduce, but can't... Perhaps it's a resource issue on your host? The module does quite a bit of DB work during the upload.
Posts: 76
1. There is no error, I could read in the server-logs...
2. I've attached the error-message, that appears (but I think you won't find a hint):
[img]http://gallery.menalto.com/files/Unbenannt-1.jpg[/img]
3. gallery 3 logfile:
Posts: 814
Rockbär: go in to your admin->settings->advanced, and let me know what you have for each 'artfu' setting in there.
(There should be about 11 of them.)
A screen shot cropped to those variables would be best.
It looks like your item->slug is being set wrong... but I can't find an issue in the code, unless one of these var's was set wrong or not validated right.
Posts: 76
Here it comes:
[img]http://gallery.menalto.com/files/advanced_settings_artfu.jpg[/img]
Posts: 814
Hmmm. That looks fine to me. I'll dig around in the code a bit and see if anything sticks out. I may try adding some debugs and have you try that as well. But I can't see anything wrong off the bat.
Posts: 76
Maybe it's a conflict with other plugins?
Posts: 814
Perhaps, unfortunately I'm at a loss...
You could try disabling other modules to see.
Posts: 76
I won't ;-) Cause some will lost completely their settings. In some cases it's horrible :-D (I've already did it a few weeks before... and that was not a quite good idea)
But at least a list of my modules I'm using (screenshot of the ftp-folder):
[img]http://gallery.menalto.com/files/list.jpg[/img]
Posts: 814
Does the error happen on every try?
Posts: 76
yep, without any exceptions... every time.
Posts: 814
If I could get access to your server I might be able to debug, but without that, I'm at a loss, there is no requirement for slug length (other than 0-255) which looks like where the error is, it appears as if the slug (URL) string is not being populated.
Wait, before that:
You could try adding these lines in your modules/artfu/helpers/artfu.php file right before $itemrec->save(); line at the bottom...
message::success("DEBUG: FIRSTPART: ".$firstpart);
message::success("DEBUG: Slug: ".$itemrec->slug);
message::success("DEBUG: Name: ".$itemrec->name);
message::success("DEBUG: Title: ".$itemrec->title);
When it fails, hit DONE or CANCEL and see what shows up in green at the top and report back here.
James
Posts: 814
I've made a small change to the module as well and updated so that if there is a failure, such as this, it doesn't rename, but still succeeds with the upload.
This won't fix your problem, but hopefully the debug lines will confirm there is one, and then we can dig further.
Posts: 76
Take a look:
The images was saved, but no changes (like without the plugin)...
Posts: 814
did you grab and install the latest, or is this debug from the old one? (All the details look good in the debug, and thus should have saved out)
Posts: 76
no, it doesn't succeed with v3 :-/
it messes up: an error message appears, I have no choice (pressing cancel-button e.g. won't have an effect. I'd have to press the x-close-button at the top and have to refresh the page to see the uploaded pic).
Posts: 76
With the newest v3:
Posts: 814
Okay, I'm testing now, and see that it is failing of slug length and title length (if set to rename both) Unfortunately, I'm not sure why though, as the item model shows 0-255 as being valid...
Will report back once I find something.
Posts: 814
Well, something changed, and I can't figure it out for the life of me... digging and digging but am stuck for the moment...
This is what is happening:
2012-05-31 14:54:48 -04:00 --- error: Validation test: SLUG: '27126' 2012-05-31 14:54:48 -04:00 --- error: Validation test: TITLE: '27126' 2012-05-31 14:54:48 -04:00 --- error: Validation errors: Array ( [slug] => length [title] => length )I'm debugging these to the log, and yet, even though slug and title are populated and have a valid 'length' - I'm seeing them fail on validation with length as the problem...
So I'll dig further.
Posts: 814
Doh, I got it... $item->id is an int, and length on an int seems to fail, so I had to convert the rename variable to a string first... all solved and codex updated to ver4.
Thanks Rockbär !
Try again after updating to latest version...
Posts: 76
I've tried the latest version, but still the same problem :-/
Posts: 814
v4 ? If so, can you post the error log again. There should be more in there this time.
Posts: 76
Here with v5
Posts: 814
According to that, you don't have v5 installed... I log more to the file in that version and v4.
Double check your files. (make sure you didn't unzip inside the artfu folder for example)
Posts: 76
Really? O,o I've downloaded it twice from here: http://gallery.menalto.com/files/artfu.zip
Posts: 76
That's the logfile (/var/logs/2012-06-02.log.php) after installing http://gallery.menalto.com/files/artfu.zip
Posts: 814
Try grabbing from here... that's the wrong link...
http://codex.gallery2.org/Gallery3:Modules:ARTFU
Edit: Updated codex to link to right file... link in codex was wrong too
Posts: 76
Cooooooool, now it's working! Awesome work, thank you very much.
Just one question: "also rename the Title:" ...but after the "title:" nothing else appears... is this still in beta-progress?
Posts: 814
nope, shouldn't be a ":" there... fixed and re-uploaded... (It's the same as also rename the URL) If checked renames URL/Title)
Thanks
Posts: 10
Long time hobbyist site builder here, first time poster on this forum.
jnash, love having clean URLs and album/pic names so was very keen to d/l ARTFU. Loving it so far, except for one tiny issue...
It successfully renames the first uploaded file, but all the rest of the upload fails on the rename. The error given is generic and nothing specific is shown as far as error codes. I have tried it maybe ten different times thinking it might be a temporary or random DB glitch but every time it consistently produces the same results.
Here are my ARTFU settings....
artfu incr_length 6
artfu incr_start 310962
artfu inc_title 1
artfu inc_url 1
artfu pad_zero empty
artfu to_incr 1
artfu to_item_id empty
artfu to_prefix 1
artfu to_prefix_val fb
artfu to_suffix empty
artfu to_suffix_val suffix
Simultaneous upload is set to "1".
Thanks in advance.
Any ideas? Need more info ?
Posts: 814
Anything in your gallery log files? I added some logging to catch any errors generated in the last version. The only thing that comes to mind is the simultaneous upload, as that was causing issues, but I see yours is set to 1. What method are you using to upload?
Posts: 10
Here's what I think are probably the highlights of the log file.
LOG ERRORS
- a bunch of these... error: date.timezone setting not detected in /home/dir/public_html/gallery/php.ini falling back to UTC. Consult http://php.net/manual/function.get-cfg-var.php for help.
- error: Can't find css file: themeroller/ui.base.css
- error: Can't find script file: gallery.form.js
- error: @todo UNKNOWN_THEME_FUNCTION: pager
- 19 entries, it looks like, related to "pager"
2012-0- error: date.timezone setting not detected in /home/etc
The number is the same as the starting ARTFU name I set so I think these are important.
(
[name] => conflict
[slug] => conflict
)
2012-08-06 21:44:03 +00:00 --- error: Validation test: [string] SLUG: '10962'
2012-08-06 21:44:03 +00:00 --- error: Validation test: [string] TITLE: '10962'
2012-08-06 21:44:03 +00:00 --- error: Validation errors: Array
(
[name] => conflict
[slug] => conflict
)
EDIT: Oh yea! Uploading using the add pics link on the album.
Posts: 814
According to the error, you have an image with that name already... which would result in repeated errors because the counter would never increment. I'd double check your numbers and ensure no other images have this name (number)
Posts: 10
Hmmm I think it was happening even with an empty destination folder but I'll be able to double check in a couple hours.
Thanks.
Posts: 814
tforward: If you have insight into your database, see if you have a table named 'artfu_counts' under your gallery DB...
I'm wondering if perhaps the table didn't get generated, or got removed, or something... In reviewing the code, this could cause a counter increment failure, resulting in the conflict error on the second image uploaded. (since you indicated the first passed, but second failed)
However, I'm doubtful this is the case, as I think Kohana would die out if the table didn't exist and the call to save the counter increment failed.
Also, please provide your artfu settings again, so I can see if I can reproduce this. (the settings you provided earlier, don't match the error results you provided last) - Thanks!