Attempting to automate ItemAddFromServer calls with Cron job

Blood_Hawk

Joined: 2008-08-26
Posts: 5
Posted: Tue, 2008-08-26 22:21

I've recently been coding a website (my first real venture into PHP) that embeds a Gallery2 instance. The embedding of the gallery is working OK (slight problem with following the URLs, but that's not what I'm here for), but I wanted to automate/simplify the maintenance of the site for another set of admins as much as possible. To this end, a lot of the site is driven from flat files/databases, I thought it would also be nice if they could quickly/easily upload picutres into the gallery (making sure the upload directory is secure and requires passwords, etc to FTP into).

To this end, I've been writing a PHP script that will then operate through a Cron job on the server every 10 minutes or so. The basic outline of the script is to check a (upload) directory for files, if any exist, move them to a staging directory and add them to the gallery before being deleted. The album I'm adding to is always the same and only files that are to be loaded into the one album will be uploaded to this directory, so it's all good. The problem I'm having is in trying to setup the Gallery classes to correctly call ItemAddFromServer::addItem (or ::addDirectory might be used instead). Basically, I'm getting "function called on null Object" errors from one or more locations, but can't figure out how I'm supposed to instantiate the needed classes to make the code work.

Start of script:

require_once('<my_web_server>/modules/core/classes/Gallery.class');
    require_once('<my_web_server>/modules/core/classes/GalleryDataCache.class');
    require_once('<my_web_server>/modules/core/classes/GalleryCoreApi.class');
    require_once('<my_web_server>/modules/core/classes/GalleryUtilities.class');
    require_once('<my_web_server>/modules/core/classes/GalleryController.class');
    require_once('<my_web_server>/modules/core/classes/GalleryView.class');
    require_once('<my_web_server>/modules/core/ItemAdd.inc');
    require_once('<my_web_server>/modules/itemadd/ItemAddFromServer.inc');
    
    $GLOBALS['gallery'] =& new Gallery();
    $gallery =& $GLOBALS['gallery'];
    @include(GALLERY_CONFIG_DIR . '/config.php');

I then proceed to do the above directory/file checks, finally reaching a foreach loop that repeatedly calls:

$ret = ItemAddFromServer::addItem($parent_id, $dir . $file, false, $text_fields, $status, $error);

where $parent_id is the album id; $dir the staging directory; $file the file to be added; $text_fields, $status and $error are empty arrays (not currently building a set of meta data for adding the files, happy with the basic/empty titles, etc until I've got the main code working).

Any help would be great!


Gallery version = 2.2.5 core 1.2.0.7
PHP version = 5.2.6 cgi
Webserver = Apache/2.2.9 (Unix) mod_ssl/2.2.9 OpenSSL/0.9.8b mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635
Database = mysqlt 5.0.51a-community-log, lock.system=flock
Toolkits = ArchiveUpload, Dcraw, Exif, Ffmpeg, Getid3, LinkItemToolkit, NetPBM, SquareThumb, Thumbnail, Gd, ImageMagick
Acceleration = none, none
Operating system = Linux hp19.hostpapa.com 2.6.18-92.1.6.el5 #1 SMP Wed Jun 25 13:45:47 EDT 2008 x86_64
Browser = Firefox 3.0

 
alecmyers

Joined: 2006-08-01
Posts: 4342
Posted: Tue, 2008-08-26 22:44

If my mind isn't playing tricks on me, somewhere in the Codex is a sample skeleton script that includes all the initialisation code for exactly what you are trying to do - but I can't find it right now. If someone doesn't post a link to it I'll have a further search tomorrow.

 
nivekiam
nivekiam's picture

Joined: 2002-12-10
Posts: 16504
Posted: Wed, 2008-08-27 02:47

I don't think your mind is playing trick. Possibly this: http://codex.gallery2.org/Gallery2:How_can_I_call_a_View_or_Controller_from_the_Command_Line

or this: http://codex.gallery2.org/Gallery2:How_To_Write_Scripts

I remember something from a few years ago too. I think there might be more references in the forums as well but that could be where the codex entries were created from.
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here

 
Blood_Hawk

Joined: 2008-08-26
Posts: 5
Posted: Wed, 2008-08-27 06:10

Cheers guys, that's pretty much what I was looking for I think, I just completely missed it yesterday (d'oh)! I'll have a look through the threads and give what I learn a try. I'll update this thread with a couple of details if it does/doesn't work!

 
Blood_Hawk

Joined: 2008-08-26
Posts: 5
Posted: Wed, 2008-08-27 20:55

Still having problems unfortunately. I seem to be missing something (no doubt obvious when I figure out what it is). I've updated my code to be the following:

require_once($gallery_home_dir . '/embed.php');

$ret = GalleryEmbed::init(array('fullInit' => true));
check($ret);

list($ret, $itemAdd) = GalleryCoreApi::newFactoryInstance( 'ItemAddPlugin', 'ItemAddFromServer' );
check($ret);

<<pseudo-code: check a directory for files, loop through files and, one at a time, call:>>
$ret = $itemAdd->addItem($parent_id, $add_path . $uploaded, false, $text_fields, $status, $error);
check($ret);

<<finishing with>>
$ret = GalleryEmbed::done();
check($ret);

I've tried the above and also using ItemAddFromServer::addItem(...) instead of using $itemAdd, but I keep getting "functions called on null object" errors (the $itemAdd var in the code above is always returned as null).

As I say, I'm guessing it's something simple, but I just can't seem to get my head around the way in which things are supposed to be instantiated/used with this G2 coding (nor can I find any examples that have helped much apart from the two URLs posted above that gave me the ::init and ::done calls... /sigh, get there eventually!)

Should I be using code that sets up and sends a request (as given in the first of the above example URLs?

$session =& $gallery->getSession();
$authToken = $session->getAuthToken();
GalleryUtilities::putRequestVariable('authToken', $authToken);

/* Here we could either call any API mehtods or declare the controller request params */
GalleryUtilities::putRequestVariable('controller', 'core.AdminMaintenance');
GalleryUtilities::putRequestVariable('form[action][runTask]', 1);
GalleryUtilities::putRequestVariable('taskId', 'BuildDerivativesTask');

Changing core.AdminMaintenance to core.ItemAdd; and BuildDerivatesTask to ItemAddFromServer? Of course, then I've got to get the file location and other parameters into the request somehow.

 
alecmyers

Joined: 2006-08-01
Posts: 4342
Posted: Wed, 2008-08-27 21:55

I think your first approach is likely to be more successful. (If you try to send a request to Gallery, you're likely to end up having to write the controller to which you're sending the request too, I suspect.)

I can't see why your $itemAdd is coming back null - but I'm no expert. You don't have to use the factory though - it's convenient for plugins where the code instancing the plugin doesn't know which/what plugins implement a particular interface at the time of writing but in your case you know exactly which class you want to instance and where the code lives so I guess you could just

GalleryCoreApi::requireOnce('modules/itemadd/ItemAddFromServer.inc');
$myItemAdd = new ItemAddFromServer;

Oh, and looking at the code for GalleryFactoryHelper_simple::newInstance(...), there are two conditions that return a null object without error, and both throw a debug message if you have debugging turned on. See for yourself, in GalleryFactoryHelper_simple.class.

Final thought: you do actually have the itemadd module installed and active, don't you? Otherwise the plugin won't be installed in the registry, and yes, you'll get a null result.

 
Blood_Hawk

Joined: 2008-08-26
Posts: 5
Posted: Thu, 2008-08-28 20:28

Got it working (at least when I call the script via my browser, waiting to see if the cron job works at the moment!)

Basically, the above works, but I needed to load all parent objects for ItemAddFromServer so everything would work (i.e. ItemAddFromServer doesn't automatically load all the classes it requires, it expects them to have been loaded already, same with all other classes this one calls). The outline of my code is now:

    /*
     * Initialize G2 (includes all necessary classes,
     * puts G2 into a state ready to process your calls)
     */
    require_once($g2_Config['path'] . '/embed.php');
    $ret = GalleryEmbed::init(array(
                                'path' => $g2_Config['path'],
                                'g2Uri' => $g2_Config['g2Uri'],
                                'fullInit' =>'true'
                                ));
    check($ret);

    //import needed classes
    GalleryCoreApi::requireOnce('modules/core/classes/GalleryController.class');
    GalleryCoreApi::requireOnce('modules/core/classes/GalleryView.class');
    GalleryCoreApi::requireOnce('modules/core/ItemAdd.inc');
    GalleryCoreApi::requireOnce('modules/itemadd/ItemAddFromServer.inc');

    //instantiate the ItemAdd class
    $itemAdd = new ItemAddFromServer();

    <<check there are files to be added>>

    //lock the album we're adding things to
    $currentLockIds = array();
    list ($ret, $currentLockIds) = GalleryCoreApi::acquireWriteLock($gallery_id, 60);
    check($ret);

    <<for every file in my upload directory>>
    $ret = $itemAdd->addItem($gallery_id, $add_path . $uploaded, false, $text_fields, $status, $error);
    check($ret);

    <<delete added files>>

    <<after all files added>>
    //release locks
    $ret = GalleryCoreApi::releaseLocks($currentLockIds);
    check($ret);

    <<finally>>
    /*
     * At the end, commit the transaction by calling ::done()
     * or all changes made above will be rolled back automatically
     */
    $ret = GalleryEmbed::done();
    check($ret);

...and set to run every 15 minutes as a cron job on my web server, merrily uploading photos when someone with the password FTPs some photos into the directory!

Thanks for all your help guys, hope this helps someone else in the future!

 
nivekiam
nivekiam's picture

Joined: 2002-12-10
Posts: 16504
Posted: Thu, 2008-08-28 20:48

Awesome! Hopefully you can get a few more people to test it, I'll try to test it if I get time.

Please post add this to the Codex under either Utilities or Mods/Patches (not sure which category it would be best under)
http://codex.gallery2.org/Gallery2:User_Contributions

Could you expand on this a little and explain more step by step what it would take for someone else to duplicate? Do I just copy and paste your code into a text file? Is there anything else I need? Do I have to take that code and put it between php tags? <?php your-code-here ?> etc...
____________________________________________
Like Gallery? Like the support? Donate now!!! See G2 live here

 
Blood_Hawk

Joined: 2008-08-26
Posts: 5
Posted: Tue, 2008-09-02 21:28

OK, I've finished working on this now I think. I've made the script into a self-contained PHP object which I basically call from a couple of scripts on my webserver (one running the add_files and one running add_dir command).

I'm sure there'll be one or two things missing/not quite right (I'm not 100% convinced about Session handling for example), so guess I'll work on it if there's need/when I've more time, but it's working for me on my server now so I'm not going to be updating it a great deal in the near future.

The script is attached (and I've linked this post from the Mods section of the user_contributions Wiki page).

To call the script:

1) Setup a PHP script file (or other if you wish, but it'll have to be able to use PHP Objects)
2) Includes/require the embed.php file from your Gallery2 installation directory
3) Includes/require the gallery_auto_upload_funcs.php file from your Gallery2 installation directory
4) Create a new AutoUploader Object, passing in the required variables [check the constructor function at the top of the Object for parameter details] (e.g. $autoUploader = new AutoUploader('/home/my_user/', 'uploads/gallery_specific_dir/', 'public_html/gallery2/', 'gallery2/');)
5) Initialise the AutoUploader Object, saving the returned list of files to be processed [note that username and password are optional here, but omitting them means all pictures/albums will be loaded using the Guest account, giving full-access to any Guests visiting your Gallery, assuming Guest can upload files, etc of course] (e.g. $files = $autoUploader->initialise_uploads($album_id, 'username', 'password');)
6) Call the AutoUpload function you wish to use, (e.g. $autoUploader->add_dir(); OR $autoUploader->add_files($files);)
7) Complete the process by calling the AutoUploader's completion function (e.g. $autoUploader->complete_upload($files);)

The initialise and completion functions must be called at the start and end of processing respectively as they basically prepare and complete the Gallery2 interactions (as well as moving files and cleaning up the file system, etc). If there is a problem during the file/directory upload, AutoUploader will attempt to move the files back to the upload directory from which they originated and rollback the changes made to Gallery2.

The script you created above can then be called via the internet (if the script is in a publicly available HTML folder) OR, for full automation, can be setup as a Cron job to run every, say, 15 minutes (e.g. cs '/uploads/gallery_specific_dir/'; PHP -q 'gallery_auto_uploads.php';).

Finally, it is a good idea to setup the required file-directory structure on your server before running the upload script. The AutoUploader will expect a sub-directory called 'upload/' to be located in the directory you specify as the gallery upload base directory (secon parameter in the AutoUploader constructor). It also uses an 'add/' sub-directory in the same base location. These sub-directories will be created this if it they don't already exist, but it's always better to be safe rather than sorry. A processing.txt file will be created (and later deleted) in the upload script's directory in order to ensure the script isn't trying to run more than once at any given time (possibly resulting in duplicate uploads and/or file read/write problems). Obviously, the script process will need all permissions appropriate to performing these tasks.

 
bt86bt

Joined: 2008-10-18
Posts: 2
Posted: Sat, 2008-10-18 11:16

Whao, this is just what I was looking for! I am looking to sync my gallery with a public image-host my friend has. I am going to give this a test right away and report back. If I understand correct I just edit this line to the php file? ::

public function AutoUploader($server_root_dir = '/home/bt/', $upload_base_dir = 'uploads/php_upload/', $gallery_dir = '/mnt/USB1/www/gallery/', $rel_gallery_dir = 'test/', $lock_wait_secs = 60, $file_mod_secs = 10, $error_log_file = '')

Edit 1: Does not seem to work... As I understood it I need to create another file first, how should that look like?

_____________________________________
http://gallery.inara.nu

 
james147

Joined: 2009-01-20
Posts: 5
Posted: Fri, 2009-01-23 21:49

Is this script being supported anymore? Not able to get script to actually work, but am getting no errors in browser. Unable to get script logging to work. Here's what I've done so far. Not sure if it's because I'm running gallery embedded into my site. Using script from attachment above, I did not make any changes to the script. I created a second php script to include and reference. Any help appreciated.

Created a php script file:
<?php
error_reporting(E_ALL);
include("/home/jbmitcom/public_html/gallery2/embed.php");
include("/home/jbmitcom/public_html/autoupload.php");
$files = $autoUploader->initialise_uploads($parent_album_id = '6900', $username = 'admin', $password = 'njm1305');
$autoUploader->complete_upload($files);
$autoUploader = new AutoUploader($server_root_dir = '/home/jbmitcom/public_html/', $upload_base_dir = 'attachments/', $gallery_dir = 'gallery2/', $rel_gallery_dir = '/gallery2/', $lock_wait_secs = '60', $file_mod_secs = '10', $error_log_file = 'addemail.log');
$files = $autoUploader->initialise_uploads($parent_album_id = '6900', $username = 'myuser', $password = 'mypassword');
$autoUploader->add_files($files);
$autoUploader->complete_upload($files);
?>

My Gallery Specs:
Gallery version = 2.2.5 core 1.2.0.7
PHP version = 5.2.6 apache2handler
Webserver = Apache
Database = mysqlt 4.1.22-standard, lock.system=flock
Toolkits = ArchiveUpload, Dcraw, Exif, Ffmpeg, Getid3, LinkItemToolkit, NetPBM, SquareThumb, Thumbnail, Gd, ImageMagick
Acceleration = none, none
Operating system = Linux

 
bplgonzo

Joined: 2009-04-03
Posts: 4
Posted: Fri, 2009-04-03 12:19

I think you need to create object instance first, and then call methods. Try like this:

<?php
include("/home/jbmitcom/public_html/gallery2/embed.php");
include("/home/jbmitcom/public_html/autoupload.php");
$autoUploader = new AutoUploader('/home/jbmitcom/public_html/', 'attachments/', 'gallery2/', '/gallery2/');
$files = $autoUploader->initialise_uploads('6900', 'admin', 'njm1305');
$autoUploader->add_files($files);
$autoUploader->complete_upload($files);
?>

 
bplgonzo

Joined: 2009-04-03
Posts: 4
Posted: Fri, 2009-04-03 12:21

Blood_Hawk (od anybody else), do you know how to include filename as title, and add watermark to resizes via this cron script? I know it can be done trough form on "Upload from local" page, so I guess that it can be done here also...

 
bplgonzo

Joined: 2009-04-03
Posts: 4
Posted: Fri, 2009-04-03 15:00

To be more specific on post above:

script for automatic adding images is working, images do appear in specific album but without EXIF/IPTC data. No title, no caption/description.
When I add images trough "Add From Local Server" page, everything working OK, title and description appears. Can someone point me to right direction?

 
Dr_Spock

Joined: 2009-02-11
Posts: 5
Posted: Mon, 2009-06-22 19:50

I have a whole bunch of video files in my Gallery, and they need their own thumbnails. My server don't have ffmpeg (and they don't want to have it either), so I can't use that to automatically generate thumbnails. Now I have to generate the thumbnails on my computer (that's OK) and then manually upload each and every thumbnail to each and every video (and that is NOT OK).

The script I read about here seems to be ALMOST what I need, but unfortunately I don't now that much scripting to tweak it to my needs. The videos are in different folders and subfolders. Some folders have just one video, others have 20+, some have both videos and photos. It would be nice to name the thumbnails (jpg) exactly the same as the videos (flv) (example: video23.flv & video23.jpg), upload the thumbnails to the same folders as the videos (in the g2data/albums), and then run a script that connects the thumbs to the videos, and at the same time move the jpgs away from the g2data/album-folders since they are not stand-alone photos.

Could someone please guide me?

 
soldstatic

Joined: 2005-06-18
Posts: 35
Posted: Wed, 2009-07-01 03:20

sorry for the uber dumb-ness here, but I'm confused, I need to create a new php script to run the functions built into the attached script here, right?

Does anyone have a decent example they could post? I'm a bit wary of cron cause I haven't been able to get it to work before, so I don't totally understand what is required (as far as php object calling ability? Isn't that just built into apache or php or whatever on my [shared hosting plan dreamhost] web server?)

Thanks, I'm just not too good at coding anymore (except excel, I can rock some macros out on excel) lol

 
simon_b

Joined: 2010-01-19
Posts: 8
Posted: Wed, 2010-01-20 14:39

Has anyone been able to get this going?

When I run objectupload.php, I get the error:

"Gallery Upload base directory (/home102b/sub006/sc49607-JHDL/mydomain.com/test-delete/modules/galleryautouploadfuncs/attachments/ not found"

I use this code to create objectupload.php

<?php
include("/home102b/sub006/sc49607-JHDL/mydomain.com/test-delete/embed.php");
include("/home102b/sub006/sc49607-JHDL/mydomain.com/test-delete/modules/galleryautouploadfuncs/gallery_auto_upload_funcs.php");
$autoUploader = new AutoUploader('(/home102b/sub006/sc49607-JHDL/mydomain.com/', 'test-delete/modules/galleryautouploadfuncs/attachments/', 'test-delete/modules/galleryautouploadfuncs/gallery2/', 'test-delete/modules/galleryautouploadfuncs/gallery2/');
$files = $autoUploader->initialise_uploads('21', 'admin', 'password');
$autoUploader->add_files($files);
$autoUploader->complete_upload($files);
?>

Is there anything I am doing wrong here? I even put some images in the attachments directory, but it doesn't make a difference.

 
commonplace

Joined: 2010-05-20
Posts: 9
Posted: Wed, 2010-06-09 00:37

Any way of making this work with Gallery 3? And if so... how? :) All of this was pretty much beyond my skill set.

/Kevin

 
alecmyers

Joined: 2006-08-01
Posts: 4342
Posted: Wed, 2010-06-09 09:30
Quote:
Any way of making this work with Gallery 3?

Best to stick to asking this in a G3 section of this forum, as this thread is for G2 only.