IPTC/EXIF and Keywords

shawnebrown

Joined: 2005-02-25
Posts: 4
Posted: Tue, 2005-04-26 13:20

I noticed in the EXIF/IPTC module that there is an option to look and see if there are IPTC keywords and if so add them to the database.

Does this mean the same KEYWORDS that will then be searchable?

Thanks in advance,
Shawn

----

Gallery URL (optional):
Gallery version:
Webserver (with version):
Datatabase (with version):
PHP version (eg 4.2.1):
phpinfo URL (optional):
Graphics Toolkit(s):
Operating system:
Web browser/version:
G1 version (for migration bugs):

 
buut
buut's picture

Joined: 2003-06-18
Posts: 196
Posted: Tue, 2005-04-26 13:57

shawnebrown,

I'am just finished conducting a very small test how this works.
In the EXIF/IPTC module you can define wich IPTC field will be shown in the summery/details photo properties below the photo.
And all the keywords which are available in the IPTC field Keyword are placed in the photo item keywords (in database), these keyword are not visible to the viewer. But these keyword are searcheable.

 
bharat
bharat's picture

Joined: 2002-05-21
Posts: 7994
Posted: Sat, 2005-04-30 21:46

We plan to allow you to extract all the EXIF/IPTC data into the database (even that data which isn't put into the general item fields) and make it searchable. It's not that hard, we just need more dev resources.

 
outpostbabu

Joined: 2005-02-26
Posts: 15
Posted: Sat, 2005-10-08 01:13

I would be interested in giving this a go because I want to be able to search G2's database for EXIF field values. I've not found anything that describes how the pipeline works with EXIF. Can somebody point me to this information?

 
outpostbabu

Joined: 2005-02-26
Posts: 15
Posted: Sat, 2005-10-08 03:00

I've gone through the code forwards and backwards and I think I've found some clue as to how I can get the EXIF data into the database. It appears that it would require modification of handleRequestAfterAdd(...) in ExifDescriptionOption.inc. The workhorse code is:

                    ExifExtractor::getMetaData(
                        array($itemId),
                        array('IPTC/Caption',
                              'ImageDescription',
                              'UserComment',
                              'IPTC/Keywords',
                              )
                        );

Am I right that populating that array with my custom EXIF fields would cause the extractor to return the data for those fields? If so, great but the current set of properties is simply assigned to the item description:

$itemDescription = $exifData[$itemId]['UserComment']['value'];

I traced the setDescription(...) calls back hoping to eventually find a database query to insert/update the description but its a long and winding trail of code that appears to include caching and other things before it reaches the database. I thought I would add a new method to GalleryEntity that would eventually store the data persistently but it seems very complicated. I'm not sure what has to change beyond GalleryEntity to get the data into the database. Anyone?

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sat, 2005-10-08 09:57

also see:
http://gallery.menalto.com/node/37994

- first you'd have to create additional g2 database tables to hold the exif / iptc data, maybe a map table
g_id, profile (exif or iptc), field, value

- then you'd have to add the exif data to this map in handleRequestAfterAdd.

- then you'd have to add a search plugin implementation similar to how it's done in the comments module: modules/comment/classes/GalleryCommentSearch.class

 
outpostbabu

Joined: 2005-02-26
Posts: 15
Posted: Mon, 2005-10-10 17:56

As a test I modified handleRequestAfterAdd(...) in ExifDescriptionOption.inc to support the fields of interest to me. I created a database table to store the itemID, profile, field, and value and cheated by directly connecting to the database to populate the fields. The test it, I added an image with the GPS fields to a test gallery. But nothing happened. The fields were not populated, I assume, because the code was never called to populate them. No errors were given and the image was added to the gallery without a problem. So my question is, when is handleRequestAfterAdd(...) called? Once I get that figured out and the database populating when images are added I will trudge through the source and do it the right way by adding the appropriate methods to entity, storage, etc.

 
jrmint

Joined: 2005-08-20
Posts: 94
Posted: Mon, 2005-10-10 18:22

I have been working this also. I added a table called g2_IPTC and added a few IPTC fields. No errros but nothing in the database.

I even tryed to just add a field to the g2_Item table called g_city same value as g_description still no information put in the database.

I think that there is another file that sets up the set comands like setKeywords and setDescription. When I tryed to use a setCity I got an error.

 
outpostbabu

Joined: 2005-02-26
Posts: 15
Posted: Mon, 2005-10-10 19:08

Yes, if you want to add methods to the objects you will have to update a long chain of classes to get from the initial creation of the data to the database. I myself have not figured that chain out fully yet except that it must require changes to the GalleryStorage class, the implementation classes for GalleryStorage, and at least some of the GalleryItem and GalleryEntity type classes. I wasn't too worried about that part just yet because I figured one of the active developers could illuminate that for us. My test was simply to see if the handler was getting called and, as you know, it is apparently not. It would be useful to have a UML sequence diagram of the item addition process but I doubt such a thing exists. If it comes to it, I will make one but I have done no other development on gallery and so am a complete noob to the source. I'll poke around in there more tonight and let you know what I find out.

 
jrmint

Joined: 2005-08-20
Posts: 94
Posted: Mon, 2005-10-10 20:24

I just got some of what I wanted to do.
To start off with I just wanted to see if I could put the IPTC/CITY metadata into the database. As I said before. I started with the current database table called g2_Item. I added a field called g_city.

I had to edit two files. the ExifDescriptionOption.inc and the GalleryItem.inc in \gallery2\modules\core\classes\interfaces

From what I could see the GalleryItem.inc is what points to the database.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Mon, 2005-10-10 21:44

wait a momement :)

1.
all files in modules/*/classes/interfaces/* are generated automatically.
also all CREATE TABLE sql statements in modules/*/classes/GalleryStorage/DatabaseStorage/schema/platform/*/*.sql is generated automatically.

how?
you create a class in modules/*/classes/ClassNameMap.class and add some markup (take a look at the existing *Map.class files.
we use php to parse our .class files for XML markup and generate from the markup the interfaces/*.inc files and the sql files for all databases (mysql, postgresql, ..).

2.
do no add columns to existing tables etc. e.g. do not add something to g2_Item.

first: get the latest G2 version. we changed some important things since 2.0. get the nightly snapshot.

you should create a new *Map.class file in modules/exif/classes/ which has the members g_id, profile (exif or iptc), field, value.
then run gmake in modules/exif/classes/exif/ . this will generate the corresponding interface file and the SQL table definition.

then add your code to add and query data into / from the map table.

finished.

 
outpostbabu

Joined: 2005-02-26
Posts: 15
Posted: Mon, 2005-10-10 22:01

Valiant, your explaination of the process is very helpful. Makes it sound like a little less work than I had anticipated. I'm off to try it out. Thanks!

 
outpostbabu

Joined: 2005-02-26
Posts: 15
Posted: Mon, 2005-10-10 23:29

OK, so I installed the latest CVS nightly and created my own *Map.class file called ExifFieldsMap.class. This I based directly on ExifPropertiesMap.class. It is as follows:

<?php
/**
 * @g2 <class-name>ExifFieldsMap</class-name>
 * @g2 <schema>
 * @g2   <schema-major>1</schema-major>
 * @g2   <schema-minor>0</schema-minor>
 * @g2 </schema>
 *
 * @package Exif
 * @subpackage Classes
 */

class ExifFieldsMap_core {

    /**
     * @g2 <map>
     * @g2   <member>
     * @g2     <member-name>itemId</member-name>
     * @g2     <member-type>INTEGER</member-type>
     * @g2     <member-size>LARGE</member-size>
     * @g2   </member>
     * @g2   <member>
     * @g2     <member-name>profile</member-name>
     * @g2     <member-type>STRING</member-type>
     * @g2     <member-size>MEDIUM</member-size>
     * @g2   </member>
     * @g2   <member>
     * @g2     <member-name>field</member-name>
     * @g2     <member-type>STRING</member-type>
     * @g2     <member-size>LARGE</member-size>
     * @g2   </member>
     * @g2   <member>
     * @g2     <member-name>value</member-name>
     * @g2     <member-type>STRING</member-type>
     * @g2     <member-size>LARGE</member-size>
     * @g2   </member>
     * @g2 </map>
     */

    /**
     * Return the target properties for the given view mode
     *
     * @param int the view mode (EXIF_SUMMARY, etc)
     * @return array object GalleryStatus a status code
     *               array logical exif property names
     * @static
     */
    function getProperties($itemId) {
        global $gallery;

        $query = '
        SELECT
          [ExifFieldsMap::itemId], [ExifFieldsMap::profile], [ExifFieldsMap::field], [ExifFieldsMap::value]
        FROM
          [ExifFieldsMap]
        WHERE
          [ExifFieldsMap::itemId] = ?
        ORDER BY
          [ExifFieldsMap::field] ASC';
        list ($ret, $searchResults) = $gallery->search($query, array($itemId));
        if ($ret->isError()) {
            return array($ret->wrap(__FILE__, __LINE__), null);
        }

        $data = array();
        while ($result = $searchResults->nextResult()) {
            $data[] = $result[0];
        }

        return array(GalleryStatus::success(), $data);
    }

    /**
     * Set the target properties for the given item ID
     *
     * @param int the item ID
     * @param array logical property key/value pairs
     * @return object GalleryStatus a status code
     * @static
     */
    function setProperties($itemId, $properties) {

        /* Remove all old map entries */
        $ret = ExifPropertiesMap::removeMapEntry(array('itemID' => $itemId));
        if ($ret->isError()) {
            return $ret->wrap(__FILE__, __LINE__);
        }

        for ($i = 0; $i < sizeof($properties); $i++) {
            $ret = ExifPropertiesMap::addMapEntry(array('property' => $properties[$i], 'itemId' => $itemId));
            if ($ret->isError()) {
                return $ret->wrap(__FILE__, __LINE__);
            }
        }

        return GalleryStatus::success();
    }
}

include(dirname(__FILE__) . '/interfaces/ExifFieldsMap.inc');
?>

I simply made changes for the appropriate database columns and reflected those changes in the methods that already existed.

There is no ExifFieldsMap.inc in interfaces, so I run gmake to generate everything. It completes without error:

cd interfaces && gmake -
gmake[1]: Entering directory `/gallery2/modules/exif/classes/interfaces'
perl ../../../../lib/tools/bin/extractClassXml.pl --dtd=../../../../../lib/tools/dtd/GalleryClass2.0.dtd --stub-ok --quiet --out-dir=tmp ../*.class
php ../../../../lib/tools/bin/generate-interfaces.php Exif
Content-type: text/html
X-Powered-By: PHP/4.4.0

gmake[1]: Leaving directory `/gallery2/modules/exif/classes/interfaces'
cd GalleryStorage/DatabaseStorage/schema && gmake -
gmake[1]: Entering directory `gallery2/modules/exif/classes/GalleryStorage/DatabaseStorage/schema'
perl ../../../../../../lib/tools/bin/extractClassXml.pl --dtd=../../../../../../lib/tools/dtd/GalleryClass2.0.dtd --quiet --out-dir=tmp/classxml ../../../*.class
php -f ../../../../../../lib/tools/bin/generate-dbxml.php
php -f ../../../../../../lib/tools/bin/generate-sql.php tmp/classxml
gmake[1]: Leaving directory `/gallery2/modules/exif/classes/GalleryStorage/DatabaseStorage/schema'

But after this, there is no matching ExifFieldsMap.inc in interfaces. I expected it because the ExifPropertiesMap.inc says its automatically generated. Additionally there is no matching database table in the database. Does something else have to be done to generate them? Or perhaps I have done something wrong?

After this I optimized the database, removed the caches, etc as listed under maintanence thinking maybe it had something to do with that but no such luck. Is this process documented somewhere? I don't want you to have to repeat information that already exists.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2005-10-11 06:52

instead of "field", let's use

     * @g2   <member>
     * @g2     <member-name>property</member-name>
     * @g2     <member-type>STRING</member-type>
     * @g2     <member-size>MEDIUM</member-size>
     * @g2   </member>

for compatibility with ExifPropertiesMap.class
and maybe we should get a better name for this map. maybe ExifPropertyValuesMap or so.

also, you forgot to set a unique key. add

     * @g2   <key>
     * @g2     <member-name>itemId</member-name>
     * @g2     <member-name>profile</member-name>
     * @g2     <member-name>property</member-name>
     * @g2   </key>
Quote:
I simply made changes for the appropriate database columns and reflected those changes in the methods that already existed.

i don't quite get that part. what did you do?
to make it clear: you don't have to edit any database table. if you mean that you just edited function getProperties($itemId) { and function setProperties($itemId) {

@ function getProperties($itemId) { and setProperties
1. i'd call these functions getPropertyValues and setPropertyValues
2. the comments right above the functions are our phpdoc style documention for each function. if you change the function, you should update the this documention (manually)

@gmake:
i've used your map class definition and changed the class name and the fields -> property stuff, added a key, didn't change the functions yet.
then i ran gmake in modules/exif/classes/ and it successfully created the interface and the sql file for each database.
did you really try: http://galleryupdates.jpmullan.com/G2/gallery-2.0+.tar.gz ?

this is what i used:

<?php
/**
 * @g2 <class-name>ExifPropertyValuesMap</class-name>
 * @g2 <schema>
 * @g2   <schema-major>1</schema-major>
 * @g2   <schema-minor>0</schema-minor>
 * @g2 </schema>
 *
 * @package Exif
 * @subpackage Classes
 */

class ExifPropertyValuesMap_core {

    /**
     * @g2 <map>
     * @g2   <member>
     * @g2     <member-name>itemId</member-name>
     * @g2     <member-type>INTEGER</member-type>
     * @g2     <member-size>LARGE</member-size>
     * @g2   </member>
     * @g2   <member>
     * @g2     <member-name>profile</member-name>
     * @g2     <member-type>STRING</member-type>
     * @g2     <member-size>MEDIUM</member-size>
     * @g2   </member>
     * @g2   <member>
     * @g2     <member-name>property</member-name>
     * @g2     <member-type>STRING</member-type>
     * @g2     <member-size>MEDIUM</member-size>
     * @g2   </member>
     * @g2   <member>
     * @g2     <member-name>value</member-name>
     * @g2     <member-type>STRING</member-type>
     * @g2     <member-size>LARGE</member-size>
     * @g2   </member>
     * @g2   <key>
     * @g2     <member-name>itemId</member-name>
     * @g2     <member-name>profile</member-name>
     * @g2     <member-name>property</member-name>
     * @g2   </key>
     * @g2 </map>
     */

    /**
     * Return the target properties for the given view mode
     *
     * @param int the view mode (EXIF_SUMMARY, etc)
     * @return array object GalleryStatus a status code
     *               array logical exif property names
     * @static
     */
    function getProperties($itemId) {
        global $gallery;

        $query = '
        SELECT
          [ExifFieldsMap::itemId], [ExifFieldsMap::profile], [ExifFieldsMap::field], [ExifFieldsMap::value]
        FROM
          [ExifFieldsMap]
        WHERE
          [ExifFieldsMap::itemId] = ?
        ORDER BY
          [ExifFieldsMap::field] ASC';
        list ($ret, $searchResults) = $gallery->search($query, array($itemId));
        if ($ret->isError()) {
            return array($ret->wrap(__FILE__, __LINE__), null);
        }

        $data = array();
        while ($result = $searchResults->nextResult()) {
            $data[] = $result[0];
        }

        return array(GalleryStatus::success(), $data);
    }

    /**
     * Set the target properties for the given item ID
     *
     * @param int the item ID
     * @param array logical property key/value pairs
     * @return object GalleryStatus a status code
     * @static
     */
    function setProperties($itemId, $properties) {

        /* Remove all old map entries */
        $ret = ExifPropertiesMap::removeMapEntry(array('itemID' => $itemId));
        if ($ret->isError()) {
            return $ret->wrap(__FILE__, __LINE__);
        }

        for ($i = 0; $i < sizeof($properties); $i++) {
            $ret = ExifPropertiesMap::addMapEntry(array('property' => $properties[$i], 'itemId' => $itemId));
            if ($ret->isError()) {
                return $ret->wrap(__FILE__, __LINE__);
            }
        }

        return GalleryStatus::success();
    }
}

include(dirname(__FILE__) . '/interfaces/ExifPropertyValuesMap.inc');
?>
 
jrmint

Joined: 2005-08-20
Posts: 94
Posted: Tue, 2005-10-11 20:38

I am getting the following error when I try and run the make. Any Ideas?

The java class is not found: com/icl/saxon/StyleSheet
make[3]: *** [iptcPropertyValuesMap.sql] Error 1
make[3]: Leaving directory `/srv/www/htdocs/gallery2/modules/exif/classes/GalleryStorage/DatabaseStorage/schema/platform/postgres'
make[2]: *** [sql] Error 2
make[2]: Leaving directory `/srv/www/htdocs/gallery2/modules/exif/classes/GalleryStorage/DatabaseStorage/schema/platform'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/srv/www/htdocs/gallery2/modules/exif/classes/GalleryStorage/DatabaseStorage/schema'
make: *** [all] Error 2

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2005-10-11 20:41

jrmint, you're not using the latest nightly snapshot. looks rather like g 2.0.
you need to cvs update -dPC or get the latest nightly snapshot.

http://galleryupdates.jpmullan.com/G2/gallery-2.0+.tar.gz

 
jrmint

Joined: 2005-08-20
Posts: 94
Posted: Tue, 2005-10-11 20:54

I just did another update.
cvs -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/gallery checkout -P gallery2
And I am still getting the same error.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2005-10-11 20:59

use
cvs -z3 update -dPC

-C replaces locally customized / changed files with original files from the cvs server. -z3 adds compression during the transfer.

and run gmake in modules/exif/classes/

cd gallery2/modules/exif/classes/
gmake

 
jrmint

Joined: 2005-08-20
Posts: 94
Posted: Tue, 2005-10-11 21:04

I forgot to add the Debugging module. Now the script looks like it is running but the .inc file is empty.
I am attaching a gmake.txt file with what all happens when I run the make.

 
jrmint

Joined: 2005-08-20
Posts: 94
Posted: Tue, 2005-10-11 21:14

here is what my iptcPropertyValuesMap.class looks like.

<?php
/**
* @g2 <class-name>iptcPropertyValuesMap</class-name>
* @g2 <schema>
* @g2 <schema-major>1</schema-major>
* @g2 <schema-minor>0</schema-minor>
* @g2 </schema>
*
* @package Exif
* @subpackage Classes
*/

class iptcPropertyValuesMap_core {

/**
* @g2 <map>
* @g2 <member>
* @g2 <member-name>itemId</member-name>
* @g2 <member-type>INTEGER</member-type>
* @g2 <member-size>LARGE</member-size>
* @g2 </member>
* @g2 <member>
* @g2 <member-name>profile</member-name>
* @g2 <member-type>STRING</member-type>
* @g2 <member-size>MEDIUM</member-size>
* @g2 </member>
* @g2 <member>
* @g2 <member-name>property</member-name>
* @g2 <member-type>STRING</member-type>
* @g2 <member-size>MEDIUM</member-size>
* @g2 </member>
* @g2 <member>
* @g2 <member-name>value</member-name>
* @g2 <member-type>STRING</member-type>
* @g2 <member-size>LARGE</member-size>
* @g2 </member>
* @g2 <key>
* @g2 <member-name>itemId</member-name>
* @g2 <member-name>profile</member-name>
* @g2 <member-name>property</member-name>
* @g2 </key>
* @g2 </map>
*/

/**
* Return the target properties for the given view mode
*
* @param int the view mode (EXIF_SUMMARY, etc)
* @return array object GalleryStatus a status code
* array logical exif property names
* @static
*/
function getProperties($itemId) {
global $gallery;

$query = '
SELECT
[iptcFieldsMap::itemId], [iptcFieldsMap::profile], [iptcFieldsMap::field], [iptcFieldsMap::value]
FROM
[iptcFieldsMap]
WHERE
[iptcFieldsMap::itemId] = ?
ORDER BY
[iptcFieldsMap::field] ASC';
list ($ret, $searchResults) = $gallery->search($query, array($itemId));
if ($ret->isError()) {
return array($ret->wrap(__FILE__, __LINE__), null);
}

$data = array();
while ($result = $searchResults->nextResult()) {
$data[] = $result[0];
}

return array(GalleryStatus::success(), $data);
}

/**
* Set the target properties for the given item ID
*
* @param int the item ID
* @param array logical property key/value pairs
* @return object GalleryStatus a status code
* @static
*/
function setProperties($itemId, $properties) {

/* Remove all old map entries */
$ret = iptcPropertiesMap::removeMapEntry(array('itemID' => $itemId));
if ($ret->isError()) {
return $ret->wrap(__FILE__, __LINE__);
}

for ($i = 0; $i < sizeof($properties); $i++) {
$ret = iptcPropertiesMap::addMapEntry(array('property' => $properties[$i], 'itemId' => $itemId));
if ($ret->isError()) {
return $ret->wrap(__FILE__, __LINE__);
}
}

return GalleryStatus::success();
}
}

include(dirname(__FILE__) . '/interfaces/iptcPropertyValuesMap.inc');
?>

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2005-10-11 22:12

jrmint, it's still using g 2.0 stuff, not the current cvs version.
delete your g2 and get the nightly snapshot http://galleryupdates.jpmullan.com/G2/gallery-2.0+.tar.gz . or install it as a second g2 on your box.
obviously you're doing something wrong with cvs or it doesn't work as it should or you have used a release / date flag in prior cvs checkouts / updates.
just get the nightly snapshot.

 
jrmint

Joined: 2005-08-20
Posts: 94
Posted: Tue, 2005-10-11 22:51

valiant
Thanks for all of your help.
I deleted the gallery and loaded the cvs and started all over with a fresh install. Then copyed my map.class and ran the make here is what I got back.
I do have information in the .xml and .inc but no table was made the Database.

Or do I need to add the table first then run the make?

cd interfaces && make -
make[1]: Entering directory `/srv/www/htdocs/gallery2/modules/exif/classes/interfaces'
perl ../../../../lib/tools/bin/extractClassXml.pl --dtd=../../../../../lib/tools/dtd/GalleryClass2.0.dtd --stub-ok --quiet --out-dir=tmp ../*.class
php ../../../../lib/tools/bin/generate-interfaces.php Exif

Notice: Undefined index: content in /srv/www/htdocs/gallery2/lib/tools/bin/generate-interfaces.php on line 75

Notice: Undefined index: content in /srv/www/htdocs/gallery2/lib/tools/bin/generate-interfaces.php on line 75

Warning: file(IptcItemsMap.inc): failed to open stream: No such file or directory in /srv/www/htdocs/gallery2/lib/tools/bin/generate-interfaces.php on line 131

Notice: Uninitialized string offset: 2 in /srv/www/htdocs/gallery2/lib/tools/bin/generate-interfaces.php on line 132

Notice: Uninitialized string offset: 24 in /srv/www/htdocs/gallery2/lib/tools/bin/generate-interfaces.php on line 133

Warning: md5_file(): Unable to open file in /srv/www/htdocs/gallery2/lib/tools/bin/generate-interfaces.php on line 139

Warning: unlink(IptcItemsMap.inc): No such file or directory in /srv/www/htdocs/gallery2/lib/tools/bin/generate-interfaces.php on line 140
make[1]: Leaving directory `/srv/www/htdocs/gallery2/modules/exif/classes/interfaces'
cd GalleryStorage/DatabaseStorage/schema && make -
make[1]: Entering directory `/srv/www/htdocs/gallery2/modules/exif/classes/GalleryStorage/DatabaseStorage/schema'
perl ../../../../../../lib/tools/bin/extractClassXml.pl --dtd=../../../../../../lib/tools/dtd/GalleryClass2.0.dtd --quiet --out-dir=tmp/classxml ../../../*.class
php -f ../../../../../../lib/tools/bin/generate-dbxml.php
php -f ../../../../../../lib/tools/bin/generate-sql.php tmp/classxml
make[1]: Leaving directory `/srv/www/htdocs/gallery2/modules/exif/classes/GalleryStorage/DatabaseStorage/schema'

 
jrmint

Joined: 2005-08-20
Posts: 94
Posted: Wed, 2005-10-12 17:22

2 questions.
1. I would like to extend the GalleryItem.inc and GalleryItem.class. These files are doing what I need them to do, except I want to point the IPTC metadata to a new database table I created. Can I just create a new .inc and .class with the code I need and do a requireonce in the original files?

2. If I do create a new module How do I get Gallery2 to run the new module?

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Wed, 2005-10-12 17:32

@ gmake issues:
maybe bharat looks into your gmake issues. he recently changed the generate-*.php files.

@2 questions:
why don't you follow the approach described here. putting the iptc/exif data into a new exifpropertyvaluesmap table? (which is linked through itemId with the items)
your approach, editing GalleryItem.class etc isn't modular. in g2, we don't want to edit the core files, you add functionality by adding modules.

you can register your own entity type in a module, which is then used instead of GalleryItem.

 
jrmint

Joined: 2005-08-20
Posts: 94
Posted: Wed, 2005-10-12 17:54

you can register your own entity type in a module, which is then used instead of GalleryItem.

What are the steps to register my own module?

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Wed, 2005-10-12 21:42

jrmint,
i didn't explain this yet because i want to know what you're trying to do here. maybe it's unnecessary. i might find a better way to implement what you have in mind since i know the g2 architecture a little better.
so what exactly is your goal and what's the exact plan?

 
jrmint

Joined: 2005-08-20
Posts: 94
Posted: Wed, 2005-10-12 22:27

What I am trying to do is make a module that will add a table called g2_ItcpItems with some indexed fields called headline, city, byline, objectname ext ext.
The module should look at the jpgs during upload and populate these fields in the database.

Then I will need to be able to search by one or more of these fields.

Thanks you again for your help.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Wed, 2005-10-12 22:37

why don't you follow the approach i sketched here?

your approach is limited in the sense that your table has a column for each iptc field, it's not flexible.
also, there's no need to create a iptcItem. a mapping for items with exif / iptc data is good enough.

 
jrmint

Joined: 2005-08-20
Posts: 94
Posted: Wed, 2005-10-12 22:57

I tryed to do the gmake *.map.class but it never finishes. I keep getting Java errors.
So I thought that maybe I should look into doing this with a differant table.

If I can get a module that will add the IPTC fields into g2_Item then populate those fields I will use it..

I am probably doing this wrong.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Wed, 2005-10-12 23:03

@java errors:
no, your latest errors are php errors and i'm sure bharat will address them soon.
and just because gmake doesn't work for you right now, you don't have to dump this approach.

but of course you're free to do it your own way. but it's clearly very limited in use (because of your architecture to add db columns for iptc fields, instead of a mapping table which maps fields / values in rows.
also, your approach will be more complicated once it comes to registering and replacing GalleryItem with your iptc item. actually, it won't be trivial at all.

so i'd encourage you to not follow this iptcitem approach anymore. it would soon get complicated and it wouldn't offer a more powerful or elegant solution than the mapping table.

 
jrmint

Joined: 2005-08-20
Posts: 94
Posted: Wed, 2005-10-12 23:19

Here is my last map.class file
What am I doing wrong? I think that I don't have the information to point to the g2_Item table..

<?php
/**
* @g2 <class-name>IptcItemMap</class-name>
* @g2 <schema>
* @g2 <schema-major>1</schema-major>
* @g2 <schema-minor>0</schema-minor>
* @g2 </schema>
*
* @package Exif
* @subpackage Classes
*/

class IptcItemMap_core {

/**
* @g2 <map>
* @g2 <member>
* @g2 <member-name>headline</member-name>
* @g2 <member-type>TEXT</member-type>
* @g2 </index>
* @g2 </member>
* @g2 <member>
* @g2 <member-name>city</member-name>
* @g2 <member-type>TEXT</member-type>
* @g2 </member>
* @g2 <member>
* @g2 <member-name>byline</member-name>
* @g2 <member-type>TEXT</member-type>
* @g2 </member>
* @g2 <member>
* @g2 <member-name>objectname</member-name>
* @g2 <member-type>TEXT</member-type>
* @g2 </index>
* @g2 </member>
* @g2 </map>
*/

/**
* Return the target properties for the given view mode
*
* @param int the view mode (EXIF_SUMMARY, etc)
* @return array object GalleryStatus a status code
* array logical exif property names
* @static
*/
function getProperties($itemId) {
global $gallery;

$query = '
SELECT
[IptcItemMap::itemId], [IptcItemMap::profile], [IptcItemMap::field], [IptcItemMap::value]
FROM
[IptcItemMap]
WHERE
[IptcItemMap::itemId] = ?
ORDER BY
[IptcItemMap::field] ASC';
list ($ret, $searchResults) = $gallery->search($query, array($itemId));
if ($ret->isError()) {
return array($ret->wrap(__FILE__, __LINE__), null);
}

$data = array();
while ($result = $searchResults->nextResult()) {
$data[] = $result[0];
}

return array(GalleryStatus::success(), $data);
}

/**
* Set the target properties for the given item ID
*
* @param int the item ID
* @param array logical property key/value pairs
* @return object GalleryStatus a status code
* @static
*/
function setProperties($itemId, $properties) {

/* Remove all old map entries */
$ret = ExifPropertiesMap::removeMapEntry(array('itemID' => $itemId));
if ($ret->isError()) {
return $ret->wrap(__FILE__, __LINE__);
}

for ($i = 0; $i < sizeof($properties); $i++) {
$ret = ExifPropertiesMap::addMapEntry(array('property' => $properties[$i], 'itemId' => $itemId));
if ($ret->isError()) {
return $ret->wrap(__FILE__, __LINE__);
}
}

return GalleryStatus::success();
}
}

include(dirname(__FILE__) . '/interfaces/IptcItemMap.inc');
?>

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Wed, 2005-10-12 23:50

first, your map doesn't make much sense.
what does it map? there's no itemId.

also, there's neither a primary key nor a unique key.

and take a look at other maps. the 2 functions are not relly necessary. they are just there in the original exif property map since it's convenient there.

finally, your gmake issues probably have nothing to do with your actual map. probably, the gmake issues just have to do with the generate-*.php scripts which aren't mature enough.

 
jrmint

Joined: 2005-08-20
Posts: 94
Posted: Thu, 2005-10-13 00:02

I knew I was doing this wrong.
I am learning this stuff as I go. as you can see..

What would you use to put the IPTC fields into the g2_item table.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2005-10-13 00:06

no, the general idea of your map table is fine, but there should also be a member "itemId".
also see my other comments.

your solution would be a specialized case of the solution i suggested. what i described would be for arbitrary exif / iptc fields/properties. your solution would only cover 4 different iptc fields. that's fine, for you. but i think it wouldn't find its way into the official g2 code that way. it's too specialized for your own purposes.

 
jrmint

Joined: 2005-08-20
Posts: 94
Posted: Thu, 2005-10-13 00:20

I tryed this and I got an error Notice: Undefined index:

<?php
/**
* @g2 <class-name>IptcItemMap</class-name>
* @g2 <schema>
* @g2 <schema-major>1</schema-major>
* @g2 <schema-minor>0</schema-minor>
* @g2 </schema>
*
* @package Exif
* @subpackage Classes
*/

class IptcItemMap_core {

/**
* @g2 <map>
* @g2 <member>
* @g2 <member-name>itemId</member-name>
* @g2 <member-type>INTEGER</member-type>
* @g2 <member-size>LARGE</member-size>
* @g2 </member>
* @g2 <member-name>headline</member-name>
* @g2 <member-type>TEXT</member-type>
* @g2 </indexed>
* @g2 </member>
* @g2 <member>
* @g2 <member-name>city</member-name>
* @g2 <member-type>TEXT</member-type>
* @g2 </member>
* @g2 <member>
* @g2 <member-name>byline</member-name>
* @g2 <member-type>TEXT</member-type>
* @g2 </member>
* @g2 <member>
* @g2 <member-name>objectname</member-name>
* @g2 <member-type>TEXT</member-type>
* @g2 </indexed>
* @g2 </member>
* @g2 <key>
* @g2 <member-name>itemId</member-name>
* @g2 <member-name>headline</member-name>
* @g2 <member-name>city</member-name>
* @g2 <member-name>byline</member-name>
* @g2 <member-name>objectname</member-name>
* @g2 </key>
* @g2 </map>
*/
}
include(dirname(__FILE__) . '/interfaces/IptcItemMap.inc');
?>

 
jrmint

Joined: 2005-08-20
Posts: 94
Posted: Thu, 2005-10-13 00:20

heres the return i got

cd interfaces && make -
make[1]: Entering directory `/srv/www/htdocs/gallery2/modules/exif/classes/interfaces'
perl ../../../../lib/tools/bin/extractClassXml.pl --dtd=../../../../../lib/tools/dtd/GalleryClass2.0.dtd --stub-ok --quiet --out-dir=tmp ../*.class
php ../../../../lib/tools/bin/generate-interfaces.php Exif

Notice: Undefined index: content in /srv/www/htdocs/gallery2/lib/tools/bin/generate-interfaces.php on line 75
XML error: mismatched tag at line 17make[1]: Leaving directory `/srv/www/htdocs/gallery2/modules/exif/classes/interfaces'
cd GalleryStorage/DatabaseStorage/schema && make -
make[1]: Entering directory `/srv/www/htdocs/gallery2/modules/exif/classes/GalleryStorage/DatabaseStorage/schema'
perl ../../../../../../lib/tools/bin/extractClassXml.pl --dtd=../../../../../../lib/tools/dtd/GalleryClass2.0.dtd --quiet --out-dir=tmp/classxml ../../../*.class
php -f ../../../../../../lib/tools/bin/generate-dbxml.php
XML error: mismatched tag at line 17php -f ../../../../../../lib/tools/bin/generate-sql.php tmp/classxml
make[1]: Leaving directory `/srv/www/htdocs/gallery2/modules/exif/classes/GalleryStorage/DatabaseStorage/schema'

 
jrmint

Joined: 2005-08-20
Posts: 94
Posted: Thu, 2005-10-13 00:23
Quote:
your solution would be a specialized case of the solution i suggested. what i described would be for arbitrary exif / iptc fields/properties. your solution would only cover 4 different iptc fields. that's fine, for you. but i think it wouldn't find its way into the official g2 code that way. it's too specialized for your own purposes.

Once I know what we are working on works. I will add the other fields into the module. I am just using a few as a test. I need more than just these 4.

 
outpostbabu

Joined: 2005-02-26
Posts: 15
Posted: Thu, 2005-10-13 02:09

Just catching up here, about to try Valiant's revised map class on my g2 install. It sounds like we could all benefit from being on the same page and working towards the same goal. Originally I only cared about getting GPS data into custom fields but obviously, as valiant has pointed out, the effort is the same to store arbitrary IPTC/EXIF fields so to make this useful for everybody, we should pursue that. I will post my results shortly. I'm glad there is so much interest in this feature.

 
outpostbabu

Joined: 2005-02-26
Posts: 15
Posted: Thu, 2005-10-13 02:42

I just cvs'ed a fresh checkout of gallery 2 to try valiant's ExifPropertyValuesMap.class. After running through the standard install procedure and making sure G2 was functional, I copied the map class into modules/exif/classes and ran gmake with this result:

$ gmake
cd interfaces && gmake -
gmake[1]: Entering directory `/cvs/gallery2/modules/exif/classes/interfaces'
perl ../../../../lib/tools/bin/extractClassXml.pl --dtd=../../../../../lib/tools/dtd/GalleryClass2.0.dtd --stub-ok --quiet --out-dir=tmp ../*.class
php ../../../../lib/tools/bin/generate-interfaces.php Exif
Content-type: text/html
X-Powered-By: PHP/4.4.0

<br />
<b>Warning</b>:  main(../../../../lib/smarty/Smarty.class.php): failed to open stream: No such file or directory in <b>/cvs/gallery2/lib/tools/bin/generate-interfaces.php</b> on line <b>29</b><br />
<br />
<b>Fatal error</b>:  main(): Failed opening required '../../../../lib/smarty/Smarty.class.php' (include_path='.:/usr/local/share/pear:/usr/local/lib/php') in <b>/cvs/gallery2/lib/tools/bin/generate-interfaces.php</b> on line <b>29</b><br />
gmake[1]: *** [inc] Error 255
gmake[1]: Leaving directory `/cvs/gallery2/modules/exif/classes/interfaces'
gmake: *** [all] Error 2

The error is because the relative directory goes up one directory too far. So I changed '../../../../lib/smarty/Smarty.class.php' to '../../../lib/smarty/Smarty.class.php' in both files. Then gmake ran fine:

$ gmake
cd interfaces && gmake -
gmake[1]: Entering directory `/cvs/gallery2/modules/exif/classes/interfaces'
perl ../../../../lib/tools/bin/extractClassXml.pl --dtd=../../../../../lib/tools/dtd/GalleryClass2.0.dtd --stub-ok --quiet --out-dir=tmp ../*.class
php ../../../../lib/tools/bin/generate-interfaces.php Exif
Content-type: text/html
X-Powered-By: PHP/4.4.0

gmake[1]: Leaving directory `/cvs/gallery2/modules/exif/classes/interfaces'
cd GalleryStorage/DatabaseStorage/schema && gmake -
gmake[1]: Entering directory `/cvs/gallery2/modules/exif/classes/GalleryStorage/DatabaseStorage/schema'
perl ../../../../../../lib/tools/bin/extractClassXml.pl --dtd=../../../../../../lib/tools/dtd/GalleryClass2.0.dtd --quiet --out-dir=tmp/classxml ../../../*.class
php -f ../../../../../../lib/tools/bin/generate-dbxml.php
php -f ../../../../../../lib/tools/bin/generate-sql.php tmp/classxml
gmake[1]: Leaving directory `/cvs/gallery2/modules/exif/classes/GalleryStorage/DatabaseStorage/schema'

At least that looks fine to me. I don't really know what to expect. But the result is the same as before. There is no ExifPropertyValuesMap.inc in interfaces and no database was created. So I am in essentially the same position as jrmint.

 
jrmint

Joined: 2005-08-20
Posts: 94
Posted: Thu, 2005-10-13 13:35

I think we are fighting a losing battle. I am looking at how the core built its GalleryItem and just for kicks I made a back up of the core module,
and ran the make on the GalleryItem.class and it didn't work either. It came up with errors
Notice: Undefined index: content in /srv/www/htdocs/gallery2/lib/tools/bin/generate-interfaces.php on line 75

Has any body looked at the GNUmakefile to see if they are updated with the right codes for these nightly snap shots?

I hope I am wrong. but it looks line the GNUmakefile is our problem.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2005-10-13 14:00

sorry that we can't solve this immediately. I'm still hoping for assistance by bharat, since he's the author of the generate-*.php scripts.
Thanks for your patience. I'm sure we get this fixed soon.

 
jrmint

Joined: 2005-08-20
Posts: 94
Posted: Thu, 2005-10-13 14:12

No Problem. I really appreciate the help you are giving us.

I am NOT giving up on this I will keep pluging away at doing differant things.

Personally I think all of this would be taken care of if we just changed the GalleryItem.class sence that core class is doing everything in that table any way.
And the exif module is just using what is already in the core GalleryItem.inc.

I do understand the whole bit about leaving the core files alone and doing this with a new or extended exif module.

I still have a few things to try that I have running threw my mind. Like copying the code from the GalleryItem.class and tweaking it in the a new class file in the
exif module. ext ext.

 
outpostbabu

Joined: 2005-02-26
Posts: 15
Posted: Fri, 2005-10-14 00:04

I'm confident that its a minor hurdle to getting this feature implemented. I'll keep poking at it to see if it moves. As jrmint said, many thanks for the help you've given us thus far.

 
outpostbabu

Joined: 2005-02-26
Posts: 15
Posted: Mon, 2005-10-17 01:00

Looking through the GNUMakefile, I don't see how it can create the appropriate .inc file in interfaces:

all:
        cd interfaces && $(MAKE) -$(MAKEFLAGS)
        cd GalleryStorage/DatabaseStorage/schema && $(MAKE) -$(MAKEFLAGS)

%:
        cd interfaces && $(MAKE) -$(MAKEFLAGS) $@
        cd GalleryStorage/DatabaseStorage/schema && $(MAKE) -$(MAKEFLAGS) $@

It looks like it just goes into the interfaces file and "compiles" whatever is in there. It doesn't look like it does anything at all with the actual generation of the .inc file such as ExifPropertiesMap.inc in interfaces.
What am I missing? Also, if you update your checkout from cvs, how do you "recompile" the whole of Gallery 2? Do you have to run through the install again?

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Mon, 2005-10-17 10:36

sorry for the wait, but you have to be a little more patient :/

@make:
it sure works.
see:

Quote:
tux classes # cd interfaces/
tux interfaces # less GNUmakefile
GALLERYDIR=../../../..
XSLFILE=$(GALLERYDIR)/modules/core/classes/interfaces/ClassCode.xsl
PACKAGE=Exif

include $(GALLERYDIR)/modules/core/classes/interfaces/GNUmakefile

modules/core/classes/interfaces/GNUmakefile is:

Quote:
TOOLDIR ?= ../../../../lib/tools
CLASSDIR ?= ..
PACKAGE ?= GalleryCore
G2_TMPDIR ?= tmp

inc: $(XMLFILES)
@if [ ! -d $(G2_TMPDIR) ]; then mkdir $(G2_TMPDIR); fi
perl $(TOOLDIR)/bin/extractClassXml.pl --dtd=../$(TOOLDIR)/dtd/GalleryClass2.0.dtd --stub-ok
--quiet --out-dir=$(G2_TMPDIR) $(CLASSDIR)/*.class
php $(TOOLDIR)/bin/generate-interfaces.php $(PACKAGE)

clean:
rm -rf $(G2_TMPDIR)

scrub: clean
rm -f *.inc

# Gmake will automatically delete $(G2_TMPDIR)/*.xml files after creating .inc files
# because it thinks that they're intermediate files. But, we want to save
# them (for now), so mark them as PRECIOUS.
#
.PRECIOUS: $(G2_TMPDIR)/%.xml

so it does a lot more than you think :)
maybe the scripts aren't robust enough yet since we (bharat) changed it since september 12th.
until 2.0, you needed java, saxon and rxp, plus perl, to generate the interface and sql files.
now, you only need php and perl. but for some reason it doesn't work on all machines yet.

 
bharat
bharat's picture

Joined: 2002-05-21
Posts: 7994
Posted: Thu, 2005-10-20 07:55

Sorry for the long delay in getting back to you guys. I realize that this is a pain to get off the ground so I spent an hour or two and created a new tool that should help. You'll need to update to the latest code from CVS or a snapshot then run php lib/tools/creator/create-module.php and answer the questions it asks you. It will create a new module with a map that has an itemId<-->itemValue mapping that lets you store a string for any item in your Gallery 2. Once the module is installed, just select the "Hello <...>" link from the action menu under any photo and give it a shot.

I literally finished pounding this out five minutes ago so I'm sure there are bugs. But let's see if this gets you up and running quickly!

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2005-10-20 10:37

bharat, this is a great idea. based on that we can expand the wizard etc. cool :)

but unless you have changed the existing xml parsing scripts etc. it probably won't help these two guys.

e.g. jrmint gets this output on gmake:

cd interfaces && make -
make[1]: Entering directory `/srv/www/htdocs/gallery2/modules/exif/classes/interfaces'
perl ../../../../lib/tools/bin/extractClassXml.pl --dtd=../../../../../lib/tools/dtd/GalleryClass2.0.dtd --stub-ok --quiet --out-dir=tmp ../*.class
php ../../../../lib/tools/bin/generate-interfaces.php Exif

Notice: Undefined index: content in /srv/www/htdocs/gallery2/lib/tools/bin/generate-interfaces.php on line 75
XML error: mismatched tag at line 17make[1]: Leaving directory `/srv/www/htdocs/gallery2/modules/exif/classes/interfaces'
cd GalleryStorage/DatabaseStorage/schema && make -
make[1]: Entering directory `/srv/www/htdocs/gallery2/modules/exif/classes/GalleryStorage/DatabaseStorage/schema'
perl ../../../../../../lib/tools/bin/extractClassXml.pl --dtd=../../../../../../lib/tools/dtd/GalleryClass2.0.dtd --quiet --out-dir=tmp/classxml ../../../*.class
php -f ../../../../../../lib/tools/bin/generate-dbxml.php
XML error: mismatched tag at line 17php -f ../../../../../../lib/tools/bin/generate-sql.php tmp/classxml
make[1]: Leaving directory `/srv/www/htdocs/gallery2/modules/exif/classes/GalleryStorage/DatabaseStorage/schema'

@jrmint:
please CVS update -dPC to get the new files and try gmake again. if it doesn't work, create a zip file of your (exif) module and i'll try gmake for the same module on my computer.
we tried this before based on the single file you edited, but let's try it again / make sure your source files are ok.

 
jrmint

Joined: 2005-08-20
Posts: 94
Posted: Thu, 2005-10-20 12:32

Thanks for all your work bharat and valiant.

I will look into this as soon as I get a chance.

FYI I don't know how much futher I can go with this project tho. With out a fulltext/boolean search I can't use this program.

I haven't giving up on the project just looking at other possibilities.

 
bharat
bharat's picture

Joined: 2002-05-21
Posts: 7994
Posted: Thu, 2005-10-20 18:33

Gallery2 will have fulltext boolean search, probably in the 2.1 release. The main problem is figuring out the proper way to cross-database support of creating the indexes and doing the matching.

The advantage of the sample module generator is that it creates something that should definitely work (barring some slight bugs). If that code gives you the same error then we should figure out what's wrong with your environment. Otherwise, we should be trying to figure out what's wrong with the code that you wrote (so zipping up what you have like valiant suggests will probably help us figure this one out).

 
outpostbabu

Joined: 2005-02-26
Posts: 15
Posted: Tue, 2005-10-25 00:38

Thanks for the help bharat. I only had a short time to play with the new script. I got a number of warnings when I ran it, which I know are because of my php environment but the script appeared to work otherwise. I still had to update the relative paths for Smarty (as I described in another post above) but after that I could see the module in the site admin and I could install and activate it.
It also appeared as an item on the list under each image. However if I select it I get the following:


Error Detail -
Error (ERROR_STORAGE_FAILURE)

    * in modules/core/classes/GalleryStorage/DatabaseStorage.class at line 305 (gallerystatus::error)
    * in modules/core/classes/GalleryStorage.class at line 250 (mysqldatabasestorage::search)
    * in modules/core/classes/Gallery.class at line 223 (gallerystorage::search)
    * in modules/hellooutpostbabu/classes/MyPageHelper.class at line 58 (gallery::search)
    * in modules/hellooutpostbabu/MyPage.inc at line 97 (mypagehelper::getitemvalue)
    * in modules/core/classes/GalleryTheme.class at line 689 (mypageview::loadtemplate)
    * in modules/core/classes/GalleryView.class at line 312 (matrixtheme::loadtemplate)
    * in main.php at line 335 (mypageview::doloadtemplate)
    * in main.php at line 88
    * in main.php at line 81

System Information
Gallery version 	2.0.1+
PHP version 	4.4.0 apache
Webserver 	Apache
Database 	mysql 4.0.18-standard-log
Toolkits 	ImageMagick, NetPBM, Gd
Operating system 	Linux 2.4.21-20.0.1.ELsmp #1 SMP Wed Nov 24 20:34:01 EST 2004 i686
Browser 	Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8b5) Gecko/20051006 Firefox/1.4.1

I checked the database and there is no new table for the module, so I assume thats the issue. I've included the system information above. Maybe that will help. Thanks for the continuing support with this.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2005-10-25 03:59

after gmake etc. you need to uninstall and reinstall the module in question (site admin -> modules ).
only installing a module creates new tables.

(alternatively, increase the module version, then you can just upgrade the module in site admin -> modules).