Larry,
The easiest (and most user-friendly) CVS software for windows is http://www.tortoisecvs.org/index.shtml. It's also integrated with Windows explorer. You don't need python, and there are no geeky details that are unneccessary.
Building, however, used to be somewhat problematic on Windows - but I think that's what Bharat is fixing right now...
Good luck!
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Thu, 2005-10-06 04:06
Hi, Virshu.
OK I've installed TortoiseCVS, but seem to be unable to reach the CVS server for Gallery. I'm using:
But then it just incessantly keeps prompting me for my password. I keep entering it, and it keeps prompting me for it.
When I cancel, here are the messages:
Quote:
In C:\My Server\gallery2\lib\tools\bin: "C:\Program Files\TortoiseCVS\cvs.exe" "-q" "checkout" "-P" "lib/tools/bin/generate-sql.php"
CVSROOT=:ext:larry_menard@cvs.sourceforge.net:/cvsroot/gallery
cvs.exe checkout: warning: unrecognized response `Access denied' from cvs server
cvs.exe [checkout aborted]: end of file from server (consult above messages if any)
Error, CVS operation failed
What am I missing?
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Thu, 2005-10-06 04:18
After reading further, I see that SSH is used only by authorized developers, for which I doubt I qualify.
So I've switched to protocol ":pserver" and user name "anonymous", and module "lib/tools/bin/generate-sql.php", but it tells me that it couldn't find that module.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Thu, 2005-10-06 04:25
Yay, I figured out that the module name needed a "gallery2/" at the front. I think I've now checked out "gallery2/lib/tools/bin/generate-sql.php".
I guess now I need to know what specific files I need to grab.
bharat
Joined: 2002-05-21
Posts: 7994
Posted: Thu, 2005-10-06 05:17
The general shape of the command that you want is:
That will check out *all* of the code. This is the right thing to do because it means that your entire G2 will be drawn from the CVS repository which means that when you want to get the latest code you just do "cvs update" and it does the rest for you. If you want to give us a patch to apply, you do "cvs diff" and it'll tell you what's changed in your version vs. what's in revision control.
bharat
Joined: 2002-05-21
Posts: 7994
Posted: Thu, 2005-10-06 05:27
FYI, I just committed one round of changes that simplifies the SQL generation code. I got rid of the makefiles in xml-src and platform, now there's only one in the schema directory and it should do everything. Because of the way that SourceForge is set up, it takes ~6 hours for anything we commit to show up in the anonymous repository (which you're drawing from).
So the way to figure out which version you have is to do "cvs status generate-sql.php". If you have revision 1.10, then copy over the prior version of generate-sql.php that posted. If you have revision 1.11 then copy over this version instead: http://www.menalto.com/.outgoing/gallery/generate-sql-2.php.txt
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2005-10-06 09:21
as you're using cygwin, you could just use the linux style cvs commands there.
and if you want to stick to windows applications, i also use http://www.tortoisecvs.org/index.shtml on my windows box. once configured, cvs is a matter of clicking a few buttongs
The files in "platform/db2" do seem to be created, so I'll try proceeding anyway. FYI, I also see that an empty directory for each platform has also been created under "schema". Perhaps this is harmless, perhaps even required... I'll leave them and carry on.
I then copied "GalleryStorage/DatabaseStorage/OracleDatabaseStorage.class" to "GalleryStorage/DatabaseStorage/Db2DatabaseStorage.class" and modified it for DB2.
I then added the DB2 particulars to "install/steps/DatabaseSetupStep.class" and "install/config.php-template".
I then added a DB2 section to "modules/core/classes/DatabaseStorage.class".
I then copied files "drivers/adodb-db2.inc.php" and "drivers/adodb-odbc.inc.php" from the "adodb" package into "lib/adodb/drivers".
Now I boldly fire up the installer.
In the "System Check" step, I see 19 modified files. I won't bother listing them here.
Something odd: As soon as the "Database Setup" page loaded it had error "You must have the MySQL PHP module installed", even though I hadn't entered anything yet.
Charging ahead, I entered the specifics for my DB2 database, but that resulted in error:
Quote:
Fatal error: Call to undefined method stdClass::NConnect() in C:\My Server\gallery2\install\steps\DatabaseSetupStep.class on line 119
Any ideas what might cause these database setup errors?
Thanks.
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Thu, 2005-10-06 20:38
@my last post: sorry didn't read the 2nd page of this topic before.
bharat
Joined: 2002-05-21
Posts: 7994
Posted: Fri, 2005-10-07 07:06
That looks like progress to me! I shadowed everything you did on my Windows box and got a little further than you do. I'm at the limit of how far I can go because I don't have ODBC and DB2 handy. But a few things:
1. I fixed the "xmllint: command not found" bug. cvs up and you'll get that fix.
2. I fixed the "empty directory for each db under the schema directory" bug. Typo on my part. cvs up for that too.
3. The "You must have the MySQL PHP module installed" is actually a bug, but we hit it incredibly rarely beacause almost everybody has mysql installed! Fixed.
None of these issues are critical, so you don't have to CVS up right away to fix them.
I reproduced the problem you were experiencing and realized that it's because I wasn't setting the $dbPlatformType to 'db2' in DatabaseSetupStep, so adodb was using the default connector which doesn't work. I've put my tweaked copy of that file here:
Try that instead. With that, I now get this error:
Quote:
For odbc Connect(), gallery2 is not used. Place dsn in 1st parameter.
Warning: odbc_connect() [function.odbc-connect]: SQL error: [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified, SQL state IM002 in SQLConnect in C:\cygwin\usr\local\gallery2\lib\adodb\drivers\adodb-odbc.inc.php on line 60
localhost: Connection error to server 'localhost' with user 'root'
So that's good news; it's using ODBC correctly and trying to connect. But I don't know the right values to provide, etc to make that happen. You probably know a lot more than me at this point. I took a quick look at the odbc_connect() call in adodb-odbc.inc.php and see that it's expecting to call:
This implies to me that instead of providing the name of the database (I used "gallery2") I need to provide a full DSN. Let me know how that goes for you.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Fri, 2005-10-07 16:11
Hi, Bharat.
Thanks for the fixes.
Yup, I got there at one point too, a couple of changes ago . There's a weirdness with DB2 via ODBC, you need to specify the parameters to odbc_connect in a different order. What Valiant and I discussed at one time was entering the database name in the "DB Hostname" field. I tested it and it works.
However... I've found an extension to PHP, called "ibm_db2". It's based on DB2's "Call Level Interface" (or "CLI"), DB2's own version of ODBC, which is apparently much better than unified ODBC. Plus, I used to work with the developer of the 'ibm_db2' extension, Dan Scott.
So I'm planning to switch from ODBC to the 'ibm_db2' PHP extension... I think it'll help in a couple of ways.
Will let you know how it goes.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Fri, 2005-10-07 16:49
Bharat,
I downloaded the "DatabaseSetupStep.class" that you pointed to, and I still get the same NConnect error (not migrated from ODBC to ibm_db2 yet). From what I can see, your file is the same as mine except that you've changed the order of the platforms in a couple of places... the code itself is the same.
Then, I changed the 'odbc_connect' to 'db2_connect' (the correct call according to the 'ibm_db2' doc), and now it says it can't find that function. I have copied the 'php_ibm_db2.dll' to my PHP directory and pointed to it in my 'php.ini' ('extension=php_ibm_db2.dll'). Any idea why it can't find that function?
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Fri, 2005-10-07 16:56
how do you plan to do this "switch" to ibm_db2?
is it supported by adodb?
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Fri, 2005-10-07 17:34
I didn't know it had to explicitly support it. I doubt it does. Oh well, back to ODBC.
but after that I'm no longer able to 'gmake'... lots of errors like:
Quote:
$ gmake 2>&1 | tee gmake.out
perl ../../../../../../lib/tools/bin/extractClassXml.pl --dtd=../../../../../../
lib/tools/dtd/GalleryClass2.0.dtd --quiet --out-dir=tmp/classxml ../../../*.class
php -f ../../../../../../lib/tools/bin/generate-dbxml.php
PHP Notice: Undefined index: TMP in c:\My Server\gallery2\lib\tools\bin\generate-dbxml.php on line 30
PHP Notice: Undefined index: content in c:\My Server\gallery2\lib\tools\bin\generate-dbxml.php on line 69
PHP Notice: Undefined variable: package in c:\My Server\gallery2\lib\tools\bin\generate-dbxml.php on line 171
PHP Notice: Undefined index: content in c:\My Server\gallery2\lib\tools\bin\generate-dbxml.php on line 69
PHP Notice: Undefined variable: package in c:\My Server\gallery2\lib\tools\bin\generate-dbxml.php on line 171
So I completely released my build tree, am getting a whole new copy, and will try yet again.
When I volunteered for this, I thought it might take a day or so to just plunk some SQL in somewhere. Hmmm...
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Fri, 2005-10-07 17:43
@ibm_db2:
well, we use adodb as our db abstraction layer, all db queries go through adodb. if adodb doesn't use the ibm_ calls, we can't use ibm_ calls...
of course you could send a a modified adodb-db2 driver file to adodb, maybe they'll add it to the distribution.
maybe it supports ibm_db2, you could find out by grepping (grep -ri "ibm_" *) in the adodb drivers/ directory of the latest adodb release (not in g2's adodb/drivers directory, since adodb might have, but we may not include it).
@cvs update:
you have changed some files. if you then do cvs update -P, it doesn't replace the changed files. depending on a few factors, cvs auto-merges your local changes with the new version of the file from the cvs repository, or it adds <<< >>> stuff in the files to mark where there are differences or it doesn't touch your changed files at all.
so this might explain your problems.
copy your changed files to a save place, then use
cvs -z3 update -dPC
-z3 for compression, speeds up the whole transmission
-C to replace your changed files with new versions from the cvs repository. thus you'll lose all your changes. but it renames your locally changed files to something like #oldfilename.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Fri, 2005-10-07 17:47
Even with brand new build tree (and no DB2 support added), same errors in 'gmake'. Might they be because of a 'php.ini' setting?
bharat
Joined: 2002-05-21
Posts: 7994
Posted: Fri, 2005-10-07 20:38
Ok, I raised the warning level to something that matches what you're using, and fixed all the warnings that it generates. This last batch was just warnings though, it shouldn't have prevented any SQL generation.
Are you still getting the NConnect error? You shouldn't be anymore with your code (or the version I provided). From looking at the DB2 example here: http://phplens.com/lens/adodb/docs-adodb.htm
It looks like you need to pass in a DSN definition as the first argument, and the username, password, etc can be empty. I tried it with this DSN:
Warning: odbc_connect() [function.odbc-connect]: SQL error: [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified, SQL state IM002 in SQLConnect in C:\cygwin\usr\local\gallery2\lib\adodb\drivers\adodb-odbc.inc.php on line 60
driver={IBM db2 odbc DRIVER};Database=sample;hostname=localhost;port=50000;protocol=TCPIP;uid=root; pwd=secret: Connection error to server 'driver={IBM db2 odbc DRIVER};Database=sample;hostname=localhost;port=50000;protocol=TCPIP;uid=root; pwd=secret' with user ''
That looks promising to me. Looks like it's actually parsing the DSN correctly and trying to do the right thing with it which means that G2 isn't really in your way at this point.
If you can get it to connect then the first thing that it's going to try to do is to run the T_InstallerTest_1.sql .. T_InstallerTest_4.sql generated sql files. So this is where you can start tweaking generate-sql.php to actually build out the right SQL for DB2.
Quote:
When I volunteered for this, I thought it might take a day or so to just plunk some SQL in somewhere. Hmmm...
I know what you mean. It's more complex than it seems at first. The big downside to doing it this way is that getting support for a new database requires a steep learning curve at first. The upside to doing it this way is that once we've gotten the transforms working properly, we rarely need to touch the SQL generation code again even as we continuously make small changes to the database schema.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Fri, 2005-10-07 21:12
When they say to use dsn as the first parm, I believe they mean "database name", not a full "datasource name". Since G2 (or whoever) passes the hostname as the first parm, put the database name "gallery2" in the hostname field. That worked for me; at least the connect didn't fail, but then I got in trouble with the "InstallTest" file missing.
Like I said above:
Quote:
I downloaded the "DatabaseSetupStep.class" that you pointed to, and I still get the same NConnect error (not migrated from ODBC to ibm_db2 yet). From what I can see, your file is the same as mine except that you've changed the order of the platforms in a couple of places... the code itself is the same.
Am I mistaken? Other than the order of the plaforms, what else did you change in it? (The Windows CRLFs in my file make "diff" pointless.) I'm very confused as to why you're telling me to use essentially the same file as my own.
In the meantime, I'll re-add my DB2/ODBC support into my new build and see how far I get now.
And FYI... Dan Scott tells me that ADOdb package should support the 'ibm_db2' PHP extension soon, perhaps in a couple of weeks.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Fri, 2005-10-07 21:59
OK, I think I'm on a roll. By putting the database name in the "hostname" field, it does successfully connect to the database. I'm now working on fixing up some syntax errors in the generated SQL.
This is where I thought I'd be 4 days ago.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Fri, 2005-10-07 23:28
Big Problem.
Quote:
CREATE INDEX g2dbtest0_Schema_80114 ON g2dbtest0_Schema(g_testCol, g_STRING, g_MEDIUM)
SQL0107N The name "G2DBTEST0_SCHEMA_80114" is too long. The maximum length is "18". SQLSTATE=42622
From what I've seen of the index names in G2, this is going to be a very pervasive problem. I believe that even some column names may exceed the DB2 limit of 30 characters, but I'm not sure of that yet.
Suggestions?
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Sat, 2005-10-08 09:44
nope that shouldn't be a problem.
so you know now that indices must be shorter than 19 characters.
in generate-sql.php
class Db2Generator extends BaseGenerator {
function createSql
around (2 occurrences):
case 'INDEX':
you define the index name. you can choose a diffferent pattern to assemble the index name, you don't have to follow the pattern "prefixTableName_CRC". you can also choose other patterns.
i would include the CRC and maybe instead of the rest, you should also find something unique, but it shouldn't be that long. maybe a truncated crc for the tablename?
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Sat, 2005-10-08 14:05
OK, cool. I was concerned that you would be concerned with cross-platform schema compatablity.
Thanks.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Sat, 2005-10-08 14:37
Possibly a stupid question...
I'm not sure what "CRC" you mean. The CRC of what? I know how to get the CRC value of a file, but, I don't see how that fits in the context of your response.
with something that is shorter.
i'd keep the second part, ('_' . $this->getIndexCrc($columns);[/code]) since it's quite short and is quite important.
what we want is to make sure 'DB_TABLE_PREFIX' . $parent['child'][0]['content'] isn't longer than 12 characters.
so what you could do is
a) omit DB_TABLE_PREFIX and just make it $parent['child'][0]['content']. but this wouldn't fix it, since $parent['child'][0]['content'] is likely longer than 6 chars.
b) compute crc32($parent['child'][0]['content']) and take the last 6 digits of it.
why? well, the resulting index name wouldn't be human readable, but we need a index name that is unique among all indices in the database, right?
i'd use
--------------------------------------------------------------------------------
(db2): ALTER TABLE g2dbtest0_Schema ADD COLUMN g_testCol VARCHAR(128)
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
(db2): CREATE INDEX G2_I_23747_80114 ON g2dbtest0_Schema(g_testCol, g_STRING, g_MEDIUM)
--------------------------------------------------------------------------------
Warning: odbc_exec() [function.odbc-exec]: SQL error: [IBM][CLI Driver][DB2/NT] SQL0205N Column or attribute "G_STRING" is not defined in "LMENARD.G2DBTEST0_SCHEMA". SQLSTATE=42703 , SQL state S0022 in SQLExecDirect in C:\My Server\gallery2\lib\adodb\drivers\adodb-odbc.inc.php on line 530
S0022: [IBM][CLI Driver][DB2/NT] SQL0205N Column or attribute "G_STRING" is not defined in "LMENARD.G2DBTEST0_SCHEMA". SQLSTATE=42703
ADOConnection._Execute(CREATE INDEX G2_I_23747_80114
ON g2dbtest0_Schema(g_testCol, g_STRING, g_MEDIUM), false) % line 855, file: adodb.inc.php
ADOConnection.Execute(CREATE INDEX G2_I_23747_80114
ON g2dbtest0_Schema(g_testCol, g_STRING, g_MEDIUM)) % line 421, file: DatabaseSetupStep.class
DatabaseSetupStep._executeSqlFile(C:\My Server\gallery2\install\steps\..\..\modules\core\classes\GalleryStorage\DatabaseStorage\schema\platform\db2\T_InstallerTes..., g2dbtest0_) % line 359, file: DatabaseSetupStep.class
DatabaseSetupStep._testPrivileges(db2, Array[0]) % line 145, file: DatabaseSetupStep.class
DatabaseSetupStep.loadTemplateData(Array[1]) % line 167, file: index.php
I have no idea where these incorrect column names are coming from. Any pointers?
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Sat, 2005-10-08 17:48
hmm, wait a second: you should use the DB_TABLE_PREFIX for all index names, since someone could want to install multiple G2 installs in the same database, but with different prefices.
@error:
obviously your CREATE INDEX SQL generation isn't ok. g_STRING, g_MEDIUM shouldn't be part of this line.
see the corresponding code in T_InstallerTest_2.sql in mysql/ or postgresql/
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Sat, 2005-10-08 18:45
Well if the DB_TABLE_PREFIX stays, then the rest of the index name needs to be shortened accordingly.
Like I said in my original posting, I am not a hardcore developer. 99% of what I know of PHP I've learned in the last week. So, I'm sorry, but with all due respect, I don't find your answer "check the code" that helpful. Trust me, I'm even more frustrated with this than you are.
I believe these to be fine, but the references to "MEDIUM" and "STRING" in the generated SQL make me suspect something isn't parsing these properly. But I can't tell where that is done.
I checked the Oracle, MySQL, and PostgreSQL files; the latter two generate the index only on the 'g_testcol' column, but Oracle is similarly afflicted. (I based my changes on the Oracle code, so if there was a bug in it, I'd have it too. For example, I did find one earlier SQL syntax problem that I wonder how it worked on Oracle either... no '\n;\n\n' between the ADD COLUMN and the CREATE INDEX.)
Also, I'm still getting lots of the following messages when I do 'gmake'. I thought Bharat said that the SQL should still be generated, but is it possible they're related to the generated SQL being bad?
Quote:
PHP Notice: Undefined index: TMP in c:\My Server\gallery2\lib\tools\bin\generate-dbxml.php on line 30
PHP Notice: Undefined index: content in c:\My Server\gallery2\lib\tools\bin\generate-dbxml.php on line 69
PHP Notice: Undefined variable: package in c:\My Server\gallery2\lib\tools\bin\generate-dbxml.php on line 171
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Sat, 2005-10-08 19:51
yep, you're right. oracle's code is also bugged. that explains your issue.
again, i'm sorry for all these unnecessary issues. you just happen to have chosen a time when we chose to restructure this code :/
but oracle is possibly the most untested code. so basing your changes on postgresql should be a more reliable choice.
i'm very sure that noone tested the generated oracle sql code that we have now. it's only a week old (cvs) and the handful of oracle users we have use G 2.0.
my advice is:
base your DB2 class on the postgresql class (bharat mainly develops g2 on/for postgresql).
i'm sure he'll be able to address the remaining issues soon.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Sat, 2005-10-08 20:22
Just my luck. I figured Oracle would be more similar to DB2 in terms of functionality provided.
On the other hand, I'm glad to hear that Oracle apparently isn't popular in your client base. I worked in the DB2 development organization for 12 years, and in IBM in general for 25 years, so Oracle is considered one of 'The Competition'.
OK, I'll re-implement my changes, based this time on PostgreSQL.
I'm also going to take all of this to mean that even Oracle and DB2 should be creating the indexes on exactly the same columns as MySQL and PostgreSQL. Please let me know if that's not the case.
Thanks again.
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Sat, 2005-10-08 20:53
my relationship to oracle is a love-hate thing. i've learnt everything about databases with oracle. but the sheer amount of errors / bugs in their products is probably the one thing that i'll never forget about that time.
Quote:
I'm also going to take all of this to mean that even Oracle and DB2 should be creating the indexes on exactly the same columns as MySQL and PostgreSQL. Please let me know if that's not the case.
of course, all databases should create the exact same indices. just with a different syntax if necessary.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Sat, 2005-10-08 21:02
For the time being, I'm going to ignore the requirement to support multiple instances of G2 in the same database. That's going to be really difficult given the short string lengths I have to work with.
I've now got the database setup working correctly (cloned from PostgreSQL this time). Drinks are on me!
But now in Step 8 ("Install Gallery Core"), I get nothing... just a blank browser.
Where do I start looking now?
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Sat, 2005-10-08 21:08
congratulations
ok, first make sure it's a clean install.
drop your db2 database and create a new one (or drop all tables etc.). goal: it should be an empty db.
also delete everything in g2data/
then start the installation again from step 1.
if you still get a blank page in step 8:
please look at g2data/install.log
you can upload it somewhere and add a link to the log here in the forums of course. we can help finding the problem.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Sat, 2005-10-08 21:44
OK, I dropped & recreated the database, & emptied out g2data. Then I closed my browser & fired up the installer in a new browser.
Same thing happens... everything fine up to step 8. There's an error recorded in "install.log" that it couldn't find table G2_PLUGINPARAMETERMAP.
I'll attach the log file here.
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Sat, 2005-10-08 22:31
as far as i see, the errors you have there are ok.
this is my install.log based on a g2/mysql. http://dev.nei.ch/install.log
as you see, there are two db errors at the top. these are logged, but should be ignored by G2. it works for mysql / postgresql / oracle.
the next thing you should see in your debug output is the system information output.
install/steps/InstallCoreModuleStep.class right before
if ($ret->isError()) {
$gallery->debug(sprintf('Error: Unable to initialize our Gallery data, this is the ' .
'error stack trace: %s', $ret->getAsText()));
$templateData['errors'][] = _('Unable to initialize our Gallery data');
$templateData['stackTrace'] = $ret->getAsHtml();
}
$this->_addSystemInformationToDebugLog();
it stops in your case.
2 things:
1. when developing php, change your php error_reporting level to E_ALL (in php.ini, then restart the webserver).
2. when it just dies, take a look at your apache error log (not the access log)
if the above two things don't improve the information you have about the situation, you'll have to find out why
in init.inc function GalleryInitFirstPass
line:
$platform->_calculateUmaskAndFilePerms(); /* Init platform umask */
it calls:
modules/core/classes/GalleryPlatform.class function _calculateUmaskAndFilePerms()
on lines:
list ($ret, $filePerms) =
GalleryCoreApi::getPluginParameter('module', 'core', 'permissions.file');
it calls:
modules/core/classes/helpers/GalleryPluginHelper_simple.class function getParameter(
which calls:
GalleryPluginHelper_simple::fetchAllParameters($pluginType, $pluginId, $itemId);
which calls
$query = '
SELECT
[GalleryPluginParameterMap::parameterName],
[GalleryPluginParameterMap::parameterValue]
FROM
[GalleryPluginParameterMap]
WHERE
[GalleryPluginParameterMap::pluginType] = ?
AND
[GalleryPluginParameterMap::pluginId] = ?
AND
[GalleryPluginParameterMap::itemId] = ?
';
list ($ret, $searchResults) =
$gallery->search($query, array($pluginType, $pluginId, $itemId));
which calls: modules/core/classes/GalleryStorage/DatabaseStorage/Db2DatabaseStorage.class function search
which actually calls modules/core/classes/GalleryStorage/DatabaseStorage.class function search since you didn't override this function with your db class implementation. that's fine, since it isn't necessary to override this method.
so in function search of DatabaseStorage.class it calls:
your goal is to find out where it dies.
the point is: even if there was an error, it shouldn't just die. we should see a debug output saying that it didn't work successfully. instead it just dies.
i'd add your own global $gallery; $gallery->debug("reached this line"); statements to find out where exactly it dies.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Sun, 2005-10-09 00:01
Yeah I've confirmed that e"rror_reporting=E_ALL" is set, and there's nothing in the apache error log.
I've determined that GalleryInitFirstPass() works, the error is in the subsequent "_addSystemInformationToDebugLog()". Specifically, it's the line:
and more specifically, the call to "$storage->_impl->getVersion()" in that line. Plus, I notice that that particular call is prefixed with a "@", which I understand suppresses errors. When I remove the "@", I see the error:
Quote:
Fatal error: Call to undefined function db2serverversion() in C:\My Server\gallery2\modules\core\classes\GalleryStorage\DatabaseStorage\Db2DatabaseStorage.class on line 204
Would anybody mind if I removed that "@"?
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Sun, 2005-10-09 02:00
I've hit a snag that I can't figure out.
After I resolved the problem in my last post, I saw an SQL error that contained references to Oracle. This made me realize that when I re-made my DB2 changes based on PostgreSQL instead of Oracle, I had only modified 'generate-sql.php', not 'Db2DatabaseStorage.class'. So I created a new 'Db2DatabaseStorage.class' based on the PostgreSQL file. But now when I run the installer I get the following error in Step 8 ("Install Gallery core"):
Quote:
Warning: GalleryCoreApi::requireOnce(C:\My Server\gallery2\modules\core\classes/../../../modules/core/classes/GalleryStorage/DatabaseStorage/Db2DatabaseStorage.class) [function.requireOnce]: failed to open stream: Permission denied in C:\My Server\gallery2\modules\core\classes\GalleryCoreApi.class on line 2505
Fatal error: GalleryCoreApi::requireOnce() [function.require]: Failed opening required 'C:\My Server\gallery2modules\core\classes/../../../modules/core/classes/GalleryStorage/DatabaseStorage/Db2DatabaseStorage.class' (include_path='.;C:\php5\pear') in C:\My Server\gallery2\modules\core\classes\GalleryCoreApi.class on line 2505
But I don't get it... the 'Db2DatabaseStorage.class' file is there. Even when I go back to the Oracle-based 'Db2DatabaseStorage.class' file, I still get the same error.
Any idea what's going wrong? I'm really confused by this one.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Sun, 2005-10-09 02:04
I should also add.. there's nothing wrong with the permissions of the file, and nothing is locking it open. I even re-booted just to be sure.
virshu
Joined: 2003-09-13
Posts: 314
Posted: Sun, 2005-10-09 02:54
for some reason there is missing backslash between gallery2 and modules...
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Sun, 2005-10-09 03:09
Sorry that's just a copy-paste error.
Quote:
Warning: GalleryCoreApi::requireOnce(C:\My Server\gallery2\modules\core\classes/../../../modules/core/classes/GalleryStorage/DatabaseStorage/Db2DatabaseStorage.class) [function.requireOnce]: failed to open stream: Permission denied in C:\My Server\gallery2\modules\core\classes\GalleryCoreApi.class on line 2505
Fatal error: GalleryCoreApi::requireOnce() [function.require]: Failed opening required 'C:\My Server\gallery2\modules\core\classes/../../../modules/core/classes/GalleryStorage/DatabaseStorage/Db2DatabaseStorage.class' (include_path='.;C:\php5\pear') in C:\My Server\gallery2\modules\core\classes\GalleryCoreApi.class on line 2505
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Sun, 2005-10-09 03:12
I found that it's coming from somewhere in a function called "_calculateUmaskAndFilePerms()":
$gallery->debug("Returned from _calculateUmaskAndFilePerms().\n");
But I can't find the source for that function.
Install.log says:
Quote:
--------------------------------------------------------
Prepare installation of the core module
--------------------------------------------------------
It was complaining about permissions, so I noticed that the permissions of the 'Db2DatabaseStorage.class' was slightly different in that there was a plus sign after it.
Quote:
$ ls -l
total 52
drwx------+ 2 lmenard None 0 Oct 7 13:39 CVS
-rwx------ 1 lmenard None 5396 Oct 9 00:10 Db2DatabaseStorage.class
-rwx------+ 1 lmenard None 2708 Jul 9 01:44 ErrorHandler.inc
-rwx------+ 1 lmenard None 4168 Aug 22 23:49 MySqlDatabaseStorage.class
-rwx------+ 1 lmenard None 5396 Aug 22 23:49 OracleDatabaseStorage.class
-rwx------+ 1 lmenard None 5845 Aug 22 23:49 PostgreSqlDatabaseStorage.class
drwx------+ 6 lmenard None 0 Oct 7 13:46 schema
The man page for "ls" said the plus sign meant that there are additional access methods for the file. Then I removed the file completely, created it brand new using my usual Windows-based editor (PFE32), and the file now has a plus sign, and that particular problem is now gone. Maybe it was a NTFS thing?
All I can say without offending anyone is... holy cow, that was 4 hours of my life that I'll never get back.
Now I'm finally back to debugging SQL again. But enough for tonight. There's always tomorrow.
bharat
Joined: 2002-05-21
Posts: 7994
Posted: Sun, 2005-10-09 06:07
Larry -- I fear that this process is going to kill you . Do you want to take a different approach? If you want, you can take representative set of the source XML files and translate them by hand into what you think is right for DB2 and then I can update generate-sql.php to generate those properly and commit all the code to get things moving. That way you won't have to do the bulk of the work; you can either just tell me what's wrong with it or you can tweak generate-sql.php until it does the right thing. How does that sound?
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Sun, 2005-10-09 13:54
Hi Bharat.
Yeah, but those years come off the end of your life, and they generally suck anyway. Plus, whatever doesn't kill you should make you stronger, right?
Seriously, I do appreciate the offer. That was actually what I had originally envisioned, but this has turned out to be a very educational experience for me, so I'd like to persevere with it. There's a fine line between stubborn and stupid, and I never can tell which side of that line I'm on. Plus, I'm currently 'between jobs' so I have the time to spend on this.
I think (hope!) things should go more smoothly now. I think I finally have my mind around your build tree structure, and I think that if I hit any more problems like that file permissions problem, then I'll take that to mean there really is a God and I must have been very bad in a previous life.
Plus if I shake any issues out, hopefully someone else will be spared the trouble.
Thanks to all of you folks for your support and patience with me.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Fri, 2005-10-14 16:39
Folks, I'm still grappling with the problem of DB2 object name limitations.
I got a number of indexes to be created OK as discussed above. I'm now trying a similar trick (using another crc instead of the too-long name 'AccessSubscriberMap'), but am getting a 'duplicate name' error.
What I'm trying (in 'generate-sql.php') is:
Quote:
for ($i = $firstNonColumn; $i < count($child); $i++) {
if ($child[$i]['name'] == 'INDEX') {
$crc = $this->getIndexCrc($child[$i]['child']);
// $output .= 'CREATE INDEX DB_TABLE_PREFIX' . $child[0]['content'] . '_' . $crc .
// ' ON DB_TABLE_PREFIX' . $child[0]['content'] . "(";
$output .= 'CREATE INDEX G2_I_' . $this->getIndexCrc(array($child[0]['content'])) . '_' . $crc .
' ON DB_TABLE_PREFIX' . $child[0]['content'] . "(";
, which generates SQL:
Quote:
CREATE INDEX G2_I_54475_83732 ON g2_AccessSubscriberMap(g_accessListId)
, but apparently that's also the name of the first index we created.
Is there something better that I should be using? (Part of the problem is that I don't really understand the meaning of things like 'parent', 'child', 'content' in this context.)
I'm attaching the 'install.log' here.
Thanks.
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Fri, 2005-10-14 17:41
looks ok.
Quote:
, but apparently that's also the name of the first index we created.
/me is looking at install.log
Quote:
05-10-14 12:19:37 [<no session id>] <hr> (db2): CREATE INDEX G2_I_54475_83732 ON g2_AccessMap(g_accessListId) <code></code>
<hr>
2005-10-14 12:19:37 [<no session id>] <hr>
(db2): CREATE INDEX G2_I_54475_69068 ON g2_AccessMap(g_userId) <code></code>
<hr>
2005-10-14 12:19:37 [<no session id>] <hr>
(db2): CREATE INDEX G2_I_54475_89328 ON g2_AccessMap(g_groupId) <code></code>
<hr>
2005-10-14 12:19:38 [<no session id>] <hr>
(db2): CREATE INDEX G2_I_54475_18058 ON g2_AccessMap(g_permission) <code></code>
<hr>
2005-10-14 12:19:38 [<no session id>] db2 error: [S0011: [IBM][CLI Driver][DB2/NT] SQL0601N The name of the object to be created is identical to the existing name "G2USER.G2_I_54475_83732" of type "INDEX". SQLSTATE=42710
] in EXECUTE("CREATE INDEX G2_I_54475_83732 ON g2_AccessSubscriberMap(g_accessListId)")
so obviously there was a collision because the last 5 digits of the CRC32 value for AccessSubscriberMap are the same as for AccessMap.
well, that was very unlikely, but it happened.
so it seems CRC32, last 5 digits, very similar table names is a bad idea
instead of CRC you could also use substr(md5($child[0]['content']), -5) or -6 or -7, ... to get the last 5, 6, 7, .. characters of the md5 hash string of the table name. the chance for a collision should be smaller with md5 and using more characters reduces this probability even more.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Fri, 2005-10-14 20:58
As Professor Farnsworth would say, "Good news, everyone."
Installer Step 8 ("Install Gallery Core") seems to have been successfully installed.
I'm now looking at Installer Step 9 ("Install More Modules"). Is there anything here that I should select or deselect, or should I just proceed with default values?
valiant
Joined: 2003-01-04
Posts: 32509
Posted: Fri, 2005-10-14 21:15
cool
either uncheck all modules and just finish without activating other modules to see if it works in general.
and if it works, you can generate the sql for all modules as the next step:
you can now run "gmake" in all the modules/*/classes/ dirs.
or maybe just a single call to /lib/tools/bin/rebuild-modules.pl, not sure though if it works.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Fri, 2005-10-14 21:24
Hmm, I'm not sure I understand your response. I don't see an "or" that matches your "either". But I think that's what I'll do anyway (deselect all) and see where it goes from there. ;-)
Thanks.
Larry Menard
Joined: 2005-10-01
Posts: 757
Posted: Fri, 2005-10-14 21:39
OK, I deselected all, and the install seemed to finish OK. But when I clicked on "Go to my Gallery" I got the following error:
Quote:
Error Detail -
Error (ERROR_MISSING_OBJECT) :
in C:\My Server\gallery2\modules\core\classes\GalleryStorage\DatabaseStorage.class at line 378 (GalleryStatus::error)
in C:\My Server\gallery2\modules\core\classes\GalleryStorage.class at line 123 (DatabaseStorage::loadEntities)
in C:\My Server\gallery2\modules\core\classes\helpers\GalleryEntityHelper_simple.class at line 82 (GalleryStorage::loadEntities)
in C:\My Server\gallery2\modules\core\classes\GalleryCoreApi.class at line 2186 (GalleryEntityHelper_simple::loadEntitiesById)
in C:\My Server\gallery2\modules\core\classes\GalleryView.class at line 360 (GalleryCoreApi::loadEntitiesById)
in C:\My Server\gallery2\modules\core\classes\GalleryView.class at line 219 (GalleryView::_getItem)
in C:\My Server\gallery2\main.php at line 301 (GalleryView::doLoadTemplate)
in C:\My Server\gallery2\main.php at line 87
in C:\My Server\gallery2\main.php at line 80
Posts: 314
Larry,
The easiest (and most user-friendly) CVS software for windows is http://www.tortoisecvs.org/index.shtml. It's also integrated with Windows explorer. You don't need python, and there are no geeky details that are unneccessary.
Building, however, used to be somewhat problematic on Windows - but I think that's what Bharat is fixing right now...
Good luck!
Posts: 757
Hi, Virshu.
OK I've installed TortoiseCVS, but seem to be unable to reach the CVS server for Gallery. I'm using:
But then it just incessantly keeps prompting me for my password. I keep entering it, and it keeps prompting me for it.
When I cancel, here are the messages:
What am I missing?
Posts: 757
After reading further, I see that SSH is used only by authorized developers, for which I doubt I qualify.
So I've switched to protocol ":pserver" and user name "anonymous", and module "lib/tools/bin/generate-sql.php", but it tells me that it couldn't find that module.
Posts: 757
Yay, I figured out that the module name needed a "gallery2/" at the front. I think I've now checked out "gallery2/lib/tools/bin/generate-sql.php".
I guess now I need to know what specific files I need to grab.
Posts: 7994
The general shape of the command that you want is:
That will check out *all* of the code. This is the right thing to do because it means that your entire G2 will be drawn from the CVS repository which means that when you want to get the latest code you just do "cvs update" and it does the rest for you. If you want to give us a patch to apply, you do "cvs diff" and it'll tell you what's changed in your version vs. what's in revision control.
Posts: 7994
FYI, I just committed one round of changes that simplifies the SQL generation code. I got rid of the makefiles in xml-src and platform, now there's only one in the schema directory and it should do everything. Because of the way that SourceForge is set up, it takes ~6 hours for anything we commit to show up in the anonymous repository (which you're drawing from).
So the way to figure out which version you have is to do "cvs status generate-sql.php". If you have revision 1.10, then copy over the prior version of generate-sql.php that posted. If you have revision 1.11 then copy over this version instead: http://www.menalto.com/.outgoing/gallery/generate-sql-2.php.txt
Posts: 32509
as you're using cygwin, you could just use the linux style cvs commands there.
and if you want to stick to windows applications, i also use http://www.tortoisecvs.org/index.shtml on my windows box. once configured, cvs is a matter of clicking a few buttongs
also see
http://gallery.menalto.com/wiki/CvsForNewDevelopers
and
http://sourceforge.net/cvs/?group_id=7130
Posts: 757
Thanks folks. I've now checked out a copy of the complete build tree.
But with Bharat's changes, the instructions given above have now changed, so here's a step-by-step of what I've done this time 'round.
As Bharat says, there's now only 1 GNUmakefile, in "schema". I no longer see a "scrub" section in there, so will ignore that step.
My copy of "generate-sql.php" is version 1.11, so I copied over "http://www.menalto.com/.outgoing/gallery/generate-sql-2.php.txt" as Bharat said.
When I do "gmake" in directory "gallery2/modules/core/classes/GalleryStorage/DatabaseStorage/schema",
I get error:
The files in "platform/db2" do seem to be created, so I'll try proceeding anyway. FYI, I also see that an empty directory for each platform has also been created under "schema". Perhaps this is harmless, perhaps even required... I'll leave them and carry on.
I then copied "GalleryStorage/DatabaseStorage/OracleDatabaseStorage.class" to "GalleryStorage/DatabaseStorage/Db2DatabaseStorage.class" and modified it for DB2.
I then added the DB2 particulars to "install/steps/DatabaseSetupStep.class" and "install/config.php-template".
I then added a DB2 section to "modules/core/classes/DatabaseStorage.class".
I then copied files "drivers/adodb-db2.inc.php" and "drivers/adodb-odbc.inc.php" from the "adodb" package into "lib/adodb/drivers".
Now I boldly fire up the installer.
In the "System Check" step, I see 19 modified files. I won't bother listing them here.
Something odd: As soon as the "Database Setup" page loaded it had error "You must have the MySQL PHP module installed", even though I hadn't entered anything yet.
Charging ahead, I entered the specifics for my DB2 database, but that resulted in error:
Any ideas what might cause these database setup errors?
Thanks.
Posts: 32509
@my last post: sorry didn't read the 2nd page of this topic before.
Posts: 7994
That looks like progress to me! I shadowed everything you did on my Windows box and got a little further than you do. I'm at the limit of how far I can go because I don't have ODBC and DB2 handy. But a few things:
1. I fixed the "xmllint: command not found" bug. cvs up and you'll get that fix.
2. I fixed the "empty directory for each db under the schema directory" bug. Typo on my part. cvs up for that too.
3. The "You must have the MySQL PHP module installed" is actually a bug, but we hit it incredibly rarely beacause almost everybody has mysql installed! Fixed.
None of these issues are critical, so you don't have to CVS up right away to fix them.
I reproduced the problem you were experiencing and realized that it's because I wasn't setting the $dbPlatformType to 'db2' in DatabaseSetupStep, so adodb was using the default connector which doesn't work. I've put my tweaked copy of that file here:
http://www.menalto.com/.outgoing/gallery/DatabaseSetupStep.class
Try that instead. With that, I now get this error:
So that's good news; it's using ODBC correctly and trying to connect. But I don't know the right values to provide, etc to make that happen. You probably know a lot more than me at this point. I took a quick look at the odbc_connect() call in adodb-odbc.inc.php and see that it's expecting to call:
This implies to me that instead of providing the name of the database (I used "gallery2") I need to provide a full DSN. Let me know how that goes for you.
Posts: 757
Hi, Bharat.
Thanks for the fixes.
Yup, I got there at one point too, a couple of changes ago
. There's a weirdness with DB2 via ODBC, you need to specify the parameters to odbc_connect in a different order. What Valiant and I discussed at one time was entering the database name in the "DB Hostname" field. I tested it and it works.
However... I've found an extension to PHP, called "ibm_db2". It's based on DB2's "Call Level Interface" (or "CLI"), DB2's own version of ODBC, which is apparently much better than unified ODBC. Plus, I used to work with the developer of the 'ibm_db2' extension, Dan Scott.
So I'm planning to switch from ODBC to the 'ibm_db2' PHP extension... I think it'll help in a couple of ways.
Will let you know how it goes.
Posts: 757
Bharat,
I downloaded the "DatabaseSetupStep.class" that you pointed to, and I still get the same NConnect error (not migrated from ODBC to ibm_db2 yet). From what I can see, your file is the same as mine except that you've changed the order of the platforms in a couple of places... the code itself is the same.
Then, I changed the 'odbc_connect' to 'db2_connect' (the correct call according to the 'ibm_db2' doc), and now it says it can't find that function. I have copied the 'php_ibm_db2.dll' to my PHP directory and pointed to it in my 'php.ini' ('extension=php_ibm_db2.dll'). Any idea why it can't find that function?
Posts: 32509
how do you plan to do this "switch" to ibm_db2?
is it supported by adodb?
Posts: 757
I didn't know it had to explicitly support it. I doubt it does. Oh well, back to ODBC.
FYI, I tried to pick up the 'optional' fixes by:
but after that I'm no longer able to 'gmake'... lots of errors like:
So I completely released my build tree, am getting a whole new copy, and will try yet again.
When I volunteered for this, I thought it might take a day or so to just plunk some SQL in somewhere. Hmmm...
Posts: 32509
@ibm_db2:
well, we use adodb as our db abstraction layer, all db queries go through adodb. if adodb doesn't use the ibm_ calls, we can't use ibm_ calls...
of course you could send a a modified adodb-db2 driver file to adodb, maybe they'll add it to the distribution.
maybe it supports ibm_db2, you could find out by grepping (grep -ri "ibm_" *) in the adodb drivers/ directory of the latest adodb release (not in g2's adodb/drivers directory, since adodb might have, but we may not include it).
@cvs update:
you have changed some files. if you then do cvs update -P, it doesn't replace the changed files. depending on a few factors, cvs auto-merges your local changes with the new version of the file from the cvs repository, or it adds <<< >>> stuff in the files to mark where there are differences or it doesn't touch your changed files at all.
so this might explain your problems.
copy your changed files to a save place, then use
cvs -z3 update -dPC
-z3 for compression, speeds up the whole transmission
-C to replace your changed files with new versions from the cvs repository. thus you'll lose all your changes. but it renames your locally changed files to something like #oldfilename.
Posts: 757
Even with brand new build tree (and no DB2 support added), same errors in 'gmake'. Might they be because of a 'php.ini' setting?
Posts: 7994
Ok, I raised the warning level to something that matches what you're using, and fixed all the warnings that it generates. This last batch was just warnings though, it shouldn't have prevented any SQL generation.
Are you still getting the NConnect error? You shouldn't be anymore with your code (or the version I provided). From looking at the DB2 example here:
http://phplens.com/lens/adodb/docs-adodb.htm
It looks like you need to pass in a DSN definition as the first argument, and the username, password, etc can be empty. I tried it with this DSN:
And got this error:
That looks promising to me. Looks like it's actually parsing the DSN correctly and trying to do the right thing with it which means that G2 isn't really in your way at this point.
If you can get it to connect then the first thing that it's going to try to do is to run the T_InstallerTest_1.sql .. T_InstallerTest_4.sql generated sql files. So this is where you can start tweaking generate-sql.php to actually build out the right SQL for DB2.
I know what you mean. It's more complex than it seems at first. The big downside to doing it this way is that getting support for a new database requires a steep learning curve at first. The upside to doing it this way is that once we've gotten the transforms working properly, we rarely need to touch the SQL generation code again even as we continuously make small changes to the database schema.
Posts: 757
When they say to use dsn as the first parm, I believe they mean "database name", not a full "datasource name". Since G2 (or whoever) passes the hostname as the first parm, put the database name "gallery2" in the hostname field. That worked for me; at least the connect didn't fail, but then I got in trouble with the "InstallTest" file missing.
Like I said above:
Am I mistaken? Other than the order of the plaforms, what else did you change in it? (The Windows CRLFs in my file make "diff" pointless.) I'm very confused as to why you're telling me to use essentially the same file as my own.
In the meantime, I'll re-add my DB2/ODBC support into my new build and see how far I get now.
And FYI... Dan Scott tells me that ADOdb package should support the 'ibm_db2' PHP extension soon, perhaps in a couple of weeks.
Posts: 757
OK, I think I'm on a roll. By putting the database name in the "hostname" field, it does successfully connect to the database. I'm now working on fixing up some syntax errors in the generated SQL.
This is where I thought I'd be 4 days ago.
Posts: 757
Big Problem.
From what I've seen of the index names in G2, this is going to be a very pervasive problem. I believe that even some column names may exceed the DB2 limit of 30 characters, but I'm not sure of that yet.
Suggestions?
Posts: 32509
nope that shouldn't be a problem.
so you know now that indices must be shorter than 19 characters.
in generate-sql.php
class Db2Generator extends BaseGenerator {
function createSql
around (2 occurrences):
case 'INDEX':
you define the index name. you can choose a diffferent pattern to assemble the index name, you don't have to follow the pattern "prefixTableName_CRC". you can also choose other patterns.
i would include the CRC and maybe instead of the rest, you should also find something unique, but it shouldn't be that long. maybe a truncated crc for the tablename?
Posts: 757
OK, cool. I was concerned that you would be concerned with cross-platform schema compatablity.
Thanks.
Posts: 757
Possibly a stupid question...
I'm not sure what "CRC" you mean. The CRC of what? I know how to get the CRC value of a file, but, I don't see how that fits in the context of your response.
Thanks.
Posts: 32509
you want to replace
with something that is shorter.
i'd keep the second part, ('_' . $this->getIndexCrc($columns);[/code]) since it's quite short and is quite important.
what we want is to make sure 'DB_TABLE_PREFIX' . $parent['child'][0]['content'] isn't longer than 12 characters.
so what you could do is
a) omit DB_TABLE_PREFIX and just make it $parent['child'][0]['content']. but this wouldn't fix it, since $parent['child'][0]['content'] is likely longer than 6 chars.
b) compute crc32($parent['child'][0]['content']) and take the last 6 digits of it.
why? well, the resulting index name wouldn't be human readable, but we need a index name that is unique among all indices in the database, right?
i'd use
which will create a 5 digits underscore 5 digits index key.
maybe someone has a better idea, e.g. keep the index name human readable and shorter than 18 characters. but this seems to be straight forward.
Posts: 757
I see. Sorry, I thought I had to calculate the CRC myself somehow.
Thanks.
Posts: 757
Apparently DB2 doesn't like index names that do not have an alphabetic charcter. So I prefix the index name with "G2_I_".
But now I've hit something I don't get. It's trying to create that index on columns that don't exist.
I have no idea where these incorrect column names are coming from. Any pointers?
Posts: 32509
hmm, wait a second: you should use the DB_TABLE_PREFIX for all index names, since someone could want to install multiple G2 installs in the same database, but with different prefices.
@error:
obviously your CREATE INDEX SQL generation isn't ok. g_STRING, g_MEDIUM shouldn't be part of this line.
see the corresponding code in T_InstallerTest_2.sql in mysql/ or postgresql/
Posts: 757
Well if the DB_TABLE_PREFIX stays, then the rest of the index name needs to be shortened accordingly.
Like I said in my original posting, I am not a hardcore developer. 99% of what I know of PHP I've learned in the last week. So, I'm sorry, but with all due respect, I don't find your answer "check the code" that helpful. Trust me, I'm even more frustrated with this than you are.
My type mapping is as follows:
I believe these to be fine, but the references to "MEDIUM" and "STRING" in the generated SQL make me suspect something isn't parsing these properly. But I can't tell where that is done.
I checked the Oracle, MySQL, and PostgreSQL files; the latter two generate the index only on the 'g_testcol' column, but Oracle is similarly afflicted. (I based my changes on the Oracle code, so if there was a bug in it, I'd have it too. For example, I did find one earlier SQL syntax problem that I wonder how it worked on Oracle either... no '\n;\n\n' between the ADD COLUMN and the CREATE INDEX.)
Also, I'm still getting lots of the following messages when I do 'gmake'. I thought Bharat said that the SQL should still be generated, but is it possible they're related to the generated SQL being bad?
Posts: 32509
yep, you're right. oracle's code is also bugged. that explains your issue.
again, i'm sorry for all these unnecessary issues. you just happen to have chosen a time when we chose to restructure this code :/
but oracle is possibly the most untested code. so basing your changes on postgresql should be a more reliable choice.
i'm very sure that noone tested the generated oracle sql code that we have now. it's only a week old (cvs) and the handful of oracle users we have use G 2.0.
my advice is:
base your DB2 class on the postgresql class (bharat mainly develops g2 on/for postgresql).
i'm sure he'll be able to address the remaining issues soon.
Posts: 757
Just my luck.
I figured Oracle would be more similar to DB2 in terms of functionality provided.
On the other hand, I'm glad to hear that Oracle apparently isn't popular in your client base. I worked in the DB2 development organization for 12 years, and in IBM in general for 25 years, so Oracle is considered one of 'The Competition'.
OK, I'll re-implement my changes, based this time on PostgreSQL.
I'm also going to take all of this to mean that even Oracle and DB2 should be creating the indexes on exactly the same columns as MySQL and PostgreSQL. Please let me know if that's not the case.
Thanks again.
Posts: 32509
my relationship to oracle is a love-hate thing. i've learnt everything about databases with oracle. but the sheer amount of errors / bugs in their products is probably the one thing that i'll never forget about that time.
of course, all databases should create the exact same indices. just with a different syntax if necessary.
Posts: 757
For the time being, I'm going to ignore the requirement to support multiple instances of G2 in the same database. That's going to be really difficult given the short string lengths I have to work with.
I've now got the database setup working correctly (cloned from PostgreSQL this time). Drinks are on me!
But now in Step 8 ("Install Gallery Core"), I get nothing... just a blank browser.
Where do I start looking now?
Posts: 32509
congratulations
ok, first make sure it's a clean install.
drop your db2 database and create a new one (or drop all tables etc.). goal: it should be an empty db.
also delete everything in g2data/
then start the installation again from step 1.
if you still get a blank page in step 8:
please look at g2data/install.log
you can upload it somewhere and add a link to the log here in the forums of course. we can help finding the problem.
Posts: 757
OK, I dropped & recreated the database, & emptied out g2data. Then I closed my browser & fired up the installer in a new browser.
Same thing happens... everything fine up to step 8. There's an error recorded in "install.log" that it couldn't find table G2_PLUGINPARAMETERMAP.
I'll attach the log file here.
Posts: 32509
as far as i see, the errors you have there are ok.
this is my install.log based on a g2/mysql.
http://dev.nei.ch/install.log
as you see, there are two db errors at the top. these are logged, but should be ignored by G2. it works for mysql / postgresql / oracle.
the next thing you should see in your debug output is the system information output.
install/steps/InstallCoreModuleStep.class right before
if ($ret->isError()) { $gallery->debug(sprintf('Error: Unable to initialize our Gallery data, this is the ' . 'error stack trace: %s', $ret->getAsText())); $templateData['errors'][] = _('Unable to initialize our Gallery data'); $templateData['stackTrace'] = $ret->getAsHtml(); } $this->_addSystemInformationToDebugLog();it stops in your case.
2 things:
1. when developing php, change your php error_reporting level to E_ALL (in php.ini, then restart the webserver).
2. when it just dies, take a look at your apache error log (not the access log)
if the above two things don't improve the information you have about the situation, you'll have to find out why
$ret = GalleryInitFirstPass(array('debug' => 'logged', 'noDatabase' => 1));doesn't work.
in init.inc function GalleryInitFirstPass
line:
$platform->_calculateUmaskAndFilePerms(); /* Init platform umask */
it calls:
modules/core/classes/GalleryPlatform.class function _calculateUmaskAndFilePerms()
on lines:
list ($ret, $filePerms) = GalleryCoreApi::getPluginParameter('module', 'core', 'permissions.file');it calls:
modules/core/classes/helpers/GalleryPluginHelper_simple.class function getParameter(
which calls:
GalleryPluginHelper_simple::fetchAllParameters($pluginType, $pluginId, $itemId);
which calls
which calls:
modules/core/classes/GalleryStorage/DatabaseStorage/Db2DatabaseStorage.class function search
which actually calls modules/core/classes/GalleryStorage/DatabaseStorage.class function search since you didn't override this function with your db class implementation. that's fine, since it isn't necessary to override this method.
so in function search of DatabaseStorage.class it calls:
$this->_traceStart(); $recordSet = $this->_db->Execute($query, $data); $this->_traceStop(); } if ($recordSet) { return array(GalleryStatus::success(), new DatabaseSearchResults($recordSet)); } else { return array(GalleryStatus::error(ERROR_STORAGE_FAILURE, __FILE__, __LINE__), null); }your goal is to find out where it dies.
the point is: even if there was an error, it shouldn't just die. we should see a debug output saying that it didn't work successfully. instead it just dies.
i'd add your own global $gallery; $gallery->debug("reached this line"); statements to find out where exactly it dies.
Posts: 757
Yeah I've confirmed that e"rror_reporting=E_ALL" is set, and there's nothing in the apache error log.
I've determined that GalleryInitFirstPass() works, the error is in the subsequent "_addSystemInformationToDebugLog()". Specifically, it's the line:
and more specifically, the call to "$storage->_impl->getVersion()" in that line. Plus, I notice that that particular call is prefixed with a "@", which I understand suppresses errors. When I remove the "@", I see the error:
Would anybody mind if I removed that "@"?
Posts: 757
I've hit a snag that I can't figure out.
After I resolved the problem in my last post, I saw an SQL error that contained references to Oracle. This made me realize that when I re-made my DB2 changes based on PostgreSQL instead of Oracle, I had only modified 'generate-sql.php', not 'Db2DatabaseStorage.class'. So I created a new 'Db2DatabaseStorage.class' based on the PostgreSQL file. But now when I run the installer I get the following error in Step 8 ("Install Gallery core"):
But I don't get it... the 'Db2DatabaseStorage.class' file is there. Even when I go back to the Oracle-based 'Db2DatabaseStorage.class' file, I still get the same error.
Any idea what's going wrong? I'm really confused by this one.
Posts: 757
I should also add.. there's nothing wrong with the permissions of the file, and nothing is locking it open. I even re-booted just to be sure.
Posts: 314
for some reason there is missing backslash between gallery2 and modules...
Posts: 757
Sorry that's just a copy-paste error.
Posts: 757
I found that it's coming from somewhere in a function called "_calculateUmaskAndFilePerms()":
But I can't find the source for that function.
Install.log says:
And that's all she wrote.
Posts: 757
Ok, I figured this one out too.
It was complaining about permissions, so I noticed that the permissions of the 'Db2DatabaseStorage.class' was slightly different in that there was a plus sign after it.
The man page for "ls" said the plus sign meant that there are additional access methods for the file. Then I removed the file completely, created it brand new using my usual Windows-based editor (PFE32), and the file now has a plus sign, and that particular problem is now gone. Maybe it was a NTFS thing?
All I can say without offending anyone is... holy cow, that was 4 hours of my life that I'll never get back.
Now I'm finally back to debugging SQL again. But enough for tonight. There's always tomorrow.
Posts: 7994
Larry -- I fear that this process is going to kill you
. Do you want to take a different approach? If you want, you can take representative set of the source XML files and translate them by hand into what you think is right for DB2 and then I can update generate-sql.php to generate those properly and commit all the code to get things moving. That way you won't have to do the bulk of the work; you can either just tell me what's wrong with it or you can tweak generate-sql.php until it does the right thing. How does that sound?
Posts: 757
Hi Bharat.
Yeah, but those years come off the end of your life, and they generally suck anyway.
Plus, whatever doesn't kill you should make you stronger, right?
Seriously, I do appreciate the offer. That was actually what I had originally envisioned, but this has turned out to be a very educational experience for me, so I'd like to persevere with it. There's a fine line between stubborn and stupid, and I never can tell which side of that line I'm on. Plus, I'm currently 'between jobs' so I have the time to spend on this.
I think (hope!) things should go more smoothly now. I think I finally have my mind around your build tree structure, and I think that if I hit any more problems like that file permissions problem, then I'll take that to mean there really is a God and I must have been very bad in a previous life.
Plus if I shake any issues out, hopefully someone else will be spared the trouble.
Thanks to all of you folks for your support and patience with me.
Posts: 757
Folks, I'm still grappling with the problem of DB2 object name limitations.
I got a number of indexes to be created OK as discussed above. I'm now trying a similar trick (using another crc instead of the too-long name 'AccessSubscriberMap'), but am getting a 'duplicate name' error.
What I'm trying (in 'generate-sql.php') is:
, which generates SQL:
, but apparently that's also the name of the first index we created.
Is there something better that I should be using? (Part of the problem is that I don't really understand the meaning of things like 'parent', 'child', 'content' in this context.)
I'm attaching the 'install.log' here.
Thanks.
Posts: 32509
looks ok.
/me is looking at install.log
so obviously there was a collision because the last 5 digits of the CRC32 value for AccessSubscriberMap are the same as for AccessMap.
well, that was very unlikely, but it happened.
so it seems CRC32, last 5 digits, very similar table names is a bad idea
instead of CRC you could also use substr(md5($child[0]['content']), -5) or -6 or -7, ... to get the last 5, 6, 7, .. characters of the md5 hash string of the table name. the chance for a collision should be smaller with md5 and using more characters reduces this probability even more.
Posts: 757
As Professor Farnsworth would say, "Good news, everyone."
Installer Step 8 ("Install Gallery Core") seems to have been successfully installed.
I'm now looking at Installer Step 9 ("Install More Modules"). Is there anything here that I should select or deselect, or should I just proceed with default values?
Posts: 32509
cool
either uncheck all modules and just finish without activating other modules to see if it works in general.
and if it works, you can generate the sql for all modules as the next step:
you can now run "gmake" in all the modules/*/classes/ dirs.
or maybe just a single call to /lib/tools/bin/rebuild-modules.pl, not sure though if it works.
Posts: 757
Hmm, I'm not sure I understand your response. I don't see an "or" that matches your "either". But I think that's what I'll do anyway (deselect all) and see where it goes from there. ;-)
Thanks.
Posts: 757
OK, I deselected all, and the install seemed to finish OK. But when I clicked on "Go to my Gallery" I got the following error:
Suggestions?