Statistics Module

aidanlister

Joined: 2005-12-07
Posts: 56
Posted: Sat, 2005-12-10 00:24

Hello,

I've written a statistics module. Here's the info on Codex:
http://codex.gallery2.org/index.php/Gallery2:Modules:Statistics

If you have any ideas for features, design or improvements on the current code, please post here or on the codex and let me know - Thanks!

 
aidanlister

Joined: 2005-12-07
Posts: 56
Posted: Sat, 2005-12-10 06:33

As of writing
* All Entries [Removed for the moment]
* General [Working]
* Disk Usage [Working]
* User Stats [Removed for the moment]
* Latest events [Working]

 
aidanlister

Joined: 2005-12-07
Posts: 56
Posted: Sun, 2005-12-11 15:08

Hello,

General Statistics is now complete, as is Latest Events.

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Sun, 2005-12-11 17:18

if you haven't already:

- think about modularizing: each module should provide statistics, your module shouldn't be aware of all other modules. that way, module authors can offer statistics when they write a new module and you don't have to keep track of all new modules.
e.g. check for modules that implement a function getStatistics() or so

- almost all (web) applications that offer a statistics view must use caching. without caching, each page impression would recalculate all the statistics (run x database queries etc etc).
Gallery offers GalleryDataCache::putToDisk etc. to cache data on the disk. another approach would be to cache the data in the database with GalleryCoreApi::setPluginParameter() or in a dedicated table, if necessary

also, please offer a .zip download or another packaged format.
if you want cvs access, we can give you cvs access to sf.net/projects/gallery-contrib/ which is used by a few module developers.

 
id01

Joined: 2005-11-07
Posts: 62
Posted: Sun, 2005-12-11 19:15

I will definitely install Gallery 2.02 and install this just to try it out! This is too tempting to ignore. Being the reason why my gallery is still running on 1.51 engine.

Thank you very much for this statistic module. I personally am not a programmer so I'm afraid I can't show you much appreciation or suggestion.

Though I do have a few idea. If you have not been aware of it yet, perhaps it would be nice to turn off certain statistic and make sure the server will not still calculate it. When a gallery become huge and populated, the statistic certainly can put a troll on the database query. I believe that is one of the reason why the old gallery slow down as the size got bigger.

I also agree with Valiant, without caching the site will be very slow.

And about the zip download, that would be a lot better. Downloading each file seem like a pain. I do not want to add more work for you but if there are anything I can help I will gladly provide it!

 
bgraabek
bgraabek's picture

Joined: 2003-11-20
Posts: 19
Posted: Mon, 2005-12-12 16:15

I've downloaded your code (with WGET to make sure I got everything). There is obviously something I need to do to enable to module and I don't know what it is. One point to make is that my gallery is a multi-site setup.

I moved everything in the statistics directory (including obviously the statistics dir. itself) into "gallery2/modules". The moment I click "siteadmin > Modules" I see the php code from you module in the web browser. At the bottom of the page isn then shown a Gallery errormessage:

Error Detail
Error (ERROR_BAD_PLUGIN) : Class statisticsModule does not exist

* in modules/core/classes/helpers/GalleryPluginHelper_simple.class at line 113 (gallerystatus::error)
* in modules/core/classes/GalleryCoreApi.class at line 187 (gallerypluginhelper_simple::loadplugin)
* in modules/core/AdminModules.inc at line 227 (gallerycoreapi::loadplugin)
* in modules/core/SiteAdmin.inc at line 153 (adminmodulesview::loadtemplate)
* in modules/core/classes/GalleryTheme.class at line 688 (siteadminview::loadtemplate)
* in modules/core/classes/GalleryView.class at line 285 (matrixtheme::loadtemplate)
* in main.php at line 287 (siteadminview::doloadtemplate)
* in main.php at line 87
* in main.php at line 80
* in /home/virt_hosts/graabek.com/domains/photo.graabek.com/html/main.php at line 3

System Information
Gallery version 2.0.2
PHP version 4.3.9 apache2handler
Webserver Apache/2.0.52 (CentOS)
Database mysqlt 4.1.12
Toolkits ImageMagick, NetPBM
Operating system Linux server.graabek.com 2.6.9-22.0.1.ELsmp #1 SMP Thu Oct 27 14:49:37 CDT 2005 x86_64
Browser Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8) Gecko/20051111 Firefox/1.5

 
VoiceX

Joined: 2005-09-26
Posts: 12
Posted: Mon, 2005-12-12 16:40

installation problem solved!

by downloading form aidanlister's site, it created me HTML-pages. By extracting the original text and update the files on the server, it works.

now download my version directly from

EDIT - sorry had to shotdown my page! problems with jemako.com - EDIT

i hope installation works by everyone!

i've got only no results at:
- all entries
- disk usage (album:today, size:Hello, total space: .../...)
- user statistics

although: a very BIG THX to aidanlister !!! very good work !!!

peace!
VoiceX

old text:
the same thing as by bgraabek happens also by me :-(

what i did additionally: i created a directory statistics

hope for solution, please!

-------------------------------------
GreetX from Switzerland

VoiceX

 
NBrady

Joined: 2005-10-07
Posts: 77
Posted: Tue, 2005-12-13 02:07
Quote:
Warning: Cannot modify header information - headers already sent by (output started at /home/nbhq27f/public_html/gallery/modules/statistics/module.inc:85) in /home/nbhq27f/public_html/gallery/modules/core/classes/GalleryPhpVm.class on line 124

 
NBrady

Joined: 2005-10-07
Posts: 77
Posted: Tue, 2005-12-13 02:21

False alarm, its working, great job on it

 
ZyreX

Joined: 2004-03-04
Posts: 43
Posted: Tue, 2005-12-13 11:00

Really interested in this module, just gon' wait a little while more before I try it :) but it has all my needs!

 
drhiii

Joined: 2003-04-27
Posts: 397
Posted: Tue, 2005-12-13 11:52

I am getting the same error. What did you do to solve this? Looks like an excellent module.

NBrady wrote:
Quote:
Warning: Cannot modify header information - headers already sent by (output started at /home/nbhq27f/public_html/gallery/modules/statistics/module.inc:85) in /home/nbhq27f/public_html/gallery/modules/core/classes/GalleryPhpVm.class on line 124

 
aidanlister

Joined: 2005-12-07
Posts: 56
Posted: Tue, 2005-12-13 12:43

Hi,

I think you may possibly be including some extra unintentional whitespace when you download the files. Check each file for spaces above <?php or below ?> and remove them. That should fix the problem.

I'm sorry it's so inconvenient to install at the moment, I'm going to finish a first revision of the code tomorrow then put that in CVS - which chris has kindly set up. Once that is done, I shall release a version and you'll be able to download it in a zip file.

 
aidanlister

Joined: 2005-12-07
Posts: 56
Posted: Wed, 2005-12-14 05:53

Hello!

I've added the DiskUsage section, and uploaded the release on CVS and SourceForge.

You can download it here:
https://sourceforge.net/project/showfiles.php?group_id=142706

 
ZyreX

Joined: 2004-03-04
Posts: 43
Posted: Thu, 2005-12-15 09:35

---------
Total space used 1.59 GB
Total space available 71.63 GB
Percentage space available 97.78%
---------

Filesystem Size Used Avail Use% Mounted on
/dev/hda6 67G 38G 30G 56% /home

Something is a little bit fishy here, don't you agree? :)
(Yes, I am sure about the partitions..)

Other than that, it seems to be working fine :)

 
VoiceX

Joined: 2005-09-26
Posts: 12
Posted: Thu, 2005-12-15 12:32

same thing happend by me:

Item Value
Total space used 75.72 MB
Total space available 79.60 GB
Percentage space available 99.9%

i've got 500 MB diskspace. Is it possible to enter this value manually?
and can i sort the listing by used diskspace?

THX ALOT for this great work !!!

-------------------------------------
GreetX from Switzerland

VoiceX

 
aidanlister

Joined: 2005-12-07
Posts: 56
Posted: Fri, 2005-12-16 08:42

Hello,

I've fixed the percentage space available, and changed the album breakdown so you can see parent/children albums in a tree. I won't sort by size, because it would break the tree.

I'll release a new version later this week, I'm glad you like it :)

 
id01

Joined: 2005-11-07
Posts: 62
Posted: Fri, 2005-12-16 22:47

This is coming alone exceptionally well. I'm glad we have someone who know what they are doing to work on this mod. While it is not something everyone will need, this will be one of the mod that give Gallery a polished touch which can be felt by every user.
I have problem with generating link that will directly display state (like a link that will display the newest upload) but I am happy about the progress already!

 
aidanlister

Joined: 2005-12-07
Posts: 56
Posted: Sat, 2005-12-17 03:21

Hi id01, I've just fixed that bug, the updated code is in CVS.
http://cvs.sourceforge.net/viewcvs.py/gallery-contrib/gallery2/modules/statistics/

 
oddessy74

Joined: 2005-12-03
Posts: 6
Posted: Sat, 2005-12-17 04:45

Thank you so much this is just what I was looking for. Keep up the great work.

 
ZyreX

Joined: 2004-03-04
Posts: 43
Posted: Mon, 2005-12-19 07:53

Looks like it's working like a charm now.. takes some time getting the disk usage page up, though.. :)

 
LFrank

Joined: 2005-02-19
Posts: 1023
Posted: Mon, 2005-12-19 18:17

hmm, my disk usage simply times out, the other info section work perfectly.
(I'm using the latest CVS - 19.12.05)

Gallery version=2.0.2+ core 1.0.10
PHP version=5.1.1 apache2handler
Webserver=Apache/2.2.0 (Win32) DAV/2 PHP/5.1.1 mod_ssl/2.2.0 OpenSSL/0.9.8a mod_perl/2.0.2 Perl/v5.8.7
Database=mysql 5.0.16-nt-log
Gallery-URL=http://lf-photodesign.de

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Mon, 2005-12-19 20:31

you'll have to add a progress bar view to compute all the statistics and then cache the result (db or GalleryDataCache::putToDisk) for future requests until a timeout is reached upon which you run the computation again.

 
sandymac
sandymac's picture

Joined: 2005-09-15
Posts: 31
Posted: Tue, 2005-12-20 02:37

RFE: An API for other modules to record stats with. My thinking was that my Coralize modules could register that it would want to keep track of some stats in various buckets and then when it did things it could add amounts to those buckets and the stats module would take care of presenting that info to the user.

I've updated: http://codex.gallery2.org/index.php/Gallery2:Modules:Statistics#Feature_request

 
valiant

Joined: 2003-01-04
Posts: 32509
Posted: Tue, 2005-12-20 08:42

I'm +1 for modularization of stats. but the stats module should keep the history / last results and return them to each module such that they can compute new results based on the old ones.

similarely to the function getRewriteRules() that can be implemented by any module and is checked by the rewrite module, there could be something like

function getStatistics($params, &$data) {
whereas $params contains options and the timestamp of the last time data was requested and $data contains the current from the last run and is meant to be updated.
the module can choose to compute everything from new or to update $data.
...
return GalleryStatus::success;
}

 
id01

Joined: 2005-11-07
Posts: 62
Posted: Wed, 2005-12-28 04:28

Is there anyway I can generates a user accessable page with the lastest statistic?

 
mincel
mincel's picture

Joined: 2005-10-14
Posts: 134
Posted: Sun, 2006-01-01 22:34

Thank you for this nice module... Great work!

http://www.mincel.com/gallery

 
turnbulm

Joined: 2004-10-03
Posts: 431
Posted: Mon, 2006-01-02 17:50

Another +1 for other modules being able to register stats. I'd like the Checkout module to be able to return things like the number of completed transactions, abandoned carts, total order values, etc, and the Statistics module seems the ideal place to send that information.

 
gbo
gbo's picture

Joined: 2005-10-06
Posts: 24
Posted: Tue, 2006-01-03 23:04

2aidanlister
Thanks for this module,

Could you describe what you plan to integrate in
- User Stats [Removed for the moment]
- All Entries [Removed for the moment]

Here is my whish list :-)

For User Stats :
- Top20 most recent requesting hosts/country
- Top50 requesting hosts/country

For All Entries :
- Most viewed image/gallery
- Weekly top50 images/galleries and daily top20 images/galleries
- List all or X galleries sorted by the number of views
- List all or X images sorted by the number of views

For Latest View :
- Will it be possible to add Hour, Host/IP and if possible country ?

Thanks

_________________
Gil
My Gallery http://mesphotos.fontismedia.com/

 
aidanlister

Joined: 2005-12-07
Posts: 56
Posted: Sat, 2006-01-21 01:26

I've just released a 0.2.0 version. Nothing much has changed, I've just fixed a few bugs and cleaned up some of the code. I'll get working on the features requested once I get some spare time! Thanks for your support!

ZyreX: I'll fix that in the 0.3.0 release by generating the statistics once, then displaying them.

LFrank: See above :)

sandymac & others: I'll start working on an API for this once I get some free time. It's a great idea.

id01: Yep, that's planned for the next release too. I'll make a statistics block available and customisable.

gbo: User Stats would have information about each of the users galleries, how many pics, how often they look at it, etc. All entries would list all the items in the gallery, not feasable for any gallery of decent size. In regards to your wish list, unfortunately gallery doesn't take or store any of that information, you're going to have to examine your server logs :(

 
id01

Joined: 2005-11-07
Posts: 62
Posted: Mon, 2006-01-23 19:50

How easy would it be to get a total page views inside an album? Instead of just views of each individial items?

 
eduo

Joined: 2003-09-10
Posts: 107
Posted: Tue, 2006-01-31 11:30

The current version doesn't work with the latest snapshot as the core APIs have changed and don't validate any more.

Is there any CVS version of the module available that would work with the new APIs?

Eduo
---
www.eduo.info
www.eduo.info/gallery/
www.hamsterspit.com

 
technobilder

Joined: 2003-02-09
Posts: 214
Posted: Sat, 2006-02-11 17:22

Would it be possible to add "last sent eCards" to the statistics module?
So that it's viewable whicheCard is sent most and possibly who wrote which text (in case of spam or other abuse).

 
technobilder

Joined: 2003-02-09
Posts: 214
Posted: Sat, 2006-02-11 17:27

Another one wit G2.0.2 and 0.2.0 stats module

My G2 is quite large:

Total Items 90850
Total Albums 956
Total Views 1863622
Total Users 3
Total Comments 878

When I click on "Disc space" I get an error after a while. But 25MB seems to be quite a lot for a script.

Fatal error: Allowed memory size of 26214400 bytes exhausted (tried to allocate 88 bytes) in /www/htdocs/techno/gallery2/modules/statistics/classes/StatisticsAdminDiskUsageHelper.class on line 153

 
solid

Joined: 2006-02-13
Posts: 1
Posted: Mon, 2006-02-13 07:53

first off:
-- great module..
-- very slick and easy to mod :)

found a bug in the 0.2.0 release, templates/StatisticsAdminLatestEvents.tpl, line 64:
instead of iterating over $StatisticsAdmin.LatestEvents.latestAdditions in the "Latest Additions" section you again iterate over $StatisticsAdmin.LatestEvents.latestViews

other than that great job!

-- solid

 
aidanlister

Joined: 2005-12-07
Posts: 56
Posted: Mon, 2006-02-27 15:25

The current CVS version (0.2.1) has been updated to use the new Gallery API. I've fixed the bug solid has reported (iterating wrong var), but have not had time to address any other issues - really sorry.

 
Eka_Mei

Joined: 2006-01-03
Posts: 132
Posted: Mon, 2006-02-27 21:20

Looking for a way to hack this module to export total page views per each album. For another word, it will add up all the total hits for all items under an album.
Any idea How I should do this to generates a page that list the views, similar to the Disk usage page?

 
vahan1

Joined: 2002-11-01
Posts: 27
Posted: Thu, 2006-03-02 05:01

First, Thanks for the awesome module!

Second...

I'm experience a weird issue where under Latest Events, if a picture has no "caption", the filename with the link doesn't show. Gets more interesting where another installation of gallery on the same codebase works fine.

I've tried matching theme to the working gallery theme - same result. Am I missing setting a permission somewhere ? What else can this be caused by??

Thanks a lot
Vahan

 
aidanlister

Joined: 2005-12-07
Posts: 56
Posted: Fri, 2006-03-03 01:48

Eka_Mei: the algorithm to do this is quite complex, when I get some spare time I'll implement it.
vahan1: that bug was fixed in 0.2.0 release, check you have this version installed.

 
vahan1

Joined: 2002-11-01
Posts: 27
Posted: Sat, 2006-03-04 07:35

aidanlister,

thanks for the tip. Actually I do have 0.2.0 installed, but I upgraded from the version with the bug.

I had to uninstall the module, delete <specific-gallery-data-directory>/cache/module/statistics, and reinstall.

worked like a charm.

thanks again :)

-v

 
Ranger187
Ranger187's picture

Joined: 2005-12-02
Posts: 204
Posted: Mon, 2006-03-13 09:11

Incompatible module!
Core API Required: 6.8 (available: 7.2)
Module API Required: 2.0 (available: 3.0)

Does this not work with 2.1-rc-2a?

 
eduo

Joined: 2003-09-10
Posts: 107
Posted: Mon, 2006-03-13 10:24

Ranger187:

It doesn't work yet with 2.1-rc-2a out-of-the-box. The module may be updated soon to accomodate for the API changes but I wouldn't expect anything final until there is a 2.1 final out as well. The statistics module accesses a lot of core functions from G2 so it needs to work on a final version of it.

Eduo
---
www.eduo.info
www.eduo.info/gallery/
www.hamsterspit.com

 
IamNear

Joined: 2006-03-08
Posts: 57
Posted: Mon, 2006-03-13 13:17

Provides statistics about Gallery
Incompatible module!
Core API Required: 6.8 (available: 7.1)
Module API Required: 2.0 (available: 3.0)

Same thing, any update on compatibility?

 
Ranger187
Ranger187's picture

Joined: 2005-12-02
Posts: 204
Posted: Mon, 2006-03-13 19:05

Iamnear,

Just like he told me, it's not fully working with 2.1 yet. Now if you are not running 2.1 you should have posted your version number to begin with. ;)

 
LFrank

Joined: 2005-02-19
Posts: 1023
Posted: Mon, 2006-03-13 19:41

if you're running 2.1 - the CVS version neccessary is available to test too.

Gallery version = 2.1-rc-2a core 1.0.34
PHP version = 5.1.1 apache2handler
Webserver = Apache/2.2.0 (Win32) DAV/2 PHP/5.1.2 mod_ssl/2.2.0 OpenSSL/0.9.8a
Database = mysql 5.0.18-nt-log,
Theme=PGtheme,
Gallery-URL=http://lf-photodesign.de

 
IamNear

Joined: 2006-03-08
Posts: 57
Posted: Sat, 2006-03-18 15:05

Okay, any time frame available for when this module will be available for 2.1?

Gallery version = 2.1-rc-1a core 1.0.28
PHP version = 4.3.11 apache
Webserver = Apache/1.3.33 (Unix) mod_auth_passthrough/1.8 mod_jk/1.2.0 mod_log_bytes/1.2 mod_bwlimited/1.4 PHP/4.3.11 FrontPage/5.0.2.2635 mod_ssl/2.8.22 OpenSSL/0.9.6b
Database = mysql 4.0.25-standard
Toolkits = Gd, Exif, Thumbnail
Operating system = Linux host.bh.xgb.net 2.4.20-28.7smp #1 SMP Thu Dec 18 11:18:31 EST 2003 i686
Browser = Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322)

 
aidanlister

Joined: 2005-12-07
Posts: 56
Posted: Mon, 2006-03-20 00:59

The current CVS version (0.2.1) has been updated to use the new Gallery API (Gallery version 2.1).

http://cvs.sourceforge.net/viewcvs.py/gallery-contrib/gallery2/modules/statistics/

 
eduo

Joined: 2003-09-10
Posts: 107
Posted: Mon, 2006-03-20 13:43

Is there any way to download the latest CVS snapshot for it in a tarball? The machine I'm currently in can't do CVS and saving each file manually is a PITA (I'll never understand why the viewcvs.py can't do this itself: Download Dir as tarball)

Eduo
---
www.eduo.info
www.eduo.info/gallery/
www.hamsterspit.com

 
longstag

Joined: 2006-03-24
Posts: 12
Posted: Mon, 2006-03-27 14:48

For all users who don't want to use cvs enclosed the statistics module for Gallery 2.1 as zip.

--
http://www.longstag.com

 
eduo

Joined: 2003-09-10
Posts: 107
Posted: Mon, 2006-03-27 23:24

Thanks a lot, longstag. Works a treat.

Eduo
---
www.eduo.info
www.eduo.info/gallery/
www.hamsterspit.com

 
IamNear

Joined: 2006-03-08
Posts: 57
Posted: Tue, 2006-03-28 13:23

I was rather disappointed with statistics, I was hoping this module would offer specific information on individual user sessions from what images (and for how long) a user views -- to tracking their entire gallery viewing sessions.

Are there any plans for this?

See logging module thread http://gallery.menalto.com/node/46376

 
shocksll
shocksll's picture

Joined: 2005-06-22
Posts: 352
Posted: Tue, 2006-03-28 14:10

I use bbclone to accomplish that type of use profiling. See http://codex.gallery2.org/index.php/Gallery2:How_to_embed_counter_code_at_the_page_bottom#BBClone

BTW, I love the statistics module and I think that it combined with BBClone gives me all the information I need. Thanks.

Steve Lineberry