LOTS of sessions

luc7v

Joined: 2010-11-05
Posts: 61

Posted: Thu, 2012-01-12 09:34

I see 112000 records in sessions table. About 150MB. Is this normal??? If not, what should I do?

Thank you,
Lucian

floridave

Joined: 2003-12-22
Posts: 27300

Posted: Fri, 2012-01-13 00:48

You can safely truncate that table if you wish. Anybody logged in will need to log back in.
I have not looked at the code to see if we check the size or table or not.
What is your issue?

Dave
_____________________________________________
Blog & G2 || floridave - Gallery Team

luc7v

Joined: 2010-11-05
Posts: 61

Posted: Fri, 2012-01-13 13:15

I'm trying to get more speed. I think sessions table is accessed for every page, so if it's so big, it's not good.

Can I filter out sessions for search emgines? Or even better, users not logged in?

Thank you,
Lucian

luc7v

Joined: 2010-11-05
Posts: 61

Posted: Sun, 2012-01-22 12:37

I truncated the sessions table.

Now I see 175.000 records. The table has 200MB!

Please help, I have speed problems! I know PHP/MySQL, but I need some starting point.

floridave

Joined: 2003-12-22
Posts: 27300

Posted: Sun, 2012-01-22 18:39

I don't know how we or if we limit that table size.
I have 155 rows in my sessions table and truncated it on 2012-01-12 the time I made my post above.

Dave
_____________________________________________
Blog & G2 || floridave - Gallery Team

luc7v

Joined: 2010-11-05
Posts: 61

Posted: Sun, 2012-01-22 21:42

Is it any reason to have sessions for guest users? Can I hack G3 to not save sessions for guests?

Thank you,
Lucian

inposure

Joined: 2010-04-23
Posts: 304

Posted: Tue, 2012-01-24 00:40

Sessions keep combined CSS and JS in the DB as a cache for returning users.

However, if you have some hard-hitting bots from RU/UA/BR and the other usual suspects, chances are they will just create a new session for every hit.

This is obviously a problem. You can solve it by blocking typical offenders (from certain "data centers" and certain countries/regions/continents).

On the other hand, you could also have 100,000 genuine visitors, and then you are to be congratulated! Up your hardware accordingly.

For the actual problem, you could truncate the sessions table via maintenance or in local.php (create it) in your gallery root folder.

luc7v

Joined: 2010-11-05
Posts: 61

Posted: Tue, 2012-01-24 11:38

I have about 1000 visitors/day.

I don't know who to block.

Why are CSS and JS for every user?! At least for guest users, I can't imagine how they can be different.

bharat

Joined: 2002-05-21
Posts: 7994

Posted: Mon, 2012-02-06 01:18

We don't actually store the combined CSS/JS in the session any more - that's stored in the caches table.

I checked my production gallery and I have 13k sessions - more than I expected. So I wrote a new module called "Session explorer" that lets you see what the top 15 user agents and IP addresses look like. You can find the module in the gallery3-contrib repo. I found that we're not properly identifying Baidu, Bing and Ezooms so I added exclusions for them. Once we identify the request as a search engine we don't save their session.

You can mimic this commit in your copy of the code: https://github.com/gallery/gallery3/commit/1bbaafcc1fc1c4997860706219a628343694e44b or just upgrade to the latest 3.0.x snapshot. After you've done this, truncate your sessions table and you're good to go.
---
Problems? Check gallery3/var/logs
file a bug/feature ticket | upgrade to the latest code! | hacking G3? join us on IRC!

udaraaka

Joined: 2012-02-03
Posts: 29

Posted: Mon, 2012-02-06 17:24

Bharat you told that combined CSS/JS are stored in caches table. What is the purpose of storing them in the table? May be I am wrong but I have heard that accessing local files are faster than accessing a database.Is that table really improves the performance? Currently I have a cache table with 37MB data. If I truncate the caches table, is that affect the performance my website? 100% of my website visitors are guest visitors.

floridave

Joined: 2003-12-22
Posts: 27300

Posted: Tue, 2012-02-07 00:23

I have created a codex page for this new module:
http://codex.gallery2.org/Gallery3:Modules:session_explorer

Dave
_____________________________________________
Blog & G2 || floridave - Gallery Team

bharat

Joined: 2002-05-21
Posts: 7994

Posted: Tue, 2012-02-07 03:52

@udaraaka - yes, accessing local files is faster but files are harder to manage and the latency difference is not huge (it should be <10ms on most machines). If you truncate the caches table it will get rebuilt on demand very quickly. One thing I noticed in my gallery is that the bulk of the cache entries don't have associated tags so I can't see what they are. I filed that as https://sourceforge.net/apps/trac/gallery/ticket/1818 and submitted a fix - you can try updating your code and clearing the cache, then when it refills we can see what's taking up the most space with a query like this:

select distinct(tags) as d, count(*) as c from caches group by d;

@floridave - thanks!!
---
Problems? Check gallery3/var/logs
file a bug/feature ticket | upgrade to the latest code! | hacking G3? join us on IRC!

bharat

Joined: 2002-05-21
Posts: 7994

Posted: Tue, 2012-02-07 04:04

Oh, and I checked out my production gallery and I think that the cache is full of display context entries from search engines. I filed that as https://sourceforge.net/apps/trac/gallery/ticket/1819 and submitted a fix for that as well.
---
Problems? Check gallery3/var/logs
file a bug/feature ticket | upgrade to the latest code! | hacking G3? join us on IRC!

luc7v Joined: 2010-11-05 Posts: 61	Posted: Thu, 2012-01-12 09:34
	I see 112000 records in sessions table. About 150MB. Is this normal??? If not, what should I do? Thank you, Lucian
	XUser login Username: * Password: * Create new account Request new password
floridave Joined: 2003-12-22 Posts: 27300	Posted: Fri, 2012-01-13 00:48
	You can safely truncate that table if you wish. Anybody logged in will need to log back in. I have not looked at the code to see if we check the size or table or not. What is your issue? Dave _____________________________________________ Blog & G2 \|\| floridave - Gallery Team

luc7v Joined: 2010-11-05 Posts: 61	Posted: Fri, 2012-01-13 13:15
	I'm trying to get more speed. I think sessions table is accessed for every page, so if it's so big, it's not good. Can I filter out sessions for search emgines? Or even better, users not logged in? Thank you, Lucian

luc7v Joined: 2010-11-05 Posts: 61	Posted: Sun, 2012-01-22 12:37
	I truncated the sessions table. Now I see 175.000 records. The table has 200MB! Please help, I have speed problems! I know PHP/MySQL, but I need some starting point.

floridave Joined: 2003-12-22 Posts: 27300	Posted: Sun, 2012-01-22 18:39
	I don't know how we or if we limit that table size. I have 155 rows in my sessions table and truncated it on 2012-01-12 the time I made my post above. Dave _____________________________________________ Blog & G2 \|\| floridave - Gallery Team

luc7v Joined: 2010-11-05 Posts: 61	Posted: Sun, 2012-01-22 21:42
	Is it any reason to have sessions for guest users? Can I hack G3 to not save sessions for guests? Thank you, Lucian

inposure Joined: 2010-04-23 Posts: 304	Posted: Tue, 2012-01-24 00:40
	Sessions keep combined CSS and JS in the DB as a cache for returning users. However, if you have some hard-hitting bots from RU/UA/BR and the other usual suspects, chances are they will just create a new session for every hit. This is obviously a problem. You can solve it by blocking typical offenders (from certain "data centers" and certain countries/regions/continents). On the other hand, you could also have 100,000 genuine visitors, and then you are to be congratulated! Up your hardware accordingly. For the actual problem, you could truncate the sessions table via maintenance or in local.php (create it) in your gallery root folder.

luc7v Joined: 2010-11-05 Posts: 61	Posted: Tue, 2012-01-24 11:38
	I have about 1000 visitors/day. I don't know who to block. Why are CSS and JS for every user?! At least for guest users, I can't imagine how they can be different.

bharat Joined: 2002-05-21 Posts: 7994	Posted: Mon, 2012-02-06 01:18
	We don't actually store the combined CSS/JS in the session any more - that's stored in the caches table. I checked my production gallery and I have 13k sessions - more than I expected. So I wrote a new module called "Session explorer" that lets you see what the top 15 user agents and IP addresses look like. You can find the module in the gallery3-contrib repo. I found that we're not properly identifying Baidu, Bing and Ezooms so I added exclusions for them. Once we identify the request as a search engine we don't save their session. You can mimic this commit in your copy of the code: https://github.com/gallery/gallery3/commit/1bbaafcc1fc1c4997860706219a628343694e44b or just upgrade to the latest 3.0.x snapshot. After you've done this, truncate your sessions table and you're good to go. --- Problems? Check gallery3/var/logs file a bug/feature ticket \| upgrade to the latest code! \| hacking G3? join us on IRC!

udaraaka Joined: 2012-02-03 Posts: 29	Posted: Mon, 2012-02-06 17:24
	Bharat you told that combined CSS/JS are stored in caches table. What is the purpose of storing them in the table? May be I am wrong but I have heard that accessing local files are faster than accessing a database.Is that table really improves the performance? Currently I have a cache table with 37MB data. If I truncate the caches table, is that affect the performance my website? 100% of my website visitors are guest visitors.

floridave Joined: 2003-12-22 Posts: 27300	Posted: Tue, 2012-02-07 00:23
	I have created a codex page for this new module: http://codex.gallery2.org/Gallery3:Modules:session_explorer Dave _____________________________________________ Blog & G2 \|\| floridave - Gallery Team

bharat Joined: 2002-05-21 Posts: 7994	Posted: Tue, 2012-02-07 03:52
	@udaraaka - yes, accessing local files is faster but files are harder to manage and the latency difference is not huge (it should be <10ms on most machines). If you truncate the caches table it will get rebuilt on demand very quickly. One thing I noticed in my gallery is that the bulk of the cache entries don't have associated tags so I can't see what they are. I filed that as https://sourceforge.net/apps/trac/gallery/ticket/1818 and submitted a fix - you can try updating your code and clearing the cache, then when it refills we can see what's taking up the most space with a query like this: select distinct(tags) as d, count(*) as c from caches group by d; @floridave - thanks!! --- Problems? Check gallery3/var/logs file a bug/feature ticket \| upgrade to the latest code! \| hacking G3? join us on IRC!

bharat Joined: 2002-05-21 Posts: 7994	Posted: Tue, 2012-02-07 04:04
	Oh, and I checked out my production gallery and I think that the cache is full of display context entries from search engines. I filed that as https://sourceforge.net/apps/trac/gallery/ticket/1819 and submitted a fix for that as well. --- Problems? Check gallery3/var/logs file a bug/feature ticket \| upgrade to the latest code! \| hacking G3? join us on IRC!

LOTS of sessions

XUser login