Thread: [Cgi-session-user] MySQL backend - multiple copies of session id's stored?
Brought to you by:
sherzodr
From: Justin S. <ju...@sk...> - 2006-07-28 22:41:44
|
Hey everyone, before I delve any further into a problem I'm having, Is it normal for the MySQL backend to store multiple copies of the session in a MySQL backend? The other backends I've tried - file, Db, Postgres, don't seem to have this behavior - I'm guessing this is because, the file backend has the name of the session_id as its filename, so any multiples get re-written - same goes for the Berkeley DB backend - keys (I think) in a tied hash have to be unique, or you'll overwrite your information already saved. No clue on Postgres ;) I've had some bug reports: http://sourceforge.net/tracker/index.php? func=detail&aid=1525557&group_id=13002&atid=113002 that, for some reason, the more sessions that are saved, the more multiples of the next session created are made, perhaps in an almost Fibonacci growth :) It gets to the point that many thousands of entries are made for the same session, slowing things right down. Before I blame CGI::Session, I was wondering if any specific behavior of my program may be at fault to create such a scenario. Given the advice of the docs and this list, every change done to the sessions - adding/editing/removing is followed by a ->flush method call. I'm willing to make a test up that I'll try running, but first wanted to see if this is behavior is actually wrong. My program is of course, pretty meaty, so making a simplified version of what's going on, may be a little bit of a chore (that I am willing to do). Justin Simoni -- :: is an eccentric artist, living and working in Denver, Colorado :: URL: http://justinsimoni.com :: Mailing List - http://justinsimoni.com/mailing_list.html ps: Sherzod - moved to the CGI::Session::ExpireSessions for my other problem - thanks! |
From: Mark S. <ma...@su...> - 2006-07-29 13:12:03
|
Justin Simoni wrote: > Hey everyone, > > before I delve any further into a problem I'm having, > > Is it normal for the MySQL backend to store multiple copies of the > session in a MySQL backend? No. But read the source of the MySQL driver module to confirm for yourself. It's rather simple. > I've had some bug reports: > > http://sourceforge.net/tracker/index.php?func=detail&aid=1525557&group_id=13002&atid=113002 The bug report clarifies something. It's not /duplicate/ sessions are being saved, but rather, lots of new, different sessions. That sounds like it might be an application error. Review the calls to CGI::Session->new(). Add debugging statements near there and try again. That should clarify who is at fault. Remember that's fine for the same session to be used before and after someone logs in. The difference might be the presence of "logged_in => 1" in the session. (And random suggestion: The YAML serializer creates output that is a lot nicer to look it for debugging than the default one. ) > Given the advice of the docs and this list, every change done to the > sessions - adding/editing/removing is followed by a ->flush method > call. I'm willing to make a test up that I'll try running, but first > wanted to see if this is behavior is actually wrong. I think flush() just means "synchronize memory with disk". So this design is OK. In theory, doing it once at the end of the request cycle would be sufficient, or just before you need to run a DB query that depends on the flushed state() > My program is of course, pretty meaty, I like to think of it a "pretty tofuey" or just "high in protein". Mark -- http://mark.stosberg.com/ |
From: Justin S. <ju...@sk...> - 2006-07-29 23:54:04
|
Here's an update: There's been a few problems in my code - the first was with login of my webapp - before login would even take place, the app makes sure you're not logged in, with a different set of credentials (basically, if you're logged in as one user, it won't let you log in as a different user, until you log out) The way it did this was to look at the saved session information and make sure everything matches - fine and dandy, but if there is no saved session information, CGI::Session will make it. Many ways to fix this, I just see if the person logged in, isn't logged in as a different user, that current session is deleted. I could probably also use load? which wouldn't create a new session, and then just try use the, is_empty() method. I'm still having trouble with accessing a session already created, something simple as: require CGI::Session; CGI::Session->name($LOGIN_COOKIE_NAME); $session = CGI::Session->load($self->{dsn}, $q, $self-> {dsn_args}); Will not just read the already created session, it'll create a new session, with the exact same session id and exact same information inside the a_session column. This snippet of code happens every time the session information is checked - basically everytime you bring a new screen up in the webapp and that's one way of how the session tables starts to get a little full (not THOUSANDS as my bug poster put it, but still) Any reason for this to happen? Justin On Jul 29, 2006, at 8:08 AM, Mark Stosberg wrote: > Justin Simoni wrote: >> Hey everyone, >> before I delve any further into a problem I'm having, >> Is it normal for the MySQL backend to store multiple copies of >> the session in a MySQL backend? > > No. But read the source of the MySQL driver module to confirm for > yourself. It's rather simple. > >> I've had some bug reports: >> http://sourceforge.net/tracker/index.php? >> func=detail&aid=1525557&group_id=13002&atid=113002 > > The bug report clarifies something. It's not /duplicate/ sessions > are being saved, but rather, lots of new, different sessions. That > sounds like it might be an application error. Review the calls to > CGI::Session->new(). Add debugging statements near there and try > again. > That should clarify who is at fault. > > Remember that's fine for the same session to be used before and > after someone logs in. The difference might be the presence of > "logged_in => 1" in the session. > > (And random suggestion: The YAML serializer creates output that is > a lot nicer to look it for debugging than the default one. ) > >> Given the advice of the docs and this list, every change done to >> the sessions - adding/editing/removing is followed by a ->flush >> method call. I'm willing to make a test up that I'll try running, >> but first wanted to see if this is behavior is actually wrong. > > I think flush() just means "synchronize memory with disk". So this > design is OK. In theory, doing it once at the end of the request > cycle would be sufficient, or just before you need to run a DB > query that depends on the flushed state() > >> My program is of course, pretty meaty, > > I like to think of it a "pretty tofuey" or just "high in protein". > > Mark > > -- > http://mark.stosberg.com/ > > |
Re: [Cgi-session-user] how load works (was: MySQL backend -
multiple copies of session id's stored?)
From: Mark S. <ma...@su...> - 2006-07-30 02:33:17
|
> > I could probably also use load? which wouldn't create a new session, and > then just try use the, is_empty() method. > > I'm still having trouble with accessing a session already created, > something simple as: > > require CGI::Session; > CGI::Session->name($LOGIN_COOKIE_NAME); > $session = CGI::Session->load($self->{dsn}, $q, > $self->{dsn_args}); There's probably a bug here. Could you boil this down to a Test::More style test case? It looks like you are nearly there. Looking at the docs for load(), it says: Constructor. Usage is identical to new(), so is the return value. Major difference is, new() can create new session if it detects expired and non-existing sessions, but load() does not. load() is useful to detect expired or non-existing sessions without forcing the library to create new sessions. This contains an apparent contradiction, because it says the return value is the same as new(), but new() returns newly created sessions. That conflicts with the next paragraph that says it doesn't create a new session. Looking at the code, load() /does/ have the ability to return new, empty sessions, which seems wrong. Do others agree? "new()" is already magical that way. It doesn't seem that we need "load" to mean "load_or_new" as well. > Will not just read the already created session, it'll create a new > session, with the exact same session id and exact same information > inside the a_session column. You mean, it's cloning the session of loading it, resulting in duplicate entries in the database? That seems weird. Mark |
Re: [Cgi-session-user] how load works (was: MySQL backend -
multiple copies of session id's stored?)
From: Sherzod R. <she...@ha...> - 2006-07-30 03:39:39
|
Mark, Right before I was coding load() I had a need for such functionality, although no matter how hard I try to remember, I can't recall what = exactly created that need. But since then I wrote lots of code, and not even = once remember having to use load(). So, I'm not exactly sure what the purpose = of load() is, except adding to the complexity of the already complex code. Usually in an application there really is no need for checking for the existence of the actual session. It's too low level to be even concerned about. Application should create a session at request, and check for existence of certain session parameters. At least that's how I do it = these days. Notifying the users about their expired sessions is also something not = quite necessary. Displaying the login screen conveys the same message, and = also, provides with an option to re-create a new login session, which in turn saves bandwidth, and at least couple lines of code, if not more. Sherzod > -----Original Message----- > From: cgi...@li...=20 > [mailto:cgi...@li...] On=20 > Behalf Of Mark Stosberg > Sent: Saturday, July 29, 2006 11:30 PM > To: Justin Simoni > Cc: List - CGI-Session > Subject: Re: [Cgi-session-user] how load works (was: MySQL=20 > backend - multiple copies of session id's stored?) >=20 >=20 > > > > I could probably also use load? which wouldn't create a=20 > new session, and > > then just try use the, is_empty() method. > > > > I'm still having trouble with accessing a session already created, > > something simple as: > > > > require CGI::Session; > > CGI::Session->name($LOGIN_COOKIE_NAME); > > $session =3D CGI::Session->load($self->{dsn}, $q, > > $self->{dsn_args}); >=20 >=20 > There's probably a bug here. Could you boil this down to a Test::More > style test case? It looks like you are nearly there. >=20 > Looking at the docs for load(), it says: >=20 > Constructor. Usage is identical to new(), so is the return=20 > value. Major > difference is, new() can create new session if it detects=20 > expired and > non-existing sessions, but load() does not. >=20 > load() is useful to detect expired or non-existing=20 > sessions without=20 > forcing the > library to create new sessions. >=20 > This contains an apparent contradiction, because it says the return > value is the same as new(), but new() returns newly created sessions. > That conflicts with the next paragraph that says it doesn't=20 > create a new > session. >=20 > Looking at the code, load() /does/ have the ability to return=20 > new, empty > sessions, which seems wrong. Do others agree? "new()" is already > magical that way. It doesn't seem that we need "load" to mean > "load_or_new" as well. >=20 > > Will not just read the already created session, it'll create a new > > session, with the exact same session id and exact same information > > inside the a_session column. >=20 > You mean, it's cloning the session of loading it, resulting=20 > in duplicate > entries in the database? That seems weird. >=20 > Mark >=20 > -------------------------------------------------------------- > ----------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the=20 > chance to share your > opinions on IT & business topics through brief surveys -- and=20 > earn cash > http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge &CID=3DDEVDEV _______________________________________________ Cgi-session-user mailing list Cgi...@li... https://lists.sourceforge.net/lists/listinfo/cgi-session-user |
Re: [Cgi-session-user] how load works (was: MySQL backend -
multiple copies of session id's stored?)
From: Justin S. <ju...@sk...> - 2006-07-30 22:32:54
|
> There's probably a bug here. Could you boil this down to a Test::More > style test case See what I can do ;) A good exercise anyways, since sometimes doing just that makes you realize some other complication in the code. Justin On Jul 29, 2006, at 9:29 PM, Mark Stosberg wrote: >> >> I could probably also use load? which wouldn't create a new >> session, and >> then just try use the, is_empty() method. >> >> I'm still having trouble with accessing a session already created, >> something simple as: >> >> require CGI::Session; >> CGI::Session->name($LOGIN_COOKIE_NAME); >> $session = CGI::Session->load($self->{dsn}, $q, >> $self->{dsn_args}); > > > There's probably a bug here. Could you boil this down to a Test::More > style test case? It looks like you are nearly there. > > Looking at the docs for load(), it says: > > Constructor. Usage is identical to new(), so is the return > value. Major > difference is, new() can create new session if it detects > expired and > non-existing sessions, but load() does not. > > load() is useful to detect expired or non-existing sessions without > forcing the > library to create new sessions. > > This contains an apparent contradiction, because it says the return > value is the same as new(), but new() returns newly created sessions. > That conflicts with the next paragraph that says it doesn't create > a new > session. > > Looking at the code, load() /does/ have the ability to return new, > empty > sessions, which seems wrong. Do others agree? "new()" is already > magical that way. It doesn't seem that we need "load" to mean > "load_or_new" as well. > >> Will not just read the already created session, it'll create a new >> session, with the exact same session id and exact same information >> inside the a_session column. > > You mean, it's cloning the session of loading it, resulting in > duplicate > entries in the database? That seems weird. > > Mark > > ---------------------------------------------------------------------- > --- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to > share your > opinions on IT & business topics through brief surveys -- and earn > cash > http://www.techsay.com/default.php? > page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Cgi-session-user mailing list > Cgi...@li... > https://lists.sourceforge.net/lists/listinfo/cgi-session-user > |
Re: [Cgi-session-user] how load works (was: MySQL backend -
multiple copies of session id's stored?)
From: Justin S. <ju...@sk...> - 2006-07-30 23:41:49
Attachments:
dup_sess_test.pl
|
> There's probably a bug here. Could you boil this down to a Test::More > style test case? It looks like you are nearly there. Done! Be gentle - this is my first real Test::More test - I know these are supposed to be pretty simple, but setting up all the DBI parameters seems a bit verbose, but I've attached it to this message, as well as attempted to place it inline (notes to follow code): [snip] #!/usr/bin/perl use Test::More tests => 4; use CGI::Session; use DBI; # SQL Paramaters - obviously, these will be different from you... my %sql_params = ( database => 'database', dbserver => 'localhost', # may just be, "localhost" port => '3306', # mysql: 3306, Postgres: 5432 dbtype => 'mysql', # 'mysql' for 'MySQL', 'Pg' for 'PostgreSQL' user => 'user', pass => 'pass', session_table => 'sessions', ); # Let's get a db handle... my $dbh = connectdb( $sql_params{dbtype}, $sql_params{database}, $sql_params{dbserver}, $sql_params{3306}, $sql_params{user}, $sql_params{pass}, ); # Some CGI::Session params... my $dsn = 'driver:mysql'; my $dsn_args = { Handle => $dbh, TableName => $sql_params{session_table}, }; # Let's start with a clean slate... $dbh->do("DELETE FROM " . $sql_params{session_table}); # Build us a session object... my $session = new CGI::Session($dsn, undef, $dsn_args); $session->param('foo', 'bar'); $session->expire('+1d'); $session->flush(); # Check the integrity of our saved information.... ok($session->param('foo') eq 'bar', "Correct information has been saved in the session..."); # Save this for later, so we can recall the info... my $session_id = $session->id; # Hey, let's see how many rows we have... my $sth = $dbh->prepare("SELECT COUNT(*) FROM " . $sql_params {session_table} ); $sth->execute(); # (Hopefully) we only have one session... ok($sth->fetchrow_array() == 1, "Only one copy of the session file..."); # In the app itself, the Session is checked upon a refresh to a new screen... # So let's get rid of what we have, and do it again... undef $session; undef $dbh; # just being thorough. # Our new DB handle... # There's no persistance in the CGI app... my $dbh2 = connectdb( $sql_params{dbtype}, $sql_params{database}, $sql_params{dbserver}, $sql_params{3306}, $sql_params{user}, $sql_params{pass}, ); # And again.. my $dsn2_args = { Handle => $dbh2, TableName => $sql_params{session_table}, }; # New Session! Should call up the same information... my $session2 = CGI::Session->load($dsn, $session_id, $dsn2_args); # Check the integrity of our saved information.... ok($session2->param('foo') eq 'bar', "Information is retrieved from past session alright..."); # NOTE: UNCOMMENTING EITHER OF THESE TWO LINES WILL CAUSE THE NEXT TEST TO FAIL # $session2->flush; # undef $session2; # ??? # How many do we have?! my $sth2 = $dbh2->prepare("SELECT COUNT(*) FROM " . $sql_params {session_table} ); $sth2->execute(); # One? Two? ok($sth2->fetchrow_array() == 1, "Still only one copy of the session..."); # This is to double check out handy work: # print "\n\nLet's see what's in the table:\n" . '-' x 72 . "\n"; my $sth3 = $dbh2->prepare("SELECT * FROM " . $sql_params {session_table} ); $sth3->execute(); $sth3->dump_results(2048); [/snip] NOTES: I tried to emulate how things are running in the (I admit, complex) web app - does a good job. In the web app, upon login, a session is created, and set in a cookie. The person will receive the cookie and the page will refresh to the administration side of things - that's why there's the creation of a new session (and db handle, etc). Plop in your own DB credentials, obviously. All the above tests (4) PASS - until you uncomment one of the two lines: Either: # $session2->flush; or: # undef $session2; For whatever reason, this will write a second, similar session into the table, with the same session id and same parameters saved. And that's leading to some problems. I've only tested this on a server running mysql, but it doesn't (without testing) seem to be a problem with Postgres which is a little mysterious. Here's the output if you uncomment one of the above lines: me@there [~/where/we/are]# perl dup_sess.pl 1..4 ok 1 - Correct information has been saved in the session... ok 2 - Only one copy of the session file... ok 3 - Information is retrieved from past session alright... not ok 4 - Still only one copy of the session... # Failed test (dup_sess.pl at line 123) Let's see what's in the table: ------------------------------------------------------------------------ 'c211b11dc99e01c9f5685fd39391d6eb', '$D = {'_SESSION_ETIME' => 86400,'_SESSION_ID' => 'c211b11dc99e01c9f5685fd39391d6eb','_SESSION_ATIME' => 1154302807,'foo' => 'bar','_SESSION_REMOTE_ADDR' => '','_SESSION_CTIME' => 1154302807};;$D' 'c211b11dc99e01c9f5685fd39391d6eb', '$D = {'_SESSION_ID' => 'c211b11dc99e01c9f5685fd39391d6eb','_SESSION_ETIME' => 86400,'_SESSION_ATIME' => 1154302807,'foo' => 'bar','_SESSION_EXPIRE_LIST' => {},'_SESSION_REMOTE_ADDR' => '','_SESSION_CTIME' => 1154302807};;$D' 2 rows # Looks like you failed 1 test of 4. me@there [~/where/we/are]# Thanks Mark for showing me how to whip up Test's :) Justin Simoni -- :: is an eccentric artist, living and working in Denver, Colorado :: URL: http://justinsimoni.com :: PHO: 720.436.7701 :: Mailing List - http://justinsimoni.com/mailing_list.html |
From: Mark S. <ma...@su...> - 2006-07-31 02:52:20
|
Justin Simoni wrote: >> There's probably a bug here. Could you boil this down to a Test::More >> style test case? It looks like you are nearly there. > > Done! Be gentle - this is my first real Test::More test - I know these > are supposed to be pretty simple, but setting up all the DBI parameters > seems a bit verbose, but I've attached it to this message, as well as > attempted to place it inline (notes to follow code): Thanks Justin. I adapted this test for the SQLite driver... and it passed when it fails for MySQL. So that points to something MySQL-specific. Looking at the code for the MySQL driver, it does do something a little different. It use the non-standard "REPLACE INTO" function. It's documented here: http://sunsite.mff.cuni.cz/MIRRORS/ftp.mysql.com/doc/en/REPLACE.html Your test excluded creating the session table itself. Notice in the docs for REPLACE INTO where it says that if there is no UNIQUE or PRIMARY KEY constraint, then it will always INSERT, never update. That sounds a lot like your case. If that's the issue. we should add a big note to the MySQL driver about that. Mark -- http://mark.stosberg.com/ |
From: Justin S. <ju...@sk...> - 2006-07-31 03:17:38
|
> Your test excluded creating the session table itself. Notice in the > docs for REPLACE INTO where it says that if there is no UNIQUE or > PRIMARY KEY constraint, then it will always INSERT, never update. Ah ha. Indeed, here is the table I've been using: CREATE TABLE sessions ( id CHAR(32) NOT NULL, a_session TEXT NOT NULL ); The docs do sort of complicate matters, as it's stated what the minimum table is for "most" SQL backends are here: http://search.cpan.org/~markstos/CGI-Session-4.14/lib/CGI/Session/ Driver/DBI.pm#STORAGE But, the docs specifically for MySQL: http://search.cpan.org/~markstos/CGI-Session-4.14/lib/CGI/Session/ Driver/mysql.pm Don't have any mention of how to create the table (so I have to guess?), but there are docs from the 3.x series: http://search.cpan.org/~sherzodr/CGI-Session-3.95/Session/MySQL.pm (which I found just searching on CPAN for, easy to forget to look at the version numbers) That do have directions for MySQL. Going through all that, I'm assuming the MySQL can use the generic table stated in the DBI.pm docs, and Postgres uses what is listed here: http://search.cpan.org/~markstos/CGI-Session-4.14/lib/CGI/Session/ Driver/postgresql.pm OK, I can apply that change to my docs as well :) Thanks for helping out with this; Justin Simoni -- :: is an eccentric artist, living and working in Denver, Colorado :: URL: http://justinsimoni.com :: PHO: 720.436.7701 :: Mailing List - http://justinsimoni.com/mailing_list.html On Jul 30, 2006, at 9:48 PM, Mark Stosberg wrote: > Justin Simoni wrote: >>> There's probably a bug here. Could you boil this down to a >>> Test::More >>> style test case? It looks like you are nearly there. >> Done! Be gentle - this is my first real Test::More test - I know >> these are supposed to be pretty simple, but setting up all the DBI >> parameters seems a bit verbose, but I've attached it to this >> message, as well as attempted to place it inline (notes to follow >> code): > > Thanks Justin. > > I adapted this test for the SQLite driver... and it passed when it > fails for MySQL. So that points to something MySQL-specific. > Looking at the code for the MySQL driver, it does do something a > little different. It use the non-standard "REPLACE INTO" function. > It's documented here: > > http://sunsite.mff.cuni.cz/MIRRORS/ftp.mysql.com/doc/en/REPLACE.html > > Your test excluded creating the session table itself. Notice in the > docs for REPLACE INTO where it says that if there is no UNIQUE or > PRIMARY KEY constraint, then it will always INSERT, never update. > > That sounds a lot like your case. If that's the issue. we should > add a big note to the MySQL driver about that. > > Mark > > -- > http://mark.stosberg.com/ > > |
From: Justin S. <ju...@sk...> - 2006-07-31 03:26:59
|
Final followup, Dropping the session table and creating this one: CREATE TABLE sessions ( id CHAR(32) NOT NULL PRIMARY KEY, a_session TEXT NOT NULL ); And running the Test::More test Mark made me create has it pass in every case I noted, with or without either of those two lines commented - looks like we're in good shape! Testing the web app itself, I'm seeing similar positive results - multiple copies of the session information are not being created - one session entry per session - perfect. Thanks for all the help, would never have figured that out :) (Justin realizes the power of the Test::More test script, will file in back of mind for later use...) Justin Simoni -- :: is an eccentric artist, living and working in Denver, Colorado :: URL: http://justinsimoni.com :: Mailing List - http://justinsimoni.com/mailing_list.html On Jul 30, 2006, at 9:48 PM, Mark Stosberg wrote: > Justin Simoni wrote: >>> There's probably a bug here. Could you boil this down to a >>> Test::More >>> style test case? It looks like you are nearly there. >> Done! Be gentle - this is my first real Test::More test - I know >> these are supposed to be pretty simple, but setting up all the DBI >> parameters seems a bit verbose, but I've attached it to this >> message, as well as attempted to place it inline (notes to follow >> code): > > Thanks Justin. > > I adapted this test for the SQLite driver... and it passed when it > fails for MySQL. So that points to something MySQL-specific. > Looking at the code for the MySQL driver, it does do something a > little different. It use the non-standard "REPLACE INTO" function. > It's documented here: > > http://sunsite.mff.cuni.cz/MIRRORS/ftp.mysql.com/doc/en/REPLACE.html > > Your test excluded creating the session table itself. Notice in the > docs for REPLACE INTO where it says that if there is no UNIQUE or > PRIMARY KEY constraint, then it will always INSERT, never update. > > That sounds a lot like your case. If that's the issue. we should > add a big note to the MySQL driver about that. > > Mark > > -- > http://mark.stosberg.com/ > > |
From: Mark S. <ma...@su...> - 2006-07-31 03:44:52
|
Justin Simoni wrote: > Final followup, > > Dropping the session table and creating this one: > > CREATE TABLE sessions ( > id CHAR(32) NOT NULL PRIMARY KEY, > a_session TEXT NOT NULL > ); > > And running the Test::More test Mark made me create has it pass in every > case I noted, with or without either of those two lines commented - > looks like we're in good shape! Great, I patched the docs and committed that change to SVN. You got credit for your help bring it to our attention. Thanks! Mark |