Thread: [SSI-users] postgresql and load balancing
Brought to you by:
brucewalker,
rogertsang
From: Oliver U. <oli...@tu...> - 2013-02-15 11:12:39
|
Hi everybody, I've installed OpenSSI on a Debian 5 system together with postgresql. I'm able to migrate the postgresql server to a second node, but only all processes together. Running the postgresql benchmark pgbench all created postgresql processes also run on that node. If I try to migrate one of these worker processes to another node using echo 2 > /proc/68240/goto the process is migrated but the benchmark crashes with different messages, e.g.: Client 3 aborted in state 11: FATAL: semop(id=491526) failed: Bezeichner wurde entfernt or Client 2 aborted in state 8: FEHLER: lock RowExclusiveLock on object 16384/16391/0 is already held Did anybody have success load balancing postgresql? Best regards, Oliver -- Dipl.-Inf. Oliver Urbann Robotics Research Institute Section Information Technology TU Dortmund University 44221 Dortmund, Germany mailto:Oli...@tu... http://www.it.irf.tu-dortmund.de Phone: +49 231 755 6165 Fax: +49 231 755 3251 |
From: Mulyadi S. <mul...@gm...> - 2013-02-17 05:48:10
|
On Fri, Feb 15, 2013 at 5:49 PM, Oliver Urbann <oli...@tu...> wrote: > the process is migrated but the benchmark crashes with different > messages, e.g.: > > Client 3 aborted in state 11: FATAL: semop(id=491526) failed: > Bezeichner wurde entfernt > > or > > Client 2 aborted in state 8: FEHLER: lock RowExclusiveLock on object > 16384/16391/0 is already held > > Did anybody have success load balancing postgresql? > https://lists.sourceforge.net/lists/listinfo/ssic-linux-users IIRC, locks in OpenSSI is not system wide. Further more, when a process (the whole thread group in this case) migrated, then it is migrated entirely in the sense that it physically move. This is different than say MOSIX which use "stub" approach, so lock might still work because there is still communication between origin node and the new node (from the migration point of view) So that might explain the lock error message. To prove it, try to install MOSIX and see if the crash/error disappear during your test. -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com |
From: Roger T. <rog...@gm...> - 2013-02-18 00:38:03
|
The errors don't say whether it is something to do with POSIX locks. POSIX locks are cluster wide and so are semaphores. In the case of POSIX locks OpenSSI has a bug that can race across the cluster. That is when two or more processes on different nodes fight over the same file lock there is no guarantee they will be served in the order their requests were submitted. They are served in the order their requests arrive at the CFS server the file belongs to. A possible fix is to serialize the requests by timestamp. On Feb 17, 2013 12:48 AM, "Mulyadi Santosa" <mul...@gm...> wrote: > On Fri, Feb 15, 2013 at 5:49 PM, Oliver Urbann > <oli...@tu...> wrote: > > the process is migrated but the benchmark crashes with different > > messages, e.g.: > > > > Client 3 aborted in state 11: FATAL: semop(id=491526) failed: > > Bezeichner wurde entfernt > > > > or > > > > Client 2 aborted in state 8: FEHLER: lock RowExclusiveLock on object > > 16384/16391/0 is already held > > > > Did anybody have success load balancing postgresql? > > https://lists.sourceforge.net/lists/listinfo/ssic-linux-users > > IIRC, locks in OpenSSI is not system wide. Further more, when a > process (the whole thread group in this case) migrated, then it is > migrated entirely in the sense that it physically move. > > This is different than say MOSIX which use "stub" approach, so lock > might still work because there is still communication between origin > node and the new node (from the migration point of view) > > So that might explain the lock error message. To prove it, try to > install MOSIX and see if the crash/error disappear during your test. > > -- > regards, > > Mulyadi Santosa > Freelance Linux trainer and consultant > > blog: the-hydra.blogspot.com > training: mulyaditraining.blogspot.com > > > ------------------------------------------------------------------------------ > The Go Parallel Website, sponsored by Intel - in partnership with Geeknet, > is your hub for all things parallel software development, from weekly > thought > leadership blogs to news, videos, case studies, tutorials, tech docs, > whitepapers, evaluation guides, and opinion stories. Check out the most > recent posts - join the conversation now. > http://goparallel.sourceforge.net/ > _______________________________________________ > Ssic-linux-users mailing list > Ssi...@li... > https://lists.sourceforge.net/lists/listinfo/ssic-linux-users > |