thanks a lot for sharing your experience. That's what we were looking for.
On Tue, Jun 25, 2013 at 6:37 PM, Alastair Bridgewater <
> I don't have experience with your particular configuration... because I
> think that MySQL is absolute trash, and doesn't belong anywhere near my
> nice, clean Lisp code. Or even my hurriedly-written, trashy Lisp code, but
> I try not to let trashy code into our source control. I'm a big fan of
> PostgreSQL, and to a lesser extent (oh, the stories I could tell)
> That said, SBCL uses a stop-the-world GC. If you don't put a hard limit
> on the number of threads that you're going to use, that will kill you. It
> may kill you anyway. We're currently running some fixed number of web
> listener threads per SBCL instance (currently 40), and eight instances per
> two-or-four-CPU-core VM (so, 320 threads per VM), with a load-balancer
> (pound or nginx for us, though apache is a plausible option) distributing
> connections, and it's plausibly overkill for what we're doing at work.
> And, provided that we can supply enough database capacity behind that (a
> rather large caveat in our case), we can scale out by adding more VMs.
> Another thing that we find is that DB connection setup and tear-down is
> expensive, but that total number of connections per DB server is a painful
> limit, especially for a situation with a single "master" server that is the
> only one that can be written to. If I were designing from scratch now, I'd
> have a dedicated read-only DB connection per web listener thread, open
> master-DB connections on an as-needed basis (the theory being that having a
> request need to write to the DB is rare, and thus may be slightly slower
> without impacting performance), and marshal all logging to a single thread
> with its own master-DB (or other log-recording) connection.
> Honestly, once you can handle a hundred and fifty to two hundred
> simultaneous connections requesting one of the most horribly slow things
> that you can process, without the system keeling over, you're off to a good
> start. And a lot of that is just figuring out where the weak points in
> your scalability are. Too many threads (and too much live data / garbage)
> per instance causing GC time to go completely insane is one choke point.
> Database connection limits (and RAM, and so on) is another. Keep running
> up the simultaneous connections until the system keels over, figure out why
> it keels over, what would be involved in fixing it so that it doesn't, and
> at what point it becomes worth fixing. Repeat as needed. While you're at
> it, driving down response latency (total time spent handling a connection)
> is good as well, since you can thus turn around more connection requests
> per unit time at maximum load.
> Having a non-stop-the-world GC would help a bit in terms of thread limits,
> but that's slow going in terms of implementation (totally worth it, for
> coolness if nothing else, but slow going). GC improvements are being
> worked on, though. Being able to scale out by starting up another SBCL
> instance or a whole second VM worth of instances is a lot easier in terms
> of implementation, and gets you further as well.
> This has been a bit of a brain dump of various things that I've run into,
> and things that are still on my mind. I hope that you find it helpful, and
> if you have any follow-on questions... Well, I'm not sure that sbcl-devel
> is the right forum for it, actually. We'll see if anyone makes a
> recommendataion for a more appropriate place to discuss this sort of thing.
> -- Alastair Bridgewater
> On Jun 25, 2013 7:39 AM, "Andreas Franke" <
> andreas.franke@...> wrote:
>> has anybody experienced serious problems with running
>> a sizable web-application with mysql back-end on SBCL?
>> Any recommendations for a benchmark setup?
>> The intended benchmark specs are:
>> - SBCL 64bit Linux
>> - MySQL DB on dedicated server with ~ 40GB RAM
>> - up to 50 -- 100 threads in parallel
>> - 4 or 8 CPU cores
>> Kind regards,
>> Andreas Franke
>> This SF.net email is sponsored by Windows:
>> Build for Windows Store.
>> Sbcl-devel mailing list