state-threads-users — State Threads users' list

[ST-users] condition variables not associated with locks?

[Jeff <jl...@ho...>]

How does State Threads make condition variables work without a lock?

In every other implementation of condition variables I have seen, 
waiting is always done in conjunction with a lock.   In NSPR, on which 
ST is based, the variable itself is permenantly associated with a lock 
at creation time.  In other systems a reference to a lock is passed in 
to the wait function as a parameter.

[ST-users] condition variables not associated with locks?

[Jeff B. <jl...@ho...>]

How does State Threads make condition variables work without a lock?

In every other implementation of condition variables I have seen, 
waiting is always done in conjunction with a lock.   In NSPR, on which 
ST is based, the variable itself is permenantly associated with a lock 
at creation time.  In other systems a reference to a lock is passed in 
to the wait function as a parameter.

[ST-users] Re: shared data

[Gene <ge...@pa...>]

Neil Conway wrote:
> 
> Hi all,
> 
> I have an application I'm designing that may or may not be suitable for
> use with ST.
> 
> The application should be able to handle a fair number (128-1024) of
> concurrent TCP connections. Of these, half will be inbound and half will
> be outbound (there is always a 1:1 relationship between inbound and
> outbound connections).
> 
> It won't need to do a lot of processing on the data that comes in; most
> of the time it will just be copying data from the inbound socket to the
> outbound socket, and vice versa.
>

Sounds like some sort of proxy.  ST works very well for proxies.

> It will need very rapid access (i.e. as scaleable & fast as possible) to
> a single shared data structure (this is a cache: the whole point of this
> is to get a lot of cache hits, so I'd like all the threads to be able to
> access a single cache). There is a single unique value for each unique
> key, so the cache could be implemented as a hash table.
> 
> This application will also need to be able to scale with the number of
> CPUs in the machine: simply using a single CPU is not optimal.
> 
> Is this a good candidate for using ST? AFAIK, it is considered good
> practice with ST to create several processes to allow for system
> scalability. This would work well, but wouldn't IPC be a significant
> performance hit? I'll be doing several operations on the hash table for
> each connection; furthermore, an operation might need to insert or
> retrieve a relatively large amount of data from the table (10 to 100
> KB).
>

Ideally, you should create one process per CPU and each process
should have its own *separate* cache.  In other words, consider
each process as a separate mini-instance of your application.
Someone called it a "cluster in a box" approach.  It may seem wasteful,
but that's how scalability is achieved in clusters.  For example,
if you double number of CPUs, you also double the amount of memory
and other relevant resources.  This approach will work well if all
connections are completely independent of each other (like in web
proxy servers).

Hope it helps,
Gene

[ST-users] shared data

[Neil C. <nc...@kl...>]

Hi all,

I have an application I'm designing that may or may not be suitable for
use with ST.

The application should be able to handle a fair number (128-1024) of
concurrent TCP connections. Of these, half will be inbound and half will
be outbound (there is always a 1:1 relationship between inbound and
outbound connections).

It won't need to do a lot of processing on the data that comes in; most
of the time it will just be copying data from the inbound socket to the
outbound socket, and vice versa.

It will need very rapid access (i.e. as scaleable & fast as possible) to
a single shared data structure (this is a cache: the whole point of this
is to get a lot of cache hits, so I'd like all the threads to be able to
access a single cache). There is a single unique value for each unique
key, so the cache could be implemented as a hash table.

This application will also need to be able to scale with the number of
CPUs in the machine: simply using a single CPU is not optimal.

Is this a good candidate for using ST? AFAIK, it is considered good
practice with ST to create several processes to allow for system
scalability. This would work well, but wouldn't IPC be a significant
performance hit? I'll be doing several operations on the hash table for
each connection; furthermore, an operation might need to insert or
retrieve a relatively large amount of data from the table (10 to 100
KB).

I would be grateful for any advice on how to design my application to
work with ST -- or, if ST isn't an appropriate choice, the best I/O
design to use.

Thanks in advance,

Neil

-- 
Neil Conway <nei...@ro...>
PGP Key ID: DB3C29FC

Re: [ST-users] Re: How to deal with long running computations?

[Lev W. <vl...@ne...>]

> Claus Assmann wrote:
> 
> >
> > I'm concerned about the possible generation of a temporary RSA key
> > (RSA_generate_key()) or similar operations (DSA_generate_parameters())
> > that can take really long. I'll probably have to put those in
> > a helper process.
> 
> Yes, if these functions take longer than OS time-slice, it's probably
> a good idea to put them in a helper process.


Correction: take longer than 5-10 time slices. Another idea (anybody
can prove this?) is to put them into an OS-provided _THREAD_.



-- 
Lev Walkin
vl...@ne...

[ST-users] Re: How to deal with long running computations?

[Gene S. <gs...@ab...>]

Claus Assmann wrote:

>
> I'm concerned about the possible generation of a temporary RSA key
> (RSA_generate_key()) or similar operations (DSA_generate_parameters())
> that can take really long. I'll probably have to put those in
> a helper process.

Yes, if these functions take longer than OS time-slice, it's probably
a good idea to put them in a helper process.

[ST-users] Re: How to deal with long running computations?

[Claus A. <sta...@es...>]

On Fri, Nov 02, 2001, Gene Shekhtman wrote:

First: thanks for your answer!

> Claus Assmann wrote:
> 
> > State threads seem like a very interesting concept for internet
> > servers (as intented :-). However, what do you do if the server may
> > perform long running (compute intensive) operations sometimes?  For
> > example, if TLS is used, some computations may take a while and
> > hence all other connections are blocked.
> 
> It's hard to give a general answer without knowing application's
> specifics.  In my experience with ST and OpenSSL, thread starvation

It's an MTA, i.e., the SMTP server.

> was not a problem because
> 
> - SSL handshake requires information exchange.  Every time you read
>   from a socket you'll probably have a context switch.
> 
> - If you send large amounts of encrypted data, the computation time
>   is still limited by the output socket buffer size.  That is, no
>   matter how much data you need to encrypt, you'll have a context
>   switch as soon as socket buffer is full.  You can probably adjust
>   this "time-slice" by changing the socket's send buffer size with
>   setsockopt(SO_SNDBUF).

I'm concerned about the possible generation of a temporary RSA key
(RSA_generate_key()) or similar operations (DSA_generate_parameters())
that can take really long. I'll probably have to put those in
a helper process.

> In general, there may be a problem if you do lots of compute
> intensive operations and very little network I/O.  In that case you
> should do your computations in "chunks" and yield control in between
> (e.g., by calling st_usleep(0)).

That can work if
- I would have control over the libraries that the MTA uses,
  e.g., OpenSSL, Cyrus-SASL;
- and I would know which parts of the computations may take
  a long time on which types of computers.
The latter is of less concern, the former may cause problems.  This
requires the application programmer to do a part of the job of the
OS (scheduler). On the one hand, it gives him more control, on the
other hand, it's simpler to do it wrong.

I have to make a decision whether I go with state threads for the
SMTP server or with a worker thread model.  I would like to hear
"real world" experience about such an application (or something
similar). My current tests favor state threads since they behave
more reasonable across different OS and single/multi processor
systems. However, I don't know yet how realistic my test programs
are.

[ST-users] Re: How to deal with long running computations?

[Gene S. <gs...@ab...>]

Claus Assmann wrote:

> State threads seem like a very interesting concept for internet
> servers (as intented :-). However, what do you do if the server may
> perform long running (compute intensive) operations sometimes?  For
> example, if TLS is used, some computations may take a while and
> hence all other connections are blocked.

It's hard to give a general answer without knowing application's
specifics.  In my experience with ST and OpenSSL, thread starvation
was not a problem because

- SSL handshake requires information exchange.  Every time you read
  from a socket you'll probably have a context switch.

- If you send large amounts of encrypted data, the computation time
  is still limited by the output socket buffer size.  That is, no
  matter how much data you need to encrypt, you'll have a context
  switch as soon as socket buffer is full.  You can probably adjust
  this "time-slice" by changing the socket's send buffer size with
  setsockopt(SO_SNDBUF).


In general, there may be a problem if you do lots of compute
intensive operations and very little network I/O.  In that case you
should do your computations in "chunks" and yield control in between
(e.g., by calling st_usleep(0)).

Hope it helps,
Gene

[ST-users] How to deal with long running computations?

[Claus A. <sta...@es...>]

State threads seem like a very interesting concept for internet
servers (as intented :-). However, what do you do if the server may
perform long running (compute intensive) operations sometimes?  For
example, if TLS is used, some computations may take a while and
hence all other connections are blocked. This increases the latency
for them and may reduce overall throughput (the clients of those
connections could do some work while the server goes back to its
computation). Is there some way around this other than placing those
computations in helper threads? I'm worried about the communication
overhead, e.g., all data has to be read by the main server, then
all of it must be send off to the helper thread, and then it goes
back to the main server. Alternatively, I could create several
processes and "hope" that the compute intensive parts are more or
less uniformly spread out to avoid delaying too many connections.
Has anyone done something like this and can share her/his experience?
Or is there a better way to deal with this problem? Maybe "playing"
around with mutexes? I'm not sure whether this can be done if
libraries (e.g., OpenSSL) are used.