Re: [naviserver-devel] Tcl nonblocking file IO in NaviServer?

[Wolfgang W. <Wol...@di...>]

We could use this as well for special purpose logs, e.g. payment logs or 
for user deletions and pseudonymizations which conform to GDPR standards.

Am 07.06.19 um 11:25 schrieb Gustaf Neumann:
> On 06.06.19 23:09, Gustaf Neumann wrote:
>> Not sure, what your performance constraints are, but open/close
>> is pretty fast these days (measured under linux on openacs.org):
>>     % time {set F [open /tmp/foo a]; puts $F x; close $F} 1000
>>     55.333 microseconds per iteration
>>
>> if this is too slow for you, one can experiment with ramdisks,
>> and other already mentioned approaches.
>>
> i did some more experiments and implemented a "ns_asyncwrite"
> wrapper for NaviServer's internal function:
>      time {ns_asyncwrite 55 hello\n} 1000
>      0.957 microseconds per iteration
> This is pretty cool. Since the AsyncWriterThread is build around
> a queue, this is also protected against concurrent write() operations.
> this means, one can do ~1mio simple write operations per second
> in the thread protected way, independent from file system blocks
> (measured on the same machine with the same load as above).
>
> The only disadvantage is that one needs commands for opening/closing
> based on the numeric file descriptors (not on Tcl file handles), which
> means at the end small wrappers on top of NaviServer's 
> ns_write()/ns_close().
> Working with the numeric file descriptors has the advantage that it can be
> passed around between interpreters/threads, etc., so it is well suited
> for multi threaded applications, the FDs of open files can be kept
> in nsvs, etc.).
>
> I see also potentials for this in some of our projects (e.g. writing 
> application
> specific log-files e.g. for ingest in elasticsearch, etc.).
>
> If nobody objects, i'll try to bring this to a state for committing 
> this the next
> days.
>
> -g
>
>
>
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
-- 

*Wolfgang Winkler*
Geschäftsführung
wol...@di...
mobil +43.699.19971172

dc:*büro*
digital concepts Novak Winkler OG
Software & Design
Landstraße 68, 5. Stock, 4020 Linz
www.digital-concepts.com <http://www.digital-concepts.com>
tel +43.732.997117.72
tel +43.699.1997117.2

Firmenbuchnummer: 192003h
Firmenbuchgericht: Landesgericht Linz