tcl-core — Tcl Core Team discussions about Tcl development

Re: [TCLCORE] Slave interpreters, events and atomicity of callbacks

[Mark J. <mpc...@gm...>]

Hi,

First of all, these types of questions are probably better posed in
comp.lang.tcl. This mailing list is for discussion regarding
development of the Tcl core.
Secondly, in absense of threads Tcl will only execute one thing at a
time (even if Tcl is thread enabled). This also means any proc used as
a callback will be ran till completion before anything else happens.
The behaviour you are seeing seems caused by the fact that variables
in a proc are not available globally. And global variables are not
available in a proc. The whole slave interp seems a red herring. If
you have a complete script that is confusing you please post it to
comp.lang.tcl.

HTH,
Mark

On Fri, Oct 23, 2009 at 8:59 PM, k2k2e6 <pa...@pi...> wrote:
>
> Guys
>
> I have a question about timer events and their atomicity in slave
> interpreters.
>
> Assume that I do the following:
> interp create slave
> slave eval after 1000 proc_foo   # proc foo does some work and repeats the
> after 1000 proc_foo at the end to repeat
>
> slave eval set  i 0
> slave eval puts $i
> slave eval set  i 0
> slave eval puts $i
> ...
>
> My question is, when the proc_foo is called, does it execute to completion
> before any of my subsequent statements get executed or my subsequent
> statements are interleaved with the statements of proc_foo?
>
> We are seeing a problem with interleaving because one can set the variable
> in the context of proc_foo and then try to get the variable "i" outside the
> context and one gets "variable not found error".
>
> So in slave interpreters, are timer callbacks executed "atomically"?
>
> We are seeing that it is not using ActiveState Tcl 8.4.19.2 threaded
> distribution. The documentation does not clearly state what will happen in
> this case.
> --
> View this message in context: http://www.nabble.com/Slave-interpreters%2C-events-and-atomicity-of-callbacks-tp26031628p26031628.html
> Sent from the tcl-core mailing list archive at Nabble.com.
>
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core
>

[TCLCORE] Slave interpreters, events and atomicity of callbacks

[k2k2e6 <pa...@pi...>]

Guys

I have a question about timer events and their atomicity in slave
interpreters.

Assume that I do the following:
interp create slave
slave eval after 1000 proc_foo   # proc foo does some work and repeats the
after 1000 proc_foo at the end to repeat

slave eval set  i 0
slave eval puts $i
slave eval set  i 0
slave eval puts $i
...

My question is, when the proc_foo is called, does it execute to completion
before any of my subsequent statements get executed or my subsequent
statements are interleaved with the statements of proc_foo?

We are seeing a problem with interleaving because one can set the variable
in the context of proc_foo and then try to get the variable "i" outside the
context and one gets "variable not found error".

So in slave interpreters, are timer callbacks executed "atomically"?

We are seeing that it is not using ActiveState Tcl 8.4.19.2 threaded
distribution. The documentation does not clearly state what will happen in
this case.
-- 
View this message in context: http://www.nabble.com/Slave-interpreters%2C-events-and-atomicity-of-callbacks-tp26031628p26031628.html
Sent from the tcl-core mailing list archive at Nabble.com.

Re: [TCLCORE] CFV TIP #348 - Substituted 'errorStack' / 'traceback'

[Alexandre F. <ale...@gm...>]

On 10/20/09, Alexandre Ferrieux <ale...@gm...> wrote:
> Hello,
>
>  I have now updated the patch for TIP #348 (errorStack) at
>
>  https://sourceforge.net/tracker/index.php?func=detail&aid=2868499&group_id=10894&atid=310894
>
>  to comply with Joe's suggestions [info errorstack ?interp?] + [dict
>  get $d -errorstack].
>
>  Please review the performance measurements there (and code).
>
>  TIA,
>
>  -Alex
>

Patch extended to allow measurement of the faster construction of
errorstack as a flat list.
Timing results at the same place:

https://sourceforge.net/tracker/index.php?func=detail&aid=2868499&group_id=10894&atid=310894

Also, an I tried the Lisp-like linked list approach (each cell being a
Tcl_Obj), only to discover that it didn't gain any speed. So the final
choice is the following:

     (1) [use==2]: [info errorstack] + options dicts + nested list, <5% slowdown

     (2a) [use==6]: [info errorstack] + options dicts + flat list, <2% slowdown

     (2b) [use==6]: [info errorstack] + postprocessed nested list, <2% slowdown

I am hereby asking you to argue on this choice, even if you don't have
time to review the code.

-Alex

Re: [TCLCORE] CFV TIP #348 - Substituted 'errorStack' / 'traceback'

[Alexandre F. <ale...@gm...>]

Hello,

I have now updated the patch for TIP #348 (errorStack) at

https://sourceforge.net/tracker/index.php?func=detail&aid=2868499&group_id=10894&atid=310894

to comply with Joe's suggestions [info errorstack ?interp?] + [dict
get $d -errorstack].

Please review the performance measurements there (and code).

TIA,

-Alex

Re: [TCLCORE] CFV TIP #348 - Substituted 'errorStack' / 'traceback'

[Alexandre F. <ale...@gm...>]

On 10/11/09, Alexandre Ferrieux <ale...@gm...> wrote:
>
> Yes, I feel sympathetic with that last remark, as I follow this
>  strategy all the time (no "debug level", always log, and delete the
>  logs if nothing went wrong)... as long as it doesn't hamper
>  performance. Here the overhead is slight but nonzero, so I'm ready to
>  work a little bit more on optimizing it, if we decide to leave the
>  feature on by default.
>
>  I believe the measured overhead comes from the ckalloc-ed element
>  arrays of the small lists built to contain argslist at each level; the
>  "backbone" list costs much less, because careful steps are taken to
>  modify it in-place whenever possible.
>
>  I can see two options to speed this up:
>
>   (1) grow a flat list instead of a list of lists, inserting integer
>  counts of arguments before each argslist:   {3 foo a b 1 bar 2 baz
>  gnats} and transform it into a true error stack only on  final [info
>  errorstack] request: {{foo a b} bar {baz gnats}}
>
>   (2) grow a Lisp-like nested linked list based on Tcl_Obj's which are
>  Cons cells (one car + one cdr). This has the advantage of relying
>  exclusively on the Tcl_Obj allocator. On final request, it must also
>  be transformed back into a true Tcl list (2a) , or (2b) at least have
>  a string representation identical to that of the wanted list (just
>  like dicts look like lists on strep examination).
>
>  If the constraint of landing in the options dict holds, then (2b) is
>  the way to go.
>  Please give me your preference.

Let me rephrase.
The principle is roughly

    lappend errorstack [info level 0]

The current implementation reuses the optimization of in-place
[lappend] so that it doesn't incur extra allocations in routine use.
But [info level 0] is actually a

    Tcl_NewListObj(varFramePtr->objc,varFramePtr->objv);

which means 2 ckallocs (one for the fixed size List intrep, one for
the elements array).

Question: should I

   (1) leave it as is with the on/off switch, off by default.
       => trivial code, slight performance cost, must be switched on.

   (2) optimize it to a ckalloc-free variant (with two alternatives
sketched earlier)
       => more complex code, zero performance cost, always on.

Please help me decide. I have no problem implementing either choice, I
just don't want to waste time,

-Alex

Re: [TCLCORE] CFV TIP #348 - Substituted 'errorStack' / 'traceback'

[Alexandre F. <ale...@gm...>]

On 10/9/09, Joe English <jen...@fl...> wrote:
>
>  Alexandre Ferrieux wrote:
>
>  > Of course I have no objection regarding the API. It is trivial to
>  > expose it through the [info] or [interp] ensembles rather than
>  > variables. I'm just surprised to see this as a strong showstopper,
>  > because I seem to recall some of you saying that the ::tcl namespace
>  > was okay for the job.
>
>  > Now to proceed is:
>  >      info errorstack --> gives the list
>  >      info errorstack <boolean> --> enables/disables the feature
>
> > Minor variations:
>  >     - "interp" ensemble instead of "info"
>  >     - boolean replaced or extended by "enable/disable"
>
>
>
> Where this really belongs is in the return options dictionary:
>
>     catch $script result options
>     dict get $options -errorstack

The only concern is that the value will then need to be built all the
time (a dict member cannot have traces), while it might be faster to
build an internal form that is not directly the errorstack Tcl value,
see below.

>  Of course to be truly useful the error stack *also*
>  needs to be accessible via [info] or [interp],
>  for the benefit of those who neglected to catch
>  the return options before making an error :-)
>
>  I'd suggest: [info errorstack ?interp?], where ?interp?
>  defaults to the calling interpreter if omitted.

OK, will do that shortly.

>  As for how to turn it on and off -- I think it'd be OK,
>  preferable even, to just leave this enabled all the time.
>  The feature doesn't cost anything under normal operations,
>  the only additional work done is during error propagation,
>  which is not a time-critical part of any reasonable program.
>
>  Also, that way users don't need to remember to turn the feature
>  on before making a mistake that they will want to debug.

Yes, I feel sympathetic with that last remark, as I follow this
strategy all the time (no "debug level", always log, and delete the
logs if nothing went wrong)... as long as it doesn't hamper
performance. Here the overhead is slight but nonzero, so I'm ready to
work a little bit more on optimizing it, if we decide to leave the
feature on by default.

I believe the measured overhead comes from the ckalloc-ed element
arrays of the small lists built to contain argslist at each level; the
"backbone" list costs much less, because careful steps are taken to
modify it in-place whenever possible.

I can see two options to speed this up:

 (1) grow a flat list instead of a list of lists, inserting integer
counts of arguments before each argslist:   {3 foo a b 1 bar 2 baz
gnats} and transform it into a true error stack only on  final [info
errorstack] request: {{foo a b} bar {baz gnats}}

 (2) grow a Lisp-like nested linked list based on Tcl_Obj's which are
Cons cells (one car + one cdr). This has the advantage of relying
exclusively on the Tcl_Obj allocator. On final request, it must also
be transformed back into a true Tcl list (2a) , or (2b) at least have
a string representation identical to that of the wanted list (just
like dicts look like lists on strep examination).

If the constraint of landing in the options dict holds, then (2b) is
the way to go.
Please give me your preference.

>  (BTW: I've since had a chance to look at the actual patch,
>  and agree that the basic mechanism is minimally intrusive;
>  I think it'd be OK for 8.6 (once amended).)

Thank you very much. Indeed I would find it very long to wait for 8.7
for a feature that's both really handy and really simple to implement.

-Alex

Re: [TCLCORE] CFV TIP #348 - Substituted 'errorStack' / 'traceback'

[Joe E. <jen...@fl...>]

Alexandre Ferrieux wrote:

> Of course I have no objection regarding the API. It is trivial to
> expose it through the [info] or [interp] ensembles rather than
> variables. I'm just surprised to see this as a strong showstopper,
> because I seem to recall some of you saying that the ::tcl namespace
> was okay for the job.

> Now to proceed is:
>      info errorstack --> gives the list
>      info errorstack <boolean> --> enables/disables the feature
> Minor variations:
>     - "interp" ensemble instead of "info"
>     - boolean replaced or extended by "enable/disable"


Where this really belongs is in the return options dictionary:

    catch $script result options
    dict get $options -errorstack

Of course to be truly useful the error stack *also*
needs to be accessible via [info] or [interp],
for the benefit of those who neglected to catch
the return options before making an error :-)

I'd suggest: [info errorstack ?interp?], where ?interp?
defaults to the calling interpreter if omitted.

As for how to turn it on and off -- I think it'd be OK,
preferable even, to just leave this enabled all the time.
The feature doesn't cost anything under normal operations,
the only additional work done is during error propagation,
which is not a time-critical part of any reasonable program.

Also, that way users don't need to remember to turn the feature
on before making a mistake that they will want to debug.

(BTW: I've since had a chance to look at the actual patch,
and agree that the basic mechanism is minimally intrusive;
I think it'd be OK for 8.6 (once amended).)


--Joe English

  jen...@fl...

Re: [TCLCORE] CFV TIP #348 - Substituted 'errorStack' / 'traceback'

[Jeff H. <je...@ac...>]

On 09/10/2009 1:05 AM, Reinhard Max wrote:
> On Fri, 9 Oct 2009 at 09:35, Alexandre Ferrieux wrote:
>
>>     - "interp" ensemble instead of "info"
>
> yes, "interp" seems to be a better fit, at least for the
> enable/disable part, because AFAICS the "info" command is purely
> introspective right now. None of the subcommands changes any
> script-visible state.
>
> Having it in "interp" would also result in shorter and simpler code
> when dealing with the errorStack of slave interpreters:
>
>      interp errorstack ?path? ?enable|disable?
>      slave errorstack ?enable|disable?

s/enable|disable/boolean/ and +1.

That's also how I implemented a portion of the optional pcre control.

Re: [TCLCORE] CFV TIP #348 - Substituted 'errorStack' / 'traceback'

[Andreas K. <and...@ac...>]

Donal K. Fellows wrote:
> Andreas Kupries wrote:
>> This is a Call For Votes on TIP #348.
>>
>> TIP #348: Substituted 'errorStack' / 'traceback'
> 
> So... given the chorus of "not opposed to the principle, but not now/in
> this way"... are we going ahead with this vote right now? If nothing
> else, if we are I want to mark the TIP as being voted on. Formal process
> and all that. :-)

I am hereby canceling this vote. We can try again after the discussion.

-- 
Sincerely,
     Andreas Kupries <an...@ac...>
     Developer @    <http://www.activestate.com/>

Re: [TCLCORE] CFV TIP #348 - Substituted 'errorStack' / 'traceback'

[Donal K. F. <don...@ma...>]

Andreas Kupries wrote:
> This is a Call For Votes on TIP #348.
> 
> TIP #348: Substituted 'errorStack' / 'traceback'

So... given the chorus of "not opposed to the principle, but not now/in
this way"... are we going ahead with this vote right now? If nothing
else, if we are I want to mark the TIP as being voted on. Formal process
and all that. :-)

Donal.

Re: [TCLCORE] CFV TIP #348 - Substituted 'errorStack' / 'traceback'

[Reinhard M. <ma...@tc...>]

Hi,

On Fri, 9 Oct 2009 at 09:35, Alexandre Ferrieux wrote:

>    - "interp" ensemble instead of "info"

yes, "interp" seems to be a better fit, at least for the 
enable/disable part, because AFAICS the "info" command is purely 
introspective right now. None of the subcommands changes any 
script-visible state.

Having it in "interp" would also result in shorter and simpler code 
when dealing with the errorStack of slave interpreters:

    interp errorstack ?path? ?enable|disable?
    slave errorstack ?enable|disable?
vs.
    info errorstack ?enable|disable?
    slave eval info errorstck ?enable|disable?
    slave invokehidden info errorstck ?enable|disable?


cu
 	Reinhard

Re: [TCLCORE] CFV TIP #348 - Substituted 'errorStack' / 'traceback'

[Alexandre F. <ale...@gm...>]

On 10/9/09, Kevin Kenny <kev...@gm...> wrote:
>
>  On the other hand, Joe's and Daniel's criticisms suggest that
>  the TIP isn't fully baked.

Yes, I admit the CFV was possibly a bit hurried. It is my fault, I
asked Jeff and Andreas for sponsorship, and they were very fast to
help me finish it and test it. When they proposed the CFV I didn't
react, thinking we had no need for an extra discussion step, since (1)
the ::tcl namespace issue had been raised before and (2) I was under
the impression that 8.6 was imminent.

Now I can see I was wrong, so my proposition is to SEND IT BACK as
Daniel says, and enter the discussion step instead. See my answer to
Daniel to proceed on the API definition.

-Alex

Re: [TCLCORE] CFV TIP #348 - Substituted 'errorStack' / 'traceback'

[Alexandre F. <ale...@gm...>]

On 10/9/09, Daniel A. Steffen <da...@us...> wrote:
> On Thu, Oct 8, 2009 at 14:55, Joe English <jen...@fl...> wrote:
>  >
>  >> This is a Call For Votes on TIP #348.
>  >>
>  >> TIP #348: Substituted 'errorStack' / 'traceback'
>  >>
>  >> Contrary to the information in the headers of the TIP (Tcl 8.7) it is my
>  >> understanding that Alexandre is asking for inclusion of this feature into
>  >>
>  >>       Tcl 8.6
>  >
>  > TIP#348: SEND IT BACK.
>  >
>  > The basic idea behind this TIP is good, but there is
>  > an IMO fatal flaw as currently specified: it introduces
>  > two new magic globals, $::tcl::errorStack and $::tcl::useErrorStack.
>  >
>  > Tcl already has too many of these (errorInfo, errorCode,
>  > tcl_precision, env, ...); this is recognized by now as
>  > having been a bad idea in all cases.
>  >
>  > I'd also suggest targeting this for 8.7.  8.6 is supposed to
>  > be in feature-freeze mode, so that it can be released.
>  > Every last-minute addition delays the release.
>  >
>  > If "SEND IT BACK" is (still) not a valid vote, then I'm
>  > sadly forced to say:
>  >
>  > TIP#348: NO.
>
>
> +1 on all counts, i.e. NO for 8.6, sounds good for 8.7 if amended to
>  not introduce yet more magic variable names, e.g. use [info
>  errorstack] or something instead.

Of course I have no objection regarding the API. It is trivial to
expose it through the [info] or [interp] ensembles rather than
variables. I'm just surprised to see this as a strong showstopper,
because I seem to recall some of you saying that the ::tcl namespace
was okay for the job.

Now to proceed is:

     info errorstack --> gives the list
     info errorstack <boolean> --> enables/disables the feature

palatable ?

Minor variations:

    - "interp" ensemble instead of "info"
    - boolean replaced or extended by "enable/disable"

-Alex

Re: [TCLCORE] CFV TIP #348 - Substituted 'errorStack' / 'traceback'

[Kevin K. <kev...@gm...>]

TIP #348: PRESENT

Jeff is right that the changes are minimally disruptive, and
should be considered for 8.6.0 - after all, it isn't as if
release is imminent.

On the other hand, Joe's and Daniel's criticisms suggest that
the TIP isn't fully baked.

And I'm not going to have time in the next few days to enter
the fray on this one. (I have my hands full trying to repair
357, which is badly flawed as it stands.) I'll trust Joe and
Daniel to thrash it out.

-- 
73 de ke9tv/2, Kevin

Re: [TCLCORE] CFV TIP #348 - Substituted 'errorStack' / 'traceback'

[Daniel A. S. <da...@us...>]

On Thu, Oct 8, 2009 at 14:55, Joe English <jen...@fl...> wrote:
>
>> This is a Call For Votes on TIP #348.
>>
>> TIP #348: Substituted 'errorStack' / 'traceback'
>>
>> Contrary to the information in the headers of the TIP (Tcl 8.7) it is my
>> understanding that Alexandre is asking for inclusion of this feature into
>>
>>       Tcl 8.6
>
> TIP#348: SEND IT BACK.
>
> The basic idea behind this TIP is good, but there is
> an IMO fatal flaw as currently specified: it introduces
> two new magic globals, $::tcl::errorStack and $::tcl::useErrorStack.
>
> Tcl already has too many of these (errorInfo, errorCode,
> tcl_precision, env, ...); this is recognized by now as
> having been a bad idea in all cases.
>
> I'd also suggest targeting this for 8.7.  8.6 is supposed to
> be in feature-freeze mode, so that it can be released.
> Every last-minute addition delays the release.
>
> If "SEND IT BACK" is (still) not a valid vote, then I'm
> sadly forced to say:
>
> TIP#348: NO.

+1 on all counts, i.e. NO for 8.6, sounds good for 8.7 if amended to
not introduce yet more magic variable names, e.g. use [info
errorstack] or something instead.

Cheers,

Daniel

Re: [TCLCORE] CFV TIP #348 - Substituted 'errorStack' / 'traceback'

[Joe E. <jen...@fl...>]

> This is a Call For Votes on TIP #348.
>
> TIP #348: Substituted 'errorStack' / 'traceback'
>
> Contrary to the information in the headers of the TIP (Tcl 8.7) it is my
> understanding that Alexandre is asking for inclusion of this feature into
>
> 	Tcl 8.6

TIP#348: SEND IT BACK.

The basic idea behind this TIP is good, but there is
an IMO fatal flaw as currently specified: it introduces
two new magic globals, $::tcl::errorStack and $::tcl::useErrorStack.

Tcl already has too many of these (errorInfo, errorCode,
tcl_precision, env, ...); this is recognized by now as
having been a bad idea in all cases.

I'd also suggest targeting this for 8.7.  8.6 is supposed to
be in feature-freeze mode, so that it can be released.
Every last-minute addition delays the release.

If "SEND IT BACK" is (still) not a valid vote, then I'm
sadly forced to say:

TIP#348: NO.



--Joe English

  jen...@fl...

Re: [TCLCORE] CFV TIP #348 - Substituted 'errorStack' / 'traceback'

[Jeff H. <je...@ac...>]

On 08/10/2009 12:55 PM, Andreas Kupries wrote:
> This is a Call For Votes on TIP #348.
>
> TIP #348: Substituted 'errorStack' / 'traceback'
>
> Contrary to the information in the headers of the TIP (Tcl 8.7) it is my
> understanding that Alexandre is asking for inclusion of this feature into
>
> 	Tcl 8.6
>
> The implementation is available at
>
> 	https://sourceforge.net/support/tracker.php?aid=2868499

TIP #348: YES

I feel that the functionality provided can be valuable to end users and 
should have no chance of compat issues.  I know 8.6 is at beta, but this 
enhancement is minor and contained.  This is something tkcon could 
definitely make use of.

At Alex's request we have reviewed and tested the current patch across 
our entire platform set and it passed.

Jeff

[TCLCORE] CFV TIP #348 - Substituted 'errorStack' / 'traceback'

[Andreas K. <and...@ac...>]

This is a Call For Votes on TIP #348.

TIP #348: Substituted 'errorStack' / 'traceback'

Contrary to the information in the headers of the TIP (Tcl 8.7) it is my 
understanding that Alexandre is asking for inclusion of this feature into

	Tcl 8.6

The implementation is available at

	https://sourceforge.net/support/tracker.php?aid=2868499

Please send your votes to the tcl-core mailing list by

	Thursday, Oct 15, 12:00 Pacific,
aka	[clock format 1255633200]

-- 
Sincerely,
     Andreas Kupries <an...@ac...>
     Developer @    <http://www.activestate.com/>

Re: [TCLCORE] TIP #358: Suppress Empty List Element Generation from the Split Command

[Andreas K. <and...@ac...>]

Tom Jackson wrote:
> On Wed, Oct 7, 2009 at 9:19 AM, Andreas Kupries
> <and...@ac...> wrote:
>> Oh, you mean
>>
>> proc string-normalize {str} {
>>        return [regsub -all {\s+} $str { }]
>> }
>>
> 
> Close:
> 
> proc string-normalize {str} {
>     return [string trim [regsub -all {\s+} $str { }]]
> }

Ah, leading, trailing spaces. Right.

 > However, the [string] command would probably be much faster than using
 > [regsub], and I can think of a few additional problems and options
 > which complicate including this as a [string] subcommand. Basically I
 > wonder if it would be better in some cases to keep the first
 > whitespace char and remove additional ones (keeping tabbed data or

> line folded text to remain closer to the original). Also, how do you
> handle windows type newlines (crlf) which is two chars?

I assume that the channel used to reading the data was in auto-translation 
mode, which means I always see only LF. If not, it is no big deal to either 
extend the regsub, or run a [string map] before the regsub which normalizes 
that too.


-- 
Sincerely,
     Andreas Kupries <an...@ac...>
     Developer @    <http://www.activestate.com/>

Re: [TCLCORE] TIP #358: Suppress Empty List Element Generation from the Split Command

[Andreas K. <and...@ac...>]

Tom Jackson wrote:
> We have [string trim], maybe something like [string normalize
> (whitespace)]. The result would be a string where adjacent internal
> whitespace chars are collapsed into one space char, and before and
> after whitespace is eliminated. Then the problem would be solved like
> this:
> 
> set mylist [split [string normalize $mystring]]

Oh, you mean

proc string-normalize {str} {
	return [regsub -all {\s+} $str { }]
}

-- 
Sincerely,
     Andreas Kupries <an...@ac...>
     Developer @    <http://www.activestate.com/>

Re: [TCLCORE] TIP #358: Suppress Empty List Element Generation from the Split Command

[Magentus <mag...@gm...>]

On Tue, 06 Oct 2009 23:43:37 +0200, 
Twylite <tw...@cr...> wrote:

> George Petasis wrote:
>>  This TIP proposes a new optional switch (*-noemptyelements*) to
>> the *split* command: 
>>        *split -noemptyelements* /string/ ?/splitChars/? 
>>  If this option is present, then *split* will not produce an empty
>> list element when the /string/ contains adjacent characters that
>> are present in /splitChars/. 
> I don't think this functionality belongs in [split]; it would be
> better handled by a higher order function like filter:
>   filter "ne {}" [split $string $splitchars]

[filter] is a bad idea, IMO...  It's going to need many of [search]s
options.  It'd be MUCH better to add a -command to [lsearch] similar to
[sort], to direct its matching.  (It can already do this job, anyhow...)

In any case, the problem with this is that you're splitting the entire
string into a list, and then throwing chunks of it away again.  In a
substantial sized string with a lot of padding characters, you can have
a LOT of little empty elements to iterate over.  The [string normalize]
idea is better, I think...  ([string collapse] kind of says something
else to me, though that could be my Glib history...)


I wonder, if it wouldn't be an idea to add a [string split] command,
with new functionality and more supportive syntax and migrate over to
that, as has been done with [chan] - this isn't the first time this
idea has been raised.  I know I for one would like to see a few simple
options added to [split]:

-defaults; adds the default spaces set to the specified split characters
set.  I've wanted to do that many times, and this brings to mind an
earlier discussion of which UTF-8 characters should be part of the
default spaces set.  [encoding sets] or something would be handy.

-noempty; the currently being discussed option.

-keepall; split BETWEEN (spans of, with -noempty) deliminator and
non-deliminator characters, instead of over them.  This can currently
be achieved with the somewhat heavier [regexp], but it would be trivial
for [split] to do this.

Specifically...  -noempty would cause it to skip all split characters,
instead of stepping over them one at a time.  In either case, you stash
away the character position first, and -keepall then simply reches back
and extracts the intervening characters as a list element.


The excellent thing with split, is that it's light-weight, fast, and
easy.  And none of this changes that.

-- 
Fredderic

Debian/unstable (LC#384816) on i686 2.6.30-1-686 2009 (up 1 day, 18:41)

Re: [TCLCORE] TIP #358: Suppress Empty List Element Generation from the Split Command

[Donald A. <as...@tr...>]

Tom Jackson <tom...@gm...> writes:

> We have [string trim], maybe something like [string normalize
> (whitespace)]. The result would be a string where adjacent internal
> whitespace chars are collapsed into one space char,

I like the idea of [string collapse $xxx]

Not [split -nomumble $zzz] though.

-- 
Donald Arseneau                          as...@tr...

Re: [TCLCORE] TIP #358: Suppress Empty List Element Generation from the Split Command

[Jordan H. <jo...@jo...>]

All,

I am inclined to agree with the filter recommendation.  If anything, split should behave in a 
symmetrical manner for composing an equivalent string.  The filter example clearly makes this an 
issue with the list and not the split function.  Further more, have a general purpose filter 
function is likely to be more useful than a 'noemptyelements' option added to split.

My two cents,
Jordan Henderson

On Tuesday 06 October 2009, Twylite wrote:
> George Petasis wrote:
> >  This TIP proposes a new optional switch (*-noemptyelements*) to the
> >  *split* command:
> >
> >        *split -noemptyelements* /string/ ?/splitChars/?
> >
> >  If this option is present, then *split* will not produce an empty list
> >  element when the /string/ contains adjacent characters that are present
> >  in /splitChars/.
>
> I don't think this functionality belongs in [split]; it would be better
> handled by a higher order function like filter:
>   filter "ne {}" [split $string $splitchars]
>
> Regards,
> Twylite
>
>
>
> ---------------------------------------------------------------------------
>--- Come build with us! The BlackBerry(R) Developer Conference in SF, CA is
> the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core

Re: [TCLCORE] TIP #358: Suppress Empty List Element Generation from the Split Command

[Tom J. <tom...@gm...>]

On Tue, Oct 6, 2009 at 10:06 AM, George Petasis
<pe...@ii...> wrote:
>
>  TIP #358: SUPPRESS EMPTY LIST ELEMENT GENERATION FROM THE SPLIT COMMAND
> =========================================================================
>  Version:      $Revision: 1.2 $
>  Author:       George Petasis <petasis_at_iit.demokritos.gr>
>  State:        Draft
>  Type:         Project
>  Tcl-Version:  8.7
>  Vote:         Pending
>  Created:      Sunday, 04 October 2009
>  URL:          http://purl.org/tcl/tip/358.html
>  WebEdit:      http://purl.org/tcl/tip/edit/358
>  Post-History:
>
> -------------------------------------------------------------------------
>
>  ABSTRACT
> ==========
>
>  The *split* command will create empty list elements when adjacent split
>  characters are found in the input. In some cases these empty list
>  elements are not desired, so this TIP proposes a new switch to disable
>  their generation.
>
>  RATIONALE
> ===========
>
>  The idea for this TIP came from a discussion in comp.lang.tcl:
>  [<URL:http://groups.google.gr/group/comp.lang.tcl/browse_thread/thread/8d46b0f10e7a5750/d7844cc739aa4310>]
>  and the (non obvious) suggestions on how tokens can be extracted from a
>  string can be performed efficiently.
>
>  It should be noted that this will allow the *split* command to be used
>  in a fashion that is very similar to how splitting works in many other
>  languages (e.g., Perl, awk, Unix shells).
>
>  SPECIFICATION
> ===============
>
>  This TIP proposes a new optional switch (*-noemptyelements*) to the
>  *split* command:
>
>       *split -noemptyelements* /string/ ?/splitChars/?
>
>  If this option is present, then *split* will not produce an empty list
>  element when the /string/ contains adjacent characters that are present
>  in /splitChars/.

I think that [split] is best reserved for well formed inputs, in fact,
if the split chars are whitespace, then [split] does what most Tcl
programmers would consider to be the wrong thing...creating empty
elements between extra whitespace chars.

The solution could be something more generally useful: maybe
whitespace normalization?

We have [string trim], maybe something like [string normalize
(whitespace)]. The result would be a string where adjacent internal
whitespace chars are collapsed into one space char, and before and
after whitespace is eliminated. Then the problem would be solved like
this:

set mylist [split [string normalize $mystring]]

Re: [TCLCORE] TIP #358: Suppress Empty List Element Generation from the Split Command

[Twylite <tw...@cr...>]

George Petasis wrote:
>  This TIP proposes a new optional switch (*-noemptyelements*) to the 
>  *split* command: 
>
>        *split -noemptyelements* /string/ ?/splitChars/? 
>
>  If this option is present, then *split* will not produce an empty list 
>  element when the /string/ contains adjacent characters that are present 
>  in /splitChars/. 
>   
I don't think this functionality belongs in [split]; it would be better 
handled by a higher order function like filter:
  filter "ne {}" [split $string $splitchars]

Regards,
Twylite