tcl-core — Tcl Core Team discussions about Tcl development

[TCLCORE] CFV: Zlib and PNG

[Kenny, K. B (G. Research) <ke...@cr...>]

   TIP#234: Add Support For Zlib Compression
YES
   TIP#244: PNG Photo Image Support for Tk
YES

And please, please, let's get the bundle-vs-depend issue sorted.
Simply building our own copy of zlib on platforms where it is
universally available is not acceptable. (But we probably *do*
need to ship it with Tcl on Windows, where zlib is *not*
universally available.
 
-- 
73 de ke9tv/2, Kevin

Re: [TCLCORE] [Fwd: Re: CFV: Zlib and PNG]

[Donal K. F. <don...@ma...>]

Donald G Porter wrote:
> Still have had no chance to review the actual substance.  May not.
> 

Note that the links to the implementation weren't working the last time
I checked. I don't know what's wrong; looks like a server misconfig. (I
suppose that's a good reason for encouraging people to not submit TIPs
that say the implementation is on their own system...)

Donal.

Re: [TCLCORE] [Fwd: Re: CFV: Zlib and PNG]

[Donald G P. <dg...@ni...>]

Donal K. Fellows wrote:
> Then you'd be in favour of (for example) Tcl_ZlibDeflate as a function
> name (I prefer concrete examples) and no use of [package require zlib]?
> That's certainly OK with me.

It was a surprise to me to see that as an option, since my impression
had been that distribution in bundled package form was the aim.

But offered it as an option, yes, I'll gladly seize it since it's
the easiest approach available.

Still have had no chance to review the actual substance.  May not.

-- 
| Don Porter          Mathematical and Computational Sciences Division |
| don...@ni...             Information Technology Laboratory |
| http://math.nist.gov/~DPorter/                                  NIST |
|______________________________________________________________________|

Re: [TCLCORE] [Fwd: Re: CFV: Zlib and PNG]

[Donal K. F. <don...@ma...>]

Donald G Porter wrote:
> Routines matching Tcl_* that are part of the Tcl C interface are
> available with only [package present Tcl].  No need to [package require]
> anything else to get the complete Tcl interface.
>
> If folks won't agree to that, then my fallback position is that we
> add no "phony" pacakges without an actual plan since we've botched
> it so many times already.  So show me the plan, and explain its value.
> These details matter and are not something to toss in as an afterthought.

Then you'd be in favour of (for example) Tcl_ZlibDeflate as a function
name (I prefer concrete examples) and no use of [package require zlib]?
That's certainly OK with me.

Donal.

Re: [TCLCORE] TIP #342: Dict Get With Default

[Jan N. <nij...@us...>]

2008/12/2 Jan Nijtmans <nij...@us...>:
>>> or
>>>    dict get ?-default /value/? /dictionary/ /key/ ?/key/ ...?
>>
>> That's worse; forces the production of the string representation of the
>> dictionary just so we can check to see if it is equal to "-default"!
>
> Agreed.

On the other hand, we can check if the argument is already a list or
a dict. A dict can never be equal to "-default" and neither can
a list with >1 arguments. So we can prevent the need for generating
the string representation if the argument is already a list or a dict.

Regards,
          jan Nijtmans

Re: [TCLCORE] TIP #342: Dict Get With Default

[Jan N. <nij...@us...>]

2008/12/2 Donal K. Fellows <don...@ma...>:
>> How about:
>>    dict get /dictionary/ ?-default /value/? /key/ ?/key/ ...?
>
> What about if we want to use "-default" as the key?

Insert "--" as the last switch.

>> or
>>    dict get ?-default /value/? /dictionary/ /key/ ?/key/ ...?
>
> That's worse; forces the production of the string representation of the
> dictionary just so we can check to see if it is equal to "-default"!

Agreed.

Regards,
      Jan Nijtmans

[TCLCORE] [Fwd: Re: CFV: Zlib and PNG]

[Donald G P. <dg...@ni...>]

Donald G Porter wrote:
> Donal K. Fellows wrote:
>> Jeff Hobbs wrote:
>>> It should be Tcl_Zlib and it should _not_ be a separate package or in 
>>> any way optional.
>>
>> Sounds like it ought to be a "Tcl_Zlib" prefix, which is fine with me.
>> Whether we support a no-op [package require zlib] is the main Q left 
>> open.
> 
> No.
> 
> Routines matching Tcl_* that are part of the Tcl C interface are
> available with only [package present Tcl].  No need to [package require]
> anything else to get the complete Tcl interface.
> 
> If folks won't agree to that, then my fallback position is that we
> add no "phony" pacakges without an actual plan since we've botched
> it so many times already.  So show me the plan, and explain its value.
> These details matter and are not something to toss in as an afterthought.
> 

DKF:
: Did you mean to send this to tcl-core?

Yes, I did.  :P


-- 
| Don Porter          Mathematical and Computational Sciences Division |
| don...@ni...             Information Technology Laboratory |
| http://math.nist.gov/~DPorter/                                  NIST |
|______________________________________________________________________|

Re: [TCLCORE] TIP #342: Dict Get With Default

[Donal K. F. <don...@ma...>]

Jan Nijtmans wrote:
> How about:
>     dict get /dictionary/ ?-default /value/? /key/ ?/key/ ...?

What about if we want to use "-default" as the key?

> or
>     dict get ?-default /value/? /dictionary/ /key/ ?/key/ ...?

That's worse; forces the production of the string representation of the
dictionary just so we can check to see if it is equal to "-default"!

Donal.

Re: [TCLCORE] CFV: Zlib and PNG

[miguel s. <mig...@gm...>]

Donal K. Fellows wrote:
> As has been thoroughly threatened, here's the Call For Votes on getting
> zlib support for Tcl (yay compression!) and PNG support for Tk (yay
> modern icons!) Votes should be sent to tcl-core by midday GMT next
> Monday (i.e. [clock format 1228737600]).
>
>    TIP#234: Add Support For Zlib Compression
>    TIP#244: PNG Photo Image Support for Tk

My votes:
     TIP#234: YES
     TIP#244: YES

Miguel

Re: [TCLCORE] TIP #342: Dict Get With Default

[Jan N. <jan...@gm...>]

2008/12/2 Lars Hellström <Lar...@re...>:
>  TIP #342: DICT GET WITH DEFAULT
>  (I consider the name of this subcommand very much open for discussion)

How about:
    dict get /dictionary/ ?-default /value/? /key/ ?/key/ ...?
or
    dict get ?-default /value/? /dictionary/ /key/ ?/key/ ...?

Regards,
         Jan Nijtmans

Re: [TCLCORE] TIP #341: Multiple 'dict filter' Patterns

[Donal K. F. <don...@ma...>]

Lars HellstrXm wrote:
>  TIP #341: MULTIPLE 'DICT FILTER' PATTERNS 

Damn! I hate it when encodings get in the way...

Donal.

Re: [TCLCORE] TIP #329 [try] updated

[Twylite <tw...@cr...>]

TIP #329 updated.

Changes:

(1) Variables are now assigned per handler

Was:  try { ... } as {em opts} on error { ... } finally { ... }

Now: try { ... } on error {em opts} { ... } finally { ... }

(2) Trap matching uses list prefix instead of glob

Was:  try { ... } trap {POSIX *} {em opts} { ... }

Now: try { ... } trap {POSIX} {em opts} { ... }

(3) Clarified exception chaining

If an exception is thrown from a handler or the finally body, the 
options dict of the exception it replaces is added into the new 
exception's options dict under the key "-during".

_If you have a suggestion for a better name than -during speak really 
really fast_


I will take a final read over the TIP when I get home (+/- 1.5 hours) 
and it should then, finally, hopefully, trapping new and unexpected 
objections, be ready for a vote.

Regards,
Twylite

[TCLCORE] TIP #341: Multiple 'dict filter' Patterns

[Lars H. <Lar...@re...>]

 TIP #341: MULTIPLE 'DICT FILTER' PATTERNS 
===========================================
 Version:      $Revision: 1.2 $
 Author:       Lars Hellström <Lars.Hellstrom_at_residenset.net>
 State:        Draft
 Type:         Project
 Tcl-Version:  8.6
 Vote:         Pending
 Created:      Thursday, 27 November 2008
 URL:          http://purl.org/tcl/tip/341.html
 WebEdit:      http://purl.org/tcl/tip/edit/341
 Post-History: 

-------------------------------------------------------------------------

 ABSTRACT 
==========

 The *key* and *value* forms of *dict filter* are generalised to allow 
 an arbitrary number of patterns. 

 SPECIFICATION 
===============

 The two *dict filter* command forms 

       *dict filter* /dictionary/ *key* /pattern/ 

       *dict filter* /dictionary/ *value* /pattern/ 

 are generalised to 

       *dict filter* /dictionary/ *key* ?/pattern/ ...? 

       *dict filter* /dictionary/ *value* ?/pattern/ ...? 

 and the results are the sub-dictionaries of those keys and values 
 respectively which match at least one of the patterns. 

 RATIONALE 
===========

 Although there are *dict* subcommands which allow deleting some keys 
 from a dictionary (*dict remove*) and inserting some keys into a 
 dictionary (*dict replace*), there is no direct way of requesting the 
 sub-dictionary which only has keys from a given list; if we think of 
 only the set of keys in the dictionary, then we have subcommands for 
 set minus and set union, but none for set intersection. A situation 
 where this would be useful is that the option dictionary for a 
 high-level procedure can contain options meant to be passed on to lower 
 level commands, and it is necessary to extract the subdictionary of 
 options that the lower level command would accept (since passing one 
 which is not supported would cause it to throw an error). 

 There is of course already the *dict filter* command, which indeed 
 returns a subdictionary of an existing dictionary, but its *key* form 
 only accepts one *string match* pattern and therefore cannot be used to 
 e.g. select all three of -foo, -bar, and -baz (it could select both 
 -bar and -baz through the pattern -ba[rz], but that's neither common 
 nor particularly readable). However, in many instances where this kind 
 of pattern is used (notably *glob*, *namespace export*, and *switch*), 
 it is possible to give several such patterns and have it interpreted as 
 the union of the patterns. Were that the case with *dict filter*, the 
 "-foo, -bar, and -baz" problem could be solved as easily as 

     dict filter $opts key -foo -bar -baz

 which is comparable to 

     dict remove $opts -foo -bar -baz
     dict replace $opts -foo 1 -bar off -baz 42

 and much nicer than the *script* counterpart 

     dict filter $opts script {key val} {
        ::tcl::mathop::in $key {-foo -bar -baz}
     }

 If the *key* form is generalised like this, then it seems appropriate 
 to also generalise the *value* form in the same way to keep the 
 symmetry, even though I have no immediate use-case for that feature. 

 Since it is generally good to Do Nothing Gracefully, the command syntax 
 is also generalised to allow the case of no patterns at all. 

 REJECTED ALTERNATIVES 
=======================

 A more direct way of meeting the motivating need would be a command 
 *dict select* with the same syntax as *dict remove* (no pattern 
 matching) but logic reversed. This would however be so close to *dict 
 filter* ... *key* that extending the syntax of the latter seemed more 
 appropriate. 

 An alternative to allowing multiple patterns with *dict filter* could 
 be to allow a regular expression pattern, since the union of two 
 regular languages is again a regular language. Any syntax that could be 
 picked for that would however on one hand already be rather close to 

     dict filter $opts script {key val} {regexp $RE $key}

 and on the other it would be rather difficult to read, as the regular 
 expression corresponding to "-foo or -bar or -baz" is 

     ^(-foo|-bar|-baz)$

 which it is tempting but incorrect to simplify to "-foo|-bar|-baz". 

 IMPLEMENTATION NOTES 
======================

 An implementation exists (it's a very trivial to modify *dict filter* 
 ... *value* to work this way: just add an inner loop over the list of 
 patterns); see SF path #2370575. 
 [<URL:https://sourceforge.net/support/tracker.php?aid=2370575>] 

 What might be tricky is the case of *dict filter* ... *key*, since this 
 currently has an optimisation for the case of a pattern without glob 
 metacharacters that would be very desirable to keep for the motivating 
 use-case of selecting specific keys from a dictionary. The natural way 
 to do that would be to make the loop over patterns the outer loop and 
 the loop over dictionary entries the inner loop, which is only entered 
 if the current pattern contains metacharacters. Such an optimisation 
 would however have the script-level-visible consequence of having the 
 keys show up in the order of the patterns rather than the order of the 
 original dictionary, so it may be a good idea to also explicitly 
 specify that *dict filter* does not guarantee keys in the result to be 
 in the same order as in the input dictionary. 

 Indeed, a *dict filter* ... *key* that reorders keys according to its 
 pattern arguments could sometimes be useful in interactive situations, 
 as a way of getting selected keys up from in a dictionary: 

     set D {-baz 0 -bar 1 -foo 2}
     dict filter $D key -foo -bar *

 On the other hand, this effect can mostly be obtained through use of 
 *dict merge* already: 

     dict merge {-foo x -bar x} $D

 COPYRIGHT 
===========

 This document has been placed in the public domain.  

-------------------------------------------------------------------------

 TIP AutoGenerator - written by Donal K. Fellows 

[TCLCORE] TIP #342: Dict Get With Default

[Lars H. <Lar...@re...>]

 TIP #342: DICT GET WITH DEFAULT 
=================================
 Version:      $Revision: 1.2 $
 Author:       Lars Hellström <Lars.Hellstrom_at_residenset.net>
 State:        Draft
 Type:         Project
 Tcl-Version:  8.6
 Vote:         Pending
 Created:      Thursday, 27 November 2008
 URL:          http://purl.org/tcl/tip/342.html
 WebEdit:      http://purl.org/tcl/tip/edit/342
 Post-History: 

-------------------------------------------------------------------------

 ABSTRACT 
==========

 A new subcommand of *dict* is proposed, which returns a dictionary 
 value if it exists and returns a per-call default otherwise. 

 SPECIFICATION 
===============

 The *dict* command will get a new subcommand 

       *dict getwithdefault* /dictionary/ /key/ ?/key/ ...? /value/ 

 (I consider the name of this subcommand very much open for discussion) 
 which modulo error messages behaves like 

    proc dict_getwithdefault {D args} {
        if {[dict exists $D {*}[lrange $args 0 end-1]]} then {
            dict get $D {*}[lrange $args 0 end-1]
        } else {
            lindex $args end
        }
    }

 i.e., it returns the value from the /dictionary/ corresponding to the 
 sequence of /key/s if it exists, or the default /value/ otherwise. As 
 with *dict exists*, it is OK (and will cause the default /value/ to be 
 returned) if one of the /key/s is missing from its dictionary, but an 
 error is thrown if this path of keys cannot be traversed because the 
 value associated with the previous key is not a dictionary. 

 RATIONALE 
===========

 It is clear that getting a value from a dictionary if it exists and 
 using a default otherwise is a common operation, but it is also clear 
 that this can be carried out with a combination of existing Tcl 
 commands. Hence the issue is whether a new subcommand for this improves 
 efficiency and convenience of this operation enough to justify the 
 possible bloat it brings. 

 ALTERNATIVE METHODS 
---------------------

 One approach that has been suggested for providing default values is to 
 combine *dict get* with *dict merge*, like so: 

     dict get [dict merge {-apa bar} $D] -apa

 This approach is however appropriate mainly in situations where several 
 keys are given fixed defaults simultaneously. Compared to *dict 
 getwithdefault*, it has the following disadvantages: 

     * It cannot be used for keys in nested dictionaries. 

     * It takes time proportional to the size of the dictionary, even 
       when only one value is inspected. Since *dict filter key* has an 
       optimisation for this kind of situation, there are apparently 
       maintainers which consider such differences relevant. 

     * The "one *dict merge* early providing defaults for all keys" 
       approach cannot deal with keys that have dynamic defaults, e.g. 
       that the default for option -foo is the effective value of option 
       -bar. 

 Hence although *dict merge* is sometimes appropriate for providing 
 defaults, it is not a universal solution. 

 The basic approach is instead to, as in the *dict_getwithdefault* proc 
 above, first use *dict exists* and then *dict get* if the value 
 existed. Problems with this approach are: 

     * It is redundant: already *dict exists* retrieves the value, but 
       doesn't return it, so *dict get* has to look it up all over 
       again. 

     * It is bulky: if the value in dictionary /D/ of option *-apa* (or 
       its default *bar*) is to be passed as an argument to the command 
       *foo*, then the complete command is 

        foo [if {[dict exists $D -apa]} then {dict get $D -apa}\
          else {return -level 0 bar}]

       or  

        foo [expr {[dict exists $D -apa] ? [dict get $D -apa] : "bar"}]

       which many programmers would find objectionable. The *dict 
       getwithdefault* counterpart is merely 

        foo [dict getwithdefault $D -apa bar]

 The only way to avoid the redundance of an extra look-up seems to be to 
 combine *dict get* with *catch*, like so: 

     if {[catch {dict get $D -apa} value]} then {set value bar} else {set value}

 but this has the disadvantage of hiding other sources of error, such as 
 /D/ not being a dictionary in the first place. This kind of error in a 
 normal processing path is also considered poor style by some. 

 IMPLEMENTATION CHOICES 
------------------------

 Even if it is deemed appropriate to have a dedicated subcommand of 
 *dict* for this, it could be argued that it needn't be part of the 
 compiled Tcl core; since *dict* is an ensemble, anyone can extend it at 
 the script level and "the core can do without this bloat". However, it 
 turns out than an in-core implementation is very easy whereas the 
 alternatives are not so easy. 

 Concretely, the necessary DictGetWithDefaultCmd is a trivial 
 modification of DictExistsCmd, to take one extra argument after the 
 /key/s and change the final 

     Tcl_SetObjResult(interp, Tcl_NewBooleanObj(valuePtr != NULL));

 to 

     Tcl_SetObjResult(interp, valuePtr != NULL ? valuePtr : objv[objc-1]);

 It is nowhere near as easy to do this in a well-behaved extension, 
 since DictExistsCmd relies on TclTraceDictPath to do most of the work, 
 and the latter is AFAICT at best available in the internal stubs table. 

 A script-level implementation is certainly possible, but the minute 
 details of producing core-looking error messages in this case appears 
 considerable both compared to the functional parts of the command and 
 compared to the amount of code needed to do it in the core. 

 REFERENCE IMPLEMENTATION 
==========================

 An implementation is provided on SF, in patch #2370575. 
 [<URL:https://sourceforge.net/support/tracker.php?aid=2370575>] 

 COPYRIGHT 
===========

 This document has been placed in the public domain.  

-------------------------------------------------------------------------

 TIP AutoGenerator - written by Donal K. Fellows 

Re: [TCLCORE] TIP #329 [try] updated

[Twylite <tw...@cr...>]

Since I'm actually far shorter on time than this mail wold suggest, this 
is as much to document my thoughts & findings for later reference as it 
is to explain my reasoning.
> Further feedback on error matching:
>
> It looks like glob matching is not going to cut it.  List prefix 
> matching will be a similarly-powerful and generally safer approach.
>   
Okay, we've done a quick analysis on our source repository (C, C++, Java 
and Tcl, more than 1m lines over hundreds of applications & utils 
developed over 10+ years).  In most cases we avoid branching based on 
errors/exceptions (a number of Best Practice authors advise against 
doing so), so we can categorise our exception handling into "log and 
ignore", "log and rethrow", "log and abort", "recover/retry" (just try 
it again, it may work) and "intelligent recover/retry" (attempt to 
overcome the specific problem then try it again).  Exceptions that 
demand user interaction are included in the "log and X" categories.  
Very few exceptions fall outside these categories.
Since this was a _quick_ analysis I can only talk in impressions, but 
our impression is that "log and X" is far and away the most common use 
case, and the vast majority (way about 80%) of these cases are "catch 
all, log and X". 
The next category down is  intelligent retry (we have applications with 
some really domain specific retry logic), which needs to catch error 
classes (IO errors) and specific errors (ApiException with cause 1125).  
In Java and C++ we catch on classes near the top of the hierarchy and 
then switch or if/then for more specific errors.  In most cases the IO 
errors are coming from a subsystem and we catch all SystemIoExceptions 
rather than (say) java.lang.IOException.

What we learned from this is that if we represent an error as a unique 
prefix word followed by a unique error name or code, then an exact 
prefix match is going to be good enough for us 80% of the time.  If we 
represent an error as a list of increasingly specific elements (API 
SUBSYSTEM ERRNAME ...) then an exact prefix match is going to be good 
enough upwards of 90% of the time _assuming we separate code into 
subsystems that have high cohesion and low coupling_ (which is generally 
a good idea), and capable of greater specificity in error handling than 
Java or C++ (but not of greater generality). 

We identified only one placed in our entire code base that cannot be 
adequately handled by a prefix match against an errorcode list.  A base 
exception class has two integer fields indicating the cause of the 
error; each function in the API has its own associated exception class 
that inherits from the base class.  Yes, it sounds very weird (it is 
very weird), but it allows very high level code to determine which 
_function_ failed, which is the essential bit of information needed to 
determine how to recover.  Without getting into more details about the 
hierarchy, let me assure you that there is no list representation that 
can be matched with a prefix that covers all types of catch we need to 
do (i.e. catch on one of the error fields or on the subclass type). 
If we constructed the errorcode as "XAPI code1 code2 FUNCNAME" then a 
string glob match _could_ work (e.g. "XAPI * FUNCNAME").  But that 
solution isn't good enough -- there is a special case subclass of one of 
the function exceptions, and it was added after the first drop of the 
product.  If we made the errorcode "XAPI code1 code2 FUNCNAME sub1" it 
would break existing trap patterns (that don't have a trailing *).  
Using "XAPI code1 code2 sub1 FUNCNAME" may or may not be backwards 
compatible (e.g. code could be trapping code1 == 5 and then logging 
FUNCNAME ... but did it use lindex end or lindex 3?).

So the general rules one must follow with -errorcode to avoid shooting 
yourself in the foot are:
(1) When trapping errors exact matching against the full errorcode is 
always a bad idea.  It prevents any future extension of the errorcode to 
distinguish between different errors that currently share the same 
errorcode (or new functionality that must conform to an existing error 
model and thus share an existing code).  A match (prefix, glob, etc.) is 
pretty much required if you want maintainable code.
(2) If you are building errorcodes with [list] and matching them with 
glob it becomes impossible to distinguish between error subclasses and 
adjacent errorcodes that share a common prefix.  e.g. "ABC 4" vs "ABC 
42", or "WIN32 INVALID_DATA" vs "WIN32 INVALID_DATATYPE".  To use glob 
you must build errorcodes as a string and add a trailing space or other 
appropriate delimiter, so that you can match "ABC 4 *" instead of "ABC 4*".
(3) If your errorcode information is represented as a list then you 
should assume that the user trapping the error is parsing the list to 
extract useful information, and you should further assume that such 
parsing involves positional arguments (e.g. lindex $errorcode 2).  It is 
therefore only safe to extend errorcodes at one end - you cannot safely 
add more fields in the middle of the errorcode.
(4) The only thing that a glob match can do - that a prefix/suffix match 
cannot - is match stuff in the middle of an errorcode.  Since you can't 
safely extended errorcodes in the middle this is of limited use unless 
you have two different dimensions on which to trap.  Most other 
languages don't support this sort of thing directly in their try/catch 
syntax.

So I'm calling it at this:  using glob is going to lead to mistakes and 
design inflexibility that are hard to overcome unless you notice them 
early, and there is little practical benefit associated with this cost.

A list prefix match (for each element in the pattern there must exist a 
corresponding element with the identical value in the list under 
consideration) is Good Enough, and far safer.

Anything else can be handled with an extension when we know more about 
the problem.

I'll update the TIP accordingly.

Regards,
Twylite

Re: [TCLCORE] TIP #329 [try] updated

[Neil M. <ne...@Cs...>]

On 1 Dec 2008, at 21:32, Joe English wrote:
> Also also: re: "trap" clauses doing a glob match or
> a list prefix match: either way sounds OK to me.
> I have a slight preference for list prefix match,
> but glob matching is also OK.  The critical thing
> is to pick *one* kind of matching and stick with it:
> we don't ever want to see
>
>     [try { ... } trap -match regexp "..." { ... }]

Why not?

-- Neil

This message has been checked for viruses but the contents of an attachment
may still contain software viruses, which could damage your computer system:
you are advised to perform your own checks. Email communications with the
University of Nottingham may be monitored as permitted by UK legislation.

Re: [TCLCORE] TIP #329 [try] updated

[Neil M. <ne...@Cs...>]

On 2 Dec 2008, at 09:21, Twylite wrote:
> Further feedback on error matching:
>
> It looks like glob matching is not going to cut it.  List prefix
> matching will be a similarly-powerful and generally safer approach.
>
> Looking at the functionality of other exception matching systems,  
> we can
> see that Java (with single inheritance) is the equivalent of matching
> against a tree, and C++ (with multiple inheritance) is the  
> equivalent of
> matching against a Directed Acyclic Graph.
>
> Neither string matching nor list prefix matching nor element-wise list
> matching give the functionality of a tree match.
>
> Example:
>
> In Java you may define a MyIOException extends IOException (extends
> Exception extends Throwable).  Code using your API will
>    catch (MyIOException e) { // handle it }
>
> You can safely subclass MyIOException and throw these subclasses  
> without
> breaking code that catches MyIOException (as in the example above).
>
> You can also safely introduce a new superclass between IOException and
> MyIOException (MyCompanysGenericIOException) without breaking code  
> that
> catches MyIOException.

This is because inheritance matching is encapsulated, whereas as  
pattern matching on a concrete data structure isn't. If you really  
want to support these use-cases then they way to go is to use TclOO.

> In C++ you could introduce a completely new base class to  
> MyIOException
> via multiple inheritance, without breaking existing code.

Likewise in Java you can happily add new interfaces (implements  
Blah...).

> With glob/list/prefix matching you can support either subclassing or
> superclassing the error, not both, and not MI.  You cannot use glob/ 
> list
> matching safely to allow for superclassing (don't got there --  
> think of
> the pattern "Throwable * FormatException *" and how many third party
> APIs are going to have something called "FormatException" and you'll
> understand the folly of this approach).
>
> So while element-wise list matching may SEEM really powerful, its  
> just a
> knapsack with a big gun and rope.  Similarly a glob match may seem
> flexible, but in most cases its going to be used a prefix match, and
> often used incorrectly such that it prevents subclassing of errors in
> future.
>
> So, in the absence of an approach to tree/DAG matching, a list prefix
> seems like the most sensible option.

Well, the most flexible form of (concrete) pattern matching is the  
algebraic sort found in functional programming languages: that can  
match anything expressible as a sum-of-products, which includes  
lists, trees, and so on. It still matches against a concrete data- 
type, however, so the use-cases you imagine would still break. The  
only real way to get around that is encapsulation of the matching,  
which can be achieved either by making the new data structure look  
like the old ("views"), or by defining something like an abstract "is- 
a" relation. If you want full flexibility and the ability to match  
arbitrary DAGs then there's always Prolog.

Most of those suggestions aren't practical, at least not with current  
Tcl. Inheritance based matching using TclOO is practical, but would  
represent a radical departure from the way -errorcode is currently  
used. Of the other options, it does look like strict list-prefix  
matching is the least surprising, but also the least flexible.

-- Neil

This message has been checked for viruses but the contents of an attachment
may still contain software viruses, which could damage your computer system:
you are advised to perform your own checks. Email communications with the
University of Nottingham may be monitored as permitted by UK legislation.

Re: [TCLCORE] TIP #329 [try] updated

[Twylite <tw...@cr...>]

>> More to come.
>
> No. Please, no.
More to come ... on whether glob matching or list prefix matching 
actually works best in practice, which will determine which one goes in 
the TIP.

Regards,
Trevor

Re: [TCLCORE] TIP #329 [try] updated

[Donal K. F. <don...@ma...>]

Twylite wrote:
> More to come.

No. Please, no.

The primary constraint is the deadline for new features, being Wednesday
next week. That which is not approved by then won't be making 8.6; we're
strict about that (especially for something as big as a major new
command). It takes a week to run a vote. A solution that answers today's
problems is what is possible to do in the time available; a solution
that is "perfect" is a solution that won't happen. We won't hold the
deadline while waiting for [try] to become "perfect".

You have (realistically) until end of business *today* to produce a
final specification that balances functionality and practicality of
implementation. That means you have to ignore input from people with
theoretically good ideas and instead focus on "good enough for now". I
had to do the same with TclOO; if I'd incorporated everyone's ideas and
dealt with every issue they had, it would never have actually happened.

One good thing is that if it becomes deemed necessary to do something as
complex as exception type tree handling, we can add it in the future
with another keyword. But not now.

Donal.

Re: [TCLCORE] TIP #329 [try] updated

[Twylite <tw...@cr...>]

Further feedback on error matching:

It looks like glob matching is not going to cut it.  List prefix 
matching will be a similarly-powerful and generally safer approach.

Looking at the functionality of other exception matching systems, we can 
see that Java (with single inheritance) is the equivalent of matching 
against a tree, and C++ (with multiple inheritance) is the equivalent of 
matching against a Directed Acyclic Graph.

Neither string matching nor list prefix matching nor element-wise list 
matching give the functionality of a tree match.

Example:

In Java you may define a MyIOException extends IOException (extends 
Exception extends Throwable).  Code using your API will
   catch (MyIOException e) { // handle it }

You can safely subclass MyIOException and throw these subclasses without 
breaking code that catches MyIOException (as in the example above).

You can also safely introduce a new superclass between IOException and 
MyIOException (MyCompanysGenericIOException) without breaking code that 
catches MyIOException.

In C++ you could introduce a completely new base class to MyIOException 
via multiple inheritance, without breaking existing code.

With glob/list/prefix matching you can support either subclassing or 
superclassing the error, not both, and not MI.  You cannot use glob/list 
matching safely to allow for superclassing (don't got there -- think of 
the pattern "Throwable * FormatException *" and how many third party 
APIs are going to have something called "FormatException" and you'll 
understand the folly of this approach).

So while element-wise list matching may SEEM really powerful, its just a 
knapsack with a big gun and rope.  Similarly a glob match may seem 
flexible, but in most cases its going to be used a prefix match, and 
often used incorrectly such that it prevents subclassing of errors in 
future.

So, in the absence of an approach to tree/DAG matching, a list prefix 
seems like the most sensible option.

More to come.

Regards,
Twylite

Re: [TCLCORE] TIP #329 [try] updated

[Twylite <tw...@cr...>]

Some feedback on JE's request to remove the "as" clause:

Approved ;)

I have spoken with two colleagues of different experience levels and 
they agree that associating the variables with each handler is a more 
readable and logical approach.

So the TIP will be updated to:

   try body ?handler ...? ?finally body?
where handler is:
   on code {?em ?opts??} body
   trap pattern {?em ?opts??} body

This is slightly different from my original suggestion in that the 
code/pattern is separate to the args.  These are separate concerns so 
they should not be combined in one list, and it also means that the 
bytecoding needs to do less sublist parsing.

Regards,
Twylite

[TCLCORE] TIP #329 [try] updated

[Andreas L. <av...@lo...>]

Twylite <tw...@cr...> wrote:
> Case in point: we have an API for accessing a hardware coprocessor 
> (let's call the API "ABC").  The coprocessor returns numeric error 
> results in the range 1 to 99.  So we defined the -errorcode to be [list 
> ABC $errnum], e.g. "ABC 4".
> A little while later we realised that error 4 is quite special -- it is 
> allowed to return an extended error information field (the coprocessor 
> had to maintain the use of error 4 for backwards compatibility, but 
> there were times when knowing the exact cause was important).  So we 
> extended the -errorcode in this case to [list ABC $errnum $extra], e.g. 
> "ABC 4 F".

Using list-patterns of course adresses this usecase just perfectly.
{ABC 4} would match "ABC 4 F" as well as "ABC 4 whatever" and also
just "ABC 4",  but still would not match "ABC 42".
{ABC 4 [A-C]} could furthermore be used to match "ABC 4 B" but not
"ABC 4 F.

I guess it will take a few years from now till Donal runs into such
a practical usecase himself, and at that point we will perhaps add
a new ltrap clause with just that type of matching.  Adding it now
would unfortunately put the TIP at risk.

Joe English <jen...@fl...> wrote:
> Just about every other language with a try/catch/finally
> statement binds variables as part of the handler clause;

tcl is different :-)   In this particular case, I even think
it's good so. The resultvar's typical name may be confusing
for the "on ok" block, but one can easily assign it to
a better variable inside the block, if the information is
later needed with an apt name. The "on ok" block is likely
to be rather a rare case.
Other languages have only one exception-variable, and that's
length is generally much smaller than the length of the 
exception name, so two more chars don't hurt. Also in the
other languags the names of the variables aren't visible
anymore after the block.  In tcl's try, having to specify
a variable for each handler would make it much more bulky:
  try {...} trap {{MYERR FOO} vMsg vDict} {...}
And since each variable would outlive the handler, we'd have
a bulk of possibly but not always defined variables afterwards.

In a nutshell:  just because in tcl the error-variables are broader
scoped, it makes sense to define them globally to the try-command.

Twylite <tw...@cr...> wrote:
> So far the opinion seems to be that a list prefix is too limited,
> and a list pattern match (per element glob) is too difficult
That's relative to its perceived usefulness.
> and has no existing reference.
Many things in tcl haven't.

> The intended manner for extending [try] is by adding new handler 
> keywords (if the existing ones are not handling the required use cases).
... ltrap {ABC 4} ...

Re: [TCLCORE] TIP #329 [try] updated

[Twylite <tw...@cr...>]

Hi,
>> In all seriousness, I've already encountered some situations that lead 
>> me to believe there will be teething problems, and some careful thinking 
>> about what conventions to adopt for -errorcode.
>>     
> What none of this does is make good error design easy, and people mess
> it up in other languages too.
Of course :)  I was just pointing out that this approach is going to 
have some limitations and surprises.  In general it is ridiculously easy 
to extend code written in Tcl while maintaining backwards compatibility 
- we regularly refactor the guts of packages to improve functionality 
while providing backwards-compatible behaviour with safe defaults.  Even 
return values can be made backwards compatible using a facade that maps 
the original interface onto a new more powerful function.  Errorcodes 
based on glob matching cannot always be extended in this manner (while 
maintaining backwards compatibility); you need to get some aspect of the 
design right up front, both in how the exception is thrown and how it is 
caught.  So, as I said, teething problems.  And I imaging an ongoing 
cause of questions to c.l.t as new developers encounter this sort of 
thing and grapple with it.
> want the [try] command to be bytecoded, it's definitely the case that
> exact matching of errorcodes is not enough, and it's definitely the case
> that we already have glob matching machinery in the bytecode engine but
> not anything more complex. Going beyond requires *much* more work.
>   
List prefix matching?
> (Myself, I'd prefer to translate the major value from the hardware
> coprocessor in your example into some kind of name token as well as
> putting the number in there afterwards. But that's because I prefer to
> not expose magic numbers to Tcl code, and it's getting into stuff that's
> wildly off-topic...)
>   
While this addresses the confusion between two similar errorcodes ("ABC 
4" vs "ABC 42") ...
(1) The potential for confusion is still there and not necessarily 
obvious to the person naming the errors.  A quick analysis of the WIN32 
platform SDK shows 42 error names that are also prefixes of other error 
names (e.g. ERROR_INVALID_HANDLE is a prefix of 
ERROR_INVALID_HANDLE_STATE, ERROR_INVALID_DATA is a prefix of 
ERROR_INVALID_DATATYPE).  Trapping "WIN32 ERROR_INVALID_HANDLE*" may 
trap errors you don't want, and "WIN32 ERROR_INVALID_HANDLE *" won't 
work unless the code throwing the error ensures that there is a trailing 
space or subclass (like "NONE").
(2) If you trap using an exact match rather than a prefix match, your 
trap will no longer work if the errorcode is subclassed or otherwise 
extended.  So it is not safe to trap using exact matching.  e.g. If I 
extend my Win32 binding to include the numeric code ("WIN32 
ERROR_INVALID_HANDLE 6" instead of "WIN32 ERROR_INVALID_HANDLE") any 
traps that use exact matching will no longer function as intended.

>> A glob against errorcode doesn't work like an OO is-a relationship.
>>     
> True, but is it "good enough"? We don't need perfection immediately.
>   
Dunno :)  The examples/problems given above tell me that an exact match 
is almost never desirable, and a (list) prefix match is likely to be the 
most common case.  A true list prefix match (not a glob match on a 
string) gives functionality equivalent to catching exceptions by class 
in a single-inheritance OO model (e.g. Java).  A glob match can 
potentially do more, but at the risk of handling yourself enough rope to 
get it wrong a lot of the time.

I'll see what else experience brings out today...

Regards,
Twylite

Re: [TCLCORE] TIP #329 [try] updated

[Donal K. F. <don...@ma...>]

Twylite wrote:
> In all seriousness, I've already encountered some situations that lead 
> me to believe there will be teething problems, and some careful thinking 
> about what conventions to adopt for -errorcode.
> 
> Case in point:
[...]

What none of this does is make good error design easy, and people mess
it up in other languages too. On the other hand, coming at things with a
mind to the Art of the Possible :-) it's definitely the case that we
want the [try] command to be bytecoded, it's definitely the case that
exact matching of errorcodes is not enough, and it's definitely the case
that we already have glob matching machinery in the bytecode engine but
not anything more complex. Going beyond requires *much* more work.

(Myself, I'd prefer to translate the major value from the hardware
coprocessor in your example into some kind of name token as well as
putting the number in there afterwards. But that's because I prefer to
not expose magic numbers to Tcl code, and it's getting into stuff that's
wildly off-topic...)

> A glob against errorcode doesn't work like an OO is-a relationship.

True, but is it "good enough"? We don't need perfection immediately.

Donal.

Re: [TCLCORE] CFV: Zlib and PNG

[Donal K. F. <don...@ma...>]

Jeff Hobbs wrote:
> It should be Tcl_Zlib and it should _not_ be a separate package or in 
> any way optional.

Sounds like it ought to be a "Tcl_Zlib" prefix, which is fine with me.
Whether we support a no-op [package require zlib] is the main Q left open.

> No 8.6 Tcl should ship without zlib support built in.

That part I definitely agree with.

Donal.