tcl-core — Tcl Core Team discussions about Tcl development

Re: [TCLCORE] Magic FDs, Was: Re: TIP #355: Stop Fast Recycling of Channel Names on Unix

[Donal K. F. <don...@ma...>]

Alexandre Ferrieux wrote:
> Just to clarify: are you talking about putting the brakes on:
>     - the proposed patch in TIP # 355 (chan name reform)

This one, yes.

>     - the proposed patch in 2827015 (code fix in TclpCreateProcess
>       dup()'s for corner cases)

No opinion on that one at this time.

Donal.

Re: [TCLCORE] Magic FDs, Was: Re: TIP #355: Stop Fast Recycling of Channel Names on Unix

[Alexandre F. <ale...@gm...>]

On 9/3/09, Donal K. Fellows <don...@ma...> wrote:
>
>  If we're going to have a semblance of a chance of getting Tcl 8.6 out
>  this month (for Tcl2009) then the chances of a change with non-error
>  incompatibilities in are pretty slim. Given that the driver for this
>  whole matter found a workaround and it's only ever an issue when dealing
>  with buggy code anyway, I'm tempted to leave alone this close to a release.

Just to clarify: are you talking about putting the brakes on:
    - the proposed patch in TIP # 355 (chan name reform)
    - the proposed patch in 2827015 (code fix in TclpCreateProcess
dup()'s for corner cases)
    - both ?

-Alex

Re: [TCLCORE] Magic FDs, Was: Re: TIP #355: Stop Fast Recycling of Channel Names on Unix

[Donal K. F. <don...@ma...>]

Lars Hellström wrote:
> Not sure how much further just the two of us can get on this. (But then 
> again, neither of us is the bug assignee. ;-) ) I was sort-of hoping to 
> invoke some interest from one of the big guys. Particularly, since 
> there would be a script-level-visible change of behaviour, it would be 
> preferable to have "incompatibilities" (if that is how they would be 
> viewed) happen at a minor version increment...

You've been got by the Schedule Monster. :-)

If we're going to have a semblance of a chance of getting Tcl 8.6 out
this month (for Tcl2009) then the chances of a change with non-error
incompatibilities in are pretty slim. Given that the driver for this
whole matter found a workaround and it's only ever an issue when dealing
with buggy code anyway, I'm tempted to leave alone this close to a release.

I'm more concerned that there are inconsistencies in the channel name
system. That's actually a bug in Tcl.

If we slip the deadline of the Conference, then we can revisit this
issue. I just don't like mad dashes to get features in before a stable
release. (My TIP currently in voting is actually just a ratification of
changes already in, and none of it was a change that would break any
code that was even vaguely sane...)

Donal.

Re: [TCLCORE] Magic FDs, Was: Re: TIP #355: Stop Fast Recycling of Channel Names on Unix

[Lars H. <Lar...@re...>]

Alexandre Ferrieux skrev:
> On 9/2/09, Lars Hellström <Lar...@re...> wrote:
>>  Since the subject of FDs 0,1,2 has come up, is anything happening with
>>  respect to bug#2827015
>>  (http://sourceforge.net/tracker/?func=detail&aid=2827015&group_id=10894&atid=110894)
>>  ?
>>
>>  I believe there is a working patch, but I'd prefer to see an
>>  independent confirmation of this...
> 
> Of course I'm biased ;-)
> What about the corner case I mentioned in the comment, and adding a
> dup_over_three() ?

Sounds right, but I'm in well over my head on this. Half the stuff I 
know about it, I learnt from reading the affected source file!

> We might continue this discussion where we left it in the tracker
> instead of tclcore though.

Not sure how much further just the two of us can get on this. (But then 
again, neither of us is the bug assignee. ;-) ) I was sort-of hoping to 
invoke some interest from one of the big guys. Particularly, since 
there would be a script-level-visible change of behaviour, it would be 
preferable to have "incompatibilities" (if that is how they would be 
viewed) happen at a minor version increment...

Lars Hellström

Re: [TCLCORE] TIP #355: Stop Fast Recycling of Channel Names on Unix

[Andreas L. <av...@lo...>]

On Thu, Sep 03, 2009 at 10:33:36AM +0200, Jan Nijtmans wrote:
> 2009/9/2 Andreas Leitgeb <av...@lo...>:
> > With "real problem", you mean the one about the possibility of reviving
> > stale file handles?
> The real problem is the impossibility to diagnose the problem
> when it happens. The program is still broken, but at least
> the user should be told that it is broken.

That's surely the noble intention.  The message, though is: no need anymore
to manage file handles properly. just catch { eof $handle } and if it returns
a true (won't ever return other codes than 0 or 1), then just dispose of the
handle.   Like, as he said, the originator intended to do.

> I'm not promoting that scripts start relying on errors for
> stale handles, ...

It wouldn't really require any promotion...

Re: [TCLCORE] TIP #355: Stop Fast Recycling of Channel Names on Unix

[Donal K. F. <don...@ma...>]

Reinhard Max wrote:
> whoops, I just noticed that
> 
>    close stderr
>    open stderr /some/file w
> 
> results in [open] to return file2, but [file channels] still has 
> {stdin stdout stderr} and both, file2 and stderr can be used.
> 
> I wonder when that was changed, because I think last time I checked 
> it, I got file2 even in [file channels].

Ugh, things are really rather messed up here. It's all now reported in 
https://sourceforge.net/support/tracker.php?aid=2849797

Donal.

Re: [TCLCORE] TIP #355: Stop Fast Recycling of Channel Names on Unix

[Jan N. <nij...@us...>]

2009/9/2 Andreas Leitgeb <av...@lo...>:
> With "real problem", you mean the one about the possibility of reviving
> stale file handles?
The real problem is the impossibility to diagnose the problem
when it happens. The program is still broken, but at least
the user should be told that it is broken.
> Since when do we consider broken programs (and even
> the author of the post that triggered all this admitted that) to be a
> "real problem" for tcl that would even justify *any* incompatibility for
> others?  (And it does, as some scripts may currently deduce the OS file
> handle from the tcl file handle - which is also broken, but not more so
> than relying on errors for stale handles.)

Yes, the loss of the ability to deduce the OS file handle from
the tcl file handle is the potential incompatibility I was talking
about. I'm not promoting that scripts start relying on errors for
stale handles, but if a stale handle is revived by accident, I
expect a good error message in stead of undefined behavior.

But let's handle this TIP as any other TIP. Even though
the TIP deadline is already closed, a majority of votes
could change that, but it should be considered
thourougly!

Regards,
          Jan Nijtmans

Re: [TCLCORE] Magic FDs, Was: Re: TIP #355: Stop Fast Recycling of Channel Names on Unix

[Alexandre F. <ale...@gm...>]

On 9/2/09, Lars Hellström <Lar...@re...> wrote:
> >
>  Since the subject of FDs 0,1,2 has come up, is anything happening with
>  respect to bug#2827015
>  (http://sourceforge.net/tracker/?func=detail&aid=2827015&group_id=10894&atid=110894)
>  ?
>
>  I believe there is a working patch, but I'd prefer to see an
>  independent confirmation of this...

Of course I'm biased ;-)
What about the corner case I mentioned in the comment, and adding a
dup_over_three() ?
We might continue this discussion where we left it in the tracker
instead of tclcore though.

-Alex

Re: [TCLCORE] TIP #355: Stop Fast Recycling of Channel Names on Unix

[Andreas L. <av...@lo...>]

> This TIP tries to solve a real problem, and introduces a
> minor potential incompatibility.

With "real problem", you mean the one about the possibility of reviving
stale file handles?  Since when do we consider broken programs (and even
the author of the post that triggered all this admitted that) to be a
"real problem" for tcl that would even justify *any* incompatibility for
others?  (And it does, as some scripts may currently deduce the OS file
handle from the tcl file handle - which is also broken, but not more so
than relying on errors for stale handles.)

Re: [TCLCORE] TIP #355: Stop Fast Recycling of Channel Names on Unix

[Joe E. <jen...@fl...>]

Larry W. Virden wrote:
> [re: TIP #355: STOP FAST RECYCLING OF CHANNEL NAMES ON UNIX]
> >  This TIP proposes to replace the file descriptor in unix channel names
> >  by an ever-increasing, process-unique counter.
>
> Will there be a method that allows one to redirect stdin, stdout,
> and/or stderr, (or for that matter, other specific file descriptiors)
> if the user needs to do that?

This TIP doesn't affect stdin/stdout/stderr redirection;
no changes there, the existing functionality will continue
to work.  For arbitrary file descriptors you can't do that
in Tcl anyway.

> I know that I've written my share of ksh scripts that make use of that
> functionality to be able to gather information from an applications
> stdout or stderr or to feed various files into an application. It
> would be unfortunate if this type of functionality were lost
> altogether.


--Joe English

[TCLCORE] Magic FDs, Was: Re: TIP #355: Stop Fast Recycling of Channel Names on Unix

[Lars H. <Lar...@re...>]

Larry W. Virden skrev:
> On Tue, Sep 1, 2009 at 6:21 AM, Alexandre
> Ferrieux<ale...@gm...> wrote:
>>  TIP #355: STOP FAST RECYCLING OF CHANNEL NAMES ON UNIX
>>
> 
>>  This TIP proposes to replace the file descriptor in unix channel names
>>  by an ever-increasing, process-unique counter.
> 
> 
> Will there be a method that allows one to redirect stdin, stdout,
> and/or stderr, (or for that matter, other specific file descriptiors)
> if the user needs to do that?
> 
> I know that I've written my share of ksh scripts that make use of that
> functionality to be able to gather information from an applications
> stdout or stderr or to feed various files into an application. It
> would be unfortunate if this type of functionality were lost
> altogether.

Since the subject of FDs 0,1,2 has come up, is anything happening with 
respect to bug#2827015 
(http://sourceforge.net/tracker/?func=detail&aid=2827015&group_id=10894&atid=110894)
?

I believe there is a working patch, but I'd prefer to see an 
independent confirmation of this...

Lars Hellström

Re: [TCLCORE] TIP #355: Stop Fast Recycling of Channel Names on Unix

[Alexandre F. <ale...@gm...>]

On 9/2/09, Donal K. Fellows <don...@ma...> wrote:
> Alexandre Ferrieux wrote:
>
> > Oh if you mean:
> >
> >    close stdout
> >    open somefile w
> >    puts Hey
> >
> > then yes, of course it will work exactly the same, since "stdout" is a
> > special alias mapping to fd 1 in all cases, *and* the allocation of
> > fds remains the OS's job. The only difference is that the handle name
> > returned by the [open] above will likely be "file345" rather than
> > "file1", but who cares ?
> >
>
>  No, it will be "stdout". This is a "special magic" moment, and why the
>  [open] code will have to not assume that the name it suggested for a
>  channel is the name actually given to the channel.

Just tested with HEAD:

 % close stdout
 puts stderr [open somefile w]
 file1

So I infer that with #355 it will return fileN, with N the current
value of the global counter.
Which doesn't prevent [puts foo] and [puts stdout foo] from still working.

But anyway, 355 is orthogonal to your other (true) remark about
suggested vs. given name. We're mereley replacing file$fd by
file$counter regardless of the std* layer.

>  Which isn't to say that I necessarily approve of doing this TIP in 8.6.
>  That case needs to be made more convincingly IMO.

OK, so much for the consensus :(
Now what is most questionable:
 - the need ?
 - the innocuity of the change ?

-Alex

Re: [TCLCORE] TIP #355: Stop Fast Recycling of Channel Names on Unix

[Reinhard M. <ma...@tc...>]

On Wed, 2 Sep 2009 at 16:40, Reinhard Max wrote:

> This brings up the question (though outside the scope of this TIP), 
> whether re-opened file descriptors 0, 1 and 2 should always get the 
> channel names stdin stdout and stderr

whoops, I just noticed that

   close stderr
   open stderr /some/file w

results in [open] to return file2, but [file channels] still has 
{stdin stdout stderr} and both, file2 and stderr can be used.

I wonder when that was changed, because I think last time I checked 
it, I got file2 even in [file channels].

cu
 	Reinhard

Re: [TCLCORE] TIP #355: Stop Fast Recycling of Channel Names on Unix

[Reinhard M. <ma...@tc...>]

On Wed, 2 Sep 2009 at 16:14, Alexandre Ferrieux wrote:

> then yes, of course it will work exactly the same, since "stdout" is 
> a special alias mapping to fd 1 in all cases, *and* the allocation 
> of fds remains the OS's job. The only difference is that the handle 
> name returned by the [open] above will likely be "file345" rather 
> than "file1", but who cares ?

This brings up the question (though outside the scope of this TIP), 
whether re-opened file descriptors 0, 1 and 2 should always get the 
channel names stdin stdout and stderr instead of the current file[012] 
or the proposed file$n, so that e.g. library code has a chance to use 
the standard channels by their standard names, even if they were 
closed and reopened.


Back to this TIP: It might be a good idea to let the new channel 
counter start at numbers that are higher than any file descriptor 
number on Unix can ever be* to avoid confusion, e.g. with dynamically 
loaded channel drivers that might still use the system fd number to 
create the channel name.

*) If that number can be determined at all, otherwise use a number 
that can be assumed to be above what one will typically see on a 
real-life system.


cu
 	Reinhard

Re: [TCLCORE] TIP #355: Stop Fast Recycling of Channel Names on Unix

[Donal K. F. <don...@ma...>]

Alexandre Ferrieux wrote:
> Oh if you mean:
> 
>     close stdout
>     open somefile w
>     puts Hey
> 
> then yes, of course it will work exactly the same, since "stdout" is a
> special alias mapping to fd 1 in all cases, *and* the allocation of
> fds remains the OS's job. The only difference is that the handle name
> returned by the [open] above will likely be "file345" rather than
> "file1", but who cares ?

No, it will be "stdout". This is a "special magic" moment, and why the
[open] code will have to not assume that the name it suggested for a
channel is the name actually given to the channel.

Which isn't to say that I necessarily approve of doing this TIP in 8.6.
That case needs to be made more convincingly IMO.

Donal.

Re: [TCLCORE] TIP #355: Stop Fast Recycling of Channel Names on Unix

[Alexandre F. <ale...@gm...>]

On 9/2/09, Larry W. Virden <lv...@gm...> wrote:
> On Tue, Sep 1, 2009 at 6:21 AM, Alexandre
>  Ferrieux<ale...@gm...> wrote:
>  >
>  >  TIP #355: STOP FAST RECYCLING OF CHANNEL NAMES ON UNIX
>  >
>
>  >
>
> >  This TIP proposes to replace the file descriptor in unix channel names
>  >  by an ever-increasing, process-unique counter.
>
>
>
> Will there be a method that allows one to redirect stdin, stdout,
>  and/or stderr, (or for that matter, other specific file descriptiors)
>  if the user needs to do that?

Oh if you mean:

    close stdout
    open somefile w
    puts Hey

then yes, of course it will work exactly the same, since "stdout" is a
special alias mapping to fd 1 in all cases, *and* the allocation of
fds remains the OS's job. The only difference is that the handle name
returned by the [open] above will likely be "file345" rather than
"file1", but who cares ?

-Alex

Re: [TCLCORE] TIP #355: Stop Fast Recycling of Channel Names on Unix

[Larry W. V. <lv...@gm...>]

On Tue, Sep 1, 2009 at 6:21 AM, Alexandre
Ferrieux<ale...@gm...> wrote:
>
>  TIP #355: STOP FAST RECYCLING OF CHANNEL NAMES ON UNIX
>

>
>  This TIP proposes to replace the file descriptor in unix channel names
>  by an ever-increasing, process-unique counter.


Will there be a method that allows one to redirect stdin, stdout,
and/or stderr, (or for that matter, other specific file descriptiors)
if the user needs to do that?

I know that I've written my share of ksh scripts that make use of that
functionality to be able to gather information from an applications
stdout or stderr or to feed various files into an application. It
would be unfortunate if this type of functionality were lost
altogether.

-- 
Tcl - The glue of a new generation.   http://wiki.tcl.tk/
Larry W. Virden
http://www.xanga.com/lvirden/
Even if explicitly stated to the contrary, nothing in this posting
should be construed as representing my employer's opinions.

Re: [TCLCORE] TIP #355: Stop Fast Recycling of Channel Names on Unix

[Alexandre F. <ale...@gm...>]

On 9/2/09, Jan Nijtmans <jan...@gm...> wrote:
>  [...]
>  So, I would vote YES for the milder form, I don't think it is
>  too late for 8.6.

Thanks Jan. If there is consensus, Donal can you s/8.7/8.6/ in the TIP header ?

-Alex

Re: [TCLCORE] TIP #355: Stop Fast Recycling of Channel Names on Unix

[Jan N. <nij...@us...>]

The thing I like about this TIP is the better error message
when the problem occurs.

2009/9/2 Alexandre Ferrieux <ale...@gm...>:

> Two questions to proceed on #355 now:
>
>  (1) Who is in favour of the more ambitious form (reform all channel
> names) vs the milder one (only unix) ?


I would favour the milder form:
- The ambitious form can always be done later (8.7?), when
 it is determined that the same problem occurs on Windows
 as well. (Don't fix what is not broken, especially not close
 to a new release date)
- less risk for potential incompatability
- What effect would this have on the "iocpsock"
 implementation, another desired feature on
 Windows?


>
>  (2) Is it too late for 8.6 ? (8.7 seems faaar away in the future)


This TIP tries to solve a real problem, and introduces a
minor potential incompatibility. The best time to do that
is with a minor release and 8.7 is still far away to be
beneficial.

So, I would vote YES for the milder form, I don't think it is
too late for 8.6.

Regards,
   Jan Nijtmans

Re: [TCLCORE] TIP #355: Stop Fast Recycling of Channel Names on Unix

[Jan N. <jan...@gm...>]

The thing I like about this TIP is the better error message
when the problem occurs.

2009/9/2 Alexandre Ferrieux <ale...@gm...>:
> Two questions to proceed on #355 now:
>
>  (1) Who is in favour of the more ambitious form (reform all channel
> names) vs the milder one (only unix) ?

I would favour the milder form:
- The ambitious form can always be done later (8.7?), when
  it is determined that the same problem occurs on Windows
  as well. (Don't fix what is not broken, especially not close
  to a new release date)
- less risk for potential incompatability
- What effect would this have on the "iocpsock"
  implementation, another desired feature on
  Windows?

>
>  (2) Is it too late for 8.6 ? (8.7 seems faaar away in the future)

This TIP tries to solve a real problem, and introduces a
minor potential incompatibility. The best time to do that
is with a minor release and 8.7 is still far away to be
beneficial.

So, I would vote YES for the milder form, I don't think it is
too late for 8.6.

Regards,
    Jan Nijtmans

Re: [TCLCORE] TIP #355: Stop Fast Recycling of Channel Names on Unix

[Alexandre F. <ale...@gm...>]

Two questions to proceed on #355 now:

 (1) Who is in favour of the more ambitious form (reform all channel
names) vs the milder one (only unix) ?

 (2) Is it too late for 8.6 ? (8.7 seems faaar away in the future)

-Alex

Re: [TCLCORE] An Emacs tuning

[Donal K. F. <don...@ma...>]

Konstantin Khomoutov wrote:
> I agree on that this was the worst possible choice for indentation, but
> it was made long time ago, Tcl source code contains spaces mixed with
> tabs while assuming tabs to be 8 spaces wide.

What I'm not saying is whether or not people should use tabs when
editing Tcl's source. What I'm saying is that *if* they use tabs, then
they've got to be 8 character tabs. We've had this rule for ages and
it's part of the original Engineering Manual/Style Guide. (The one real
advantage of the Emacs way is that it puts the noise at the end of the
file, well out of the way.)

Extensions and scripts not part of Tcl itself can do anything they want
(there's even code on the wiki to allow the use of Python-style indent
blocks through clever preprocessing; weird, but it exists). I don't
recommend using 40 character tabs though... ;-)

Donal.

Re: [TCLCORE] An Emacs tuning

[Larry M. <lm...@bi...>]

On Tue, Sep 01, 2009 at 10:42:45AM -0700, Brian Griffin wrote:
> On Tue, Sep 1, 2009 at 1:14 AM, Kristoffer Lawson <se...@fi...>wrote:
> 
> >  mixing tabs and spaces makes no sense whatsoever. It's the worst
> > of both worlds.
> >
> 
> Hear hear!

Coming into this late but we use ts=8 and sw=4 because while control 
blocks are all tabs, continuation lines are 4.

	if (whatever) {
		blah;
	}

	if (whatever && something_else && some_long_frigging_java_whatever &&
	    another_long_frigging_java_whatever && blah) {
		blech;
		blech2;
		blech3;
	}

Bill Shannon normal form, and he's the only guy with an option in indent
just for him :)
-- 
---
Larry McVoy                lm at bitmover.com           http://www.bitkeeper.com

Re: [TCLCORE] An Emacs tuning

[Brian G. <bri...@ea...>]

On Tue, Sep 1, 2009 at 1:14 AM, Kristoffer Lawson <se...@fi...>wrote:

>  mixing tabs and spaces makes no sense whatsoever. It's the worst
> of both worlds.
>

Hear hear!

Re: [TCLCORE] An Emacs tuning

[Konstantin K. <fla...@us...>]

Kristoffer Lawson wrote:

>>>> Q: Is there any reason not to add an explicit:
>>>> * tab-width: 8
>>>> so that the source will be laid out correctly in all emacses
>>>> regardless of their local tab-width ?
>>> Is there any reason the files should actually contain any tabs at  
>>> all? I'd
>>> prefer spaces only.
>> Well, expecting tens of developers to all follow a non-default rule
>> is...optimistic :)
>> Or is there an Emacs version after which the default for C mode is to
>> never output a tab ?
> I'm not that sure about non-default... Many developers with other  
> editors will not be outputting tabs. Just spaces. If you think about  
> it, mixing tabs and spaces makes no sense whatsoever. It's the worst  
> of both worlds.
I agree on that this was the worst possible choice for indentation, but
it was made long time ago, Tcl source code contains spaces mixed with
tabs while assuming tabs to be 8 spaces wide.
So I think it's better to document the format so that the users of text
editors other than Emacs at least know how to tweak their software to
get sensible display and properly save modified files if needed.

For the record -- the Vim modeline matching the formatting of the Tcl
source code is:
/* vim: set ts=8 sts=4 sw=4 sts=4 noet: */