tcl-core

[TCLCORE] TIP #206: Add an [ftruncate] Command

[Joe M. <jo...@mi...>]

 TIP #206: ADD AN [FTRUNCATE] COMMAND 
======================================
 Version:      $Revision: 1.1 $
 Author:       Joe Mistachkin <joe_at_mistachkin.com>
 State:        Draft
 Type:         Project
 Tcl-Version:  8.5
 Vote:         Pending
 Created:      Friday, 25 June 2004
 URL:          http://purl.org/tcl/tip/206.html
 WebEdit:      http://purl.org/tcl/tip/edit/206
 Post-History: 

-------------------------------------------------------------------------

 ABSTRACT 
==========

 This TIP proposes a new command *ftruncate* for truncating open files. 

 RATIONALE 
===========

 In Tcl, there is currently no way to truncate a file while it is open 
 even though all modern operating systems support such an operation. The 
 current workaround is to close the file and reopen it. However, in 
 cases where the file must be held open (e.g. on Unix where the file has 
 already been deleted and is just being used as scratch space) this 
 workaround cannot be used. 

 SPECIFICATION 
===============

       *ftruncate* /channelId/ ?/length/? 

 The channel specified by /channelId/ must refer to a file that was 
 opened for writing. The /length/ argument must be greater than or equal 
 to zero and can be bigger than the current size of the file. If the 
 /length/ argument is omitted, the file will be truncated to zero bytes. 
 This command will return any errors received from the operating system. 

 EXAMPLE USAGE 
---------------

   # Open the file for reading and writing...
   set file [open "test.txt" {RDWR}]
   # Write some data to the file...
   puts $file "test data"
   
   # Some time later...
   # ... ... ...
   ftruncate $file
   # ... ... ...
   
   # We are done, close the file...
   close $file

 PROPOSED C API CHANGES 
========================

 A new function named *Tcl_TruncateChannel* would be added to the Tcl C 
 API (taking two arguments, the channel to truncate and the length (as a 
 wide integer to facilitate use with large files) to truncate it to in 
 bytes). The channel API would be modified to add the new functionality 
 for standard disk file channels and to allow extension authors to 
 implement it for their custom channel types through specifying in their 
 /Tcl_ChannelType/ structure a value for the new field /truncateProc/ 
 (of type pointer to /Tcl_DriverTruncateProc/, which will be a function 
 with the obvious type signature). Finally, the maximum 
 TCL_CHANNEL_VERSION would be increased to TCL_CHANNEL_VERSION_4 to 
 accommodate this new field. 

 REFERENCE IMPLEMENTATION 
==========================

 A reference implementation does /not/ yet exist. 

 COPYRIGHT 
===========

 This document has been placed in the public domain. 

-------------------------------------------------------------------------

 TIP AutoGenerator - written by Donal K. Fellows 

Re: [TCLCORE] TIP #206: Add an [ftruncate] Command

[Lars <Lar...@ma...>]

At 14.33 +0200 04-06-25, Joe Mistachkin wrote:
>
>       *ftruncate* /channelId/ ?/length/?
>
[snip]
> If the
> /length/ argument is omitted, the file will be truncated to zero bytes.

Wouldn't it make more sense to truncate to the current file position, i.e.,

  ftruncate $F

would be equivalent to

  ftruncate $F [tell $F]

?

Lars Hellstr=F6m

Re: [TCLCORE] TIP #206: Add an [ftruncate] Command

[Neil M. <ne...@cs...>]

On 25 Jun 2004, at 13:33, Joe Mistachkin wrote:
>
>  TIP #206: ADD AN [FTRUNCATE] COMMAND
> ======================================

I'll get the obvious question out of the way - why not [file truncate]?

Cheers,

Neil.

Re: [TCLCORE] TIP #206: Add an [ftruncate] Command

[Jacob L. <jy...@mo...>]

Joe answered that -- the file may already have been deleted from the (user
visible) file system but still held open by the Tcl program. It no longer
has a name by which to refer to it.

However, I also would prefer this as [file truncate] since it's a logical
file operation and does not apply to sockets, fifos and other channel
types. The issue then becomes how to refer to the file. I don't have any
good ideas here. Also on the grounds of prefering a sub-command rather
than a whole new command, I prefer [file truncate].

For the last time (I promise) "oh but for an object oriented
implementation of channels, we would not have this problem". Imagine a
channel type with some basic operations such as "$c read", "$c write", and
"$c close". Then you could have a derived channel type representing a file
and have it have additional operations e.g. "$f tell", "f seek", and "$f
truncate".... Oh well, too late for that, I'm sure.

Finally a question for Joe about the implementation -- is this well
supported on all systems on which Tcl must run?

--JYL

> On 25 Jun 2004, at 13:33, Joe Mistachkin wrote:
>>
>>  TIP #206: ADD AN [FTRUNCATE] COMMAND
>> ======================================
>
> I'll get the obvious question out of the way - why not [file truncate]?
>
> Cheers,
>
> Neil.
>
>
>
> -------------------------------------------------------
> This SF.Net email sponsored by Black Hat Briefings & Training.
> Attend Black Hat Briefings & Training, Las Vegas July 24-29 -
> digital self defense, top technical experts, no vendor pitches,
> unmatched networking opportunities. Visit www.blackhat.com
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core

RE: [TCLCORE] TIP #206: Add an [ftruncate] Command

[Joe M. <jo...@mi...>]

A lot of people wrote:
>
> Why not [file truncate]?

The [file] command operates on files by name.  Logically, a
[file truncate] command would be used to truncate a file based
on it's name.  That is NOT what this TIP proposes.

This TIP proposes the ability to truncate an open file and then
optionally rewrite the contents of that file, something that
cannot be readily accomplished without this TIP.

Lars Hellstr=F6m wrote:
>
> Wouldn't it make more sense to truncate to the current file
> position, i.e.,

TIP modified.

Jacob Levy wrote:
>
> Finally a question for Joe about the implementation -- is this well
> supported on all systems on which Tcl must run?

Yes.

Joe English wrote:
>
> File locking on Unix is a swamp; experienced Unix hackers tend
> to avoid it altogether.

Agreed.

--

Joe Mistachkin <jo...@mi...>

RE: [TCLCORE] TIP #206: Add an [ftruncate] Command

[Reinhard M. <ma...@tc...>]

Hi,

On Fri, 25 Jun 2004 at 13:48, Joe Mistachkin wrote:

> Logically, a [file truncate] command would be used to truncate a file
> based on it's name.

a [file truncate] command could cover both cases:

   file truncate fileName
   file truncate -channel channelId


cu
	Reinhard

RE: [TCLCORE] TIP #206: Add an [ftruncate] Command

[Joe M. <jo...@mi...>]

Reinhard Max wrote:
>
> a [file truncate] command could cover both cases:
> 
>    file truncate fileName
>    file truncate -channel channelId
>

Truncating a file by name is trivially easy now.  I see no 
need to complicate this TIP with functionality that already 
exists.

--

Joe Mistachkin <jo...@mi...>

Re: [TCLCORE] TIP #206: Add an [ftruncate] Command

[Reinhard M. <Rei...@m4...>]

On Fri, 25 Jun 2004 at 15:23, Neil Madden wrote:

> I'll get the obvious question out of the way - why not [file truncate]?

That was my first thought as well, when I saw the name. While I like the
proposed functionality I don't feel comfortable with adding it as a separate
command.

One reason for proposing a separate command could be, that no subcommand of
[file] currently takes an open file (i.e. a channel identifier) as it's
argument. They all work on file names.

Another candidate might be fconfigure. It always takes a channel argument,
but current options manipulate the state of the channel rather than the
underlaying file.

But would it really be so bad to add a new class of subcommands to [file],
that are called with channels, or add the truncate functionality to
fconfigure, which is the central command for manipulating channels?

BTW, while thinking about more missing file operations, flock() comes into
mind, which I think would be a good addition to fconfigure, next to
-blocking, and -buffering.

cu
	Reinhard

Re: [TCLCORE] TIP #206: Add an [ftruncate] Command

[Tom K. <tom...@fr...>]

<..snip..>

> BTW, while thinking about more missing file operations, flock() comes into
> mind, which I think would be a good addition to fconfigure, next to
> -blocking, and -buffering.
> 
> cu
> 	Reinhard

My experience with flock is that it is not a "reliable" command. It isn't 
reliable across nfs (i.e. shared file systems). I think it would be 
misleading to provide a command that isn't "reliable".

Tom K.

Re: [TCLCORE] TIP #206: Add an [ftruncate] Command

[Joe E. <jen...@fl...>]

Tom Krehbiel wrote:
>[Reinhard]
> > BTW, while thinking about more missing file operations, flock() comes into
> > mind, which I think would be a good addition to fconfigure, next to
> > -blocking, and -buffering.
>
> My experience with flock is that it is not a "reliable" command. It isn't
> reliable across nfs (i.e. shared file systems). I think it would be
> misleading to provide a command that isn't "reliable".

Seconded.  Also, there are (at least) three different
styles of file locking in various Unix systems: there's the
BSD "flock" system call, the SYSV "lockf" call, and POSIX
file locking, which uses "fcntl".

There are multiple options -- blocking or nonblocking, shared
or exclusive locks, mandatory or advisory locks -- and not all
options are supported by all APIs.  Different unices support
different subsets of the three APIs (including the empty set),
and while most modern systems support all three, there's no
guarantee that they interact (e.g., if one process obtains a
lock with flock(), it might not be recognized by a process
that uses fcntl() locks.)  And, as Tom mentioned, locking
doesn't usually work over network file systems.  (NFS *claims*
to support POSIX advisory locks, but in my experience it's not 
at all reliable.)

File locking on Unix is a swamp; experienced Unix hackers tend
to avoid it altogether.


--Joe English

  jen...@fl...

Re: [TCLCORE] TIP #206: Add an [ftruncate] Command

[Donal K. F. <don...@ma...>]

Joe English wrote:
> Seconded.  Also, there are (at least) three different
> styles of file locking in various Unix systems: there's the
> BSD "flock" system call, the SYSV "lockf" call, and POSIX
> file locking, which uses "fcntl".

IIRC, some of those other calls *may* be implemented using the fcntl()
call underneath. It's not just a swamp, it's a jungle complete with
man-eating wild beasts.

> And, as Tom mentioned, locking
> doesn't usually work over network file systems.  (NFS *claims*
> to support POSIX advisory locks, but in my experience it's not 
> at all reliable.)

Again, IIRC with NFS it depends on whether the system running the nfsd
is also running a fully-compliant lockd (fairly commonly the case on
Solaris, less common elsewhere). And there are other networked
filesystems (for example AFS) that absolutely do not support locking at
all because they are designed to work with different levels of network
connectivity; I believe you can disconnect an AFS client machine from
its network and not have the whole machine seize solid, which is
definitely not how highly-networked NFS clients tend to operate.

Donal.