#1872 exec (fork) from huge process fails

obsolete: 8.3.3
open
5
2002-04-30
2002-04-30
Anonymous
No

See comp.lang.tcl thread "Unix forking error". When
your tclsh
has a huge process size (say 1Gb virtual memory) and
you
"exec ls", it fails on solaris unless you have 1Gb of
virtual
memory free. The c.l.t. thread contains a test
program. The
discussion in the thread from Peter Da Silva and others
indicates that you can't just use vfork instead of
fork, but
they have outlined a solution.

(Should I have pasted in big pieces of the c.l.t
discussion?)

- Dave Allen: dallen@sequencedesign.com

Discussion

  • Donal K. Fellows

    • labels: 105657 --> 24. Channel Commands
    • assigned_to: dkf --> andreas_kupries
     
  • Andreas Kupries

    Andreas Kupries - 2002-04-30

    Logged In: YES
    user_id=75003

    Pasting big pieces of the cl.t. discussion: IMHO no. The
    reason for that is that it would have been sent with each
    notification email created for every change of this item.

    OTOH, attaching them to the report as separate file (see
    below) has no such problem and should be done.

     
  • David Allen

    David Allen - 2002-04-30

    Logged In: YES
    user_id=530697

    I have created a file (~100 lines) containing the useful
    bits from the c.l.t thread. But, there is no button I can
    see to attach a file to an existing problem report. I have
    just created a sourceforge login and come back, but
    there is still no button.

    OK to paste in 100 lines as a "followup"?

     
  • Andreas Kupries

    Andreas Kupries - 2002-04-30

    Logged In: YES
    user_id=75003

    On this page here, where you attached you comment you can
    attach files too. Immediately after the text entry field
    for comment you will see a section "Followups" which
    displays the comments so far. After that is the
    section "Check to Upload & Attach File". Check the
    checkbox, enter the name of the file in the entry field
    below the checkbox (or use the Browse button), optionally
    enter a file description in the next entry field. Then use
    the button "Submit changes" to upload the new attachment.

     
  • David Allen

    David Allen - 2002-04-30

    Logged In: YES
    user_id=530697

    Sorry, there is no such checkbox. After "followups" is a
    section
    which says "Attached files", and then "Changes". Do I have
    to be a member of the tcl group? If you send me an email to
    the above address, I will send you a screenshot of what I
    see.

     
  • Andreas Kupries

    Andreas Kupries - 2002-04-30

    Logged In: YES
    user_id=75003

    Wow. For the time being paste the 100 lines in as a
    followup. Better have this part of the report now. I can't
    say how long it will take to understand why file upload is
    disabled for you. Note that my page says "Existing files"
    instead of "Attached files".

     
  • David Allen

    David Allen - 2002-04-30

    Logged In: YES
    user_id=530697

    Excerpts from discussion on comp.lang.tcl thread "Unix
    forking error encountered", mid-April 2002:

    I wrote:

    > Here's something you can try. Take your favorite
    > tclsh and add a command with this C code. (Yes, I
    > know this command leaks memory. That's the point :-)
    >
    > int i; void *p;
    > for (i=0; i<512; i++) p = malloc(1024*1024);
    >
    > Each time you call it, it increases the virtual
    > memory size of your process by 512MB. Now, in your
    > tcl shell, call this until the virtual memory size of
    > your process is greater than your remaining free
    > virtual memory.
    >
    > Then "exec ls", and you will get the out of memory
    > message.
    >
    > What I'm hoping to find is a way to run "ls" or other
    > similar unix commands when my virtual memory size is
    > larger than the free virtual memory.

    Dan Smart (cppdan@dansmart.com) wrote:

    > > > [...] how about: vfork(), then without diddling
    > > > any signal handlers immediately exec() a new
    > > > tclsh with command line arguments telling it that
    > > > it's completing a vfork/exec sequence so it can
    > > > tweak the signal handlers where they're supposed
    > > > to be in the child?
    > >
    > > Referring to the tcl 8.3.3 source code, "fork()"
    > > appears only once, in unix/tclUnixPipe.c, function
    > > TclpCreateProcess. After fork()'ing, the child
    > > process code (pid == 0 means child process)
    > > executes RestoreSignals() before execvp()'ing.
    > >
    > > Based on my limited understanding, it seems to me
    > > that the code already does what you suggest.
    > >
    > > I think that line of the tcl source code could be
    > > changed to use vfork() instead of fork() and my VM
    > > problem would go away with no side effects.
    > >
    > > Am I missing something?
    >
    > Yes, an extra exec.
    >
    > Doing anything with signal handlers in the child
    > branch of a vfork is not a good idea, in fact doing
    > anything other than an exec is pretty much frowned
    > upon. If you want to use vfork(), you have to replace
    > the current process image (i.e. the copy that is the
    > child), with a new process image. The new process
    > image can then mess with its environment in anyway it
    > chooses, including change signal handling, mess with
    > file descriptors... Thus to replace fork() with
    > vfork(), you need an intermediate process, that takes
    > the desired eventual command line, sets up a "sane
    > environment", and then execs the final process. The
    > definition of "sane environment" would also have to
    > be passed to the intermediate.
    >
    > Peter's suggestion was that the intermediate process
    > also be an incarnation of tclsh, with a magic flag to
    > tell it to perform the above function. In other words
    > the current (pseudoTCL):
    >
    > if {[fork] == 0} {
    > RestoreSignals
    > exec myProgram and an arg
    > }
    >
    > would become (more pseudoTCL)
    >
    > if {[vfork] == 0} {
    > exec "tclsh -magic myProgram and an arg"
    > }
    >
    > and tclsh would include the following (even more
    > pseudoTCL)
    >
    > if {[string equal [lindex argv 0] "-magic"]} {
    > eval [lreplace $argv 0 0 exec]
    > }
    >
    > Note that this does not include passing the necessary
    > context information to actually restore the signal
    > handlers (we need to set them to whatever the
    > original tclsh inherited from its parent, rather than
    > just to default).