squeak-vmdev — CVS-related Virtual Machine Development Issues

Re: [Squeak-VMdev] OSS (was: something completely different)

[Tim R. <ti...@su...>]

In message <Pin...@ti...>
          Ian Piumarta <ian...@in...> wrote:


> (Or I could just give up right now and go make all my stuff compatible
> with VMMaker instead, which was supposed to be the next thing on the
> "to-do" list two weeks ago. ;-)
Much more sensible; after all, the sound code will realise it is being
ignored and then immediately start to play nice.

tim

-- 
Tim Rowledge, ti...@su..., http://sumeru.stanford.edu/tim
Don't anthropomorphize comupters. They _hate_ that.

[Squeak-VMdev] OSS (was: something completely different)

[Ian P. <ian...@in...>]

Lex,

> That wasn't the issue.  It was something along the lines of:
> StartRecording does not get called if CanRecordWhilePlaying is turned
> on.

Ok, thanks.  I'll watch out for this.

> Right.  Though using a sound server is nice, too.  :)

I'll resist the temptation (for now) to air my observations about NAS
running over the net between different endian machines.  ;-)

> There is a difference in the handling of stereo: with playback, monaural
> sounds are encoded as stereo but with one chanel always set to 0.

Ahh, I'd missed that one.

Err, then again... are you sure?  Without digging as deeply as I should,
here's the beginning of playLoop:

	| bytesPerSlice count willStop mayStop |
	mayStop _ Preferences soundStopWhenDone.
	bytesPerSlice _ Stereo ifTrue: [4] ifFalse: [2].

which seems to suggest that mono really is 1 channel.

> Also, I don't see why recording could not be stereo -- there is a flag
> for it in StartRecording.

I meant it's (currently) always set to mono in the image.  (The support
code makes no assumptions about # of Squeak channels in either direction.
Everything is completely symmetric.)

> True, but in what case will this happen?  The device should still be
> operating a fragment at a time.

Absolutely.  There's no other way it can work, given that something is
eventually going to DMA all those samples.  The problem is with select()
itself.

Try this:
  - compile the attached program
  - launch xmixer (or somesuch)
  - select CD digital output as your recording source
  - set all relevant in/out volumes so you'll here something from the CD
    (analog CD and/or "monitor" at zero [if you have it] but PCM and
    IGain turned up [just a little -- if you value your speakers ;])
  - put a very calming CD into the drive (you'll need it ;)
  - start the CD playing (with cdplay, or somesuch)
  - run the program you just compiled.

If all is well you should hear music with no gaps, clicks, or repeated
fragments.  The program should be printing lots and lots of `.'s
(indicating that select() is twiddling thumbs furiously) with the
occasional `r' and `w' interspersed (approx the same number of each).

On the other hand, if your OSS drivers are as broken as all those that
I've come across so far, you'll see something like this:

  - lines of `rw.' with the cursor sitting mostly over the `w'.  This
    means that select() is saying the input device is readable even
    when it isn't, and read is blocking on input.

or maybe:

  - lots of repeated `.w<N' which means the input side of the device
    is never being select()ed for read.

(The offending drivers ignore O_NONBLOCK too, just to add insult to
injury.)

FWIW, all of the OSS-based sound programs/daemons that I've downloaded and
autopsied in the last week (disregarding the trivial ones that just pump
data using blocking read/write) use clocked i/o rather than select().

FWIW #2, it was the first problem that started this saga.  Squeak was so
stuck in blocked reads (blocking every time it did a quickCheckForIntrs)
that it became literally uninterruptable.

I wouldn't hesitate to put SIGALRM back into the VM (I've already got all
the code to implement ITIMER-based periodic "triggers") if I weren't so
terrified by the potential number of syscalls that might break from EINTR
and which are currently not checked (not to mention the amount of work
needed to resuscitate my slowly-rusting SunOS box to be absolutely sure
about it ;).

> The only bad case I can think of is
> because of the 100 frame limitation: the device may have 10 frames left
> in a fragment, but Squeak might refuse to use them.  If that limitation
> were removed, then select() would work fine, wouldn't it?

I don't think so.  It was one of the first things I tried.  (I removed the
minimum limit altogether.)  The only thing that did work was preventing
the output from being select()ed on each and every poll.  (Failing
snd_Start when semaIndex > 0 makes the image use the old-style polling
loop.  Then don't bother registering the dsp output fd with aio, and use
GETOSPACE in snd_availSpace.  Works like a dream: Squeak eats zero CPU and
sound output is flawless.  If the same aproach would worked for input
there would be no problem, but O_NONBLOCK is broken in several drivers
which also don't implement [or return garbage for] GETISPACE.)

So the problem really is in the totally broken interaction between the OSS
drivers and select().

I started fixing this for PowerBook (at least on the driver side) but
having been dropped into xmon for the Nth time due to a kernel exception
down in the bowels of do_select() I rapidly lost motivation for it.  It's
the wrong (short-term) solution anyway: the OSS drivers are thoroughly
broken on PowerMac and NM256/AC97 (which crops up in a _lot_ of PC
hardware), and it's anybody's guess on how many others too.  So I think it
makes more sense to assume an absolute minimum of functionality from the
OSS drivers (including the absence of select()) and then try to make the
VM code sufficiently intelligent to cope using only what's left.

(Or I could just give up right now and go make all my stuff compatible
with VMMaker instead, which was supposed to be the next thing on the
"to-do" list two weeks ago. ;-)

Ian

PS: A quick-and-dirty solution would be to spawn a thread to talk to the
device, connected to the VM through a pipe or two with which select()
could happily work.  But this is total overkill.

Re: Please read and speak up! (was RE: [Squeak-VMdev] RE: Pending 3.2 and VM sources @ sourceforge)

[Lex S. <le...@cc...>]

Ian Piumarta <ian...@in...> wrote:
> > If so, and thus the question is the Squeak code, you might look at the
> > OSS code on SourceForge.
> 
> This is the code I started with.  I fell over the recording problems while
> testing the merge from SF last week.  (FWIW, sound is the last of the
> merges: everything else was done 10 days ago. ;-)  

Super!

> > It's not full duplex, but that's because
> > the last time I checked full-duplex didn't work in the image, either.
> 
> Preferences at: #canRecordWhilePlaying.

That wasn't the issue.  It was something along the lines of:
StartRecording does not get called if CanRecordWhilePlaying is turned
on.


> (Tip-of-the-week #2: `Preferences at: #stopSoundWhenDone put: true' will
> make Squeak coexist much more amicably with any other sounds apps that
> might be interested in opening /dev/dsp from time to time.)

Right.  Though using a sound server is nice, too.  :)  All the new
GUI_ish programs for Linux seem to support at least one sound server --
new Linux programs seem to make a lot more beeps and bloops than
traditional Unix programs would!


> > There are some functions squeakToCard()
> > and cardToSqueak() which might be handy, even if you rewrite all the rest
> > of it.
> 
> Oops -- I think the format conversions were the first thing I decided had
> to go. ;-)  They now don't contain any tests at all: there is one
> convertor for every possible combination of { squeak, driver } x { size,
> end, sign, channels }

That's cool too.  I decided not to do it that way because the CPU usage
was already pretty small.  But if you are writing individual conversion
routines, then macro tricks are great!


> > Incidentally, the formats are different for recording and playback.
> 
> The only difference that I've spotted is that recording is always mono
> whereas playback has the choice.


There is a difference in the handling of stereo: with playback, monaural
sounds are encoded as stereo but with one chanel always set to 0.  With
recording, monaural sounds are encoded with just one chanel.

Also, I don't see why recording could not be stereo -- there is a flag
for it in StartRecording.


> I'll get back to you if/when I suspect something in the image smells
> funny.  So far I can happily record and playback in half-duplex, so the
> formats don't seem to be a problem.  The real headache is getting the
> timing just right with full-duplex: the fragment size can be different (so
> there's no single, simple relationship between read/write timing) and
> select() is utterly useless because (i) it says "write me" when the image
> refuses to write less than 100 frames and (ii) it says "read me" when as
> little as one frame is available

True, but in what case will this happen?  The device should still be
operating a fragment at a time.  The only bad case I can think of is
because of the 100 frame limitation: the device may have 10 frames left
in a fragment, but Squeak might refuse to use them.  If that limitation
were removed, then select() would work fine, wouldn't it?


Lex

Re: Please read and speak up! (was RE: [Squeak-VMdev] RE: Pending 3.2 and VM sources @ sourceforge)

[Ian P. <ian...@in...>]

Hi Lex,

> Does "cat < /dev/dsp > testfile" and then "cat testfile > /dev/dsp" work?
> ie, is recording working at all on your Linux box?

Actually it's better than that.  On my PowerBook (OSS/dmasound and
AWACS low-level driver):

  cat < /dev/dsp > /dev/dsp

works just fine (and as it says somewhere on opensound.com, the above
makes me a perfectly serviceable "cheesy little delay line" ;-).

FWIW, SNDCTL_DSP_GETCAPS reports DSP_CAP_DUPLEX for dmasound/AWACS, so I'd
expect to be able to open the same device twice.

On Intel (nm256av + ac97 drivers):

  cat < /dev/dsp > /dev/dsp

fails (resource not available), but

  cat < /dev/dsp > /dev/dsp1

works just fine.  DSP_CAP_DUPLEX is not reported for the nm256, so I
shouldn't expect to be able to open the same device twice -- and /dev/dsp
and /dev/dsp1 do indeed seem to be different devices with dsp1 being the
output half and dsp being the input half (but which also knows how to turn
into a "shadow" of dsp1 when it's opened for writing).

The above is all pretty much consistent with what it says in the latest
OSS API documentation.  For AWACS I can also open /dev/dsp O_RDWR (as
claimed in the OSS doc for DSP_CAP_DUPLEX devices) and then perform both i
and o through the same fd.  (This locks both directions into the same
format/rate/channels, but that's a bug in the OSS API and not an intrinsic
limitation in the AWACS LL driver.)

> If so, and thus the question is the Squeak code, you might look at the
> OSS code on SourceForge.

This is the code I started with.  I fell over the recording problems while
testing the merge from SF last week.  (FWIW, sound is the last of the
merges: everything else was done 10 days ago. ;-)  I've spent the
intervening time learning more than I ever really wanted to about
low-latency sound programming in general, OSS in particular and the
internals of its low-level drivers in gorey detail.  (But I'm a much
wiser, and therefore happier, hacker now. ;-)

(Once I got the output timing sorted out properly Squeak immediately
dropped to between 0.1 and 0.5 % CPU while playing the stereoBachFugue --
which makes me realise just how _much_ dsp could potentially be done in
Squeak without ever risking underruns...)

> It's not full duplex, but that's because
> the last time I checked full-duplex didn't work in the image, either.

Preferences at: #canRecordWhilePlaying.

(Tip-of-the-week #2: `Preferences at: #stopSoundWhenDone put: true' will
make Squeak coexist much more amicably with any other sounds apps that
might be interested in opening /dev/dsp from time to time.)

> Anyway, getting full duplex to work should be a lot easier than getting
> all the format conversions to work.

That's what I thought... but the format conversions turned out to be
utterly trivial and timing issues are turning out to be a horrendous
nightmare.

> There are some functions squeakToCard()
> and cardToSqueak() which might be handy, even if you rewrite all the rest
> of it.

Oops -- I think the format conversions were the first thing I decided had
to go. ;-)  They now don't contain any tests at all: there is one
convertor for every possible combination of { squeak, driver } x { size,
end, sign, channels } (which comes to 24 converters after you've deleted
the meaningless ones -- and just 24 lines of code when you've figured out
the right macros ;-) where each one is a trivial, tight, ultrafast loop.
Appropriate input/output converters are chosen when the device format is
queried/set after opening dsp.  Overhead: one indirect fn call per
buffer's worth of i or o.

I decided the rest had to go when I started trying to do full duplex.  In
short:

1.  Everything now respects *rigorously* the OSS API.
2.  The image was spending most of its time in tight loops because of
    the (inconsistent) way it uses the semaphores, the arbitrary 100
    frame minimum in output, and the arbitrary min read size for input
    based on the buffer allocated in the image.
3.  The conversion routines are now optimal (modulo any weirdo string
    handling insns that the compiler might not be clever enough
    to emit).
4.  select() is basically useless for full-duplex sound i/o.
5.  Different cards have different capabilities so any one of two
    (and a half ;) mutually incompatible mechanisms might be needed to
    achieve full-duplex in all cases where it's possible.
6.  The old code had more bubble gum and scotch tape holding it together
    than it had actual implementation.

And maybe this is a teeny bit extreme, but...

7.  Doing things "right" in the Squeak code panics the latest Linux
    kernel: the perfect incentive to go fix the kernel!  (The problem
    is obvious and seems trivial to fix, so I'm sucking in bitkeeper
    repositories as I type... ;-)

> Incidentally, the formats are different for recording and playback.

The only difference that I've spotted is that recording is always mono
whereas playback has the choice.  (So initiating recording when stereo
playback is already active, with a driver like dmasound which imposes the
same # channels for full-duplex [rather stupidly since the LL drivers can
happily cope with different formats in duplex] will cause
SNDCTL_DSP_CHANNELS to answer `2' when 1 channel is requested and the
support code to subsequently select a 2->1 channel converter correctly.  
Seems to work.  If you have pathological cases, I'm interested!)

> posted a changeset that updates the comments for this, but I don't know
> if they got included in any image.  I can dig up those comments again if
> it would help -- it took me a few hours to trace it all down!

I'll get back to you if/when I suspect something in the image smells
funny.  So far I can happily record and playback in half-duplex, so the
formats don't seem to be a problem.  The real headache is getting the
timing just right with full-duplex: the fragment size can be different (so
there's no single, simple relationship between read/write timing) and
select() is utterly useless because (i) it says "write me" when the image
refuses to write less than 100 frames, and (ii) it says "read me" when as
little as one frame is available -- both of which tend to keep the
player/recorder processes in such tight loops that the image has a hard
time getting screen updates out, let alone noticing incoming keyboard
interrupts.

Regards,

Ian

PS: There's a performance bug in the player process.  Rather than

       [   [self isSpaceAvailable] whileFalse: [self twiddleThumbs].
           lotsOfSounds do: [sound playInto: buffer].
           self outputNoisesFrom: buffer.
       ] rinseAndRepeat.

    I think it might make more sense to pregenerate a lead time's worth
    of samples into the buffer before deciding whether to twiddleThumbs.
    Otherwise we're effectively wasting a fragment's worth of latency
    and/or bandwidth when lotsOfSounds and/or sample frequency is large
    w.r.t. CPU speed.  Just a thought...

Re: Please read and speak up! (was RE: [Squeak-VMdev] RE: Pending 3.2 and VM sources @ sourceforge)

[Lex S. <le...@cc...>]

On Thu, May 23, 2002 at 12:55:31PM +0200, Ian Piumarta wrote:
> > - Everyone is happy.
> 
> I'll be happy when I persuade the OSS recording code to produce something
> other than silence, 144 dB noise, VM freeze or kernel panic.  (The last
> one's my fault for thinking it might work better with the 2.4.19-pre8
> drivers.  Stupid me.  ;-)

Does "cat < /dev/dsp > testfile" and then "cat testfile > /dev/dsp" work?
ie, is recording working at all on your Linux box?

If so, and thus the question is the Squeak code, you might look at the
OSS code on SourceForge.  It's not full duplex, but that's because
the last time I checked full-duplex didn't work in the image, either.
Anyway, getting full duplex to work should be a lot easier than getting
all the format conversions to work.  There are some functions squeakToCard()
and cardToSqueak() which might be handy, even if you rewrite all the rest
of it.

Incidentally, the formats are different for recording and playback.  I
posted a changeset that updates the comments for this, but I don't know
if they got included in any image.  I can dig up those comments again if
it would help -- it took me a few hours to trace it all down!

Lex

Re: Please read and speak up! (was RE: [Squeak-VMdev] RE: Pending 3.2 and VM sources @ sourceforge)

[Ian P. <ian...@in...>]

On Wed, 15 May 2002 gor...@bl... wrote:
> I am actually waiting for people to say "Ok, let's do it that way."

Ok, let's do it that way.

> - Everyone is happy.

I'll be happy when I persuade the OSS recording code to produce something
other than silence, 144 dB noise, VM freeze or kernel panic.  (The last
one's my fault for thinking it might work better with the 2.4.19-pre8
drivers.  Stupid me.  ;-)

Ciao,

Ian

RE: [Squeak-VMdev] RE: Pending 3.2 and VM sources @ sourceforge

[Rob W. <rwi...@at...>]

Ian Piumarta <ian...@in...> wrote:
> On Mon, 13 May 2002, Rob Withers wrote:
> > 
> > Do we know that we can restrict access to the main trunk to the
> > maintainers, then issue expanded rights to select individuals to their
> > branches?
> 
> You can do it with a trivial commitinfo script to filter the usernames
> that are allowed to commit to a particular area.  (The script is called on
> every commit with env vars and arguments telling who, what, where, etc.  
> If the script exits with nonzero then the commit is aborted.  A few lines
> of case/sed/test/etc. does the trick.)
> 
> Of course, for this to be in any way secure, SF must only allow admins to
> meddle with the contents of CVSROOT/commitinfo.

It seems this would work technically.  I must ask if you are really,
really...really sure you aren't comfortable with just leaving it open
and tagging things voraciously, so we can recover the stable points?  
Adding technical complexity to publishing code sounds scary to me, but
then I'm not the one who gets yelled at.  :)

Are you aware of the commit mailing list for the squeak SF?  In this
way, you can see all commits that occur and the 'VM harvesters' could
verify operation.

welcome back,
Rob

RE: [Squeak-VMdev] RE: Pending 3.2 and VM sources @ sourceforge

[<gor...@bl...>]

Bert Freudenberg <be...@is...> wrote:
> With the commitinfo script it is easy to restrict access on a directory
> basis. It's somewhat harder to limit access to individual files. But I see
> no way to ensure a file is indeed committed to a branch, rather than to 
> the trunk.

See hits on:
http://groups.google.com/groups?q=commitinfo+restrict+branch

It looks doable. I haven't followed up the leads found though.

regards, Göran

RE: [Squeak-VMdev] RE: Pending 3.2 and VM sources @ sourceforge

[Bert F. <be...@is...>]

With the commitinfo script it is easy to restrict access on a directory
basis. It's somewhat harder to limit access to individual files. But I see
no way to ensure a file is indeed committed to a branch, rather than to 
the trunk.

See http://www.gnu.org/manual/cvs-1.9/html_node/cvs_134.html
and http://www.loria.fr/~molli/fom-serve/cache/93.html

-- Bert

On Tue, 21 May 2002, Andreas Raab wrote:

> Ian,
> 
> I think this would be great. My major concern is mostly making errors so
> I wouldn't even require this to be "secure" in any way. What it should
> do is to make it harder to do stupid mistakes and for that it seems
> perfectly applicable.
> 
> Cheers,
>   - Andreas
> 
> 
> > -----Original Message-----
> > From: squ...@li... 
> > [mailto:squ...@li...] On Behalf 
> > Of Ian Piumarta
> > Sent: Tuesday, May 21, 2002 4:44 PM
> > To: Rob Withers
> > Cc: squ...@li...
> > Subject: RE: [Squeak-VMdev] RE: Pending 3.2 and VM sources @ 
> > sourceforge
> > 
> > 
> > On Mon, 13 May 2002, Rob Withers wrote:
> > > 
> > > Do we know that we can restrict access to the main trunk to the
> > > maintainers, then issue expanded rights to select 
> > individuals to their
> > > branches?
> > 
> > You can do it with a trivial commitinfo script to filter the usernames
> > that are allowed to commit to a particular area.  (The script 
> > is called on
> > every commit with env vars and arguments telling who, what, 
> > where, etc.  
> > If the script exits with nonzero then the commit is aborted.  
> > A few lines
> > of case/sed/test/etc. does the trick.)
> > 
> > Of course, for this to be in any way secure, SF must only 
> > allow admins to
> > meddle with the contents of CVSROOT/commitinfo.
> > 
> > Ciao,
> > Ian

RE: [Squeak-VMdev] RE: Pending 3.2 and VM sources @ sourceforge

[Andreas R. <And...@gm...>]

Ian,

I think this would be great. My major concern is mostly making errors so
I wouldn't even require this to be "secure" in any way. What it should
do is to make it harder to do stupid mistakes and for that it seems
perfectly applicable.

Cheers,
  - Andreas


> -----Original Message-----
> From: squ...@li... 
> [mailto:squ...@li...] On Behalf 
> Of Ian Piumarta
> Sent: Tuesday, May 21, 2002 4:44 PM
> To: Rob Withers
> Cc: squ...@li...
> Subject: RE: [Squeak-VMdev] RE: Pending 3.2 and VM sources @ 
> sourceforge
> 
> 
> On Mon, 13 May 2002, Rob Withers wrote:
> > 
> > Do we know that we can restrict access to the main trunk to the
> > maintainers, then issue expanded rights to select 
> individuals to their
> > branches?
> 
> You can do it with a trivial commitinfo script to filter the usernames
> that are allowed to commit to a particular area.  (The script 
> is called on
> every commit with env vars and arguments telling who, what, 
> where, etc.  
> If the script exits with nonzero then the commit is aborted.  
> A few lines
> of case/sed/test/etc. does the trick.)
> 
> Of course, for this to be in any way secure, SF must only 
> allow admins to
> meddle with the contents of CVSROOT/commitinfo.
> 
> Ciao,
> Ian
> 
> 
> _______________________________________________________________
> 
> Don't miss the 2002 Sprint PCS Application Developer's Conference
> August 25-28 in Las Vegas -- http://devcon.sprintpcs.com/adp/index.cfm
> 
> _______________________________________________
> Squeak-VMdev mailing list
> Squ...@li...
> https://lists.sourceforge.net/lists/listinfo/squeak-vmdev
> 

RE: [Squeak-VMdev] RE: Pending 3.2 and VM sources @ sourceforge

[Ian P. <ian...@in...>]

On Mon, 13 May 2002, Rob Withers wrote:
> 
> Do we know that we can restrict access to the main trunk to the
> maintainers, then issue expanded rights to select individuals to their
> branches?

You can do it with a trivial commitinfo script to filter the usernames
that are allowed to commit to a particular area.  (The script is called on
every commit with env vars and arguments telling who, what, where, etc.  
If the script exits with nonzero then the commit is aborted.  A few lines
of case/sed/test/etc. does the trick.)

Of course, for this to be in any way secure, SF must only allow admins to
meddle with the contents of CVSROOT/commitinfo.

Ciao,
Ian

Re: [Squeak-VMdev] VMMaker on 3.3a

[Dan I. <Da...@Sq...>]

>Hi Guys,
>
>Just to let you know: With a variation of Hendriks CS (which he just
>sent to the Squeak mailing list) 3.3a is back in the VMBuilder arena.
>I've posted a slight variation of Hendriks CS, opened a VMMaker in 3.3a
>"generated all", typed "build" and it built out of the box. Yayy!
>
>Cheers,
>  - Andreas

Yayy!

This is great news, guys!

	- D

[Squeak-VMdev] VMMaker on 3.3a

[Andreas R. <And...@gm...>]

Hi Guys,

Just to let you know: With a variation of Hendriks CS (which he just
sent to the Squeak mailing list) 3.3a is back in the VMBuilder arena.
I've posted a slight variation of Hendriks CS, opened a VMMaker in 3.3a
"generated all", typed "build" and it built out of the box. Yayy!

Cheers,
  - Andreas

Re: Please read and speak up! (was RE: [Squeak-VMdev] RE: Pending 3.2 and VM sources @ sourceforge)

[Tim R. <ti...@su...>]

I'll happily agree to this; especially if we can then make sqcvs tools
that make it difficult to screwup :-)

tim

RE: Please read and speak up! (was RE: [Squeak-VMdev] RE: Pending 3.2 and VM sources @ sourceforge)

[Bert F. <be...@is...>]

On Wed, 15 May 2002, Andreas Raab wrote:

> I thought I said that already ;-) So "let's do it that way".

I'm in. Surely Goran's policy document will clarify the actual procedure.  

I think we should put it at http://squeak.sourceforge.net, which is the
most obvious location people will be looking for information. That URL
maps to /home/groups/s/sq/squeak/htdocs on your SF shell account.
Currently there is only a PHP script redirecting traffic to
http://sourceforge.net/projects/squeak ...

-- Bert

RE: Please read and speak up! (was RE: [Squeak-VMdev] RE: Pending 3.2 and VM sources @ sourceforge)

[Andreas R. <And...@gm...>]

> I am actually waiting for people to say "Ok, let's do it that way."
> so that:

I thought I said that already ;-) So "let's do it that way".

Cheers,
  - Andreas

Re: Please read and speak up! (was RE: [Squeak-VMdev] RE: Pending 3.2 and VM sources @ sourceforge)

[<cg...@cd...>]

gor...@bl... said:
> I am actually waiting for people to say "Ok, let's do it that way." so
> that: 

As far as my vote counts in this eminent company, I'd say "go ahead". The 
proposed model seems quite workable to me.

And count me in for CVS coaching - I've been working with the stuff since 
~1990, so I think I can claim to be a bit experienced with it.

Regards,

Cees


-- 
Cees de Groot               http://www.cdegroot.com     <cg...@cd...>
GnuPG 1024D/E0989E8B 0016 F679 F38D 5946 4ECD  1986 F303 937F E098 9E8B

Please read and speak up! (was RE: [Squeak-VMdev] RE: Pending 3.2 and VM sources @ sourceforge)

[<gor...@bl...>]

Rob Withers <rwi...@at...> wrote:
[SNIP]
> I think we just tag the main trunk for the major releases.  This way we
> can get back to that point for testing reported bugs, etc.  I pretty
> much see everyone hacking in the trunk and this will cause ongoing
> problems of inconsistent platform support for new features.  We have
> been fighting this since day one of using SF with VMMaker, but the
> response time is typically very short.   I don't know how we can resolve
> this. 

Eh, have you (and everyone else) read Ian's, mine and others postings on
the proposed "work model"?
If not then please do - it is the larger posts in this thread starting
with one from Ian I believe. :-)

I am actually waiting for people to say "Ok, let's do it that way." so
that:

- Andreas and Ian joins up. And anyone else that was hesitating before.
- I can sit down and write the "CVS howto + policy draft" so that chaos
and confusion does not need to reign.
- Everyone is happy.

regards, Göran

RE: [Squeak-VMdev] RE: Pending 3.2 and VM sources @ sourceforge

[<gor...@bl...>]

Rob Withers <rwi...@at...> wrote:
> gor...@bl... wrote:
> > Well, if you are interested in hacking on sqcvs - which is a CVS
> > pserver-client/server protocol implementation - then I should write down
> > an email for you with the most pressing todo-list. Most of those todos
> > are pretty damn easy to implement so I think it is a "rewarding point in
> > time" to jump in! :-)
> > 
> > Sqcvs is meant to be "plugged in" with VMMaker making it even easier to
> > build your own VM. Then of course - a nice Morphic UI on top of Sqcvs
> > wouldn't hurt - then we have our very own supercrossplatform CVS client!
> > :-)
> > 
> > One thing to start with would be to integrate it in FileList and
> > friends. That would perhaps be nice. Anyway - if you are still
> > interested then I can whip up an email to get you started. 
> > 
> > Regarding cvstproj - which is a different thing - you should talk to
> > Martin Kobetic. Currently it looks like Avi Bryant is building something
> > along the same lines as cvstproj (same goal, different approach) so you
> > may want to talk to him too.
> 
> Hey Goran,
> 
> I would welcome the email you mention writing, basically a roadmap of
> integrating cvs support into Squeak.  As sqcvs currently stands, we can

Sure.

> probably d/l the trunk to the file-system, then generate our code,
> right?  Otherwise we should get that working, first.  Following that,

That already works. I posted Tim a couple of lines of code that does
that and he said it worked fine. :-)
The only thing stopping us from having such a "button" in VMMaker is
that such a checkout doesn't create compatible CVS clientside "state"
files, so you can't use another CVS client on that workingcopy. Oops.
=Not very useful. That is one of the things on the todo.

> you mention other protocol features (commit, etc), then a UI evolving
> through increasing complexity.

Commit actually already works. It's basically add/remove that is missing
today.
Well, some other things of course...

> I am certainly up for helping you get the VMMakerTool using cvs, but I

Cool. I will write that roadmap.

> am not so sure about the rest.  It seems we have competing
> implementations, already.  I don't have a whole lot of time, and it has

Eh, the rest? Competing implementations?

> to be spread out, but I can help.
> 
> > "branched tipped with 3.2g-experimental-protected-memory?" - I am not
> > following your english here.
> 
> What english?  :)
> 
> > No problem - keep hammering, hopefully other people not versant in CVS
> > will learn something too! :-)
> 
> I think the important point to agree on is when to tag the main trunk
> with a root.  I feel this should be done at major release points, or
> experimental branches only.  I have certainly learned something!

Yes, the typical tagpoints for the trunk would be:
- When a release is made. Those tags should be called "yadayada-release"
or similar.
- When a branch is made. Those tags mark the root of the branch and
shoule be called "yadayada-root" or similar.
- When some other interesting point in time is needed to be marked -
typically before and after mergework. When you work with branches you
often want to tag before merging and when the merging is done. This is
mainly to facilitate further mergework in the future. But of course -
since we will mostly be merging in one direction from the branches into
the trunk such tags are mostly important on the branch.

regards, Göran

RE: [Squeak-VMdev] RE: Pending 3.2 and VM sources @ sourceforge

[Rob W. <rwi...@at...>]

"Lex Spoon" <le...@cc...> wrote:
> rwi...@at... wrote:
> > Heh.  This is what I did this morning for the changes, 
> > but I had to re-checkout the toplevel to add 
> > osExports.c.  Of course, I had done a 'cvs diff squeak', 
> > first, to verify that I was only commiting the files I 
> > had planned on commiting.
> > 
> 
> If I commit a whole directory, I cd to platforms/unix first.  Usually,
> though, I commit one file at a time, anyway.

That works and is safe.  Either the cvs diff or the cvs update will both
inform you which files are modified, so you can make sure you don't
commit anything accidentally.

BTW, did my unix commits look ok to you, for VMMaker32-7.5?

> 
> 
> "Andreas Raab" <And...@gm...> wrote:
> > There are a few points about this that I'm a bit unhappy with. One is
> > that it appears to me that working on a branch may be complicated. We
> > might argue that this is the price a person has to pay in order to get
> > write permission at SF but if so, we should all agree on it. Please
> > vote.
> 
> I'm not happy with it, either.  We are making this more complicated than
> it deserves.  Are we truly going to maintain old versions?  And which VM
> heads *truly* want to manage merging patches in?  I bet it's just one. 
> We all do realize, don't we, that it is very easy to revert changes that
> you don't like?
> 
> But the tides are moving.  We may as well do it that way I suppose, just
> to have a point of agreement.

I think we just tag the main trunk for the major releases.  This way we
can get back to that point for testing reported bugs, etc.  I pretty
much see everyone hacking in the trunk and this will cause ongoing
problems of inconsistent platform support for new features.  We have
been fighting this since day one of using SF with VMMaker, but the
response time is typically very short.   I don't know how we can resolve
this. 

Cheers,
Rob

RE: [Squeak-VMdev] RE: Pending 3.2 and VM sources @ sourceforge

[Rob W. <rwi...@at...>]

gor...@bl... wrote:
> Well, if you are interested in hacking on sqcvs - which is a CVS
> pserver-client/server protocol implementation - then I should write down
> an email for you with the most pressing todo-list. Most of those todos
> are pretty damn easy to implement so I think it is a "rewarding point in
> time" to jump in! :-)
> 
> Sqcvs is meant to be "plugged in" with VMMaker making it even easier to
> build your own VM. Then of course - a nice Morphic UI on top of Sqcvs
> wouldn't hurt - then we have our very own supercrossplatform CVS client!
> :-)
> 
> One thing to start with would be to integrate it in FileList and
> friends. That would perhaps be nice. Anyway - if you are still
> interested then I can whip up an email to get you started. 
> 
> Regarding cvstproj - which is a different thing - you should talk to
> Martin Kobetic. Currently it looks like Avi Bryant is building something
> along the same lines as cvstproj (same goal, different approach) so you
> may want to talk to him too.

Hey Goran,

I would welcome the email you mention writing, basically a roadmap of
integrating cvs support into Squeak.  As sqcvs currently stands, we can
probably d/l the trunk to the file-system, then generate our code,
right?  Otherwise we should get that working, first.  Following that,
you mention other protocol features (commit, etc), then a UI evolving
through increasing complexity.

I am certainly up for helping you get the VMMakerTool using cvs, but I
am not so sure about the rest.  It seems we have competing
implementations, already.  I don't have a whole lot of time, and it has
to be spread out, but I can help.

> "branched tipped with 3.2g-experimental-protected-memory?" - I am not
> following your english here.

What english?  :)

> No problem - keep hammering, hopefully other people not versant in CVS
> will learn something too! :-)

I think the important point to agree on is when to tag the main trunk
with a root.  I feel this should be done at major release points, or
experimental branches only.  I have certainly learned something!

Cheers,
Rob

RE: [Squeak-VMdev] RE: Pending 3.2 and VM sources @ sourceforge

[<gor...@bl...>]

"Lex Spoon" <le...@cc...> wrote:
[SNIP]
> "Andreas Raab" <And...@gm...> wrote:
> > There are a few points about this that I'm a bit unhappy with. One is
> > that it appears to me that working on a branch may be complicated. We
> > might argue that this is the price a person has to pay in order to get
> > write permission at SF but if so, we should all agree on it. Please
> > vote.
> 
> I'm not happy with it, either.  We are making this more complicated than
> it deserves.  Are we truly going to maintain old versions?  And which VM

Perhaps not. But if the situation arises then such a releasebranch is
good to have.
Note that we don't create the release branch UNTIL we really want to
commit fixes on it.

> heads *truly* want to manage merging patches in?  I bet it's just one. 

Well, at least two I think. Andreas and Ian! :-) But sure - if the
maintainers turns out to be a bottleneck then we will just have to fix
the working process. The good point here is that Ian and Andreas are "in
the game".

> We all do realize, don't we, that it is very easy to revert changes that
> you don't like?

I hope so! :-) You can even with some trickery erase "history" from the
RCS files. Well, that is not so easy - but doable.

> But the tides are moving.  We may as well do it that way I suppose, just
> to have a point of agreement.

Yes.

> -Lex

regards, Göran

PS. Time is up for my offer. Sorry guys! ;-) Well... Let's put it this
way - until all people signed up on SF have in some way vocalized a
"yes" for the model described then I won't begin writing the policy
document. I just don't want to spend time on something as boring as
documentation if it doesn't get used in the end... DS

RE: [Squeak-VMdev] RE: Pending 3.2 and VM sources @ sourceforge

[Lex S. <le...@cc...>]

rwi...@at... wrote:
> Heh.  This is what I did this morning for the changes, 
> but I had to re-checkout the toplevel to add 
> osExports.c.  Of course, I had done a 'cvs diff squeak', 
> first, to verify that I was only commiting the files I 
> had planned on commiting.
> 

If I commit a whole directory, I cd to platforms/unix first.  Usually,
though, I commit one file at a time, anyway.


"Andreas Raab" <And...@gm...> wrote:
> There are a few points about this that I'm a bit unhappy with. One is
> that it appears to me that working on a branch may be complicated. We
> might argue that this is the price a person has to pay in order to get
> write permission at SF but if so, we should all agree on it. Please
> vote.

I'm not happy with it, either.  We are making this more complicated than
it deserves.  Are we truly going to maintain old versions?  And which VM
heads *truly* want to manage merging patches in?  I bet it's just one. 
We all do realize, don't we, that it is very easy to revert changes that
you don't like?

But the tides are moving.  We may as well do it that way I suppose, just
to have a point of agreement.



-Lex

RE: [Squeak-VMdev] RE: Pending 3.2 and VM sources @ sourceforge

[<gor...@bl...>]

Rob Withers <rwi...@at...> wrote:
> gor...@bl... wrote:
> > rwi...@at... wrote:
> > > our process.  Do we want to complicate it further, 
> > > technically, by trying to tackle cvssupport?
> > 
> > Honestly, no! :-) But we can check it out later.
> 
> Whew.  That's good.  :)   My opinion is that we are much better off
> using a trust system and minimizing the technical barriers.  We all want
> to create the highest quality system we can.

I agree - I am just "playing along" since there are a few people worried
about these things.
But as you say - I also think that a trustsystem is quite enough - if we
can't trust each other then we shouldn't give each other commit
capability IMHO. :-)

> > > How far off are we to 3.2final?
> > 
> > No idea. ;-)
> 
> Me neither, but it feels like we are getting close.  What can I do to
> help cvsproj and sqcvs along?

Well, if you are interested in hacking on sqcvs - which is a CVS
pserver-client/server protocol implementation - then I should write down
an email for you with the most pressing todo-list. Most of those todos
are pretty damn easy to implement so I think it is a "rewarding point in
time" to jump in! :-)

Sqcvs is meant to be "plugged in" with VMMaker making it even easier to
build your own VM. Then of course - a nice Morphic UI on top of Sqcvs
wouldn't hurt - then we have our very own supercrossplatform CVS client!
:-)

One thing to start with would be to integrate it in FileList and
friends. That would perhaps be nice. Anyway - if you are still
interested then I can whip up an email to get you started. 

Regarding cvstproj - which is a different thing - you should talk to
Martin Kobetic. Currently it looks like Avi Bryant is building something
along the same lines as cvstproj (same goal, different approach) so you
may want to talk to him too.

> > Since you tagged those on the line above I assume you still have those
> > revisions :-) so the "-r 3.2g" is actually unnecessary (but correct).
> 
> Note that this has been a hypothetical; I didn't actually tag the unix
> stuff as I feel that that is Lex's or Ian's perogative.  

I know! I was just "pretending along". :-)

> There could be a lot of branches off of the trunk, which could get
> unweildy with all of the Root-of-XXX.  I like John's approach of using a
> <major>.<minor>.<patch>.0 (i.e. 3.2.7.0), then branches could be
> <major>.<minor>.<patch>.<branch>...(<subbranch>)  (i.e. 3.2.7.1).  I
> suppose we could even tag a particular revision of a branch as the root
> of a subbranch, like 3.2.7.1.0, and the subbranch would be 3.2.7.1.1 and
> so on.

I still want branchtags to have a special prefix/suffix like "-branch"
and the branch root tags to have another special prefix/suffix like
"-root" or "Rootof-" or whatever. A CVS repository quickly gets a LOT of
tags so you want to have a good naming standard for them and be able to
see what tags are branch tags etc. This is advice coming directly from
Karl Fogel's CVS book btw.

> > There is also the command "rtag" that can tag stuff without having a
> > workingcopy but I never use that - I feel in more control with "tag".
> 
> I agree, especially given your discussion of verifying with cvs update.
> 
> > > to checkout the branch:
> > > 
> > > cvs checkout -r 3.2g-experimental-protected-memory
> > 
> > Yes, this would check out the tip of the branch. This will place a so
> > called "sticky tag" on your files in your workingcopy so that all
> > operations from now on happen on the branch. Quite convenient. 
> 
> I see.  Now you won't be affecting the trunk at all.

Right.

> > You can
> > also move an existing checked out trunk onto the branch by using:
> > cvs update -r 3.2g-experimental-protected-memory
> 
> hmmm.  I don't follow you here.  What exactly does this do?  Oh.  Are

I am saying that an update operation with a revision option will update
your workingcopy to the state of the tag.
In my example I had checked out the trunk and wanted suddenly to work on
the branch - so I updated my trunk over to the branch by giving update
the branch tag as an argument. This will make CVS update/patch only
those files that are different in the branch. The end result is the same
as if I had done a checkout of the branch.

> you saying that if I have done
> 
> cvs co -r 3.2g

That would check out the trunk but as it was when tagged with "3.2g" -
which is not a branch tag. You are still on the trunk but with a sticky
tag "3.2g". The effect is that you can't commit anything because you can
not change "history"! But that is a sidepoint.

> then I could do 
> 
> cvs up -r 3.2g-experimental-protected-memory

Sure, this would move the workingcopy over to the tip/head of the branch
- since the branch tag always refer to the tip of the branch (only
changing the files that needs to be changed).

> and my working copy will now be the latest in the branched tipped with
> 3.2g-experimental-protected-memory?

"branched tipped with 3.2g-experimental-protected-memory?" - I am not
following your english here.
But I think the answer is Yes - your workingcopy is now the latest in
the branch called "3.2g-experimental-protected-memory".

> > This would have the same result as the checkout but with much less IO.
> > And moving it back to the trunk is done by clearing the sticky tag:
> > 
> > cvs update -A
> 
> Will this take the latest from the trunk, or the root?

Right. All sticky tags are cleared so CVS will then take the latest on
the trunk - can also be referred to as HEAD (a pseudo-tag representing
the head of the trunk).

> > My impression is that LinCVS is a poor clone of WinCVS/MacCVS (which are
> > pretty good by now). TkCVS looks interesting and has more advanced
> > features.
> 
> Very nice - I am d/ling tkcvs as I write this.
> 
> thanks and cheers,
> Rob

No problem - keep hammering, hopefully other people not versant in CVS
will learn something too! :-)

regards, Göran

RE: [Squeak-VMdev] RE: Pending 3.2 and VM sources @ sourceforge

[Rob W. <rwi...@at...>]

gor...@bl... wrote:
> rwi...@at... wrote:
> > our process.  Do we want to complicate it further, 
> > technically, by trying to tackle cvssupport?
> 
> Honestly, no! :-) But we can check it out later.

Whew.  That's good.  :)   My opinion is that we are much better off
using a trust system and minimizing the technical barriers.  We all want
to create the highest quality system we can.
 

> > How far off are we to 3.2final?
> 
> No idea. ;-)

Me neither, but it feels like we are getting close.  What can I do to
help cvsproj and sqcvs along?

> Since you tagged those on the line above I assume you still have those
> revisions :-) so the "-r 3.2g" is actually unnecessary (but correct).

Note that this has been a hypothetical; I didn't actually tag the unix
stuff as I feel that that is Lex's or Ian's perogative.  

There could be a lot of branches off of the trunk, which could get
unweildy with all of the Root-of-XXX.  I like John's approach of using a
<major>.<minor>.<patch>.0 (i.e. 3.2.7.0), then branches could be
<major>.<minor>.<patch>.<branch>...(<subbranch>)  (i.e. 3.2.7.1).  I
suppose we could even tag a particular revision of a branch as the root
of a subbranch, like 3.2.7.1.0, and the subbranch would be 3.2.7.1.1 and
so on.

> There is also the command "rtag" that can tag stuff without having a
> workingcopy but I never use that - I feel in more control with "tag".

I agree, especially given your discussion of verifying with cvs update.

> > to checkout the branch:
> > 
> > cvs checkout -r 3.2g-experimental-protected-memory
> 
> Yes, this would check out the tip of the branch. This will place a so
> called "sticky tag" on your files in your workingcopy so that all
> operations from now on happen on the branch. Quite convenient. 

I see.  Now you won't be affecting the trunk at all.

> You can
> also move an existing checked out trunk onto the branch by using:
> cvs update -r 3.2g-experimental-protected-memory

hmmm.  I don't follow you here.  What exactly does this do?  Oh.  Are
you saying that if I have done

cvs co -r 3.2g

then I could do 

cvs up -r 3.2g-experimental-protected-memory

and my working copy will now be the latest in the branched tipped with
3.2g-experimental-protected-memory?
  
> This would have the same result as the checkout but with much less IO.
> And moving it back to the trunk is done by clearing the sticky tag:
> 
> cvs update -A

Will this take the latest from the trunk, or the root?

> My impression is that LinCVS is a poor clone of WinCVS/MacCVS (which are
> pretty good by now). TkCVS looks interesting and has more advanced
> features.

Very nice - I am d/ling tkcvs as I write this.

thanks and cheers,
Rob