Re: [TCLCORE] TCT/TIP reform

[Steve L. <st...@di...>]

Hi Rolf,

Yes, I would see the TCT reform (i.e. expansion) as the first step.

-- Steve
On 12 May 2023 at 7:08 AM +0800, Rolf Ade <tcl...@po...>, wrote:
>
> That's a lot of (well sorted and presented) thoughs and things at once.
> Thanks for that!
>
> One remark to your preamble: I don't think that all changes made to the
> core which caused frictions have been resolved - most identified,
> perhaps, some important indeed resolved, others are worked on or have to
> be further discussed and then decided.
>
> Als your subject already say this is about two things: a TCT reform and
> a TIP reform. I guess the new or modified rules and processes for core
> changes and their documentation will be finalized by TCT vote?
>
> It seems natural then to first fill the TCT seats so that the new voices
> are heared while sorting out the rules and processes.
>
> rolf
>
> Steve Landers writes:
> > During the last few months there has been a lot of friction in the Tcl
> > community resulting from concern about the direction of some changes
> > being made to the core. Although these now look to have been resolved
> > there was a lot of time, effort and emotional energy expended to get
> > there. Not to mention the wasted effort on the TIPs that were
> > replaced.  And there are a number of other issues simmering with
> > respect to implementation and coding style as evidenced by ongoing
> > chatter on the Tclers Chat and mailing list.
> >
> > I have been reflecting on the situation and wondering how we might
> > reduce the friction and turn activity into achievement,  working from
> > the assumption that all involved have the best interests of Tcl/Tk at
> > heart.   In the process I have leaned on several Tclers whose wisdom
> > and opinions I greatly value:  Ashok Nadkarni, Brian Griffin, Scott
> > Goodwin and Kevin Walzer.  This document records my thoughts in the
> > hope of stimulating a discussion and concrete proposals that lead to
> > positive change.
> >
> > My conclusion is that the TCT/TIP process mostly works but needs improving. Let me explain why ...
> >
> > TIP #0 is the foundation of the TCT. Although it was written for a
> > different time and different circumstances the objective remains the
> > same: to stimulate progress within reasonable constraints and avoid
> > doing harm.  It mostly works but the TYANNOTT policy (Two Yesses and
> > No Nos or Two Thirds) breaks down at a time when there is a flood of
> > TIPs and not enough active TCT members for adequate review. Also, TIP
> > #0 talks about three stages of a proposed change - approval,
> > implementation and integration - but only the first has been
> > formalised.  And finally there is a broader issue,  the Tcl/Tk release
> > cadence is glacially slow when compared with other Open Source
> > projects, which means useful features may not get wide adoption for
> > years, if not decades. That results in pressure to get proposed
> > changes into a release when perhaps they are not ready.
> >
> > So this proposal addresses three broad areas in TIP #0 that need refining:
> >
> > • the need for broader participation in reviewing and voting on TIPs
> > • the need for a more formal implementation and integration process once a TIP is approved
> > • the need for an improved release cadence
> >
> > Reviewing and Voting on TIPs
> >
> > There are currently 13 members of the TCT of which three have not been
> > active recently, which  leaves a small group of people available for
> > reviewing proposals. Based on recent voting it is apparent that many
> > of active members have struggled to keep up with the load of reviewing
> > TIPs, myself included.  The result is that many recent TIPs have
> > passed with just a few YES votes and (arguably) have not had adequate
> > scrutiny.   This is worse if a particular developer is running ahead
> > of the pack and identifying problems that need addressing.
> >
> > So what's the solution?  I've come to the conclusion that we need a lot more people on the TCT, to spread the load and get more eyes viewing proposals.
> >
> > But what sort of people?
> >
> > Tcl is more than the core Tcl language , it includes the APIs to
> > access the core, the documentation and the distributions.  And
> > currently we may have more core maintainers outside of the TCT than
> > within, a situation that was not foreseen in TIP #0.   So (in my
> > opinion) the TCT should not just include core developers but also need
> > people with an interest in the stability, performance and direction of
> > the core. That would include people who depend on the APIs such as
> > extension maintainers, people who have experience in deploying
> > applications based on Tcl, people who integrate Tcl (e.g. distribution
> > maintainers), and people who maintain interfaces to Tcl and Tk from
> > other languages. Complementary skills are needed: some people will be
> > better at core coding, some at code review, others at API design,
> > others at documentation and communication.
> >
> > And who should the people be?
> >
> > I suggest any long term active community member who is well respected
> > and who is willing to serve should be considered.  Especially people
> > who have to interface with Tcl (and live with the consequences of bad
> > API design) and those who have deployed systems based on Tcl (so their
> > knowledge is experiential and not just theoretical).  The more the
> > merrier, in the hope that at least a few more people with different
> > skills and interests will be voting on each TIP. Put differently (and
> > perhaps ungraciously) this is an attempt to dilute the influence of
> > any one individual and to harness the wisdom of the crowd.
> >
> > Sure some people can be "prickly" (can't we all at times?) but most
> > long term members of the Tcl community have the best interest of Tcl
> > at heart and have a critical mindset that will lead to potential
> > problems being identified earlier and addressed in a way that is
> > widely acceptable.  Hopefully the various skills and weaknesses will
> > balance and we'll have more active involvement.
> >
> > One other consideration is we might choose to have Tcl and Tk subgroups, and inviting people from other language communities to get involved in the Tk language direction.
> >
> > I further suggest that TCT members should be expected to be involved
> > in a vote or review (more on that later) at least once per year,
> > otherwise they are marked as "inactive".  Not that there would any
> > consequence to being inactive, since the TCT voting is 2/3 of those
> > present not of active members.
> >
> > TIP Implementations
> >
> > The question of implementation needs clarification. Should an
> > implementation be required before voting on a TIP, given that it
> > commits the proposer to work without any guarantee of the work being
> > used? Over the years it has been implied that this should be the case,
> > but there is nothing in TIP #0 that says this - arguably the
> > opposite.   This needs to be clarified.
> >
> > I can see both sides of the argument: on one hand why should someone
> > invest time and energy with no indication of whether it will be
> > used,  on the other hand why should a proposal be evaluated if there
> > is no guarantee an implementation will be forthcoming.
> >
> > We could explicitly to allow TIPs to be voted on without an
> > implementation (as happens now) but there is a risk this will be seen
> > as a “Feature Request” and not a commitment to implement if approved.
> >
> > Another alternative is to require an implementation before a vote, but
> > to add extra states to the TIP workflow to explicitly require a
> > discussion phase before implementation.  This will give some
> > indication to the TIP proposers as to whether a TIP has a good chance
> > to succeed before they expend effort on an implementation.  Currently
> > we have Draft, Voting, Accepted and Final (plus Rejected or Withdrawn
> > or Obsolete).   What if we refined the TIP system to support Draft,
> > Discussion, Implementation, Voting, Accepted and Final (i.e.
> > integrated) plus Rejected/Withdrawn/Obsolete?  In other words require
> > an implementation before a vote but only after the developer has
> > “tested the waters”.  A larger TCT and more engagement from the
> > community will make this more practical.
> >
> > Feature integration
> >
> > Integration is desperately in need of a more formal process:
> >
> > • there is no formal mechanism to ensure that code merged to trunk actually reflects the TIP or meets acceptable code quality standards
> > • there have been recent instances of TIPs and implementations being changed during a discussion, making it difficult to assess the merits of a proposal
> > • core devs are branching off different locations: some branch off 8.7
> > and merge changes forward, others branch off 9.0 and merge backwards -
> > making it more difficult to support parallel development and for
> > reviewing changes
> > • some branches contain multiple, intertwined changes so it is difficult to review individual changes
> > • there have been instances of people re-writing other people’s code
> > in amongst functional changes:   some were egregious - e.g. changing
> > spacing - while others were more subtle, but the net effect is to make
> > it difficult to identify and review functional changes
> >
> > My suggestion is that we formalise the integration process, along the lines of:
> >
> > • there should be one branch per TIP containing only changes pertaining to that TIP (for ease of review)
> > • every feature branch must be based on the latest development branch (i.e. Tcl 9.0 at the moment) for consistency and ease of review
> > • there should be a review and formal sign-off before by a TCT member before integration into the trunk
> > • the branch must pass the test suite on key platforms before being submitted for review (it may be possible to automate this through the existing Continuous Integration facility)
> >
> > Looking at the process for approving a merge, it needn't be onerous.
> > Even a single reviewer signing-off on a merge would be better than the
> > current situation.   That means a single positive code review is
> > sufficient but if someone objects then a 2/3 vote is needed to
> > integrate.  Does this add complexity?  Definitely.  Would it avoid the
> > "commit wars"?  Hopefully. Would it lead to a better core? Surely.
> >
> > Release procedures
> >
> > With more people involved and a commitment to more regular releases we
> > could start addressing the load on the release manager.  This might
> > include documentation of the steps involved in creating a release with
> > a view to enlisting the help of others,  automation where possible and
> > checklists to monitor progress.
> >
> > Conclusion
> >
> > Change is needed but it can be evolutionary rather than
> > revolutionary.  With that change will come inevitable pressures to
> > change policy and procedures but those can be dealt with in
> > time.  However the current situation is wasting time and energy that
> > could be better directed to improving Tcl, Tk and associated
> > technologies.
> >
> > I intend to turn this proposal into a series of TIPs after a suitable period for feedback and discussion.
> >
> > Steve Landers
> > May 2023
> >
> > _______________________________________________
> > Tcl-Core mailing list
> > Tcl...@pu...
> > https://lists.sourceforge.net/lists/listinfo/tcl-core
>
>
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@li...
> https://lists.sourceforge.net/lists/listinfo/tcl-core