Re: [TCLCORE] TCT/TIP reform

[Rolf A. <tcl...@po...>]

That's a lot of (well sorted and presented) thoughs and things at once.
Thanks for that!

One remark to your preamble: I don't think that all changes made to the
core which caused frictions have been resolved - most identified,
perhaps, some important indeed resolved, others are worked on or have to
be further discussed and then decided.

Als your subject already say this is about two things: a TCT reform and
a TIP reform. I guess the new or modified rules and processes for core
changes and their documentation will be finalized by TCT vote?

It seems natural then to first fill the TCT seats so that the new voices
are heared while sorting out the rules and processes.

rolf

Steve Landers writes:
> During the last few months there has been a lot of friction in the Tcl
> community resulting from concern about the direction of some changes
> being made to the core. Although these now look to have been resolved
> there was a lot of time, effort and emotional energy expended to get
> there. Not to mention the wasted effort on the TIPs that were
> replaced.  And there are a number of other issues simmering with
> respect to implementation and coding style as evidenced by ongoing
> chatter on the Tclers Chat and mailing list.
>
> I have been reflecting on the situation and wondering how we might
> reduce the friction and turn activity into achievement,  working from
> the assumption that all involved have the best interests of Tcl/Tk at
> heart.   In the process I have leaned on several Tclers whose wisdom
> and opinions I greatly value:  Ashok Nadkarni, Brian Griffin, Scott
> Goodwin and Kevin Walzer.  This document records my thoughts in the
> hope of stimulating a discussion and concrete proposals that lead to
> positive change.
>
> My conclusion is that the TCT/TIP process mostly works but needs improving. Let me explain why ...
>
> TIP #0 is the foundation of the TCT. Although it was written for a
> different time and different circumstances the objective remains the
> same: to stimulate progress within reasonable constraints and avoid
> doing harm.  It mostly works but the TYANNOTT policy (Two Yesses and
> No Nos or Two Thirds) breaks down at a time when there is a flood of
> TIPs and not enough active TCT members for adequate review. Also, TIP
> #0 talks about three stages of a proposed change - approval,
> implementation and integration - but only the first has been
> formalised.  And finally there is a broader issue,  the Tcl/Tk release
> cadence is glacially slow when compared with other Open Source
> projects, which means useful features may not get wide adoption for
> years, if not decades. That results in pressure to get proposed
> changes into a release when perhaps they are not ready.
>
> So this proposal addresses three broad areas in TIP #0 that need refining:
>
> • the need for broader participation in reviewing and voting on TIPs
> • the need for a more formal implementation and integration process once a TIP is approved
> • the need for an improved release cadence
>
> Reviewing and Voting on TIPs
>
> There are currently 13 members of the TCT of which three have not been
> active recently, which  leaves a small group of people available for
> reviewing proposals. Based on recent voting it is apparent that many
> of active members have struggled to keep up with the load of reviewing
> TIPs, myself included.  The result is that many recent TIPs have
> passed with just a few YES votes and (arguably) have not had adequate
> scrutiny.   This is worse if a particular developer is running ahead
> of the pack and identifying problems that need addressing.
>
> So what's the solution?  I've come to the conclusion that we need a lot more people on the TCT, to spread the load and get more eyes viewing proposals.
>
> But what sort of people?
>
> Tcl is more than the core Tcl language , it includes the APIs to
> access the core, the documentation and the distributions.  And
> currently we may have more core maintainers outside of the TCT than
> within, a situation that was not foreseen in TIP #0.   So (in my
> opinion) the TCT should not just include core developers but also need
> people with an interest in the stability, performance and direction of
> the core. That would include people who depend on the APIs such as
> extension maintainers, people who have experience in deploying
> applications based on Tcl, people who integrate Tcl (e.g. distribution
> maintainers), and people who maintain interfaces to Tcl and Tk from
> other languages. Complementary skills are needed: some people will be
> better at core coding, some at code review, others at API design,
> others at documentation and communication.
>
> And who should the people be?
>
> I suggest any long term active community member who is well respected
> and who is willing to serve should be considered.  Especially people
> who have to interface with Tcl (and live with the consequences of bad
> API design) and those who have deployed systems based on Tcl (so their
> knowledge is experiential and not just theoretical).  The more the
> merrier, in the hope that at least a few more people with different
> skills and interests will be voting on each TIP. Put differently (and
> perhaps ungraciously) this is an attempt to dilute the influence of
> any one individual and to harness the wisdom of the crowd.
>
> Sure some people can be "prickly" (can't we all at times?) but most
> long term members of the Tcl community have the best interest of Tcl
> at heart and have a critical mindset that will lead to potential
> problems being identified earlier and addressed in a way that is
> widely acceptable.  Hopefully the various skills and weaknesses will
> balance and we'll have more active involvement.
>
> One other consideration is we might choose to have Tcl and Tk subgroups, and inviting people from other language communities to get involved in the Tk language direction.
>
> I further suggest that TCT members should be expected to be involved
> in a vote or review (more on that later) at least once per year,
> otherwise they are marked as "inactive".  Not that there would any
> consequence to being inactive, since the TCT voting is 2/3 of those
> present not of active members.
>
> TIP Implementations
>
> The question of implementation needs clarification. Should an
> implementation be required before voting on a TIP, given that it
> commits the proposer to work without any guarantee of the work being
> used? Over the years it has been implied that this should be the case,
> but there is nothing in TIP #0 that says this - arguably the
> opposite.   This needs to be clarified.
>
> I can see both sides of the argument: on one hand why should someone
> invest time and energy with no indication of whether it will be
> used,  on the other hand why should a proposal be evaluated if there
> is no guarantee an implementation will be forthcoming.
>
> We could explicitly to allow TIPs to be voted on without an
> implementation (as happens now) but there is a risk this will be seen
> as a “Feature Request” and not a commitment to implement if approved.
>
> Another alternative is to require an implementation before a vote, but
> to add extra states to the TIP workflow to explicitly require a
> discussion phase before implementation.  This will give some
> indication to the TIP proposers as to whether a TIP has a good chance
> to succeed before they expend effort on an implementation.  Currently
> we have Draft, Voting, Accepted and Final (plus Rejected or Withdrawn
> or Obsolete).   What if we refined the TIP system to support Draft,
> Discussion, Implementation, Voting, Accepted and Final (i.e.
> integrated) plus Rejected/Withdrawn/Obsolete?  In other words require
> an implementation before a vote but only after the developer has
> “tested the waters”.  A larger TCT and more engagement from the
> community will make this more practical.
>
> Feature integration
>
> Integration is desperately in need of a more formal process:
>
> • there is no formal mechanism to ensure that code merged to trunk actually reflects the TIP or meets acceptable code quality standards
> • there have been recent instances of TIPs and implementations being changed during a discussion, making it difficult to assess the merits of a proposal
> • core devs are branching off different locations: some branch off 8.7
> and merge changes forward, others branch off 9.0 and merge backwards -
> making it more difficult to support parallel development and for
> reviewing changes
> • some branches contain multiple, intertwined changes so it is difficult to review individual changes
> • there have been instances of people re-writing other people’s code
> in amongst functional changes:   some were egregious - e.g. changing
> spacing - while others were more subtle, but the net effect is to make
> it difficult to identify and review functional changes
>
> My suggestion is that we formalise the integration process, along the lines of:
>
> • there should be one branch per TIP containing only changes pertaining to that TIP (for ease of review)
> • every feature branch must be based on the latest development branch (i.e. Tcl 9.0 at the moment) for consistency and ease of review
> • there should be a review and formal sign-off before by a TCT member before integration into the trunk
> • the branch must pass the test suite on key platforms before being submitted for review (it may be possible to automate this through the existing Continuous Integration facility)
>
> Looking at the process for approving a merge, it needn't be onerous.
> Even a single reviewer signing-off on a merge would be better than the
> current situation.   That means a single positive code review is
> sufficient but if someone objects then a 2/3 vote is needed to
> integrate.  Does this add complexity?  Definitely.  Would it avoid the
> "commit wars"?  Hopefully. Would it lead to a better core? Surely.
>
> Release procedures
>
> With more people involved and a commitment to more regular releases we
> could start addressing the load on the release manager.  This might
> include documentation of the steps involved in creating a release with
> a view to enlisting the help of others,  automation where possible and
> checklists to monitor progress.
>
> Conclusion
>
> Change is needed but it can be evolutionary rather than
> revolutionary.  With that change will come inevitable pressures to
> change policy and procedures but those can be dealt with in
> time.  However the current situation is wasting time and energy that
> could be better directed to improving Tcl, Tk and associated
> technologies.
>
> I intend to turn this proposal into a series of TIPs after a suitable period for feedback and discussion.
>
> Steve Landers
> May 2023
>
> _______________________________________________
> Tcl-Core mailing list
> Tcl...@pu...
> https://lists.sourceforge.net/lists/listinfo/tcl-core