Re: [TCLCORE] Pain points as a distro maintainer

[Pietro C. <ga...@ga...>]

On Aug 12 2025, 17:01 +0000, apn...@ya... wrote:
>Pietro,
>
>Thanks for the detailed write-up. My thoughts below.
>
>1. versioning
>
>I think a lot of people would be happy (starting with me) if Tcl supported
>exactly two versions at any one time, though I'm not sure if that would mean
>the latest minor versions of two major versions, or the last two minor
>versions period (i.e. 8.6.17 + 9.1, say, or 9.0 + 9.1).

I think before asking that question, we should ask ourselves what 
meaning we impute to our major and minor versions. If it's similarly 
hard to migrate from 8.6 to 9.0 and from 9.0 to 9.1, then I don't see 
why we don't skip 9.1 and do 10.0 instead.

Put another way, can we *commit* to do breaking changes on major 
versions, and maintain two major versions where the previous-to-last 
only gets bug fixes?

>2. file provisioning. I learnt something today. Does this apply even if
>the >files installed by each package to the same location are in fact
>identical (the Windows installer for example maintains a reference
>count).

Interesting, first time I hear about that. We don't have that on 
FreeBSD.

>But in any case, doesn't the use of --prefix, --with-tcl, and friends,
>already allow you build and install Tcl as well as extensions into
>different paths? What other changes would you require here?

All ports install under the same prefix. I could install Tcl8 under 
/usr/local/tcl8/{lib,include,man,...}, but that goes against the best 
practives. It would still require at least different pkg-config files 
for tcl8 and tcl9, and tweaks to the loader to find shlibs in 
non-standard places.

I would like to know how other people deal with having many versions 
installed at the same time. For example, includes and manual pages are 
different between 8.6 and 9.0. Do people install whole trees under 
different prefixes?

>3. single-source extensions. Most extensions I know are already single
>source, thread and Tk being the exceptions. I think Tk is too closely
>coupled with Tcl and can stay as an exception. As for thread, I committed a
>branch thread9-for-tcl8 in the thread repository that builds (and passes the
>test suite for whatever that's worth) with both Tcl 8 and Tcl 9.

That's wonderful, thanks! I gave it a quick go and it behaves nicely.

>I'm not sure if that is too late to help you but in any case, I don't
>think you need to worry about needing a separate 8/9 ports for every
>extension. They may of course have to be installed in separate
>directories as most pkgIndex files
>are not written to choose the appropriate shared library to load.

We have a way to build the same port with different options, we call 
that concept "flavors". That would allow to build the same extension 
against Tcl8 and Tcl9 and install them both together. They would still 
require different set of installed files, though.
That was my plan for thread too, but it turns out that having different 
sources for different flavors goes against the grain a bit. I might be 
able to get back to that now :)

>4. tcltls - true. Unfortunately, the reality is pay work and real life 
>take priority. Hopefully this should be resolved soon.

Yep!

>
>With regards to the points you make in your Conclusion, I think everyone is
>painfully aware of the need for an updated site, a better distribution
>mechanism for extensions etc. and there are folks working on those aspects.
>But again, it will take some time.

I think we need direction and a common view of what we want to offer our 
users, more than we need time.

I argue that working on 9.1 when 9.0 is just released and the ecosystem 
hasn't caught up, is probably not the best way to spend the time.

And if it is, I fear I don't understand the reasons.

>The other point about more interaction /
>feedback from users, I have a somewhat contrary view. It seems to me no one
>(may be some exaggeration there) tests or tries out the alpha or even beta
>releases to provide feed back before the release and it's kind of too late
>afterwards.

I'm not on the core team so I don't know what happens there. But I read 
the mailing lists, most of the TIPs that are drafted (I admit I don't 
follow Tk that much), and I occasionally take a look at what's up on the 
fossil repos. My feeling is that I don't know what's going on. Sometimes 
dgp sends an email with a release schedule. I don't know where it comes 
from. I don't know why we're doing 9.1 now. I'm sure there is rationale, 
but it doesn't come across.

>Even presence at the bi-weekly or monthly meetups would be very
>helpful in determining the issues being faced and the priorities for future
>work.

Yes, being able to attend would be good. I'll see if I can make it some 
time.

Thanks for the time you took and for the writeup!

>
>/Ashok
>
>-----Original Message-----
>From: Pietro Cerutti via Tcl-Core <tcl...@li...>
>Sent: Monday, August 11, 2025 4:40 PM
>To: tcl...@li...
>Subject: [TCLCORE] Pain points as a distro maintainer
>
>Hi,
>
>I main a number of Tcl/Tk ports on FreeBSD, under the umbrella of the
>tcltk@ team: https://portscout.freebsd.org/tc...@fr....
>
>I would like to express my pain points and concerns when it comes to
>maintain Tcl/Tk as a downstream packager, particularly in light of the
>upcoming 9.1 release. I think I have expressed some (most?) of these
>concerns in the past, in a way or another, either on this ML and/or on
>IRC.
>
>What I say below is only partially specific to how the FreeBSD ports
>work.  I expect different downstreams to have intersecting but not
>identical sets of concerns.
>
>1. versioning
>
>Problem:
>We used to maintain flavours of Tcl/Tk for 8.4, 8.5, 8.6, 8.7, and 9.0.
>Right now we only have 8.6 and 9.0 in the tree. I'd like to keep it that
>way, i.e., one port per major version. We need to figure out sooner than
>later whether it's going to be a pain to upgrade from 9.0 to 9.1. Better
>yet: we need to make sure that it's *not* going to be a pain.
>Ideally, I'd like to have one port for tcl8 and one for tcl9. Ideally, I
>wouldn't have to care about minor versions.
>
>Proposal:
>I propose that we TIP specifics about our versioning schema, and we
>abide to the post-conditions we set forward. TIP 439 (Semantic
>Versioning) is marked as obsoleted. We should resurrect it. I think we
>should state that any 9.x is going to be source/binary compatible with
>9.0. Only backwards compatibile enhancements should be included.
>
>2. file provenance
>
>Problem:
>One of the problems that package managers solve for users is to answer
>the question: which package installs /usr/local/foo/bar? Each file must
>come from a unique 3rd party package. When two different packages
>install the same file, they are marked as conflicting, and the user
>cannot have them both installed at the same time.
>Tcl 8.x doesn't have disjoint sets of installation files for different
>values of x. The only versioned things are the shared library and tclsh.
>This is true also for 8.x and 9.x.
>We go through a lot of gymnics in FreeBSD to make sure people can
>install 8.6 and 9.0 together, e.g., by suffixing man pages with .tcl86
>and .tcl90, and by installing header files to ${prefix}/include/tcl${ver}.
>
>Proposal:
>This is linked to the previous item: if we can't seamlessy upgrade from
>one version to another, we need to make sure people can keep both at the
>same time. If we want that to be a painless process, we need to make
>sure the packages don't install overlapping files.
>I propose that we adopt a solution similar to what it's done on FreeBSD,
>where every files is installed under a path that contains the version in
>a way or another. Here's how we do it on FreeBSD, where %%TCL_VER%% is
>expanded to e.g., 8.6 and %%SHORT_TCL_VERSION%% to e.g., 86:
>https://cgit.freebsd.org/ports/tree/lang/tcl86/pkg-plist
>
>3. source-level compatibility between Tcl8 and Tcl9, Thread
>
>Problem:
>The Thread package now comes in two branches, 2.x for Tcl8 and 3.x for
>Tcl9. This was a major pain point and required me to split the previous
>tclthread port into two separate ports, tcl8-thread and tcl9-thread.
>I hope this won't be the case for many extensions.
>To me, not providing a Thread extension that's buildable with both Tcl8
>and Tcl9 is a very bad message to give to downstream and extension
>implementors.  TIP 486 suggested it, but it was never discussed / voted.
>
>Proposal:
>I fear the ship has sailed on this one, but it would be great if the 3.x
>branch could be made buildable with both Tcl8 and Tcl9.
>
>4. tcltls
>
>Problem:
>We now have 9.0 out of the door and work is already underways for 9.1.
>However, it's 2025 and we don't have a way to talk https, as the
>currently released tcltls extension doesn't build with Tcl9.  I know
>there's a tcltls 1.8 tagged, but not released; I am testing it right now
>and it seems to work fine for my use caess. I also see that work is
>being done (is it?) for 2.0.  But no matter what happens in the future,
>they're missing *now*, as 9.x is supposed to be adopted by users.
>
>Solution:
>Some packages are A-class citizens and get distributed together with
>Tcl: https://sourceforge.net/projects/tcl/files/Tcl/9.0.2/.
>I suggest that tcltls be upgraded to A-class.
>
>Conclusion
>
>I have the feeling that the people doing the work (and thanks for that!)
>are sometimes disconnected from the people who are supposed to integrate
>and use the result of the work.
>
>For example, to reiterate, I don't see how 9.1 can be a priority when
>9.0 can't talk on the web because of a missing TLS solution.
>
>I understand that this is opensource and people get to work on whatever
>pleases them. But I also think that if we don't want to become
>irrelevant on the landscape of programming languages, we should offer
>more of an integrated solution and less of individual pieces. For a
>parallel, in OCaml land, there's this concept of an OCaml platform from
>some years now: https://ocaml.org/platform. I think we could draw some
>inspriration.
>
>I ask the people putting in the hours and the efforts to try to connect
>with the users base and the downstream pacakgers to offer a viable,
>usable, and maintainable Tcl solution.
>
>I ask that big changes, especially those affecting users, be TIP'd and
>not just implemented, merged, and forgotten.
>
>I am willing to put hours in, if people agree on the general direction.
>
>I am happy to start multiple threads on the different subjects, too.
>
>Thanks,
>
>Pietro
>
>-- 
>Pietro Cerutti
>I have pledged to give 10% of income to effective charities
>and invite you to join me - https://givingwhatwecan.org
>
>
>_______________________________________________
>Tcl-Core mailing list
>Tcl...@li...
>https://lists.sourceforge.net/lists/listinfo/tcl-core
>

-- 
Pietro Cerutti
I have pledged to give 10% of income to effective charities
and invite you to join me - https://givingwhatwecan.org