[TCLCORE] Pain points as a distro maintainer

[Pietro C. <ga...@ga...>]

Hi,

I main a number of Tcl/Tk ports on FreeBSD, under the umbrella of the 
tcltk@ team: https://portscout.freebsd.org/tc...@fr...ml.

I would like to express my pain points and concerns when it comes to 
maintain Tcl/Tk as a downstream packager, particularly in light of the 
upcoming 9.1 release. I think I have expressed some (most?) of these 
concerns in the past, in a way or another, either on this ML and/or on 
IRC.

What I say below is only partially specific to how the FreeBSD ports 
work.  I expect different downstreams to have intersecting but not 
identical sets of concerns.

1. versioning

Problem:
We used to maintain flavours of Tcl/Tk for 8.4, 8.5, 8.6, 8.7, and 9.0.  
Right now we only have 8.6 and 9.0 in the tree. I'd like to keep it that 
way, i.e., one port per major version. We need to figure out sooner than 
later whether it's going to be a pain to upgrade from 9.0 to 9.1. Better 
yet: we need to make sure that it's *not* going to be a pain.
Ideally, I'd like to have one port for tcl8 and one for tcl9. Ideally, I 
wouldn't have to care about minor versions.

Proposal:
I propose that we TIP specifics about our versioning schema, and we 
abide to the post-conditions we set forward. TIP 439 (Semantic 
Versioning) is marked as obsoleted. We should resurrect it. I think we 
should state that any 9.x is going to be source/binary compatible with 
9.0. Only backwards compatibile enhancements should be included.

2. file provenance

Problem:
One of the problems that package managers solve for users is to answer 
the question: which package installs /usr/local/foo/bar? Each file must 
come from a unique 3rd party package. When two different packages 
install the same file, they are marked as conflicting, and the user 
cannot have them both installed at the same time.
Tcl 8.x doesn't have disjoint sets of installation files for different 
values of x. The only versioned things are the shared library and tclsh.
This is true also for 8.x and 9.x.
We go through a lot of gymnics in FreeBSD to make sure people can 
install 8.6 and 9.0 together, e.g., by suffixing man pages with .tcl86 
and .tcl90, and by installing header files to ${prefix}/include/tcl${ver}.

Proposal:
This is linked to the previous item: if we can't seamlessy upgrade from 
one version to another, we need to make sure people can keep both at the 
same time. If we want that to be a painless process, we need to make 
sure the packages don't install overlapping files.
I propose that we adopt a solution similar to what it's done on FreeBSD, 
where every files is installed under a path that contains the version in 
a way or another. Here's how we do it on FreeBSD, where %%TCL_VER%% is 
expanded to e.g., 8.6 and %%SHORT_TCL_VERSION%% to e.g., 86:
https://cgit.freebsd.org/ports/tree/lang/tcl86/pkg-plist

3. source-level compatibility between Tcl8 and Tcl9, Thread

Problem:
The Thread package now comes in two branches, 2.x for Tcl8 and 3.x for 
Tcl9. This was a major pain point and required me to split the previous 
tclthread port into two separate ports, tcl8-thread and tcl9-thread.
I hope this won't be the case for many extensions.
To me, not providing a Thread extension that's buildable with both Tcl8 
and Tcl9 is a very bad message to give to downstream and extension 
implementors.  TIP 486 suggested it, but it was never discussed / voted.

Proposal:
I fear the ship has sailed on this one, but it would be great if the 3.x 
branch could be made buildable with both Tcl8 and Tcl9.

4. tcltls

Problem:
We now have 9.0 out of the door and work is already underways for 9.1.  
However, it's 2025 and we don't have a way to talk https, as the 
currently released tcltls extension doesn't build with Tcl9.  I know 
there's a tcltls 1.8 tagged, but not released; I am testing it right now 
and it seems to work fine for my use caess. I also see that work is 
being done (is it?) for 2.0.  But no matter what happens in the future, 
they're missing *now*, as 9.x is supposed to be adopted by users.

Solution:
Some packages are A-class citizens and get distributed together with 
Tcl: https://sourceforge.net/projects/tcl/files/Tcl/9.0.2/.
I suggest that tcltls be upgraded to A-class.

Conclusion

I have the feeling that the people doing the work (and thanks for that!) 
are sometimes disconnected from the people who are supposed to integrate 
and use the result of the work.

For example, to reiterate, I don't see how 9.1 can be a priority when 
9.0 can't talk on the web because of a missing TLS solution.

I understand that this is opensource and people get to work on whatever 
pleases them. But I also think that if we don't want to become 
irrelevant on the landscape of programming languages, we should offer 
more of an integrated solution and less of individual pieces. For a 
parallel, in OCaml land, there's this concept of an OCaml platform from 
some years now: https://ocaml.org/platform. I think we could draw some 
inspriration.

I ask the people putting in the hours and the efforts to try to connect 
with the users base and the downstream pacakgers to offer a viable, 
usable, and maintainable Tcl solution.

I ask that big changes, especially those affecting users, be TIP'd and 
not just implemented, merged, and forgotten.

I am willing to put hours in, if people agree on the general direction.

I am happy to start multiple threads on the different subjects, too.

Thanks,

Pietro

-- 
Pietro Cerutti
I have pledged to give 10% of income to effective charities
and invite you to join me - https://givingwhatwecan.org