Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo

Close

#3365 package provide/package ifneeded mismatch errors

obsolete: 8.5a3
closed-out-of-date
Don Porter
5
2008-02-23
2006-02-07
Joe English
No

After the fix for #1162286, it is now an error if a
[package ifneeded] script for a particular version of a
package ends up providing a different version of that
package. Please consider making this a warning
instead, like it is in 8.4.12.

There are a lot of existing packages that make this
mistake, since historically it has not been an error.
Turning it into one makes it harder for end-users and
administrators to upgrade to 8.5, since they need to
get all the packages fixed first.

The current situation penalizes the wrong group of
people. Package authors should be nagged to fix their
packages, certainly, but end users shouldn't be left
stranded in the meantime.

Discussion

  • Don Porter
    Don Porter
    2006-02-08

    Logged In: YES
    user_id=80530

    It's a fair observation, and it's
    worth considering softening the
    penalty as Tcl 8.5 gets to beta
    and/or stable releases.

    During alpha development, though,
    I think an actual error makes for
    a more effective "nag the authors
    of broken packages" function.

     
  • Twylite
    Twylite
    2007-04-18

    Logged In: YES
    user_id=91629
    Originator: NO

    I'm going to add a curve ball into this problem, since it is making my migration to 8.5 very unhappy.

    I have used (possibly abused) the ability of "package ifneeded" to take only a major number, than then have the sourced script provide the latest version of the package havingthat major number.

    e.g:
    package ifneeded myutils 1 [list source [file join $dir myutils1.tcl]]
    package ifneeded myutils 2 [list source [file join $dir myutils2.tcl]]

    This allows me to use my version control system to track the minor versions of packages for me, by doing the following:

    package provide myutils "1.[lindex {$Revision: 1 $} 1]"

    (That's for Visual SourceSafe; CVS should be similar)

    The same technique allows you to specify major & minor numbers with ifneeded, and allow the SCM to handle the patchlevel for you.

    My point: if CompareVersions is used in Tcl_PkgRequireProc in the current manner, it is not possible to let the SCM handle any aspect of version numbering for you; you must do it manually. Which is a pain, IMHO.

    I consider this ability of the 8.4 ifneeded/require/provide system to be a feature.

    My suggestion: if the provided version is compatible with the ifneeded version (the ifneeded version is effectively the prefix of the provided version) then the package require should succeed (no warnings or errors). If there is any conflict between the provided and ifneeded versions then raise an error.

    e.g.
    package ifneeded myutils 1 [list source [file join $dir myutils2.tcl]]
    package require myutils
    --> ERROR (ifneeded 1, provided 2.1)

    package ifneeded myutils 3 [list source [file join $dir myutils3.tcl]]
    package require myutils
    --> 3.4

    This would make the version matching logic between ifneeded and provided identical to the requirements satisfied logic for a -exact requirement (if my understanding of the latter is correct).

    This behaviour is also consistent with "package present". i.e. "package present myutils 1" will happily return "1.2" if that is the actual version of the package that is loaded.

    Required change to do this:

    Replace line 520:
    res = CompareVersions(pvi, vi, NULL);
    With:
    Tcl_Obj *ov;
    ov = ExactRequirement(versionToProvide);
    res = ! AllRequirementsSatisfied(pvi, 1, &ov);

    A cleaner version of this change is given in the patch below (against the latest version in CVS):

    Index: tclPkg.c

    RCS file: /cvsroot/tcl/tcl/generic/tclPkg.c,v
    retrieving revision 1.26
    diff -u -r1.26 tclPkg.c
    --- tclPkg.c 5 Dec 2006 15:36:12 -0000 1.26
    +++ tclPkg.c 18 Apr 2007 06:48:46 -0000
    @@ -504,7 +504,6 @@
    " provided", NULL);
    } else {
    char *pvi, *vi;
    - int res;

    if (CheckVersionAndConvert(interp, pkgPtr->version, &pvi,
    NULL) != TCL_OK) {
    @@ -514,11 +513,14 @@
    ckfree(pvi);
    code = TCL_ERROR;
    } else {
    - res = CompareVersions(pvi, vi, NULL);
    + Tcl_Obj *ov;
    + ov = ExactRequirement(versionToProvide);
    + satisfies = AllRequirementsSatisfied(pvi, 1, &ov);
    +
    ckfree(pvi);
    ckfree(vi);

    - if (res != 0) {
    + if (!satisfies) {
    code = TCL_ERROR;
    Tcl_AppendResult(interp,
    "attempt to provide package ", name, " ",

     
  • Twylite
    Twylite
    2007-04-18

    Logged In: YES
    user_id=91629
    Originator: NO

    The change will break test pkg-2.34.

    Interestingly, although the test fails, the test summary for all.tcl gives Failed 0 !?

     
  • Don Porter
    Don Porter
    2007-04-24

    Logged In: YES
    user_id=80530
    Originator: NO

    I'm sorry that you've fallen into
    those habits, but they're simply
    not compatible with the package
    selection logic used by
    [package require].

    When you do

    package ifneeded p 1 {package provide p 1.1}

    and Tcl doesn't detect and prevent the error,
    then several things get broken:

    [package require p 1.1] will fail, even
    though version 1.1 is available.

    [package require -exact p 1] will "succeed"
    but it won't deliver the exact version
    requested.

    Tcl's tolerated this brokenness for a long time,
    and there will be some pain in migrating the
    broken packages. It may be that we'll have
    to soften the errors to warnings in order to
    ease the transition, and I'll accept that if
    it turns out to be necessary. However, the
    release of a new minor version of Tcl is exactly
    the right time to at least attempt to work
    through the pain.

     
  • Twylite
    Twylite
    2007-04-25

    Logged In: YES
    user_id=91629
    Originator: NO

    Then the selection logic is broken.

    The package manager is a straightforward dynamic linker, which means it must do two things:

    (a) If I ask for "version 1", it must give me the most appropriate version 1.x that it has. Tcl does this.

    (b) But if I say that I am providing "version 1", then it must use that as a last-resort fallback if it doesn't have any more specific information to help it satisfy a request for some version 1.x.

    > [package require p 1.1] will fail, even though version 1.1 is available.

    See (b) above -- it should fall back to using the "ifneeded p 1" statement, which is the best information it has.

    > [package require -exact p 1] will "succeed" but it won't deliver the exact version requested.

    This is a consequence of the implementation of -exact. There is no reason why it shouldn't be implemented so that it fails. Similarly a [package require -exact p 1.1] should fall back to the "ifneeded p 1" statement, get a "provide p 1.1", and subsequently succeed.

     
  • Twylite
    Twylite
    2007-04-25

    Logged In: YES
    user_id=91629
    Originator: NO

    Also, the Tcl 8.5 documentation has the following to say about -exact:

    ---
    package require -exact package version
    This form of the command is translated to the form below using the bounded requirement "version-(version+1)", making only the given version acceptable, within the specified level of detail. Deeper levels are allowed to vary. Examples:

    -exact 8 => 8-9
    -exact 8.4 => 8.4-8.5
    -exact 8.4.14 => 8.4.14-8.4.15
    ---

    > [package require -exact p 1] will "succeed" but it won't deliver the exact version requested.

    According to the docs, that is supposed to succeed.

    If you didn't have -exact then [package require p 1] could provide v2.x instead.

    What won't succeed is [package require -exact p 1.1], because the selection logic can't find something suitable to load for version 1.1.

     
  • Don Porter
    Don Porter
    2007-04-25

    Logged In: YES
    user_id=80530
    Originator: NO

    Thanks for pointing that out.

    Someone has documented a bug.
    That should not have happened.
    See 1578344. We're deep in
    other bug fixing now, but I'll
    try to get back to this and
    get it resolved properly.

     
  • Don Porter
    Don Porter
    2007-04-25

    Logged In: YES
    user_id=80530
    Originator: NO

    You say

    "(a) If I ask for "version 1", it must give me the most appropriate version
    1.x that it has. Tcl does this."

    This is another problem with the revised
    scheme you are attempting to use. Tcl
    can only accomplish selection of the
    "most appropriate" version if each available
    version is accurately labeled with the real
    version it provides.

    With versions 1.0, 1.1, 1.2 installed properly
    with these index scripts:

    package ifneeded p 1.0 {package provide p 1.0}

    package ifneeded p 1.1 {package provide p 1.1}

    package ifneeded p 1.2 {package provide p 1.2}

    Then the [package] command has the data it needs
    so you can know that [package require p 1] will
    actually load p 1.2, the most appropriate one.

    If you atttempt your scheme:

    package ifneeded p 1 {package provide p 1.0}

    package ifneeded p 1 {package provide p 1.1}

    package ifneeded p 1 {package provide p 1.2}

    Then it's basically undefined what version
    will be loaded in reaction to [package require p 1].

    It's a feature of [package require] to select
    the latest compatible version. Your scheme
    interferes with the workings of that feature.

     
  • Twylite
    Twylite
    2007-05-03

    Logged In: YES
    user_id=91629
    Originator: NO

    Okay, I have come up with an alternative proposal that may cover both views (dgp & mine).

    Enhance the "package provide" to accept a version as a list of two elements. The first element is the version that the package manager cares about, and the second is additional sub-version information (one could call it a patchlevel) that is stored by the package manager but not used in decision making.

    e.g. I want to version my packages at minor level for the purposes of management, but for diagnostics I want to be able to report the exact patchlevel or "build" of code that is running.

    In Tcl 8.4.x I could do this:
    package ifneeded p 1.1 { package provide p 1.1.7 }

    In Tcl 8.5a currently I must do this:
    package ifneeded p 1.1 { package provide p 1.1 ; set ::my_package_info(p) 1.1.7 }

    Note that I now need a custom work-around in 8.5 for something Tcl did for me in 8.4.

    Proposal:
    package ifneeded p 1.1 { package provide p {1.1 .7} }

    The "full" version of the package is [join $p_ver ""], i.e. 1.1.7

    This proposal is fully backwards compatible, and draws a clear line between version information that is seen by the package manager, and info that is not.

    Additionally:

    a) The "full" version would probably be returned by "package present p".

    b) I propose to extend "package present" to be usable without arguments, in which case it will return a list of name/full_version pairs (an alternative to looping over [package names] doing a [package present $name])

    c) I think this proposal can help to clarify/solve the ideal behaviour of -exact. i.e. -exact will do an exact (effectively a string identity) match on the part of the version that the package manager takes decisions on (i.e. [lindex $pkg_ver 0]), and ignore sub-version information.

     
  • Joe English
    Joe English
    2008-02-23

    • status: open --> closed-out-of-date
     
  • Joe English
    Joe English
    2008-02-23

    Logged In: YES
    user_id=68433
    Originator: YES

    Withdrawing this request, as 8.5 has been released.