Fink

On 12/28/06, mn <mnork0@...> wrote:
> Hi.
> I am using the 0.8.1 binary-install of fink, kept upgrading it to
> fink-0.26.0-31 via rsync/unstable/source.
> I managed to keep up with compiling most of the packages until a new
> version of kde-related stuff gets uploaded...
> Everything worked beautifully and just the latest kdelibs and
> dependencies were outdatded.
> Now the system keeps downgrading after every selfupdate run, stating
> in /sw/fink/VERSION that I am now on 0.7.0cvs again?

Nope, that's normal.  I believe that was the last version for which
that file indicated anything.

> "fink --version" still says it's 0.26.0-31, the compiled binaries
> still seem to work as expected.
> "fink list -o" now returns zero packages to upgrade, but "fink list"
> gives a list with seemingly all previously installed packs.
> fink install fink gives an error like (debian-rescue inspired) "sudo
> dpkg -i /sw/fink/debs/fink_0.26.0-31_darwin-powerpc.deb" does:
>
> "
> (Reading database ... 101343 files and directories currently installed.)
> Preparing to replace fink 0.26.0-31 (using .../fink_0.26.0-31_darwin-
> powerpc.deb) ...
> Unpacking replacement fink ...
> Setting up fink (0.26.0-31) ...
> Checking system... powerpc-apple-darwin8.8.0
> This system is supported and tested.
>
> Needs an update...
> cannot rename /sw/fink/dists to /sw/fink/10.2 at /sw/lib/fink/
> postinstall.pl line 124.
> /sw/bin/dpkg: error processing fink (--install):
> subprocess post-installation script returned error exit status 2
> Errors were encountered while processing:
> fink
> "
>
> 10.2 ?? I know I started from scratch with a 10.4-tree when 10.4-
> transitional went obsolete.
> the path in question indeed doesnt't exist on my system, and all
> files under /sw/fink/10.4 state what they should
> (0.8.1, rsync, VERSION, DISTRIBUTION)
>
>
> other attempts to install anything results in something like:
>
> "
> fink install sed
> Scanning package description files
> not a reference at /sw/lib/perl5/Fink/Services.pm line 1827
> "
>
>
> What went possibly wrong and how to recover from this mess?
> thanks & greetings
> mn
>
>
>

That is indeed strange.  Verify that the Distribution: line in
/sw/etc/fink.conf does in fact say "10.4".

-- 
Alexander K. Hansen
(akh)
Fink Documenter (still)

On 12/28/06, mn <mnork0@...> wrote:
> thx for answering so quickly.
>
> On 28.12.2006, at 18:57, Alexander Hansen wrote:
>
> > On 12/28/06, mn <mnork0@...> wrote:
> >>
> >> Now the system keeps downgrading after every selfupdate run, stating
> >> in /sw/fink/VERSION that I am now on 0.7.0cvs again?
> >
> > Nope, that's normal.  I believe that was the last version for which
> > that file indicated anything.
>
> If there is no technical reason for keeping it,
> may I suggest the removal of this confusing hd-space-eater?
>

n$ ls -l /sw/fink/VERSION
-rw-r--r-- 1 hansen admin 11 Mar 17  2004 /sw/fink/VERSION

It doesn't eat that much space. :-)

There may be a technical reason why it's still needed--I'm not sure.

> >> "fink --version" still says it's 0.26.0-31, the compiled binaries
> >> still seem to work as expected.
> >> "fink list -o" now returns zero packages to upgrade, but "fink list"
> >> gives a list with seemingly all previously installed packs.
> >> fink install fink gives an error like (debian-rescue inspired) "sudo
> >> dpkg -i /sw/fink/debs/fink_0.26.0-31_darwin-powerpc.deb" does:
> >>
> >> "
> >> (Reading database ... 101343 files and directories currently
> >> installed.)
> >> Preparing to replace fink 0.26.0-31 (using .../fink_0.26.0-31_darwin-
> >> powerpc.deb) ...
> >> Unpacking replacement fink ...
> >> Setting up fink (0.26.0-31) ...
> >> Checking system... powerpc-apple-darwin8.8.0
> >> This system is supported and tested.
> >>
> >> Needs an update...
> >> cannot rename /sw/fink/dists to /sw/fink/10.2 at /sw/lib/fink/
> >> postinstall.pl line 124.
> >> /sw/bin/dpkg: error processing fink (--install):
> >> subprocess post-installation script returned error exit status 2
> >> Errors were encountered while processing:
> >> fink
> >> "
> >>
> >> 10.2 ?? I know I started from scratch with a 10.4-tree when 10.4-
> >> transitional went obsolete.
> >> the path in question indeed doesnt't exist on my system, and all
> >> files under /sw/fink/10.4 state what they should
> >> (0.8.1, rsync, VERSION, DISTRIBUTION)
> >>
> >>
> >> other attempts to install anything results in something like:
> >>
> >> "
> >> fink install sed
> >> Scanning package description files
> >> not a reference at /sw/lib/perl5/Fink/Services.pm line 1827
> >> "
> >>
> >
> > That is indeed strange.  Verify that the Distribution: line in
> > /sw/etc/fink.conf does in fact say "10.4".
>
>
> Just double-checked. It does say 10.4.
> btw:
> "fink index" used to take quite some time in the past, finishes now
> as soon as I press return.

the fink package itself seems to be running correctly then.

> "fink scanpackages" seems to work, but "fink list -i" now just gives
> me installed virtual packages.
>
> greetings
> mike
>

Do "ls -l /sw/fink/dists" and make sure that it's pointing to the 10.4
tree, then.

-- 
Alexander K. Hansen
(akh)
Fink Documenter (still)

Just to revisit this topic again:

 >The issue is this: whenever we distribute a binary, we need to also 
 >make available the exact sources which were used to create that 
binary.  >My interpretation of this has been that we should supply not 
only the >upstream source, as found in our distfiles mirror system, but 
also the >fink info file and patch file.

Fink info and patch files are publically available from the CVS tree 
that is anonymously accessible.

 >So we would need a way of archiving info and source files, and being 
 >able to point someone to the precise info and source files which were 
 >used to create the binary.

The different versions of info and patch file are already archived in 
the CVS tree. I guess all you need to do is add a little TXT to the 
packaged binary which references the correct version number.

 >We would need a system like that (an automated one, obviously) before 
 >making any public distribution.

With that I do not agree, It does not state in the GPL (anywhere) that 
this data needs to be available in a coherent, automatic form. It only 
says, rather convoluted, that is had to be accessible. Which it 
definitely is.

But then again, you all know what I think about this rather 'precise' 
interpretation of GPL requirements.

Thanks

On Dec 18, 2006, at 7:37 AM, David H. wrote:

>
> But then again, you all know what I think about this rather  
> 'precise' interpretation of GPL requirements.
>

Right.

Just to carefully state my own view on this: I am proud that Fink has  
made efforts to comply with the spirit, and not just the letter, of  
the GPL, and I hope that we continue to do so.

   -- Dave

On Dec 18, 2006, at 10:28 AM, David R. Morrison wrote:
> So we would need a way of archiving info and source files, and  
> being able to point someone to the precise info and source files  
> which were used to create the binary.  We do this already for the  
> official bindists, by tagging the source and info files in CVS and  
> storing a copy of the tagged files in the appropriate directory  
> tree within the "Archive Browser" http://bindist.finkmirrors.net/ 
> bindist/dists/.

We can quite easily determine the exact CVS path and revision of  
the .info/.patch files used for any build. It wouldn't be a bad idea  
to stuff this information in every deb, maybe in %p/share/doc/%n/fink- 
info or some such. (Alternatively, it could go into the Description  
field.) We could also include a direct link to the ViewVC interface  
for the files in question, eg: http://fink.cvs.sourceforge.net/fink/ 
dists/10.4/unstable/main/finkinfo/base/fink.info? 
revision=1.18&view=markup

Would that be sufficient?

Dave

On Dec 18, 2006, at 10:23 AM, David H. wrote:
> I promised Simon that there would not be any excessive traffic.  
> What we can do is that we mirror built packages to finch ( another  
> sponsored machine) and we make them available from there. But that  
> would have to be done sparsely.

Yes, I wasn't suggesting that we directly download from the build  
farm, mirroring to finch is what I had in mind. May I ask what  
exactly is the point of syncing only "sparsely"? If we use rsync, it  
shouldn't really matter how often we sync--debs that haven't changed  
will be left alone.

Dave

The Main reason why that company has the 2Mbit line is for VoIP 
applications. So rather than blocking the line with little data packages 
constantly. I would think it is better to sync in large blocks during 
none business hours.

And the initial sync is less bothersome that way (to Amphora)

-d

On Dec 18, 2006, at 7:48 AM, Dave Vasilevsky wrote:

>
> On Dec 18, 2006, at 10:28 AM, David R. Morrison wrote:
>> So we would need a way of archiving info and source files, and  
>> being able to point someone to the precise info and source files  
>> which were used to create the binary.  We do this already for the  
>> official bindists, by tagging the source and info files in CVS and  
>> storing a copy of the tagged files in the appropriate directory  
>> tree within the "Archive Browser" http://bindist.finkmirrors.net/ 
>> bindist/dists/.
>
> We can quite easily determine the exact CVS path and revision of  
> the .info/.patch files used for any build. It wouldn't be a bad  
> idea to stuff this information in every deb, maybe in %p/share/doc/% 
> n/fink-info or some such. (Alternatively, it could go into the  
> Description field.) We could also include a direct link to the  
> ViewVC interface for the files in question, eg: http:// 
> fink.cvs.sourceforge.net/fink/dists/10.4/unstable/main/finkinfo/ 
> base/fink.info?revision=1.18&view=markup

That's a nice idea.  While we're at it, we could give URL's to the  
source files in the distfiles, so that there was a single place to  
look to find pointers to all the sources used to create the .deb.

But I have a question: in rsync updating, don't we throw away the CVS  
directories?  How shall we keep the CVS version data available to  
fink at deb-building time?

Or maybe we just make it a build-time option about whether to include  
this 'pointer-to-the-source' file or not... but that of course  
contravenes the long standing hope that a .deb will be the same no  
matter where it is built.

   -- Dave

On Dec 18, 2006, at 7:56 AM, David R. Morrison wrote:

>
> On Dec 18, 2006, at 7:48 AM, Dave Vasilevsky wrote:
>
>>
>> On Dec 18, 2006, at 10:28 AM, David R. Morrison wrote:
>>> So we would need a way of archiving info and source files, and  
>>> being able to point someone to the precise info and source files  
>>> which were used to create the binary.  We do this already for the  
>>> official bindists, by tagging the source and info files in CVS  
>>> and storing a copy of the tagged files in the appropriate  
>>> directory tree within the "Archive Browser" http:// 
>>> bindist.finkmirrors.net/bindist/dists/.
>>
>> We can quite easily determine the exact CVS path and revision of  
>> the .info/.patch files used for any build. It wouldn't be a bad  
>> idea to stuff this information in every deb, maybe in %p/share/doc/ 
>> %n/fink-info or some such. (Alternatively, it could go into the  
>> Description field.) We could also include a direct link to the  
>> ViewVC interface for the files in question, eg: http:// 
>> fink.cvs.sourceforge.net/fink/dists/10.4/unstable/main/finkinfo/ 
>> base/fink.info?revision=1.18&view=markup
>
> That's a nice idea.  While we're at it, we could give URL's to the  
> source files in the distfiles, so that there was a single place to  
> look to find pointers to all the sources used to create the .deb.
>
> But I have a question: in rsync updating, don't we throw away the  
> CVS directories?  How shall we keep the CVS version data available  
> to fink at deb-building time?
>
> Or maybe we just make it a build-time option about whether to  
> include this 'pointer-to-the-source' file or not... but that of  
> course contravenes the long standing hope that a .deb will be the  
> same no matter where it is built.
>
>   -- Dave
>

Actually, there is a much simpler solution: stuff the .info  
and .patch files themselves into the .deb (along with pointers to the  
distfiles copy of the source file(s)).  This would not increase .deb  
size much (except in the case of huge patches, which we discourage in  
any case) and is both reliable and easy to implement.

   -- Dave

I want to flesh out this idea from the past few emails into a formal  
proposal.

I propose creating %p/sw/share/doc/%n/BuildData/Readme in every .deb  
file, which would say one of the following (either A or B+B1 or B+B2):

A) The binary file for the %n package was built in a group with other  
packages.  Detailed build information, including information about  
the source files used, can be found in the %N package which was  
distributed together with this one.

B) The binary file for the %n package was built with fink-0.26.0 on  
18 December 2006 at 0800 on 10.4, using a core2duo processor (and  
whatever other diagnostic data we wish to include here... or should  
that go in a separate file?).  The fink build script and patch file 
(s) that were used are reproduced in the directory %p/share/doc/%n/ 
BuildData which contains this Readme file.

B.1) The build was based on the source file(s):
   foo.tgz
   bat.tar.bz2
If the binary file for this package was distributed by the Fink  
project, then those source file(s) can be obtained from
   http://bindist.finkmirrrors.net/1111111111111111111111/foo.tgz
   http://bindist.finkmirrrors.net/1111111111111111111111/bar.tar.bz2
If you obtained a binary file for this package from another source,  
please consult the distributor for instructions about obtaining the  
source file(s).

B.2) The build used source files which the Fink project does not have  
permission to redistribute.   Please consult the file %f.info in this  
directory for information about that file(s) which were used.
EOF

Also in the %p/share/doc/%n/BuildData directory would be copies of % 
f.info and any patch files, unless the package is a splitoff.

(For those who don't know, the definitive copy of a source file is  
stored in a directory whose name is the MD5-sum of the package; by  
using links of that form, we ensure that users are pointed to the  
actual version of the sourcefile which was used in compiling the .deb.)

Questions? Comments?

   -- Dave

On Mon, Dec 18, 2006 at 09:31:24AM -0800, David R. Morrison wrote:
> I want to flesh out this idea from the past few emails into a formal  
> proposal.
> 
> I propose creating %p/share/doc/%n/BuildData/Readme in every .deb file,

Alternative would be in %p/share/doc/finkinfo/%n/Readme (or %f). I've
seen a package or two where a SplitOff's doc/%n is a symlink to its
parent doc/%N, which would make doc/%{n,N}/BuildData collide. IIRC, it
may be essentially hopeless to change those, because dpkg has trouble
when a symlink is replaced by a real directory.

dan

-- 
Daniel Macks
dmacks@...
http://www.netspace.org/~dmacks

On Dec 18, 2006, at 10:01 AM, Daniel Macks wrote:

> On Mon, Dec 18, 2006 at 09:31:24AM -0800, David R. Morrison wrote:
>> I want to flesh out this idea from the past few emails into a formal
>> proposal.
>>
>> I propose creating %p/share/doc/%n/BuildData/Readme in every .deb  
>> file,
>
> Alternative would be in %p/share/doc/finkinfo/%n/Readme (or %f). I've
> seen a package or two where a SplitOff's doc/%n is a symlink to its
> parent doc/%N, which would make doc/%{n,N}/BuildData collide. IIRC, it
> may be essentially hopeless to change those, because dpkg has trouble
> when a symlink is replaced by a real directory.

Good point.  But then why not %p/share/finkinfo/%n, or %p/share/ 
BuildData/%n for the directory?  In any event, we are going to need  
to point users to where it is, with some documentation... not sure  
why it needs to be inside "doc" if its not going in the existing per- 
package doc/%n ...

   -- Dave

On Dec 18, 2006, at 10:12, Dave Vasilevsky wrote:

> I have limited knowledge of buildfink, Matt is really the only one  
> with the answers, so take this all with a grain of salt. Although  
> buildfink wasn't originally designed to run continuously, I'm  
> pretty sure we could run buildfink repeatedly in order to  
> continually update the set of built packages.

And if there's any functionality that it would be useful for me to  
add...

> 2. I suspect that repeated runs of buildfink will attempt to  
> rebuild every failing package, every run--even if the .info file  
> hasn't changed.

If you don't give it an explicit package list, it will attempt to  
rebuild every package, even ones in which the latest version was  
successfully built.  It shouldn't be too much work to write a script  
that sits on top of buildfink and makes the policy decision about  
what to try building for a given run.

> Well, we'd almost certainly want X11User/SDK installed. Aside from  
> that, just make sure /usr/local isn't present (buildfink already  
> checks this).

buildfink itself doesn't do this, the owbuild wrapper script I use  
for my runs does.  It also checks for X11.

On Dec 18, 2006, at 12:31, David R. Morrison wrote:

> B) The binary file for the %n package was built with fink-0.26.0 on
> 18 December 2006 at 0800 on 10.4, using a core2duo processor (and
> whatever other diagnostic data we wish to include here... or should
> that go in a separate file?).  The fink build script and patch file

Various software versions:
	*OS (sw_vers)
	*gcc (gcc -v)
	*cctools (ld -v)
I'd also recommend some basic hardware info.  At least the output of  
`arch`.  More detailed stuff is available from system_profiler -xml  
SPHardwareDataType.  The Xcode version would be nice too -- it can  
probably be inferred from the gcc version, but you can get it  
directly via system_profiler -xml SPApplicationsDataType.

Matthew Sachs wrote:
[]
> If you don't give it an explicit package list, it will attempt to  
> rebuild every package, even ones in which the latest version was  
> successfully built.  It shouldn't be too much work to write a script  
> that sits on top of buildfink and makes the policy decision about  
> what to try building for a given run.

If we are going to have a "live" bindist, that is one that is 
continually renewed, we have to take care of a problem with apt-get that 
has annoyed me for a long time (and I have mentioned it a couple of 
times, filed even a bug #1065267, closed by vasi "Assuming this is fixed"):

When deciding whether to update an installed package from the bindist, 
apt-get looks not only at %v-%r, but also at some home-made hash sum 
which involves among other things some sort of file size. Therefore it 
often reinstalls packages that are already installed with the exact same 
%v-%r, just because the bindist has changed.

We have to make sure this won't happen with the automatically rebuilt 
packages. Otherwise people will continually reinstall their already 
isntalled packages and won't be happy about it.

-- 
Martin

On Dec 19, 2006, at 1:58 AM, Martin Costabel wrote:
> When deciding whether to update an installed package from the bindist,
> apt-get looks not only at %v-%r, but also at some home-made hash sum
> which involves among other things some sort of file size. Therefore it
> often reinstalls packages that are already installed with the exact  
> same
> %v-%r, just because the bindist has changed.

As I mentioned last time this came up[1], this only happens if  
the .deb for the installed package is unknown to apt. This situation  
used to happen quite often because 'fink scanpackages' used to be  
extremely slow. When they built and installed a package, but hadn't  
yet run scanpackages, apt had no way of finding the .deb, and  
sillyness ensued. Fink 0.25 has much faster scanpackages, and with  
AutoScanpackages will automatically run it. This takes away the vast  
majority of cases where this bug showed up.[2]


To get back on topic, the upshot is that if we build a .deb in the  
bindist, and later remove it in favor of a rebuilt .deb, apt will go  
nuts. So the bindist build script should *not* replace existing .debs  
in the bindist. If for some strange reason we do want to rebuild  
packages which have already been built (same %v-%r), we should keep  
the old deb somewhere so that apt knows about it and doesn't go crazy.

Incidentally...has there been any progress on the new dpkg and apt  
which were being worked on awhile ago? I'm not really sure whether  
Debian considers this weird apt behavior a bug, but it might be  
changed in more recent versions.

Dave



1. http://www.mail-archive.com/fink-devel@...> 
msg12052.html

2. Remaining cases which will still result in apt re-installing things:
	- If you manually turn off AutoScanpackages
	- If you delete your local .debs
	- If you use multiple bindists, and turn different ones on and off

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dave Vasilevsky wrote:

> Incidentally...has there been any progress on the new dpkg and apt which
> were being worked on awhile ago? I'm not really sure whether Debian
> considers this weird apt behavior a bug, but it might be changed in more
> recent versions.

The updated dpkg I think is ready for public consumption.  The updated
apt acts like it does stuff, but magically never finds anything to
install.  :P


- --
Benjamin Reed a.k.a. Ranger Rick
Fink, KDE, and Mac OS X development
http://www.racoonfink.com/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFh+hxUu+jZtP2Zf4RAiuTAJwPS53RQXWB9iStqkRm0kQxXNwwLgCfZ4/c
7ohwdodoK6fzQDVu0InE2i0=
=e4xN
-----END PGP SIGNATURE-----

Daniel E. Macks wrote:
[]
> I just committed a patch for it to 10.3/unstable (last segment of
> base/dpkg.patch). Could some folks verify that it's good for 10.4
> also?

It works for me.

-- 
Martin