From: J. op d. B. <MSQ...@st...> - 2001-12-10 16:07:01
|
Hi all, is there no way HTDIG files are availably by FTP? If so, how do we make a copy of the files directory? If FTP is no longer available, I'll update the mirror-howto page accordingly. --jesse -------------------------------------------------------------------- J. op den Brouw Johanna Westerdijkplein 75 Haagse Hogeschool 2521 EN DEN HAAG Faculty of Engeneering Netherlands Electrical Engeneering +31 70 4458936 -------------------- J.E...@st... -------------------- Linux - because reboots are for hardware changes |
From: Gilles D. <gr...@sc...> - 2001-12-11 19:17:08
|
According to J. op den Brouw: > is there no way HTDIG files are availably by FTP? If so, how do > we make a copy of the files directory? If FTP is no longer > available, I'll update the mirror-howto page accordingly. No, from the SourceForge site, all we have right now is HTTP acces. FTP access has been completely cut off. You will need to use HTTP for mirroring. The files are at http://www.htdig.org/files/ -- Gilles R. Detillieux E-mail: <gr...@sc...> Spinal Cord Research Centre WWW: http://www.scrc.umanitoba.ca/~grdetil Dept. Physiology, U. of Manitoba Phone: (204)789-3766 Winnipeg, MB R3E 3J7 (Canada) Fax: (204)789-3930 |
From: J. op d. B. <ms...@st...> - 2001-12-11 21:00:16
|
Okay, I'm currently setting up a wget mirror for the "files" section. With a dozen or so commands you can make a nice copy of the files section. Instructions follow shortly. Isn't it better (for mirroring anyway) to put files section into a CVS repository? Updating your local copy would be very simple then. Gilles Detillieux wrote: > > No, from the SourceForge site, all we have right now is HTTP acces. > FTP access has been completely cut off. You will need to use HTTP > for mirroring. The files are at http://www.htdig.org/files/ --Jesse |
From: Gilles D. <gr...@sc...> - 2001-12-12 15:43:02
|
According to J. op den Brouw: > Okay, I'm currently setting up a wget mirror for the "files" section. > With a dozen or so commands you can make a nice copy of the files > section. Instructions follow shortly. > > Isn't it better (for mirroring anyway) to put files section into > a CVS repository? Updating your local copy would be very simple then. > > Gilles Detillieux wrote: > > > > No, from the SourceForge site, all we have right now is HTTP acces. > > FTP access has been completely cut off. You will need to use HTTP > > for mirroring. The files are at http://www.htdig.org/files/ That's a good question. On the old htdig.org site hosted by Verio, before the move to SourceForge, we used CVS for everything including the files section. I'm not too sure of the exact reasons for moving away from CVS for files on SourceForge. (We still use CVS for the source trees and maindocs, though.) I know there are some strange quirks with CVS on SourceForge, so maybe it caused problems with the large binary files in the files section. I've also heard that CVS doesn't deal with binary files all that efficiently (it's hard to "diff" them) so maybe it was a concern over the amount of repository space we'd use up. Geoff would probably know better than me, as he's the one who coordinated the switch to SourceForge. -- Gilles R. Detillieux E-mail: <gr...@sc...> Spinal Cord Research Centre WWW: http://www.scrc.umanitoba.ca/~grdetil Dept. Physiology, U. of Manitoba Phone: (204)789-3766 Winnipeg, MB R3E 3J7 (Canada) Fax: (204)789-3930 |
From: Geoff H. <ghu...@ws...> - 2001-12-14 06:01:19
|
At 9:42 AM -0600 12/12/01, Gilles Detillieux wrote: >I'm not too sure of the exact reasons for moving away from CVS for >files on SourceForge. It's something of a disk hog considering you rarely ever revise a file. When we moved, I basically felt it wasn't such a great idea to put them in CVS, but we could change if this is what people want. >Geoff would probably know better than me, as he's the one who coordinated >the switch to SourceForge. So far SourceForge hasn't put a quota on CVS (it's about the only thing they *haven't* quota'ed. So it's up to the group. If this is a good idea now that FTP has disappeared, sure. Keep in mind, I still find strange quirks with the CVS on SourceForge in terms of connectivity. Some of my cron jobs hang because the cvs command never hangs. (And I don't know how you'd set an alarm in a shell script.) -Geoff |
From: J. op d. B. <MSQ...@st...> - 2001-12-14 11:19:17
|
On Thu, 13 Dec 2001, Geoff Hutchison wrote: > It's something of a disk hog considering you rarely ever revise a > file. When we moved, I basically felt it wasn't such a great idea to > put them in CVS, but we could change if this is what people want. It's ideal for reproduction/mirroring. You can use CVS the same way as you use it for mirroring the maindocs. Therefor we don't need extra software, like mirror or wget. It would be nice if Joe's patch site would be available by CVS too.... But fair is fair, rsync would be the most preferred..... > So far SourceForge hasn't put a quota on CVS (it's about the only > thing they *haven't* quota'ed. So it's up to the group. If this is a > good idea now that FTP has disappeared, sure. > Keep in mind, I still find strange quirks with the CVS on SourceForge > in terms of connectivity. Some of my cron jobs hang because the cvs > command never hangs. (And I don't know how you'd set an alarm in a > shell script.) Is there a possibility for a trail period? I'm currently testing new ways for mirroring the htdig sites, since we have lost FTP access, and a combination of CSV and wget is getting some successfull results. Later more on that. --jesse -------------------------------------------------------------------- J. op den Brouw Johanna Westerdijkplein 75 Haagse Hogeschool 2521 EN DEN HAAG Faculty of Engeneering Netherlands Electrical Engeneering +31 70 4458936 -------------------- J.E...@st... -------------------- Linux - because reboots are for hardware changes |
From: Gilles D. <gr...@sc...> - 2001-12-14 19:10:08
|
According to J. op den Brouw: > On Thu, 13 Dec 2001, Geoff Hutchison wrote: > > It's something of a disk hog considering you rarely ever revise a > > file. When we moved, I basically felt it wasn't such a great idea to > > put them in CVS, but we could change if this is what people want. > > It's ideal for reproduction/mirroring. You can use CVS the same > way as you use it for mirroring the maindocs. Therefor we don't > need extra software, like mirror or wget. > > It would be nice if Joe's patch site would be available by CVS too.... > > But fair is fair, rsync would be the most preferred..... The drawback to using CVS is that everyone has to use it. You can't really have an alternate method of updating the files, otherwise the repository doesn't get updated. Another option open to htdig developers is ssh access and the scp command. Personally I like being able to use scp to copy in any contributed works. I find this more convenient than the old method of using CVS. Is there some way of setting up automatic updating/mirroring using ssh access? > > So far SourceForge hasn't put a quota on CVS (it's about the only > > thing they *haven't* quota'ed. So it's up to the group. If this is a > > good idea now that FTP has disappeared, sure. > > > > Keep in mind, I still find strange quirks with the CVS on SourceForge > > in terms of connectivity. Some of my cron jobs hang because the cvs > > command never hangs. (And I don't know how you'd set an alarm in a > > shell script.) > > Is there a possibility for a trail period? I'm currently testing new > ways for mirroring the htdig sites, since we have lost FTP access, > and a combination of CSV and wget is getting some successfull results. > Later more on that. Given the quirks with CVS on SF, I'd be a bit leary about using it, even for a trial period, because it's sort of an all or nothing proposition. Mind you, the maindocs updates have been going pretty well now, haven't they? -- Gilles R. Detillieux E-mail: <gr...@sc...> Spinal Cord Research Centre WWW: http://www.scrc.umanitoba.ca/~grdetil Dept. Physiology, U. of Manitoba Phone: (204)789-3766 Winnipeg, MB R3E 3J7 (Canada) Fax: (204)789-3930 |
From: J. op d. B. <ms...@st...> - 2001-12-16 12:23:34
|
Gilles Detillieux wrote: > The drawback to using CVS is that everyone has to use it. You can't > really have an alternate method of updating the files, otherwise the > repository doesn't get updated. Everyone already uses it. It's the best way to create a mirror for the maindocs. Mostly, if not all, mirrors are ran on 24/7 machines with a big internet pipe, and are managed on a routinely basis by profesionals, and they know CVS for sure and are using it. I don't think that there are 'personal' mirrors. Better still, is to use one program only for updating mirrors. That's why it would be nice if the patch site ran CVS (but it only has advantage if the 'files' can be mirrored with CVS). > Another option open to htdig developers is ssh access and the scp command. > Personally I like being able to use scp to copy in any contributed works. > I find this more convenient than the old method of using CVS. Is there > some way of setting up automatic updating/mirroring using ssh access? SSH/SCP can be used to copy, but I don't know if it can be used to update. I've used rsync over SSH for updating directories, and that works very well, so you'll need a third program to generate a list of files to be tranferred for updating the mirror. > Given the quirks with CVS on SF, I'd be a bit leary about using it, even > for a trial period, because it's sort of an all or nothing proposition. > Mind you, the maindocs updates have been going pretty well now, haven't > they? Ahhh, mirorring maindocs can go haywire if 'files' are added to the repository. Hmm, if that's so, let's not do it. --Jesse op den Brouw |
From: Geoff H. <ghu...@ws...> - 2001-12-17 04:03:53
|
At 1:25 PM +0100 12/16/01, J. op den Brouw wrote: > > Given the quirks with CVS on SF, I'd be a bit leary about using it, even >> for a trial period, because it's sort of an all or nothing proposition. >> Mind you, the maindocs updates have been going pretty well now, haven't >> they? > >Ahhh, mirorring maindocs can go haywire if 'files' are added to the >repository. >Hmm, if that's so, let's not do it. I'm just worried about complaints I've gotten from the SF sysadmins. The scripts I use to update the website itself seem to hang frequently--the CVS request never quits. This is obviously something I'd like to work out before we put new things in the repository! Of course if someone knows how you could do some sort of timeout/alarm in the update script, that would solve the problem. I could even stick the mirror scripts as examples on the website. -Geoff |
From: J. op d. B. <MSQ...@st...> - 2001-12-17 15:11:54
|
On Sun, 16 Dec 2001, Geoff Hutchison wrote: > Of course if someone knows how you could do some sort of > timeout/alarm in the update script, that would solve the problem. I > could even stick the mirror scripts as examples on the website. Something like: <command> & sleep <seconds> if <command> still on ps list, then kill it. Found this on a list: ---------------------------- command & sleep 20 if [ $(ps -p $! | wc -l ) -eq 2 ]; then echo "ERROR"; else echo "OK"; fi --------------------------- --jesse -------------------------------------------------------------------- J. op den Brouw Johanna Westerdijkplein 75 Haagse Hogeschool 2521 EN DEN HAAG Faculty of Engeneering Netherlands Electrical Engeneering +31 70 4458936 -------------------- J.E...@st... -------------------- Linux - because reboots are for hardware changes |