Thread: [A-A-P-develop] Caching of ftp connections
Brought to you by:
vimboss
From: Bram M. <Br...@mo...> - 2004-03-04 20:11:49
|
Jacob Lerner complained about ftp download not working with Python 1.5. After some debugging I narrowed it down to urlretrieve() not using a passive connection. I'm not 100% sure but it looks like using a passive connection solved the problem (obtaining the Vim runtime files). While I was changing the ftp code anyway, I implemented an item from the todo list: Keep an ftp connection open instead of re-opening it for every upload and download. This should speed up ftp transfer of small files considerably. However, the network code is complicated, I cannot oversee all the side effects of using ftplib instead of urllib. Please try out the new version. For now I only checked it into CVS. A zip archive is available here: http://www.a-a-p.org/files/aap-latest.zip I wonder if the setting for passive mode should be made available to the recipe writer. Googling I find a remark that it should be set automatically depending on the server, but I don't know how. -- [Another hideous roar.] BEDEVERE: That's it! ARTHUR: What? BEDEVERE: It's The Legendary Black Beast of Aaaaarrrrrrggghhh! "Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD /// Bram Moolenaar -- Bram@Moolenaar.net -- http://www.Moolenaar.net \\\ /// Sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\ Project leader for A-A-P -- http://www.A-A-P.org /// \\\ Buy at Amazon and help AIDS victims -- http://ICCF.nl/click1.html /// |
From: Yakov L. <ql...@ne...> - 2004-03-06 20:18:13
|
Bram Moolenaar wrote: > Jacob Lerner complained about ftp download not working with Python 1.5. > After some debugging I narrowed it down to urlretrieve() not using a > passive connection. I'm not 100% sure but it looks like using a passive > connection solved the problem (obtaining the Vim runtime files). > > While I was changing the ftp code anyway, I implemented an item from the > todo list: Keep an ftp connection open instead of re-opening it for > every upload and download. This should speed up ftp transfer of small > files considerably. > > However, the network code is complicated, I cannot oversee all the side > effects of using ftplib instead of urllib. Please try out the new > version. For now I only checked it into CVS. A zip archive is > available here: http://www.a-a-p.org/files/aap-latest.zip I tried aap-latest.zip with python 2.2.2 under cygwin. I got error: > aap -f ftp://ftp.vim.org/pub/vim/runtime/main.aap "main.aap" exists, overwrite? (y/n) y Aap: Attempting download of "ftp://ftp.vim.org/pub/vim/runtime/main.aap" Aap: Error in recipe: Cannot download "ftp://ftp.vim.org/pub/vim/runtime/main.aap" to "main.aap": [Errno 2] No such file or directory: '/home/Administrator/.netrc' I did 'touch ~/.netrc' and aap proceeded without errors. Is it possible to avoid errors if ~/.netrc is not present ? Jacob > > I wonder if the setting for passive mode should be made available to the > recipe writer. Googling I find a remark that it should be set > automatically depending on the server, but I don't know how. |
From: Bram M. <Br...@mo...> - 2004-03-07 12:15:09
|
Yakov Lerner wrote: > I tried aap-latest.zip with python 2.2.2 under cygwin. > I got error: > > aap -f ftp://ftp.vim.org/pub/vim/runtime/main.aap > "main.aap" exists, overwrite? (y/n) y > Aap: Attempting download of "ftp://ftp.vim.org/pub/vim/runtime/main.aap" > Aap: Error in recipe: Cannot download > "ftp://ftp.vim.org/pub/vim/runtime/main.aap" to "main.aap": > [Errno 2] No such file or directory: '/home/Administrator/.netrc' > > I did 'touch ~/.netrc' and aap proceeded without errors. Is it possible > to avoid errors if ~/.netrc is not present ? That was the intention. It appears this problem was already present, but it wasn't noticed, because the code was only used for uploading until now. The fix is simple: *** CopyMove.py~ Thu Mar 4 19:38:26 2004 --- CopyMove.py Sun Mar 7 12:56:15 2004 *************** *** 222,228 **** user = '' else: user, acct, passwd = res ! except aapnetrc.NetrcParseError, e: pass # Try to open the connection to the ftp server. --- 222,228 ---- user = '' else: user, acct, passwd = res ! except (IOError, aapnetrc.NetrcParseError), e: pass # Try to open the connection to the ftp server. -- "Software is like sex... it's better when it's free." -- Linus Torvalds, initiator of the free Linux OS Makes me wonder what FSF stands for...? /// Bram Moolenaar -- Bram@Moolenaar.net -- http://www.Moolenaar.net \\\ /// Sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\ Project leader for A-A-P -- http://www.A-A-P.org /// \\\ Buy at Amazon and help AIDS victims -- http://ICCF.nl/click1.html /// |
From: Yakov L. <ql...@ne...> - 2004-03-08 00:19:25
|
Bram Moolenaar wrote: > Jacob Lerner complained about ftp download not working with Python 1.5. > ... A zip archive is > available here: http://www.a-a-p.org/files/aap-latest.zip Thanks, it works now under python 1.5.2. Only it bombs out if ~/.netrc does not contain an entry for the server (the ftp.vim.org), with ugly output. The output is below. Jacob P.S. Output when ~/.netrc does not contain the entry for the server: # aap -f ftp://ftp.vim.org/pub/vim/runtime/main.aap "main.aap" exists, overwrite? (y/n) y Aap: Attempting download of "ftp://ftp.vim.org/pub/vim/runtime/main.aap" Aap: Internal Error Traceback (innermost last): File "/usr/local/lib/aap/Exec-1.058/Main.py", line 101, in do_the_work work = doread(find_recipe) File "/usr/local/lib/aap/Exec-1.058/DoRead.py", line 158, in doread {"name" : dest}, {"interactive" : 1}, 0, errmsg = 1) File "/usr/local/lib/aap/Exec-1.058/CopyMove.py", line 519, in remote_copy_move tmpfile, rtime = url_download(recdict, fname, destpath) File "/usr/local/lib/aap/Exec-1.058/Remote.py", line 203, in url_download ftp, msg = ftpConnect(fmach) File "/usr/local/lib/aap/Exec-1.058/CopyMove.py", line 220, in ftpConnect res = n.authenticators(machname) File "/usr/local/lib/aap/Exec-1.058/aapnetrc.py", line 92, in authenticators elif 'default' in self.hosts: TypeError: 'in' or 'not in' needs sequence right argument Aap: Traceback (innermost last): File "/usr/local/lib/aap/Exec-1.058/Main.py", line 101, in do_the_work work = doread(find_recipe) File "/usr/local/lib/aap/Exec-1.058/DoRead.py", line 158, in doread {"name" : dest}, {"interactive" : 1}, 0, errmsg = 1) File "/usr/local/lib/aap/Exec-1.058/CopyMove.py", line 519, in remote_copy_move tmpfile, rtime = url_download(recdict, fname, destpath) File "/usr/local/lib/aap/Exec-1.058/Remote.py", line 203, in url_download ftp, msg = ftpConnect(fmach) File "/usr/local/lib/aap/Exec-1.058/CopyMove.py", line 220, in ftpConnect res = n.authenticators(machname) File "/usr/local/lib/aap/Exec-1.058/aapnetrc.py", line 92, in authenticators elif 'default' in self.hosts: TypeError: 'in' or 'not in' needs sequence right argument |
From: Bram M. <Br...@mo...> - 2004-03-08 10:27:39
|
Yakov Lerner wrote: > Bram Moolenaar wrote: > > Jacob Lerner complained about ftp download not working with Python 1.5. > > ... A zip archive is > > available here: http://www.a-a-p.org/files/aap-latest.zip > > Thanks, it works now under python 1.5.2. I'm glad you kept on reporting the problems you encountered. > Only it bombs out if ~/.netrc does not contain an entry for the server > (the ftp.vim.org), with ugly output. The output is below. This patch should fix that: *** aapnetrc.py~ Thu Mar 4 17:15:21 2004 --- aapnetrc.py Mon Mar 8 10:46:13 2004 *************** *** 89,95 **** """Return a (user, account, password) tuple for given host.""" if host in self.hosts.keys(): return self.hosts[host] ! elif 'default' in self.hosts: return self.hosts['default'] else: return None --- 89,95 ---- """Return a (user, account, password) tuple for given host.""" if host in self.hosts.keys(): return self.hosts[host] ! elif 'default' in self.hosts.keys(): return self.hosts['default'] else: return None -- NEIL INNES PLAYED: THE FIRST SELF-DESTRUCTIVE MONK, ROBIN'S LEAST FAVORITE MINSTREL, THE PAGE CRUSHED BY A RABBIT, THE OWNER OF A DUCK "Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD /// Bram Moolenaar -- Bram@Moolenaar.net -- http://www.Moolenaar.net \\\ /// Sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\ Project leader for A-A-P -- http://www.A-A-P.org /// \\\ Buy at Amazon and help AIDS victims -- http://ICCF.nl/click1.html /// |