Menu

#92 About the --enable-http-keep-alive option

closed
nobody
None
5
2012-09-28
2008-06-02
Anonymous
No

Hi, Mr. Tujikawa.

I often need to download large number (more than 1000) of small (<100KB) files from a certain server using the parameterized URI feature of aria2. Because this server is not very stable, I can only establish a connection with it about 30% of the times, but once the connection is established, I can always download exactly 101 files from it without problem.

For example, if I use curl to download files [0001-1000].pdf from this server, I often need several retries to begin download the first file, but once the downloading of 0001.pdf is started, curl will download the first 101 files with no problem, then it will often get a "can't connect to host" error when downloading the 102nd file. I think this happens because when download from this certain server, curl reuse the http connection for exactly 101 files.

Now I'm using aria2 for all my downloads, and it downloads way faster than curl. But I found the "reuse http connection" behavior of aria2 is different from curl's. When I using aria2 with the options "--enable-http-keep-alive=true -m0" to download files [0001-1000].pdf from this server, I often get timeout errors after the successful download of several (<10) files instead of 101 files, and the #cuid keeps changing. So I guess aria2 only reused the connection for several files. It would be great if the --enable-http-keep-alive feature of aria2 can reuse the http connection for a larger number of files (101 files in this case), thank you very much.

p.s. This is not a feature request, but could you tell me how to uninstall the older version of aria2 if a newer version is available in the future. I "make install" it using the tar.bz2 file downloaded form sourceforge, thanks.

Discussion

  • tujikawa

    tujikawa - 2008-06-02

    Logged In: YES
    user_id=1450148
    Originator: NO

    If you're not using the latest release of aria2, socket reuse doesn't work between files.
    Could you try 0.13.2+1 release?
    I've rewritten http-keep-alive support in 0.13.2 and it behaves in a way you want.
    BTW, since 0.13.2 release, --http-keep-alive is enabled by default.

    For uinstall, if you build aria2 from tar ball and installed using 'make install', then build new one and do 'make install'. This will overwrite existing files and that's ok.
    If your aria2 is installed via package system such as deb, or rpm, then you need to use a package system command;
    for example in debian:
    dpkg -r aria2

     
  • Nobody/Anonymous

    Logged In: NO

    Hi, Mr. Tujikawa.

    Thanks for the reply, I was using 0.13.1+2 when I wrote the first post, the lastest version works great.

     
  • tujikawa

    tujikawa - 2008-06-03

    Logged In: YES
    user_id=1450148
    Originator: NO

    I'm grad to hear that:) Let's close the issue.

     

Log in to post a comment.

MongoDB Logo MongoDB