#24042 Resuming a finished http download wrongly return `HTTP 302`+`HTTP 200´ instead of the expected `HTTP 416`

[Saulius Krasuckas]

I believe this is generic http server issue (not related to the ReactOS project).

If I start downloading the ReactOS-0.4.14-live.zip file using curl and then cancel it (or truncate the full download), I can resume the download following way:

$ curl -Lo ReactOS-0.4.14-live.zip -C - 'https://sourceforge.net/projects/reactos/files/ReactOS/0.4.14/ReactOS-0.4.14-live.zip/download'; echo "Exit code: $?"
** Resuming transfer from byte position 78643200
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1012    0  1012    0     0   2784      0 --:--:-- --:--:-- --:--:--  2803
100   369  100   369    0     0    215      0  0:00:01  0:00:01 --:--:--   282
100 5004k  100 5004k    0     0  1354k      0  0:00:03  0:00:03 --:--:-- 2789k
Exit code: 0

But if the download is already complete, the command fails because of the server response:

$ curl -Lo ReactOS-0.4.14-live.zip -C - 'https://sourceforge.net/projects/reactos/files/ReactOS/0.4.14/ReactOS-0.4.14-live.zip/download'; echo "Exit code: $?"
** Resuming transfer from byte position 83767364
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1008    0  1008    0     0   2339      0 --:--:-- --:--:-- --:--:--  2344
100   365  100   365    0     0    261      0  0:00:01  0:00:01 --:--:-- 73000
  0   154    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
  0  1027    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0
curl: (33) HTTP server doesn't seem to support byte ranges. Cannot resume.
Exit code: 33

Using -v switch I noticed that several SF mirrors (served both by Apache and NGINX) in a row return HTTP 302 + wrong HTTP 200 to me with the latter response containing an HTML telling about some mirroring error.

I get this instead of the HTTP 416 which is expected way to say that requested range is not satisfiable. Eg. some Japanese server serving the same file:

$ curl -vLo ReactOS-0.4.14-live.zip -C - 'http://www.cs.tohoku-gakuin.ac.jp/pub/Linux/ReactOS/ReactOS-0.4.14-live.zip'; echo "Exit code: $?"
** Resuming transfer from byte position 83767364
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 157.118.89.2:80...

* Connected to www.cs.tohoku-gakuin.ac.jp (157.118.89.2) port 80 (#0)
> GET /pub/Linux/ReactOS/ReactOS-0.4.14-live.zip HTTP/1.1
> Host: www.cs.tohoku-gakuin.ac.jp
> Range: bytes=83767364-
> User-Agent: curl/7.86.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 416 Requested Range Not Satisfiable
< Content-Type: text/html
< Content-Length: 385
< Date: Tue, 15 Nov 2022 19:23:00 GMT
< Server: Apache 1.3.29
<
{ [385 bytes data]
100   385  100   385    0     0    605      0 --:--:-- --:--:-- --:--:--   607
* Connection #0 to host www.cs.tohoku-gakuin.ac.jp left intact
Exit code: 0

My guess is that configuration of http mirroring server needs some fine tuning.

Can this be fixed, please:)?
It would make the bootstrapping script resistant to download interruptions, because the same curl command would finish the download when repeated.

[Saulius Krasuckas]

Regarding my last statement:

It would make the bootstrapping script resistant to download interruptions, because the same curl command would finish the download when repeated.

`
I mean returning the HTTP 416 for resumes of completed files would make the same curl -C - ... to return success in every three cases:

• when download of the file hadn't been started yes and it finishes.
• when the file to download is incomplete and the download finishes.
• when the file to download is already complete.

`
Now the 3rd case returns exit code 33 which breaks shell scripts containing set -e.

[SourceForge Support]

Hi,

Thank you for letting us know.
I have escalated this to our DevOps team to investigate further.

Sincerely,

SourceForge Support

[SourceForge Support]

Hello,

Unfortunately, that is how our File Release System is designed.

A link to a file, like: https://sourceforge.net/projects/reactos/files/ReactOS/0.4.14/ReactOS-0.4.14-live.zip/download redirects to https://downloads.sourceforge.net/... which in turn redirects to a mirror that has the actual file, like https://versaweb.dl.sourceforge.net/... which may vary based on region.

Resuming the final mirror URL should work; but the original URL isn't the actual file so can't be resumed.

And its not safe for a script to hardcode a specific mirror, because it may or may not have a file (or may be on the opposite side of the world for some people).

It might be possible to do some advanced scripting to resolve the download URL to the final one, and then re-use that for resuming.

Sincerely,
SourceForge Support

[milahu]

Resuming the final mirror URL should work

only 7 of 30 sourceforge.net mirrors support http range requests

good mirrors:
freefr jaist liquidtelecom nchc netcologne tenet versaweb

bad mirrors:
altushost-swe cfhcable cyfuture cytranet-dal deac-fra deac-riga excellmedia gigenet icolo ixpeering kumisystems netactuate onboardcloud phoenixnap pilotfiber psychz razaoinfo sinalbr sitsa unlimited webwerks yer zenlayer

name=lineage-18.1-20240914-VANILLA-waydroid_x86_64-system.zip
path=project/waydroid/images/system/lineage/waydroid_x86_64/$name
mirrors="freefr jaist liquidtelecom nchc netcologne tenet versaweb"

for mirror in $mirrors; do
  echo "mirror: $mirror"
  url="https://$mirror.dl.sourceforge.net/$path?viasf=1"
  timeout 5 \
  wget --continue -O "$name" "$url"
done

i tried to use aria2 for parallel download from multiple http mirrors,
but its rather slow...

name="lineage-18.1-20240914-VANILLA-waydroid_x86_64-system.zip"
path="project/waydroid/images/system/lineage/waydroid_x86_64/$name"
mirrors="freefr jaist liquidtelecom nchc netcologne tenet versaweb"

aria2c --metalink-file <(
  echo "<metalink xmlns='urn:ietf:params:xml:ns:metalink'>"
  echo "<file name='$name'>"
  for mirror in $mirrors; do
    url="https://$mirror.dl.sourceforge.net/$path?viasf=1"
    echo "<url>$url</url>"
  done
  echo "</file>"
  echo "</metalink>"
)

[SourceForge Support]

Hello,

We have not heard from you and are now closing this ticket. If you need any help moving forward, please submit another ticket. Thank you.

Sincerely,
SourceForge Support