sfk wget not behaving correctly
One hundred command line tools in a small and portable binary.
Brought to you by:
stahlworks
Menu
▾
▴
1 Attachments
#49 sfk wget not behaving correctly
Let me clarify real quick. When I say that "sfk wget" is not working correctly, I am comparing the sfk wget command to how the tool wget itself would perform the operation (aka "wget http://url.com/file.exe -O filename.exe"). Now, one of the things that makes wget so incredibly useful is that it is able to follow the redirections it encounters and/or allows users to specify the exact file (name, extension and all) for urls which may have a different MIME type or extension than the final file that is downloaded (I assume the web server processes the file before serving the download when downloaded regularly). However, I found that "sfk wget", while it does support this functionality as well TO SOME EXTENT, the area where it runs into trouble is due to the fact that it parses the URL that is passed to it prior to processing the request. This has led to many, many instances where sfk has determined that a completely valid download URL (which has been masked or has a redirection or something which prevents the full file path from appearing) is not valid and errors out before even trying to perform the download. Attached is a screenshot not only showing how sfk wget fails to process one such download, but also shows a) that the download link supplied is pulled straight from the URL of the item in Chrome's download manager, and b) that the regular wget utility is able to process the equivilant of the same command with no errors at all, and the executable works just fine. A word of advice: Just don't parse the URL. If there is an error connecting to the web server/webpage/file indicated in the URL, that is when the error should be thrown and the processing stopped. NOT BEFORE!
your example image does not show any redirect.
instead it uses an https:// url meaning something should be downloaded by SSL encryption.
sfk does not support SSL encryption at all.
because your wget tool happens to support SSL it can download from that URL.
if you could provide an example that really applies a redirect, using plain http:// urls, I would like to look into that.
Ah. I see. I guess I misspoke. SFK redirection works fine, guess its the SSL that doesn't work. Any particular reason why it doesn't? I also thought the error was due to parsing of the URL according to the error message, which says "invalid url"
Last edit: William Higgs 2017-07-14
so far, ssl inclusion is too complex.
it means to add a hundred source files with a complicated compile, over three platforms.
and then it doesn't even work properly and fails to connect with many websites.
I see. Well, go ahead and close this ticket I suppose. Since SSL is the actual issue, then your tool is fine