Menu

cli-downloader

big-bass
Attachments
cli-downloader.bac (9187 bytes)
cli-downloader1.png (81994 bytes)
cli-downloader2.png (66971 bytes)

top
🔥NEW🔥

cli-downloader QT5 baconized 70kb

you can run some tests to get familiar with the command line

simple

./cli-downloader http://www.basic-converter.org/hug.bac
  • if you installed the cli-downloader (remove the ./) or are running it from the compiled .bac
  • the file will automatically go to home Downloads

multiple with the -o option

  • this allow you to send the downloads to a folder that you choose
  • in this example we will use TESTME as the folders name
  • and use the command if it is installed in /usr/bin as a deb package
cli-downloader -o ~/TESTME/ http://www.basic-converter.org/doc_frame.html \
http://www.basic-converter.org/hug.bac \
http://www.basic-converter.org/stable/CHANGES \
http://www.basic-converter.org/stable/bacon-4.5.tar.gz

cli-downloader.bac

download the source code

cli-downloader.bac
depends at least qtbase5-dev libqt5websockets5

*This also has an additional cmake_build that can generate a debian package for you
will show how to do it at the bottom of this post *

'---VERSION 2.1 Aug 2 2022  cli-downloader
'---option -o  ~/Downloads/  url url 
'---must have trailing slash when using the ~/target_dir/ !
'---ported https://code.qt.io/cgit/qt/qtbase.git/tree/examples/network/download/main.cpp?h=5.15 
'---and removed class code heavily modified for bacon use by bigbass
'   fmt$ = "%s%s\n"  fix for mint thanks vovchik
'   fixed empty arguments display help
'   removed c++ code from bacons PRINT
'   handle c++ with PRINTOUT and PRINTABLE
'   changed i to start at 2 this fixed empty urls
'   added application/x-www-form-urlencoded   download are much faster now
'   -Wno-format-security  tested on mint 21
'   reply warning fixed
'---found a way to mix bacon commands !  QString to char* hack
'---MAKEDIR (char*) PRINTABLE(target_dir)
'--- added default folder to Downloads and fixed the index many improvements

PRAGMA INCLUDE <iostream> <QtCore/QDebug>  <QtCore/QDir> <QtCore/QDir> 
PRAGMA INCLUDE <QtCore/QStringList> <QtCore/QFile> <QtCore/QFileInfo>
PRAGMA INCLUDE <QtCore/QMimeDatabase> <QtCore/QMimeType> <QtCore/QTextStream>
PRAGMA INCLUDE <QtCore/QUrl>  <QtCore/QUrlQuery> <QtNetwork/QNetworkReply> 
PRAGMA INCLUDE <QtCore/QDateTime> <QtCore/QSize>
PRAGMA INCLUDE <QtNetwork/QNetworkAccessManager> <QtNetwork/QNetworkRequest>
PRAGMA INCLUDE <cstdio> <QtCore/QTimer> <QtCore/QCoreApplication> 
PRAGMA INCLUDE <QtCore/QCommandLineParser> <QtCore/QCommandLineOption>
PRAGMA LDFLAGS  -lQt5Network -lQt5Core  -lpthread -latomic 
PRAGMA COMPILER g++
'cli-downloader light version for cli only no widget dependencies added 

'RPI3
PRAGMA OPTIONS -Wno-return-type -Wno-write-strings -Wno-pointer-arith -Wformat=1
PRAGMA OPTIONS -I/usr/include/arm-linux-gnueabihf/qt5  
PRAGMA OPTIONS -I/usr/include/arm-linux-gnueabihf/qt5/QtNetwork -fPIC


'x86
'PRAGMA OPTIONS -Wno-return-type -Wno-write-strings -Wno-pointer-arith -Wno-format-security  -Wformat=1
'PRAGMA OPTIONS -I/usr/include/x86_64-linux-gnu/qt5 
'PRAGMA OPTIONS -I/usr/include/x86_64-linux-gnu/qt5/QtNetwork  -fPIC

'depends at least  qtbase5-dev libqt5websockets5

OPTION PARSE FALSE

DEF FN  PRINTOUT(x) =  qDebug(x)  
DEF FN  PRINTABLE(p) =  qPrintable(p)  

DECLARE app TYPE QCoreApplication*
DECLARE manager TYPE QNetworkAccessManager*
'DECLARE reply TYPE QNetworkReply*
DECLARE request TYPE QNetworkRequest
DECLARE currentDownloads TYPE QList<QNetworkReply *>
DECLARE dlcounter = 0 TYPE int 
DECLARE fincounter = 0 TYPE int 


DECLARE target_dir TYPE QString
DECLARE filePath TYPE QString
DECLARE cli_option TYPE QString

manager = new QNetworkAccessManager()


'had trouble globally seeing and comparing the option
cli_option = "-o"



'--------------------------------------------
SUB DownloadManager()
'--------------------------------------------
 '(1)
    COLOR FG TO INTENSE WHITE
    'PRINT "(1) initialize the Download manager"
    COLOR RESET
    QObject::connect(manager, &QNetworkAccessManager::finished, downloadFinished)
END SUB 


'--------------------------------------------
SUB  execute() 
'--------------------------------------------
    COLOR FG TO INTENSE WHITE
    'PRINT "(2) Process the command line urls"  
    COLOR RESET
    LOCAL nameofapp TYPE const char*
    LOCAL str TYPE QString
    LOCAL arg2 TYPE QString



    QStringList args = QCoreApplication::instance()->arguments()
    nameofapp = PRINTABLE(args.at(0).toLocal8Bit())

    ' remove the first argument, which is the program's name
    args.takeFirst()

    '#NO ARGUMENTS GIVEN
    IF  args.isEmpty()  == true THEN
        COLOR FG TO INTENSE BLUE
        PRINT "Qt Downloader  - downloads a single or multiple URLs by bigbass July 31 2022"
        PRINT "--------------------------------------------------------"
        PRINT "Usage: ",nameofapp, " url"
        PRINT "Downloads the URL or URLs passed in the command-line to Downloads by default"
        PRINT "If the target file already exists it will get over written with the new download"
        PRINT "--------------------------------------------------------"
        PRINT "Usage2: ",nameofapp, " -o  ~/Downloads/  url url "
        PRINT "Must have the trailing slash when using the -o ~/target_dir/"
        PRINT "Note  the target_directory  will be created automatically"
        COLOR RESET
        END
    ELSE

    '#NO -o OPTION GIVEN
    '---we need to create a default directory when none is given ver 2.1
    IF  args.at(0).toLocal8Bit() ISNOT cli_option THEN
        ' you forgot the target directory 
        ' make a default to ~/Downloads/
        HOME$ = CHOP$(EXEC$("echo  $HOME" ))
        target_dir = HOME$&"/Downloads/"
        PRINT "Automatically sent it to the Downloads folder"
        'chop extra spaces
        args.replaceInStrings(QRegExp("^\\s+"),"")
    END IF



    '# -o OPTION GIVEN  and your target directory also
    '---option homemade -o parser hack to a target_dir
    IF args.at(0).toLocal8Bit() == cli_option THEN

        'get the second arg then remove it from the list later
        arg2 = args.at(1).toLocal8Bit()

        'copy arg2 to the target_dir delete arg2 from the download list
        target_dir = arg2

        COLOR FG TO INTENSE BLUE
        PRINT "-o  DIR you want the download to go "
        PRINTOUT(PRINTABLE(target_dir)) 
        '---found a way to mix bacon commands !
        MAKEDIR (char*) PRINTABLE(target_dir)

        COLOR RESET
        'remove the option  -o
        args.replaceInStrings(cli_option,"")
        'remove the  arg2  which is the target_dir 
        args.replaceInStrings(arg2,"")
        args.replaceInStrings(QRegExp("^\\s+"),"")
     END IF

     '#INITIALIZE INDEX
     'changed i to 0     ver 2.1
     LOCAL Multi_url TYPE QString
     LOCAL i = 0 TYPE int
     LOCAL num TYPE int
     num = args.size()



    '# PASS the urls to the doDownload function
     REPEAT  
        Multi_url = args.at(i).toLocal8Bit()
        IF (Multi_url.isEmpty()) THEN
        ' do nothing and stay here for a moment if a bad url happens
        ELSE 
            CALL doDownload(Multi_url)
        END IF 
            'PRINTOUT(PRINTABLE(args.at(i).toLocal8Bit()))
            'PRINT "i = ",i
            INCR i      
     UNTIL num  == i
        END IF
END SUB


'--------------------------------------------
SUB doDownload(const QUrl &url)
'--------------------------------------------
    'PRINT "(3)" 
    IF (url.isEmpty()) THEN 
     'if you arrive here i needs to be set correctly
    ELSE

        QList<QNetworkReply *> currentDownloads
        QNetworkRequest request(url)
        'force a user agent 
        request.setRawHeader( "User-Agent" , "Mozilla Firefox" )
        request.setHeader(QNetworkRequest::ContentTypeHeader, "application/x-www-form-urlencoded")
        QNetworkReply *dlreply = manager->get(request)
        currentDownloads.append(dlreply)
        '--- use a dlcounter to see how many urls are used 
        INCR dlcounter
    END IF
END SUB


'--------------------------------------------
SUB  downloadFinished(QNetworkReply *reply)
'--------------------------------------------
    'PRINT "(4)" 
    LOCAL url TYPE QUrl
    LOCAL fmt$ TYPE STRING
    url = reply->url()
    IF (reply->error()) THEN
        COLOR FG TO INTENSE WHITE
        EPRINT "Download of  "
        PRINTOUT(PRINTABLE(url.toEncoded().constData()))
        PRINT " failed: " 
        PRINTOUT(PRINTABLE(reply->errorString()))
        COLOR RESET
    ELSE
        IF (isHttpRedirect(reply)) THEN
            COLOR FG TO INTENSE BLUE
            fprintf(stderr, "Request was redirected.\n")
            COLOR RESET
       ELSE
            '--- use a fincounter to see how many urls finshed and downloaded 
            QString filename = saveFileName(url)
            filePath = target_dir  + filename
            IF (saveToDisk(filePath, reply)) THEN
                COLOR FG TO INTENSE YELLOW
                'PRINT "Download of  "
                PRINTOUT(PRINTABLE(url.toEncoded().constData()))
                PRINT  "succeeded! "  
                'PRINT "saved as " 
                'PRINTOUT(PRINTABLE(filename))
                INCR fincounter
                COLOR RESET
            END IF
        END IF
    END IF



    IF (dlcounter == fincounter ) THEN
        COLOR FG TO GREEN
        PRINT  "All downloads finished !"
        COLOR RESET
        END
    END IF
END SUB




'--------------------------------------------
FUNCTION  isHttpRedirect(QNetworkReply *reply) TYPE bool
'--------------------------------------------
    'PRINT "(5)"
    LOCAL statusCode TYPE int
    statusCode = reply->attribute(QNetworkRequest::HttpStatusCodeAttribute).toInt()
    RETURN (statusCode == 301 || statusCode == 302 || statusCode == 303 || \
     statusCode == 305 || statusCode == 307 || statusCode == 308)
END FUNCTION


'--------------------------------------------
FUNCTION saveFileName(const QUrl &url) TYPE   QString 
'--------------------------------------------
    'PRINT "(6)" 
    LOCAL path TYPE QString
    LOCAL basename TYPE QString
    path = url.path()
    basename = QFileInfo(path).fileName()

    IF (basename.isEmpty()) THEN 
        basename = "download"
    ELSE
        RETURN basename
    END IF
END FUNCTION 





'--------------------------------------------
FUNCTION saveToDisk(const QString &filename, QIODevice *data) TYPE bool
'--------------------------------------------
   'PRINT "(7)"
        QFile file(filename)
   IF  (file.open(QIODevice::WriteOnly)) == false THEN
        COLOR FG TO INTENSE WHITE
        EPRINT "Could not open " 
        PRINTOUT(PRINTABLE(filename))
        PRINT " for writing:  " 
        PRINTOUT(PRINTABLE(file.errorString()))
        COLOR RESET
        RETURN FALSE
   ELSE
        file.write(data->readAll())
        file.close()
        RETURN TRUE
   END IF
END FUNCTION




    '---main
    app = new QCoreApplication(argc, argv)
    'special note we use  QCoreApplication  
    'instead of the standard QApplication when its all cli
    CALL DownloadManager
    CALL execute
    app->exec()

EXTRA

RPI 3 and 4

cmake source to build the package

https://sourceforge.net/projects/bassix-gtk3-webkit-browser/files/CMAKE_BUILDS/CMAKE_CLI_DOWNLOADER_2.1.0.tar.gz/download

x86_64 amd

debian package only

https://sourceforge.net/projects/bassix-gtk3-webkit-browser/files/amd64-deb-packages/cli-downloader_2.1.0_amd64.deb/download

cmake_build

cmake source to build the package

https://sourceforge.net/projects/bassix-gtk3-webkit-browser/files/CMAKE_BUILDS/CMAKE_CLI_DOWNLOADER_x86_64v_2.1.0.tar.gz/download

*Just three commands and always the same three

cmake .
cmake --build .
cpack

TOP


Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.