#4 Support for large request/response

closed-rejected
nobody
None
5
2001-01-24
2000-12-01
Vladimir Kralik
No

Discussion

  • None

     
    Attachments
  • !!! Before You apply patch, You need apply my patches
    101932 Webware as python module
    102188 Basic security for mod_python

    * support for large request added ( file upload )
    Transaction si now created from dictionary, which has added 'input' and
    'output' keys. ( For output see response )
    Values of that keys are files ( file, StringIO ... ).
    AppServer now does not read all input from browser into memory ( string
    ), but simply put opened file into Application and in Application into
    CGI module.

    In AsyncoreThreadedAppServer request and response are buffered if are
    smaller than configuring size.
    
    \( In CGI module is bug, all file lines are stored in list 
    cgi.FieldStorage.lines, serch and delete lines containing string 
    "self.lines", 4 ocurences \)
    
    cgi.FieldStorage contais two methods to access upladed file.
        form\["userfile"\].value - which read all file into string
        form\["userfile"\].file and form\["userfile"\].filename
            form\["userfile"\].file contains opened tmp-file
                containing uploaded file
            form\["userfile"\].filename contains filename from
                browser \( see cgi-documentation \)
    HTTPRequest destroy value from object FieldStorage, and load all file
    into memory. I change this behavior. If it's a uploaded file
    HTTPRequest.field\["userfile"\] return tuple \( string filename and opened
    file \)
    
    Example :
    req = self.request\(\)
    if req.hasField\("userfile"\):
        if type\(req.field\("userfile"\)\)==TupleType:
            filename,file=req.field\("userfile"\)
            poc=0
            while 1:
                data=file.read\(1024\)
                if not data: break
                poc=poc+len\(data\)
                del data
            self.writeln\("Size of %s is %d bytes" % \(
                filename, poc \)\)
        else:
            self.writeln\("%s is not file" %req.field\("userfile"\)\)
    

    * support for large response added ( generate file from database and sent
    it directly to browser )
    In HTTPResponse I change behavior of method deliver. When this method
    is called all texts buffered into memory are send to browser. But
    response can continue to generating more output. When I want sent large
    bundle of data, I call transaction deliver method ( which is the same
    as Response.deliver ).

    If deliver is called, HTTP-header is send to browse, this mean that
    Error messeges cannot overwrite output. Traceback is simple append to
    previusly send content of that page. \( Constructor of Transaction must
    be changed \).
    
    Output of respond is send to file, which respond gets from transaction,
    when transation is created from dictionary \( output \).
    
    Example :
    class bigfile\(Page\):
        def respond\(self,trans\):
            resp=self.response\(\)
            resp.setHeader\('Content-type', 'application/data'\)
            trans.deliver\(\)
            file=open\('/dev/hda','r'\) \# some data
            for i in range\(1000\):
                data=file.read\(1024\)
                if not data: return
                self.write\(data\)
    

    * protokol on which Adapter communication with AppServer is changed from marshal
    to cPickle ( I need this for implementing buffer in AsyncThreadedServer)
    * I add argument pidfile main methon of AsyncThreadedAppServer and
    ThreadedAppServer. Pid of main-process is write to file $pidfile. This
    is needed by running AppServer as demon on Unix.

    * some security is added to Examples/Colorize.py

     
    • status: open-rejected --> closed-rejected