Webware for Python

marcelo ametller wrote:
> I have a servlet, with a form, a button with label "generate".
> When the user press the button, de "action" run a function, it generate
> a pdf (reportlab).
> When the pdf have 200 items, that's ok. But when the pdf have 6000
> items, the appserver crash. The dmesg inform that there isn't memory...

Does this crash immediately after you start the appserver and generate 
the very first pdf? Or does it only happen after some time when more 
users have been on the server?

What do you get in the Webware logfile?

Do you create the pdf in memory only or do you write it to a file?

-- Christoph

On 1/21/06, Christoph Zwerschke <cito@...> wrote:
> marcelo ametller wrote:
> > I have a servlet, with a form, a button with label "generate".
> > When the user press the button, de "action" run a function, it generate
> > a pdf (reportlab).
> > When the pdf have 200 items, that's ok. But when the pdf have 6000
> > items, the appserver crash. The dmesg inform that there isn't memory...
>
> Does this crash immediately after you start the appserver and generate
> the very first pdf? Or does it only happen after some time when more
> users have been on the server?
>
> What do you get in the Webware logfile?
>
> Do you create the pdf in memory only or do you write it to a file?
>
> -- Christoph

Also, can do you this generation outside of Webware, say as a
standalone Python program? In the past, most, if not all, crashes have
been associated with compiled Python extensions.

-Chuck

Christoph Zwerschke wrote:
> marcelo ametller wrote:
> 
>> I have a servlet, with a form, a button with label "generate".
>> When the user press the button, de "action" run a function, it generate
>> a pdf (reportlab).
>> When the pdf have 200 items, that's ok. But when the pdf have 6000
>> items, the appserver crash. The dmesg inform that there isn't memory...
> 
> 
> Does this crash immediately after you start the appserver and generate
> the very first pdf? Or does it only happen after some time when more
> users have been on the server?

it only crash after some time... the time for generate de pdf is 20min.

> 
> What do you get in the Webware logfile?
nothing...


> 
> Do you create the pdf in memory only or do you write it to a file?
i not know that...
i paste de code here:

action for the botton:

	def generar(self):		
		fecha1=self.form.field('fecha1').value()
		fecha2=self.form.field('fecha2').value()
		
		self.session().setValue('fecha1', fecha1)
		self.session().setValue('fecha2', fecha2) 		
		url='/menu/fac/inf/v/libFactR1'
		self.response().sendRedirect(url)
		self.writeBody()	

part of the servlet:

def writeHTML(self):
		req=self.request()		
		self.fecha1=self.session().value('fecha1')
		self.fecha2=self.session().value('fecha2')
		self.empresa = empresa.nombre	
		self.titulo = "Resumen Libro Facturacion - Facturas generadas entre %s
- %s" % (self.fecha1,self.fecha2)

		fact= Fact1()
		#consulta sql (return 6000 regs for 10 columns)
		self.listado= fact.lisLibFactR1(self.fecha1,self.fecha2)
		
		buffer=StringIO()
		doc = SimpleDocTemplate(buffer,pagesize=A4,leftMargin=3*cm)
		doc.setTitle=self.titulo
		doc.setAuthor='operador'
		
		Story = []
		data=self.listado	
		colwidths=(10*cm,2*cm)
		t=Table( data, colwidths)
		t.setStyle(TableStyle([('VALIGN',(0,0),(-1,-1),'MIDDLE'),					
('GRID',(0,0),(-1,-1),  0,colors.grey),						('BOX', (0,0), (-1,-1),
1, colors.black),						]))

		Story.append(t)
		Story.append(Spacer(1,0.2*cm))
			
		doc.build(Story, onFirstPage=self.myFirstPage,
onLaterPages=self.myLaterPages)
		pdf=buffer.getvalue()
		buffer.close()
		
		self.response().setHeader('Content-Type', 'application/pdf')
		self.response().setHeader('Content-Length', str(len(pdf)))
		self.response().setHeader('Content-Disposition', 'inline;
filename="ReslibFact.pdf"')
		self.write(pdf)


thanks for you time...

Marcelo

marcelo ametller wrote:
> it only crash after some time... the time for generate de pdf is 20min.

That's way too long. If it really must take so long, you should spawn a 
separate process to create the pdf. Your webserver will probably quit 
the connection with timeout anyway.

>> What do you get in the Webware logfile?
> nothing...

You can insert print statements, and use unbuffered output for the 
AppServer, to see where the servlet is spinning. But after a look on 
your servlet I am almost sure it will be the doc.build() call.

>> Do you create the pdf in memory only or do you write it to a file?
> i not know that...
> i paste de code here:

That would be in memory. Can you check whether the StringIO is imported 
from cStringIO (not StringIO)?

But the real problem is probably here:

> 		t=Table( data, colwidths)

You should definitely use "LongTable" instead of "Table" if your table 
has so many rows. The time used by Table goes quadratic with the number 
of rows...

The rest looks fine.

Let us know whether that helps.

-- Christoph

Christoph Zwerschke wrote:
> marcelo ametller wrote:
> 
>> it only crash after some time... the time for generate de pdf is 20min.
> 
> 
> That's way too long. If it really must take so long, you should spawn a
> separate process to create the pdf. Your webserver will probably quit
> the connection with timeout anyway.
> 
>>> What do you get in the Webware logfile?
>>
>> nothing...
> 
> 
> You can insert print statements, and use unbuffered output for the
> AppServer, to see where the servlet is spinning. But after a look on
> your servlet I am almost sure it will be the doc.build() call.
> 
>>> Do you create the pdf in memory only or do you write it to a file?
>>
>> i not know that...
>> i paste de code here:
> 
> 
> That would be in memory. Can you check whether the StringIO is imported
> from cStringIO (not StringIO)?
yes: from cStringIO import StringIO
I suppose that use the memory of the servidor?



> 
> But the real problem is probably here:
> 
>>         t=Table( data, colwidths)

now use:

from reportlab.platypus import SimpleDocTemplate, Paragraph, Spacer,
LongTable, TableStyle

the time of that form is smaller...

> 
> 
> You should definitely use "LongTable" instead of "Table" if your table
> has so many rows. The time used by Table goes quadratic with the number
> of rows...
> 
> The rest looks fine.
> 
> Let us know whether that helps.


it's the appserver inf:

empezo doc.build
   13  2006-01-24 09:51:39  /menu/fac/inf/v/auxIva1

termino doc.build
   13  234.81 secs          /menu/fac/inf/v/auxIva

the pdf contain 180 pages.

the appserver not crash...

thanks...




-- 
Marcelo

Marcelo Ametller wrote:
> yes: from cStringIO import StringIO
> I suppose that use the memory of the servidor?

Yes, it provides a kind of memory file for Reportlab to write to instead 
a real file on the harddisk. In this case, it would not make much 
difference, but if you serve many small PDFs it is better. You also 
don't have to care about a unique temporary file name. Sometimes it may 
also make sense to cache the PDFs you created. I.e. you don't create a 
temporary filename, but a filename that is made from the input 
parameters (e.g. weekly_report_03_2006). If somebody else requests the 
same document, you can serve it directly from disk.

-- Christoph