Webware for Python

On Thursday, September 25, 2003, at 10:31 PM, Edmund Lian wrote:
> I've been seeing intermittent _formID_ KeyErrors with FunFormKit of 
> the form:
>
> Traceback (most recent call last):
>   File "WebKit/Application.py", line 415, in dispatchRequest
>   File "WebKit/Application.py", line 567, in handleGoodURL
>   File "WebKit/Application.py", line 776, in respond
>   File "WebKit/Transaction.py", line 105, in respond
>   File "WebKit/HTTPServlet.py", line 38, in respond
>   File "/opt/Mefa/SiteLib/Page.py", line 113, in respondToPost
>     self._respond(transaction)
>   File "/opt/Mefa/SiteLib/Page.py", line 134, in _respond
>     self.mainResponse()
>   File "/opt/Mefa/Contexts/Home/Login.py", line 50, in mainResponse
>     formProcessed, data = self.processForm()
>   File "/opt/Webware/FunFormKit/Form.py", line 127, in processForm
>     currentForm = self._formDefinitions[req.field('_formID_')]
> KeyError: form1
>
>
> I've been trying to figure out when these are triggered, and how best 
> to deal with them.
>
> I know for example, that if I have a form on display, then restart the 
> appserver, then try to submit the form, the error will pop up. It also 
> pops up if the browser's back and forward navigation buttons are used 
> prior to form submission. So, the issue seems to be related to the 
> browser caching pages, as well as the fact that the formID was 
> autogenerated so that between restarts, the same form might have 
> different formIDs.

The problem is you are using mutable forms, and the forms are stored in 
a module global and keyed to these dynamic names (like "form1").  When 
you restart the AppServer there may be people with reference to forms 
that no longer exist.

Ideally these should be stored in some other persistent form.  It would 
be nice if the session mechanism could be reused in some global 
fashion, and this stuff could be stored in there.  (There's several 
places where global persistent data would be nice, e.g., randomly 
generated keys used to sign hidden fields)

   Ian

Ian Bicking wrote:

> The problem is you are using mutable forms, and the forms are stored in 
> a module global and keyed to these dynamic names (like "form1").  When 
> you restart the AppServer there may be people with reference to forms 
> that no longer exist.

But I get the error even on pages that don't use mutable forms, and even 
on pages that have only one form.

There must be something else going on here. It's maddening because I 
can't seem to reproduce the error consistently, so I can't figure out 
what the problem is.


> Ideally these should be stored in some other persistent form.  It would 
> be nice if the session mechanism could be reused in some global fashion, 
> and this stuff could be stored in there.  (There's several places where 
> global persistent data would be nice, e.g., randomly generated keys used 
> to sign hidden fields)

Assuming this to be the problem, is the workaround to explicitly name 
each form? <shudder>


...Edmund.

Ian Bicking wrote:

> The problem is you are using mutable forms, and the forms are stored in 
> a module global and keyed to these dynamic names (like "form1").  When 
> you restart the AppServer there may be people with reference to forms 
> that no longer exist.

Just trying to understand what's happening... I notice that sometimes, 
the very same page has a formID of form1, sometimes form2, etc. When 
does FFK decide to issue a different formID to the same form?

...Edmund.

Ian Bicking wrote:

> The problem is you are using mutable forms, and the forms are stored in 
> a module global and keyed to these dynamic names (like "form1").  When 
> you restart the AppServer there may be people with reference to forms 
> that no longer exist.

Just looked at how formIDs were generated:

def getName():
     global nameCount, nameCountLock
     nameCountLock.acquire()
     name = 'form%i' % nameCount
     nameCount = nameCount + 1
     nameCountLock.release()
     return name

So if I read this correctly, if form names are not explicitly assigned, 
then the name a form has will depend on when it was first instantiated.

If one has two different forms fA in servlet sA, and fB in servlet sB, 
then if servlet sA is instantiated before sB, form fA will have formID = 
form1, and form fB will have formID = form2.

If, on a restart, servlet sB is instantiated but sA is not, then form fB 
will now have formID = form1, and form2 will not exists in the global space.

So, if this is correct, FFK is mighty sensitive to servlet instantiation 
order. The immediate workaround is to assign formIDs in a manner that is 
not instantiation order sensitive. E.g., use the absolute path of the 
servlet as the formID. Or, assign names to forms explicitly.

Is this correct?

...Edmund.

Edmund Lian wrote:

> So, if this is correct, FFK is mighty sensitive to servlet instantiation 
> order. The immediate workaround is to assign formIDs in a manner that is 
> not instantiation order sensitive. E.g., use the absolute path of the 
> servlet as the formID. Or, assign names to forms explicitly.
> 
> Is this correct?

Answering my own question, yikes--FFK's assignment of formIDs is indeed 
sensitive to form/servlet order. Argh! Not quite sure what the best fix 
is here...


...Edmund.

On Friday, September 26, 2003, at 12:21 AM, Edmund Lian wrote:
> Just looked at how formIDs were generated:
>
> def getName():
>     global nameCount, nameCountLock
>     nameCountLock.acquire()
>     name = 'form%i' % nameCount
>     nameCount = nameCount + 1
>     nameCountLock.release()
>     return name
>
> So if I read this correctly, if form names are not explicitly 
> assigned, then the name a form has will depend on when it was first 
> instantiated.
>
> If one has two different forms fA in servlet sA, and fB in servlet sB, 
> then if servlet sA is instantiated before sB, form fA will have formID 
> = form1, and form fB will have formID = form2.
>
> If, on a restart, servlet sB is instantiated but sA is not, then form 
> fB will now have formID = form1, and form2 will not exists in the 
> global space.
>
> So, if this is correct, FFK is mighty sensitive to servlet 
> instantiation order. The immediate workaround is to assign formIDs in 
> a manner that is not instantiation order sensitive. E.g., use the 
> absolute path of the servlet as the formID. Or, assign names to forms 
> explicitly.
>
> Is this correct?

Sounds correct.  You should be able to do it simply off the form 
definition's __class__.__name__ and __class__.__module__, which 
together should be unique.

   Ian

I've checked in a change to FFK's Form.py so that the formID generation 
function assigns and track formIDs based on servlet URL and a URL 
specific counter. This should mean that auto-generated formIDs are no 
longer sensitive to instantiation servlet order. The patch for this 
modification is appended below FYI.

I also checked in a minor change to Field.SelectField to have the field 
display the label instead of value when it is static.

...Edmund.




Index: Form.py
===================================================================
RCS file: /cvsroot/funformkit/FunFormKit/Form.py,v
retrieving revision 1.17
diff -r1.17 Form.py
500c500
< nameCount = 1
---
 > nameCount = {}
502c502
< def getName():
---
 > def getName(servletName):
505,506c505,509
<     name = 'form%i' % nameCount
<     nameCount = nameCount + 1
---
 >     if servletName in nameCount:
 >         nameCount[servletName] += 1
 >     else:
 >         nameCount[servletName] = 1
 >     name = '%s%i' % (servletName, nameCount[servletName])
542c545
<             name = getName()
---
 >             name = getName(handlerServletURL)