addanc-developer — Addanc developer discussions, debates, and suggestions

[Addanc-developer] FASD project: Online survey launched

[<ben...@id...>]

Dear Open Source developer

I am doing a research project on "Fun and Software Development" in which I kindly invite you to participate.
You will find the online survey under http://fasd.ethz.ch/qsf/. The questionnaire consists of 53 questions and you will need about 15 minutes to complete it.

With the FASD project (Fun and Software Development) we want to define the motivational significance of fun when software developers decide to engage in Open Source projects. What is special about our research project is that a similar survey is planned with software developers in commercial firms. This procedure allows the immediate comparison between the involved individuals and the conditions of production of these two development models. Thus we hope to obtain substantial new insights to the phenomenon of Open Source Development.


With many thanks for your participation,
Benno Luthiger


PS:
The results of the survey will be published under http://www.isu.unizh.ch/fuehrung/blprojects/FASD/.
We have set up the mailing list fa...@we... for this study. Please see http://fasd.ethz.ch/qsf/mailinglist_en.html for registration to this mailing list.

_______________________________________________________________________

Benno Luthiger
Swiss Federal Institute of Technology Zurich
8092 Zurich

Mail: benno.luthiger(at)id.ethz.ch
_______________________________________________________________________

[Addanc-developer] Decoupling the iterator from vcScript

[Doug F. <dou...@do...>]

I now understand why we need a deep copy of the script in every VC. The function
vcScript.run() acts as an iterator on the _scriptList.

I propose replacing vcScript.run() with an explicit __iter__ member that creates
a vcScriptIter object to hold the iteration state for each VC. This actually
wouldn't be much more efficient than the deep copy, but I think it would be
clearer (at least to me).

I would like to code up an example, unless you have other priorities.
--
Doug Fort
Addanc
Downright Software Collective
------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/

Re: [Addanc-developer] Python Code Support (PCS) in Addanc Scripts - Draft 1

[Doug F. <dou...@do...>]

This sounds like a good start. I suggest a multi-phase approach: we can
refactor the existing code, one object at a time (session, request, response),
before we get into scripting. We may have to modify the infrastructure when we
start embedding Python, but it should be pretty well in place.

Quoting Joe Griffin <joe...@us...>:

> Here's a first draft of the 'spec' for Addanc's Python Code Support.
>
> Comments/ectc. requested.
>
> I want to review the sample syntax a bit more. And generate a more 'test'
> scripts.
>
> Joe
>
> =================================================
> Proposal for Python Code Support in Addanc Scripts
>
> Background/Justification
>
> The existing Addanc scripting support can handle only simply record/playback
> scenarios for form fields and query strings.
>
> More realistic and revealing tests could be accomplished if the Addanc
> scripting
> support gave programmatic access to transaction status and content (headers,
> form fields, querystrings, body/DOM, and attachments). Branching and
> "on-the-fly" request generation would further enhance the testing
> capabilities.
>
> The Addanc scripting model can be extended by using embedded Python and
> "describing" the behavior of each page using python code operating on a
> standard set of objects like a session object, a transaction object, and a
> response object.
>
> Python Code Support
>
> The Addanc script will include Python Code Support (PCS) if there exists a
> file
> in the same directory as the script .XML file with the same "root" name as
> the
> script file but with a .py extension; e.g, my_script.xml and my_script.py
> both
> in the /home/myacct/addanctest directory.
>
> For the purposes of this discussion, we will use this nomenclature:
>  - Script Description File (SDF) will refer to the .XML file
>  - Script Code File (SCF) will refer to the .py file
>
> The Script Code File (SCF) is a Python source file containing one or more
> function definitions. Functions in the SCF file are associated with
> transactions ing the SDF via the Transaction ID. Functions with the names
> <transaction_id>_pre() and <transaction_id_>_post() should be defined in the
> SCF  for any transaction in the SDF file that requires preparation or special
> post handling.
>
> The <transaction_id>_pre(session_object, request_object[, response_object])
> will
> be called after the request object has been built using the "default" values
> from the SDF file but before starting the transaction. The response_object
> will
> be the response_object after the _post() processing is performed on the
> previous  transaction.
>
> The <transaction_id>_post(session_object, response_object) will be called
> after
> the transaction is completed or has failed.
>
> PCS Objects
>
> Addanc's Pythoc Code Support exposes three objects:
>  - a Session Object packaging information the extends over the lifetime of
>    a VC,
>  - a Request Object packaging the information/data to be sent to s SUT as
>    part of the HTTP request/response sequence.
>  - a Response Object packaging information received from ther SUT as part
>    of the HTTP request/response sequence
>
> The Session Object supports the following properties/members:
>   .globals - Contains generic script globals for this session
>   .cookies - Container for cookies
>   .session_status - One of OK|REQUEST_ERROR|RESPONSE_ERROR|INTERNAL_ERROR
>   .next_page - Page Id of the next transaction to execute.
>                If .eq. None - stop this VC.
>                If current PAGE has multiple transactions, execution branching
>                will happen only if this property is changed to a value that
>                differs from the current Page Id)
>   .next_wait - Defines delay time after receiving current transaction before
>                starting request for next transaction. (AKA Sleep or read
> time.)
>
> The Request Object has the following properties/members:
>   .operation - One of POST|GET
>   .query_string_dict - dictionary of Query String key/val pairs
>   .field_dict - dictionary of form field key(field)/val pairs
>   .headers - dictionary of header key/val pairs
>   .url - target URL
>
> The Response Object has the following properties/members:
>   .url_feched - url of informatrion returned as result of request
>   .redirect_count - NUmber of URLs visited to resolve Response Object
>   .HTTPstatus - HTTP status of result
>   .headers - Dictionary of headers returned
>   .source - Raw text response returned
>
>
> =======================================================
> Simple PCS Example:
>
> Snippet from file PCS_test_001.xml
>
>     <page id='WestWingHome'>
>         <url
> id="Potus">http://addanc.testsite.downright.com/cgi-bin/potus-v1</url>
>         <wait>2.000</wait>
>     </page>
>
>     <page id='WestWingLogin'>
>         <url id="PotusLogin"
> op="post">http://addanc.testsite.downright.com/cgi-bin/potus-v1
>            <formfield name="loginid" type="text">joeg</formfield>
>            <formfield name="passwd" type="password">geoj</formfield>
>          </url>
>         <wait>2.000</wait>
>     </page>
>
>
>
> Sinppet fom filePCS_test_001.py:
>
> #filePCS_test_001.py
>
> def Potus_pre(session_object, request_object):
>     """ Called BEFORE executing operation on page """
>     # Build a dictionaries of Login Identies for the post test
>     # Dictionary is indexed by ID, returns password
>     session_object.globals.identies = {}
>     session_object.globals.identies['joeg':'geoj']
>     session_object.globals.identies['doug':'guod']
>     session_object.globals.identies['addanc':'cnadda']
>
>     return
>
> import random
>
> def Potus_post(session_object, response_object):
>     """ Called AFTER executing operation on page """
>     if session_object.session_status == OK:
>         # Set the wait time to a random 1 to 5 second delay
>         session_object.next_wait.set(1 + random.random()*4.0)
>         return
>     else:
>         # Stop test if transaction not OK
>         session_object.next_txn.set(None)
>         return
>
> def PotusLogin_pre(session_object, request_object):
>     """ Called BEFORE executing operation on page """
>     # Choose a random user ID to log in anf collect identity info
>     indexID = random.randrange(0, len(session_object.globals.identies))
>     user = session_object.globals.identies[indexID].getkey()
>     passwd = session_object.globals.identies[user]
>     # Set the fields
>     request_object.field_dict['loginid'] = user
>     request_object.field_dict['passwd'] = passwd
>
>     return
>
>
> def PotusLogin_post(session_object, response_object):
>     """ Called AFTER executing operation on page """
>     if session_object.session_status != OK:
>        # Stop test if transaction not OK
>         session_object.next_txn.set(None)
>     return
>
>
> ================== End Example ========================
>
>
> --
> Joe Griffin
> Addanc
> Downright Software Collective
>
>
>
> ------------------------------------------------------------------------
> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they
> are addressed. If you have received this email in error please notify
> sec...@do....
> ------------------------------------------------------------------------
> This mail sent through Downright Software's implementation of HORDE/IMP.
> For info, see: http://horde.org/imp/
>
>
>
> -------------------------------------------------------
> SF.Net is sponsored by: Speed Start Your Linux Apps Now.
> Build and deploy apps & Web services for Linux with
> a free DVD software kit from IBM. Click Now!
> http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
> _______________________________________________
> Addanc-developer mailing list
> Add...@li...
> https://lists.sourceforge.net/lists/listinfo/addanc-developer
>


--
Doug Fort
http://www.dougfort.net
------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/

[Addanc-developer] Python Code Support (PCS) in Addanc Scripts - Draft 1

[Joe G. <joe...@us...>]

Here's a first draft of the 'spec' for Addanc's Python Code Support.

Comments/ectc. requested.

I want to review the sample syntax a bit more. And generate a more 'test'
scripts.

Joe

=================================================
Proposal for Python Code Support in Addanc Scripts

Background/Justification

The existing Addanc scripting support can handle only simply record/playback
scenarios for form fields and query strings.

More realistic and revealing tests could be accomplished if the Addanc scripting
support gave programmatic access to transaction status and content (headers,
form fields, querystrings, body/DOM, and attachments). Branching and
"on-the-fly" request generation would further enhance the testing capabilities.

The Addanc scripting model can be extended by using embedded Python and
"describing" the behavior of each page using python code operating on a
standard set of objects like a session object, a transaction object, and a
response object.

Python Code Support

The Addanc script will include Python Code Support (PCS) if there exists a file
in the same directory as the script .XML file with the same "root" name as the
script file but with a .py extension; e.g, my_script.xml and my_script.py both
in the /home/myacct/addanctest directory.

For the purposes of this discussion, we will use this nomenclature:
 - Script Description File (SDF) will refer to the .XML file
 - Script Code File (SCF) will refer to the .py file

The Script Code File (SCF) is a Python source file containing one or more
function definitions. Functions in the SCF file are associated with
transactions ing the SDF via the Transaction ID. Functions with the names
<transaction_id>_pre() and <transaction_id_>_post() should be defined in the
SCF  for any transaction in the SDF file that requires preparation or special
post handling.

The <transaction_id>_pre(session_object, request_object[, response_object]) will
be called after the request object has been built using the "default" values
from the SDF file but before starting the transaction. The response_object will
be the response_object after the _post() processing is performed on the
previous  transaction.

The <transaction_id>_post(session_object, response_object) will be called after
the transaction is completed or has failed.

PCS Objects

Addanc's Pythoc Code Support exposes three objects:
 - a Session Object packaging information the extends over the lifetime of
   a VC,
 - a Request Object packaging the information/data to be sent to s SUT as
   part of the HTTP request/response sequence.
 - a Response Object packaging information received from ther SUT as part
   of the HTTP request/response sequence

The Session Object supports the following properties/members:
  .globals - Contains generic script globals for this session
  .cookies - Container for cookies
  .session_status - One of OK|REQUEST_ERROR|RESPONSE_ERROR|INTERNAL_ERROR
  .next_page - Page Id of the next transaction to execute.
               If .eq. None - stop this VC.
               If current PAGE has multiple transactions, execution branching
               will happen only if this property is changed to a value that
               differs from the current Page Id)
  .next_wait - Defines delay time after receiving current transaction before
               starting request for next transaction. (AKA Sleep or read time.)

The Request Object has the following properties/members:
  .operation - One of POST|GET
  .query_string_dict - dictionary of Query String key/val pairs
  .field_dict - dictionary of form field key(field)/val pairs
  .headers - dictionary of header key/val pairs
  .url - target URL

The Response Object has the following properties/members:
  .url_feched - url of informatrion returned as result of request
  .redirect_count - NUmber of URLs visited to resolve Response Object
  .HTTPstatus - HTTP status of result
  .headers - Dictionary of headers returned
  .source - Raw text response returned


=======================================================
Simple PCS Example:

Snippet from file PCS_test_001.xml

    <page id='WestWingHome'>
        <url
id="Potus">http://addanc.testsite.downright.com/cgi-bin/potus-v1</url>
        <wait>2.000</wait>
    </page>

    <page id='WestWingLogin'>
        <url id="PotusLogin"
op="post">http://addanc.testsite.downright.com/cgi-bin/potus-v1
           <formfield name="loginid" type="text">joeg</formfield>
           <formfield name="passwd" type="password">geoj</formfield>
         </url>
        <wait>2.000</wait>
    </page>



Sinppet fom filePCS_test_001.py:

#filePCS_test_001.py

def Potus_pre(session_object, request_object):
    """ Called BEFORE executing operation on page """
    # Build a dictionaries of Login Identies for the post test
    # Dictionary is indexed by ID, returns password
    session_object.globals.identies = {}
    session_object.globals.identies['joeg':'geoj']
    session_object.globals.identies['doug':'guod']
    session_object.globals.identies['addanc':'cnadda']

    return

import random

def Potus_post(session_object, response_object):
    """ Called AFTER executing operation on page """
    if session_object.session_status == OK:
        # Set the wait time to a random 1 to 5 second delay
        session_object.next_wait.set(1 + random.random()*4.0)
        return
    else:
        # Stop test if transaction not OK
        session_object.next_txn.set(None)
        return

def PotusLogin_pre(session_object, request_object):
    """ Called BEFORE executing operation on page """
    # Choose a random user ID to log in anf collect identity info
    indexID = random.randrange(0, len(session_object.globals.identies))
    user = session_object.globals.identies[indexID].getkey()
    passwd = session_object.globals.identies[user]
    # Set the fields
    request_object.field_dict['loginid'] = user
    request_object.field_dict['passwd'] = passwd

    return


def PotusLogin_post(session_object, response_object):
    """ Called AFTER executing operation on page """
    if session_object.session_status != OK:
       # Stop test if transaction not OK
        session_object.next_txn.set(None)
    return


================== End Example ========================


--
Joe Griffin
Addanc
Downright Software Collective



------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/

Re: [Addanc-developer] A Possible Addanc Extension

[Doug F. <dou...@do...>]

On further consideration, I'm afraid you're right. In the Unix philosophy,
Addanc is a tool that should do one thing extremely well. Adding FTP wouldn't
push it out of shape, but mail QOS would start turning it into a blob.


Quoting Joe Griffin <joe...@us...>:

>
> My first reaction was ... yea, just change the protocol and away you go...
>
> Second thoughts... the mail protocol is not strickly send/receive .. and the
> process you describe requires coupling two transactions...while this may be
> accomplished in addanc, I think you may rewrite moe than you reuse.
>
> Final thoughts, addanc may not be the best framework for a set of cooperating
> agents...it would provide the scheduling and "lifetime" support for a type of
> agent, but it doesn't provide any other import features of an agent
> framework;
> e.g., interaction/communicaion between agents.
>
> While I think it's neat to try to exapand addanc, I'm not too sure that
> addanc
> will provide enough support to make this set of agents work.
>
> --
> Joe Griffin
> Addanc
> Downright Software Collective
>
> Quoting Doug Fort <dou...@do...>:
>
> > The current VC is strictly set up for HTTP. I have an itch to scratch that
> > might fit with addanc in a different protocol.
> >
> > As an unemployed drone, I am very interested in receiving my email
> promptly.
> > I
> > have several accounts (4) all forwarded to the Horde server. For various
> > reasons, I sometimes don't get mail from one or more accounts; or it takes
> a
> > long time.
> >
> > When I'm worried about it, I send out test emails to see when I get them.
> I'm
> > thinking about automating this procedure as an ongoing QOS check.
> >
> > The idea is to send out timestamped emails to each of my accounts at some
> > interval, say once an hour. Then have an agent watching my inbox for these
> > mails to record and report the elapsed time. Ideally, there should be some
> > form of alert if an expected mail fails to arrive in a specified interval.
> >
> > I'm contemplating developing this as an exercise. However, in the abstract,
> > the functionality seems analogous to addanc. With the proposed scripting
> > capability, I wonder if addanc could be made flexible enough to handle this
> > requirement.
> >
> > --
> > Doug Fort
> > Addanc
> > Downright Software Collective
> > ------------------------------------------------------------------------
> > This email and any files transmitted with it are confidential and
> > intended solely for the use of the individual or entity to whom they
> > are addressed. If you have received this email in error please notify
> > sec...@do....
> > ------------------------------------------------------------------------
> > This mail sent through Downright Software's implementation of HORDE/IMP.
> > For info, see: http://horde.org/imp/
> >
> >
> >
> > -------------------------------------------------------
> > The SF.Net email is sponsored by EclipseCon 2004
> > Premiere Conference on Open Tools Development and Integration
> > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
> > http://www.eclipsecon.org/osdn
> > _______________________________________________
> > Addanc-developer mailing list
> > Add...@li...
> > https://lists.sourceforge.net/lists/listinfo/addanc-developer
> >
>
> ------------------------------------------------------------------------
> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they
> are addressed. If you have received this email in error please notify
> sec...@do....
> ------------------------------------------------------------------------
> This mail sent through Downright Software's implementation of HORDE/IMP.
> For info, see: http://horde.org/imp/
>
>
>
> -------------------------------------------------------
> The SF.Net email is sponsored by EclipseCon 2004
> Premiere Conference on Open Tools Development and Integration
> See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
> http://www.eclipsecon.org/osdn
> _______________________________________________
> Addanc-developer mailing list
> Add...@li...
> https://lists.sourceforge.net/lists/listinfo/addanc-developer
>


--
Doug Fort
http://www.dougfort.net
------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/

Re: [Addanc-developer] A Possible Addanc Extension

[Joe G. <joe...@us...>]

My first reaction was ... yea, just change the protocol and away you go...

Second thoughts... the mail protocol is not strickly send/receive .. and the
process you describe requires coupling two transactions...while this may be
accomplished in addanc, I think you may rewrite moe than you reuse.

Final thoughts, addanc may not be the best framework for a set of cooperating
agents...it would provide the scheduling and "lifetime" support for a type of
agent, but it doesn't provide any other import features of an agent framework;
e.g., interaction/communicaion between agents.

While I think it's neat to try to exapand addanc, I'm not too sure that addanc
will provide enough support to make this set of agents work.

--
Joe Griffin
Addanc
Downright Software Collective

Quoting Doug Fort <dou...@do...>:

> The current VC is strictly set up for HTTP. I have an itch to scratch that
> might fit with addanc in a different protocol.
>
> As an unemployed drone, I am very interested in receiving my email promptly.
> I
> have several accounts (4) all forwarded to the Horde server. For various
> reasons, I sometimes don't get mail from one or more accounts; or it takes a
> long time.
>
> When I'm worried about it, I send out test emails to see when I get them. I'm
> thinking about automating this procedure as an ongoing QOS check.
>
> The idea is to send out timestamped emails to each of my accounts at some
> interval, say once an hour. Then have an agent watching my inbox for these
> mails to record and report the elapsed time. Ideally, there should be some
> form of alert if an expected mail fails to arrive in a specified interval.
>
> I'm contemplating developing this as an exercise. However, in the abstract,
> the functionality seems analogous to addanc. With the proposed scripting
> capability, I wonder if addanc could be made flexible enough to handle this
> requirement.
>
> --
> Doug Fort
> Addanc
> Downright Software Collective
> ------------------------------------------------------------------------
> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they
> are addressed. If you have received this email in error please notify
> sec...@do....
> ------------------------------------------------------------------------
> This mail sent through Downright Software's implementation of HORDE/IMP.
> For info, see: http://horde.org/imp/
>
>
>
> -------------------------------------------------------
> The SF.Net email is sponsored by EclipseCon 2004
> Premiere Conference on Open Tools Development and Integration
> See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
> http://www.eclipsecon.org/osdn
> _______________________________________________
> Addanc-developer mailing list
> Add...@li...
> https://lists.sourceforge.net/lists/listinfo/addanc-developer
>

------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/

[Addanc-developer] A Possible Addanc Extension

[Doug F. <dou...@do...>]

The current VC is strictly set up for HTTP. I have an itch to scratch that
might fit with addanc in a different protocol.

As an unemployed drone, I am very interested in receiving my email promptly. I
have several accounts (4) all forwarded to the Horde server. For various
reasons, I sometimes don't get mail from one or more accounts; or it takes a
long time.

When I'm worried about it, I send out test emails to see when I get them. I'm
thinking about automating this procedure as an ongoing QOS check.

The idea is to send out timestamped emails to each of my accounts at some
interval, say once an hour. Then have an agent watching my inbox for these
mails to record and report the elapsed time. Ideally, there should be some
form of alert if an expected mail fails to arrive in a specified interval.

I'm contemplating developing this as an exercise. However, in the abstract,
the functionality seems analogous to addanc. With the proposed scripting
capability, I wonder if addanc could be made flexible enough to handle this
requirement.

--
Doug Fort
Addanc
Downright Software Collective
------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/

Re: [Addanc-developer] Fixes for mySQLRecorder support

[Doug F. <dou...@do...>]

I would say IF rather than IFF. That is we, or some other developer, may add a
recorder in the future. I could see Oracle or Jabber. The new recorder should
be able to use TestID and BrickID. I would keep them as global options with a
default of "None".


Quoting Joe Griffin <joe...@us...>:

> I have just checked in updates to brick.py, brickcontrolthread.py,
> instrumentationrecorder.py, mySQLRecorder.py, and fileRecorder.py that
> integrate the mySQL support.
>
> The command line now requires THREE args... the last being a TestID.
>
> The brick will also pick up a brickID from the config file IFF a
> mySQLRecorder
> is configured.
>
> These changes allow tests to be run in both deamon (jabber) and non-deamon
> mode
> while using the mySQLRecorder module.
>
> Couple of comments/questions:
> 1.- The command line arg for TestID should be required IFF the mySQLRecorder
> module is configured.
>
> 2.- The TestID should be an attribute of the Test Object. It is currrently a
> global option. This will require setTestID()/getTestID() functions for the
> test
> object and appropiate changes in the startTest() function in the
> brickcontrolthread.
>
> 3.- Should the BrickID used for mySQLRecorder be a recorder element
> attribute,
> specified in that part of the config file?
>
>
> --
> Joe Griffin
> Addanc
> Downright Software Collective
> ------------------------------------------------------------------------
> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they
> are addressed. If you have received this email in error please notify
> sec...@do....
> ------------------------------------------------------------------------
> This mail sent through Downright Software's implementation of HORDE/IMP.
> For info, see: http://horde.org/imp/
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: SF.net Giveback Program.
> Does SourceForge.net help you be more productive?  Does it
> help you create better code?  SHARE THE LOVE, and help us help
> YOU!  Click Here: http://sourceforge.net/donate/
> _______________________________________________
> Addanc-developer mailing list
> Add...@li...
> https://lists.sourceforge.net/lists/listinfo/addanc-developer
>


--
Doug Fort
http://www.dougfort.net
------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/

[Addanc-developer] Fixes for mySQLRecorder support

[Joe G. <joe...@us...>]

I have just checked in updates to brick.py, brickcontrolthread.py,
instrumentationrecorder.py, mySQLRecorder.py, and fileRecorder.py that
integrate the mySQL support.

The command line now requires THREE args... the last being a TestID.

The brick will also pick up a brickID from the config file IFF a mySQLRecorder
is configured.

These changes allow tests to be run in both deamon (jabber) and non-deamon mode
while using the mySQLRecorder module.

Couple of comments/questions:
1.- The command line arg for TestID should be required IFF the mySQLRecorder
module is configured.

2.- The TestID should be an attribute of the Test Object. It is currrently a
global option. This will require setTestID()/getTestID() functions for the test
object and appropiate changes in the startTest() function in the
brickcontrolthread.

3.- Should the BrickID used for mySQLRecorder be a recorder element attribute,
specified in that part of the config file?


--
Joe Griffin
Addanc
Downright Software Collective
------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/

[Addanc-developer] MySQLRecorder

[Doug F. <dou...@do...>]

I've attached MySQLRecorder.py. It basicaslly works, but I'm having trouble with
MySQL permissions.

--
Doug Fort
Addanc
Downright Software Collective
------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/

[Addanc-developer] Revised Config File

[Doug F. <dou...@do...>]

I've attached the revised config file.

I've got the fileRecorder working and a shell of the mySQLRecorder. I'm in the
process of refactoring instrumentationRecorder to load the recorders and
publish the Txn metrics each one. When that is complete (this afternoon I
hope), I propose to check everything in before starting work on the actual
implementation of mySQLRecorder.

--
Doug Fort
http://www.dougfort.net
------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/

Re: [Addanc-developer] Config File for Recorders

[Doug F. <dou...@do...>]

Quoting Joe Griffin <joe...@us...>:

> Just a couple (four really) of comments/questions:
> 1.- We probably do NOT need a console recorder right away. The
> instrumentation
> data stream is huge...and not very useful on the screen. (Am I interperting
> the
> function of a console recorder correctly?) And, useful info can be obtained
> using tail/grep on the file recorder output.

The consoleRecorder is the fileRecorder with 'print' instead of 'write'. I threw
it in to have a variety of recorders. I won't check it in: it's easy enough to
recreate if the need arises.

>
> 2.- The end of the appsettings should be:
>                 <!-- Non-command line configuration option(s), DO NOT DELETE
> any
> of the following lines -->
>                 <add key="connectthreadcount" value="6" />
>                 <add key="vc_inventory_level" value="4" />
>                 <add key="max_new_vc_queue_depth" value="100" />
>                 <add key="max_run_queue_depth" value="0" />
>                 <add key="max_process_queue_depth" value="0" />
>                 <add key="max_wait_queue_depth" value="0" />
>
> (I may not have checked in an up-to-date .config file for the brick.)
>

I'll add these guys. We'll probably still have to merge the final form. Note
that at one point you suggested a scheme for checking in config files without
trashing users active files. This might be a good point to implement that.

> 3.- Does this mean we can drop the config line:
> <add key="instumentationfile" value="addanc.brick.instrumentation" />
> and have the output handled by the fileRecorder?

Yes

> 4.- If yes, can we have the default name for the fileRecorder match the old
> default, "addanc.brick.instrumentation"?

Affirmative: I'll fix it right now.

--
Doug Fort
http://www.dougfort.net
------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/

Re: [Addanc-developer] Config File for Recorders

[Joe G. <joe...@us...>]

Quoting Doug Fort <dou...@do...>:

> I have attached a proposed modification to the Brick configuration file. This
> declares the recorders to subscribe to InstrumentationRecorder. It may be a
> bit
> much to load the recorders dynamically, however it's a technique I'm
> interested
> in and it will allow us (or others) to add new recorders without changing the
> code.
>
I like the idea of "dynamically" adding recorders...

Just a couple (four really) of comments/questions:
1.- We probably do NOT need a console recorder right away. The instrumentation
data stream is huge...and not very useful on the screen. (Am I interperting the
function of a console recorder correctly?) And, useful info can be obtained
using tail/grep on the file recorder output.

2.- The end of the appsettings should be:
                <!-- Non-command line configuration option(s), DO NOT DELETE any
of the following lines -->
                <add key="connectthreadcount" value="6" />
                <add key="vc_inventory_level" value="4" />
                <add key="max_new_vc_queue_depth" value="100" />
                <add key="max_run_queue_depth" value="0" />
                <add key="max_process_queue_depth" value="0" />
                <add key="max_wait_queue_depth" value="0" />

(I may not have checked in an up-to-date .config file for the brick.)

3.- Does this mean we can drop the config line:
<add key="instumentationfile" value="addanc.brick.instrumentation" />

and have the output handled by the fileRecorder?

4.- If yes, can we have the default name for the fileRecorder match the old
default, "addanc.brick.instrumentation"?


--
Joe Griffin
Addanc
Downright Software Collective
------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/

[Addanc-developer] MySQL Script

[Doug F. <dou...@do...>]

I've attached a script for creating the Instrumentation table using MySQL. It
includes a restricted user for bricks. We (and other Addanc users) are going to
need something like this to start testing a new site.

--
Doug Fort
Addanc
Downright Software Collective
------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/

[Addanc-developer] Config File for Recorders

[Doug F. <dou...@do...>]

I have attached a proposed modification to the Brick configuration file. This
declares the recorders to subscribe to InstrumentationRecorder. It may be a bit
much to load the recorders dynamically, however it's a technique I'm interested
in and it will allow us (or others) to add new recorders without changing the
code.

--
Doug Fort
Addanc
Downright Software Collective
------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/

Re: [Addanc-developer] An ElementTree Compromise...

[Joe G. <joe...@us...>]

Quoting Doug Fort <dou...@do...>:

> I would like to use ElementTree instead of xml.dom.minidom in config.py. I
> will
> retain config.appSettings, which should make it transparent to brick. I will
> add config.tree which gives access to the full tree, which would make
> configuring the reporters much easier. It does mean that ElementTree has to
> be
> installed on every brick.
>
> What do you think?
>
Sounds like the way to go... and we already have a boatload of extras that need
to be installed on each brick, what's one more.

--
Joe Griffin
Addanc
Downright Software Collective
------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/

[Addanc-developer] An ElementTree Compromise...

[Doug F. <dou...@do...>]

I would like to use ElementTree instead of xml.dom.minidom in config.py. I will
retain config.appSettings, which should make it transparent to brick. I will
add config.tree which gives access to the full tree, which would make
configuring the reporters much easier. It does mean that ElementTree has to be
installed on every brick.

What do you think?

--
Doug Fort
Addanc
Downright Software Collective
------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/

[Addanc-developer] Updated Instrumentation Recorder Schema

[Joe G. <joe...@us...>]

See Below - also as an attachment...

--
Joe Griffin
Addanc
Downright Software Collective

Table            Create Table
---------------
---------------------------------------------------------------------------
Instrumentation  CREATE TABLE `Instrumentation` (
                   `instrumentationID` bigint(20) NOT NULL auto_increment,
                   `brickID` text NOT NULL,
                   `testID` text NOT NULL,
                   `scriptID` text NOT NULL,
                   `vcID` text NOT NULL,
                   `pageID` text NOT NULL,
                   `txnID` text NOT NULL,
                   `conStart` double unsigned zerofill default NULL,
                   `conEnd` double unsigned zerofill default NULL,
                   `conTime` double unsigned zerofill default NULL,
                   `sendStart` double unsigned zerofill default NULL,
                   `sendEnd` double unsigned zerofill default NULL,
                   `sendTime` double unsigned zerofill default NULL,
                   `recvStart` double unsigned zerofill default NULL,
                   `recvEnd` double unsigned zerofill default NULL,
                   `recvTime` double unsigned zerofill default NULL,
                   `procStart` double unsigned zerofill default NULL,
                   `procEnd` double unsigned zerofill default NULL,
                   `procTime` double unsigned zerofill default NULL,
                   `txnStart` double unsigned zerofill default NULL,
                   `txnEnd` double unsigned zerofill default NULL,
                   `txnTime` double unsigned zerofill default NULL,
                   `txnSendBytes` bigint(20) unsigned zerofill default NULL,
                   `txnRecvBytes` bigint(20) unsigned zerofill default NULL,
                   `txnHTTPStatus` text,
                   `txnTxnStatus` text,
                   PRIMARY KEY  (`instrumentationID`),
                   KEY `brickID` (`brickID`(25)),
                   KEY `testID` (`testID`(25)),
                   KEY `scriptID` (`scriptID`(25)),
                   KEY `vcID` (`vcID`(25)),
                   KEY `pageID` (`pageID`(25)),
                   KEY `txnID` (`txnID`(25))
                 ) TYPE=MyISAM

------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/

Re: [Addanc-developer] InstrumentationRecorder with MySQL

[Doug F. <dou...@do...>]

Quoting Joe Griffin <joe...@us...>:

> Quoting Doug Fort <dou...@do...>:
>
> > How do you feel about going with ElementTree for the config file?
> >
> If the recorder has a separate config files, GREAT. Otherwise, I'm not quite
> ready to refactor your config file object out of the rest of the code. We'll
> probably still need some sort of 'link' in the legacy file to configure the
> presence of recorders.
>

I think proliferating config files would cause more trouble than it's worth.
I'll stick with the config object in order to maintain a single file.

--
Doug Fort
Addanc
Downright Software Collective
------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/

Re: [Addanc-developer] InstrumentationRecorder with MySQL

[Joe G. <joe...@us...>]

Quoting Doug Fort <dou...@do...>:

> As we discussed, I'd like to have a go at adding MySQL support to
> InstrumentationRecorder. Actually, I have something more fundamental in mind.
> I
> propose to modify InstrumentationRecorder to use the 'Publish/Subscribe"
> design
> pattern to support various 'recorders': ConsoleRecorder, FileRecorder and
> MySQLRecorder.
>
> InstrumentationRecorder would load any or all of these modules based on a
> config
> file, and 'publish' each VC's results to all subscribed recorders. This gives
> us the option to plug in new recorders with a minimum of disturbance
> (OracleRecorder ...).
>
Great design pattern...

> How do you feel about going with ElementTree for the config file?
>
If the recorder has a separate config files, GREAT. Otherwise, I'm not quite
ready to refactor your config file object out of the rest of the code. We'll
probably still need some sort of 'link' in the legacy file to configure the
presence of recorders.

Joe Griffin
Addanc
Downright Software Collective
------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/

[Addanc-developer] InstrumentationRecorder with MySQL

[Doug F. <dou...@do...>]

As we discussed, I'd like to have a go at adding MySQL support to
InstrumentationRecorder. Actually, I have something more fundamental in mind. I
propose to modify InstrumentationRecorder to use the 'Publish/Subscribe" design
pattern to support various 'recorders': ConsoleRecorder, FileRecorder and
MySQLRecorder.

InstrumentationRecorder would load any or all of these modules based on a config
file, and 'publish' each VC's results to all subscribed recorders. This gives
us the option to plug in new recorders with a minimum of disturbance
(OracleRecorder ...).

How do you feel about going with ElementTree for the config file?

--
Doug Fort
Addanc
Downright Software Collective
------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/

Re: [Addanc-developer] ElementTree

[Joe G. <jgr...@do...>]

>
> I've been looking at ElementTree by Frederik Lundh
>
> http://effbot.org/zone/element-index.htm
>
> It's a simplified hierarchy of Python containers that can hold XML. We wrote
> something similar (but cruder) for StressMy.com. If we're going to make
> extensive use of XML in addanc, it may be worth incorporating ElementTree.
>
Thanks for passing that on ... it could be of great use for the script object. I
have to look into re-factoring it in after completing the high volume testing.

(BTW, I've run a couple of test run and found some problems when writing to
queues with nowait set... and so is another thread... gets returned as
Queue.Full... so ALL blocking is the same... locks and queue capacity... one
more to fix and I'm ready for another test... I have a report I'm getting ready
to post as a html page on malfoy...)


> I got into this by way of PythonDoc http://effbot.org/zone/pythondoc.htm
> which
> we may also want to incorporate.
>
This looks cool... have you run it against the current codebase?

Joe Griffin
Addanc
Downright Software Collective

Current MeatSpace coordinates:
    38 degrees, 048.009 minutes N latitude
   078 degrees, 047.437 minutes W longitude
Nearest Geographical Feature: Downright Basye Labs

------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/

[Addanc-developer] ElementTree

[Doug F. <dou...@do...>]

I've been looking at ElementTree by Frederik Lundh

http://effbot.org/zone/element-index.htm

It's a simplified hierarchy of Python containers that can hold XML. We wrote
something similar (but cruder) for StressMy.com. If we're going to make
extensive use of XML in addanc, it may be worth incorporating ElementTree.

I got into this by way of PythonDoc http://effbot.org/zone/pythondoc.htm which
we may also want to incorporate.

--
Doug Fort
Addanc
Downright Software Collective
------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/

[Addanc-developer] Full implementation of maketree

[Doug F. <dou...@do...>]

I have checked in a working version of the 'maketree' utility. It takes two
required arguments

--template an XML file describing a directory tree (see sample.xml in CVS)
--startdir the directory where the tree is to be created

--
Doug Fort
Addanc
Downright Software Collective
------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/

[Addanc-developer] 'maketree' utility

[Doug F. <dou...@do...>]

I am developing the utility program that we discussed. This program will read an
XML file that lays out a directory tree and create that tree on disk. (See
attached sample file).

The purpose of this program is to create a shell image of a site to be tested.
This enables us to test addanc scripts and bricks before applying them to a
'live' site.

I propose to call the program 'maketree'. It will have two mandatory arguments:
-f --file the name of the XML file. -r --root the path where the root directory
will be created.

Eventually I would like to add the capability to handle this sort of file to
config.py, but for now it is going to be separate.
--
Doug Fort
Addanc
Downright Software Collective
------------------------------------------------------------------------
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
sec...@do....
------------------------------------------------------------------------
This mail sent through Downright Software's implementation of HORDE/IMP.
For info, see: http://horde.org/imp/