phpesp-devel

[phpesp-dev] Respondent database

[Creative M. <Cre...@Pu...>]

Hi There,

I see a lack of functionality in the respondent database. First of all, normally when 
you conduct a study you would not add respondents one by one but rather want to upload 
an entire set of respondents. May I recommend using a non-predefined format for this. 
Respondent information can be very different from survey to survey.

I think you should focus on a Comma Separated file for upload with first line 
containing variable name. After that a guide (as in Excel for instance) where you 
define what kind of variable is in the field.

This would be a good next phase in development before we head on to quota management 
etc. Because if have a survey where you need 500 males and 500 females, you should be 
able to create a field inside the survey which then stores values to the respondent 
database and not just to a datafile. And inside the questionnaire you should be able to 
check whether you have or do not have enough respondents of a certain quota group and 
make a condition to end the survey for that respondent with the motivation that the 
quota group is full.

Any opinions?

// Creative Minds

Re: [phpesp-dev] Respondent database

[James E. F. <jf...@ac...>]

[ Please try to make your email wrap at something slightly
less than 80 characters, thanks. ]

A few points here. Having the data fields associated with
each respondent be arbitrarily chosen by the designer is
difficult. Databases are designed to, and work best when,
the fields are pre-determined. Adding a new table per
survey to store this information is a bad idea for several
reasons. The other option is to somehow store the data in
one (or two) predefined tables -- this is sort of how the
survey questions work, its dirty and slow.

Having a way to add respondents in bulk is certainly a good
idea. I agree that an option to import them from either
text (csv or xml) would be helpful.

When you are analyzing the data, it should be very easy to
include the rest of your respondent information. One of the
great thing about relational databases is that you can join
tables. Simple use one of the phpESP fields (username,
email, etc) as a key into your separate database with all
your custom data.

I am constantly amazed by people wanting to take the data
out of a database and put it in excel. To me it seems that
in doing so, you give up so many tools available in SQL. Oh
well, I suppose it's because excel is pointy-clicky, and to
take full advantage of a database requires substantial
knowledge of SQL. It is certainly well worth learning if
you need to do any substantial data analysis. There are
also several tools to help you with this, I know that SAS
can use an ODBC database directly as a data source.

(More at a later date on quota management.)

-James

On Sat, 22 Jun 2002, Creative Minds wrote:

> Hi There,
>
> I see a lack of functionality in the respondent
> database. First of all, normally when you conduct a
> study you would not add respondents one by one but
> rather want to upload an entire set of respondents. May
> I recommend using a non-predefined format for this.
> Respondent information can be very different from
> survey to survey.
>
> I think you should focus on a Comma Separated file for
> upload with first line containing variable name. After
> that a guide (as in Excel for instance) where you
> define what kind of variable is in the field.
>
> This would be a good next phase in development before
> we head on to quota management etc. Because if have a
> survey where you need 500 males and 500 females, you
> should be able to create a field inside the survey
> which then stores values to the respondent database and
> not just to a datafile. And inside the questionnaire
> you should be able to check whether you have or do not
> have enough respondents of a certain quota group and
> make a condition to end the survey for that respondent
> with the motivation that the quota group is full.
>
> Any opinions?

Re: [phpesp-dev] Respondent database

[Creative M. <Cre...@Pu...>]

[Sorry about the e-mail wrapping =)]

A couple of issues on your comments...

2002-06-23 16:37:11, "James E. Flemer" <jf...@ac...> wrote:

>A few points here. Having the data fields associated with
>each respondent be arbitrarily chosen by the designer is
>difficult. Databases are designed to, and work best when,
>the fields are pre-determined. Adding a new table per
>survey to store this information is a bad idea for several
>reasons. The other option is to somehow store the data in
>one (or two) predefined tables -- this is sort of how the
>survey questions work, its dirty and slow.

I must say I'm a bit confused here. You refer to two tables
as a dirty and slow solution or the "none" predefined?

I would imagine a new table for each survey (to store data, 
not descriptions) would be very fast. What I figure here is
that a table exists which describes all surveys in the system.
This table has a unique identifier for each project, a project
number so to say. This number could be used in an SQL statement
as the table name in the creation process (possibly in a respondent
Dbase???). After that, the fields in this table can be unique as
it is defined by a new create statement. When the project is put to
Inactive or actually to a later status --> Archived, the table could
be restructured into an archiving table containing one row for 
each field/row in the respondent base.

Then we could keep the system clean of two many tables and still gain the 
flexibility of having special respondent information. If this idea is
completely out the window, I would recommend a table with load of fields
so it can be adapted to different sets of information.

>Having a way to add respondents in bulk is certainly a good
>idea. I agree that an option to import them from either
>text (csv or xml) would be helpful.

=)

>I am constantly amazed by people wanting to take the data
>out of a database and put it in excel. To me it seems that
>in doing so, you give up so many tools available in SQL. Oh
>well, I suppose it's because excel is pointy-clicky, and to
>take full advantage of a database requires substantial
>knowledge of SQL.

Oh but you don't see my friend, Excel is not where the information
is placed in the next phase, but into a powerful analyses program
such as SPSS or Answertree or SAS or for a tabulation package which
offers true power such as Diana or Quantum.

>(More at a later date on quota management.)

Look forward to it.

If you're interested to see some of the leading MR applications
I can send you thorough documentation of how it works and some
demo versions. If you have Remote Administrator from Famatech
I can show you some ASP solutions similar to this one.

Interested?

// Creative Minds


>
>-James
>
>On Sat, 22 Jun 2002, Creative Minds wrote:
>
>> Hi There,
>>
>> I see a lack of functionality in the respondent
>> database. First of all, normally when you conduct a
>> study you would not add respondents one by one but
>> rather want to upload an entire set of respondents. May
>> I recommend using a non-predefined format for this.
>> Respondent information can be very different from
>> survey to survey.
>>
>> I think you should focus on a Comma Separated file for
>> upload with first line containing variable name. After
>> that a guide (as in Excel for instance) where you
>> define what kind of variable is in the field.
>>
>> This would be a good next phase in development before
>> we head on to quota management etc. Because if have a
>> survey where you need 500 males and 500 females, you
>> should be able to create a field inside the survey
>> which then stores values to the respondent database and
>> not just to a datafile. And inside the questionnaire
>> you should be able to check whether you have or do not
>> have enough respondents of a certain quota group and
>> make a condition to end the survey for that respondent
>> with the motivation that the quota group is full.
>>
>> Any opinions?
>
>

Re: [phpesp-dev] Respondent database

[Matthew G. <gr...@mu...>]

On Sun, Jun 23, 2002 at 10:37:11AM -0400, James E. Flemer wrote:
> [ Please try to make your email wrap at something slightly
> less than 80 characters, thanks. ]
> 
> A few points here. Having the data fields associated with
> each respondent be arbitrarily chosen by the designer is
> difficult. Databases are designed to, and work best when,
> the fields are pre-determined. Adding a new table per
> survey to store this information is a bad idea for several
> reasons. The other option is to somehow store the data in
> one (or two) predefined tables -- this is sort of how the
> survey questions work, its dirty and slow.
> 
> Having a way to add respondents in bulk is certainly a good
> idea. I agree that an option to import them from either
> text (csv or xml) would be helpful.
Yup this would be worthwhile.  I'll add a feature request and work on 
implementing.

> 
> When you are analyzing the data, it should be very easy to
> include the rest of your respondent information. One of the
> great thing about relational databases is that you can join
> tables. Simple use one of the phpESP fields (username,
> email, etc) as a key into your separate database with all
> your custom data.
This is exactly the way to go.  We need to be careful not to "feature
creep" a great survey tool into a lousy statistics package :)

> 
> I am constantly amazed by people wanting to take the data
> out of a database and put it in excel. To me it seems that
> in doing so, you give up so many tools available in SQL. Oh
> well, I suppose it's because excel is pointy-clicky, and to
> take full advantage of a database requires substantial
> knowledge of SQL. 
The answer to this is simple...  People know Excel, it's easy to use and
they aren't will or don't have the time to learning anything else.

> It is certainly well worth learning if
> you need to do any substantial data analysis. There are
> also several tools to help you with this, I know that SAS
> can use an ODBC database directly as a data source.
> 
> (More at a later date on quota management.)
> 
> -James
> 
> On Sat, 22 Jun 2002, Creative Minds wrote:
> 
> > Hi There,
> >
> > I see a lack of functionality in the respondent
> > database. First of all, normally when you conduct a
> > study you would not add respondents one by one but
> > rather want to upload an entire set of respondents. May
> > I recommend using a non-predefined format for this.
> > Respondent information can be very different from
> > survey to survey.
> >
> > I think you should focus on a Comma Separated file for
> > upload with first line containing variable name. After
> > that a guide (as in Excel for instance) where you
> > define what kind of variable is in the field.
> >
> > This would be a good next phase in development before
> > we head on to quota management etc. Because if have a
> > survey where you need 500 males and 500 females, you
> > should be able to create a field inside the survey
> > which then stores values to the respondent database and
> > not just to a datafile. And inside the questionnaire
> > you should be able to check whether you have or do not
> > have enough respondents of a certain quota group and
> > make a condition to end the survey for that respondent
> > with the motivation that the quota group is full.
> >
> > Any opinions?
> 
> 
> 
> Sponsored by:
> ThinkGeek at http://www.ThinkGeek.com/
> _______________________________________________
> phpESP-devel mailing list
> php...@li...
> https://lists.sourceforge.net/lists/listinfo/phpesp-devel
> 

-- 
brought to you by, Matthew Gregg...
one of the friendly folks in the IT Lab.
--------------------------------------\
The IT Lab (http://www.itlab.musc.edu) \____________________
Probably the world's premier software development center.
Serving: Programming, Tools, Ice Cream, Seminars