openwebwork-devel — WeBWorK development discussion

Re: [WWdevel] Re: [Spam:******** SpamScore] [WWcvs] webwork2: Moved stylesheet into a separate file.

[Sam H. <sh...@ma...>]

On Jan 28, 2005, at 11:01 AM, Davide P.Cervone wrote:

>> Log Message:
>> -----------
>> Moved stylesheet into a separate file. Created new htdocs/css 
>> directory,
>> added stylesheet to that directory (ur.css). Include stylesheet in
>> ur.template with new "url" template escape:
>>
>> <!--#url type="webwork" name="stylesheet"-->
>>
>> This is 132 lines that Template.pm doesn't have to parse and 4368 
>> bytes
>> that don't have to get sent to the client with every request.
>>
>> Modified Files:
>> --------------
>>     webwork2/conf/templates:
>>         ur.template
>>     webwork2/lib/WeBWorK:
>>         ContentGenerator.pm
>>
>> Added Files:
>> -----------
>>     webwork2/htdocs/css:
>>         ur.css
>
> Sam:
>
> I think this also requires an addition to global.conf to define a 
> webworkURL for stylesheet.  It looks like this hasn't been added to 
> the CVS tree either in HEAD or in the backport.
>
> I noticed this because I recently did a CVS update in my testbed 
> server (that is supposed to be just a plain copy of the HEAD 
> revision), and it started to produce error messages.  I tracked it 
> down to the missing webworkURL value.

Yeah, you're right. I added it to global.conf but not global.conf.dist. 
Whoops!
-sam

Re: [WWdevel] professor accounts

[B. D. <b.d...@cs...>]

Thanks, Sam.  That works a charm.

>On Jan 26, 2005, at 12:12 PM, B. Duffee wrote:
>
>> I'm having a little trouble creating professor accounts from the  
>> system admin
>> webpage.  I have managed to log in as the user, but it immediately  
>> comes up with
>> "User is not authorized to create or delete courses".  I've tried  
>> setting the
>> permission levels up to 7.  Should I go up to 10 or is there something  
>> I'm
>> missing?
>
>Hi,
>
>The default permission level for professors is 10. By default, only  
>professors are allowed to administrate courses. This is controlled in  
>the "Authorization system" section of the global.conf file. The example  
>below if from the development version -- yours may look slightly  
>different.
>
>global.conf reads:
>
>> ####################################################################### 
>> #########
>> # Authorization system
>> ####################################################################### 
>> #########
>>
>> # This lets you specify a minimum permission level needed to perform  
>> certain
>> # actions. For each pair in the hash below, in order to perform the  
>> action
>> # described by the key, the user must have a permission level greater  
>> than or
>> # equal to the value.
>>
>> my $guest     = -1;
>> my $student   = 0;
>> my $ta        = 5;
>> my $professor = 10;
>> my $nobody    = undef;
>>

>Hope this helps.
>-sam
>

---
Boyd Duffee,		Keele University Computer Science (01782) 583437 
Computing Officer	BScH, BOFH, tinLC, ROTFLMAO
  I told Inland Revenue that I didn't owe them a penny
  since I lived by the seaside. -- Ken Dodd.

[WWdevel] Re: [Spam:******** SpamScore] [WWcvs] webwork2: Moved stylesheet into a separate file.

[Davide P.C. <dp...@un...>]

> Log Message:
> -----------
> Moved stylesheet into a separate file. Created new htdocs/css 
> directory,
> added stylesheet to that directory (ur.css). Include stylesheet in
> ur.template with new "url" template escape:
>
> <!--#url type="webwork" name="stylesheet"-->
>
> This is 132 lines that Template.pm doesn't have to parse and 4368 bytes
> that don't have to get sent to the client with every request.
>
> Modified Files:
> --------------
>     webwork2/conf/templates:
>         ur.template
>     webwork2/lib/WeBWorK:
>         ContentGenerator.pm
>
> Added Files:
> -----------
>     webwork2/htdocs/css:
>         ur.css

Sam:

I think this also requires an addition to global.conf to define a 
webworkURL for stylesheet.  It looks like this hasn't been added to the 
CVS tree either in HEAD or in the backport.

I noticed this because I recently did a CVS update in my testbed server 
(that is supposed to be just a plain copy of the HEAD revision), and it 
started to produce error messages.  I tracked it down to the missing 
webworkURL value.

Davide

Re: [WWdevel] preparing for WeBWorK 2.1.1

[John J. <jj...@as...>]

Sam Hathaway wrote:

> I've backported a good number of bugfixes from HEAD to rel-2-1-patches 
> in preparation for tagging a 2.1.1 release. I've attached a list of 
> the changes that have and have not been backported. Over the next few 
> days, I'll be testing rel-2-1-patches to ensure that it is stable.
>
> If you have some time, please test this branch as well so we can make 
> sure nothing falls through the cracks. If you have a favorite bug 
> you'd like to see fixed in 2.1.1, please fix it or let me know about it. 

Yesterday I had someone come to me with a variety of bugs on the scoring 
page.

   1. There are 4 checkboxes.  Their values are not sticky.
   2. The two middle checkboxes are checked by default (ok so far), but
      if you uncheck them it does no good.  The columns are still produced.
   3. If you forget to check some sets to score, you get a webwork error
      (with stack trace).  You should get a more pleasant "badmessage".
   4. The cvs files have padding in them to right justify the columns. 
      This messes up sorting them in at least some spreadsheets.  Since
      webwork doesn't give you the option to sort the lines for you, you
      pretty much need to sort them in your spreadsheet.

The first three are clear bugs which I can fix when I get around to it.  
The last one may be a big more of a judgement call.

The last one is there presumably to make the scoring output look nice 
when it is displayed in a pre group on the page.  I think the right fix 
would be to not pad when creating the file (that much is easy).  Then 
when producing a copy of the cvs file on the web page, padding would 
have to be added at that stage.  So, that step is not a simple read file 
and dump its output on the screen, but it should still should not be too 
complicated.

A second way to fix this which is simpler, but less pleasing (in terms 
of programming aesthetics) would be to generate two copies of cvs files 
when scoring - one without padding for download and one with padding for 
display.

In addition to removing the padding, we may want to add the possibility 
of sorting lines of the spreadsheet (allow specifying up to three 
columns to sort on).

Thoughts on the .csv files?

John

[WWdevel] preparing for WeBWorK 2.1.1

[Sam H. <sh...@ma...>]

I've backported a good number of bugfixes from HEAD to rel-2-1-patches 
in preparation for tagging a 2.1.1 release. I've attached a list of the 
changes that have and have not been backported. Over the next few days, 
I'll be testing rel-2-1-patches to ensure that it is stable.

If you have some time, please test this branch as well so we can make 
sure nothing falls through the cracks. If you have a favorite bug you'd 
like to see fixed in 2.1.1, please fix it or let me know about it.

Thanks for your help!
-sam

Re: [WWdevel] professor accounts

[Sam H. <sh...@ma...>]

On Jan 26, 2005, at 12:12 PM, B. Duffee wrote:

> I'm having a little trouble creating professor accounts from the  
> system admin
> webpage.  I have managed to log in as the user, but it immediately  
> comes up with
> "User is not authorized to create or delete courses".  I've tried  
> setting the
> permission levels up to 7.  Should I go up to 10 or is there something  
> I'm
> missing?

Hi,

The default permission level for professors is 10. By default, only  
professors are allowed to administrate courses. This is controlled in  
the "Authorization system" section of the global.conf file. The example  
below if from the development version -- yours may look slightly  
different.

global.conf reads:

> ####################################################################### 
> #########
> # Authorization system
> ####################################################################### 
> #########
>
> # This lets you specify a minimum permission level needed to perform  
> certain
> # actions. For each pair in the hash below, in order to perform the  
> action
> # described by the key, the user must have a permission level greater  
> than or
> # equal to the value.
>
> my $guest     = -1;
> my $student   = 0;
> my $ta        = 5;
> my $professor = 10;
> my $nobody    = undef;
>
> %permissionLevels = (
> 	login                          => $guest,
> 	report_bugs                    => $student,
> 	submit_feedback                => $student,
> 	change_password                => $student,
> 	change_email_address           => $student,
> 	
> 	view_multiple_sets             => $ta,
> 	view_unopened_sets             => $ta,
> 	view_unpublished_sets          => $ta,
> 	view_answers                   => $ta,
> 	
> 	become_student                 => $professor,
> 	access_instructor_tools        => $ta,
> 	score_sets                     => $professor,
> 	send_mail                      => $professor,
> 	receive_feedback               => $ta,
> 	
> 	create_and_delete_problem_sets => $professor,
> 	assign_problem_sets            => $professor,
> 	modify_problem_sets            => $professor,
> 	modify_student_data            => $professor,
> 	modify_classlist_files         => $professor,
> 	modify_set_def_files           => $professor,
> 	modify_scoring_files           => $professor,
> 	modify_problem_template_files  => $professor,
> 	
> 	create_and_delete_courses      => $professor,
> 	fix_course_databases           => $professor,
> 	
> 	##### Behavior of the interactive problem processor #####
> 	
> 	show_correct_answers_before_answer_date         => $ta,
> 	show_solutions_before_answer_date               => $ta,
> 	avoid_recording_answers                         => $ta,
> 	check_answers_before_open_date                  => $ta,
> 	check_answers_after_open_date_with_attempts     => $ta,
> 	check_answers_after_open_date_without_attempts  => $guest,
> 	check_answers_after_due_date                    => $guest,
> 	check_answers_after_answer_date                 => $guest,
> 	record_answers_when_acting_as_student           => $nobody,
> 	# "record_answers_when_acting_as_student" takes precedence
> 	# over the following for professors acting as students:
> 	record_answers_before_open_date                 => $nobody,
> 	record_answers_after_open_date_with_attempts    => $student,
> 	record_answers_after_open_date_without_attempts => $nobody,
> 	record_answers_after_due_date                   => $nobody,
> 	record_answers_after_answer_date                => $nobody,
> 	dont_log_past_answers                           => $professor,
> 	
> 	##### Behavior of the Hardcopy Processor #####
> 	
> 	download_hardcopy_multiuser  => $ta,
> 	download_hardcopy_multiset   => $ta,
> 	download_hardcopy_format_tex => $ta,
> );

The %permissionLevels hash indicates the minimum permission level  
necessary to perform an action in the system. In this example, the  
"create_and_delete_courses" action requires at least the permission  
level $professor. The variable $professor has been assigned a value of  
10, above:

> my $professor = 10;

Hope this helps.
-sam

[WWdevel] professor accounts

[B. D. <b.d...@cs...>]

I'm having a little trouble creating professor accounts from the system admin 
webpage.  I have managed to log in as the user, but it immediately comes up with 
"User is not authorized to create or delete courses".  I've tried setting the 
permission levels up to 7.  Should I go up to 10 or is there something I'm 
missing?

thanks,
---
Boyd Duffee,		Keele University Computer Science (01782) 583437 
Computing Officer	BScH, BOFH, tinLC, ROTFLMAO
  I wish there was a knob on the TV to turn up the intelligence. There's 
  a knob called "brightness", but it doesn't seem to work. -- Gallagher

[WWdevel] Re: CVS alternative

[John J. <jj...@as...>]

William Ziemer wrote:

> Just found time to look over arch.  It is very cool!
> Perhaps more importantly for me, I have never understood cvs 
> completely, but arch makes perfect sense.

I set it up once and got the basic thing to compile but failed to get it 
to compile with security settings turned on.  It was increasingly 
frustrating that it loves weird characters in filenames (weird meaning 
that they needed quoting when issuing shell commands).

Have you tried it?

John



> On Jan 10, 2005, at 9:16 AM, Sam Hathaway wrote:
>
>>
>> On Jan 10, 2005, at 12:08 PM, Michael Gage wrote:
>>
>>> We've been considering this, but haven't made any moves yet -- 
>>> mostly because we've
>>> had other things to do.  Sam, John, do you have any comments?
>>
>>
>> Arch is more ambitious and has a greater "cool" factor, but I think 
>> Subversion is the way to go right now, especially since it's possible 
>> to convert a repository from CVS to Subversion: 
>> <http://svnbook.red-bean.com/en/1.0/apas11.html>.
>> -sam
>>
>>> On Jan 10, 2005, at 11:52 AM, William Ziemer wrote:
>>>
>>>> Most of the projects I link with are going to subversion:
>>>> http://subversion.tigris.org/
>>>> Maybe use this for the problem database?
>>>> Good to see you all again,
>>>> Bill
>>>
>>
>>
> -- 
> William Ziemer
> Mathematics Department
> CSULB

[WWdevel] Re: CVS alternative

[William Z. <wz...@cs...>]

Just found time to look over arch.  It is very cool!
Perhaps more importantly for me, I have never understood cvs 
completely, but arch makes perfect sense.


On Jan 10, 2005, at 9:16 AM, Sam Hathaway wrote:

>
> On Jan 10, 2005, at 12:08 PM, Michael Gage wrote:
>
>> We've been considering this, but haven't made any moves yet -- mostly 
>> because we've
>> had other things to do.  Sam, John, do you have any comments?
>
> Arch is more ambitious and has a greater "cool" factor, but I think 
> Subversion is the way to go right now, especially since it's possible 
> to convert a repository from CVS to Subversion: 
> <http://svnbook.red-bean.com/en/1.0/apas11.html>.
> -sam
>
>> On Jan 10, 2005, at 11:52 AM, William Ziemer wrote:
>>
>>> Most of the projects I link with are going to subversion:
>>> http://subversion.tigris.org/
>>> Maybe use this for the problem database?
>>> Good to see you all again,
>>> Bill
>
>
--
William Ziemer
Mathematics Department
CSULB

[WWdevel] page contents/organization/versioned sets

[P G. L. <gl...@um...>]

Hi all,

(Is that a vague enough subject line?)

I'm starting to get geared up for more work on support for versioned sets
and gateway tests in WeBWorK, and have a couple of things that I'd welcome
others' comments on.

 - When viewing students' progress by set, I think there is more
   information than is necessarily useful in most cases.  In particular,
   I think it would be useful to be able to suppress columns, not unlike
   the filter that now exists for the Instructor Tools page, et al.  For
   gateway tests, I'd like the ability to suppress everything after the
   "OutOf" column.  Any thoughts?  Do other people find columns like
   "Ind" or "LoginName" essential (or even useful)?

 - Along the same lines, if one is only interested in the total score on
   an assignment, it becomes possible to display multiple assignments in a
   single table.  Actually, this is probably possible no matter what,
   though it's easier to see if we leave out the per problem information.
   For example,
                        Set1         Set2
      Name          Score OutOf  Score OutOf   Section  Recitation
      Gavin LaRose    8     8      10   10       000       000
      etc.

   This does require a way of selecting multiple sets for viewing, though.
   The ability to do this is desirable for gateway tests, where we
   routinely have two assignments, one for practicing the test and one for
   taking it.  Does anyone else think this would be useful?

 - More generally, I think it would be useful to have a "professor of a
   specific section" authorization level.  I'm thinking of the case of a
   multi-section course, where we want a "professor" (the course
   coordinator) to see everything, but generally want other professors to
   only see and work with students in their section.  I think Mark may
   have done something like this somewhere along the line.  This would
   require a bunch of new permission levels, however, viz.,

        become_student_in_section      => $sectionprofessor,
        send_mail_to_section           => $sectionprofessor,
        assign_problem_sets_in_section => $sectionprofessor,
        modify_problem_sets_in_section => $sectionprofessor,
        modify_student_data_in_section => $sectionprofessor,
        etc.,

   and then would require that authorization checks verify
        has_auth( 'become_student' ) ||
        ( has_auth( 'become_student_in_section' ) && (in_same_section) )
   Display of students in various pages (e.g., student progress) would
   also want to be limited to those students in the correct section.  It
   also becomes more complicated if an instructor is teaching two sections
   of a course.

   Thoughts?

 - For gateway tests, I'm going to rearrange the preview/check output.
   Currently, previewing gives the following display:
           -----------------------------------------------------------
           | parsed input    |  formatted input     |  answer result |
           -----------------------------------------------------------
       1. text of problem
           input box: [ student input ]

   I think I like the following output better
       1. text of problem
           input box: [ student input ]
          answer result
           input read as: parsed input
           input preview: formatted input

   (This is at least partly because for gateway tests we're putting
   multiple problems on one page.)  Is there any reason that I'm missing
   that argues that the current arrangement is a better interface?  (When
   problems are presented one-by-one it brings the results to the fore,
   which is nice, but it still seems a little odd to me that it's coming
   before the problem that's being considered.)

Thanks,
Gavin

-- 
P. Gavin LaRose, Ph.D.  Program Manager (Instructional Tech.)
Math Dept., University of Michigan
gl...@um...                         "It's snowing still.  And freezing.
734.764.6454                              However,  we  haven't had an earth-
http://www.math.lsa.umich.edu/~glarose/	  quake lately."         - A.A. Milne

Re: [WWdevel] Re: [WWcvs] webwork-modperl: Commented out escapeHTML() of warning messages since it

[Michael G. <ga...@ma...>]

I get the same phenomenon for warnings within problems themselves.

Commenting out line 797  in pg/lib/WeBWorK/PG/Translator.pm   restores 
the warning messages from problems.

We'll need to look at the whole warning mechanism again.  Maybe we can 
devise some unit tests
to make sure existing warning functions don't break when we add others.

Can't do that this second though.

Take care,

Mike


On Jan 15, 2005, at 6:20 PM, John Jones wrote:

> Michael Gage wrote:
>
>> I'm not sure how to resolve this.  We use "warn" routinely inside 
>> problems to
>> help write and debug problems.  pretty_print_rh formats complicated
>> structures in tables.  This is very useful and I'd rather not loose 
>> it.
>>
>> Hard coded warn messages can have < > manually replaced by &lt; and 
>> &gt; but
>> there is a problem if the warn message has a variable.
>
> Being able to use warn statements from the pg side is useful, but all 
> times I had done it recently didn't work.  I higher level webwork 
> error is generated and you don't get the output of the warn.  Does it 
> currently work for you?
>
> John
>
>
>
> -------------------------------------------------------
> The SF.Net email is sponsored by: Beat the post-holiday blues
> Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
> It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt
> _______________________________________________
> OpenWeBWorK-Devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/openwebwork-devel
>

Re: [WWdevel] Re: [WWcvs] webwork-modperl: Commented out escapeHTML() of warning messages since it

[John J. <jj...@as...>]

Michael Gage wrote:

> I'm not sure how to resolve this.  We use "warn" routinely inside 
> problems to
> help write and debug problems.  pretty_print_rh formats complicated
> structures in tables.  This is very useful and I'd rather not loose it.
>
> Hard coded warn messages can have < > manually replaced by &lt; and 
> &gt; but
> there is a problem if the warn message has a variable.

Being able to use warn statements from the pg side is useful, but all 
times I had done it recently didn't work.  I higher level webwork error 
is generated and you don't get the output of the warn.  Does it 
currently work for you?

John

Re: [WWdevel] Re: [WWcvs] webwork-modperl: Commented out escapeHTML() of warning messages since it

[Michael G. <ga...@ma...>]

I'm not sure how to resolve this.  We use "warn" routinely inside 
problems to
help write and debug problems.  pretty_print_rh formats complicated
structures in tables.  This is very useful and I'd rather not loose it.

Hard coded warn messages can have < > manually replaced by &lt; and 
&gt; but
there is a problem if the warn message has a variable.

I suppose that the long term solution is to build a protected print or 
debugging statement
within the PG environment that allows us to produce debugging output 
without using the warm
mechanism.

For now I'd like to leave it the way it is, with HTML allowed in warn 
messages, and you have to
do your own escape.  If this causes too many problems I'll up the 
priority of finding another method
for reporting debugging errors in problems.

Take care,

Mike

On Jan 15, 2005, at 2:51 PM, Samuel Hathaway wrote:

> On Sat, 15 Jan 2005, Mike Gage via activitymail wrote:
>
>> Commented out escapeHTML() of warning messages since it prevented
>> using tables when outputing info about answer evaluators.
>>
>> Can someone tell me whether there is another reason why not allowing
>> HTML in warning messages is or would be important?  If so we can
>> figure out another way to debug answer evaluators.
>
> The intent is not to prohibit HTML in warning messages but to allow the
> literal display of characters like '<' and '&' in warnings.
>
> It's going to be a problem if we need to pass HTML through the warning
> facility, since there would be no way for the warning handler to
> differentiate between a warning that just happened to use characters 
> like
> '<' and '&' (say, generated by Perl), and a warning that has purposeful
> HTML in it that ought to be passed through unmolested.
> -sam
>

[WWdevel] Re: [WWcvs] webwork-modperl: Commented out escapeHTML() of warning messages since it

[Samuel H. <sh...@ma...>]

On Sat, 15 Jan 2005, Mike Gage via activitymail wrote:

> Commented out escapeHTML() of warning messages since it prevented
> using tables when outputing info about answer evaluators.
>
> Can someone tell me whether there is another reason why not allowing
> HTML in warning messages is or would be important?  If so we can
> figure out another way to debug answer evaluators.

The intent is not to prohibit HTML in warning messages but to allow the
literal display of characters like '<' and '&' in warnings.

It's going to be a problem if we need to pass HTML through the warning
facility, since there would be no way for the warning handler to
differentiate between a warning that just happened to use characters like
'<' and '&' (say, generated by Perl), and a warning that has purposeful
HTML in it that ought to be passed through unmolested.
-sam

Re: [WWdevel] Re: CVS alternative

[John J. <jj...@as...>]

Sam Hathaway wrote:

> On Jan 10, 2005, at 5:47 PM, John Jones wrote:
>
>> Sam Hathaway wrote:
>>
>>> On Jan 10, 2005, at 4:13 PM, Michael Gage wrote:
>>>
>>> Can anyone give me more details on how the repository of problem 
>>> sources and the "database" will interact? Based on what little I 
>>> know, it seems to me that the problem source should be part of the 
>>> problem's database record.
>>
>>
>> In a sense, things are reversed.  The problem files initially contain 
>> all of the information; the extra information comes in the form of 
>> special comments in those files.  We then have a script to set up a 
>> mysql database, and to extract the information from the files and 
>> load it into the database.
>
>
> Would it be fair to say that the MySQL database does nothing more than 
> act as an index on the metadata associated with each problem? Or am I 
> missing something?

I think that is a reasonable description.

>> I like this approach since it is easy to reload the database if 
>> something goes wrong, and we are shipping mainly flat text files 
>> (except for the images).
>
>
> I like the simplicity of this, and in a distributed system like this 
> the more we can do with a version control system the better.
>
>>> By the way, has anyone thought about how problems will be packaged? 
>>> Many problems consist of more than one file and it might be worth 
>>> laying out a packaging format, so that a problem and all of its 
>>> auxiliary files and metadata can be distributed as a single file.
>>
>>
>> I hadn't thought of the extra files.  Thus far, the problems were 
>> basically not packaged in any special way.
>>
>> The distribution method I had in mind was that webwork would handle 
>> it behind the scenes.  It would fetch files over http from perl (I 
>> think the perl module is LWP, or something like that).  The entry 
>> point would be an extra tab in the admin course (along with add 
>> course, ..., and then Problem Database).  If you ask it to update 
>> your Problem Library Database, then it fetches the current list of 
>> files/version via http, checks it against your current list, and gets 
>> whatever is new and reloads the database.
>
>
> Shouldn't we leverage the version control system checkout features to 
> fetch and update problem libraries? It seems like a waste to keep the 
> problems in CVS (or Subversion) and then ignore the versioning 
> features of that system and track versions separately and fetch via HTTP.

At one time, there were two reasons for this.  Maybe neither is compelling.

The first was that I thought people might now want the most current 
version of some problems.  This would introduce various complications.  
After talking with Bill about this in Atlanta, we decided against this.  
Along with that, if someone modifies a problem and submits the change 
and it isn't a strict improvement, then we would fork the problem rather 
than replace the original.  Anyway, if people might prefer older 
versions of problems, we would not want the equivalent of "cvs up".

The second reason was that I knew I could get perl to do http requests.  
If someone was missing the needed perl module, it would just become one 
more cpan module to fetch.  If webwork is going to access the cvs 
command on their system, it may be more inconvenient at installation time.

Now that I think about it, the first reason is now out.  The second is 
connected to what system we choose.  If webwork is keeping track of the 
versions and just getting files as needed (i.e., doing some functions 
cvs can do) then it doesn't matter which system is storing files at 
Problem Library Central.  If sites get new versions of problems by the 
equivalent of "cvs up" and we are using subversion, presumably they need 
to have subversion installed.

What I think I have just talked myself into is:

    * if webwork does some of the cvs work and uses http to get files,
      we have more flexibility in how problems are maintained at the
      repository.
    * if we have files transmitted by cvs, cvs will do some of the work
      for us.  But, we then either use cvs itself, or increase the
      installation hassle of webwork (the latter is something I would
      not want to do).

I just thought of another complication with using a cvs-like system for 
fetching files.  Part of the repository structure currently envisioned 
is that there will be several directories with the same internal 
structure, and the process of downloading the problem library should 
amount to taking their union.  If we polish a problem, then it will move 
in the original repository, but it's location should not change the 
individual sites' machines.  Just its version number increases.  It is 
not insurmountable, but it is a complication.

>> My guess is that this is how the perl cpan module works, and it is 
>> how the xemacs package system works.
>
>
> By the way, CPAN modules are packaged in "distributions", tarballs 
> which have a predictable naming scheme and layout and a standard way 
> to build and install them.

There are lots of aspects to the cpan process.  I was only thinking of 
the part where it first seems to fetch a file which gives the modules 
available from a site and their current versions.

>> Since knowing which files need updating keys off of version numbers, 
>> we may have to keep those as part of the files' metadata.
>
>
> Would that still be a problem if you were to keep the local copy of 
> the problem database as a checked-out CVS (or Subversion) working copy?

No, then it shouldn't be a problem.  If the individual sites access the 
library via a cvs-like system, then all version control (e.g. the 
manifest mentioned below) would be handled by cvs.

>> This approach should still be ok with extra associated files.  They 
>> are listed in the manifest along with the problem files.  So, if you 
>> don't have one at the time of an update, it will be fetched for you.
>
>
> What is the manifest? I don't think you'd need any such thing if you 
> were to use a version control system to track files.

The manifest would be a list of files and version information.  It can 
also contain dependency information if we choose.  The main role would 
be to do simple version tracking, so when updating you only get the new 
and updated files.

> Thanks for explaining this all to me. If you get sick of it, just let 
> me know. I always have opinions about things that aren't really my 
> business, but if you'd like to be left alone, say the word. :)

It is good to have some discussion before moving farther forward.

John

Re: [WWdevel] Re: CVS alternative

[Sam H. <sh...@ma...>]

On Jan 10, 2005, at 6:48 PM, Michael Gage wrote:

> On Jan 10, 2005, at 4:29 PM, Sam Hathaway wrote:
>
>> By the way, has anyone thought about how problems will be packaged? 
>> Many problems consist of more than one file and it might be worth 
>> laying out a packaging format, so that a problem and all of its 
>> auxiliary files and metadata can be distributed as a single file.
>
> This is a really good thing to look at.  I still regret that we 
> couldn't use the "resource fork" idea of mac files systems, so that a 
> problem and all of its pictures stay together.  Are there equivalent 
> schemes used on unix and windows?  What is being done on the mac these 
> days?

For distribution packaging there are many archive formats -- tar, zip, 
pkg, etc. with various features. For installed files, though, there 
isn't much. Win32 can embed certain resources (icons, for example) in 
executables and libraries. In UNIX, there's never been much of an 
attempt to do this. On OS X, of course, there are bundles.

> Our current system, which works, but is a bit fragile, is to use the 
> same name for the directory and the .pg file when packaging a file 
> with its accompanying pictures, applets or whatever.  e.g.    
> prob1/prob1.pg  accompanied by prob1/picture1.png, prob1/picture2.png, 
> etc.  We usually place applets in a common area so that they can be 
> used in multiple places, but that also means that frequently problems 
> using applets don't work when you are first setting up a course.  You 
> have to tweak the addresses in the problems and/or find the applets 
> and install them in the right place.
>
> Any ideas for making this situation more robust and straightforward?

A simple tweak would be to put the problem file in the directory with 
the auxiliary files. This sits better with me than having the problem 
file and the directory of auxiliary files side-by-side, since it would 
be harder for a problem to be separated from its auxiliary files. These 
directories could be tarred up for distribution if need be.

You mentioned the applets, and I'm thinking that it might be worth 
talking about dependancy information. Perhaps a piece of metadata about 
a problem should be a list of other resources on which it depends 
(maybe along with version ranges, if needed). The more I think about 
it, the more the needs of the problem database converge with the needs 
of operating system distribution packagers.

For example, Debian packages consist of an archive containing the 
package files and several several well-defined metadata files that give 
information about the package and its dependancies, how to build it, 
etc. To build an archive, the metadata in each package is assembled 
into a Packages.gz file that can be read by the APT system to build a 
dependancy graph, etc.

Of course, this is overkill, but what's currently proposed seems 
essentially like a simpler version of this, if you view each problem 
source file as a package, and the MySQL database as the Packages.gz 
file. It might be worth formalizing ways of tracking dependancies for 
situations where a problem depends on some external resource like an 
applet. On the other hand, it might not be worth it.

By the way, all involved with the problem database project are free to 
use the WeBWorK Wiki to share ideas and write up proposals: 
<http://devel.webwork.rochester.edu/>.
-sam

[WWdevel] Re: best way to archive courses

[Sam H. <sh...@ma...>]

On Dec 19, 2004, at 3:13 PM, Sam Hathaway wrote:

>
> On Dec 16, 2004, at 4:52 PM, Arnold Pizer wrote:
>
>> Hi Sam,
>>
>> Every semester we archive the current courses keeping them active for  
>> another semester (with a different name, e.g. fall04-mth161) and then  
>> create a new course (mth161).  My question is how best to do this  
>> with WeBWorK 2.  This is a standard question which will come up so we  
>> should have instructions on what to do (and eventually an easy  
>> automatic way to do it).
>
> There isn't really an easy way to rename courses, since the database  
> names are given in terms of the course name. However, I think we can  
> get around this by renaming the databases manually, and this can  
> certainly be automated. (In fact, a renameCourse() routine was  
> originally planned for CourseManagement.pm, but we never completed  
> it.)

Just wanted to let you know that I wrote renameCourse() a while back,  
and it works, with some caveats. First, it only works with courses that  
use the "sql_single" database layout. However, you can convert "sql"  
courses to "sql_single" courses using the "sql2sql_single" command-line  
utility. See  
<http://devel.webwork.rochester.edu/twiki/bin/view/Webwork/ 
CourseAdministrationManual> for more information.

Second, no command-line script exists -- the functionality is only  
available via the admin course.

I've stopped work on this for now, since our short-term priorities lie  
elsewhere. However, it shouldn't be hard for someone to add support for  
the other database layouts or the command-line interface.
-sam

Re: [WWdevel] Re: CVS alternative

[Sam H. <sh...@ma...>]

On Jan 10, 2005, at 5:47 PM, John Jones wrote:

> Sam Hathaway wrote:
>
>> On Jan 10, 2005, at 4:13 PM, Michael Gage wrote:
>>
>>>>
>>>>  Problems start in the non-tagged side, basically however we find 
>>>> them.  Once this thing is initialized, I guess we can start filling 
>>>> that up with lots of pg files.  When it gets tagged, then it is 
>>>> moved to the tagged-side, which will be organized to mirror the 
>>>> heirarchical topic structure of the database.
>>>>
>>>>  We may not be able to "polish" every problem, but as that is done, 
>>>> it simply gives an updated version of the problem on the tagged 
>>>> side.  The setup as described above basically gives up on the 
>>>> notion of systematically polishing problems.  If we want to keep 
>>>> that alive, we should have 3 basic sub-divisions (raw, tagged, and 
>>>> tagged-and-polished).  Actually, this 3-part version might be a 
>>>> good way to go.
>>>>
>>> I like the 3 part version.  Possibly even a 4th part for  problems 
>>> which can be used as models for future problems (exhibiting best 
>>> practices, etc. etc.)  This fourth part could be fairly small 
>>> however, and may not need to be a CVS.
>>
>>
>> Can anyone give me more details on how the repository of problem 
>> sources and the "database" will interact? Based on what little I 
>> know, it seems to me that the problem source should be part of the 
>> problem's database record.
>
> In a sense, things are reversed.  The problem files initially contain 
> all of the information; the extra information comes in the form of 
> special comments in those files.  We then have a script to set up a 
> mysql database, and to extract the information from the files and load 
> it into the database.

Would it be fair to say that the MySQL database does nothing more than 
act as an index on the metadata associated with each problem? Or am I 
missing something?

> I like this approach since it is easy to reload the database if 
> something goes wrong, and we are shipping mainly flat text files 
> (except for the images).

I like the simplicity of this, and in a distributed system like this 
the more we can do with a version control system the better.

>> By the way, has anyone thought about how problems will be packaged? 
>> Many problems consist of more than one file and it might be worth 
>> laying out a packaging format, so that a problem and all of its 
>> auxiliary files and metadata can be distributed as a single file.
>
> I hadn't thought of the extra files.  Thus far, the problems were 
> basically not packaged in any special way.
>
> The distribution method I had in mind was that webwork would handle it 
> behind the scenes.  It would fetch files over http from perl (I think 
> the perl module is LWP, or something like that).  The entry point 
> would be an extra tab in the admin course (along with add course, ..., 
> and then Problem Database).  If you ask it to update your Problem 
> Library Database, then it fetches the current list of files/version 
> via http, checks it against your current list, and gets whatever is 
> new and reloads the database.

Shouldn't we leverage the version control system checkout features to 
fetch and update problem libraries? It seems like a waste to keep the 
problems in CVS (or Subversion) and then ignore the versioning features 
of that system and track versions separately and fetch via HTTP.

> My guess is that this is how the perl cpan module works, and it is how 
> the xemacs package system works.

By the way, CPAN modules are packaged in "distributions", tarballs 
which have a predictable naming scheme and layout and a standard way to 
build and install them.

> Since knowing which files need updating keys off of version numbers, 
> we may have to keep those as part of the files' metadata.

Would that still be a problem if you were to keep the local copy of the 
problem database as a checked-out CVS (or Subversion) working copy?

> If we use cvs for the files, we can just use the cvs revision number.  
> If we use subversion, then there is one number for all files, so every 
> change would make it look like all of the files need updating, so that 
> would be a case where a problem's version number would have to be 
> kept.

I don't really know, but I would expect Subversion to provide some way 
of identifying a version of a particular file. I know that it has the 
concept of a changeset, and that might be more like what you want 
anyway. Each changeset would encompass a small set a files, usually a 
single problem file but sometimes a problem and its auxiliary files.

> This approach should still be ok with extra associated files.  They 
> are listed in the manifest along with the problem files.  So, if you 
> don't have one at the time of an update, it will be fetched for you.

What is the manifest? I don't think you'd need any such thing if you 
were to use a version control system to track files.

Thanks for explaining this all to me. If you get sick of it, just let 
me know. I always have opinions about things that aren't really my 
business, but if you'd like to be left alone, say the word. :)
-sam

Re: [WWdevel] Re: CVS alternative

[Michael G. <ga...@ma...>]

On Jan 10, 2005, at 4:29 PM, Sam Hathaway wrote:

> By the way, has anyone thought about how problems will be packaged? 
> Many problems consist of more than one file and it might be worth 
> laying out a packaging format, so that a problem and all of its 
> auxiliary files and metadata can be distributed as a single file.

This is a really good thing to look at.  I still regret that we 
couldn't use the "resource fork" idea of mac files systems, so that a 
problem and all of its pictures stay together.  Are there equivalent 
schemes used on unix and windows?  What is being done on the mac these 
days?

Our current system, which works, but is a bit fragile, is to use the 
same name for the directory and the .pg file when packaging a file with 
its accompanying pictures, applets or whatever.  e.g.    prob1/prob1.pg 
  accompanied by prob1/picture1.png, prob1/picture2.png, etc.  We 
usually place applets in a common area so that they can be used in 
multiple places, but that also means that frequently problems using 
applets don't work when you are first setting up a course.  You have to 
tweak the addresses in the problems and/or find the applets and install 
them in the right place.

Any ideas for making this situation more robust and straightforward?

Take care,

Mike

Re: [WWdevel] Re: CVS alternative

[John J. <jj...@as...>]

Sam Hathaway wrote:

> On Jan 10, 2005, at 4:13 PM, Michael Gage wrote:
>
>>>
>>>  Problems start in the non-tagged side, basically however we find 
>>> them.  Once this thing is initialized, I guess we can start filling 
>>> that up with lots of pg files.  When it gets tagged, then it is 
>>> moved to the tagged-side, which will be organized to mirror the 
>>> heirarchical topic structure of the database.
>>>
>>>  We may not be able to "polish" every problem, but as that is done, 
>>> it simply gives an updated version of the problem on the tagged 
>>> side.  The setup as described above basically gives up on the notion 
>>> of systematically polishing problems.  If we want to keep that 
>>> alive, we should have 3 basic sub-divisions (raw, tagged, and 
>>> tagged-and-polished).  Actually, this 3-part version might be a good 
>>> way to go.
>>>
>> I like the 3 part version.  Possibly even a 4th part for  problems 
>> which can be used as models for future problems (exhibiting best 
>> practices, etc. etc.)  This fourth part could be fairly small 
>> however, and may not need to be a CVS.
>
>
> Can anyone give me more details on how the repository of problem 
> sources and the "database" will interact? Based on what little I know, 
> it seems to me that the problem source should be part of the problem's 
> database record.

In a sense, things are reversed.  The problem files initially contain 
all of the information; the extra information comes in the form of 
special comments in those files.  We then have a script to set up a 
mysql database, and to extract the information from the files and load 
it into the database.

I like this approach since it is easy to reload the database if 
something goes wrong, and we are shipping mainly flat text files (except 
for the images).

> By the way, has anyone thought about how problems will be packaged? 
> Many problems consist of more than one file and it might be worth 
> laying out a packaging format, so that a problem and all of its 
> auxiliary files and metadata can be distributed as a single file.

I hadn't thought of the extra files.  Thus far, the problems were 
basically not packaged in any special way.

The distribution method I had in mind was that webwork would handle it 
behind the scenes.  It would fetch files over http from perl (I think 
the perl module is LWP, or something like that).  The entry point would 
be an extra tab in the admin course (along with add course, ..., and 
then Problem Database).  If you ask it to update your Problem Library 
Database, then it fetches the current list of files/version via http, 
checks it against your current list, and gets whatever is new and 
reloads the database.  My guess is that this is how the perl cpan module 
works, and it is how the xemacs package system works.

Since knowing which files need updating keys off of version numbers, we 
may have to keep those as part of the files' metadata.  If we use cvs 
for the files, we can just use the cvs revision number.  If we use 
subversion, then there is one number for all files, so every change 
would make it look like all of the files need updating, so that would be 
a case where a problem's version number would have to be kept.

This approach should still be ok with extra associated files.  They are 
listed in the manifest along with the problem files.  So, if you don't 
have one at the time of an update, it will be fetched for you.

John

Re: [WWdevel] Re: CVS alternative

[Sam H. <sh...@ma...>]

On Jan 10, 2005, at 4:13 PM, Michael Gage wrote:

>>
>>  Problems start in the non-tagged side, basically however we find=20
>> them.=A0 Once this thing is initialized, I guess we can start filling=20=

>> that up with lots of pg files.=A0 When it gets tagged, then it is =
moved=20
>> to the tagged-side, which will be organized to mirror the=20
>> heirarchical topic structure of the database.
>>
>>  We may not be able to "polish" every problem, but as that is done,=20=

>> it simply gives an updated version of the problem on the tagged=20
>> side.=A0 The setup as described above basically gives up on the =
notion=20
>> of systematically polishing problems.=A0 If we want to keep that =
alive,=20
>> we should have 3 basic sub-divisions (raw, tagged, and=20
>> tagged-and-polished).=A0 Actually, this 3-part version might be a =
good=20
>> way to go.
>>
> I like the 3 part version.  Possibly even a 4th part for  problems=20
> which can be used as models for future problems (exhibiting best=20
> practices, etc. etc.)  This fourth part could be fairly small however,=20=

> and may not need to be a CVS.

Can anyone give me more details on how the repository of problem=20
sources and the "database" will interact? Based on what little I know,=20=

it seems to me that the problem source should be part of the problem's=20=

database record.

By the way, has anyone thought about how problems will be packaged?=20
Many problems consist of more than one file and it might be worth=20
laying out a packaging format, so that a problem and all of its=20
auxiliary files and metadata can be distributed as a single file.
-sam

Re: [WWdevel] Re: CVS alternative

[Michael G. <ga...@ma...>]

>
>  Problems start in the non-tagged side, basically however we find=20
> them.=A0 Once this thing is initialized, I guess we can start filling=20=

> that up with lots of pg files.=A0 When it gets tagged, then it is =
moved=20
> to the tagged-side, which will be organized to mirror the heirarchical=20=

> topic structure of the database.
>
>  We may not be able to "polish" every problem, but as that is done, it=20=

> simply gives an updated version of the problem on the tagged side.=A0=20=

> The setup as described above basically gives up on the notion of=20
> systematically polishing problems.=A0 If we want to keep that alive, =
we=20
> should have 3 basic sub-divisions (raw, tagged, and=20
> tagged-and-polished).=A0 Actually, this 3-part version might be a good=20=

> way to go.
>
I like the 3 part version.  Possibly even a 4th part for  problems=20
which can be used as models for future problems (exhibiting best=20
practices, etc. etc.)  This fourth part could be fairly small however,=20=

and may not need to be a CVS.

Take care,

Mike

Re: [WWdevel] Re: CVS alternative

[John J. <jj...@as...>]

First, I am copying Bill and Jeff since I don't know if they get 
openwebwork-devel e-mail.  Also, since Jeff (and some openwebwork-devel 
readers) may not know what we are talking about, here is the plan.

For the .pg files of the National Problem Library (perhaps to be renamed 
the National Problem Database), we will start using a cvs-like system.  
There will be two repositories, or maybe two directories of one 
repository.  The basic distinction is tagged vs. non-tagged problems.

Problems start in the non-tagged side, basically however we find them.  
Once this thing is initialized, I guess we can start filling that up 
with lots of pg files.  When it gets tagged, then it is moved to the 
tagged-side, which will be organized to mirror the heirarchical topic 
structure of the database.

We may not be able to "polish" every problem, but as that is done, it 
simply gives an updated version of the problem on the tagged side.  The 
setup as described above basically gives up on the notion of 
systematically polishing problems.  If we want to keep that alive, we 
should have 3 basic sub-divisions (raw, tagged, and 
tagged-and-polished).  Actually, this 3-part version might be a good way 
to go.

Thinking of the operations we will need in a cvs-like system:

    * add directories and files - I would hope all systems are good at this
    * move a file - cvs may be weaker here since it looses version
      history if you move a file, but maybe we don't really care.  We
      don't expect much revision to take place before tagging.
    * look at recent updates, and maybe the diffs.  This would be
      important as new versions are committed to the tagged files (e.g.,
      to be sure no one deleted the tags, or to see if the new version
      should be forked instead of being a new version of the same
      problem).  I don't know which is better here.  It probably hinges
      on how useful the status commands are for pulling information
      (since I wouldn't want to browse through thousands of files
      looking for recent changes).

I don't know enough about different systems to know which will be better 
for us on these things.

John


Sam Hathaway wrote:

> On Jan 10, 2005, at 12:08 PM, Michael Gage wrote:
>
>> We've been considering this, but haven't made any moves yet -- mostly 
>> because we've
>> had other things to do.  Sam, John, do you have any comments?
>
>
> Arch is more ambitious and has a greater "cool" factor, but I think 
> Subversion is the way to go right now, especially since it's possible 
> to convert a repository from CVS to Subversion: 
> <http://svnbook.red-bean.com/en/1.0/apas11.html>.


> -sam
>
>> On Jan 10, 2005, at 11:52 AM, William Ziemer wrote:
>>
>>> Most of the projects I link with are going to subversion:
>>> http://subversion.tigris.org/
>>> Maybe use this for the problem database?
>>> Good to see you all again,
>>> Bill
>>
>
>
>
> -------------------------------------------------------
> The SF.Net email is sponsored by: Beat the post-holiday blues
> Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
> It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt
> _______________________________________________
> OpenWeBWorK-Devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/openwebwork-devel

[WWdevel] Re: CVS alternative

[Sam H. <sh...@ma...>]

On Jan 10, 2005, at 12:08 PM, Michael Gage wrote:

> We've been considering this, but haven't made any moves yet -- mostly 
> because we've
> had other things to do.  Sam, John, do you have any comments?

Arch is more ambitious and has a greater "cool" factor, but I think 
Subversion is the way to go right now, especially since it's possible 
to convert a repository from CVS to Subversion: 
<http://svnbook.red-bean.com/en/1.0/apas11.html>.
-sam

> On Jan 10, 2005, at 11:52 AM, William Ziemer wrote:
>
>> Most of the projects I link with are going to subversion:
>> http://subversion.tigris.org/
>> Maybe use this for the problem database?
>> Good to see you all again,
>> Bill

Re: [WWdevel] File Manager and linebreaks

[P G. L. <gl...@um...>]

Hi Davide,

Re: line break conversion, my inclination would be to go with the
text/binary convention of FTP, perhaps renamed to be more obvious.  I'd
probably put binary as the default.

Gavin

On Sat, 8 Jan 2005 at 12:08 Davide P.Cervone wrote:

> Folks:
>
> Ken Appel's recent problems with uploading class list files suggests
> that there is an issue that might need to be addressed in the File
> Manager.  Currently, when text files are uploaded, their contents are
> save verbatim.  In particular, nothing is done to adjust line
> terminators for PC and Mac files to be in the unix form.  This may be
> the cause of some of Ken's troubles.
>
> The question is, how should this be handled in the File Manager?  It is
> probably a bad idea to ALWAYS convert line breaks, as if the professor
> is uploaded an image, for example, this would damage it.  There are a
> couple of solutions:
>
> 	1.  Have a checkbox under the UPLOAD button that is "convert line
> breaks to unix format"
> 	     or some such wording, with a warning about not doing this for
> images or binary
> 	     data.  It could be checked by default, since most transfers would
> be text.
>
> 	2.  Have another action button on the right for "Convert Line Breaks".
>
> 	3.  Try to use the file's extension (e.g., .lst) and contents to
> determine if it is a text
> 	     file, and do the conversion automatically.  (Easiest for users,
> when it works, but
> 	     prone to errors.)
>
> 	4.  Some combination of the above.
>
> What do you think?
>
> A related question is should the File Manager try to be smarter about
> where you are putting the files.  For example, if someone puts a .lst
> file in the top level rather than in templates, should there be a
> warning about that?  If so, what are the filetype-to-directory
> mappings?  I know that .lst and set.def files should go in templates,
> and .pg files should be somewhere below templates.  What other files
> would people be uploading, and what are the restrictions on where they
> should go?  Is a warning sufficient, or should there be some sort of
> confirmation dialog box?
>
> Davide
>
>
>
> -------------------------------------------------------
> The SF.Net email is sponsored by: Beat the post-holiday blues
> Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
> It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt
> _______________________________________________
> OpenWeBWorK-Devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/openwebwork-devel
>

-- 
P. Gavin LaRose, Ph.D.  Program Manager (Instructional Tech.)
Math Dept., University of Michigan
gl...@um...                         "It's snowing still.  And freezing.
734.764.6454                              However,  we  haven't had an earth-
http://www.math.lsa.umich.edu/~glarose/	  quake lately."         - A.A. Milne