You can subscribe to this list here.
2004 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(16) |
Nov
(10) |
Dec
(4) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2005 |
Jan
(34) |
Feb
(12) |
Mar
(21) |
Apr
|
May
(5) |
Jun
(13) |
Jul
(50) |
Aug
(62) |
Sep
(72) |
Oct
(17) |
Nov
(16) |
Dec
(19) |
2006 |
Jan
(26) |
Feb
(9) |
Mar
|
Apr
(8) |
May
(5) |
Jun
(7) |
Jul
(21) |
Aug
(33) |
Sep
(17) |
Oct
(4) |
Nov
(9) |
Dec
|
2007 |
Jan
|
Feb
(4) |
Mar
|
Apr
|
May
(6) |
Jun
(16) |
Jul
(8) |
Aug
(1) |
Sep
|
Oct
(2) |
Nov
(2) |
Dec
(2) |
2008 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(4) |
Jul
(11) |
Aug
(6) |
Sep
|
Oct
|
Nov
|
Dec
|
2010 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(3) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2011 |
Jan
|
Feb
|
Mar
|
Apr
(1) |
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
(1) |
2012 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(4) |
Nov
|
Dec
|
2014 |
Jan
(2) |
Feb
(4) |
Mar
|
Apr
(1) |
May
|
Jun
|
Jul
(1) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2015 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(1) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
(2) |
2016 |
Jan
(4) |
Feb
(4) |
Mar
(3) |
Apr
|
May
(1) |
Jun
(1) |
Jul
(1) |
Aug
(2) |
Sep
(1) |
Oct
(1) |
Nov
(1) |
Dec
|
2017 |
Jan
|
Feb
(1) |
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2018 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
|
Nov
|
Dec
|
From: John J. <jj...@as...> - 2005-08-27 22:49:06
|
Webwork 1 had a fuller logging procedure, where basically every click was logged. I found that useful since I sometimes had to figure out what a teacher had done (in addition to answering questions about what students did). The webwork 2 logging framework made it easy to replicate this: one line in global.conf to define the log file, and then one (long) line in ContentGenerator to call the writeCourseLog to record the url and the cgi parameters. So, this raised a couple of questions: * would we want this facility in the main webwork? * if so, should it be more refined (so individual modules can give more finely tuned log entries)? Thoughts? John |
From: Arnold P. <ap...@ma...> - 2005-08-27 21:35:18
|
At 03:09 PM 8/26/2005, John Jones wrote: OK. Then zoom hack has been commented out. Arnie >Hi, > >I just tried to see the peek-a-boo bug, and I can't seem to trigger >it anymore. I removed the zoom fix and tried two computers with >different generations of explorer, tried using the questionnaire, >varied a change near the bottom of the css file that says it is >related to peek-a-boo, and I still can't see it. Maybe something >else was changed along the way that fixed it. > >So, by all means, remove it or comment it out from ur.css and assume >it is not being used. > >John Prof. Arnold K. Pizer Dept. of Mathematics University of Rochester Rochester, NY 14627 (585) 275-7767 |
From: John J. <jj...@as...> - 2005-08-26 19:09:24
|
Hi, I just tried to see the peek-a-boo bug, and I can't seem to trigger it anymore. I removed the zoom fix and tried two computers with different generations of explorer, tried using the questionnaire, varied a change near the bottom of the css file that says it is related to peek-a-boo, and I still can't see it. Maybe something else was changed along the way that fixed it. So, by all means, remove it or comment it out from ur.css and assume it is not being used. John Arnold Pizer wrote: > At 11:59 AM 8/25/2005, John Jones wrote: > > Hi, > > Two things. > > First a very reliable way to see the peek-a-boo bug in MSIE is to > view the WeBWorK questionnaire. > > Second is to recall that the hack > * {zoom: 1;} > we added to get around MSIE's peek-a-boo causes MSIE to drop labels > with the <OL> tag. > > The latter is I think a worst problem than the peek-a-boo bug so I > would vote to comment out the hack (which is in fact what we did last > year at Rochester but that "fix" has now been over ridden) >> Davide P.Cervone wrote: >> >>> Folks: >>> >>> Does anyone know if the >>> >>> * {zoom: 1;} >>> >>> in the ur.css file is actually helping anything? It is causing >>> problems for jsMath on MSIE on the PC, and I have been having >>> trouble getting around it. Is there any reason that I can't remove >>> it? I know that it helps the lists in the left panel in MSIE, but >>> that can be handled by adding the zoom attribute to the >>> 'td.LeftPanel' and 'td.LeftPanel ul' selectors. >> >> >> We had it because it was an easy fix for the IE peekaboo bug so that >> large parts of problem text sometimes disappeared. I will try to >> test without it, but that is slow since I don't normally use a >> windows machine. If it is ok in testing, I can try unleashing it on >> our students. The best chance for testing it, however, is almost >> over. The bug shows most often when you have very long problems, and >> the introductory set has some of the longest problems in it (and some >> of our classes have already closed that set). >> >> In any case, I'll let you know what I find out. Meanwhile, don't let >> my promise to test stop anyone else from testing too :). >> >> John >> >> >> >> ------------------------------------------------------- >> SF.Net email is Sponsored by the Better Software Conference & EXPO >> September 19-22, 2005 * San Francisco, CA * Development Lifecycle >> Practices >> Agile & Plan-Driven Development * Managing Projects & Teams * Testing >> & QA >> Security * Process Improvement & Measurement * >> http://www.sqe.com/bsce5sf >> _______________________________________________ >> OpenWeBWorK-Devel mailing list >> Ope...@li... >> https://lists.sourceforge.net/lists/listinfo/openwebwork-devel > > > Prof. Arnold K. Pizer > Dept. of Mathematics > University of Rochester > Rochester, NY 14627 > (585) 275-7767 > > > > ------------------------------------------------------- > SF.Net email is Sponsored by the Better Software Conference & EXPO > September 19-22, 2005 * San Francisco, CA * Development Lifecycle > Practices > Agile & Plan-Driven Development * Managing Projects & Teams * Testing > & QA > Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf > _______________________________________________ > OpenWeBWorK-Devel mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openwebwork-devel |
From: Sam H. <sh...@ma...> - 2005-08-26 16:34:34
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 26, 2005, at 12:27, Sam Hathaway wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Has anyone else run into this problem? I can't replicate it. > - -sam Ok, it seems I'm a little behind on this one. Arnie has seen it, and there appears to be a fix. This is what I get for not reading the discussion board. -sam > Begin forwarded message: > > >> From: Sam Hathaway <sh...@ma...> >> Date: August 26, 2005 12:13:17 EDT >> To: Jeff Denny <DEN...@Me...> >> Subject: Re: pdf generation problem >> >> On Aug 24, 2005, at 15:46, Jeff Denny wrote: >> >> >>> Sam, >>> >>> We have identified a pdf generation problem. We created our >>> usernames using the university's e-mail addresses for the >>> students by taking their e-mail userids as the webwork usernames. >>> Unfortunately, our school uses names like "jeff.k.denny" When >>> webwork tries to create a pdf, latex gets confused by the periods >>> in the username when looks for the png extension. Then, webwork >>> produces an error message. >>> >>> The only thing I can come up with is to change the usernames to >>> the students' university id numbers. Of course, the problem is >>> that the username in webwork appears to be the one thing that >>> can't be changed in the database for each user. >>> >>> Do you have a suggestion? >>> >> >> Hi Jeff, >> >> I can't replicate this problem. I created a user with user ID >> 'foo.bar' and student ID 'this.is.the.student.id'. I tried >> generating hardcopy for this user both while logged in as myself >> ('sam') and while logged in as the user itself. >> >> The problem I tested had graphs in it (it is the graph example >> from set0), which were converted to PNGs and embedded in the PDF. >> >> If I can find the source of your problem, I can probably fix it in >> the source and give you a patch to apply to your system. >> -sam >> > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.2.4 (Darwin) > > iD8DBQFDD0MR14CQX+2WvVgRAhMXAJ9SrgLHGUzUWfDWXN2aWoI3W45ffgCfdZ0s > m7qn03IkgVSWEOYmiEg+JrI= > =iBp3 > -----END PGP SIGNATURE----- > > > ------------------------------------------------------- > SF.Net email is Sponsored by the Better Software Conference & EXPO > September 19-22, 2005 * San Francisco, CA * Development Lifecycle > Practices > Agile & Plan-Driven Development * Managing Projects & Teams * > Testing & QA > Security * Process Improvement & Measurement * http://www.sqe.com/ > bsce5sf > _______________________________________________ > OpenWeBWorK-Devel mailing list > Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openwebwork-devel > -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) iD8DBQFDD0SM14CQX+2WvVgRAgZAAJ93tHp+UG7p/oX45V35IOmpVOeSeACcCS3n hqs7LR/gfJmBGM747fWxdnQ= =qZC3 -----END PGP SIGNATURE----- |
From: Sam H. <sh...@ma...> - 2005-08-26 16:28:18
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Has anyone else run into this problem? I can't replicate it. - -sam Begin forwarded message: > From: Sam Hathaway <sh...@ma...> > Date: August 26, 2005 12:13:17 EDT > To: Jeff Denny <DEN...@Me...> > Subject: Re: pdf generation problem > > On Aug 24, 2005, at 15:46, Jeff Denny wrote: > >> Sam, >> >> We have identified a pdf generation problem. We created our >> usernames using the university's e-mail addresses for the students >> by taking their e-mail userids as the webwork usernames. >> Unfortunately, our school uses names like "jeff.k.denny" When >> webwork tries to create a pdf, latex gets confused by the periods >> in the username when looks for the png extension. Then, webwork >> produces an error message. >> >> The only thing I can come up with is to change the usernames to >> the students' university id numbers. Of course, the problem is >> that the username in webwork appears to be the one thing that >> can't be changed in the database for each user. >> >> Do you have a suggestion? > > Hi Jeff, > > I can't replicate this problem. I created a user with user ID > 'foo.bar' and student ID 'this.is.the.student.id'. I tried > generating hardcopy for this user both while logged in as myself > ('sam') and while logged in as the user itself. > > The problem I tested had graphs in it (it is the graph example from > set0), which were converted to PNGs and embedded in the PDF. > > If I can find the source of your problem, I can probably fix it in > the source and give you a patch to apply to your system. > -sam -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) iD8DBQFDD0MR14CQX+2WvVgRAhMXAJ9SrgLHGUzUWfDWXN2aWoI3W45ffgCfdZ0s m7qn03IkgVSWEOYmiEg+JrI= =iBp3 -----END PGP SIGNATURE----- |
From: John J. <jj...@as...> - 2005-08-26 04:04:08
|
Michael Gage wrote: > Can we fix this to allow periods in the user names? I think we have > already done this > to a large extent, but we probably missed the latex ramifications. I'm sure it can be done fairly easily. I ran into the analogous problem when I allowed periods in set names some time ago. I thought Zig had been using periods in user names for a long time. I wonder what they have been doing all this time about hardcopy with graphs. John > Begin forwarded message: > >> *From: *Jeff Denny <DEN...@Me... <mailto:DEN...@Me...>> >> *Date: *August 24, 2005 2:43:06 PM EDT (CA) >> *To: *Michael Gage <ga...@ma... >> <mailto:ga...@ma...>> >> *Subject: **pdf generation problem* >> >> >> Mike, >> >> We have identified the pdf generation problem. We created our >> usernames using the university's e-mail addresses for the students by >> taking their e-mail userids as the webwork usernames. Unfortunately, >> our school uses names like "jeff.k.denny" When webwork tries to >> create a pdf, latex gets confused by the periods in the username when >> looks for the png extension. Then, webwork produces an error message. >> The only thing I can come up with is to change the usernames to the >> students' university id numbers. Of course, the problem is that the >> username in webwork appears to be the one thing that can't be changed >> in the database for each user. >> >> Can we somehow change the userids in sql or get around this problem >> somehow? >> >> Thanks! >> >> Jeff >> >> >> >> > |
From: Michael G. <ga...@ma...> - 2005-08-26 03:49:08
|
Can we fix this to allow periods in the user names? I think we have already done this to a large extent, but we probably missed the latex ramifications. Take care, Mike Begin forwarded message: > From: Jeff Denny <DEN...@Me...> > Date: August 24, 2005 2:43:06 PM EDT (CA) > To: Michael Gage <ga...@ma...> > Subject: pdf generation problem > > > Mike, > > We have identified the pdf generation problem. We created our > usernames using the university's e-mail addresses for the students > by taking their e-mail userids as the webwork usernames. > Unfortunately, our school uses names like "jeff.k.denny" When > webwork tries to create a pdf, latex gets confused by the periods > in the username when looks for the png extension. Then, webwork > produces an error message. > The only thing I can come up with is to change the usernames to the > students' university id numbers. Of course, the problem is that > the username in webwork appears to be the one thing that can't be > changed in the database for each user. > > Can we somehow change the userids in sql or get around this problem > somehow? > > Thanks! > > Jeff > > > > |
From: Arnold P. <ap...@ma...> - 2005-08-25 17:19:52
|
At 11:59 AM 8/25/2005, John Jones wrote: Hi, Two things. First a very reliable way to see the peek-a-boo bug in MSIE is to view the WeBWorK questionnaire. Second is to recall that the hack * {zoom: 1;} we added to get around MSIE's peek-a-boo causes MSIE to drop labels with the <OL> tag. The latter is I think a worst problem than the peek-a-boo bug so I would vote to comment out the hack (which is in fact what we did last year at Rochester but that "fix" has now been over ridden) Arnie >Davide P.Cervone wrote: > >>Folks: >> >>Does anyone know if the >> >> * {zoom: 1;} >> >>in the ur.css file is actually helping anything? It is causing problems >>for jsMath on MSIE on the PC, and I have been having trouble getting >>around it. Is there any reason that I can't remove it? I know that it >>helps the lists in the left panel in MSIE, but that can be handled by >>adding the zoom attribute to the 'td.LeftPanel' and 'td.LeftPanel ul' >>selectors. > >We had it because it was an easy fix for the IE peekaboo bug so that large >parts of problem text sometimes disappeared. I will try to test without >it, but that is slow since I don't normally use a windows machine. If it >is ok in testing, I can try unleashing it on our students. The best >chance for testing it, however, is almost over. The bug shows most often >when you have very long problems, and the introductory set has some of the >longest problems in it (and some of our classes have already closed that set). > >In any case, I'll let you know what I find out. Meanwhile, don't let my >promise to test stop anyone else from testing too :). > >John > > > >------------------------------------------------------- >SF.Net email is Sponsored by the Better Software Conference & EXPO >September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices >Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA >Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf >_______________________________________________ >OpenWeBWorK-Devel mailing list >Ope...@li... >https://lists.sourceforge.net/lists/listinfo/openwebwork-devel Prof. Arnold K. Pizer Dept. of Mathematics University of Rochester Rochester, NY 14627 (585) 275-7767 |
From: John J. <jj...@as...> - 2005-08-25 15:59:04
|
Davide P.Cervone wrote: > Folks: > > Does anyone know if the > > * {zoom: 1;} > > in the ur.css file is actually helping anything? It is causing > problems for jsMath on MSIE on the PC, and I have been having trouble > getting around it. Is there any reason that I can't remove it? I > know that it helps the lists in the left panel in MSIE, but that can > be handled by adding the zoom attribute to the 'td.LeftPanel' and > 'td.LeftPanel ul' selectors. We had it because it was an easy fix for the IE peekaboo bug so that large parts of problem text sometimes disappeared. I will try to test without it, but that is slow since I don't normally use a windows machine. If it is ok in testing, I can try unleashing it on our students. The best chance for testing it, however, is almost over. The bug shows most often when you have very long problems, and the introductory set has some of the longest problems in it (and some of our classes have already closed that set). In any case, I'll let you know what I find out. Meanwhile, don't let my promise to test stop anyone else from testing too :). John |
From: Davide P.C. <dp...@un...> - 2005-08-25 00:42:33
|
Folks: Does anyone know if the * {zoom: 1;} in the ur.css file is actually helping anything? It is causing problems for jsMath on MSIE on the PC, and I have been having trouble getting around it. Is there any reason that I can't remove it? I know that it helps the lists in the left panel in MSIE, but that can be handled by adding the zoom attribute to the 'td.LeftPanel' and 'td.LeftPanel ul' selectors. Davide |
From: John J. <jj...@as...> - 2005-08-24 23:45:43
|
Davide P.Cervone wrote: > I also added that no one starts out with old answers if we are > past the due date. So, this is somewhat hard-wired. If this > should only apply to some users, let me know - we just add another > permission cut off to global.conf. > > > I would find it helpful to show past answers even after the due date > when I am looking at student problems (since I usually am either > working with a student who had a question, or want to see what they > answered when they didn't get it right). I always found it > inconvenient in WW1 when these were not shown. OK, I'll put that in. John |
From: Davide P.C. <dp...@un...> - 2005-08-24 20:44:09
|
> I also added that no one starts out with old answers if we are past=20= > the due date.=A0 So, this is somewhat hard-wired.=A0 If this should = only=20 > apply to some users, let me know - we just add another permission cut=20= > off to global.conf. I would find it helpful to show past answers even after the due date=20 when I am looking at student problems (since I usually am either=20 working with a student who had a question, or want to see what they=20 answered when they didn't get it right). I always found it=20 inconvenient in WW1 when these were not shown. Davide |
From: John J. <jj...@as...> - 2005-08-24 20:28:56
|
Arnold Pizer wrote: > At 01:30 PM 8/24/2005, John Jones wrote: > > After a number of comments on showOldAnswers in WW1, we arrived at the > default implementation (if memory serves me correctly) of (1) not > showing old answers for practice users and (2) for regular users > showing old answers before the due date but not after (and we may have > used the answer date, not the due date). The reason was that after > the due date, students submitting answers were probably practicing for > an exam, etc and having the answers already filled in tended to negate > the exercise. In all cases, the user could change the showOldAnswers > option. I think this worked pretty well and would be reasonable to > reproduce in WW2 if it can be done cleanly. I don't think it's a > critical issue but someone new to WW logging in as a practice user > would certainly be confused by seeing the answers already filled in. I just committed a change which does this. Apply options should still work for everyone. The only change is if showOldAnswers (the default value) is set to 1, sometimes we pretend it is set to 0. Now by default, practice users start out with not showing saved answers. This can be adjusted to everyone or no one or anything in between in global.conf. I also added that no one starts out with old answers if we are past the due date. So, this is somewhat hard-wired. If this should only apply to some users, let me know - we just add another permission cut off to global.conf. John >> Hi, >> >> A question came up on the discussion board. I am not entirely sure >> of what people are expecting, but one thing seems to be a feature >> from ww 1.x where by default, practice users did not have the last >> submitted answer shown. >> >> Should we put this in ww 2? I think it would be relatively easy to do: >> >> * add a permission setting in global.conf of can_show_old_answers >> and set the default level to $student. >> * check for this permission when setting the value in >> Problem.pm. It is not quite parallel to the other items with >> the can/want/will system since it just refers to using the >> default value, but it can be handled in pre_header_initialize. >> >> While looking at it just now, I saw another bug in showOldAnswers >> (i.e., in addition to the one I just fixed in cvs). I'll wait to see >> what people want on this question before addressing it. >> >> I don't have a strong preference, so what do you think about having a >> way to make practice users not see old answers by default? >> >> John > > > Prof. Arnold K. Pizer > Dept. of Mathematics > University of Rochester > Rochester, NY 14627 > (585) 275-7767------------------------------------------------------- > SF.Net email is Sponsored by the Better Software Conference & EXPO > September 19-22, 2005 * San Francisco, CA * Development Lifecycle > Practices Agile & Plan-Driven Development * Managing Projects & Teams > * Testing & QA Security * Process Improvement & Measurement * > http://www.sqe.com/bsce5sf > _______________________________________________ OpenWeBWorK-Devel > mailing list Ope...@li... > https://lists.sourceforge.net/lists/listinfo/openwebwork-devel > |
From: Arnold P. <ap...@ma...> - 2005-08-24 19:38:12
|
<html> <body> At 01:30 PM 8/24/2005, John Jones wrote:<br><br> After a number of comments on showOldAnswers in WW1, we arrived at the default implementation (if memory serves me correctly) of (1) not showing old answers for practice users and (2) for regular users <br> showing old answers before the due date but not after (and we may have used the answer date, not the due date). The reason was that after the due date, students submitting answers were probably practicing for an exam, etc and having the answers already filled in tended to negate the exercise. In all cases, the user could change the showOldAnswers option. I think this worked pretty well and would be reasonable to reproduce in WW2 if it can be done cleanly. I don't think it's a critical issue but someone new to WW logging in as a practice user would certainly be confused by seeing the answers already filled in.<br><br> Arnie<br><br> <br> <blockquote type=cite class=cite cite="">Hi,<br> <br> A question came up on the discussion board. I am not entirely sure of what people are expecting, but one thing seems to be a feature from ww 1.x where by default, practice users did not have the last submitted answer shown.<br><br> Should we put this in ww 2? I think it would be relatively easy to do: <ul> <li>add a permission setting in global.conf of can_show_old_answers and set the default level to $student. <li>check for this permission when setting the value in Problem.pm. It is not quite parallel to the other items with the can/want/will system since it just refers to using the default value, but it can be handled in pre_header_initialize. </ul>While looking at it just now, I saw another bug in showOldAnswers (i.e., in addition to the one I just fixed in cvs). I'll wait to see what people want on this question before addressing it.<br><br> I don't have a strong preference, so what do you think about having a way to make practice users not see old answers by default?<br><br> John<br> </blockquote> <x-sigsep><p></x-sigsep> <br> Prof. Arnold K. Pizer<br> Dept. of Mathematics<br> University of Rochester<br> Rochester, NY 14627<br> (585) 275-7767</body> </html> |
From: Davide P.C. <dp...@un...> - 2005-08-24 19:10:54
|
OK, I've changed the way Parser Reals stringify themselves to not use the (fuzzy) check for zero that I was using, and instead use the zeroLevelTol value directly, and only convert to zero when the value is less than that. This should take care of the problems most of you had with the current behavior when absolute tolerances are in effect. Get the current lib/Parser/Real.pm to get the new behavior. Davide |
From: John J. <jj...@as...> - 2005-08-24 17:30:44
|
Hi, A question came up on the discussion board. I am not entirely sure of what people are expecting, but one thing seems to be a feature from ww 1.x where by default, practice users did not have the last submitted answer shown. Should we put this in ww 2? I think it would be relatively easy to do: * add a permission setting in global.conf of can_show_old_answers and set the default level to $student. * check for this permission when setting the value in Problem.pm. It is not quite parallel to the other items with the can/want/will system since it just refers to using the default value, but it can be handled in pre_header_initialize. While looking at it just now, I saw another bug in showOldAnswers (i.e., in addition to the one I just fixed in cvs). I'll wait to see what people want on this question before addressing it. I don't have a strong preference, so what do you think about having a way to make practice users not see old answers by default? John |
From: Davide P.C. <dp...@un...> - 2005-08-24 16:20:16
|
> Also the rounding should not depend on the problem otherwise students > will get very confused as to what WeBWorK is doing. It already does, as this depends on the format parameter and (and the default number format set in global.conf, which I think can be overridden in the problem). > I have no problem with presenting double precision or scientific > notation but if others want say only 8 significant digits, then I > think all answers should be rounded to this accuracy before they are > processed to see if they are correct within the tolerance of the > individual problem. That is not what currently gets done, as the values are used at the full precision regardless of what format is used to display them. > WeBWork should use the value .377964473... to decide whether or not > the student's answer is correct. It does, both in the traditional and Parser-based version of num_cmp > A similar philosophy is (or at least was) used in the string answer > comparisons. In numerical comparisons, things like 1/sqrt(7) are > translated to .377964473... which WeBWorK then sends on to the answer > checker. This is correct for the PROFESSOR'S answer, but the STUDENT'S answer is handled entirely by the answer checker. itself (or its pre- and post-filters). > In case insensitive string compare, things like AbCdE are translated > to ABCDE which are then sent to the answer checker. > In case sensitive strict string compare, the original string is just > reproduced. > > Since students see exactly what they entered in the body of the > problem (sticky answers), what should be displayed under entered is > the numerical or string equivalent of their entered answers. This is NOT the case for the string answer checkers. If you use str_cmp("AbCdE"), for example, then the students response will be shown in upper case in both the entered and preview columns. The professor's answer will show as "AbCdE" if you don't enter an answer in the blank and ask to show answers, but will be shown as "ABCDE" if you DO enter an answer in the blank. (I didn't check what hardcopy with answers produces, but I suspect "AbCdE"). So it looks like the actual evaluation of the answer checker will cause a difference in this case. I agree that they should see "the numerical or string equivalent of their entered answers", but we may disagree about what that means. To me, in a problem where the tolerances are at .5, showing an answer as .377964473 is misleading, but perhaps that is just me. Davide |
From: Arnold P. <ap...@ma...> - 2005-08-24 15:54:20
|
At 11:05 AM 8/24/2005, John Jones wrote: Hi, I'll add my two cents on this one. Basically I agree with John. As "entered" I think WeBWorK should always display the numerical equivalent of what the student entered without significant rounding. This is what WeBWorK uses to decide whether or not an answer is correct and this is what should be displayed. Also the rounding should not depend on the problem otherwise students will get very confused as to what WeBWorK is doing. I have no problem with presenting double precision or scientific notation but if others want say only 8 significant digits, then I think all answers should be rounded to this accuracy before they are processed to see if they are correct within the tolerance of the individual problem. Otherwise what WeBWorK is doing may be very mysterious albeit in very rare cases. Giving 0 as the entered answer when the absolute tol is .5 and the student entered 1/sqrt(7) I think is misleading. WeBWork should use the value .377964473... to decide whether or not the student's answer is correct. Thus in any problem where the correct answer is in the interval [-.12203..., .87796...], the above answer will be marked correct. I don't see where zero has anything to do with it. If I got0 as a response, my first reaction would be to look at the code of the problem or answer checker but students don't have that option. They will just get frustrated. A similar philosophy is (or at least was) used in the string answer comparisons. In numerical comparisons, things like 1/sqrt(7) are translated to .377964473... which WeBWorK then sends on to the answer checker. In case insensitive string compare, things like AbCdE are translated to ABCDE which are then sent to the answer checker. In case sensitive strict string compare, the original string is just reproduced. Since students see exactly what they entered in the body of the problem (sticky answers), what should be displayed under entered is the numerical or string equivalent of their entered answers. Arnie >Davide P.Cervone wrote: > >>John and Gavin: >> >>I'm rerouting this to the developer mailing list, since it now >>relates to feature decisions. >> >>On Aug 24, 2005, at 12:19 AM, John Jones wrote: >> >>>Davide P. Cervone wrote: >>> >>>>>For the num_cmp() evaluator, setting 'tol'=>0.5 gives the result >>>>> Entered Answer Preview Correct Result >>>>> 0 0 1/sqrt(7) correct >>>>>and taking out the tol gives >>>>> Entered Answer Preview Correct Result >>>>> 0.25 0.25 1/sqrt(7) incorrect >>>>> >>>>>For the number_list_cmp evaluator, with the 'tol' option I get >>>>> Entered Answer Preview Correct Result >>>>> -0, 0 -0,0 0, 0 correct >>>>>and without, >>>>> Entered Answer Preview Correct Result >>>>> -0.25, 0.25 -0.25,0.25 -0.37796.., 0.37796.. incorrect >>>> >>>> >>>> >>>>The reason that the parser shows the student's answer as 0 is >>>>that when it stringifies a real number, it shows it as zero if it >>>>is "equal" to zero according to the current equality >>>>conditions. Since your answers actually ARE equal to zero >>>>(within an absolute tolerance of 0.5), they get printed as >>>>zeros. When a RELATIVE tolerance is used, there is a special >>>>tolerance for zero that gets used for checking against >>>>zero. This is not the case for absolute tolerance, because you >>>>are specifying exactly how close you want the answer to be. >>> >>> >>>I am not so sure it is good to simplify a student's answer to 0 >>>for a few reasons. It exposes some of webwork's internal tricks >>>to the students. In general, other answers are not simplified in this way. >>>For example, 1.0001 is not simplified to 1. And, it would be >>>confusing to a student to see their answer not register. I tried >>>setting tolType=>'strict' in num_cmp and taking the default setting. >>>Then, I submitted 0.09, webwork told me I submitted 0. >> >>Well, first of all, tolType should be one of 'relative' or >>'absolute' from what I can see. (I am unable to find a reference >>to tolType being 'strict' anywhere.) num_cmp doesn't check the >>value of tolType to make sure it is legal, and perhaps it >>should. What happens when you tolType to something other than >>'relative' or 'absolute' is somewhat random, as some tests are for >>tolType equal to 'relative' and others are for 'absolute' and both >>assume if it is not the one checked, then it is the other. So when >>set to 'strict' you will sometimes get the absolute branch of the >>test and other times the relative branch. For the example above, >>you end up getting an absolute test with the tolerance set to 1, so >>the answer 0.09 is properly considered to be 0. > >My mistake on using strict instead of absolute for the tolType, and >you do have to be closer to 0 to see the rounding effect with absolute. > >I tried > >ANS( num_cmp(5, tolType=>'absolute') ); > >and entered 0.000999. Webwork tells me that I entered 0 and the preview is 0. > >I can see that tolType relative acts differently. With > >ANS( num_cmp(5) ); > >webwork will correctly report what I enter until I get down to >0.0000000000009 which is interpreted as 0. > >I understand that students get confused when a number is listed in >scientific notation. We get that complaint, but only when the value >appearing in the problem text is in scientific notation (it happens >by accident, and so we just fix the problem). > >But, one of the frustrations I hear about is "webwork won't take my >answer". If someone enters a non-zero answer and the system says >they entered 0, a common perception would be that the number they >entered didn't take. Then they try to submit the same thing 10 more >times with the same result, and get mad at the system. > >The defaults for relative comparison seem ok. If someone enters >0.0000000000009, I think you can reasonably tell them that the >computer rounded it off because it was so close to 0. The main >reason we currently use Parser-based answer checkers is that they >make things less confusing for students. But on the absolute error >example above, entering 0.0009 and being told it is 0 seems confusing. > >Now that I am thinking about it, I don't understand why 0 is special >at all when using absolute error. I see why it is special when >dealing with relative error, but it should be just like any other >number when we have absolute error, or am I missing something? > >John > > > >------------------------------------------------------- >SF.Net email is Sponsored by the Better Software Conference & EXPO >September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices >Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA >Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf >_______________________________________________ >OpenWeBWorK-Devel mailing list >Ope...@li... >https://lists.sourceforge.net/lists/listinfo/openwebwork-devel Prof. Arnold K. Pizer Dept. of Mathematics University of Rochester Rochester, NY 14627 (585) 275-7767 |
From: Davide P.C. <dp...@un...> - 2005-08-24 15:45:04
|
> Now that I am thinking about it, I don't understand why 0 is special > at all when using absolute error. I see why it is special when > dealing with relative error, but it should be just like any other > number when we have absolute error, or am I missing something? As I mentioned, I had originally done it with the scientific notation in mind, and hadn't been thinking about the absolute tolerance case, so the result was s little surprising to me, too. I also said that I didn't have a problem changing it in the absolute case. Another possibility is to use the zeroLevel tolerances for this all the time so as to get scientific notation turned into zero in both absolute and relative cases, but not use the standard tolerances for that at all. But I did want to consider the possibility of make the output results be more realistic for ALL answers (not just zero) by using the tolerances to show the displayed results with the proper number of digits of precision as set by the problem's tolerances. This would make the student answer more closely correspond to how WW actually treats it, and would make it much clearer how many digits of precision must be used in the student's answer. This would also help problem authors be aware of the effect of the tolerances on their answers, particularly when those answer involve either very large or very small numbers. The standard tolerances won't distinguish between 12345 and 12335, for example, and I doubt most problem writers would take that into account. Davide |
From: John J. <jj...@as...> - 2005-08-24 15:05:12
|
Davide P.Cervone wrote: > John and Gavin: > > I'm rerouting this to the developer mailing list, since it now relates > to feature decisions. > > On Aug 24, 2005, at 12:19 AM, John Jones wrote: > >> Davide P. Cervone wrote: >> >>>> For the num_cmp() evaluator, setting 'tol'=>0.5 gives the result >>>> Entered Answer Preview Correct Result >>>> 0 0 1/sqrt(7) correct >>>> and taking out the tol gives >>>> Entered Answer Preview Correct Result >>>> 0.25 0.25 1/sqrt(7) incorrect >>>> >>>> For the number_list_cmp evaluator, with the 'tol' option I get >>>> Entered Answer Preview Correct Result >>>> -0, 0 -0,0 0, 0 correct >>>> and without, >>>> Entered Answer Preview Correct Result >>>> -0.25, 0.25 -0.25,0.25 -0.37796.., 0.37796.. incorrect >>> >>> >>> >>> The reason that the parser shows the student's answer as 0 is that >>> when it stringifies a real number, it shows it as zero if it is >>> "equal" to zero according to the current equality conditions. Since >>> your answers actually ARE equal to zero (within an absolute >>> tolerance of 0.5), they get printed as zeros. When a RELATIVE >>> tolerance is used, there is a special tolerance for zero that gets >>> used for checking against zero. This is not the case for absolute >>> tolerance, because you are specifying exactly how close you want the >>> answer to be. >> >> >> I am not so sure it is good to simplify a student's answer to 0 for a >> few reasons. It exposes some of webwork's internal tricks to the >> students. In general, other answers are not simplified in this way. >> For example, 1.0001 is not simplified to 1. And, it would be >> confusing to a student to see their answer not register. I tried >> setting tolType=>'strict' in num_cmp and taking the default setting. >> Then, I submitted 0.09, webwork told me I submitted 0. >> > > Well, first of all, tolType should be one of 'relative' or 'absolute' > from what I can see. (I am unable to find a reference to tolType > being 'strict' anywhere.) num_cmp doesn't check the value of tolType > to make sure it is legal, and perhaps it should. What happens when > you tolType to something other than 'relative' or 'absolute' is > somewhat random, as some tests are for tolType equal to 'relative' and > others are for 'absolute' and both assume if it is not the one > checked, then it is the other. So when set to 'strict' you will > sometimes get the absolute branch of the test and other times the > relative branch. For the example above, you end up getting an > absolute test with the tolerance set to 1, so the answer 0.09 is > properly considered to be 0. My mistake on using strict instead of absolute for the tolType, and you do have to be closer to 0 to see the rounding effect with absolute. I tried ANS( num_cmp(5, tolType=>'absolute') ); and entered 0.000999. Webwork tells me that I entered 0 and the preview is 0. I can see that tolType relative acts differently. With ANS( num_cmp(5) ); webwork will correctly report what I enter until I get down to 0.0000000000009 which is interpreted as 0. I understand that students get confused when a number is listed in scientific notation. We get that complaint, but only when the value appearing in the problem text is in scientific notation (it happens by accident, and so we just fix the problem). But, one of the frustrations I hear about is "webwork won't take my answer". If someone enters a non-zero answer and the system says they entered 0, a common perception would be that the number they entered didn't take. Then they try to submit the same thing 10 more times with the same result, and get mad at the system. The defaults for relative comparison seem ok. If someone enters 0.0000000000009, I think you can reasonably tell them that the computer rounded it off because it was so close to 0. The main reason we currently use Parser-based answer checkers is that they make things less confusing for students. But on the absolute error example above, entering 0.0009 and being told it is 0 seems confusing. Now that I am thinking about it, I don't understand why 0 is special at all when using absolute error. I see why it is special when dealing with relative error, but it should be just like any other number when we have absolute error, or am I missing something? John |
From: Davide P.C. <dp...@un...> - 2005-08-24 12:27:22
|
> Just one additional comment about this behavior. I agree that in > cases where an answer is of smaller magnitude than the absolute > numerical tolerance, it is mathematically reasonable to treat the > answer as zero. I am a little worried that students will think that > the fact that they entered an answer that is non-zero and it is > previewed as zero will make them think that the system isn't working > correctly, however. For example, if the correct answer is evaluated > with num_cmp(sqrt(7)) and the student enters 1/sqrt(7), the system > reports > > Entered Answer Preview Result > 0 1/sqrt(7) incorrect That would only be the case with num_cmp(sqrt(7),tol=>.5), or some tolerance bigger than 1/sqrt(7), not num_cmp(sqrt(7)), which would be using relative tolerances, so the zeroLevel tolerances would be used for determining zero. Remember that this is ONLY a problem when ABSOLUTE tolerances are used, and when they are larger than the student's answer. I'm willing to say that when absolute tolerances are used, don't change to zero, but it is important to remember that, as far as WW is concerned, if tol=>.5 is in effect, 1/sqrt(7) IS zero. If you don't want 1/sqrt(7) to be considered as zero, the tolerances are set wrong, and if it is considered to be zero, why SHOULDN'T the student be made aware of that? Personally, I think that displaying ALL answers using the precision of the problem would be a help, as then students would start to become aware of such issues, which they aren't, but I'm sure some of you will disagree. :-) Davide |
From: P G. L. <gl...@um...> - 2005-08-24 12:10:04
|
Hi Davide, Just one additional comment about this behavior. I agree that in cases where an answer is of smaller magnitude than the absolute numerical tolerance, it is mathematically reasonable to treat the answer as zero. I am a little worried that students will think that the fact that they entered an answer that is non-zero and it is previewed as zero will make them think that the system isn't working correctly, however. For example, if the correct answer is evaluated with num_cmp(sqrt(7)) and the student enters 1/sqrt(7), the system reports Entered Answer Preview Result 0 1/sqrt(7) incorrect I suspect the zero might be confusing to some students. The one case where I use absolute tolerances like this is when I have students estimating values from data or from a graph. In this case I can determine what maximum error I expect them to have, and code that as an absolute tolerance. I suppose I could construct the problems to avoid answers that are zero within the tolerance, but that seems like an artificial solution. Thanks, Gavin On Wed, 24 Aug 2005 at 07:39 Davide P.Cervone wrote: > John and Gavin: > > I'm rerouting this to the developer mailing list, since it now relates to > feature decisions. > > Well, first of all, tolType should be one of 'relative' or 'absolute' from > what I can see. (I am unable to find a reference to tolType being 'strict' > anywhere.) num_cmp doesn't check the value of tolType to make sure it is > legal, and perhaps it should. What happens when you tolType to something > other than 'relative' or 'absolute' is somewhat random, as some tests are for > tolType equal to 'relative' and others are for 'absolute' and both assume if > it is not the one checked, then it is the other. So when set to 'strict' you > will sometimes get the absolute branch of the test and other times the > relative branch. For the example above, you end up getting an absolute test > with the tolerance set to 1, so the answer 0.09 is properly considered to be > 0. > > The intent of changing the student answer to zero was to change answers of > the form 1E-13 (which are confusing to some students) to 0. In the case were > RELATIVE tolerances are used, this is in fact the effect, since the relative > checks have a special and much more restricted test for zero than absolute > checks do (this is when the zeroLevel and zeroLevelTol values come into > play), so only values less than 1E-12 get affected. But when people are > setting absolute tolerances on the order of 1 or .5, the behavior is a bit > less satisfying. I think the real problem with these examples is that the > absolute tolerance is artificially large. > > On the other hand, since it seems that it is easy to get absolute tolerances > accidentally, I think it might be a good idea if the problem author were made > aware of this by the fact that small answers are being treated as zero, > otherwise he or she will be surprised when a student's answer of 0 is counted > as correct for an answer of .09 (as happened in Gavin's problem). The fact > that the Parser shows you what is happening could be considered a GOOD thing > in terms of problem authoring. Certainly it alerted Gavin to the error in > his problem. > > Perhaps changing to zero should be suppressed when absolute checks are being > made, but I still think it works for relative checks. I had considered using > the tolerance when stringifying the number so that it would appear with the > number of digits that where appropriate for the tolerance, so that an > absolute tolerance of .01 would print 1.0001 as 1.00 and relative tolerance > of 1 percent would print 10001 as 10000, but couldn't decide what a tolerance > of .05 meant in this setting, and thought that perhaps it was too slick. But > it has always bothered me that the answers showed large numbers of digits as > though they were significant, when they aren't. I'd be interested in hearing > what Mike, Arnie and Sam have to say about these issues. > > Davide -- P. Gavin LaRose, Ph.D. Program Manager (Instructional Tech.) Math Dept., University of Michigan gl...@um... "There's no use in trying," [Alice] 734.764.6454 said. "One Can't believe impossible http://www.math.lsa.umich.edu/~glarose/ things." "I daresay you haven't had much practice," said the Queen. - Lewis Carrol |
From: Davide P.C. <dp...@un...> - 2005-08-24 11:39:44
|
John and Gavin: I'm rerouting this to the developer mailing list, since it now relates to feature decisions. On Aug 24, 2005, at 12:19 AM, John Jones wrote: > Davide P. Cervone wrote: > >>> For the num_cmp() evaluator, setting 'tol'=>0.5 gives the result >>> Entered Answer Preview Correct Result >>> 0 0 1/sqrt(7) correct >>> and taking out the tol gives >>> Entered Answer Preview Correct Result >>> 0.25 0.25 1/sqrt(7) incorrect >>> >>> For the number_list_cmp evaluator, with the 'tol' option I get >>> Entered Answer Preview Correct Result >>> -0, 0 -0,0 0, 0 correct >>> and without, >>> Entered Answer Preview Correct Result >>> -0.25, 0.25 -0.25,0.25 -0.37796.., 0.37796.. incorrect >> >> >> The reason that the parser shows the student's answer as 0 is that >> when it stringifies a real number, it shows it as zero if it is >> "equal" to zero according to the current equality conditions. Since >> your answers actually ARE equal to zero (within an absolute tolerance >> of 0.5), they get printed as zeros. When a RELATIVE tolerance is >> used, there is a special tolerance for zero that gets used for >> checking against zero. This is not the case for absolute tolerance, >> because you are specifying exactly how close you want the answer to >> be. > > I am not so sure it is good to simplify a student's answer to 0 for a > few reasons. It exposes some of webwork's internal tricks to the > students. In general, other answers are not simplified in this way. > For example, 1.0001 is not simplified to 1. And, it would be > confusing to a student to see their answer not register. I tried > setting tolType=>'strict' in num_cmp and taking the default setting. > Then, I submitted 0.09, webwork told me I submitted 0. > Well, first of all, tolType should be one of 'relative' or 'absolute' from what I can see. (I am unable to find a reference to tolType being 'strict' anywhere.) num_cmp doesn't check the value of tolType to make sure it is legal, and perhaps it should. What happens when you tolType to something other than 'relative' or 'absolute' is somewhat random, as some tests are for tolType equal to 'relative' and others are for 'absolute' and both assume if it is not the one checked, then it is the other. So when set to 'strict' you will sometimes get the absolute branch of the test and other times the relative branch. For the example above, you end up getting an absolute test with the tolerance set to 1, so the answer 0.09 is properly considered to be 0. The intent of changing the student answer to zero was to change answers of the form 1E-13 (which are confusing to some students) to 0. In the case were RELATIVE tolerances are used, this is in fact the effect, since the relative checks have a special and much more restricted test for zero than absolute checks do (this is when the zeroLevel and zeroLevelTol values come into play), so only values less than 1E-12 get affected. But when people are setting absolute tolerances on the order of 1 or .5, the behavior is a bit less satisfying. I think the real problem with these examples is that the absolute tolerance is artificially large. On the other hand, since it seems that it is easy to get absolute tolerances accidentally, I think it might be a good idea if the problem author were made aware of this by the fact that small answers are being treated as zero, otherwise he or she will be surprised when a student's answer of 0 is counted as correct for an answer of .09 (as happened in Gavin's problem). The fact that the Parser shows you what is happening could be considered a GOOD thing in terms of problem authoring. Certainly it alerted Gavin to the error in his problem. Perhaps changing to zero should be suppressed when absolute checks are being made, but I still think it works for relative checks. I had considered using the tolerance when stringifying the number so that it would appear with the number of digits that where appropriate for the tolerance, so that an absolute tolerance of .01 would print 1.0001 as 1.00 and relative tolerance of 1 percent would print 10001 as 10000, but couldn't decide what a tolerance of .05 meant in this setting, and thought that perhaps it was too slick. But it has always bothered me that the answers showed large numbers of digits as though they were significant, when they aren't. I'd be interested in hearing what Mike, Arnie and Sam have to say about these issues. Davide |
From: John J. <jj...@as...> - 2005-08-22 02:52:05
|
Hi, This past week I gave a workshop for community college teachers. In explaining how they could sharing problems and problem sets, two things struck me about the File Manager module. The first was when I told them "the first thing you will always do when you get to the File Manager is double click on templates...". I would guess that only webwork experts would deal with a file outside of templates, so why not have it start there. Experts can use the up directory button if they want to access something else. The second thing was "to make an archive of a group of files, click on them ... and then click the button labelled GZIP". Maybe "Make archive" would be a better label for the button for non-unix webwork users. John |
From: Sam H. <sh...@ma...> - 2005-08-19 20:42:07
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 18, 2005, at 14:45, Davide P.Cervone wrote: >> On Aug 17, 2005, at 15:33, John Jones wrote: >> >> >> >>> Personally, I liked the way the boxes looked now. To build this >>> in as an option, my suggestion is to add two variables to >>> global.conf, something like problemHtmlStart and problemHtmlEnd. >>> They would be inserted by beginproblem() and ENDDOCUMENT() when >>> displaying a HTML related mode. >>> >>> >> >> I'm hesitant to add more wiring between global.conf and the >> problem environment, especially if it requires adding to >> defineProblemEnvir in WeBWorK::PG. Could this be added to >> specialPGEnvironmentVars, since that entire hash gets passed >> through to the problem environment? >> >> > > I would be pleased to see a method of including these boxes in > general (right now, they are actually part of the problem file, > which is a sub-optimal solution from way back when I first started > using WW and didn't want to modify the system code any more than > necessary). > > How about having something like the hardcopySnippets and > screenSnippets hashes but for htmlSnippets, and make it available > to the problems in the environment hash? There are several other > things that I would like to have available, but customizable, like > the wording of common instructions ("Use 'inf' for infinity and '- > inf' for negative infinity", or "Enter 'NONE' if there are no > solutions", and so on) that can be inserted into problems so that > they all have consistent wording. > > While this is extra wiring, it is only one extra piece, and any > additional items like this can piggyback off it. Just a suggestion. > Sounds like a good idea to me. I've added this as a feature request in the bugzilla: http://bugs.webwork.rochester.edu/show_bug.cgi?id=817 -sam - -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) iD8DBQFDBkOo14CQX+2WvVgRAmnEAKDTIhIEu7eVsc8Dk/6DbFe0pQPNvgCgl9+C Nlw2BOFbOzatFCyiEztEQ3I= =tq9O - -----END PGP SIGNATURE----- -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) iD8DBQFDBkPF14CQX+2WvVgRAgyIAKCM9nYDM07s9EWNa4pnVYDu+d2EOQCfTGQP GFO/h+ItXn/rWHgdP98fSD0= =5DrS -----END PGP SIGNATURE----- |