[Cobalt-developers] BOSS Development, new features, improvements and bugfixes

[Harri <har...@tu...>]

During the past 12 months that we've been working with BOSS here at=20
Tampere University of Technology, the following improvements, features=20
and bugfixes have been developed.

We offer these updates to be committed to the latest revision of BOSS at=20
Warwick, so that they will be available to the public in next version=20
to be released, or accessible via public CVS access before the next relea=
se.

Bugfixes:

- In Submissions Frame
 - In Submissions frame the Advance-button works even when the rows are=20
   sorted
 - Running autotests for several submissions while the rows are=20
   sorted works
 - Running autotest updates the row to show "Autotested"
 - Penalise lateness for multiple students works correctly
 - View extension date in its own column, not in Date column

- In MarkingCategories
  - After visiting another Problem's Marking Category and creating=20
    a new Marking Category in another problem the Class Library actually
    reflects the current Problem

- In Marking Submission
  - Submission can be viewed in a source code viewer without downloading =
the
    files to local filesystem
  - The report from the tests run for the student's submission when they
    submitted is stored and can be viewed in Marking Submission.

- Getting and viewing the results of a multicomponent sums up the=20
  results correctly. Previously only the results from the last component
  were summed in.

- Submission
  - Allow submission after Finish if extended deadline has been assigned
    to a student.

Features and improvements:
- Allow negative weights for mark categories
  - a failed test can reduce from the final mark from the positive=20
    weight categories.

- In Submissions frame the rows can be sorted in ascending and=20
  descending order.

- Marking Categories can be ordered by setting a priority to each of=20
  them
  - Categories with priorities of 10 and under are considered special.
    If such a test fails, the whole test run will be aborted.
    - TODO: Probably the priority field in the database and the object=20
      shouldn't control both of these features. I would be a better idea
      to have to separate variables.
  - Marking Categories will appear in the priority order and the
    associated tests will be executed in the priority order

- In Assessment Frame
  - Reordered the check boxes in Assessment Frame so that "Allow Access
    to this Assessment" is above the other access controls.
  - Disabling "Allow Access to this Assessment" disables the other=20
    access controls.
  - Submission policy has a setting for maximum number of submission=20
    attempts allowed per student
  - It is possible to give extra submissions to a student
  - The Time limit set in Assessment Frame is used for the new test types
    - Currently the Simple Auto Test has a fixed 10 second time limit
  - A fallback time limit can be set in Assessment frame. The whole test =
run
    will be aborted if running all of the tests takes longer than that.
  - A low mark limit can be set, so that submissions with too low a
    score will not be accepted
  - A low mark limit for assessment queue can be set, so that submissions
    with too low a score will not be accepted to the assessment queue.
    Such submissions are still taken into account in submissino count and
    maximum submissions.

- In Problem Frame
  - A low mark limit for passing a problem can be set. If the student get=
s
    higher or equal mark, ** PASSED ** will be added to the feedback emai=
l.
    Otherwise ** FAILED ** will be added.

- In Tree Frame
  - Refreshing the Submissions frame no longer mess up the tree

- Automatic tests can award marks from 0 to 10 instead of just 0 and 10.

- A commandline tool for staff
  - Listing submissions to a problem
  - Listing marks from autotests
  - Downloading a submission

- Submission
  - A commandline interface for students for easy submission
  - BOSS can be configured so that whenever a student submits,
    the student-tests will be run, and the results will be shown to
    the student. In this mode the student can't execute the student
    tests more times than they are allowed to submit.

- A commandline interface to register a student to boss

- Added possibility to bypass username+password authentication
  - The authentication is based on the current unix user on a trusted
    shell server
  - Also possible to add new authentication methods by writing an
    authentication module

- Separated the different test types (JUnit, Simple Test,=20
  etc.) into separate classes.
  - There are corresponding classes inherited from
    - GenericAction, ActionPanel and TestAction for each test type

- Implemented new test types
  - Compile
    - Can be used to execute a compile command
    - Compiler and the parameters can be set
    - Can be used to execute other commands too. Maybe the name of the
      test and the class should be change to "Execute" or "Command"
  - Diff
    - The student's progarm is run with some input. The output from
      the student's program is compared to the expected output in a=20
      file saved in the Class Library.
    - The mark is determined by the percentage of correctness in the=20
      output
    - TODO: Specifying a certain piecewise linear scaling to the=20
      percentage to get the final mark should be made possible
  - Regex
    - The student's program is run with some input. The output is
      compared to a list of rules in a file saved in the Class Library.
    - The rules specify a regular expression to match and a number of
      points to be added or subtracted when matched in the output.
    - The points can be adjusted by a piecewise linear scaling.
    - The format of rules is same as in Ceilidh's oracle.
  - External Tool
    - An external program can be used to test some aspect of the=20
      student's submission.
    - BOSS will expect a line of the format "CAA-score: 10"
    - Other output with "CAA-" prefix is handled specially
    - External tools should be installed in a certain directory
    - TODO: Choosing the tool to use from a list would be nice
      - The list could be generated by reading through the files in the
        external tools directory, or by reading a list of tools file
        in that directory.
    - External Tool test makes it easy to add new test types to BOSS=20
      without needing to modify BOSS itself.
    - The same external tools could work in other CAA (Computer assisted=20
      assessment) systems, if they would understand the same output=20
      format and are expected to be executed in the same directory as
      the student's submission. In other cases it might be possible to
      adjust the tool to work with small changes.

- To allow smoother working and making it easier to provide help for stud=
ents
  regarding their submissions a moderator can look at a submission even b=
efore
  it has been marked

- Administrator can add and remove administrators

- While looking for junit.jar from the classpath, junit-X.Y.Z.jar will ma=
tch
  too.

- Displaying the cause exception in GUI helps understand the error messag=
e and
  the problem at hand better.

- Catch and report unexpected exceptions in test runs

- Listening to a fixed port in StudentServer, StaffServer and SlaveServer
  makes it easier to establish secure firewall rules


Other improvements:
- Fixed some typos in comments
- Removed unused obsoleted functions
- Cleaned up some code a bit
- Fixed some bugs


Yours,
Harri J=E4rvi