wcgresults

wcgresults

Configuring wcgresults isn't really necessary if you don't want to download statistical data, because there will be certain options that still work, especially when BOINC is running on your device.

When using an unconfigured wcgresults you are e.g. able to see all running (R) tasks:

Fig. 1a: wcgresults -r
Note that the state of each running task is denoted by the letter R between round brackets here.

There's much more to see with an unconfigured wcgresults; this next example shows what BOINC-tasks are available in the queue on your device (see Option -Q under Options below for more explanation):

Fig. 1b: wcgresults -Q

Also, in short, how many BOINC-tasks there are in the queue:

Fig. 1c: wcgresults -N#

If you pause one or more tasks, you will also be able to see any suspended (S) tasks:

Fig. 1d: wcgresults -z

---

Logfile

After configuring wcgresults, see [Configuration], having specified the name of your account plus your verificationcode on World Community Grid, you will be able to find out what BOINC-tasks from World Community Grid are available on all your devices, e.g.:

Fig. 2a: wcgresults -dqql0

You can then also start downloading data pertaining to your validated results, using a logfile, e.g. like so:
wcgresults -daf /tmp/wcgresults.log

To make this more persistent, logging and preserving your data on a permanent basis, you could repeat this action each twelve hours using an entry in your crontab, e.g. like so:
0 8,20 * * * bin/wcgresults -l0 -daf $HOME/BOINC/wcgresults.log

In this way it is possible to do a tail -f or rather use wcglog -f on the logfile. Or use the option -l of [wcglog] instead, e.g.:

Fig. 2b: wcglog -l3

NB: A word of caution if you use the option -f. Since your logfile will get larger by returning tasks, it might be a good idea to not let it grow out of bounds. The more tasks you return, the more it will grow, the longer it may take to store data, since wcgresults uses a simple heuristic to find out if a task has already been stored: it uses the name of the task, e.g. MCM1_0189956_7440_2, its validation-state and any credit it was granted, and will store that information in memory before deciding to add any new entry to the logfile.
Once a logfile reaches the size of one gigabyte, it will take much time and memory to hold all the tasknames when searching for a match. To be a little more safe, an active logfile shouldn't probably hold much more than 4 million records, the equivalent of about one gigabyte.

An alternative way to store data is through sqlite3 using the option -F of wcgresults.

As noted in the Help pages at World Community Grid on the subject "API" (or even more precisely at https://www.worldcommunitygrid.org/help/topic.s?shortName=api#665) it is necessary to set your data sharing preferences to "Display my data" for this API to return data for your account.

---

Options

As we've already seen, wcgresults can take several options, of which some are just modifiers.
Without options or with only modifier options (see Modifier options below), wcgresults will read from standard input (or from any specified files that - usually - were already downloaded before by using option -d).

Main options

Option -c

This can be used to calculate the maximal size of the workcache to fit the maximal number of tasks per core. In a loop, you are asked to provide the average duration of one task. Example: if there is an imposed maximum of 70 tasks per core and you specify a duration of 2h24m (that's exactly 0.1 day) per task, than the program will answer 7.00 days.
To end the loop of your session you can type a dot as input. (Other ways to end a session are: a dash as input, no input or typing a Ctrl-C or a ctrl-D.)

Option -d

This option is used to download statistical data where results are concerned.
You can download your data in two formats: JSON or XML, see option -J in the list of Modifier options.
Without any other option, a summary of available tasks and results will be shown.
With modifier option -f, you can record the results that have been reported back to the server (unless they're still "Pending validation" or "Pending verification") into a logfile. (See option -f.)
Modifier option -F is almost the same as -f: when using -F, results will be recorded into a logfile using sqlite3. (See option -F.)

Option -h

This is the help option. It shows which options are available (with concise explanations) and how to use them.

Option -i

This option takes a selection from five possible uppercase letters (E, F, O, S and V, each one is the first letter from one of the items ExitStatus, FileDeleteState, Outcome, ServerState and ValidateState) as argument and each item will list some possible values and their meanings. Example: wcgresults -i SV (this will show a list of values for ServerState and ValidateState and their meanings).

Specifying '?' as argument will list all possible, recognized values by this option.

Option -M

This option will show a list of tasknames in the queue on your device, one name per line.

Option -m

This option will show a list of tasks in the queue on your device.
The option may be repeated yielding differing effects.
In the form -m three columns will be shown: Deadline, Fraction and Name. The column Name refers to the name of the task. If the task has been started, Fraction will have a value in the range 0.000000 - 1.000000.
In the form -mm four columns will be shown: Deadline, CPUtime, Percentage and Name. If the task has been started, Percentage will have a value in the range 0.0000% - 100.0000%.
In the form -mmm the same four columns will be shown of tasks that have been started only.
In the form -mmmm five columns will be shown of tasks that have been started only: Deadline, CPUtime, Remaining, Percentage and Name. The value of Remaining is the estimated remaining time for the task to finish, determined by the BOINC-client.

Option -N

This option will show a list of tasks in the queue on your device according to the columns and the selection of tasks that you specify as argument.
The selection of tasks is accomplished using lowercase letters and some symbols. The columns are denoted by uppercase letters.
The uppercase letters that can be used, are: A = App version number; B = Elapsed time; C = CPUtime; D = Deadline; E = our own Estimated time; F = Fraction; G = Resources; L = (CPU)time since Last checkpoint; O = slOt; P = Percentage; R = Remaining time; S = Status; T = estimated Total time; U = PID; V = Virtual memory size; W = Working set size.
The lowercase letters, digits and symbols that can be used, are: # = total task count; 0 = uninitialized tasks; 1 = initialized tasks; + = tasks without status '-'; . = uploaded tasks; a = aborted tasks; c = computation errors; d = downloading tasks; n = new tasks; r = running tasks; s = suspended tasks; u = uploading tasks; w = waiting tasks.
It is also possible to use % instead of P, you will then see the progress per task inside the field Percentage in steps of 10%, using ANSI.
(All available lowercase and uppercase letters and symbols that are understood by wcgresults when using the option -N can also be found by specifying the option -N without argument or by option -h only.)

The columns in the output are separated from each other by a tab and are listed in a fixed order.
Example 1: wcgresults -NN
This will show only one column containing all tasknames. Explanation: since this option needs an argument and the default is to show only tasknames, it is possible to show tasknames only by specifying N as an argument. (Note: in this way it's also an uppercase letter that can be used, yes.)
Example 2: wcgresults -N OSru
This will show each running (r) or uploading (u) task together with its Status and its slOt.
Example 3: wcgresults -N.#
This will just show the number (#) of tasks that are 'Ready to report' (.).

Note: uninitialized tasks generally have status '-'; as an example, this will change into 'S' when they are suspended, so they will be listed by using '+' as an argument of option -N; after resumption their status will change back into '-'.

Specifying '?' as argument will list all possible, recognized values by this option.

Option -Q

This option will show the number of tasks in the queue on your device, grouped by the name of their project.
Let's have a look at a sample from a line of output: MCM1 _#_:49 [_0_:35 _1_:11 _2_:3] (R):15 (.):1
Here, the number of tasks by the project- or groupname MCM1 is 49, the number of tasks with suffix '0' is 35, with suffix '1' it's 11 and with suffix '2' it's 3, and 35 + 11 + 3 = 49. There are 15 running (R) tasks and 1 task is 'Ready to report' (.).
By adding another -Q the meaning of all the characters between parentheses will also be shown: (A) = Aborted; (C) = Computation error; (D) = Downloading; (N) = New; (R) = Running; (S) = Suspended; (U) = Uploading; (W) = Waiting to run; (.) = Ready to report.
See also option -q.

Option -r

This option will show a list of running tasks in the queue on your device, together with their percentages.
In the form -r the headerline is concise, showing something like this: 51 tasks: 41 downloaded, 10 uploaded
In the form -rr the headerline is extended, showing something like this: 51 tasks: 41 downloaded, 10 uploaded, 1 running, 0 waiting

Option -S

This option is used to retrieve the statistics for you (as a member) and your team.
In the form -S the output will be in XML.
In the form -SSS the output will be userfriendly, much easier to read and understand.
In the form -SSSS the program will mimic the output of the program [wcgstats] (without options).

Option -T

This option will show a list of tasks in the queue on your device, one per line, together with its status.

Option -t

This option will show a list of tasks in the queue on your device, if possible in two or more columns, together with their statuses.

Option -U

This option lets you update the program. Its argument specifies the file that is the new version. In this way your credentials inside the program itself will be kept intact.

Option -V

This option shows the version of the program.

Option -X

This option will show a list of current transfers between the server and your client. Whenever a new task is being downloaded or a result is being uploaded, there will be one or more files in the transferqueue.
Example:

Up/Down    Speed   Sticky   Active    Elapsed    Xferred    Filename
  up    550 kB/s       no      yes    0:00:27   14286848    ARP1_0029510_135_0_r739599116_4
  up    551 kB/s       no      yes    0:00:21   11010048    ARP1_0029510_135_0_r739599116_5
  up      0  B/s       no       no    0:00:00          0    ARP1_0029510_135_0_r739599116_6

Option -x

This option will retry, in a loop, the current transfer of any file(s) in the transferqueue, using a lockfile. If the lockfile is still present, wcgresults will exit (with exitcode 1) like this:
[wcgresults] Lockfile ”/tmp/wcgxmit.LOCK” already exists, exiting.
If there is work to do (having files in the transferqueue) and the lockfile is absent, wcgresults will create the lockfile and write its process ID to the lockfile.

Each time after the loop has gone through, one iteration is completed. As soon as the queue is empty, the total number of iterations will be reported.

During operation, when there are files to be transferred, a session may look like this:

[Sat 24 Sep 2022 09:39:45 CEST] Creating lockfile /tmp/wcgxmit.LOCK ...
(1 file remaining)
09:41:15 retry downloading task MCM1_0190870_2627_MCM1_0190870_2627.txt
(total 1 iteration)
[Sat 24 Sep 2022 09:42:45 CEST] Removing lockfile /tmp/wcgxmit.LOCK ...

Note: sometimes a file will already have succeeded in being transferred when being retried, which will cause this type of harmless message:

22-Sep-2022 15:13:55: GUI RPC error: No such transfer waiting
Operation failed: Error -1

By adding another -x the command boinccmd --network_available will be issued instead, which means that the BOINC-client will retry a deferred network communication.

NB: You may need to install the Term::Size module of Perl. In that case, this is what you should do as superuser:
For users of Fedora: dnf install perl-Term-Size
For users of Ubuntu: apt install libterm-size-perl

Option -z

This option will show a list of suspended tasks in the queue on your device.
In the form -z the headerline is concise, showing something like this:
51 tasks: 41 downloaded, 10 uploaded
By adding another -z or in the form -zz the headerline is extended, showing something like this:
51 tasks: 41 downloaded, 10 uploaded, 0 suspended, 0 waiting

Modifier options

Option -A

This is used to omit ANSI-escape-sequences in the output on your screen. This might come in handy if you want to process its output any further.
E.g., instead of the output in Figure 2a, you would see this:
2022-02-20 18:11:35 desktop mcm1:13 opn1:8 laptop mcm1:15 opn1:26 smartphone mcm1:1 = 63 @ 3

Option -a

This option is an addition to option -d. Together with option -f it will also record results in the logfile that have gotten 0 points, e.g. after being aborted, after being marked as Too Late or after a Computation Error.

Option -b

This option is an addition to option -d. Its argument specifies in which order to sort the results. Default is to sort by ReceivedTime from the viewpoint of the server.
Possible numerical values of its argument are: 0 - sort by DeviceId; 1 - sort by SentTime; 2 - sort by ReportDeadline; 3 - sort by ReceivedTime; 4 - sort by CpuTime.

Specifying '?' as argument will list all possible, recognized values by this option.

Option -D

This option is an addition to option -d. Its argument specifies the directory in which to download results. Default is /var/tmp/.

Options -f and -F

These two options both take a filename as argument. See also main option -d. Downloaded results are, by default, sorted by CpuTime.
Examples:

wcgresults -l0 -daf wcg.log
wcgresults -l0 -daF wcg.db

(See also [wcglog] on how to use sqlite3 in the case of using option -F.)

Option -H

This option will omit the header when using option -M, -m, -N, -Q, -r, -t or -z. If you add another -H, only the header will be shown.

Option -J

This option will use JSON instead of XML. It is especially useful when or after using the option -d to download batches.
You can choose to download the batches in two formats: JSON or XML. However, on 24-01-2024 downloading in the JSON-format resulted in "Internal Server Error", that is why this option was added. Downloading batches in XML-format is now the default.
One of the advantages of using JSON is that the size of a downloaded JSON-batch is somewhat smaller than a downloaded XML-batch.

Option -l

This option (ell) is an addition to option -d. This is to limit the number of results to download in a batch. Its argument specifies that limit. Specifying a value of '0' (-l0) will download all available results. When the option -F, -f or -q is not specified, only one batch will be downloaded; when the value 0 is specified, all available batches will be downloaded. A batch will maximally contain 250 results by default.

Option -O

This option is an addition to option -d. Its numerical argument specifies to select only results by value of their Outcome (1 = success; 3 = error; 4 = no reply; 6 = validate error; 7 = abandoned).

Specifying '?' as argument will list all possible, recognized values by this option.

Option -o

This option is an addition to option -d. Its argument specifies the number of results to skip (the offset) when downloading. This can be used when you are downloading results in batches, which is the default (see option -l).

Option -q

This option is an addition to option -d. It will show total numbers of tasks in progress across all your devices on one line. Most useful in combination with options -d and -l0 (ell zero).
Using -q will show the numbers per device, -qq will show a total per app per device, -qqq will show the totals per app.

Option -R

This option can be used in combination with one of the options -M, -m, -N, -Q, -r, -T, -t, -X, -x and -z.
It will allow making a connection to a remote BOINC-client, using a symbolic name for that client. You will then be referring to that client by using its symbolic name. An example, using the symbolic name xyz for a certain BOINC-client:

$ wcgresults -R xyz -NO1
slOt    Name----------------
   0    ARP1_0026243_129_0
   1    MCM1_0191025_0127_0
   2    MCM1_0191025_0193_1
   3    MCM1_0191025_0135_0

Here, in this example, the symbolic name xyz was used to connect to a certain BOINC-client, request its list of tasks and then act as if the command was issued on the local host with that same list of tasks.

To put this to work, you will have to make up a symbolic name for the client and then configure two shell-variables. The symbolic name needs to adhere to being part of the name of a shell-variable, so you can only use the characters 0-9, a-z, A-Z and _ (that's 63 in total) for it.
The names of the two shell-variables start out with REMOTEHOSTNAME_ and REMOTEPASSWORD_. You need to add the symbolic name to the end of their names, e.g. if the symbolic name is xyz, the resulting two shell-variables would be REMOTEHOSTNAME_xyz and REMOTEPASSWORD_xyz. You then need to supply at least the real name or the IP-address of the client, optionally followed by a colon and a portnumber, and optionally the password that is needed for connecting to the client. E.g. if the client's real name would be example.com and its password would be topsecret, then you would have:

REMOTEHOSTNAME_xyz="example.com"
REMOTEPASSWORD_xyz="topsecret"

Now, the default portnumber for a BOINC-client is 31416. If you have chosen to run that remote client on a different portnumber, then you should add the value of that portnumber to the value of the remote hostname, separated by a colon, like so: "example.com:portnumber". If the portnumber would be 12345, then you would have:

REMOTEHOSTNAME_xyz="example.com:12345"
REMOTEPASSWORD_xyz="topsecret"

Note: it must be made clear that the password cannot contain a space character, nor a tab or even a newline. Be careful using exclamation marks, backslashes, apostrophes and quotation marks, they might interfere. The password may be empty (or absent).
Then, verify that this setup works by running boinccmd like it is shown below. E.g. if the client's symbolic name is xyz, if the client's real name is example.com and its password topsecret, then you would use something like this:

$ opt_R=xyz
$ REMOTEHOSTNAME_xyz="example.com"
$ REMOTEPASSWORD_xyz="topsecret"
$ eval arg=\$REMOTEHOSTNAME_$opt_R; [ -n "$arg" ] && ARG_HOSTNAME="--host $arg"
$ eval arg=\$REMOTEPASSWORD_$opt_R; [ -n "$arg" ] && ARG_PASSWORD="--passwd $arg"
$ BOINCCMD="boinccmd $ARG_HOSTNAME $ARG_PASSWORD"
$ $BOINCCMD --get_host_info
  timezone: 7200
  domain name: example.com
  IP addr: 93.184.216.34
  #CPUS: 2
  …

You could put this in a shell-script like shown below, first fill out the values for the variables opt_R, REMOTEHOSTNAME_$opt_R (and optionally REMOTEPASSWORD_$opt_R) where you see two quotation marks next to each other (empty valued), then try it out yourself:

#!/bin/bash
opt_R=""
eval REMOTEHOSTNAME_$opt_R=""
eval REMOTEPASSWORD_$opt_R=""
eval arg=\$REMOTEHOSTNAME_$opt_R; [ -n "$arg" ] && ARG_HOSTNAME="--host $arg"
eval arg=\$REMOTEPASSWORD_$opt_R; [ -n "$arg" ] && ARG_PASSWORD="--passwd $arg"
BOINCCMD="boinccmd $ARG_HOSTNAME $ARG_PASSWORD"
$BOINCCMD --get_host_info

It may be worth noting that the remote BOINC-client needs to have <allow_remote_gui_rpc>1</allow_remote_gui_rpc> configured in the file "cc_config.xml" in its BOINC-directory, or was started by using the option --allow_remote_gui_rpc, to allow RPC-connections from remote hosts. See also the manual pages of boinc(1) and boinccmd(1).

Option -s

This option is an addition to option -d. Its numerical argument specifies to select only results by value of their ValidateState (0 = pending validation; 1 = valid; 2 = invalid; 4 = pending verification; 5 = results failed to validate within given deadline).

Specifying '?' as argument will list all possible, recognized values by this option.

Option -u

This option is an addition to option -d. If you are managing other accounts, this option lets you specify which other account to use. Probably best practice is to number your accounts 1, 2, 3, etc. Example: wcgresults -da -l0 -u1 -f /tmp/wcg-u1.log says to download all results for account 1 to logfile /tmp/wcg-u1.log.
Note: if you're using the options -d and -f (e.g. in your crontab) to save your own statistical data to a logfile, say LOGFILE.log, then the best practice would be to use the name LOGFILE-u1.log for the logfile of account 1 and LOGFILE-u2.log for the logfile of account 2, etc. (i.e. with -u1 and -u2 in the name of the logfile between "LOGFILE" and ".log", respectively) when you want to use [wcglog].

Note: of course it is also possible to use (other) symbolic names for your other accounts, e.g. use -u xx for account 'xx', which then corresponds with variables MEMBERxx and VERIFYxx. As always, you can do this as long as the characters of the symbolic names produce valid variable names. So, for clarification, you can only use the characters 0-9, a-z, A-Z and _ (that's 63 in total).

Option -w

This option is an addition to option -d. It will only select results that are still 'In Progress'. Example:

$ wcgresults -dw -D /tmp -J
--2022-02-21 02:05:17--  https://www.worldcommunitygrid.org/api/members/***/results?code=***&format=json&ServerState=4&SortBy=ReceivedTime&Limit=250
Resolving www.worldcommunitygrid.org (www.worldcommunitygrid.org)... 169.47.63.74
Connecting to www.worldcommunitygrid.org (www.worldcommunitygrid.org)|169.47.63.74|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 8409 (8.2K) [application/json]
Saving to: ‘/tmp/wcgresults.2022-02-21T02:05:17.1584265’

/tmp/wcgresults.2022-02-21T02:05:17.1584265    100%[==================================>]   8.21K  --.-KB/s    in 0s      

2022-02-21 02:05:17 (291 MB/s) - ‘/tmp/wcgresults.2022-02-21T02:05:17.1584265’ saved [8409/8409]

Results returned: 13
(DeviceName)     (App)  (Outcome) (ValidateState)      (Exit) (Claim/Grant) (CpuTime/Elapsed)  (Name)
laptop           opn1   (waiting) In progress          -      ....../......  ......./.......   OPN1_0109506_00072_1
ubuntulaptop     opn1   (waiting) In progress          -      ....../......  ......./.......   OPN1_0109506_02289_1
fedoralaptop     mcm1   (waiting) In progress          -      ....../......  ......./.......   MCM1_0190033_7278_2
ubuntulaptop     mcm1   (waiting) In progress          -      ....../......  ......./.......   MCM1_0190032_4824_2
laptop           mcm1   (waiting) In progress          -      ....../......  ......./.......   MCM1_0190039_6270_2
ubuntulaptop     mcm1   (waiting) In progress          -      ....../......  ......./.......   MCM1_0190039_0766_2
ubuntulaptop     opn1   (waiting) In progress          -      ....../......  ......./.......   OPN1_0109405_01935_2
ubuntulaptop     opn1   (waiting) In progress          -      ....../......  ......./.......   OPN1_0109556_02075_1
ubuntulaptop     mcm1   (waiting) In progress          -      ....../......  ......./.......   MCM1_0190116_8961_2
ubuntulaptop     mcm1   (waiting) In progress          -      ....../......  ......./.......   MCM1_0190146_2504_2
smartphone       mcm1   (waiting) In progress          -      ....../......  ......./.......   MCM1_0190149_9394_3
ubuntulaptop     mcm1   (waiting) In progress          -      ....../......  ......./.......   MCM1_0190233_0925_2
ubuntulaptop     mcm1   (waiting) In progress          -      ....../......  ......./.......   MCM1_0190219_5700_3

Authentication

For some of the options, wcgresults relies on using boinccmd internally. This means that boinccmd will look in the current directory if there is a file "gui_rpc_auth.cfg" and if there is, use its contents as the password for access to BOINC. If the authentication fails, wcgresults will try to change directory to $BOINCDIR (as defined by wcgresults), before issuing any further boinccmd commands internally.