#6 PS Does not return results for all test requests.
Sometimes the PS does not return all results for
platforms tested,
At present the NS requests sends a reqest to run 5
tests on 3 platforms to each platform. Looking at the
NS logs, it seems that for some tests no results are
returned (i.e. there are 12 or 8 or so on results sent
back to the NS by the PS, not 15 as you would expect).
It seems that the missing results relate to ACL message
from normal platforms to platforms with problems
sending results back becase of network/firewall issues
(i.e. UCR and IPN).
The PS should be returning failures results for these
tests (which i assume are timing out.
This is manifest in the diagnostics page where several
results are missing for tests to IPN and URC, i have
checked the NS and there are definately fewer results
coming back than requested.
Logged In: YES
user_id=39190
This behaviour is aparently caused by a clash with timouts
and the way that the PS schedules test behaviours (and
possibly with the way that jade shedules those behaviours?)
such that the test-query must be responded to before the
tests themselves have completed, yeilding empty or unknown
results for some platforms.
It is not easily fixable at the moment, but we should think
about a long-term solution.
One possible solution is to run the tests anyway and ensure
that the run results can be returned the next time they are
requested, this involves some engineering in the NS (for a
correct accept-results-from field in the test query) and
some engineering in the PS (to cache and return old results)