CLI

CLI Commands

The CLI is currently comprised of the following commands:

• benchmark
• soak
• smoke
• stress
• operations

Each command is discussed in detail on this page, you can also see the relevant javadoc for the commands at http://sparql-query-bm.sourceforge.net/javadoc/latest/cmd/

This documentation will refer to the *nix style commands but any command can run on Windows by adding .bat to the command name.

benchmark command

The benchmark command as the name implies runs a benchmark. A benchmark consists of some number of warmups followed by some fixed number of runs after which the best and worst outliers are discarded and summary statistics are generated.

The benchmark command is run like so:

./benchmark -q http://host/query -m mix.tsv -r 250 -w 10 -c results.csv

The above runs a benchmark against the given query endpoint http://host/query using the operation mix mix.tsv running 10 warmups with 250 timed runs and the final results being output to results.csv. To learn more about operation mixes please see the Operation Mix page.

There are many other benchmarking options supported, please run the following to see built-in help for all available options:

./benchmark --help

For details on the benchmarking methodology used please see the Benchmarking Methodology page.

soak command

The soak command runs a soak test. A given operation mix is run continuously for some period of time, the period to run for may be specified in terms of number of runs, runtime or both.

The soak command is run like so:

> ./soak -q http://host/query -m mix.tsv --max-runtime 180 --max-runs 250

The above runs a soak test against the given query endpoint http://host/query using the operation mix mix.tsv running for either 180 minutes or 250 runs whichever comes first. To learn more about operation mixes please see the Operation Mix page.

There are many other soak testing options supported, please run the following to see built-in help for all available options:

> ./soak --help

smoke command

The smoke command runs a smokr test. A given operation mix is run once in order to see whether any operations fail. It is intended for doing simple regression/status checking on a SPARQL system.

The smoke command is run like so:

> ./smoke -q http://host/query -m mix.tsv

The above runs a smoke test against the given query endpoint http://host/query using the operation mix mix.tsv running it once only. To learn more about operation mixes please see the Operation Mix page.

There are many other smoke testing options supported, please run the following to see built-in help for all available options:

> ./smoke --help

stress command

The stress command runs a stress test. A given operation mix is run continuously under progressively increasing load to see how the system reacts to high load. The test starts with using the number of parallel clients specified by the -p/--parallel option and has each client do a complete run of the mix in parallel. The number of parallel clients is then multiplied by the ramp up factor which defaults to 2 and can be configured using the --ramp-up option and the process is repeated. This repeats until either the maximum runtime or the maximum number of threads are reached.

The stress command is run like so:

> ./stress -q http://host/query -m mix.tsv --max-runtime 180 --max-threads 128 --ramp-up 2

The above runs a stress test against the given query endpoint http://host/query using the operation mix mix.tsv running for either 180 minutes or 128 threads whichever comes first with the number of threads being increased by a factor of 2 each time. This would mean the test would run with 1 thread, then 2, 4, 8, 16, 32 and so on until it runs with 128 threads before terminating.

To learn more about operation mixes please see the Operation Mix page.

There are many other stress testing options supported, please run the following to see built-in help for all available options:

> ./stress --help

operations command

The operations command provides a built-in help service detailing what operations are supported by the system and can be included in your operation mixes as detailed on the Operation Mix page.

To see the list of available operations run the following:

> ./operations

To see operation specific help run the following:

> ./operations --op name

Where name is the name of an operation you wish to see help for, adding the --verbose flag to this will print additional information about the supported arguments for configuring an operation.