Menu
▾
▴
iverilog-devel
|
From: Maciej S. <mac...@ce...> - 2014-09-02 08:16:19
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I would like to ask for your opinion about executing the ivtest suite on parallel cores. I have modified the vvp_reg.pl script [1], so it can use multiple instances of vvp at the same time. The script uses Parallel::ForkManager (available in CPAN) to run iverilog & vvp instances in separate processes to decrease the time required to perform tests on modern CPUs. I have managed to go down from ~1 minute to ~15 seconds on 8-core i7 with SSD. The approximate performance boost increases linearly up to 4 cores, then the difference becomes insignificant. You may specify the number of used processes by modifying the CPUS environmental variable, e.g. "CPUS=8 ./par_vvp_reg.pl" runs with 8 processes. Otherwise the number of available cores is autodetected (works only in Linux, relies on /proc/cpuinfo). Unfortunately, there are drawbacks too: * I modified fscanf* tests, because they were using the same file and therefore stayed in conflict. It does not change results for the default vvp_reg.pl script. It is an example of restrictions that test writers would have to obey. * The output is not fully compatible with the default output stored in the repository. I decided to sort the test results in lexical order, otherwise the order would be random each run (depends on the time when a test ends). * There are two tests (pr2509349a & sys_func_task_error) that I could not fix. I guess this might be a result of changing the iverilog output file name from vsim to $tname.vsim (to avoid conflicts). I did not modified them, because I do not want to break the default test scripts. * Output files are saved in the main directory, they are removed at the end of the script. I do not remove ivl_vhdl_work directory between tests, because it could be used in another process. More elegant solution would be to use separate directories. If you think this is completely wrong - please let me know. * This is the first time I write code in Perl, therefore I am not sure if everything is done in accordance with the art. With all the mentioned shortcomings, I think that it might be a good tool for quick, preliminary checks that could save developer's time. What do you think? Regards, Orson [1] https://github.com/orsonmmz/ivtest/tree/parallel_test -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBAgAGBQJUBXzJAAoJEBRwGu1hpbJ1etMIAJyQSFeF0/p4YWlBQgpWQPml NIv2YZeKlCQHYSYfNFDIVkevnfCSNXvv6+4l0BWy4F66yfOJiZ8Y01KOH/sPWg11 2NLfnlYxbMNKLpnvERsxp6NVe41ylxjg/QPS9RFqg4NvXEAhbIW4rtd9wT1GRG3y 1+tSj4IIrC0PrVS7t0vB18dA/MB1X38aiP8yRav0EzZoRPGQHKT+yhvuoesfjtbO fD10GPqjaZnVTK+2TBfh7R00XwwVgIVuxaqall78OuKk5s1Xr5nWD7XZjsGSQGRu S/gjOtd77KSTXRNq+1jkOf/DR1DhVsBXLtaSk5tvLn1JJtjS431vMyxXxro7bjo= =w7yz -----END PGP SIGNATURE----- |
|
From: Cary R. <cy...@ya...> - 2014-09-05 23:58:31
|
Hi Orson, Conceptually I like the idea of running the tests in parallel, but this implementation seems a bit fragile When I have though about this it always seemed like each run need its own environment so we did not need to make sure test results/files were unique. This also allows the test files to continue to have a standard name that should avoid the failing tests you mentioned. As far as the order is concerned, can the test results be cached as needed and then emitted in the correct order as the jobs finish. I realize that output blocks when a long test is running since any results for tests after the long test must be held in the cache until the long test finishes and then the cache can be flushed. Basically as each thread is started it is given a new entry at the end of a buffer to put its result and the emitting routine works off the other end emitting and reclaiming results as they become available. Something like the following for two threads: Thread 1 is given the first test and the first buffer location. Thread 2 is given the second test and the second buffer location. Thread 2 finishes and puts the result in buffer location two. Thread 2 is given the third test and the third buffer location. Thread 2 finishes and puts the result in the third buffer location. Thread 2 is given the fourth test and the fourth buffer location. Thread 1 finishes and puts the result in buffer location one Thread 1 is given the fifth test and the fifth buffer location. The emitting routine print the results for tests one, two and three and reclaims the space. etc. Of course in perl this may get more complicated than this, but I'm guessing a list may work for this though I'm not sure since I have never looked at multi-threaded code in perl. I used to be very fluent in perl so I can help refine the code as needed. On Tuesday, September 2, 2014 9:26 AM, Maciej Sumiński <mac...@ce...> wrote: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I would like to ask for your opinion about executing the ivtest suite on parallel cores. I have modified the vvp_reg.pl script [1], so it can use multiple instances of vvp at the same time. The script uses Parallel::ForkManager (available in CPAN) to run iverilog & vvp instances in separate processes to decrease the time required to perform tests on modern CPUs. I have managed to go down from ~1 minute to ~15 seconds on 8-core i7 with SSD. The approximate performance boost increases linearly up to 4 cores, then the difference becomes insignificant. You may specify the number of used processes by modifying the CPUS environmental variable, e.g. "CPUS=8 ./par_vvp_reg.pl" runs with 8 processes. Otherwise the number of available cores is autodetected (works only in Linux, relies on /proc/cpuinfo). Unfortunately, there are drawbacks too: * I modified fscanf* tests, because they were using the same file and therefore stayed in conflict. It does not change results for the default vvp_reg.pl script. It is an example of restrictions that test writers would have to obey. * The output is not fully compatible with the default output stored in the repository. I decided to sort the test results in lexical order, otherwise the order would be random each run (depends on the time when a test ends). * There are two tests (pr2509349a & sys_func_task_error) that I could not fix. I guess this might be a result of changing the iverilog output file name from vsim to $tname.vsim (to avoid conflicts). I did not modified them, because I do not want to break the default test scripts. * Output files are saved in the main directory, they are removed at the end of the script. I do not remove ivl_vhdl_work directory between tests, because it could be used in another process. More elegant solution would be to use separate directories. If you think this is completely wrong - please let me know. * This is the first time I write code in Perl, therefore I am not sure if everything is done in accordance with the art. With all the mentioned shortcomings, I think that it might be a good tool for quick, preliminary checks that could save developer's time. What do you think? Regards, Orson [1] https://github.com/orsonmmz/ivtest/tree/parallel_test -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBAgAGBQJUBXzJAAoJEBRwGu1hpbJ1etMIAJyQSFeF0/p4YWlBQgpWQPml NIv2YZeKlCQHYSYfNFDIVkevnfCSNXvv6+4l0BWy4F66yfOJiZ8Y01KOH/sPWg11 2NLfnlYxbMNKLpnvERsxp6NVe41ylxjg/QPS9RFqg4NvXEAhbIW4rtd9wT1GRG3y 1+tSj4IIrC0PrVS7t0vB18dA/MB1X38aiP8yRav0EzZoRPGQHKT+yhvuoesfjtbO fD10GPqjaZnVTK+2TBfh7R00XwwVgIVuxaqall78OuKk5s1Xr5nWD7XZjsGSQGRu S/gjOtd77KSTXRNq+1jkOf/DR1DhVsBXLtaSk5tvLn1JJtjS431vMyxXxro7bjo= =w7yz -----END PGP SIGNATURE----- ------------------------------------------------------------------------------ Slashdot TV. Video for Nerds. Stuff that matters. http://tv.slashdot.org/ _______________________________________________ Iverilog-devel mailing list Ive...@li... https://lists.sourceforge.net/lists/listinfo/iverilog-devel |
|
From: Martin W. <mai...@ma...> - 2014-09-08 20:16:41
|
My initial thought on this was that the regression tests are sufficiently
quick (~1 minute on my PC as well, even with a standard hard disk) that it is
hardly worth the effort. The regression tests have usually completed by the
time I have finished composing the commit message!
Having said this, we are continually adding tests and increasing the test
time, so I certainly wouldn't discourage you from doing this.
One thing I do think would be useful, if we are reworking the test scripts,
would be to split regress.list into a set of separate files, something like:
regress-vlg.list traditional Verilog tests that should
work with any Verilog compiler
regress-sv.list SystemVerilog tests that should
work with any SystemVerilog compiler
regress-vhdl.list VHDL tests that should
work with any VHDL compiler
regress-ivl.list tests using Icarus Verilog extensions
regress-syn.list tests using Icarus Verilog synthesis
regress-0.9.list exceptions for V0.9
regress-0.8.list exceptions for V0.8
and then selecting the appropriate subset of files depending on the compiler
version. This would eliminate a lot of the V0.8 and V0.9 exceptions (and save
us adding a lot more).
What do other people think of this idea?
Martin
Maciej Sumiński wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi,
>
> I would like to ask for your opinion about executing the ivtest suite
> on parallel cores. I have modified the vvp_reg.pl script [1], so it
> can use multiple instances of vvp at the same time.
>
> The script uses Parallel::ForkManager (available in CPAN) to run
> iverilog & vvp instances in separate processes to decrease the time
> required to perform tests on modern CPUs.
> I have managed to go down from ~1 minute to ~15 seconds on 8-core i7
> with SSD. The approximate performance boost increases linearly up to 4
> cores, then the difference becomes insignificant.
> You may specify the number of used processes by modifying the CPUS
> environmental variable, e.g. "CPUS=8 ./par_vvp_reg.pl" runs with 8
> processes. Otherwise the number of available cores is autodetected
> (works only in Linux, relies on /proc/cpuinfo).
>
> Unfortunately, there are drawbacks too:
> * I modified fscanf* tests, because they were using the same file and
> therefore stayed in conflict. It does not change results for the
> default vvp_reg.pl script. It is an example of restrictions that test
> writers would have to obey.
> * The output is not fully compatible with the default output stored in
> the repository. I decided to sort the test results in lexical order,
> otherwise the order would be random each run (depends on the time when
> a test ends).
> * There are two tests (pr2509349a & sys_func_task_error) that I could
> not fix. I guess this might be a result of changing the iverilog
> output file name from vsim to $tname.vsim (to avoid conflicts). I did
> not modified them, because I do not want to break the default test
> scripts.
> * Output files are saved in the main directory, they are removed at
> the end of the script. I do not remove ivl_vhdl_work directory between
> tests, because it could be used in another process. More elegant
> solution would be to use separate directories. If you think this is
> completely wrong - please let me know.
> * This is the first time I write code in Perl, therefore I am not sure
> if everything is done in accordance with the art.
>
> With all the mentioned shortcomings, I think that it might be a good
> tool for quick, preliminary checks that could save developer's time.
> What do you think?
>
> Regards,
> Orson
>
> [1] https://github.com/orsonmmz/ivtest/tree/parallel_test
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2
>
> iQEcBAEBAgAGBQJUBXzJAAoJEBRwGu1hpbJ1etMIAJyQSFeF0/p4YWlBQgpWQPml
> NIv2YZeKlCQHYSYfNFDIVkevnfCSNXvv6+4l0BWy4F66yfOJiZ8Y01KOH/sPWg11
> 2NLfnlYxbMNKLpnvERsxp6NVe41ylxjg/QPS9RFqg4NvXEAhbIW4rtd9wT1GRG3y
> 1+tSj4IIrC0PrVS7t0vB18dA/MB1X38aiP8yRav0EzZoRPGQHKT+yhvuoesfjtbO
> fD10GPqjaZnVTK+2TBfh7R00XwwVgIVuxaqall78OuKk5s1Xr5nWD7XZjsGSQGRu
> S/gjOtd77KSTXRNq+1jkOf/DR1DhVsBXLtaSk5tvLn1JJtjS431vMyxXxro7bjo=
> =w7yz
> -----END PGP SIGNATURE-----
>
> ------------------------------------------------------------------------------
> Slashdot TV.
> Video for Nerds. Stuff that matters.
> http://tv.slashdot.org/
> _______________________________________________
> Iverilog-devel mailing list
> Ive...@li...
> https://lists.sourceforge.net/lists/listinfo/iverilog-devel
>
|
Thanks for helping keep SourceForge clean.
X