Menu
▾
▴
#87 Problems with parallel invocation of GHDL.
One of the nice things about GHDL is that it does not require a license and thus one can always utilize the number of processor cores to the max. However when trying to run multiple simulations in parallel with GHDL --elab-run using the same top level but with different generic values I sometimes run into these kinds of errors:
Example 1.:
ghdl: cannot open e~tb_generated-a.lst
Example 2.:
/home/kraigher/repo/vunit/vunit_out/ghdl/libraries/lib/e~tb_generated-a.o: file not recognized: File truncated
collect2: error: ld returned 1 exit status
ghdl: compilation error
Example 3.:
/home/kraigher/repo/vunit/vunit_out/ghdl/libraries/lib/e~tb_generated-a.o: In function `lib__tb_generated__LASTARCH__ELAB':
ortho:(.text+0x12): undefined reference to `lib__tb_generated__ARCH__a__ELAB'
/home/kraigher/repo/vunit/vunit_out/ghdl/libraries/lib/e~tb_generated-a.o: In function `__ghdl_ELABORATE':
ortho:(.text+0x51): undefined reference to `lib__tb_generated__ARCH__a__RTI'
ortho:(.text+0x68): undefined reference to `lib__tb_generated__PKG_ELAB'
ortho:(.text+0x7d): undefined reference to `vunit_lib__run_types_pkg__runner_cfg_default'
ortho:(.text+0xa8): undefined reference to `vunit_lib__run_types_pkg__runner_cfg_default'
ortho:(.text+0xc8): undefined reference to `lib__tb_generated__ARCH__a__ELAB'
collect2: error: ld returned 1 exit status
ghdl: compilation error
I seems GHDL creates e~tb_generated-a.o and e~tb_generated-a.lst files which cause problems when multiple parallel instances of GHDL are working on the same top level.
I am already making sure the output executable has an unique location for each invocation but it would be nice if one could also control the name of the e~ files by for instance being able to add an unique suffix for each invocation. Maybe a --unique-id flag that will cause the id to get suffixed to temporary files. What do you think about this proposal, is there a better way to solve this problem?
The easiest way to solve this issue is to elaborate only once (using -e), and then run in parallel (using -r).
I think that using a different directory could fix the issue.
In any case, it shouldn't be difficult to change ghdl so that the e~ name is based on the output file.
Tristan.
Well I could always copy all libraries into separate sandbox directories for each thread. Elaborating once does not work for me since each invocation requires different generic values. If I understand correctly the
-rflag does not support-g?No, no. The -g flag is handled only when the design is simulated, not at elaboration.
(Although that could change in the future).
I don't think you need to copy libraries. Just use --elab-run in a different directory, but add a -P switch to reference the work library.
The
.ofile is still created in the library folder and not the current working directory so just changing working directory will not be enough. Were you thinking about the.lstfile?Ok, another way to fix that issue.
Did you try to only run in parallel ghdl -r ?
Yes that is the workaround I just implemented. I separated the commands into
-eand-rand then I make a threading lock around the-ecommand. I am currently optimizing the lock such that elaborations with different top level names can still execute in parallel. However for a tool like VUnit that encourages short tests and splitting the same top level test bench into multiple independent tests having a lock on elaboration for short tests takes away a lot of the opportunities for parallel execution.The output name (the one given by -o) is now used for the elaboration files.
So, you should be able to do parallel elaboration (-e or --elab-run) as long as you use different output names.
I think that should fix your issue. Please confirm!
Tristan.
A problem with splitting
-eand-rinto separate commands is that the-oargument of-ethat sets the executable to write which cannot be fed back into-r. That is why I prefer--elab-run. It would therefore be really nice if all temporary files were created with names based on the argument of-oflag of--elab-runas you suggested in your first comment.Great! Will try it right away. Did not see your latest reply before commenting.
Sorry to bother you again. It is better now but... the files still end up in the library folder, I assumed they would be located side-by-side with the location of the
-ofile with the same base name. When I said that the argument of the-ofile was unique I did not mean that it was unique by just looking at the base name. The path to it can be unique but not the base name making the whole path unique. Basically I create a separate output folder for each test run and the output file is named based on the top level name which is not always unique but the test folder name is.I could still workaround this easily though by generating unique base names but the solution I had in mind would be nicer.
Ah, ah. No problem, I will rework it.
I suppose in case of full output pathname (ticket 51), you want to use it.
Btw the
.ofile still cause conflict:Yes it would be possible to run in parallel with
-oif all artifacts were written to the same directory with the same prefix as the file specified by-o. Regardless if the argument to-ois relative or absolute. Also omitting-owould most logically be equivalent with-o .which would cause everything to be written to the current working directory. The flag becomes illogical the behavior is different when it is omitted, relative or absolute... if that was what you suggested or what do you mean with "in case of full output pathname"?All artefacts are now written to the same directory as the output file.
Does this fix your issue ? (again :-)
I tried it but it seems now the
.ofile gets written to the current working directory rather than the same directory as the file name specified with the-oflag.I hope this is now correct!
Yes now I works, I can run in parallel without race conditions. Thank you!
Great! So I can finally close this ticket
Tristan.