Re: [Iverilog-devel] Why use SFT files?

[Martin W. <ic...@ma...>]

I started by creating a separate utility for automatically generating SFT 
files, but it was so trivial that it seemed less work to just integrate it 
into the compiler proper.

I've done the merge. Let me know if there are any problems. I'll retest the 
Windows build now.

Stephen Williams wrote:
> Interesting idea from Niels, but I feel comfortable with Martin's
> description of things, so go ahead and merge it. I can run regression
> tests on my Mac to verify it doesn't break anything Mac.
> 
> On Mon, Nov 4, 2019 at 12:32 PM Martin Whitaker
> <ic...@ma...> wrote:
>>
>> I've verified on Linux and Windows. I can't verify on Mac OS - I don't own
>> any Apple hardware. But I can see no special handling of DLLs and VPI
>> modules for Mac OS in the make files, so I expect it to work. Note that the
>> new callback table for making things work in Windows is only included when
>> building for Windows - VPI modules are unchanged for Linux and Mac OS,
>> apart from a few bug fixes.
>>
>> I have left the code for reading SFT files in the compiler, so users'
>> existing test scripts won't break if they pass those files on the command line.
>>
>> I was not intending to backport this to v10.
>>
>> Stephen Williams wrote:
>>> I don't object.
>>>
>>> My only remaining concern is that running the compiler might be
>>> slightly more complicated when VPI modules are involved, but I believe
>>> the costs on that front can be minimized.
>>>
>>> Have you verified your work on Linux, Windows, and Mac OS?
>>>
>>> I would keep this out of the v10 branch.
>>>
>>> On Mon, Nov 4, 2019 at 12:32 AM Martin Whitaker
>>> <ic...@ma...> wrote:
>>>>
>>>> So, any objections to merging this?
>>>>
>>>> Martin Whitaker wrote:
>>>>> It's a user experience thing. The information is all available via the
>>>>> standard VPI API - why should we expect users to provide it again in a
>>>>> different format. No other Verilog compiler I know of requires that, and
>>>>> apart from the Windows ugliness, it's easy to fix.
>>>>>
>>>>> My current solution doesn't require the user to do anything special, other
>>>>> than to build against our vpi_user.h and libvpi.a. And they need to do that
>>>>> with the existing solution too.
>>>>>
>>>>> I will replace the macros though. The more I think about it, the more I
>>>>> dislike it.
>>>>>
>>>>> Stephen Williams wrote:
>>>>>> Ugh! I forgot about that.
>>>>>>
>>>>>> Yeah, this is getting increasingly awkward. Might it be better compile
>>>>>> into the VPI a data structure that the environment can extract without
>>>>>> callbacks? In a sense we already have that in the form of the
>>>>>> s_vpi_systf_data struct. Pile all of that into a section in the .vpi
>>>>>> and the loader binary may be able to get at it without callbacks. That
>>>>>> would require a coding standard for the vpi modules to make those
>>>>>> tables static const tables.
>>>>>>
>>>>>> Is that any better then having .sft files?
>>>>>>
>>>>>> What's the problem we're trying to solve, again?
>>>>>>
>>>>>> On Sat, Oct 26, 2019 at 12:29 PM Martin Whitaker
>>>>>> <ic...@ma...> wrote:
>>>>>>>
>>>>>>> I'm already reusing the vvp/ivl_dlfcn.h header. Dynamically loading the
>>>>>>> modules is not the problem; allowing the modules to call back to the VPI
>>>>>>> routines implemented in the main program (e.g. vpi_register_systf) is.
>>>>>>>
>>>>>>> In Linux (and I guess in MacOS), those routines are left as undefined
>>>>>>> references in the VPI module:
>>>>>>>
>>>>>>> % objdump -t v2005_math.vpi | grep vpi_register_systf
>>>>>>> 0000000000000000         *UND*  0000000000000000
>>>>>>> vpi_register_systf
>>>>>>>
>>>>>>> When the module is dynamically loaded, these references get resolved by
>>>>>>> name, using whatever is provided by the loading program (or other libraries
>>>>>>> it has dynamically loaded). So any program can load and use the module,
>>>>>>> providing it implements the necessary functions.
>>>>>>>
>>>>>>> In Windows, the VPI modules are linked to vvp/libvpi.a, which is created by
>>>>>>> dlltool from vvp.exe. This contains the relocation information for all the
>>>>>>> symbols exported by vvp, and that information gets hardwired into the VPI
>>>>>>> modules. So if a VPI module is dynamically loaded by a different program,
>>>>>>> the addresses are all wrong, calls to the VPI routines jump to some random
>>>>>>> place in the loading program, and chaos ensues.
>>>>>>>
>>>>>>> I searched for a way to make Windows resolve references at load time, but
>>>>>>> couldn't find anything, hence I did it manually.
>>>>>>>
>>>>>>> If you don't like the use of macros to translate the calls, we could
>>>>>>> provide wrapper functions in the new libvpi. It would be a little less
>>>>>>> efficient, but most likely insignificant in the overall cost of making VPI
>>>>>>> calls.
>>>>>>>
>>>>>>> Stephen Williams wrote:
>>>>>>>> VPI modules can be dynamically loaded by vvp in Windows and Mac, so
>>>>>>>> shouldn't that same functionality work with ivl on Windows? The
>>>>>>>> vvp/ivl_dlfcn.h header file encapsulates this. Perhaps that header can
>>>>>>>> be moved to libmisc.
>>>>>>>>
>>>>>>>> On Wed, Oct 23, 2019 at 8:18 AM Martin Whitaker
>>>>>>>> <ic...@ma...> wrote:
>>>>>>>>>
>>>>>>>>> This was triggered by my writing a regression test for the recently
>>>>>>>>> reported bug, and finding that vpi_reg.pl didn't really support tests
>>>>>>>>> with
>>>>>>>>> SFT files...
>>>>>>>>>
>>>>>>>>> I've implemented the compiler change to read VPI modules directly in the
>>>>>>>>> sft-rework branch. Apart from exposing another issue in the vlog95
>>>>>>>>> target,
>>>>>>>>> all the existing tests still pass.
>>>>>>>>>
>>>>>>>>> Implementing this for Linux was pretty easy. Fixing the problems in
>>>>>>>>> Windows
>>>>>>>>> took a bit longer. As far as I can tell, there's no way for a DLL to
>>>>>>>>> automatically link to functions provided by the client application. The
>>>>>>>>> existing approach of linking the VPI module to vvp meant it couldn't then
>>>>>>>>> be used by the compiler. I solved that by passing a jump table down to
>>>>>>>>> the
>>>>>>>>> VPI module when it was loaded.
>>>>>>>>>
>>>>>>>>> See what you think. There's a sft_rework branch in the test suite which
>>>>>>>>> makes vpi_reg.pl use the new functionality.
>>>>>>>>>
>>>>>>>>> Stephen Williams wrote:
>>>>>>>>>> Because the .sft has some specificity that the .vpi does not have?
>>>>>>>>>> Also, to read the .vpi file, you need to dynamically link it in so
>>>>>>>>>> that you can read the tables. The ivl program currently does not need
>>>>>>>>>> to load .vpi files, it can use the .sft files instead.
>>>>>>>>>>
>>>>>>>>>> That may be obsolete thinking, though...
>>>>>>>>>>
>>>>>>>>>> On Sun, Oct 20, 2019 at 12:19 PM Martin Whitaker
>>>>>>>>>> <ic...@ma...> wrote:
>>>>>>>>>>>
>>>>>>>>>>> It is easy enough to extract the information from the .vpi file. I've
>>>>>>>>>>> written a small standalone utility that does just that and writes out a
>>>>>>>>>>> .sft file. But why not get the compiler to do it on the fly, rather
>>>>>>>>>>> than go
>>>>>>>>>>> though an intermediate step?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Iverilog-devel mailing list
>>>>>>>>>>> Ive...@li...
>>>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/iverilog-devel
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Iverilog-devel mailing list
>>>>>>>>> Ive...@li...
>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/iverilog-devel
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Iverilog-devel mailing list
>>>>>>> Ive...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/iverilog-devel
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Iverilog-devel mailing list
>>>>> Ive...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/iverilog-devel
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Iverilog-devel mailing list
>>>> Ive...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/iverilog-devel
>>>
>>>
>>>
>>
>>
>>
>> _______________________________________________
>> Iverilog-devel mailing list
>> Ive...@li...
>> https://lists.sourceforge.net/lists/listinfo/iverilog-devel
> 
> 
>