myhdl-list

[myhdl-list] VHDL Cosim Initial Thoughts

[Daryl W. <dw...@ou...>]

Okay, I have read through _Cosimulation.py, _Simulation.py, and myhdl_vpi.c, and 
Jan's description here: http://www.myhdl.org/doc/0.7/manual/cosimulation.html 

I think I have a grasp on what the Verilog simulator is doing during 
cosimulation: Basically, run the simulation, schedule a callback that occurs 
when all signals are in steady-state, make successive callbacks to simulate 
Delta cycles from VHDL, and use delay callbacks to make sure the simulation 
times match up. At least if I'm interpreting everything correctly...

I don't think this exact method will work with the ModelSim Foreign Language 
Interface (FLI) since it appears to me that some of these call backs are 
missing. However, I do see an easier/compatible option I think... 

My thoughts: The FLI allows one to register a VHDL process, create signals, 
assign a sensitivity list to the process, and read/drive signals from C. My plan 
(barring any problems in my interpretation so far) is to programmatically create 
a "MyHDL" process in the simulator with all of the from_myhdl signals in its 
sensitivity list, and read all of the output to_myhdl signals whenever this 
process is triggered and send them back to the Cosimulation object. These may be 
defined in the VHDL interface file somehow or else creating using FLI... I'm 
still working out details there.

Also, I haven't yet figured out how to deal with delays from the MyHDL testbench 
(which will be important for sure)... But I'm sure I will find a way soon.

In addition, I am opting to subclass Cosimulation as CosimulationWithSockets. 
This would override the methods of Cosimulation and modify them to implement 
their interprocess communication through network sockets instead of the pipes 
that are used now. This would allow the interprocess communication to occur over 
a network (letting me work on my laptop while running the simulation on a server 
for example). Also, it adds local cosimulation functionality to windows (via 
localhost) without Cygwin (with some performance loss maybe). It should work 
equally well (with appropriate preprocessing directives) under Windows, 
Unix/Linux, or Mac OS X. I've done some preliminary testing connecting python 
code to C code using sockets. So it looks like this is feasible. 

Any comments/questions/ideas are welcome. Thoughts and feedback on pros/cons of 
this approach would be greatly appreciated. Also, would this be useful for 
anyone else? If so, I'll post code once I have it working. 

Once I have something that I believe is working, I'll test it by running the 
tests in the cosimulation directory (appropriately modified and using VHDL).

Thanks for any feedback,
Daryl

Re: [myhdl-list] VHDL Cosim Initial Thoughts

[Christopher F. <chr...@gm...>]

On 1/4/13 2:55 PM, Daryl Wasden wrote:
> Okay, I have read through _Cosimulation.py, _Simulation.py, and myhdl_vpi.c, and
> Jan's description here: http://www.myhdl.org/doc/0.7/manual/cosimulation.html
>
> I think I have a grasp on what the Verilog simulator is doing during
> cosimulation: Basically, run the simulation, schedule a callback that occurs
> when all signals are in steady-state, make successive callbacks to simulate
> Delta cycles from VHDL, and use delay callbacks to make sure the simulation
> times match up. At least if I'm interpreting everything correctly...

Sounds like the gist of it, I will have to review as a
refresher and see if any main points are not being
considered.

>
> I don't think this exact method will work with the ModelSim Foreign Language
> Interface (FLI) since it appears to me that some of these call backs are
> missing. However, I do see an easier/compatible option I think...
>
> My thoughts: The FLI allows one to register a VHDL process, create signals,
> assign a sensitivity list to the process, and read/drive signals from C. My plan
> (barring any problems in my interpretation so far) is to programmatically create
> a "MyHDL" process in the simulator with all of the from_myhdl signals in its
> sensitivity list, and read all of the output to_myhdl signals whenever this
> process is triggered and send them back to the Cosimulation object. These may be
> defined in the VHDL interface file somehow or else creating using FLI... I'm
> still working out details there.

This sounds reasonable, instead of the $to_myhdl and
$from_myhdl task (stubs to the VPI code) you can have
equivalent VHDL processes.

I believe one of the items the FLI interface will need
to support is getting a list of the signals from the
VHDL simulator.  Resolving the signal interface from
the MyHDL side to the VHDL simulator side, maybe a
little tricky.  It might be simplified with the automatic
generation of the VHDL processes?

>
> Also, I haven't yet figured out how to deal with delays from the MyHDL testbench
> (which will be important for sure)... But I'm sure I will find a way soon.

I am not sure what you are asking or not understanding
here?  The standard way to "delay" in a testbench is

       def testbench()
          ...
          @instance
          def tb_stimulus():
              yield delay(10)

The above will delay for 10 simulation cycles.
>
> In addition, I am opting to subclass Cosimulation as CosimulationWithSockets.
> This would override the methods of Cosimulation and modify them to implement
> their interprocess communication through network sockets instead of the pipes
> that are used now. This would allow the interprocess communication to occur over
> a network (letting me work on my laptop while running the simulation on a server
> for example). Also, it adds local cosimulation functionality to windows (via
> localhost) without Cygwin (with some performance loss maybe). It should work
> equally well (with appropriate preprocessing directives) under Windows,
> Unix/Linux, or Mac OS X. I've done some preliminary testing connecting python
> code to C code using sockets. So it looks like this is feasible.

I think this sounds like a good idea and as you state
might be a better option for other platforms (i.e. windows).
I don't know enough about pipes or sockets to know if there
are any issues.  My guess, is there should not be issues,
but possibly performance?

>
> Any comments/questions/ideas are welcome. Thoughts and feedback on pros/cons of
> this approach would be greatly appreciated. Also, would this be useful for
> anyone else? If so, I'll post code once I have it working.

The small consensus seem to support Modelsim as the most
popular simulator for VHDL (obviously it would be nice to
have an open-source option but it doesn't look like that
is possible).  I don't recall anyone else stating another
simulator (Aldec, ncsim, , etc)

Regards,
Chris

>
> Once I have something that I believe is working, I'll test it by running the
> tests in the cosimulation directory (appropriately modified and using VHDL).
>
> Thanks for any feedback,
> Daryl
>
>
>
> ------------------------------------------------------------------------------
> Master HTML5, CSS3, ASP.NET, MVC, AJAX, Knockout.js, Web API and
> much more. Get web development skills now with LearnDevNow -
> 350+ hours of step-by-step video tutorials by Microsoft MVPs and experts.
> SALE $99.99 this month only -- learn more at:
> http://p.sf.net/sfu/learnmore_122812
>

Re: [myhdl-list] VHDL Cosim Initial Thoughts

[Daryl W. <dw...@ou...>]

Christopher Felton <chris.felton <at> gmail.com> writes:

> 
> On 1/4/13 2:55 PM, Daryl Wasden wrote:
<snip> 
> This sounds reasonable, instead of the $to_myhdl and
> $from_myhdl task (stubs to the VPI code) you can have
> equivalent VHDL processes.
> 
> I believe one of the items the FLI interface will need
> to support is getting a list of the signals from the
> VHDL simulator.  Resolving the signal interface from
> the MyHDL side to the VHDL simulator side, maybe a
> little tricky.  It might be simplified with the automatic
> generation of the VHDL processes?

It would be possible and easy to use a VHDL process to 
accomplish this with a wait statement terminating it, 
but I feel that may involve more work for the end user... 
However, I have settled on a different route that reads 
the port list of the module and if a signal is "in" it 
is used to populate the to_myhdl signals and if it is 
"out" it is used to populate the from_myhdl signals. This 
all occurs during initialization (right after elaboration 
has finished in ModelSim). I like this solution so far, 
but it will always be easy to go back to the approach that 
resembles the VPI Cosimulation approach more closely if 
desired (though it should appear the same from Python's
point-of-view). 

My view of how the VHDL Cosimulation is (only) slightly 
different from the Verilog Cosimulation. 

There will be a MyHDL top-level object containing the
testbench code as well as instantiations of any MyHDL
modules that will be used in the test. The simulator is
invoked with the Cosimulation() except that the arguments
that define the ports (by passing them to $to_myhdl and
$from_myhdl) are not required. These are instead defined
in the port list of a VHDL module that I am currently 
calling MyHDL_bridge and it is found in MyHDL_bridge.vhd.
This module is defined with the "foreign" attribute in 
its architecture and as such calls the C code that sets 
up the VHDL-side. There is also a top_level.vhd file 
that instantiates the MyHDL_bridge module as well as 
any VHDL modules that are used in the cosimulation and 
connects the ports from the MyHDL bridge to the 
instantiated modules. That's the track that I am 
currently pursuing...

> > Also, I haven't yet figured out how to deal with delays...
> I am not sure what you are asking or not understanding
> here? 

Sorry, I wasn't clear before. I meant to say I'm not sure 
how the C-side of the cosimulation knows that it needs to 
look for a delay. It looks like it is a well-defined 
ordering (it reads it in the read_only_callback shortly 
after sending the current HDL-state of the the signals to 
the MyHDL-side). Maybe this is due to the fact that delays 
are only allowed on the MyHDL-side, so a future event must 
be scheduled after all signals have settled? That's just a 
guess. If so, then I have found the way in FLI using a 
prioritized process (it should occur at exactly the same 
time as the read only callback does). Only one 
process/function is required in C, so I have to test to 
see if I am on a delta cycle iteration (at the same time 
step) or the first iteration at the current simulation 
time. Basically, all three of the call backs from the VPI 
method are collapsed into a single function in the FLI 
method. I haven't decided if this is going to be a 
problem yet... But I don't think so.

Anyway, like I said, I'm fairly confident I can resolve
this even if I'm wrong at the moment on exactly why it
works in the current implementation, but if it doesn't
work for the reasons I stated, it may take me some time
to figure out why and resolve it. But I will...

> > I am opting to subclass Cosimulation as CosimulationWithSockets.
> I think this sounds like a good idea and as you state
> might be a better option for other platforms (i.e. windows).
> I don't know enough about pipes or sockets to know if there
> are any issues.  My guess, is there should not be issues,
> but possibly performance?

I'm trying to have some preliminary code executing a 
cosimulation with Icarus Verilog and the old tests in 
the cosimulation directory using the socket-based 
communication in the next day or so, so I'll update 
this thread then with my findings. Hopefully, it will 
be a positive experience.
 
> The small consensus seem to support Modelsim as the most
> popular simulator for VHDL (obviously it would be nice to
> have an open-source option but it doesn't look like that
> is possible).  I don't recall anyone else stating another
> simulator (Aldec, ncsim, , etc)

There may be a way to do it using GHDL actually. GHDL does
support the VHPI for subprograms... I think with some 
creativity it might be possible to adopt an approach 
that would work... But it wouldn't be straightforward,
that's why I'm trying to get the FLI working. I want to 
start using MyHDL with VHDL as soon as I can...

Thanks for the responses,
Daryl

Re: [myhdl-list] VHDL Cosim Initial Thoughts

[Daryl W. <dw...@ou...>]

I have a suggestion for a change to the file _Simulation.py 
that will help my socket implementation succeed. Right now, 
the _finalize method of the Simulation class contains two 
lines (76 and 77) that explicitly close the cosim._rt and 
cosim._wf pipes... Also line 78 waits for the child process. 
I think it would be better to call a cosim._finalize method 
that performs these operations on its own member data 
(preserving encapsulation of the Cosimulation object). 

I can do a temporary modification for myself, but I'd like 
to maintain as much compatibility with MyHDL as possible so 
I can benefit from future updates from the MyHDL community.
>From the user viewpoint, the behavior would be identical. 
At some point, I may want to propose a MEP with socket 
support added to Cosimulation (either as a subclass or 
built into the main class) in the main repository if people 
think this implementation will be useful to them, but I 
want a proof-of-concept first... Just to ensure it is 
feasible.

Any thoughts/feedback? Is posting here the right place for 
this type of suggestion? 

Thanks,
Daryl

Re: [myhdl-list] VHDL Cosim Initial Thoughts

[Daryl W. <dw...@ou...>]

> > > Also, I haven't yet figured out how to deal with delays...
> > I am not sure what you are asking or not understanding
> > here? 
I see how this works now. I didn't completely understand the 
format. Now, I see the protocol structure. The time is sent 
first, then any signal changes. Doing the socket-based 
cosimulation in verilog was helpful. 

I am still fixing one bug, but the simulation is running 
correctly using socket communication on windows. I ran
the test_bin2gray using Icarus. The bug is related to 
reusing the same port when moving between tests... If I 
get this up and running and fix the bug, then I'll work 
on porting it over to Linux/Mac OSX as well.

-Daryl 

Re: [myhdl-list] VHDL Cosim Initial Thoughts

[Christopher F. <chr...@gm...>]

On 1/8/13 6:30 PM, Daryl Wasden wrote:
>>>> Also, I haven't yet figured out how to deal with delays...
>>> I am not sure what you are asking or not understanding
>>> here?
> I see how this works now. I didn't completely understand the
> format. Now, I see the protocol structure. The time is sent
> first, then any signal changes. Doing the socket-based
> cosimulation in verilog was helpful.
>
> I am still fixing one bug, but the simulation is running
> correctly using socket communication on windows. I ran
> the test_bin2gray using Icarus. The bug is related to
> reusing the same port when moving between tests... If I
> get this up and running and fix the bug, then I'll work
> on porting it over to Linux/Mac OSX as well.
>
> -Daryl
>
>

Sounds like you are making good progress.  If you want, we
can setup a bitbucket repository and I can test out your
changes as well.  I don't have a windows system right now
but I can test it out on a linux.

Regards,
Chris

Re: [myhdl-list] VHDL Cosim Initial Thoughts

[Daryl W. <dw...@ou...>]

Christopher Felton <chris.felton <at> gmail.com> writes:

> 
> On 1/8/13 6:30 PM, Daryl Wasden wrote:
> >>>> Also, I haven't yet figured out how to deal with delays...
> >>> I am not sure what you are asking or not understanding
> >>> here?
> > I see how this works now. I didn't completely understand the
> > format. Now, I see the protocol structure. The time is sent
> > first, then any signal changes. Doing the socket-based
> > cosimulation in verilog was helpful.
> >
> > I am still fixing one bug, but the simulation is running
> > correctly using socket communication on windows. I ran
> > the test_bin2gray using Icarus. The bug is related to
> > reusing the same port when moving between tests... If I
> > get this up and running and fix the bug, then I'll work
> > on porting it over to Linux/Mac OSX as well.
> >
> > -Daryl
> >
> >
> 
> Sounds like you are making good progress.  If you want, we
> can setup a bitbucket repository and I can test out your
> changes as well.  I don't have a windows system right now
> but I can test it out on a linux.
> 
> Regards,
> Chris
> 
> ------------------------------------------------------------------------------
> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> MVPs and experts. SALE $99.99 this month only -- learn more at:
> http://p.sf.net/sfu/learnmore_122912
> 

Sure, that would be great. My free time is going away quickly, 
but I should be able to put in at least a few hours every week. 
With where I am now, that should still get things done
relatively fast. Currently, the simulation runs okay, but 
if you try to execute two simulations quickly, there can be
a problem because ports don't register as closed immediately
after you close them. Also, it only works on localhost
right now because of how it is designed... This wouldn't be
too difficult to change, but it would require the user to
start the process on the server manually prior to starting
the MyHDL simulation which deviates from the current 
implementation, but may be useful for me later. So, I'll 
look into that as well at some point. 

I've never used bit bucket before, but I will look into it. 
Should I email you personally when I've got everything up or 
just keep posting to the mailing list? Also, should I put 
the 0.8dev code up there and contribute to it or just 
include the subclassed code and the (modified) test code?

The one problem is that this method (or any method other than 
the one provided in the present version) requires a change 
to _Simulation.py as well because the Cosimulation pipes are 
closed by the Simulation class methods manually and not by a 
call to the Cosimulation class methods. This would be easily 
changed, but I'm not sure how to suggest such a patch. I 
looked on the website and it says to send ideas to the mailing 
list for discussion. I put it in an earlier reply on this 
thread, but maybe I should make a new thread and put it there? 

I'm sorry for all my questions, I'm trying to figure out the 
proper etiquette for contributing to the project still. The 
modification would not change any existing behavior, it would 
just make it possible to subclass the Cosimulation object and 
use other methods for inter-process communication. At some 
point, I think I'll look into putting windows pipes. 

Also, the class CosimulationWithSockets is based heavily on 
the original Cosimulation class with the only changes being 
initialization, communication, and clean up of the sockets 
(instead of the pipes) and the use of the multiprocessing
module to generate a child process instead of os.fork().
I should include the original license at the top of the
file in this case, correct? 

Thanks again, I really appreciate you helping me to 
understand these things. I know that you are busy, too... 

-Daryl

Re: [myhdl-list] VHDL Cosim Initial Thoughts

[Christopher F. <chr...@gm...>]

On 1/13/13 3:26 PM, Daryl Wasden wrote:
> Christopher Felton <chris.felton <at> gmail.com> writes:
>
>>
>> On 1/8/13 6:30 PM, Daryl Wasden wrote:
>>>>>> Also, I haven't yet figured out how to deal with delays...
>>>>> I am not sure what you are asking or not understanding
>>>>> here?
>>> I see how this works now. I didn't completely understand the
>>> format. Now, I see the protocol structure. The time is sent
>>> first, then any signal changes. Doing the socket-based
>>> cosimulation in verilog was helpful.
>>>
>>> I am still fixing one bug, but the simulation is running
>>> correctly using socket communication on windows. I ran
>>> the test_bin2gray using Icarus. The bug is related to
>>> reusing the same port when moving between tests... If I
>>> get this up and running and fix the bug, then I'll work
>>> on porting it over to Linux/Mac OSX as well.
>>>
>>> -Daryl
>>>
>>>
>>
>> Sounds like you are making good progress.  If you want, we
>> can setup a bitbucket repository and I can test out your
>> changes as well.  I don't have a windows system right now
>> but I can test it out on a linux.
>>
>> Regards,
>> Chris
>>
>>
>
> Sure, that would be great. My free time is going away quickly,
> but I should be able to put in at least a few hours every week.
> With where I am now, that should still get things done
> relatively fast. Currently, the simulation runs okay, but
> if you try to execute two simulations quickly, there can be
> a problem because ports don't register as closed immediately
> after you close them. Also, it only works on localhost
> right now because of how it is designed... This wouldn't be
> too difficult to change, but it would require the user to
> start the process on the server manually prior to starting
> the MyHDL simulation which deviates from the current
> implementation, but may be useful for me later. So, I'll
> look into that as well at some point.
>
> I've never used bit bucket before, but I will look into it.
> Should I email you personally when I've got everything up or
> just keep posting to the mailing list? Also, should I put
> the 0.8dev code up there and contribute to it or just
> include the subclassed code and the (modified) test code?

You can create a repository and push your changes or I
created a repository on bitbucket.  If you want to use
the clone I created:

https://bitbucket.org/cfelton/myhdl_vhdl_cosim/overview

The following should get you started.

    >> hg clone https://bitbucket.org/cfelton/myhdl_vhdl_cosim
       <local_path>
    # Add a <user>@bitbucket.org if you want to push later
    # after cloned the bitbucket clone of myhdl, push your changes
    >> cd <to your current working >
    >> hg commit # commit any modifications
    >> hg push <path to your local clone, first command above>

You will need to create a bitbucket user and then you can push,
bitbucket is essentially the same as github just mercurial
based and not git.

The conversation can continue on this mailing-list unless
anyone else objects.  The feedback from other users/devs
can be very useful.

>
> The one problem is that this method (or any method other than
> the one provided in the present version) requires a change
> to _Simulation.py as well because the Cosimulation pipes are
> closed by the Simulation class methods manually and not by a
> call to the Cosimulation class methods. This would be easily
> changed, but I'm not sure how to suggest such a patch. I
> looked on the website and it says to send ideas to the mailing
> list for discussion. I put it in an earlier reply on this
> thread, but maybe I should make a new thread and put it there?

Yes, a new thread with this specific issue.

>
> I'm sorry for all my questions, I'm trying to figure out the
> proper etiquette for contributing to the project still. The
> modification would not change any existing behavior, it would
> just make it possible to subclass the Cosimulation object and
> use other methods for inter-process communication. At some
> point, I think I'll look into putting windows pipes.

Questions are perfect!  Jan D. has outlined how to communicate
changes and commit bundles (patches).

http://www.myhdl.org/doku.php/dev:patches

>
> Also, the class CosimulationWithSockets is based heavily on
> the original Cosimulation class with the only changes being
> initialization, communication, and clean up of the sockets
> (instead of the pipes) and the use of the multiprocessing
> module to generate a child process instead of os.fork().
> I should include the original license at the top of the
> file in this case, correct?
>
> Thanks again, I really appreciate you helping me to
> understand these things. I know that you are busy, too...
>
> -Daryl
>

No problem, these two topics are asked once in a while,
VHDL cosimulation and windows cosimulation.  I think
their are others that might be interested as well but
probably don't have the cycles right now.

Regards,
Chris

Re: [myhdl-list] VHDL Cosim Initial Thoughts

[Daryl W. <dw...@ou...>]

Thanks again. I cloned the code to my computer. I will push 
the changes once I have the cross-platform C-code completed. 
It works more or less, with the one problem I mentioned. I 
have some ideas for fixing it, just need to find a little 
more time.... 

-Daryl