#17 systemverilog support for ctags

open
nobody
None
5
2015-01-21
2010-05-13
No

Hi all,

I worked on a personal project last fall to develop a Systemverilog parser for ctags-5.8.

It supports most language features (based on the BNF from the IEEE 1800 spec draft). I\'ve been using it for actual development for the last few months and I can safely venture to call it beta quality.

I\'m not quite sure about the submission procedure and whether I need some sort of developer access to be able to checkin my changes? Could one of the existing developers perhaps point me the way?

As far as I can tell, no one else has been working on SV support -- I\'m willing to become a maintainer of this language support if warranted.
.
Thanks,
Vikram Khosa

Related

Support Requests: #17

Discussion

  • Steve

    Steve - 2012-10-29

    I have also been working on support for System Verilog, but have not made much progress. (Too much writing Verilog, not enough work on supporting it.) I'd be happy to help you test your patch, if you want.

     
  • Vikram Khosa

    Vikram Khosa - 2012-11-12

    thanks steve! I'm attaching a gzipped archive of the entire ctags-5.8 directory with the systemverilog support files inside as well as the modified taglist.vim. let me know how it goes and please feel free to flag issues as you find them. good luck!

    some release notes:

    A. Unzip the archive and cd to the ctags-5.8 directory it creates.
    • ./configure –prefix=<LOCAL_CTAGS_PATH>, where <LOCAL_CTAGS_PATH> is a user-defined location for installation .e.g your home (by default, “make install” will try to install in /usr/local or /usr/local/man)
    • make
    • make install

    B. Copy the taglist.vim provided to your vim-plugin directory. Currently all tags are enabled (including procedural control statements, begin..end blocks). If you’d like to filter the types of tags generated, you can edit the SV language settings at line 486.

    C. Add the following settings to your .vimrc (the first points to your user ctags dir; some of the rest are optional; the last adds keyboard shortcut F8 for toggling the taglist window for any vim buffer):
    let Tlist_Ctags_Cmd='<LOCAL_PATH>/ctags'
    let Tlist_Auto_Open = 1
    let Tlist_Auto_Update = 1
    let Tlist_Show_Menu = 1
    let Tlist_Process_File_Always = 1
    let Tlist_Exit_OnlyWindow = 1
    let Tlist_Show_One_File = 1
    let Tlist_Enable_Fold_Column = 0
    let Tlist_File_Fold_Auto_Close = 1
    let Tlist_WinWidth = 44
    nnoremap <silent> <F8> :TlistToggle<CR>

    Note: I currently attempt to tag all (including user-defined) type-definitions and declarations for variables of built-in types but declarations of user-defined types are not supported yet (this would probably require creating a separate symbol table). Support for labels is a bit tenuous as well.

    A few potentially useful todos that I can think of: jumping tags across files/directories, integration with emacs etc.

    If you do decide to try this out, let me know if you have trouble installing. Also feel free to report bugs / suggest improvements / code-review source (systemverilog.c file in the ctags-5.8 directory). If you choose to debug on your own for whatever reason: <LOCAL_CTAGS_PATH>/ctags-5.8/bin/ctags –v <SV filename> will generate a verbose output and the “tags” file if successful.

     
  • Vikram Khosa

    Vikram Khosa - 2012-11-12

    taglist.vim with SV support

     
  • Vikram Khosa

    Vikram Khosa - 2012-11-12

    hi steve,

    instead of uploading an archive of the entire modified source, I've instead attached a gzip archive of the set of files that need to be patched over (to the ctags-5.8 directory). systemverilog_diff.tar.gz. Also attached is the modified taglist.vim. let me know if i missed something. again thanks for your help.

    vikram

     
  • Johannes Schaefer

    Hi Vikram,

    I am also interested in better system verilog support for ctags (using vim). I tried modifying the existing verilog parser and adding some regexp statements for the new system verilog parts, but this does not work for all constructs.
    Now that I saw there is a more sophisticated solution, I tried it out.
    Unfortunately I get a segmentation fault when parsing the following line:
    initial begin clk = 0; #(Tclk_ns/2) clk = 1; end

    Well, this is against our coding guidelines, but still valid verilog code. I am no software expert so its not easy to track down that in your code, but I hope the info that there are more people outside interested in a system verilog parser might be a motivation for you and the ctags team to fix and include this parser into the main distribution.

    Johannes

     
  • Steve

    Steve - 2012-11-13

    Vikram,

    I have downloaded your patch and am looking it over.

    The first thing I notice is that you appear to have built against the 5.8 release source, rather than the trunk. There are changes in both parsers.h and source.mak to support Object-C that you do not have in your version. I have merged the changes and will be working from a patched trunk rather than 5.8.

     
  • Vikram Khosa

    Vikram Khosa - 2012-11-14

    Johannes, thank you for your encouragement. I will debug the segfault with the input you provided.

    Steve, yes the patch was developed with 5.8 (almost 3 years ago). so thanks for merging it with latest trunk source. I think it would help to create a SV dev branch off trunk to share code, debug issues and test out various fixes.

     
  • Steve

    Steve - 2012-11-20

    Sorry I've been silent. I finally got it running against both a simple package file and a source file that imports the package.

    It properly tags the typedef struct {} and typedef enum {} entries in the package, but does not follow down into them to tag the members and enumeration values.

    In the source file, it creates tags for if, else, always_comb, etc., which is quite useless, while failing to recognize and tag instances of the user defined typedefs.

    All three of these defects seem like killers to me. Are you expecting any specific command line flags?

     
  • Steve

    Steve - 2012-11-20

    Another data point. Here is the start of the port list for one module in my design.

    module upa #( parameter IS_FPGA = 0,
    ...
    parameter RESPONSE_LOG2_DEPTH = 0 ) (
    input logic clock,
    input logic reset_n,
    input logic HCLK,
    input logic HRESETn,
    ahb_lite_if ahb_bus,
    output logic [UPDATE_FIFO_WIDTH-1:0] upa_update_fifo,
    output logic upa_upd_not_empty,
    output logic upa_update_fifo_full,
    input logic upa_update_fifo_pop,
    input logic [RESPONSE_FIFO_WIDTH-1:0] upa_response_fifo,
    output logic upa_response_fifo_empty,
    ...
    output logic upa_final

    );

    Most of the single inputs and output are correctly identified and tagged. None of the busses are, they show up a objects named "logic". :-P

    The clock input doesn't tag properly, instead ctags seems to think that the parameter RESPONSE_LOG2_DEPTH extends through that line. Similarly, the upa_final signal is tagged, but it is tagged as being on the line with the closing parenthesis rather than on the correct line.

    Interesting lines from the tags file:
    RESPONSE_LOG2_DEPTH upa.sv /^ input logic clock,$/;" i undefined/global:upa
    logic upa.sv /^ output logic [UPDATE_FIFO_WIDTH-1:0] upa_update_fifo,$/;" C undefined/global:upa
    upa_final upa.sv /^);$/;" K undefined/global:upa

     
  • Steve

    Steve - 2012-11-20

    Sorry, word wrap messed that up pretty bad. There are three lines from the tags file shown. I'm trying again here, trying to prevent word wrap, and with a blank line between them (in case it wraps again.)

    RESPONSE_LOG2_DEPTH upa.sv /^ input logic clock,$/;" i undefined/global:upa

    logic upa.sv /^ output logic [UPDATE_FIFO_WIDTH-1:0] upa_update_fifo,$/;" C undefined/global:upa

    upa_final upa.sv /^);$/;" K undefined/global:upa

     
  • Vikram Khosa

    Vikram Khosa - 2012-12-02

    Johannes, I debugged your segfault -- the problem is that I was exiting the loop (early) on encountering a semicolon within a block scope. I think commenting out line 1112 : "loop = FALSE" inside the ";" case of processBlockAssocTag(...) in systemverilog.c should work as a simple fix for now (although the "clk" identifier will still not be tagged with the implicit "reg" or "logic" type). I will need to test this a bit more thoroughly and include it in a consolidated patch later.

     
  • Vikram Khosa

    Vikram Khosa - 2012-12-02

    Steve, thanks for testing it out. Yes, agree that tagging for typedefs & module parameters absolutely needs to be fixed. I will work on that and notify you once I have a patch ready.

    As for the useless block tags, I had included them for completeness sake. These can be easily filtered out by editing the following line in your taglist.vim (remove the mappings for the ones you don't care about).

    "systemverilog language
    let s:tlist_def_verilog_systemverilog_settings = 'verilog_systemverilog;a:access;b:always;c:assertion;' .
    \ 'd:case;e:clocking;f:chandle;g:checker;h:config;i:constant;' .
    \ 'j:constraint;l:covergroup;m:coverpoint;n:coverbin;o:class;' .
    \ 'p:enum;q:event;r:final;s:fork;t:function;u:generate;v:imexport;' .
    \ 'w:implementation;x:include;y:inheritance;z:initial;A:interface;' .
    \ 'B:int_atomic;C:int_vector;E:lifetime;F:linkage;G:module;H:net;' .
    \ 'I:non-integral;J:package;K:port;L:primitive;M:program;N:property;' .
    \ 'O:qualifier;P:randsequence;Q:seqblk;R:sequence;S:specify;T:string;' .
    \ 'U:strunion;V:table;W:task;X:timespec;Y:typedef;Z:void'

     
  • Johannes Schaefer

    Hi Vikram,
    I tested your proposition by commenting out line 1112 and it solves the segfault with the clock generator.
    I wouldn't expect to tag the clk in this case which was defined some lines before. I guess, only implicit defined single bit wires are allowed and it would be no killer, if these don't get tagged.

    The problem is, that I now got further through our design database encountering another segmentation fault. This has to do with labeled fork-join blocks.

    Some other minor problems show up, when multiple signals are defined in the same statement.

    The following dummy module shows the problems I discoverd up to now:

    module test (read_data);

    input [7:0]read_data; // eats up first letter -> ead_data

    logic select, ready, busy; // only "busy" gets its entry in tag list

    wire signed [7:0] wave; // gets entered as "wire" not as "wave"
    // might be the same problem as already reported by steve

    // uncomment the label after fork and it will lead to a segfault
    initial
    begin
    fork//: labeled_block
    begin
    $display("Hello World!");
    end

    $display("Hello World!");

    begin
    $display("Hello World!");
    end
    join
    end

    always @(wave)
    a_no_undefined: assert (!$isunknown(wave) && // I would expect a tag here, but
    !$isunknown(busy)) // tag gets positioned to this line
    else
    $warning("Internal signal is undefined!");
    endmodule

    Anyways, it looks very promising and I hope you can fix the problems.
    I can test you patches on our design database, which seems to include enough different constructs to discover hidden problems.

    Johannes

     
  • justrajdeep

    justrajdeep - 2013-07-11

    Hi
    it will help a lot for young people like us who are just starting with both system verilog and vim to get ctags support.
    Really looking forward to this.

     
  • Suresh Mathew

    Suresh Mathew - 2013-12-03

    Hi Vikram, Was this code released into the main distribution?

     
  • Adam Krolnik

    Adam Krolnik - 2013-12-18

    Yes, would love to be able to get this together for 2014.

     
    • Suresh Mathew

      Suresh Mathew - 2013-12-18

      Hi adam, do you have a date for the release?
      On Dec 18, 2013 12:08 PM, "Adam Krolnik" adamkr@users.sf.net wrote:

      Yes, would love to be able to get this together for 2014.

      Status: open
      Created: Thu May 13, 2010 03:53 AM UTC by Vikram Khosa
      Last Updated: Tue Dec 03, 2013 02:19 PM UTC
      Owner: nobody

      Hi all,

      I worked on a personal project last fall to develop a Systemverilog parser
      for ctags-5.8.

      It supports most language features (based on the BNF from the IEEE 1800
      spec draft). I\'ve been using it for actual development for the last few
      months and I can safely venture to call it beta quality.

      I\'m not quite sure about the submission procedure and whether I need some
      sort of developer access to be able to checkin my changes? Could one of the
      existing developers perhaps point me the way?

      As far as I can tell, no one else has been working on SV support -- I\'m
      willing to become a maintainer of this language support if warranted.
      .
      Thanks,
      Vikram Khosa


      Sent from sourceforge.net because you indicated interest in
      https://sourceforge.net/p/ctags/support-requests/17/

      To unsubscribe from further messages, please visit
      https://sourceforge.net/auth/subscriptions/

       

      Related

      Support Requests: #17

  • Steve

    Steve - 2013-12-18

    Honestly, I found the support pretty weak, as noted in my prior comments. It really needs to be able to deal with tagging of user defined objects (like typedef struct) correctly, needs to lose all the non-sense tags for every begin and end, and properly treat them like the braces they are, etc. I have a version partially finished that is based on c.c that includes proper support for these things plus compiler directives, namespaces, etc. It certainly isn't ready for prime time yet, but I'd be more than happy to share the load if another developer or two wanted to help get it there.

     
    • Adam Krolnik

      Adam Krolnik - 2014-01-09

      Hi Steve;
      How about posting your current work for others to see ?

       
      • Steve

        Steve - 2014-01-09

        Hi Adam,

        Not being shy, it just isn't in a proper state (won't even compile) right now. If someone were interested in helping it get done, I'd be happy to send the files. I'll warn you, it has a lot left to do.

         
        • Steve

          Steve - 2014-02-28

          Sorry guys, I got sucked off on other priorities for a bit. I've got it back to compiling, but seem to have broken support for modules of all things! I was working on better support for classes and some of the more fancy things they can do, and ran into a problem telling the difference between a function prototype and an actual function declaration within the body of the class declaration. I probably broke modules in the process of messing with that part of the parser.

           
          • Suresh Mathew

            Suresh Mathew - 2014-05-02

            Guys - any update on this?

            On Fri, Feb 28, 2014 at 4:42 PM, Steve snarum-micron@users.sf.net wrote:

            Sorry guys, I got sucked off on other priorities for a bit. I've got it back
            to compiling, but seem to have broken support for modules of all things! I
            was working on better support for classes and some of the more fancy things
            they can do, and ran into a problem telling the difference between a
            function prototype and an actual function declaration within the body of the
            class declaration. I probably broke modules in the process of messing with
            that part of the parser.


            [support-requests:#17] systemverilog support for ctags

            Status: open
            Group:

            Created: Thu May 13, 2010 03:53 AM UTC by Vikram Khosa
            Last Updated: Fri Feb 28, 2014 12:00 AM UTC
            Owner: nobody

            Hi all,

            I worked on a personal project last fall to develop a Systemverilog parser
            for ctags-5.8.

            It supports most language features (based on the BNF from the IEEE 1800 spec
            draft). I\'ve been using it for actual development for the last few months
            and I can safely venture to call it beta quality.

            I\'m not quite sure about the submission procedure and whether I need some
            sort of developer access to be able to checkin my changes? Could one of the
            existing developers perhaps point me the way?

            As far as I can tell, no one else has been working on SV support -- I\'m
            willing to become a maintainer of this language support if warranted.
            .
            Thanks,
            Vikram Khosa


            Sent from sourceforge.net because you indicated interest in
            https://sourceforge.net/p/ctags/support-requests/17/

            To unsubscribe from further messages, please visit
            https://sourceforge.net/auth/subscriptions/

             

            Related

            Support Requests: #17

            • Suresh Mathew

              Suresh Mathew - 2014-05-02

              I am more than happy to validate!

              On Thu, May 1, 2014 at 9:44 PM, Suresh Mathew sureshamathew@gmail.com wrote:

              Guys - any update on this?

              On Fri, Feb 28, 2014 at 4:42 PM, Steve snarum-micron@users.sf.net wrote:

              Sorry guys, I got sucked off on other priorities for a bit. I've got it back
              to compiling, but seem to have broken support for modules of all things! I
              was working on better support for classes and some of the more fancy things
              they can do, and ran into a problem telling the difference between a
              function prototype and an actual function declaration within the body of the
              class declaration. I probably broke modules in the process of messing with
              that part of the parser.


              [support-requests:#17] systemverilog support for ctags

              Status: open
              Group:

              Created: Thu May 13, 2010 03:53 AM UTC by Vikram Khosa
              Last Updated: Fri Feb 28, 2014 12:00 AM UTC
              Owner: nobody

              Hi all,

              I worked on a personal project last fall to develop a Systemverilog parser
              for ctags-5.8.

              It supports most language features (based on the BNF from the IEEE 1800 spec
              draft). I\'ve been using it for actual development for the last few months
              and I can safely venture to call it beta quality.

              I\'m not quite sure about the submission procedure and whether I need some
              sort of developer access to be able to checkin my changes? Could one of the
              existing developers perhaps point me the way?

              As far as I can tell, no one else has been working on SV support -- I\'m
              willing to become a maintainer of this language support if warranted.
              .
              Thanks,
              Vikram Khosa


              Sent from sourceforge.net because you indicated interest in
              https://sourceforge.net/p/ctags/support-requests/17/

              To unsubscribe from further messages, please visit
              https://sourceforge.net/auth/subscriptions/

               

              Related

              Support Requests: #17

  • Adam Krolnik

    Adam Krolnik - 2013-12-18

    I would love to have a package from here that provided the contributors work.
    That way more things could be fixed faster.

    Does the parser for SV cover the whole language ? (Obviously some parts - block labels aren't handled well.)

     
    • Suresh Mathew

      Suresh Mathew - 2013-12-18

      Not to sound selfish, but I'd be more than happy to validate it! :)

      On Wed, Dec 18, 2013 at 3:36 PM, Adam Krolnik adamkr@users.sf.net wrote:

      I would love to have a package from here that provided the contributors
      work.
      That way more things could be fixed faster.

      Does the parser for SV cover the whole language ? (Obviously some parts -
      block labels aren't handled well.)


      Status: open
      Created: Thu May 13, 2010 03:53 AM UTC by Vikram Khosa
      Last Updated: Wed Dec 18, 2013 07:51 PM UTC
      Owner: nobody

      Hi all,

      I worked on a personal project last fall to develop a Systemverilog parser
      for ctags-5.8.

      It supports most language features (based on the BNF from the IEEE 1800
      spec draft). I\'ve been using it for actual development for the last few
      months and I can safely venture to call it beta quality.

      I\'m not quite sure about the submission procedure and whether I need some
      sort of developer access to be able to checkin my changes? Could one of the
      existing developers perhaps point me the way?

      As far as I can tell, no one else has been working on SV support -- I\'m
      willing to become a maintainer of this language support if warranted.
      .
      Thanks,
      Vikram Khosa


      Sent from sourceforge.net because you indicated interest in
      https://sourceforge.net/p/ctags/support-requests/17/

      To unsubscribe from further messages, please visit
      https://sourceforge.net/auth/subscriptions/

       

      Related

      Support Requests: #17

  • Vikram Khosa

    Vikram Khosa - 2014-02-28

    Hi guys,

    Unfortunately, my professional and other commitments have left me little time to contribute to this effort all this while.

    I would like to resume helping out with enhancements and bug-fixes. However I don't expect to have enough solo bandwidth to get this to production quality -- Steve, Adam or anyone else interested, any help with development and/or validation would be appreciated.

    The parser was originally meant to be comprehensive (i.e. cover all of the language). However, I do agree with Steve -- there's a lot left to do (especially with the SV standard having evolved since I started).

    Regards,
    Vikram

     
    Last edit: Vikram Khosa 2014-05-05
  • Vitor Antunes

    Vitor Antunes - 2014-06-03

    Hi everyone,

    I have started extending the Verilog parser a bit, but I've been working in a github fork [1] of fishman's repository.
    Masatake has kindly pointed out this thread, to which I feel I can contribute.
    Since I don't want to wait for a github/sourceforge flow to be defined, I decided to reply directly to be able to receive comments from the verilog parser maintainers.

    At the moment I've only done the following:

    1. Implemented support for the new Verilog 2001 port declaration format.
      In these cases I've simply ommited the ports from the net declaration groups and only kept the port declaration itself.
    2. Added scope extraction, such that taglist like plugins can extract hierarchical information.
    3. Started extending the Verilog parser to be able to support SystemVerilog keywords.
      This one is still in a proof of concept state and would probably benefit from Vikram's work if we could somehow coordinate the merge of our work.

    I am not very experienced in C, so any suggestions originating from a code review are welcome.

    Until now I have only made a very brief check of Vikram's work and only have the following comments to make:

    1. I don't know if it makes sense to have a "undefined/global" in the scope. Why not use module/class/etc?
    2. Function/task ports/arguments are not extracted.

    I'll try to analyse Vikram's code in more detail as soon as I have some free time.

    As a final topic of discussion, I'd like to know if anyone has any ideas on how we can support dynamic types, like classes.

    [1] https://github.com/vhda/ctags

     
  • Steve

    Steve - 2014-06-04

    Vitor,

    I have a partial start in my working copy of class support. Unfortunately, it has stalled behind paying work. I'd be happy to have more people working on it, and am certainly interested in getting a single version merged rather than three partial ones.

    I think the first thing we need to figure out is, what is the official head, and how do we go about getting it updated? Official release 5.8 from 2009 is in serious need of update, and nobody seems to be interested / know / care about who owns this thing now, as Mr. Hiebert seems to have disappeared.

    ================================

    From: Vitor Antunes [mailto:vantunes@users.sf.net]
    Sent: Tuesday, June 03, 2014 11:46 AM
    To: [ctags:support-requests]
    Subject: [ctags:support-requests] #17 systemverilog support for ctags

    Hi everyone,
    I have started extending the Verilog parser a bit, but I've been working in a github fork [1] of fishman's repository.
    Masatake has kindly pointed out this thread, to which I feel I can contribute.
    Since I don't want to wait for a github/sourceforge flow to be defined, I decided to reply directly to be able to receive comments from the verilog parser maintainers.
    At the moment I've only done the following:
    1. Implemented support for the new Verilog 2001 port declaration format.
    In these cases I've simply ommited the ports from the net declaration groups and only kept the port declaration itself.
    2. Added scope extraction, such that taglist like plugins can extract hierarchical information.
    3. Started extending the Verilog parser to be able to support SystemVerilog keywords.
    This one is still in a proof of concept state and would probably benefit from Vikram's work if we could somehow coordinate the merge of our work.
    I am not very experienced in C, so any suggestions originating from a code review are welcome.
    Until now I have only made a very brief check of Vikram's work and only have the following comments to make:
    1. I don't know if it makes sense to have a "undefined/global" in the scope. Why not use module/class/etc?
    2. Function/task ports/arguments are not extracted.
    I'll try to analyse Vikram's code in more detail as soon as I have some free time.
    As a final topic of discussion, I'd like to know if anyone has any ideas on how we can support dynamic types, like classes.
    [1] https://github.com/vhda/ctags

     
  • Vitor Antunes

    Vitor Antunes - 2014-06-04

    Steve,

    I think we can do both things in parallel: we can start working towards a merge of our work and when we figure out the head of the code we rebase over it.
    My suggestion would be to work in github because there are other people already working in it. Do you think you would be able to apply your work on a github fork of fishman's repository? Please apply it over commit 1d395ed, which is before my work got merged.

    Vitor

     
  • Vitor Antunes

    Vitor Antunes - 2014-06-24

    Hi Steve,

    The github fork now includes the complete subversion tree merged into it (search for the git-svn tags in the commit descriptions). This should help tracking the differences between SF and github.

    I'm still looking forward to your review of my patches :)

    Vitor

     
  • MJ1

    MJ1 - 2015-01-21

    Hi,

    Just curious as to where this was left. Are there any implementations that are available for testing?

     
  • Vitor Antunes

    Vitor Antunes - 2015-01-21

    SystemVerilog support is implemented in exuberant-ctags github development area [1]. Note: this is different from the version provided by Vikram.

    [1] https://github.com/fishman/ctags

     

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks