doxygen-develop

[Doxygen-develop] Fortran code test suite

[Albert <alb...@gm...>]

Dear all,

Currently my main interest for Doxygen is with the Fortran part. When
using Doxygen, for Fortran, I've noticed some bugs and I've tried to
fix some bugs.
A type of bug that is quite annoying is of the form:
********************************************************************
Error in file /cygdrive/c/Users/user/AppData/Local/Temp/dg/Wall.f90
line: 14, state: 10
********************************************************************
as most of the time it is quite unclear where the problem is and can
only be found by stripping the code to the bare minimum.

I think it would be a good idea to have some sort of test suite to
test if all Fortran constructs and especially the newer ones are
covered by Doxygen.
I've been looking around the Internet and found a test suite with
gfortran / gcc. This test suite is quite large and does not only
contain correct programs but also tests to have the compiler emit
errors. For Doxygen, I think, we should concentrate on programs that
do compile (and are standard conforming).

I've only looked if the programs didn't throw an error message, I have
not looked at the results.
With the aid of this suite I did find some problems eg:
- variable with the name "type" confuses Doxygen (Doxygen thinks it is
the start of a type definition)
- construct "type is" (new since Fortran 2003) confuses Doxygen
- block data is not handled by Doxygen (existing bug 567494)
- double complex is not handled by Doxygen (this is not standard
conforming code)
- the "extends" keyword gives an error message from the file
fortrancode.l (When in the Doxyfile the option SOURCE_BROWSER = YES)
For the first 2 I'm going to submit a bug report, found for the "type
is" already a solution. The "double complex" is not standard Fortran
so I won't submit a bug report for it, users should use
complex(kind=1.0d1) or similar). For the "extends" problem I'll append
my example to bug 598481.

The reason for this email is: does anybody know more places with test
suites for Fortran that can be used to find possible problems ?

Best Regards,

Albert

Re: [Doxygen-develop] Fortran code test suite

[Oleg B. <og...@gm...>]

Hi Albert,

On Tue, Mar 1, 2011 at 9:25 PM, Albert <alb...@gm...> wrote:
> Dear all,
> I think it would be a good idea to have some sort of test suite to
> test if all Fortran constructs and especially the newer ones are
> covered by Doxygen.
This is a good idea. Especially, if it is possible to match doxygen
output against predefined answer, so not only program errors are
caught but also failures. I doubt however that it is easy to achieve.

> The reason for this email is: does anybody know more places with test
> suites for Fortran that can be used to find possible problems ?
The only idea I have is to look into Fortran parsers,e.g.
OpenFortranParser
http://fortran-parser.sourceforge.net/
Phortran
http://www.eclipse.org/photran/

Best,
Oleg

>
> Albert
>
> ------------------------------------------------------------------------------
> Free Software Download: Index, Search & Analyze Logs and other IT data in
> Real-Time with Splunk. Collect, index and harness all the fast moving IT data
> generated by your applications, servers and devices whether physical, virtual
> or in the cloud. Deliver compliance at lower cost and gain new business
> insights. http://p.sf.net/sfu/splunk-dev2dev
> _______________________________________________
> Doxygen-develop mailing list
> Dox...@li...
> https://lists.sourceforge.net/lists/listinfo/doxygen-develop
>

Re: [Doxygen-develop] Fortran code test suite

[Albert <alb...@gm...>]

Hi Oleg,

Thanks for your support.
I've had a quick look at the OpenFortranParser site and in the
downloads there are quite a few test files (so I'll have a go at
them). Photran is packed in jar files so I have to Install it, I will
have a go at it (didn't see a test suite there yet).

Best Regards,

Albert

On Wed, Mar 2, 2011 at 07:05, Oleg Batrashev <og...@gm...> wrote:
> Hi Albert,
>
> On Tue, Mar 1, 2011 at 9:25 PM, Albert <alb...@gm...> wrote:
>> Dear all,
>> I think it would be a good idea to have some sort of test suite to
>> test if all Fortran constructs and especially the newer ones are
>> covered by Doxygen.
> This is a good idea. Especially, if it is possible to match doxygen
> output against predefined answer, so not only program errors are
> caught but also failures. I doubt however that it is easy to achieve.
>
>> The reason for this email is: does anybody know more places with test
>> suites for Fortran that can be used to find possible problems ?
> The only idea I have is to look into Fortran parsers,e.g.
> OpenFortranParser
> http://fortran-parser.sourceforge.net/
> Phortran
> http://www.eclipse.org/photran/
>
> Best,
> Oleg
>
>>
>> Albert
>>
>> ------------------------------------------------------------------------------
>> Free Software Download: Index, Search & Analyze Logs and other IT data in
>> Real-Time with Splunk. Collect, index and harness all the fast moving IT data
>> generated by your applications, servers and devices whether physical, virtual
>> or in the cloud. Deliver compliance at lower cost and gain new business
>> insights. http://p.sf.net/sfu/splunk-dev2dev
>> _______________________________________________
>> Doxygen-develop mailing list
>> Dox...@li...
>> https://lists.sourceforge.net/lists/listinfo/doxygen-develop
>>
>

Re: [Doxygen-develop] Fortran code test suite

[Arnold.Steve <arn...@en...>]

Howdy:

I haven't played with Photran for a while, but I'm pretty sure it also relies on the gcc/gfortran toolchain (or some other Fortran compiler) for much of the work.  That said, the Java stuff in the .jar files should include the language-sensitive parts (for the Eclipse editor) and maybe some basic analysis tools (the latter is just a guess).

I'm thinking the OpenFortranParser thing would be your best bet...

Steve

________________________________________
From: Albert [alb...@gm...]
Sent: Wednesday, March 02, 2011 10:12 AM
To: Oleg Batrashev
Cc: dox...@li...
Subject: Re: [Doxygen-develop] Fortran code test suite

Hi Oleg,

Thanks for your support.
I've had a quick look at the OpenFortranParser site and in the
downloads there are quite a few test files (so I'll have a go at
them). Photran is packed in jar files so I have to Install it, I will
have a go at it (didn't see a test suite there yet).

Best Regards,

Albert

On Wed, Mar 2, 2011 at 07:05, Oleg Batrashev <og...@gm...> wrote:
> Hi Albert,
>
> On Tue, Mar 1, 2011 at 9:25 PM, Albert <alb...@gm...> wrote:
>> Dear all,
>> I think it would be a good idea to have some sort of test suite to
>> test if all Fortran constructs and especially the newer ones are
>> covered by Doxygen.
> This is a good idea. Especially, if it is possible to match doxygen
> output against predefined answer, so not only program errors are
> caught but also failures. I doubt however that it is easy to achieve.
>
>> The reason for this email is: does anybody know more places with test
>> suites for Fortran that can be used to find possible problems ?
> The only idea I have is to look into Fortran parsers,e.g.
> OpenFortranParser
> http://fortran-parser.sourceforge.net/
> Phortran
> http://www.eclipse.org/photran/
>
> Best,
> Oleg
>
>>
>> Albert
>>
>> ------------------------------------------------------------------------------
>> Free Software Download: Index, Search & Analyze Logs and other IT data in
>> Real-Time with Splunk. Collect, index and harness all the fast moving IT data
>> generated by your applications, servers and devices whether physical, virtual
>> or in the cloud. Deliver compliance at lower cost and gain new business
>> insights. http://p.sf.net/sfu/splunk-dev2dev
>> _______________________________________________
>> Doxygen-develop mailing list
>> Dox...@li...
>> https://lists.sourceforge.net/lists/listinfo/doxygen-develop
>>
>

------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in
Real-Time with Splunk. Collect, index and harness all the fast moving IT data
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business
insights. http://p.sf.net/sfu/splunk-dev2dev
_______________________________________________
Doxygen-develop mailing list
Dox...@li...
https://lists.sourceforge.net/lists/listinfo/doxygen-develop

The information contained in this email message is intended only for the use of the individual(s) to whom it is addressed and may contain information that is privileged and sensitive. If you are not the intended recipient, or otherwise have received this communication in error, please notify the sender immediately by email at the above referenced address and note that any further dissemination, distribution or copying of this communication is strictly prohibited.

The U.S. Export Control Laws regulate the export and re-export of technology originating in the United States. This includes the electronic transmission of information and software to foreign countries and to certain foreign nationals. Recipient agrees to abide by these laws and their regulations -- including the U.S. Department of Commerce Export Administration Regulations and the U.S. Department of State International Traffic in Arms Regulations -- and not to transfer, by electronic transmission or otherwise, any content derived from this email to either a foreign national or a foreign destination in violation of such laws.

Re: [Doxygen-develop] Fortran code test suite

[Oleg B. <og...@gm...>]

Hi,

>>> I think it would be a good idea to have some sort of test suite to
>>> test if all Fortran constructs and especially the newer ones are
>>> covered by Doxygen.
>> This is a good idea. Especially, if it is possible to match doxygen
>> output against predefined answer, so not only program errors are
>> caught but also failures. I doubt however that it is easy to achieve.

I found there is an xml output from Doxygen. Has anyone tried to write
regression tests based on it, using XPath or any similar technique?
It seems to be good tool to verify parsers.

Oleg

Re: [Doxygen-develop] Fortran code test suite

[Oleg B. <og...@gm...>]

Hi,

> I found there is an xml output from Doxygen. Has anyone tried to write
> regression tests based on it, using XPath or any similar technique?
> It seems to be good tool to verify parsers.
I've made a try and quite satisfied by now.
https://github.com/ogbash/doxygen-tests

Currently there are only 8 tests, where 5 are taken from bug reports.
Release-1.7.1 gives 7 failures.
Release-1.7.3-20110217 gives 3 failures out of 8 tests.

runTest (tests.blocks.interface_abst_f90) ... ok
runTest (tests.blocks.interface_gen_f90) ... ok
runTest (tests.blocks.interface_op_f90) ... ok
runTest (tests.blocks.interface_spec_f90) ... ok
runTest (tests.syntax.label_endsub) ... ok
runTest (tests.linecont.parcomment_f) ... FAIL
runTest (tests.linecont.parcomment_f90) ... FAIL
runTest (tests.linecont.varcomment_f90) ... FAIL

89e2c4 from https://github.com/ogbash/doxygen-f90 gives no errors.

Writing tests is not trivial but managable after making yourself
familiar to XPath syntax.
For example, testing that 'example' subroutine exists in
'parcomment.f' file and its 'val' parameter is documented:

        file = self.getFile("parcomment.f")
        example = self.getSubprogram(file, "example")

        desc = self.getParamDescription(example, "val")
        self.assertEqual(desc.getContent().strip(), "[in] scalar double input")

Testing type of the variable 'as' with XPath:

        var = self.getModuleVariable(varcomment, "as")
        self.assertEqual(var.xpathEval("type")[0].getContent().strip(),
                         "integer, dimension(:,:), pointer")


Any suggestions, comments, or even new tests are welcome,
Oleg

Re: [Doxygen-develop] Fortran code test suite

[<pau...@no...>]

Hi Oleg,

We have working with Doxygen to generate DITA XML. When we started this we generated C++ reference tests for each tag (and tag combinations) then we could compare Doxygen's DITA XML output with a reference output and test if there were any regressions. This was integrated into our CI server.

We are in the process of preparing a patch for the latest DITA support in Doxygen and updating the DITA specialisation to version 0.7.0 (currently 0.6.0 is available at http://sourceforge.net/projects/dita-ot/files/Plug-in_C%2B%2B%20APIRef/ ).

Our DITA XML test suite is a bit old and needs some work to align it with all the stuff we now support with the current (patched) version of Doxygen. It is, of course, C++ input and DITA XML output but there is no reason why it could not be extended to other input languages and output formats. We found it very useful for detecting errors, especially regressive ones.

We'd be quite happy to share this with the Doxygen community if people think that it would be useful. If so I'll see if we can spend a bit of time getting it into good shape.

Best regards,

Paul.

8<------------------------------->8
Paul Ross
Senior Technology Architect
System Documentation Tools
D RD SD SSS System Doc & Tools GB
Nokia London, UK.

> -----Original Message-----
> From: ext Oleg Batrashev [mailto:og...@gm...]
> Sent: 14 March 2011 18:13
> To: Arnold.Steve
> Cc: dox...@li...
> Subject: Re: [Doxygen-develop] Fortran code test suite
> 
> Hi,
> 
> > I found there is an xml output from Doxygen. Has anyone tried to
> write
> > regression tests based on it, using XPath or any similar technique?
> > It seems to be good tool to verify parsers.
> I've made a try and quite satisfied by now.
> https://github.com/ogbash/doxygen-tests
> 
> Currently there are only 8 tests, where 5 are taken from bug reports.
> Release-1.7.1 gives 7 failures.
> Release-1.7.3-20110217 gives 3 failures out of 8 tests.
> 
> runTest (tests.blocks.interface_abst_f90) ... ok
> runTest (tests.blocks.interface_gen_f90) ... ok
> runTest (tests.blocks.interface_op_f90) ... ok
> runTest (tests.blocks.interface_spec_f90) ... ok
> runTest (tests.syntax.label_endsub) ... ok
> runTest (tests.linecont.parcomment_f) ... FAIL
> runTest (tests.linecont.parcomment_f90) ... FAIL
> runTest (tests.linecont.varcomment_f90) ... FAIL
> 
> 89e2c4 from https://github.com/ogbash/doxygen-f90 gives no errors.
> 
> Writing tests is not trivial but managable after making yourself
> familiar to XPath syntax.
> For example, testing that 'example' subroutine exists in
> 'parcomment.f' file and its 'val' parameter is documented:
> 
>         file = self.getFile("parcomment.f")
>         example = self.getSubprogram(file, "example")
> 
>         desc = self.getParamDescription(example, "val")
>         self.assertEqual(desc.getContent().strip(), "[in] scalar double
> input")
> 
> Testing type of the variable 'as' with XPath:
> 
>         var = self.getModuleVariable(varcomment, "as")
>         self.assertEqual(var.xpathEval("type")[0].getContent().strip(),
>                          "integer, dimension(:,:), pointer")
> 
> 
> Any suggestions, comments, or even new tests are welcome,
> Oleg
> 
> -----------------------------------------------------------------------
> -------
> Colocation vs. Managed Hosting
> A question and answer guide to determining the best fit
> for your organization - today and in the future.
> http://p.sf.net/sfu/internap-sfd2d
> _______________________________________________
> Doxygen-develop mailing list
> Dox...@li...
> https://lists.sourceforge.net/lists/listinfo/doxygen-develop

Re: [Doxygen-develop] Fortran code test suite

[Dimitri V. H. <do...@gm...>]

Hi Oleg,

I once made a couple of simple bash scripts for testing regression, also based
on doxygen's XML output. I've attached them. The scripts assume a set of test
files, each starting with a letter t. With the update script you can make a
reference, and with the check script compare the actual results against 
the reference. Maybe it is of use.

Re: [Doxygen-develop] Fortran code test suite

[Oleg B. <og...@gm...>]

Hi,

> I once made a couple of simple bash scripts for testing regression, also based
> on doxygen's XML output. I've attached them. The scripts assume a set of test
> files, each starting with a letter t. With the update script you can make a
> reference, and with the check script compare the actual results against
> the reference. Maybe it is of use.
Thats much simpler solution with seemingly the same outcome. I think
I'll play with my for some time.

Oleg

>
> Regards,
>  Dimitri