srcml-discussion — General discussions about the SrcML framework

[SrcML] [ srcml-Bugs-1372850 ] line numbers wrong for javadoc

[SourceForge.net <no...@so...>]

Bugs item #1372850, was opened at 2005-12-04 15:15
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=641278&aid=1372850&group_id=105448

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: View
Group: None
Status: Open
Resolution: None
Priority: 6
Submitted By: Raiser Frank (crashchaos)
Assigned to: Leif Bladt (iknowjack)
Summary: line numbers wrong for javadoc

Initial Comment:
When linenumbers:true is set for the ViewJava plugin
there is an error in the numbering of lines for
multi-line comments looking like this:

..
10: something();
11: /**
   * this is a javadoc
   * comment spanning
   * multiple lines
   */
12: something_else();

Possible solution: don't output the multiline comments
as one string, but split them at newlines and force
output of the corresponding newlines.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=641278&aid=1372850&group_id=105448

Re: [SrcML] problem with absolute paths

[Frank R. <fra...@un...>]

On Fri, Dec 02, 2005 at 01:21:33PM +0100, Icaro wrote:
> Hi to all

Hello,

> I have the following problem:
> I have a java file in c:\ (i.e.: exist c:\Policy.java)
> If I run
> 
> C:\>java -classpath "C:\cartella condivisa\Software 
> Downloads\dev\SrcML\srcml-0.2.2.jar" de.srcml.parser.Parser Policy.java
> it work, but if I run:
> C:\>java -classpath "C:\cartella condivisa\Software 
> Downloads\dev\SrcML\srcml-0.2.2.jar" de.srcml.parser.Parser c:\Policy.java

Interesting.. It works on Linux no matter wether you use a full or a
relative path. Therefore I never encountered this error before. I also
don't have access to a windows machine here, so that'll have to wait
until I'm at the uni again. But this is indeed an annoying error which
just shows again that we're having way too few testers.

Thanks for pointing out this bug Domenico. I'll try to reproduce and fix
it tomorrow at the uni. I'll keep you updated on this here.

> Is it a bug? How can I use an absolute path to specify the file to parse?

Yes. Apparently some part of the framework has a problem with those
absolute paths in Windows. As we currently only have developers on Linux
and Mac this kind of slipped through the testing. Sorry for that.

PS: Everyone can (and is encouraged to even ;) post such bugs through
the SourceForge Bug Tracking system. I'll try to also post bugs sent to
this list through the bug tracker. This isn't really necessary, but
provides us with some additional advantages: We can assign bugs to
developers, site visitors can see we're actively working on getting
those bugs fixed, it adds to the activity ranking thus improving the
project's SF ranking. Or in simpler words: it might help us to find more
developers or at least testers :)

-- 
Raiser, Frank
Student @ University of Ulm (www.uni-ulm.de)

It's not the prevention of bugs but the recovery 
-- the ability to gracefully exterminate them -- that counts. 
(Victoria Livschitz)

[SrcML] ViewPlatform and IndentEvent.java

[<lei...@un...>]

I found some spare time today, so I tried to get the new IndentEvent working...

But I'm a bit confused, if it's enough to change the outputIndent() method in
ViewPlugin.java and the sendEvent() method in View.java. How is indenting done
'til now? So Frank, could give me a short draft, how it works right now?

Thank you and have a nice weekend!
> Leif

[SrcML] problem with absolute paths

[Icaro <ic...@gm...>]

Hi to all

I have the following problem:
I have a java file in c:\ (i.e.: exist c:\Policy.java)
If I run

C:\>java -classpath "C:\cartella condivisa\Software 
Downloads\dev\SrcML\srcml-0.2.2.jar" de.srcml.parser.Parser Policy.java
it work, but if I run:
C:\>java -classpath "C:\cartella condivisa\Software 
Downloads\dev\SrcML\srcml-0.2.2.jar" de.srcml.parser.Parser c:\Policy.java

(note the *c:\* ) I obtain the following Exception:

java.lang.NullPointerException
        at de.srcml.parser.java.Preprocessor.read(Unknown Source)
        at de.srcml.parser.java.Preprocessor.filterReader(Unknown Source)
        at de.srcml.parser.java.Preprocessor.<init>(Unknown Source)
        at de.srcml.parser.java.ParserJava.parseFile(Unknown Source)
        at de.srcml.parser.java.ParserJava.parse(Unknown Source)
        at de.srcml.parser.Parser.parse(Unknown Source)
        at de.srcml.parser.Parser.main(Unknown Source)
de.srcml.parser.ParserException: Exception occured while parsing: 
java.lang.NullPointerException
        at de.srcml.parser.java.ParserJava.parseFile(Unknown Source)
        at de.srcml.parser.java.ParserJava.parse(Unknown Source)
        at de.srcml.parser.Parser.parse(Unknown Source)
        at de.srcml.parser.Parser.main(Unknown Source)
<?xml version="1.0" encoding="ISO-8859-1"?>
<unit filename="null" author="Domenico" language="java" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xsi:noNamespaceSchemaLocation="http://srcml.sourceforge.net/srcml.xsd">

Is it a bug? How can I use an absolute path to specify the file to parse?


-- 
Domenico (aka ^I^caro)

http://preponderante.splinder.com  <-- personale
http://icaro.blogspot.com          <-- perhaps technical
          
     ~~~~     "Dove mai andiamo?"......"Sempre a casa"....   ~~~~

[SrcML] documentation of ViewPlatform

[Frank R. <fra...@un...>]

I've just received feedback from Simon, that his report was created in
German and will not be published, so I went about creating the missing
documentation for the new ViewPlatform. I hope this will help everyone
to get into the event system easier.

@Leif: You may want to add the parts about the *Done events and maybe
add a page about the design of the XHTML plugin later.

PS: I also sent a support request to SF for removing the directories of
the old view plugins. I left the ViewXTHML directory though, as Leif
will be reusing that one.

-- 
Raiser, Frank
Student @ University of Ulm (www.uni-ulm.de)

The software said it requires Windows 95 or better, so I installed Unix.

Re: [SrcML] problems in plugin building

[Frank R. <fra...@un...>]

On Fri, Nov 25, 2005 at 09:25:31PM +0100, Icaro wrote:
> I am trying to turn the code XML of SrcML into an useful format to one 
> project of mine. (Fine-grained Configuration management). 
> Practically I want to eliminate details for me superfluous, and to add 
> other information. 
> what I want is to get from this:
> 
> import javax.swing.*;
> public class testFrame extends Frame{   
>    public int ELEMENTO = 0;
> 
>    public void addbuttons(int n){       
>        for(int i =1; i<=n; i++){           
>            this.add( new Button(""+i));
>        }
>    }
> }
> 
> something similar to this:
> 
> <?xml version="1.0" encoding="UTF-8"?>
> <ARTEFACT id="012345" filename="test.java">
>    <CONTAINER id="012345.1" name ="test" type="class" 
> sinceFileVersion="1.0" autor="AuthorXyz" >
> 
>        <AIU   id="012345.1.1" name="ELEMENTO" type="field" 
> sinceFileVersion="1.0">
>            <value version="1.0" time="19081979 20:20" autor="AuthorKjf">
>                  public int ELEMENTO = 0;
>            </value>
>         </AIU>
>        
>         <AIU  id="012345.1.2" name="addbuttons" type="method" 
> sinceFileVersion="1.2">
>            <value version="1.1" time="19081979 20:20" 
> autor="AuthorXyz">   
>                  public void addbuttons(int n)
>            </value>           
>          </AIU>
> 
>    </CONTAINER>
> </ARTEFACT>
> 
> I think that the thing is easier if I take advantage of the XML of SrcML 
> rather than to work on the file source native. 
> Considering that then SrcML makes a platform available to simplify the 
> transformation from XML in something else I have thought about trying to 
> use it
> Then, in conclusion, I need at least to reconstruct the signatures of 
> the classes, of the methods and of the global variable.

If it wasn't for reconstructing those method signatures you could have
gotten away with a XSLT script.
When writing a ViewPlugin however you can get those signatures easily:
- first detach() the nodes you don't want (thus creating the level of
  detail you want)
- then call SrcMLElement.getAsString which gives you the signature you
  want

> ViewJava is made with the new ViewPlatform? This is the simplest example 
> available of the use of the platform?
> I had chosen ViewXHTML because it seemed closer to my purposes. (from 
> XML to XML)

Currently it's the most simple example, because it's the only plugin
using the new ViewPlatform. The ViewXHTML plugin however will probably
not be as simple, because it involves hooking into the ViewPlatform and
the working of the ViewJava plugin. In that way the XTHML plugin is not
responsible for the actual text produced, but only modifies the creation
process in such a way that the plaintext is enhanced with the
corresponding XHTML tags.

> for me there are a lot of not important information in the SrcML files 
> and therefore are useless to produce.
> Besides, in the case I decided not to use the ViewPlattform but to 
> create my own parser of the XML of SrcML to have less useless things in 
> front of me will make simplest the job.

You could run a simple XSLT script in order to cut out those parts of
the SrcML document you're not interested in. No need to write a parser
of any kind for that.

> not in the wiki, but here: http://srcml.sourceforge.net/bericht.html

Ok. This report is clearly flagged as deprecated on the wiki. Really a
lot of things have changed since then and it's probably hard to find
anything which still is applicable to the current version.

> thanks for the detailed and exhaustive answers.

You're welcome.

-- 
Raiser, Frank
Student @ University of Ulm (www.uni-ulm.de)

Re: [SrcML] problems in plugin building

[Icaro <ic...@gm...>]

Frank Raiser ha scritto:

>>I am trying to create a ViewPlugin but I have found some problems. 
>>    
>>
>
>Any specific idea on what the plugin is supposed to do or are you just
>toying around with the ViewPlugins? In the first case let us know and
>maybe we can include it in the official SrcML distribution.
>  
>

I am trying to turn the code XML of SrcML into an useful format to one 
project of mine. (Fine-grained Configuration management). 
Practically I want to eliminate details for me superfluous, and to add 
other information. 
what I want is to get from this:

import javax.swing.*;
public class testFrame extends Frame{   
    public int ELEMENTO = 0;

    public void addbuttons(int n){       
        for(int i =1; i<=n; i++){           
            this.add( new Button(""+i));
        }
    }
}

something similar to this:

<?xml version="1.0" encoding="UTF-8"?>
<ARTEFACT id="012345" filename="test.java">
    <CONTAINER id="012345.1" name ="test" type="class" 
sinceFileVersion="1.0" autor="AuthorXyz" >

        <AIU   id="012345.1.1" name="ELEMENTO" type="field" 
sinceFileVersion="1.0">
            <value version="1.0" time="19081979 20:20" autor="AuthorKjf">
                  public int ELEMENTO = 0;
            </value>
         </AIU>
        
         <AIU  id="012345.1.2" name="addbuttons" type="method" 
sinceFileVersion="1.2">
            <value version="1.1" time="19081979 20:20" 
autor="AuthorXyz">   
                  public void addbuttons(int n)
            </value>           
          </AIU>

    </CONTAINER>
</ARTEFACT>

I think that the thing is easier if I take advantage of the XML of SrcML 
rather than to work on the file source native. 
Considering that then SrcML makes a platform available to simplify the 
transformation from XML in something else I have thought about trying to 
use it
Then, in conclusion, I need at least to reconstruct the signatures of 
the classes, of the methods and of the global variable.

>>I have created a new project (using Eclipse) inserting srcml-0.2.2.jar 
>>in the classpath  and
>>then I have copied the sources of ViewXHTML.java to have a starting point. 
>>    
>>
>
>Oops. That's not going to work. The ViewXHTML.java currently in CVS is
>still building upon the old ViewPlatform. Leif is right now working on
>porting it over to the new platform.
>  
>

Ok, I have understood.

[...]

>I'll have to publish a
>SourceForge support request for the removal of those outdated
>directories soon.
>  
>
I think that this is a good think.

>For now I suggest you take a look at the current ViewJava plugin:
>http://cvs.sourceforge.net/viewcvs.py/srcml/views/ViewJava/de/srcml/view/java/ViewJava.java?view=markup
>  
>

ViewJava is made with the new ViewPlatform? This is the simplest example 
available of the use of the platform?
I had chosen ViewXHTML because it seemed closer to my purposes. (from 
XML to XML)

[...]

>But for now I'll quickly mark the documentation on the wiki as deprecated.
>Thanks for pointing this out.
>  
>
of nothing

>>PS: is it possible to have in the output of SrcML the line in which a 
>>statement was present in the original source file? (e.g. <class name = 
>>"StatementTest" row=28>  )
>>    
>>
>
>Short answer: officially not
>
>Detailed answer: As SrcML is abstracting from the actual line-by-line
>representation of source code there is nothing like a positional
>information in terms of 'lines' contained in a SrcML document. In our
>vision even something like the 'original source file' is not really
>applicable, as you would save your sources directly in SrcML which
>pretty much replaces the meaning of what the 'original source file' is.
>  
>
I don't intend to use SrcML as a format of the information but as medium 
to reach something else. 
my project supposes to be the most possible transparency to the user.

>Hackish answer: Despite the above there is of course the technical
>possibilty to get what you want. If you modify the existing parser to
>include the positioning of tokens in the resulting SrcML document you
>(currently [1]) break the Schema, but apart from that you get the
>information you wanted. If you want to go that way I can direct you
>towards the java.tree.g file in the parser-java module. It shouldn't be
>too difficult to add this information as ANTLR tokens always know their
>original location in the source code. It's just a lot of effort to add
>this information everywhere, because the grammar file is several
>thousand lines long.
>  
>

for my purposes it would be enough to have information for the classes, 
the methods and the global variable.
but however,  for now, it is not so important and I don't have the time 
to modify SrcML  :-)

>>PPS: is it still working the possibility to have an XML output with a 
>>specified level of detail? (I believe it was the option - g...I'have 
>>read something in the SrcML documentation)
>>    
>>
>
>No. This was an idea we incorporated from similar projects some years
>ago [2]. However I'd like to know what use you have for such a feature. The
>only advantage I can come up with is that the parsing process would be
>faster (of course it would still be linear in the file size). But if you
>can live with a full SrcML document you get all levels of details for
>free, as you can simply choose how deep you walk down the DOM tree.
>  
>

for me there are a lot of not important information in the SrcML files 
and therefore are useless to produce.
Besides, in the case I decided not to use the ViewPlattform but to 
create my own parser of the XML of SrcML to have less useless things in 
front of me will make simplest the job.

>[...] The current schema is very java centric 
>
Yes, it seems also to me.

>[2] Have you found the information about detail levels in the parser on
>any of the current wiki pages? If yes could you please tell us which
>one, so it can be updated accordingly.
>
not in the wiki, but here: http://srcml.sourceforge.net/bericht.html

<cite>
Src2SrcML converter version 0.1 (for Java)
Usage: java2srcml <options> <sourcecode file>
Supported options:
 -g n  : granularity [1 = classes only : 4 = full markup ]
 -t n  : number of spaces to replace \t by. (0 = don't replace)
 -n    : add <line_break/> tags
 -v    : print version info and exit
</cite>



thanks for the detailed and exhaustive answers.

-- 
Domenico (aka ^I^caro)

http://icaro.blogspot.com
          
     ~~~~     "Dove mai andiamo?"......"Sempre a casa"....   ~~~~

Re: [SrcML] problems in plugin building

[Frank R. <fra...@un...>]

On Fri, Nov 25, 2005 at 04:39:08PM +0100, Icaro wrote:
> Hi 

Hello,

> I am trying to create a ViewPlugin but I have found some problems. 

Any specific idea on what the plugin is supposed to do or are you just
toying around with the ViewPlugins? In the first case let us know and
maybe we can include it in the official SrcML distribution.

> I have created a new project (using Eclipse) inserting srcml-0.2.2.jar 
> in the classpath  and
> then I have copied the sources of ViewXHTML.java to have a starting point. 

Oops. That's not going to work. The ViewXHTML.java currently in CVS is
still building upon the old ViewPlatform. Leif is right now working on
porting it over to the new platform.

> However there are some problems because the following names cannot be 
> resolved 
> m_outupt, SelectionPair, output, m_line and m_column 

Right. The whole SelectionPair process was rather complicated and tough
to get into. The new ViewPlatform is a lot easier in that respect. The
problem right now is that we just recently switched to this new platform
and so we still have some of the older stuff left in CVS (mainly because
we didn't know if the new concept was going to work out and wanted to be
able to do a rollback on the old platform). I'll have to publish a
SourceForge support request for the removal of those outdated
directories soon.
For now I suggest you take a look at the current ViewJava plugin:
http://cvs.sourceforge.net/viewcvs.py/srcml/views/ViewJava/de/srcml/view/java/ViewJava.java?view=markup

I'm sorry that the documentation on the wiki is deprecated in that area,
but I have to get into contact with Simon first, as I'm not sure if his
last report contained the documentation for the new ViewPlatform too and
it would be quite a waste of time to unneccessarily rewrite it. But for
now I'll quickly mark the documentation on the wiki as deprecated.
Thanks for pointing this out.

> PS: is it possible to have in the output of SrcML the line in which a 
> statement was present in the original source file? (e.g. <class name = 
> "StatementTest" row=28>  )

Short answer: officially not

Detailed answer: As SrcML is abstracting from the actual line-by-line
representation of source code there is nothing like a positional
information in terms of 'lines' contained in a SrcML document. In our
vision even something like the 'original source file' is not really
applicable, as you would save your sources directly in SrcML which
pretty much replaces the meaning of what the 'original source file' is.

Hackish answer: Despite the above there is of course the technical
possibilty to get what you want. If you modify the existing parser to
include the positioning of tokens in the resulting SrcML document you
(currently [1]) break the Schema, but apart from that you get the
information you wanted. If you want to go that way I can direct you
towards the java.tree.g file in the parser-java module. It shouldn't be
too difficult to add this information as ANTLR tokens always know their
original location in the source code. It's just a lot of effort to add
this information everywhere, because the grammar file is several
thousand lines long.

> PPS: is it still working the possibility to have an XML output with a 
> specified level of detail? (I believe it was the option - g...I'have 
> read something in the SrcML documentation)

No. This was an idea we incorporated from similar projects some years
ago [2]. However I'd like to know what use you have for such a feature. The
only advantage I can come up with is that the parsing process would be
faster (of course it would still be linear in the file size). But if you
can live with a full SrcML document you get all levels of details for
free, as you can simply choose how deep you walk down the DOM tree.

[1] Breaking the SrcML schema is rather unimportant at this point. I've
come to the conclusion, that the current schema is our major drawback
and I'm trying to shift my diploma thesis such that I can work on
creating a proper schema. The current schema is very java centric and
should work fine as long as you don't mix in new languages, but even
then it's unneccessarily complicated at a lot of places due to the
inherent similarity to the parser's abstract syntax tree. Also right now
the schema is very stringent in that it does not allow you to extend
SrcML documents which I also consider a major drawback (say you want to
include some MathML or SVG or your design diagrams in the form of XMI -
currently not allowed by the schema)

[2] Have you found the information about detail levels in the parser on
any of the current wiki pages? If yes could you please tell us which
one, so it can be updated accordingly.

-- 
Raiser, Frank
Student @ University of Ulm (www.uni-ulm.de)

"Only wimps use tape backup: _real_ men just upload their important stuff on
 ftp, and let the rest of the world mirror it ;)"  -- Linus Torvalds

[SrcML] problems in plugin building

[Icaro <ic...@gm...>]

Hi 
 
I am trying to create a ViewPlugin but I have found some problems. 
I have created a new project (using Eclipse) inserting srcml-0.2.2.jar 
in the classpath  and
then I have copied the sources of ViewXHTML.java to have a starting point. 
However there are some problems because the following names cannot be 
resolved 
m_outupt, SelectionPair, output, m_line and m_column 
 
in effects, looking in the source in CVS I don't succeed in finding such 
classes and variables. 
Which could be the problem? 
 
Is the version 0.2.2 very different from the version in CVS?

Domenico
 
PS: is it possible to have in the output of SrcML the line in which a 
statement was present in the original source file? (e.g. <class name = 
"StatementTest" row=28>  )
 
PPS: is it still working the possibility to have an XML output with a 
specified level of detail? (I believe it was the option - g...I'have 
read something in the SrcML documentation)

-- 
Domenico (aka ^I^caro)
http://icaro.blogspot.com
          
     ~~~~     "Dove mai andiamo?"......"Sempre a casa"....   ~~~~

[SrcML] de.srcml.dom.supports package

[Frank R. <fra...@un...>]

On Sat, Nov 19, 2005 at 02:37:11PM +0100, Frank Raiser wrote:
> Good idea. Hm... we could flag those elements which can have names with
> an ISupportsName interface with the implementation checking against
> keyords and reserved words then before setting the name attribute. But
> that's something totally different again :)

I've added the keywords and reserved words yesterday and finished adding
ISupportsNames now. 
There's now a new package de.srcml.dom.supports available which holds
the various ISupports* Interfaces. These interfaces are used for all DOM
classes which share certain concepts and make them easily accessible
through the API. This is supposed to avoid the need for developers to
work directly on the DOM tree (or once it's matured further to even
avoid knowing the structure of that tree). Also consistency checks of
any kind can be enforced better when these methods are used. [1]
Currently there's only the following interfaces available:

ISupportsModifiers
ISupportsMethods
ISupportsName

There's also ISupports*Impl classes with default implementations
available in the same package. I'm currently planning to add the
following interfaces too:

ISupportsBaseclasses
ISupportsInterfaces
ISupportsVariables
probably ISupportsJavadoc and ISupportsComment too

If there are any other important concepts which should have extended
support through another such interface and you think it should be added
too please let me know in a reply to this mail. Of course every tiny
little idea of every programming language could be made into one of
those interfaces, but I'm looking for those for which at least a few
of our current classes in de.srcml.dom are applicable.

[1] For example setting a name for an object implementing ISupportsName
will check if the given name is a keyword in the target language and
refuse to set it (returning false and logging the problem)

PS: Leif informed me that after the last major change of the ViewJava
plugin I forgot to run it through the whole srcml module again. This was
done today and while unit tests worked fine on my machine here I cannot
guarantee that the current CVS state is ok. I'd like to get some
feedback here from someone else to see if the current CVS version
compiles and the unit tests work if possible.

-- 
Raiser, Frank
Student @ University of Ulm (www.uni-ulm.de)

When you are a Bear of Very Little Brain, and you Think of Things, you find
sometimes that a Thing which seemed very Thingish inside you is quite
different when it gets out into the open and has other people looking at it.
(A A Milne)

Re: [SrcML] XML rationale

[Frank R. <fra...@un...>]

On Sat, Nov 19, 2005 at 03:15:34PM +0100, Icaro wrote:
> From the SrcML home page: "You can get the current version of the Schema
> __here__"
> .
> but the Schema is not available :-(

Sorry. The link was wrong there. I corrected it and you can find the
schema here:
http://cvs.sourceforge.net/viewcvs.py/srcml/srcml/srcml.xsd?view=markup

Thanks for the notice.

> Where I can find documentation about the "rationale" about the XML
> structure you have chosen to be language independent?

The current schema is rather heavily leaned against similar efforts from
the srcML and javaML projects. There is a definite need to improve this
schema. Currently it's still too closely related to a parser's AST which
I find is rather confusing at times. Maybe I have some time during my
diploma thesis to work on that a bit.

To give a simple idea of what I'm talking about here imagine a normal
class and a method in that class. Right now the schema dictates that the
structure has to look like this:
<class ...>
  <block>
    <method...>
    </method>
  </block<
</class>

simply because the java syntax uses these { .. } brackets for the class'
body. So the block tag is not really containing any useful information
at this point and methods (as well as variable declarations) should
rather go directly under class. It's also more intuitive for XPath users
to write class[@name="MyClass"]/method[@name="foo"] instead of having
to write class[@name="MyClass"]/block/method[@name="foo"].
Of course all this will be transparent when using the supplied API, but
it's still a bit of a hazzle when working directly on the XML f.ex. with
an XSL transformation.

So in short: There's still a lot of work to do on the schema and it
can't be considered ready for handling other languages nor should it be
considered stable.

PS: Also every new language will come with new features which have not
been added to the schema yet and require a modification to it. Hopefully
we can modify the schema to a point where these modifications are only
additions (as opposed to a real modification)

-- 
Raiser, Frank
Student @ University of Ulm (www.uni-ulm.de)

We stand today at a crossroads: One path leads to despair and utter
hopelessness. The other leads to total extinction. Let us hope we have
the wisdom to make the right choice. (Woody Allan)

[SrcML] XML rationale

[Icaro <ic...@gm...>]

 From the SrcML home page: "You can get the current version of the Schema
__here__"
.
but the Schema is not available :-(
the server respond:
---
      An Exception Has Occurred
srcml/api/srcml.xsd: unknown location

        HTTP Response Status

404 Not Found
---

where I can find the Schema?
Where I can find documentation about the "rationale" about the XML
structure you have chosen to be language independent?
thanks.

Icaro

-- 
http://icaro.blogspot.com

Re: [SrcML] XHTML-ViewPlugin

[Icaro <ic...@gm...>]

Frank Raiser ha scritto:

>As you already pointed out below CSS is a good way too and CSS has two
>major advantage over a property file as you described it:
>
>  
>
Yes, CSS is better! :-)


>>PS: is srcML in some way related to this: 
>>http://www.sdml.info/projects/srcml/       (SDML: srcML)
>>    
>>
>
>Note that our project goes by the name SrcML (with a capital S). Both
>project names simply originate from the obvious name "source code markup
>language" with "markup language" generally being abbreviated as ML (as
>in XML, HTML and various other lesser-known markup languages). As we
>didn't want to choose a name not including that it's about source code
>and a markup language we just went for the obvious.
>
>But other than the basic idea of an XML represenation of source code
>there is not much SrcML and srcML have in common. The resulting XML
>files are fundamentally different (mostly due to the srcML project
>insisting on a 1:1 relation between the source code file and the XML
>file, whereas SrcML only cares about the semantics). Also the srcML
>project has a C++ parser only while we have a Java parser only (with
>both projects being able to support more parser but lacking the
>developers to write them ;) The supporting API and Platforms are unique
>to SrcML too, as srcML only consists of a parser afaik.
>

thank you for the clarifications.

Icaro

-- 
http://icaro.blogspot.com

Re: [SrcML] Java language keywords

[Frank R. <fra...@un...>]

On Sat, Nov 19, 2005 at 02:08:58PM +0100, Leif Bladt wrote:
> So, keeping the list busy...

which is always a good idea ;)

> According to the list from Sun (http://java.sun.com/docs/books/ 
> tutorial/java/nutsandbolts/_keywords.html) in de.srcml.language.Java  
> there are two keywords missing: assert and enum. I'm not sure, if it  
> will result in now work in other parts of the framework (e.g. the  
> Parser or View platform), so I didn't add them to the keyword list :)

Indeed. The keyword list was probably last changed during the initial
1.3 parser time. assert comes from 1.4 and enum from 1.5. Changing them
shouldn't cause anything to break, as the parser is using it's own list
of keywords - it can't really work together with the API, as the
semantic of those keywords is important to parsing too.

> Java (and I think other languages too) has reserved words (true,  
> false, and null). Can we support these reserved words like the  
> keywords with an is Reserved(String)? This would be an advantage to  
> easily tag these words in the XHTML file as well.

Good idea. Hm... we could flag those elements which can have names with
an ISupportsName interface with the implementation checking against
keyords and reserved words then before setting the name attribute. But
that's something totally different again :)

I'll add those missing keywords and the reserved keywords right away. It
should be in CVS in about an hour.

-- 
Raiser, Frank
Student @ University of Ulm (www.uni-ulm.de)

I had to make some optimistic assumptions to meet the revenue target.
In week three, we're visited by an alien named D'utox Inag who offers
to share his advanced technology. (Scott Adams)

[SrcML] Java language keywords

[Leif B. <lei...@un...>]

So, keeping the list busy...

According to the list from Sun (http://java.sun.com/docs/books/ 
tutorial/java/nutsandbolts/_keywords.html) in de.srcml.language.Java  
there are two keywords missing: assert and enum. I'm not sure, if it  
will result in now work in other parts of the framework (e.g. the  
Parser or View platform), so I didn't add them to the keyword list :)

Java (and I think other languages too) has reserved words (true,  
false, and null). Can we support these reserved words like the  
keywords with an is Reserved(String)? This would be an advantage to  
easily tag these words in the XHTML file as well.


Leif

[SrcML] Publications

[Frank R. <fra...@un...>]

Hello everyone,

I've added a page to the wiki where all publications concerning SrcML
will be gathered. Right now only last year's report and the reports of
Leif and me are available there:
http://www.uni-ulm.de/~s_fraise/cgi-bin/moin.cgi/Publications


If anyone of the other participants wants his or her report posted
please let me know and either upload your report to some public webspace
or just send it along to me and I'll upload it.

-- 
Raiser, Frank
Student @ University of Ulm (www.uni-ulm.de)

You can either have software quality or you can have pointer arithmetic,
but you cannot have both at the same time. (Bertrand Meyer)

Re: [SrcML] XHTML-ViewPlugin

[Frank R. <fra...@un...>]

On Fri, Nov 18, 2005 at 11:22:13AM +0100, Leif Bladt wrote:
> >What about adding a boolean variable then for the different events
> >specifying whether they should fire DoneEvents and an accessor  
> >method to
> >set this behavior?
> Nice idea. So this variable could be in ViewEvent itself, with a get-  
> and set-method. And the default value for a plugin is set in its  
> constructor? In this case, you have to change thee default every time  
> you create an event. Could we get this into the view platform, so  
> that we could define the behaviours only when we create the view  
> platform?

Actually I was thinking about having it in the ViewPlatform right from
the start. So that for every platform you create you can customize what
events you want to be handled.

Let me elaborate this a bit further:

These are the event classes currently existing:
NewlineEvent, NextElementEvent, StringEvent, WhitespaceEvent

Let's add corresponding Done-Events:
NewlineDoneEvent, NextElementDoneEvent, StringDoneEvent, WhitespaceDoveEvent

Now create a Set<java.lang.Class> in the ViewPlatform which by default
contains all those event classes. Add methods to add/remove/clear
classes to/from this set. And in View.sendEvent(ViewEvent) perform a
single lookup whether the event's class is in the set. If not the event
is immediately thrown away.
For performance reasons the ViewPlatform can also keep some boolean
variables f.ex. for the generation of the *DoneEvent objects. Those
variables will have to be consistent with the set of course and can be
used to even avoid the instantiation of those events.

How does that sound to you? Anything I'm missing there?

-- 
Raiser, Frank
Student @ University of Ulm (www.uni-ulm.de)

Every technology really needs to be shipped with a special manual--
not how to use it but why, when and for what. (Alan Kay)

Re: [SrcML] XHTML-ViewPlugin

[Frank R. <fra...@un...>]

On Fri, Nov 18, 2005 at 01:12:46AM +0100, Icaro wrote:
> You could have a language independnt generator but a language specific 
> color property file for each language.

We definitely plan on having the generator as language independent as
possible (through the possibility of using de.srcml.language as a
way to query anything necessary about the supported target languages).

> the property file could be a simple set of couple "keyword"<->"color"
> 
> e.g. for java :
> class = #BBFFFF,italic (a color code)
> public = #66FFAA,bold,Courier
> ...
> 
> e.g. for python
> class = #66FFAA,bold
> def = #FFFFFF,bold,arial
> self = #3345AA,,Courier
> ...
> 
> moreover you could assign a color to word that are not reserved but have 
> special convention (like 'self' in python or String in java etc.)

As you already pointed out below CSS is a good way too and CSS has two
major advantage over a property file as you described it:

1) a CSS coloration can be exchanged without the need of having to rerun
the generation of the XHTML file, whereas a property file would only be
available at generation time.

2) CSS is already available. No need for us to implement anything beyond
the very colors we want to see. A property file like you suggest would
take additional coding effort (not a lot of it, but still avoidable).

Also CSS can be customized very aggressively when taking attributes into
account. So if we blow up our generated XHTML enough we can just about
color everything in whatever color we want. Here's an example:

span.modifier { color:red; }
span.modifier[name=public] { color:blue; }
span.modifier[name=private] { color:grey; }

> these are only extemporaneous ideas, I don't know in deep the 
> architecture of srcML and probably they would be not applicable at all.
> So, excuseme for the stupidities that I have written :-)
> 
> PS: is srcML in some way related to this: 
> http://www.sdml.info/projects/srcml/       (SDML: srcML)

Note that our project goes by the name SrcML (with a capital S). Both
project names simply originate from the obvious name "source code markup
language" with "markup language" generally being abbreviated as ML (as
in XML, HTML and various other lesser-known markup languages). As we
didn't want to choose a name not including that it's about source code
and a markup language we just went for the obvious.

But other than the basic idea of an XML represenation of source code
there is not much SrcML and srcML have in common. The resulting XML
files are fundamentally different (mostly due to the srcML project
insisting on a 1:1 relation between the source code file and the XML
file, whereas SrcML only cares about the semantics). Also the srcML
project has a C++ parser only while we have a Java parser only (with
both projects being able to support more parser but lacking the
developers to write them ;) The supporting API and Platforms are unique
to SrcML too, as srcML only consists of a parser afaik.

-- 
Raiser, Frank
Student @ University of Ulm (www.uni-ulm.de)

Then anyone who leaves behind him a written manual, and likewise anyone who
receives it, in the belief that such writing will be clear and certain,
must be exceedingly simple-minded. (Plato)

Re: [SrcML] XHTML-ViewPlugin

[Icaro <ic...@gm...>]

Frank Raiser ha scritto:

>Ok and of course for everyone to fully understand the problem we have to
>add that the XHTML generation should be as language independent as
>possible.
>  
>

You could have a language independnt generator but a language specific 
color property file for each language.
the property file could be a simple set of couple "keyword"<->"color"

e.g. for java :
class = #BBFFFF,italic (a color code)
public = #66FFAA,bold,Courier
...

e.g. for python
class = #66FFAA,bold
def = #FFFFFF,bold,arial
self = #3345AA,,Courier
...

moreover you could assign a color to word that are not reserved but have 
special convention (like 'self' in python or String in java etc.)

>Therefore you cannot compare the String to "class" and color it
>accordingly, as a new language might use "cls" instead. You also have to
>be able to deal with a language which can legally have a class called
>"class" (and the coloration needs to reflect that it's the name of the
>class and not a keyword).
>  
>

the property file may be loaded depending on the kind of source and 
then  for all the token in the source file we search it in the small 
list of significant word. 
(If 'cls' or 'class' is in the list you color it with the specified 
color. )

>What about the name of the class being highlighted differently
>(identifier color or something)? Well we have the DOM node for the
>class, so we can get the name of this class and compare the output
>string to that in order to find out.
>  
>
This is not so simple to accomplish with property files (perhaps storing 
regular expression in the property file...) but they can make simple the 
customization of the result for different purposes:
i.e. you could also allow the specification of different combination of 
color-font for a same language that could be specified as command line 
paramiters.

for example:
dark_java_color.property
light_java_color.property
etc.

Another solution could be the use of CSS in combination with the 
generated HTML.

these are only extemporaneous ideas, I don't know in deep the 
architecture of srcML and probably they would be not applicable at all.
So, excuseme for the stupidities that I have written :-)

PS: is srcML in some way related to this: 
http://www.sdml.info/projects/srcml/       (SDML: srcML)

-- 
Domenico
http://icaro.blogspot.com

Re: [SrcML] XHTML-ViewPlugin

[Leif B. <lei...@ma...>]

> What about adding a boolean variable then for the different events
> specifying whether they should fire DoneEvents and an accessor  
> method to
> set this behavior?
Nice idea. So this variable could be in ViewEvent itself, with a get-  
and set-method. And the default value for a plugin is set in its  
constructor? In this case, you have to change thee default every time  
you create an event. Could we get this into the view platform, so  
that we could define the behaviours only when we create the view  
platform?


Greetings
Leif

Re: [SrcML] XHTML-ViewPlugin

[Frank R. <fra...@un...>]

On Fri, Nov 18, 2005 at 10:21:47AM +0100, Leif Bladt wrote:
> >What about extending your NextElementDoneEvent into a more generic
> >DoneEvent? Shouldn't require that much of a change.
> Done :) It's in the CVS although the Unittests don't compile (the  
> JavaHelper.java doesn't compile anymore).

Ops. I'll fix that when I'm back from uni tonight.

> >Only problem might be the performance when every single whitespace
> >results in a whole bunch of events being fired through the whole
> >machinery. But we can still add some event filtering later if it  
> >proves
> >to be a real problem and only send the events when requested.
> This is a real problem, so at the moment I will fire DoneEvents only  
> after StringEvents are processed. That's acceptable, but also  
> noticeable slower.

What about adding a boolean variable then for the different events
specifying whether they should fire DoneEvents and an accessor method to
set this behavior?

-- 
Raiser, Frank
Student @ University of Ulm (www.uni-ulm.de)

This report, by its very length, defends itself
against the risk of being read. (Winston Churchill)

Re: [SrcML] XHTML-ViewPlugin

[Leif B. <lei...@un...>]

> What about extending your NextElementDoneEvent into a more generic
> DoneEvent? Shouldn't require that much of a change.
Done :) It's in the CVS although the Unittests don't compile (the  
JavaHelper.java doesn't compile anymore).

> Only problem might be the performance when every single whitespace
> results in a whole bunch of events being fired through the whole
> machinery. But we can still add some event filtering later if it  
> proves
> to be a real problem and only send the events when requested.
This is a real problem, so at the moment I will fire DoneEvents only  
after StringEvents are processed. That's acceptable, but also  
noticeable slower.


Greetings
Leif

Re: [SrcML] XHTML-ViewPlugin

[Frank R. <fra...@un...>]

On Thu, Nov 17, 2005 at 11:50:09PM +0100, Leif Bladt wrote:
> Another thing are the new events which are thrown, when one of the  
> "old" events are completed. Should we implement a "Done" event for  
> each of our events? My suggestion is a single DoneEvent which  
> contains the event currently finished, so you can also see which  
> event is finished.

What about extending your NextElementDoneEvent into a more generic
DoneEvent? Shouldn't require that much of a change.
Only problem might be the performance when every single whitespace
results in a whole bunch of events being fired through the whole
machinery. But we can still add some event filtering later if it proves
to be a real problem and only send the events when requested.

-- 
Raiser, Frank
Student @ University of Ulm (www.uni-ulm.de)

You can be discouraged by failure, or you can learn from it. So go ahead and 
make mistakes, make all you can. Because, remember that's where you'll find
success - on the far side of failure. (Thomas J Watson Sr)

Re: [SrcML] XHTML-ViewPlugin

[Frank R. <fra...@un...>]

On Thu, Nov 17, 2005 at 12:25:23PM +0100, Leif Bladt wrote:
> for testing purpose I implemented a NextElementDone-Event, which is  
> called, after all Listeners and default-methods were called. So I can  
> react on the end of a tag. That's fine so far.

If it works without breaking the unit test for the ViewPlatform feel
free to commit it to CVS too.

> But if I would like to color only the "class"-string in the example,  
> I have no clue where to start with. Can I realise it with the  
> existing structure (e.g. the StringEvent) or do we have to extend the  
> event structure?
> 
> My test class:
> >public class Test {}
> 
> Its SrcML representation:
> ><?xml version="1.0" encoding="UTF-8"?>
> ><unit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
> >language="java" filename="Test.java" author="leif" xsi:noNam
> >espaceSchemaLocation="http://srcml.sourceforge.net/srcml.xsd">
> >
> >  <class name="Test">
> >    <modifiers>
> >      <modifier name="public"/>
> >    </modifiers>
> >    <block/>
> >  </class>
> ></unit>

Ok and of course for everyone to fully understand the problem we have to
add that the XHTML generation should be as language independent as
possible.

Therefore you cannot compare the String to "class" and color it
accordingly, as a new language might use "cls" instead. You also have to
be able to deal with a language which can legally have a class called
"class" (and the coloration needs to reflect that it's the name of the
class and not a keyword).

So what options do we have left?
Let's take a look at the information available at the time of the
generation of the XHTML code:

- we know the string s to print equals "class"
- we know the de.srcml.dom.Class instance it's related to

Now why do we want to color the "class"? Well because it is a keyword in
Java of course. So this is easily solved:
boolean de.srcml.language.Language.isKeyword(String)

What about the name of the class being highlighted differently
(identifier color or something)? Well we have the DOM node for the
class, so we can get the name of this class and compare the output
string to that in order to find out.

Also the coloration can be customized for other keywords depending on
the type of the related DOM node. For example a "public" modifier can be
colored differently than the "class" keyword although both return True
for isKeyword. The distinction can be made based on the DOM node's
class.

So right now I'm not seeing much of a problem here. However if you have
a good idea of how to improve the event mechanism to include this necessary
information just try and explain it. We could subclass the StringEvent
of course, but the question arises as to what subclasses to create. And
I don't know if having too many subclasses again doesn't raise the
complexity of the platform without any real need for that. If you
consider the extreme amount of possibilites to configure the XTHML
colors we seem to get a huge bunch of subclasses as each new color
option should be a subclass then.

-- 
Raiser, Frank
Student @ University of Ulm (www.uni-ulm.de)

"A mathematician is a blind man in a dark room looking for a black cat 
which isn't there."                -- Charles Darwin

[SrcML] XHTML-ViewPlugin

[Leif B. <lei...@un...>]

Hi,
for testing purpose I implemented a NextElementDone-Event, which is  
called, after all Listeners and default-methods were called. So I can  
react on the end of a tag. That's fine so far.

But if I would like to color only the "class"-string in the example,  
I have no clue where to start with. Can I realise it with the  
existing structure (e.g. the StringEvent) or do we have to extend the  
event structure?

My test class:
> public class Test {}

Its SrcML representation:
> <?xml version="1.0" encoding="UTF-8"?>
> <unit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
> language="java" filename="Test.java" author="leif" xsi:noNam
> espaceSchemaLocation="http://srcml.sourceforge.net/srcml.xsd">
>
>   <class name="Test">
>     <modifiers>
>       <modifier name="public"/>
>     </modifiers>
>     <block/>
>   </class>
> </unit>


Greetings
Leif