Work at SourceForge, help us to make it a better place! We have an immediate need for a Support Technician in our San Francisco or Denver office.

Close

#691 Unable to use the fileExtensions property

release_5.0
closed
nobody
1
2013-10-26
2012-12-13
Flavien Huynh
No

Hi,

I need to tell CheckStyle to avoid parsing files other than .java.
My source files directory can contain big non-code files.
If these files get too large, CheckStyle fails with an "OutOfMemory" error.

I've been trying to use the fileExtensions property in the TreeWalker module, without any success.
Even if I set the fileExtensions to "java" (which is supposed to be the default value anyway), then all files are parsed.

There already is a post concerning this issue, unfortunately with no answers yet
http://checkstyle.2069334.n4.nabble.com/no-subject-td4654102.html

To reproduce, just add a big non-java file (a 200MB exe file showed the error in my case) to the directory parsed by CheckStyle, and the following stack trace is produced:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Unknown Source)
at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
at java.lang.AbstractStringBuilder.append(Unknown Source)
at java.lang.StringBuilder.append(Unknown Source)
at com.puppycrawl.tools.checkstyle.api.FileText.<init>(FileText.java:139)
at com.puppycrawl.tools.checkstyle.Checker.process(Checker.java:259)
at com.puppycrawl.tools.checkstyle.Main.main(Main.java:121)

Related

Bugs: #691

Discussion

  • Bewied
    Bewied
    2013-04-09

    I doubt this is an error in checkstyle.
    Either way, you might be interested in the SuppressionsFilter:

    <module name="SuppressionFilter">
        <property name="file" value="/path/to/your/suppressions.xml"/>
    </module>
    

    The contents of my suppressions.xml:

    <?xml version="1.0"?>
    <!DOCTYPE suppressions PUBLIC
        "-//Puppy Crawl//DTD Suppressions 1.1//EN"
        "http://www.puppycrawl.com/dtds/suppressions_1_1.dtd">
    <suppressions>
        <suppress files=".*\.png" checks="[a-zA-Z0-9]*"/>
    </suppressions>
    

    If you want checkstyle to absolutely and only ever look at *.java files, this should do what you want (again, suppressions.xml):

    <?xml version="1.0"?>
    <!DOCTYPE suppressions PUBLIC
        "-//Puppy Crawl//DTD Suppressions 1.1//EN"
        "http://www.puppycrawl.com/dtds/suppressions_1_1.dtd">
    <suppressions>
        <suppress files="(.{0,4}$)|([^.]java$)|(\.[^j]ava$)|(\.j[^a]va$)|(\.ja[^v]a$)|(\.jav[^a])$"
            checks="[a-zA-Z0-9]*"/>
    </suppressions>
    

    I admit I'm not too regex savvy, and this is only a brute-force approach of the phrase "everything that doesn't end in .java".

    If this regex flavor (at least included in PCRE) is supported by Checkstyle, then the following regex should work, too:

    <suppress files="(?!\.java)(.*)$" checks="[a-zA-Z0-9]*"/>
    

    Again, this does not sound like a bug in Checkstyle, Suppressing files this way works perfectly fine for me.

     
    • Flavien Huynh
      Flavien Huynh
      2013-04-10

      Hi, and thank you for the answer.

      I did try to use the "suppressions" option, but the result is still the same, unfortunately.

      What I did is use a suppressions.xml file, which suppresses every file not having the .java extension, and launch a checkstyle analysis in various situations:

      1. The project only contains java files
        All works fine.

      2. I added to the project one java files which extension was renamed as ".java2"
        All works fine, and the checkstyle result file contains an empty section for that file
        <file name="file.java2">
        </file>
        Note that if I remove the suppressions file, the ".java2" file section is no longer empty in the checkstyle result file

      3. I added to the project one small 100KB ".exe" file
        All works fine, I now have an additional empty section in the checkstyle result file, associated to the exe file

      4. Now I added to the project a big 200MB ".exe" file
        This time checkstyle ends with an error
        Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

      It seems the suppression works correctly (suppressed files are not analyzed by checkstyle), but they are still processed in some way.
      And if suppressed files are big enough, then memory becomes an issue

      It seems some people have been having a somewhat similar problem, and had to wrap checkstyle so that only Java files are provided to checkstyle

       
  • Oliver Burn
    Oliver Burn
    2013-04-10

    Adding a file to the suppressions file does not prevent Checkstyle from
    still attempting to parse it. So, stop following this train of thought.

    You need to configure Checkstyle to only parse files that end with ".java".
    How are you running Checkstyle? With the ANT Task and the Command Line,
    this is easy to setup.

    Regards,
    Oliver

    On Wed, Apr 10, 2013 at 8:24 PM, Flavien Huynh flavien-sf@users.sf.netwrote:

    Hi, and thank you for the answer.

    I did try to use the "suppressions" option, but the result is still the
    same, unfortunately.

    What I did is use a suppressions.xml file, which suppresses every file not
    having the .java extension, and launch a checkstyle analysis in various
    situations:

    1.

    The project only contains java files
    All works fine.
    2.

    I added to the project one java files which extension was renamed as
    ".java2"
    All works fine, and the checkstyle result file contains an empty
    section for that file

    Note that if I remove the suppressions file, the ".java2" file section
    is no longer empty in the checkstyle result file
    3.

    I added to the project one small 100KB ".exe" file
    All works fine, I now have an additional empty section in the
    checkstyle result file, associated to the exe file
    4.

    Now I added to the project a big 200MB ".exe" file
    This time checkstyle ends with an error
    Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

    It seems the suppression works correctly (suppressed files are not
    analyzed by checkstyle), but they are still processed in some way.
    And if suppressed files are big enough, then memory becomes an issue

    It seems some peoplehttp://stackoverflow.com/questions/11365125/checkstyle-suppressions-what-am-i-doing-wronghave been having a somewhat similar problem, and had to wrap checkstyle so
    that only Java files are provided to checkstyle


    Status: open
    Labels: fileExtensions
    Created: Thu Dec 13, 2012 10:54 AM UTC by Flavien Huynh
    Last Updated: Tue Apr 09, 2013 11:29 PM UTC
    Owner: nobody

    Hi,

    I need to tell CheckStyle to avoid parsing files other than .java.
    My source files directory can contain big non-code files.
    If these files get too large, CheckStyle fails with an "OutOfMemory" error.

    I've been trying to use the fileExtensions property in the TreeWalker
    module, without any success.
    Even if I set the fileExtensions to "java" (which is supposed to be the
    default value anyway), then all files are parsed.

    There already is a post concerning this issue, unfortunately with no
    answers yet
    http://checkstyle.2069334.n4.nabble.com/no-subject-td4654102.html

    To reproduce, just add a big non-java file (a 200MB exe file showed the
    error in my case) to the directory parsed by CheckStyle, and the following
    stack trace is produced:

    Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Unknown Source)
    at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
    at java.lang.AbstractStringBuilder.append(Unknown Source)
    at java.lang.StringBuilder.append(Unknown Source)
    at com.puppycrawl.tools.checkstyle.api.FileText.(FileText.java:139)
    at com.puppycrawl.tools.checkstyle.Checker.process(Checker.java:259)
    at com.puppycrawl.tools.checkstyle.Main.main(Main.java:121)


    Sent from sourceforge.net because you indicated interest in
    https://sourceforge.net/p/checkstyle/bugs/691/

    To unsubscribe from further messages, please visit
    https://sourceforge.net/auth/subscriptions/

     

    Related

    Bugs: #691

  • Flavien Huynh
    Flavien Huynh
    2013-04-10

    Hi,

    Adding a file to the suppressions file does not prevent Checkstyle from
    still attempting to parse it ...

    Thanks Oliver for the confirmation, the tests showed indeed that "suppression" does not mean that files are not parsed.

    As for limiting the analysis to .java files, I can see how this can be achieved using the Ant task, since it accepts filesets.

    In my environment though, CS is launched using the command line (on either Windows or Unix systems), and I didn't see a file extension parameter, nor how to achieve a "fileset" equivalent in the CS command line documentation.

    I feel I'm missing something obvious ...

    Do I have to implement a custom check, as described in the documentation?

    Regards
    Flavien

     
  • Roman Ivanov
    Roman Ivanov
    2013-10-26

    • status: open --> closed