Analysis on Cppcheck source code with SmPL

[Markus Elfring]

The source files of this evolving software can be analysed also by the means of the semantic patch language to some degree.

The following script can show places where match expressions are passed so that they will eventually be reconsidered.

@display@
constant text;
@@
*Token::Match(..., text, ...)

Another analysis approach can import some information into a database.

@initialize:python@
@@
import sqlalchemy, sys
sys.stderr.write("\n".join(["Using SQLAlchemy version:",
                            sqlalchemy.__version__]))
sys.stderr.write("\n")
from sqlalchemy import create_engine, Column, Integer, String
engine = create_engine("sqlite:///:memory:", echo = False)

from sqlalchemy.ext.declarative import declarative_base
base = declarative_base()

class position(base):
   __tablename__ = "positions"
   source_file = Column(String, primary_key = True)
   line = Column(Integer, primary_key = True)
   column = Column(Integer, primary_key = True)
   pattern = Column(String)

from sqlalchemy.orm import sessionmaker
configured_session = sessionmaker(bind = engine)
session = configured_session()
base.metadata.create_all(engine)

def store_positions(text, places):
    """Add source code positions to an internal table."""
    for place in places:
       entry = position(source_file = place.file,
                        line = place.line,
                        column = int(place.column) + 1,
                        pattern = text
                       )
       session.add(entry)

@filter@
constant text;
position pos;
@@
 Token::Match(..., text@pos, ...)

@script:python collection@
text << filter.text;
places << filter.pos;
@@
store_positions(text, places)

@finalize:python@
@@
session.commit()
from sqlalchemy import func
entries = session.query(func.count("*")).select_from(position).scalar()

if entries > 0:
   delimiter = "|"
   sys.stdout.write(delimiter.join(["pattern", "incidence"]))
   sys.stdout.write("\r\n")

   for pattern, \
       count in session.query(position.pattern, func.count("*")) \
                             .group_by(position.pattern).all():
      sys.stdout.write(delimiter.join([pattern, str(count)]))
      sys.stdout.write("\r\n")
else:
   sys.stderr.write("No result for this analysis!\n")

So it can be determined that 1127 patterns are passed to a function for evaluation so far.

Do you get any ideas for further software evolution around such data processing possibilities?

[Markus Elfring]

I got into the mood to repeat this analysis approach for the tool version “2.9.3”.

Thus I determined by my SmPL script “list_often_used_match_expressions.cocci” that 197 patterns were passed more than once to the member function “Token::Match”.
🔮 Will such incidence statistics trigger further software evolution?

[Daniel Marjamäki]

I do not see what value it has. It's like count number of character 'x' in the code... I do not see any use for that information.

[Markus Elfring]

This approach shows data processing possibilities which are supported by specific software development tools.
It demonstrates another try to extract useful facts from changing source files.

• Would you dare to try such scripts for the semantic patch language out any more?
• Do you occasionally work with data mining tools?
• How do you think about any further adjustments around the presented variety of search patterns?

[Daniel Marjamäki]

I am not against smpl.

[Markus Elfring]

• Will the semantic patch language (Coccinelle software) be used by any more developers (besides me here) for the concrete clarification of implementation details?
• Would you find it interesting to determine where a Cppcheck search pattern is applied more than once?
• How much do variations matter in these search patterns?