◐ Shell
reader mode source ↗
Skip to content

Add SARIF output support.#4651

Closed
mario-campos wants to merge 46 commits into
cppcheck-opensource:mainfrom
mario-campos:mario-campos/sarif
Closed

Add SARIF output support.#4651
mario-campos wants to merge 46 commits into
cppcheck-opensource:mainfrom
mario-campos:mario-campos/sarif

Conversation

@mario-campos

@mario-campos mario-campos commented Dec 16, 2022

Copy link
Copy Markdown
Contributor

This has been on my TODO list for a while! I've been wanting to integrate CppCheck's results into GitHub since Code Scanning came out. To do that, however, the tool needs to spit out SARIF.

To test it out, run cppcheck --output-format=sarif --quiet foo.c

Now, this is only an initial stab, and it's probably a bit ugly. I'm not quite done just yet (hence why it's a draft PR). For one, it's not truly outputting the correct line/column numbers. So, if you have any suggestions or comments, please let me know.

Also, I initially tried to work SARIF output into the original design by adding a ErrorMessage::toSARIF method (ala ErrorMessage::toXML), but I quickly realized that this wouldn't work very well, since the SARIF document has to be "constructed", not "streamed" the way the XML document is. Which meant that I had to rework the way that findings ("errors") were collected and aggregated, since the SARIF document kinda needs to see everything all at once.

@mario-campos mario-campos force-pushed the mario-campos/sarif branch 2 times, most recently from 2e352ed to 3395a1b Compare December 17, 2022 18:39
@firewave

Copy link
Copy Markdown
Collaborator

Thanks for your contribution.

Without actually looking at what it does - please add some unit and/or system tests for it.

@danmar

danmar commented Dec 18, 2022

Copy link
Copy Markdown
Collaborator

I haven't looked into your implementation but I really like this. 👍

@mario-campos mario-campos marked this pull request as ready for review December 19, 2022 22:10
@mario-campos

Copy link
Copy Markdown
Contributor Author

@firewave, any idea how to include certain .cpp files into the Visual Studio build? I don't have any experience with Windows development.

cppcheckexecutor.obj : error LNK2019: unresolved external symbol "public: __cdecl XMLAnalysisReport::XMLAnalysisReport(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const &)" (??0XMLAnalysisReport@@QEAA@AEBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@Z) referenced in function "protected: bool __cdecl CppCheckExecutor::parseFromArgs(class CppCheck *,int,char const * const * const)" (?parseFromArgs@CppCheckExecutor@@IEAA_NPEAVCppCheck@@HQEBQEBD@Z) [D:\a\cppcheck\cppcheck\test\testrunner.vcxproj]

Currently, for XML output, the XML generation is tightly-coupled with the `ErrorMessage` class, which makes it harder to implement other formats (e.g. SARIF) that may need to see all of the errors/findings at once.

Furthermore, implementing the serialization in the `ErrorMessage` class as individual methods (e.g. `toXML`) does not scale very well when using multiple formats. This approach uses the `AnalysisReport` abstract class to sub-class the different serialization formats.
This treats the CLI output as just another format (default) for the findings/report.
This should look and feel like the rest of the CppCheck code.
This is a good start, but it still needs some work. In particular, I need to fix the startLine/endLine/startColumn/endColumn part. Right now, I am using placeholder values ("1") instead of the real line/column numbers.
This matches the other AnalysisReport classes, and it's what clang-tidy recommends.
SARIF's precision property maps to Cppcheck's certainty, which is essentially the confidence level.
53 hidden items Load more…
@mario-campos mario-campos requested review from danmar and firewave and removed request for firewave December 24, 2022 18:06
@firewave

Copy link
Copy Markdown
Collaborator

Sorry for the late reply but I currently out sick. So still no code review...

@firewave, any idea how to include certain .cpp files into the Visual Studio build? I don't have any experience with Windows development.

cppcheckexecutor.obj : error LNK2019: unresolved external symbol "public: __cdecl XMLAnalysisReport::XMLAnalysisReport(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const &)" (??0XMLAnalysisReport@@QEAA@AEBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@Z) referenced in function "protected: bool __cdecl CppCheckExecutor::parseFromArgs(class CppCheck *,int,char const * const * const)" (?parseFromArgs@CppCheckExecutor@@IEAA_NPEAVCppCheck@@HQEBQEBD@Z) [D:\a\cppcheck\cppcheck\test\testrunner.vcxproj]

If #4652 is merged (could happen any day), dmake will take care of that. Until then you need to do it manually in the *.vcxproj files.

@firewave

Copy link
Copy Markdown
Collaborator

Since GitHub actions apparently support SARIF output could also also adjust the selfcheck workflow to use SARIF output instead of relying on the exitcode?

Also if SARIF is used should the exitcode be forced to 0? I have no idea how other tools implement this.

@firewave firewave left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

Also run dmake to actually update the Makefile.

@mario-campos

mario-campos commented Jan 10, 2023

Copy link
Copy Markdown
Contributor Author

@firewave, unfortunately dmake does not create the Makefile target with the picojson include -isystem externals/picojson:

cli/sarifanalysisreport.o: cli/sarifanalysisreport.cpp cli/analysisreport.h cli/sarifanalysisreport.h externals/picojson/picojson.h lib/color.h lib/config.h lib/errorlogger.h lib/errortypes.h lib/suppressions.h
	$(CXX) ${INCLUDE_FOR_CLI} $(CPPFLAGS) $(CXXFLAGS) -c -o $@ cli/sarifanalysisreport.cpp

See https://github.com/mario-campos/cppcheck/actions/runs/3887481113/jobs/6633822871. Which means that, without tampering with the Makefile, dmake will generate a Makefile that fails to build cppcheck. Any suggestions?

@danmar

danmar commented Jan 18, 2023

Copy link
Copy Markdown
Collaborator

I think this looks really interesting. There are some refactorings I would like to see.. but maybe instead of writing lots of nits it's quicker to just take over the finishing touches..

@danmar

danmar commented Jan 18, 2023

Copy link
Copy Markdown
Collaborator

unfortunately dmake does not create the Makefile target with the picojson include -isystem externals/picojson:

I have the feeling that you can adjust dmake.cpp here:
https://github.com/danmar/cppcheck/blob/main/tools/dmake.cpp#L73

I guess it should say something like:

    else if (filename.compare(0, 4, "cli/") == 0) {
        getDeps("lib" + filename.substr(filename.find('/')), depfiles);
        for (const std::string & external : externalfolders)
            getDeps(external + filename.substr(filename.find('/')), depfiles);
    }

@danmar

danmar commented Jan 21, 2023

Copy link
Copy Markdown
Collaborator

unfortunately dmake does not create the Makefile target with the picojson include -isystem externals/picojson:

I have made some fixes in this branch:

https://github.com/danmar/cppcheck/tree/4651

The fixes I made are here: https://github.com/danmar/cppcheck/commit/acca98d8f33e8f1f4bb5e06c4fb88277970830a2.diff
Feel free to apply those fixes locally. In particular I think the changes in dmake.cpp will help you.

There are still some tests that fail. You can run those tests with these commands:

 cd cppcheck/test/cli
 python3 -m pytest test-helloworld.py

If the tests are fixed I have the feeling that we can merge this PR.

@mario-campos

Copy link
Copy Markdown
Contributor Author

I think this looks really interesting. There are some refactorings I would like to see.. but maybe instead of writing lots of nits it's quicker to just take over the finishing touches..

That would be helpful. Unfortunately, now that the holiday season is over, I don't have as much time for this PR as I did before. But, I will still continue to address the remaining problems as much as I can.

@firewave

Copy link
Copy Markdown
Collaborator

FYI there's already an PR which adds that output format command-line switch: #3365.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants