Code Intelligence Alternatives

Advanced fuzzing solution that combines guided fuzzing with symbolic execution, a patented technology from CMU. Mayhem is an advanced fuzz testing solution that dramatically reduces manual testing efforts with autonomous defect detection and validation. Deliver safe, secure, reliable software with less time, cost, and effort. Mayhem’s unique advantage is in its ability to acquire intelligence of its targets over time. As Mayhem’s knowledge grows, it deepens its analysis and maximizes its code coverage. All reported vulnerabilities are exploitable, confirmed risks. Mayhem guides remediation efforts with in-depth system level information, such as backtraces, memory logs, and register state, expediting issue diagnosis and fixes. Mayhem utilizes target feedback to custom generate test cases on the fly -- meaning no manual test case generation required. Mayhem offers access to all of its test cases to make regression testing effortless and continuous.

Go-fuzz is a coverage-guided fuzzing solution for testing Go packages. Fuzzing is mainly applicable to packages that parse complex inputs (both text and binary) and is especially useful for hardening systems that parse inputs from potentially malicious users (anything accepted over a network). go-fuzz has recently added preliminary support for fuzzing Go Modules. If you encounter a problem with modules, please file an issue with details. Data is a random input generated by go-fuzz, note that in most cases it is invalid. The function must return 1 if the fuzzer should increase the priority of the given input during subsequent fuzzing if the input must not be added to the corpus even if it gives new coverage, and 0 otherwise; other values are reserved for future use. The fuzz function must be in a package that go-fuzz can import. This means the code you want to test can't be in package main. Fuzzing internal packages is supported, however.

CI Fuzz ensures robust and secure code with test coverage up to 100%. Use CI Fuzz from the command line or in the IDE of choice to generate thousands of test cases automatically. CI Fuzz analyzes code as it runs, just like a unit test, but with AI support to efficiently cover all paths through the code. Uncover real bugs in real-time and say goodbye to theoretical issues and false positives. Find real issues with all the information needed to quickly reproduce and fix them. Test your code with maximum code coverage and automatically detect typical security-relevant bugs like injections and remote code executions automatically in one go. Get fully covered to deliver the highest quality software. Conduct real-time code analysis with CI Fuzz. Take unit tests to the next level. It employs AI for comprehensive code path coverage and the automatic generation of thousands of test cases. Maximize pipeline performance that doesn't compromise software integrity.

LibFuzzer is an in-process, coverage-guided, evolutionary fuzzing engine. LibFuzzer is linked with the library under test, and feeds fuzzed inputs to the library via a specific fuzzing entry point (or target function); the fuzzer then tracks which areas of the code are reached, and generates mutations on the corpus of input data in order to maximize the code coverage. The code coverage information for libFuzzer is provided by LLVM’s SanitizerCoverage instrumentation. LibFuzzer is still fully supported in that important bugs will get fixed. The first step in using libFuzzer on a library is to implement a fuzz target, a function that accepts an array of bytes and does something interesting with these bytes using the API under test. Note that this fuzz target does not depend on libFuzzer in any way so it is possible and even desirable to use it with other fuzzing engines like AFL and/or Radamsa.

Echidna is a Haskell program designed for fuzzing/property-based testing of Ethereum smart contracts. It uses sophisticated grammar-based fuzzing campaigns based on a contract ABI to falsify user-defined predicates or Solidity assertions. We designed Echidna with modularity in mind, so it can be easily extended to include new mutations or test specific contracts in specific cases. Generates inputs tailored to your actual code. Optional corpus collection, mutation and coverage guidance to find deeper bugs. Powered by Slither to extract useful information before the fuzzing campaign. Source code integration to identify which lines are covered after the fuzzing campaign. Interactive terminal UI, text-only or JSON output. Automatic test case minimization for quick triage. Seamless integration into the development workflow. Maximum gas usage reporting of the fuzzing campaign. Support for a complex contract initialization with Etheno and Truffle.

afl-unicorn lets you fuzz any piece of binary that can be emulated by Unicorn Engine. If you can emulate the code you’re interested in using the Unicorn Engine, you can fuzz it with afl-unicorn. Unicorn Mode works by implementing the block-edge instrumentation that AFL’s QEMU mode normally does into Unicorn Engine. Basically, AFL will use block coverage information from any emulated code snippet to drive its input generation. The whole idea revolves around the proper construction of a Unicorn-based test harness. The Unicorn-based test harness loads the target code, sets up the initial state, and loads in data mutated by AFL from disk. The test harness then emulates the target binary code, and if it detects that a crash or error occurred it throws a signal. AFL will do all its normal stuff, but it’s actually fuzzing the emulated target binary code. Only tested on Ubuntu 16.04 LTS, but it should work smoothly with any OS capable of running both AFL and Unicorn.

OSS-Fuzz offers continuous fuzzing for open source software. Fuzz testing is a well-known technique for uncovering programming errors in software. Many of these detectable errors, like buffer overflow, can have serious security implications. Google has found thousands of security vulnerabilities and stability bugs by deploying guided in-process fuzzing of Chrome components, and we now want to share that service with the open source community. OSS-Fuzz aims to make common open source software more secure and stable by combining modern fuzzing techniques with scalable, distributed execution. Projects that do not qualify for OSS-Fuzz can run their own instances of ClusterFuzz or ClusterFuzzLite. Currently, OSS-Fuzz supports C/C++, Rust, Go, Python, and Java/JVM code. Other languages supported by LLVM may work too. OSS-Fuzz supports fuzzing x86_64 and i386 builds.

Atheris is a coverage-guided Python fuzzing engine. It supports fuzzing of Python code, but also native extensions written for CPython. Atheris is based on libFuzzer. When fuzzing native code, Atheris can be used to catch extra bugs. Atheris supports Linux (32- and 64-bit) and Mac OS X, with Python versions 3.6-3.10. It comes with a built-in libFuzzer, which is fine for fuzzing Python code. If you plan to fuzz native extensions, you may need to build from source to ensure the libFuzzer version in Atheris matches your Clang version. Atheris relies on libFuzzer, which is distributed with Clang. Apple Clang doesn't come with libFuzzer, so you'll need to install a new version of LLVM. Atheris is based on a coverage-guided mutation-based fuzzer (LibFuzzer). This has the advantage of not requiring any grammar definition for generating inputs, making its setup easier. The disadvantage is that it will be harder for the fuzzer to generate inputs for code that parses complex data types.

Honggfuzz is a security-oriented software fuzzer. Supports evolutionary, feedback-driven fuzzing based on code coverage (SW and HW-based). It’s multi-process and multi-threaded, there’s no need to run multiple copies of your fuzzer, as Honggfuzz can unlock the potential of all your available CPU cores with a single running instance. The file corpus is automatically shared and improved between all fuzzed processes. It’s blazingly fast when the persistent fuzzing mode is used. A simple/empty LLVMFuzzerTestOneInput function can be tested with up to 1mo iteration per second on a relatively modern CPU. Has a solid track record of uncovered security bugs, the only (to date) vulnerability in OpenSSL with the critical score mark was discovered by Honggfuzz. As opposed to other fuzzers, it will discover and report hijacked/ignored signals from crashes (intercepted and potentially hidden by a fuzzed program).

BFuzz is an input-based fuzzer tool that takes HTML as an input, opens up your browser with a new instance, and passes multiple test cases generated by domato which is present in the recurve folder of BFuzz, more over BFuzz is an automation that performs the same task repeatedly and it doesn't mangle any test cases. Running BFuzz will ask for the option of whether to fuzz Chrome or Firefox, however, this will open Firefox from recurve and create the logs on the terminal. BFuzz is a small script that enables you to open the browser and run test cases. The test cases in recurve are generated by the domato generator and contain the main script. It contains additional helper code for DOM fuzzing.

American fuzzy lop is a security-oriented fuzzer that employs a novel type of compile-time instrumentation and genetic algorithms to automatically discover clean, interesting test cases that trigger new internal states in the targeted binary. This substantially improves the functional coverage for the fuzzed code. The compact synthesized corpora produced by the tool are also useful for seeding other, more labor or resource-intensive testing regimes down the road. Compared to other instrumented fuzzers, afl-fuzz is designed to be practical, it has a modest performance overhead, uses a variety of highly effective fuzzing strategies and effort minimization tricks, requires essentially no configuration, and seamlessly handles complex, real-world use cases, say, common image parsing or file compression libraries. It's an instrumentation-guided genetic fuzzer capable of synthesizing complex file semantics in a wide range of non-trivial targets.

Radamsa is a test case generator for robustness testing or fuzzer. It is typically used to test how well a program can withstand malformed and potentially malicious inputs. It works by reading sample files of valid data and generating interestingly different outputs from them. The main selling points of Radamsa are that it has already found a slew of bugs in programs that actually matter, it is easily scriptable, and, easy to get up and running. Fuzzing is one of the techniques to find unexpected behavior in programs. The idea is simply to subject the program to various kinds of inputs and see what happens. There are two parts to this process: getting the various kinds of inputs and how to see what happens. Radamsa is a solution to the first part, and the second part is typically a short shell script. Testers usually have a more or less vague idea of what should not happen, and they try to find out if this is so.

Awesome Fuzzing is a list of fuzzing resources including books, courses, both free and paid, videos, tools, tutorials, and vulnerable applications to practice in order to learn fuzzing and initial phases of exploit development like root cause analysis. Courses/training videos on fuzzing, videos talking about fuzzing techniques, tools, and best practices. Conference talks and tutorials, blogs, tools that help in fuzzing applications, and fuzzers that help in fuzzing applications that use network-based protocols like HTTP, SSH, SMTP, etc. Search and pick the exploits, that have respective apps available for download, and reproduce the exploit by using the fuzzer of your choice. Set of tests for fuzzing engines. Includes different well-known bugs. A corpus, including various file formats for fuzzing multiple targets in the fuzzing literature.

Defensics is a comprehensive, versatile, automated black box fuzzer that enables organizations to efficiently and effectively discover and remediate security weaknesses in software. Identify defects and zero-day vulnerabilities in services and protocols​. The generational fuzzer takes an intelligent, targeted approach to negative testing. Advanced file and protocol template fuzzers enable users to build their own test cases. The SDK allows expert users to use the Defensics framework to develop their own test cases. Defensics is a black box fuzzer, meaning it doesn’t require source code to run. With Defensics, users can secure their cyber supply chain to ensure the interoperability, robustness, quality, and security of software and devices before introducing them into IT or lab environments. Properly executed fuzzing techniques can provide a low-cost, efficient means of finding vulnerabilities, covering more code paths and value iterations than a manual analysis can perform.

Discover code weaknesses and certify the security strength of any product without access to source code. Test any protocol or hardware with beSTORM, even those used in IoT, process control, CANbus compatible automotive and aerospace. Realtime fuzzing, doesn’t need access to the source code, no cases to download. One platform, one GUI to learn, with over 250+ prebuilt protocol testing modules and the ability to add custom and proprietary ones. Find the security weaknesses before deployment that are most often discovered by external actors after release. Certify vendor components and your own applications in your own testing center. Self-learning software module and propriety software testing. Customization and scalability for any business sizes up or down. Automatically generate and deliver near-infinite attack vectors and document any product failures. Record every pass/fail and hand engineering the exact command that produced each fail.

Sulley is a fuzzing engine and fuzz testing framework consisting of multiple extensible components. Sulley (IMHO) exceeds the capabilities of most previously published fuzzing technologies, commercial and public domain. The goal of the framework is to simplify not only data representation but to simplify data transmission and instrumentation. A pure-Python fully automated and unattended fuzzing framework. Sulley not only has impressive data generation but has taken this a step further and includes many other important aspects a modern fuzzer should provide. Sulley watches the network and methodically maintains records. Sulley instruments and monitors the health of the target, capable of reverting to a known good state using multiple methods. Sulley detects, tracks, and categorizes detected faults. Sulley can fuzz in parallel, significantly increasing test speed. Sulley can automatically determine what unique sequence of test cases triggers faults.

Jazzer is a coverage-guided, in-process fuzzer for the JVM platform developed by Code Intelligence. It is based on libFuzzer and brings many of its instrumentation-powered mutation features to the JVM. You can use Docker to try out Jazzer's autofuzz mode, which automatically generates arguments to a given Java function and reports unexpected exceptions and detected security issues. You can also use GitHub release archives to run a standalone Jazzer binary that starts its own JVM configured for fuzzing.

Solidity Fuzzing Boilerplate is a template repository intended to ease fuzzing components of Solidity projects, especially libraries. Write tests once and run them with both Echidna and Foundry's fuzzing. Fuzz components that use incompatible Solidity versions by deploying those into a Ganache instance via Etheno. Use HEVM's FFI cheat code to generate complex fuzzing inputs or to compare outputs with non-EVM executables while doing differential fuzzing. Publish your fuzzing experiments without worrying about licensing by extending the shell script to download specific files. Turn off FFI if you don't intend to make use of shell commands from your Solidity contracts. Note that FFI is slow and should only be used as a workaround. It can be useful for testing against things that are difficult to implement within Solidity and already exist in other languages. Before executing tests of a project that has FFI enabled, be sure to check what commands are actually being executed.

Fuzz testing or fuzzing is a software testing technique, that basically consists in finding implementation bugs using malformed/semi-malformed data injection in an automated fashion. Let’s consider an integer in a program, which stores the result of a user’s choice between 3 questions. When the user picks one, the choice will be 0, 1, or 2, which makes three practical cases. Integers are stored as a static size variable. If the default switch case hasn’t been implemented securely, the program may crash and lead to “classical” security issues. Fuzzing is the art of automatic bug finding, and its role is to find software implementation faults and identify them if possible. A fuzzer is a program that automatically injects semi-random data into a program/stack and detects bugs. The data-generation part is made of generators, and vulnerability identification relies on debugging tools. Generators usually use combinations of static fuzzing vectors.

APIFuzzer reads your API description and step-by-step fuzzes the fields to validate if your application can cope with the fuzzed parameters, and it does not require coding. Parse API definition from a local file or remote URL. JSON and YAML file format support. All HTTP methods are supported. Fuzzing of the request body, query string, path parameter, and request header is supported. Relies on random mutations and supports CI integration. Generate JUnit XML test report format. Send a request to an alternative URL. Support HTTP basic auth from the configuration. Save the report of the failed test in JSON format into the pre-configured folder.

Balance your effort spent on writing applications and test code. Use the most sophisticated code coverage tool for Java and Groovy. OpenClover measures code coverage for Java and Groovy and collects over 20 code metrics. It not only shows you untested areas of your application but also combines coverage and metrics to find the riskiest code. The Test Optimization feature tracks which test cases are related to each class of your application code. Thanks to this OpenClover can run tests relevant to changes made in your application code, significantly reducing test execution time. Do testing getters and setters bring much value? Or machine-generated code? OpenClover outruns other tools in its flexibility to define the scope of coverage measurement. You can exclude packages, files, classes, methods, and even single statements. You can focus on testing important parts of your code. OpenClover not only records test results but also measures individual code coverage for every test.

ClusterFuzz is a scalable fuzzing infrastructure that finds security and stability issues in software. Google uses ClusterFuzz to fuzz all Google products and as the fuzzing backend for OSS-Fuzz. ClusterFuzz provides many features to seamlessly integrate fuzzing into a software project’s development process. Fully automatic bug filing, triage, and closing for various issue trackers. Supports multiple coverages guided fuzzing engines for optimal results (with ensemble fuzzing and fuzzing strategies). Statistics for analyzing fuzzer performance, and crash rates. Easy to use web interface for management and viewing crashes. Support for various authentication providers using Firebase. Support for black-box fuzzing, test case minimization, and regression finding through bisection.

ClusterFuzz is a scalable fuzzing infrastructure that finds security and stability issues in software. Google uses ClusterFuzz to fuzz all Google products and as the fuzzing backend for OSS-Fuzz. ClusterFuzz provides many features to seamlessly integrate fuzzing into a software project’s development process. Fully automatic bug filing, triage, and closing for various issue trackers. Supports multiple coverages guided fuzzing engines for optimal results (with ensemble fuzzing and fuzzing strategies). Statistics for analyzing fuzzer performance, and crash rates. Easy to use web interface for management and viewing crashes. Support for various authentication providers using Firebase. Support for black-box fuzzing, test case minimization, and regression finding through bisection.

syzkaller is an unsupervised coverage-guided kernel fuzzer. Supports FreeBSD, Fuchsia, gVisor, Linux, NetBSD, OpenBSD, and Windows. Initially, syzkaller was developed with Linux kernel fuzzing in mind, but now it's being extended to support other OS kernels as well. Once syzkaller detects a kernel crash in one of the VMs, it will automatically start the process of reproducing this crash. By default, it will use 4 VMs to reproduce the crash and then minimize the program that caused it. This may stop the fuzzing, since all of the VMs might be busy reproducing detected crashes. The process of reproducing one crash may take from a few minutes up to an hour depending on whether the crash is easily reproducible or non-reproducible at all.

Wapiti is a web application vulnerability scanner. Wapiti allows you to audit the security of your websites or web applications. It performs "black-box" scans (it does not study the source code) of the web application by crawling the webpages of the deployed web app, looking for scripts and forms where it can inject data. Once it gets the list of URLs, forms, and their inputs, Wapiti acts like a fuzzer, injecting payloads to see if a script is vulnerable. Search for potentially dangerous files on the server. Wapiti supports both GET and POST HTTP methods for attacks. It also supports multipart forms and can inject payloads in filenames (upload). Warnings are raised when an anomaly is found (for example 500 errors and timeouts). Wapiti is able to make the difference between permanent and reflected XSS vulnerabilities. Generates vulnerability reports in various formats (HTML, XML, JSON, TXT, CSV).

The Fuzzbuzz workflow is very similar to other CI/CD testing workflows. However, unlike other testing workflows, fuzz testing requires multiple jobs to run simultaneously, which results in a few extra steps. Fuzzbuzz is a fuzz testing platform. We make it trivial for developers to add fuzz tests to their code and run them in CI/CD, helping them catch critical bugs and vulnerabilities before they hit production. Fuzzbuzz completely integrates into your environment, following you from the terminal to CI/CD. Write a fuzz test in your environment and use your own IDE, terminal, or build tools. Push to CI/CD and Fuzzbuzz will automatically start running your fuzz tests against your latest code changes. Get notified when bugs are found through Slack, GitHub, or email. Catch regressions as new changes are automatically tested and compared to previous runs. Code is built and instrumented by Fuzzbuzz as soon as a change is detected.

FuzzDB was created to increase the likelihood of finding application security vulnerabilities through dynamic application security testing. It's the first and most comprehensive open dictionary of fault injection patterns, predictable resource locations, and regex for matching server responses. FuzzDB contains comprehensive lists of attack payload primitives for fault injection testing. These patterns, categorized by the attack and where appropriate platform type, are known to cause issues like OS command injection, directory listings, directory traversals, source exposure, file upload bypass, authentication bypass, XSS, HTTP header crlf injections, SQL injection, NoSQL injection, and more. For example, FuzzDB catalogs 56 patterns that can potentially be interpreted as a null byte and contains lists of commonly used methods and name-value pairs that trigger debug modes.

Fuzzing is a powerful strategy to find bugs in software. The idea is quite simple, which is to generate a large number of randomly malformed inputs for the software to parse and see what happens. If the program crashes then something is likely wrong. While fuzzing is a well-known strategy, it is surprisingly easy to find bugs, often with security implications, in widely used software. Memory access errors are the errors most likely to be exposed when fuzzing software that is written in C/C++. While they differ in the details, the core problem is often the same, the software reads or writes to the wrong memory locations. A modern Linux or BSD system ships a large number of basic tools that do some kind of file displaying and parsing. In their current state, most of these tools are not suitable for untrusted inputs. On the other hand, we have powerful tools these days that allow us to find and analyze these bugs.

OpenCppCoverage is an open-source code coverage tool for C++ under Windows. The main usage is for unit testing coverage, but you can also use it to know the executed lines in a program for debugging purposes. Support compiler with a program database file (.pdb). Just run your program with OpenCppCoverage, no need to recompile your application. Exclude a line based on a regular expression. Coverage aggregation, to run several code coverages and merge them into a single report. Requires Microsoft Visual Studio 2008 or higher for all editions including the Express edition. It should also work with the previous version of Visual Studio. You can run the tests with the Test Explorer window.

SimpleCov is a code coverage analysis tool for Ruby. It uses Ruby's built-in Coverage library to gather code coverage data, but makes processing its results much easier by providing a clean API to filter, group, merge, format, and display those results, giving you a complete code coverage suite that can be set up with just a couple lines of code. SimpleCov/Coverage track covered ruby code, gathering coverage for common templating solutions like erb, slim, and haml is not supported. In most cases, you'll want overall coverage results for your projects, including all types of tests, Cucumber features, etc. SimpleCov automatically takes care of this by caching and merging results when generating reports, so your report actually includes coverage across your test suites and thereby gives you a better picture of blank spots. SimpleCov must be running in the process that you want the code coverage analysis to happen on.

Try your first CodeRush feature right now and see instantly just how powerful it is. Refactoring for C#, Visual Basic, and XAML, with the fastest test .NET runner available, next generation debugging, and the most efficient coding experience on the planet. Quickly find symbols and files in your solution and easily navigate to code constructions related to the current context. CodeRush includes the Quick Navigation and Quick File Navigation features, which make it fast and easy to find symbols and open files. Using the Analyze Code Coverage feature, you can discover what parts of your solution are covered by unit tests, and find the at-risk parts of your application. The Code Coverage window shows percentage of statements covered by unit tests for each namespace, type, and member in your solution.

BMC Compuware Xpediter is a family of debuggers and interactive analysis tools for COBOL, Assembler, PL/I, and C programs that helps developers quickly understand applications, make changes, and fix problems in a secure environment—even if they’re unfamiliar with the source code. Xpediter enables developers to get into an interactive test session with minimal effort and quickly move applications into production with greater confidence. See line-by-line code execution and control all aspects of program execution and data. Use Code Coverage to see proof of execution and view metrics on multi-platform applications. Access Abend-AID diagnostic capabilities from within a debugging session. See a graphical view of source code through an integration with Topaz for Program Analysis. Leverage Topaz for Total Test for building a comprehensive portfolio of automated virtualized test cases. Intercept and debug mainframe transactions initiated remotely.

jscoverage tool, both node.js and JavaScript support. Enhance the coverage range. Use mocha to load the jscoverage module, then it works. jscoverage will append coverage info when you select list or spec or tap reporter in mocha. You can use covout to specify the reporter, like HTML, and detail. The detail reporter will print the uncovered code in the console directly. Mocha runs test case with jscoverage module. jscoverage will ignore files while listing in covignore file. jscoverage will output a report in HTML format. jscoverage will inject a group of functions into your module exports. default jscoverage will search covignore in the project root. jscoverage will copy exclude files from the source directory to the destination directory.

On-the-fly error detection and suggestions for fixes, quick and safe refactorings with one-step undo, intelligent code completion, dead code detection, and documentation hints help all Go developers, from newbies to experienced professionals, to create fast, efficient, and reliable code. Exploring and understanding team, legacy, or foreign projects takes a lot of time and effort. GoLand code navigation helps you get around with instant switching to shadowed methods, implementations, usages, declarations, or interfaces implemented by types. Jump between types, files or any other symbols, or find their usages and examine them with convenient grouping by usage type. Powerful built-in tools help to run and debug your applications. You can write and debug tests without any additional plugins or configuration effort, and test your applications right in the IDE. A built-in Code Coverage tool will make sure that your tests don’t miss anything important.

Xdebug is an extension for PHP, and provides a range of features to improve the PHP development experience. A way to step through your code in your IDE or editor while the script is executing. An improved var_dump() function, stack traces for notices, warnings, errors, and exceptions to highlight the code path to the error. Writes every function call, with arguments and invocation location to disk. Optionally also includes every variable assignment and return value for each function. Allows you, with the help of visualization tools, to analyze the performance of your PHP application and find bottlenecks. Shows which parts of your code base are executed when running unit tests with PHPUnit. Installing Xdebug with a package manager is often the fastest way. You can substitute the PHP version with the one that matches the PHP version that you are running. You can install Xdebug through PECL on Linux & macOS with Homebrew.

Thousands of autonomously generated tests run every minute to pinpoint vulnerabilities and guide rapid remediation. Mayhem takes the guesswork out of untested code by autonomously generating test suites that produce actionable results. No need to recompile the code, since Mayhem works with dockerized images. Self-learning ML continually runs thousands of tests per second probing for crashes and defects, so developers can focus on features. Continuous testing runs in the background to surface new defects and increase code coverage. Mayhem delivers a copy/paste reproduction and backtrace for every defect, then prioritizes them based on your risk. See all the results, duplicated and prioritized by what you need to fix now. Mayhem fits into your existing build pipeline and development tools, putting actionable results at your developers' fingertips. No matter what language or tools your team uses.

Defensics is a comprehensive, versatile, automated black box fuzzer that enables organizations to efficiently and effectively discover and remediate security weaknesses in software. The generational fuzzer takes an intelligent, targeted approach to negative testing. Advanced file and protocol template fuzzers enable users to build their own test cases. The SDK allows expert users to use the Defensics framework to develop their own test cases. Defensics is a black box fuzzer, meaning it doesn’t require source code to run. With Defensics, users can secure their cyber supply chain to ensure the interoperability, robustness, quality, and security of software and devices before introducing them into IT or lab environments. Defensics fits nearly any development workflow, whether in a traditional SDL or CI environment. Its API and data export capabilities also enable it to integrate with surrounding technologies, making it a true plug-and-play fuzzer.

Peach is a SmartFuzzer that is capable of performing both generation and mutation-based fuzzing. Peach requires the creation of Peach Pit files that define the structure, type information, and relationships in the data to be fuzzed. It additionally allows for the configuration of a fuzzing run including selecting a data transport (publisher), logging interface, etc. Peach has been under active development since 2004 and is in its third major version. Fuzzing continues to be the fastest way to find security issues and test for bugs. Effective hardware fuzzing with Peach will introduce students to the fundamentals of device fuzzing. Peach was designed to fuzz any type of data consumer from servers to embedded devices. Researchers, corporations, and governments already use Peach to find vulnerabilities in hardware. This course will focus on using Peach to target embedded devices and collect information from the device in the event of a crash.

BullseyeCoverage is an advanced C++ code coverage tool used to improve the quality of software in vital systems such as enterprise applications, industrial control, medical, automotive, communications, aerospace and defense. The function coverage metric gives you a quick overview of testing completeness and indicates areas with no coverage at all. Use this metric to broadly raise coverage across all areas of your project. Condition/decision coverage provides detail at the control structure level. Use this metric to attain high coverage in specific areas, for example during unit testing. C/D coverage provides better detail than statement coverage or branch coverage, and provides much better productivity than more complex coverage metrics.

The JCov open-source project is used to gather quality metrics associated with the production of test suites. JCov is being opened in order to facilitate the practice of verifying test execution of regression tests in OpenJDK development. The main motivation behind JCov is the transparency of test coverage metrics. The advantage to promoting standard coverage based on JCov is that OpenJDK developers will be able to use a code coverage tool that stays in the 'lock step' with Java language and VM developments. JCov is a pure java implementation of a code coverage tool that provides a means to measure and analyze dynamic code coverage of Java programs. JCov provides functionality to collect method, linear block, and branch coverage, as well as show uncovered execution paths. It is also able to show a program's source code annotated with coverage information. From a testing perspective, JCov is most useful to determine execution paths.

Coco® is a multi-language code coverage tool. Automatic source code instrumentation is used to measure test coverage of statements, branches and conditions. Executing a test suite against an instrumented application produces data that can later be analyzed. This analysis can be used to understand how much of the source code has been hit by tests, which additional tests need to be written, how the test coverage changed over time and more. Identify redundant tests, untested or dead code. Identify the impact of a patch on the code and code coverage & your testing. Coco supports statement coverage, branch coverage, MC/DC and other levels. Linux, Windows, RTOS and others. Using GCC, Visual Studio, embedded compilers and more. Choice of different report formats (text, HTML, XML, JUnit, Cobertura). Coco can also be integrated with various build, test and CI frameworks like JUnit, Jenkins and SonarQube.

NCover Desktop is a Windows application that helps you collect code coverage statistics for .NET applications and services. After coverage is collected, Desktop displays charts and coverage metrics in a browser-based GUI that allows you to drill all the way down to your individual lines of source code. Desktop also allows you the option to install a Visual Studio extension called Bolt. Bolt offers built-in code coverage that displays unit test results, timings, branch visualization and source code highlighting right in the Visual Studio IDE. NCover Desktop is a major leap forward in the ease and flexibility of code coverage tools. Code coverage, gathered while testing your .NET code, shows the NCover user what code was exercised during the test and gives a specific measurement of unit test coverage. By tracking these statistics over time, you gain a concrete measurement of code quality during the development cycle.

Testwell CTC++ is a powerful instrumentation-based code coverage and dynamic analysis tool for C and C++ code. With certain add-on components CTC++ can be used also on C#, Java and Objective-C code. Further, again with certain add-on components, CTC++ can be used to analyse code basically at any embedded target machines, also in very small ones (limited memory, no operating system). CTC++ provides Line Coverage, Statement Coverage, Function Coverage, Decision Coverage, Multicondition Coverage, Modified Condition/Decision Coverage (MC/DC), Condition Coverage. As a dynamic analysis tool, CTC++ shows the execution counters (how many times executed) in the code, i.e. more than a plain executed/not executed information. You can also use CTC++ to measure function execution costs (normally time) and to enable function entry/exit tracing at test time. CTC++ is easy to use.

It works with .NET Framework on Windows and .NET Core on all supported platforms. Coverlet supports coverage for deterministic builds. The solution at the moment is not optimal and need a workaround. If you want to visualize coverlet output inside Visual Studio while you code, you can use the following addins depending on your platform. Coverlet also integrates with the build system to run code coverage after tests. Enabling code coverage is as simple as setting the CollectCoverage property to true. The coverlet tool is invoked by specifying the path to the assembly that contains the unit tests. You also need to specify the test runner and the arguments to pass to the test runner using the --target and --targetargs options respectively. The invocation of the test runner with the supplied arguments must not involve a recompilation of the unit test assembly or no coverage result will be generated.

ToothPicker is an in-process, coverage-guided fuzzer for iOS. It was developed to specifically target iOS's Bluetooth daemon and to analyze various Bluetooth protocols on iOS. As it is built using FRIDA, it can be adapted to target any platform that runs FRIDA. This repository also includes an over-the-air fuzzer with an exemplary implementation to fuzz Apple's MagicPairing protocol using InternalBlue. Additionally, it contains the ReplayCrashFile script that can be used to verify crashes the in-process fuzzer has found. This is a very simple fuzzer that only flips bits and bytes of inactive connections. No coverage, no injection, but nice as a demo and stateful. Runs just with Python and Frida, no modules or installation are required. ToothPicker is built on the codebase of frizzer. It is recommended to set up a virtual Python environment for frizzer. Starting from the iPhone XR/Xs, PAC has been introduced.

Automating the creation and deployment of component test harnesses, test stubs and test drivers is a cinch thanks to OneTest Embedded. With a single click from any development environment, one can profile memory and performance, analyze code coverage and visualize program execution behavior. Additionally, OneTest Embedded helps be more proactive in debugging, while identifying and assisting in fixing code before it breaks. Allows for a virtual cycle of test generation, while executing, reviewing and testing improvement to rapidly achieve full test coverage. One click is all it takes to build, execute on the target, and generate reports. Helps preempt performance issues and program crashes. Additionally, can be adapted to work with custom memory management methods used in embedded software. Provides visibility on thread execution and switching to develop a deep understanding of the behavior of the system under test.

dotCover is a .NET unit testing and code coverage tool that works right in Visual Studio and in JetBrains Rider, helps you know to what extent your code is covered with unit tests, provides great ways to visualize code coverage, and is Continuous Integration ready. dotCover calculates and reports statement-level code coverage in applications targeting .NET Framework, .NET Core, Mono for Unity, etc. dotCover is a plug-in to Visual Studio and JetBrains Rider, giving you the advantage of analyzing and visualizing code coverage without leaving the code editor. This includes running unit tests and analyzing coverage results right in the IDEs, as well as support for different color themes, new icons and menus. dotCover comes bundled with a unit test runner that it shares with another JetBrains tool for .NET developers, ReSharper. dotCover supports continuous testing, a modern unit testing workflow whereby dotCover figures out on-the-fly which unit tests are affected by your code changes.

A self-contained CodeCoverage compatible driver for PHP. When PCOV is left unset, PCOV will attempt to find src, lib or, app in the current working directory, in that order; If none are found the current directory will be used, which may waste resources storing coverage information for the test suite. If PCOV contains test code, it's recommended to set the exclude command to avoid wasting resources. To avoid unnecessary allocation of additional arenas for traces and control flow graphs, PCOV should be set according to the memory required by the test suite. To avoid reallocation of tables, PCOV should be set to a number higher than the number of files that will be loaded during testing, inclusive of test files. interoperability with Xdebug is not possible. At an internal level, the executor function is overridden by PCOV, so any extension or SAPI which does the same will be broken. PCOV is zero cost, code runs at full speed.

Save time and money by finding and fixing defects earlier. Reduce the effort and cost of delivering high-quality software by preventing more complicated and expensive problems down the line. Ensure your C# or VB.NET code complies with a wide range of safety and security industry standards, including the requirement traceability mandated and the documentation required to verify compliance. Parasoft's C# testing tool, Parasoft dotTEST, automates a broad range of software quality practices for your C# and VB.NET development activities. Deep code analysis uncovers reliability and security issues. Code coverage, requirements traceability, and automated compliance reporting helps achieve compliance for security standards and safety-critical industries.

Meet Agile development cycles while maintaining high-quality code. Use Jtest’s comprehensive set of Java testing tools to ensure defect-free coding through every stage of software development in the Java environment. Streamline Compliance With Security Standards. Ensure your Java code complies with industry security standards. Have compliance verification documentation automatically generated. Release Quality Software, Faster. Integrate Java testing tools to find defects faster and earlier. Save time and money by mitigating complicated and expensive problems down the line. Increase Your Return From Unit Testing. Achieve code coverage targets by creating a maintainable and optimized suite of JUnit tests. Get faster feedback from CI and within your IDE using smart test execution. Parasoft Jtest integrates tightly into your development ecosystem and CI/CD pipeline for real-time, intelligent feedback on your testing and compliance progress.