This is a small project for fuzzing antlr v4 with the jazzer fuzzing tool.

See Fuzzing for a general description of the theory behind fuzzy testing.

Because Java uses a runtime environment which does not crash on invalid actions of an application (unless native code is invoked), Fuzzing of Java-based applications focuses on the following:

verify if only expected exceptions are thrown
verify any JNI or native code calls
find cases of unbounded memory allocations

antlr does not use JNI or native code, therefore the fuzzing target mainly tries to trigger unexpected exceptions and unbounded memory allocations.

How to fuzz

Build the fuzzing target:

./gradlew shadowJar

Prepare a corpus of test-files (i.e. valid and invalid grammars) and put them into directory corpus

E.g. to fetch the large set of sample grammar-files from the grammars-v4 repository, you can do the following:

git clone https://github.com/antlr/grammars-v4 corpus

You can add more documents to the corpus to help Jazzer in producing "nearly" proper queries which will improve fuzzing a lot. Slightly broken queries seem to be a good seed for fuzzing as well.

Download Jazzer from the releases page, choose the latest version and select the file jazzer-<os>-<version>.tar.gz

Unpack the archive:

tar xzf jazzer-*.tar.gz

Invoke the fuzzing:

./jazzer --cp=build/libs/antlr-fuzz-all.jar --instrumentation_includes=org.antlr.** --target_class=org.dstadler.antlr.fuzz.Fuzz -rss_limit_mb=1024 --jvm_args=-Xss4m corpus

In this mode Jazzer will stop whenever it detects an unexpected exception or crashes.

You can use --keep_going=10 to report a given number of exceptions before stopping.

See ./jazzer for options which can control details of how Jazzer operates.

Found issues

The following issues were detected:

ClassCastException
StringIndexOutOfBoundsException
NullPointerException
ArrayIndexOutOfBoundsException
IndexOutOfBoundsException

For now the Fuzzing target ignores these, see the class FuzzTest for reproducing test-cases for all of these.

License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.