This repository is related to fuzzing of CNCF projects. It holds fuzzers as well as documentation on fuzzing.
See the blog post Improving security by fuzzing the cncf landscape for details.
Are you a CNCF project and would like to have it fuzzed? Please create an issue on this repository requesting to be fuzzed and we will assist.
CNCF has a Fuzzing Handbook which introdues fuzzing in a pragmatic manner, and also has a chapter that goes into details with how to use OSS-Fuzz to establish a continuous fuzzing set up.
The handbook is available here
Fuzzing is a technique for automating stress testing of applications and it can be used to find reliability and security issues. The technique is traditionally used by security researchers to find security vulnerabilities, however, fuzzing has become more integrated into the software development lifecycle and is increasingly being used by developers.
CNCF projects that use fuzzing include:
- Argo
- Containerd
- CRI-O
- Envoy
- Fluent-bit
- FluxCD and full report, section 5
- Kubernetes
- Linkerd2-proxy
- Prometheus
- RunC
- Vitess
- Distribution
Talks on CNCF fuzzing:
- Finding Bugs and Vulnerabilities Automatically
- Fuzzing the CNCF Landscape, Cloud Native SecurityCon, 2022
- Securing Fluent Bit by Way of Fuzzing, FluentCon, 2022
- Fuzz Testing of Envoy - Adi Peleg & Teju Nareddy
- Lightning Talk: Securing Envoy: Catching Vulnerabilities With Continuous Fuzz Testing - Teju Nareddy
Dedicated fuzzing audit reports:
Integrating fuzzing into a project takes a lot effort and is often done over a long period of time. Fuzzing can be integrated into your project with various levels of maturity. There are three essential tasks when integrating fuzzing into your project:
- Develop fuzzers
- Execute the fuzzers
- Analyse crashes
The following describes three common steps in integrating fuzzing into your project.
The first step in integrating fuzzing into a project is to develop a set of fuzz drivers for your project. The specific fuzzer you need to use depends on the programming language of your project. The following list provides links to common fuzzers for various languages:
- C/C++: libFuzzer
- Rust: Cargo-fuzz
- Go: Go-fuzz and native go fuzzing
- Python: Atheris fuzzer
- Java: jazzer fuzzer
The specific purpose of a fuzz driver vary greatly. In essence, they are closely related to unit tests and the difference is the fuzz driver takes a random input which is used to enforce diverse code execution of the target code. Common goals of a fuzz driver include:
- Execute large amounts of code to achieve high code coverage
- Execute a specific complex piece of code, e.g. parsing routines
- Execute code relative to the threat model of project
This step is usually the most time-consuming and making it possible to write fuzz drivers for your project can sometimes be a large effort. However, once you have fuzz drivers for you project you should be able to run these locally and observe results.
Once you have developed a local fuzzing set up for your project, the next step is to run the fuzzers in a continuous manner. Modern fuzzers rely on genetic algorithms to build up an input corpus, which, in a simplified manner, means that the fuzzer by nature increases it’s quality in proportion to how long it has run. Continuously running a fuzzer is thus important to ensure high quality of the fuzzing and continuous fuzzing is also important in order to capture bugs that may occur as a project progresses.
OSS-Fuzz is a service for running fuzzers continuously for open source projects. OSS-Fuzz comes with a convenient management infrastructure with a dashboard as well as bug-tracking features, which makes managing running of the fuzzers easy. We recommend integrating with OSS-Fuzz, and several CNCF projects are integrated already.
Fuzzing can be integrated in your CI, e.g. a GitHub action, such that the fuzzers run for a short amount of time on pull requests and/or push actions. This is in many ways similar to running tests as part of your CI to ensure regressions don’t occur. Once you have integrated with OSS-Fuzz, you can get CI integration by way of CIFuzz for free.
Fuzzing works best with projects that have high code complexity, e.g. parsers, decoders, etc. but can be used in many other projects. You can fuzz projects in many languages and the type of bug you will find depends on which language your project is written in.
- Envoy has invested significantly in fuzzing and OSS-Fuzz has reported more than 700 bugs as well as 81 security relevant bugs
- Fluent-bit has been fuzzed for slightly more than a year, and OSS-Fuzz has reported more than 100 reliability issues and more than 50 security issues.
For an example where fuzzing was determined to have limited effects consider Cloud custodian. Cloud custodian is a project written in Python and is very horisontal in its architecture in that it does not have deep code complexities. This is an example where fuzzing will have limited results as discussed in detail in a PR on the Cloud Custodian repository. However, Cloud Custodian still benefited from fuzzing finding a bug in the code of Cloud Custodian where fuzzing could be applied, but, in comparison to the other projects mentioned above Cloud Custodian is not integrated into OSS-Fuzz.
The following list indicates some common software properties that means your code is likely to benefit from fuzzing
- High code complexity
- Deep code paths
- Accepts untrusted input
- If a reliability or reliability issue occur then it can have significant consequences for systems
- Is used as a library by other applications
- Projects in memory unsafe languages should have a high prority for being fuzzed (but fuzzing is not exclusive to memory unsafe languages)
We have montly fuzzing hours on every second Friday of each month at 2PM GMT+1.
URL: https://us05web.zoom.us/j/86181802116 Passcode: LBw6fY