Differential Fuzzing of URL Parsers (Chromium/Go) Part 1

2026-08-01 · 24 min read

This is the first part of (hopefully) a series about me learning to build a differential fuzzer in LibAFL. The code can be found on github; code snippets shown here refer to the tag part1. It is a late writeup of an experiment I did at the end of 2025 to learn more about the topic. The build instructions are retroactively updated and versions are pinned to work on a current Debian 13 install (as of today).

While browsing once, I wondered how URLs are parsed, and more specifically, how parser differentials in URL parsing can give an interesting semantic crash oracle / objective feedback for a fuzzer. So I decided to experiment with it. Hopefully this will be able to reproduce/rediscover known URL-confusion classes.

Prior work#

Of course, many have done this before.

A quick search revealed e.g. this paper by Kallus and Smith, “dippy_gram: Grammar-Aware, Coverage-Guided Differential Fuzzing of URL Parsers”, and the accompanying code. The paper goes into much detail and differentially fuzzes various implementations against each other: e.g. ada, boost_url, curl, furl, hyperlink, libwget, rfc3986, urllib, urllib3, yarl. I definitely recommend reading it.

Another good source is Claroty Team82 and Snyk’s “Exploiting URL Parsing Confusion”, which compares 16 URL parsing libraries – including both of the implementations I use here. It also performs a much better classification of the differentials than the ad-hoc bucketing I do in the results below.

With that said, I wanted to do my own experiments as an exercise and to learn more about the topic. This blog post is the result of that; the approach is by no means complete, and many things can be improved.

Specifications#

The following documents are important specs for URIs / URLs, and they do not always agree on the details:

The URL Living Standard
RFCs RFC3986 and RFC3987

Conflicting specifications mean that we will probably find lots of differentials. It also means that it isn’t that clear or easy to say, given a specific input, what the actual desired behavior should be. There may very well be no clear right or wrong way to choose.

It also just so happens that the two implementations used here (Chromium and Go stdlib) each do in fact exactly follow different specifications.

Threat model#

I do not have a fully thought-out threat model / security boundary in mind; however, I wanted to test parser differentials in URL parsing between important server and client implementations. To not explode the scope I chose to build a harness only for the following implementations:

Go stdlib net/url; motivation: used by Let’s Encrypt ACME protocol server implementation (server-side).

This implementation follows the RFC:

// Package url parses URLs and implements query escaping.
//
// See RFC 3986. This package generally follows RFC 3986, except where
// it deviates for compatibility reasons.
// RFC 6874 followed for IPv6 zone literals.
package url

Chromium GURL; motivation: as representative for a commonly used browser (client-side). This implementation seems to follow the URL Living Standard, since the code and documentation often references it.

Building the fuzzer#

Building the fuzzer requires three compilation steps (because of large external dependencies, the harnesses and the fuzzer are compiled separately): Compiling the Chromium and Go harnesses to a shared library, and then compiling the rust fuzzer – which also contains the main method as program driver – and linking everything together.

These individual steps are described in the following sections. We assume a VM with fixed hardcoded paths: the fuzzer code repository at /data, and the build directory for the Chromium harness at /disk/build-chrome. Also, the following basic dependencies are installed:

sudo apt install -y git wget curl python3 lsb-release sudo golang rustup file clang llvm

Chromium#

Chromium is written in C++. Building it is well documented in the Linux build instructions. The following instructions worked for me (with the version pinned to the one in my initial experiment at the end of 2025). This assumes enough available disk space; here I mounted a 150GB disk image into the VM at /disk/.

First, we have to install depot_tools. While the Chromium source code lives in git repositories, actually getting the source is not a simple git clone but instead handled through these wrapper tools:

mkdir -p /disk/build-chrome ; cd /disk/build-chrome

git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
git -C depot_tools checkout 967382fdcd1477dab3a7fec8eec3e3d5368bb0fe
echo 'export PATH="/disk/build-chrome/depot_tools:$PATH"; export DEPOT_TOOLS_UPDATE=0;' >> ~/.bashrc ; source ~/.bashrc
ensure_bootstrap

mkdir /disk/build-chrome/chromium ; cd /disk/build-chrome/chromium

Then we can download a specific Chromium version with a shallow clone (i.e. without downloading the full source code history). Usually source code download is handled in one go via fetch; for pinning the version, we need to split up the commands and adjust one of them. We can print the commands that fetch would execute with --dry-run:

$ fetch --dry-run --nohooks chromium
Running: gclient root
Running: gclient config --spec 'solutions = [
  {
    "name": "src",
    "url": "https://chromium.googlesource.com/chromium/src.git",
    "managed": False,
    "custom_deps": {},
    "custom_vars": {},
  },
]
'
Running: gclient sync --nohooks
cd /disk/build-chrome/chromium/src
Running: git config --add remote.origin.fetch '+refs/tags/*:refs/tags/*'
Running: git config diff.ignoreSubmodules dirty

And then we pick the commands we need: The first half we can copy verbatim; for the second half we need to specify the desired version from December 2025:

cd /disk/build-chrome/chromium/
gclient root
gclient config --spec 'solutions = [ { "name": "src", "url": "https://chromium.googlesource.com/chromium/src.git", "managed": False, "custom_deps": {}, "custom_vars": {}, }, ]'
gclient sync --nohooks --revision src@143.0.7499.40 --delete_unversioned_trees --force --reset
cd /disk/build-chrome/chromium/src
git config --add remote.origin.fetch '+refs/tags/*:refs/tags/*'
git config diff.ignoreSubmodules dirty

Now the source code is present on-disk and we can continue with installing the build dependencies:

cd /disk/build-chrome/chromium/src
sudo ./build/install-build-deps.sh
gclient runhooks

The next step then is to apply the provided patch that introduces a self-contained fuzz harness as an additional build target and compile it with SanCov coverage feedback as a dynamic library (.so).

Usually I would prefer building a static library (.a) and linking it directly into the final fuzzer binary, because this allows for a self-contained fuzzer. But when trying to do that for multiple hours, trying various build options, learning about “thin” and “thick” .a archives, and running into conflicts with internal bundled libraries on linking with the fuzzer, I chose the dynamic library route instead. (As we will see later, this results in needing to specify LD_LIBRARY_PATH for running the fuzzer to actually resolve the correct dynamic library). It also means that link-time optimization (LTO) can not be done with the fuzzer and the Chromium library together.

Here are the commands for applying the patch: (AI helped a lot with getting FFI and Chromium build options right)

cp -r /data/harnesses-extern/chromium/* /disk/build-chrome/chromium/src/
cd /disk/build-chrome/chromium/src/
git apply chromium-src.patch

The harness code is provided partially as patch file (for changes to individual files), and partially as a folder for fully new source files (not included in a single diff patch file, since it is more readable individually in this repository).

Then we can create a new build config named harness, set our own config options, and regenerate the config with updated values like so:

gn gen out/harness

cat <<'EOF' | tee out/harness/args.gn
is_debug = false
symbol_level = 2
is_component_build = false
is_clang = true
is_official_build = false
use_thin_lto = false
use_lld = true
use_sanitizer_coverage = true
sanitizer_coverage_flags = "trace-pc-guard"
EOF

gn gen out/harness

Displaying all arguments as a check shows that the desired LLVM SanCov option has correctly been applied:

$ gn args out/harness --list | grep trace-pc-guard -C2
sanitizer_coverage_flags
    Current value = "trace-pc-guard"
      From //out/harness/args.gn:9
    Overridden from the default = ""
--
        -fsanitize=fuzzer-no-link
    Default value when unset and use_fuzzing_engine=true:
        trace-pc-guard
    Default value when unset and use_sanitizer_coverage=true:
        trace-pc-guard,indirect-calls

save_reproducers_on_lld_crash

With these preparations done, we can finally build the Chromium URL parsing harness:

ninja -C out/harness url_ffi_chromium:url_ffi_chromium

The resulting shared library indeed now includes Undefined symbols for the LLVM SanCov symbols/coverage callbacks, whose implementation is provided later by the rust fuzzer code:

$ nm out/harness/liburl_ffi_chromium.so | grep __sanitizer_cov
                 U __sanitizer_cov_trace_pc_guard
                 U __sanitizer_cov_trace_pc_guard_init

Go#

While Go has a built-in fuzzer engine (go test -fuzz), I couldn’t find an easy way to access the coverage data myself for the custom fuzzer. (Correction: Somehow I didn’t know of golibafl at the time; if I were doing the project from scratch again, I would probably choose this). Hence, this implementation is not instrumented for coverage feedback.

Since the Chromium harness is built and linked dynamically, and we therefore cannot get a combined static self-contained fuzzer binary out of the build process either way, I chose to also link the Go harness dynamically.

The Go harness can be built as follows:

$ ./harnesses-extern/go/build.sh
++ dirname ./harnesses-extern/go/build.sh
+ cd ./harnesses-extern/go
+ go build -buildmode=c-shared -o liburl_ffi_go.so

Rust fuzzer (main)#

With both harnesses compiled as shared libraries .so, we can link them dynamically and then run the fuzzer, additionally specifying the library paths via LD_LIBRARY_PATH.

Here I didn’t fully automate building the harness shared libraries through build.rs. It is assumed that they have already been built manually at the exact paths mentioned previously. Then, the fuzzer can be built like so:

cd /data/
cargo build --release

In the final binary we can clearly see the imported dynamic harness libraries/ functions, and the exported callbacks for the C/C++ Chromium LLVM sancov hooks.

$ readelf -d target/release/fuzzer | grep liburl    # linked harness libraries
 0x0000000000000001 (NEEDED)             Shared library: [liburl_ffi_chromium.so]
 0x0000000000000001 (NEEDED)             Shared library: [liburl_ffi_go.so]

$ nm target/release/fuzzer | grep parse_url         # unresolved (imports)
                 U parse_url_chromium
                 U parse_url_go

$ nm target/release/fuzzer | grep __sanitizer_cov   # provided in .text section (exports)
000000000028f400 T __sanitizer_cov_trace_pc_guard
000000000028f420 T __sanitizer_cov_trace_pc_guard_init

Note that build.rs explicitly links the glibc c library first before the harnesses:

build.rs
 8
 9
10
11
12
13
14
    println!("cargo:rustc-link-lib=dylib=c");

    println!("cargo:rustc-link-search=/disk/build-chrome/chromium/src/out/harness/");
    println!("cargo:rustc-link-lib=dylib=url_ffi_chromium");

    println!("cargo:rustc-link-search=harnesses-extern/go/");
    println!("cargo:rustc-link-lib=dylib=url_ffi_go");

$ readelf -d target/release/fuzzer | grep NEEDED -C2
Dynamic section at offset 0x2ecdb8 contains 30 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [liburl_ffi_chromium.so]
 0x0000000000000001 (NEEDED)             Shared library: [liburl_ffi_go.so]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x000000000000001e (FLAGS)              BIND_NOW
 0x000000006ffffffb (FLAGS_1)            Flags: NOW PIE

The reason for this is that Chromium includes its own standard library, which also exports frequently used functions, e.g. such related to memory management and memory allocation:

$ nm /disk/build-chrome/chromium/src/out/harness/liburl_ffi_chromium.so \
    | grep -w -e malloc -e calloc -e realloc -e free
0000000000198990 T calloc
0000000000198870 T free
00000000001987b0 T malloc
00000000001988a0 T realloc

If glibc is not explicitly specified to resolve before the Chromium harness, the harness’s functions are used globally by the whole fuzzer program. Since the Chromium functions are instrumented for code-coverage, this slows down the fuzzer as a whole. And even worse, it skews the coverage map and drives the fuzzer into exploring the wrong code areas.

Running the fuzzer#

After building it, the fuzzer can be run as follows (here on an in-memory tmpfs to increase io speeds for the corpus; although I don’t know if this is really necessary):

export LD_LIBRARY_PATH=/disk/build-chrome/chromium/src/out/harness/:/data/harnesses-extern/go/
cp ./target/release/fuzzer /tmp/ ; cd /tmp/
./fuzzer --core 0

Fuzzer architecture#

Here is some background information on code structure and fuzzer architecture. The code orients itself by the LibAFL components. The following diagram shows an overview of data flow during one iteration of the fuzz loop, focusing on the custom components:

Here is the outline of the code repository:

$ tree
|-- build.rs
|-- Cargo.lock
|-- Cargo.toml
|-- harnesses-extern
|   |-- chromium
|   |   |-- chromium-src.patch
|   |   `-- url_ffi_chromium
|   |       |-- BUILD.gn -> BUILD.gn.dynamic-library
|   |       |-- BUILD.gn.dynamic-library
|   |       |-- url_ffi.cc
|   |       `-- url_ffi.h
|   `-- go
|       |-- build.sh
|       |-- go.mod
|       `-- lib.go
|-- README.md
`-- src
    |-- cliargs.rs
    |-- generator.rs
    |-- harnesses
    |   |-- chromium.rs
    |   |-- go.rs
    |   `-- mod.rs
    |-- input.rs
    |-- main.rs
    |-- mutator.rs
    `-- output.rs

./harnesses-extern/: harness implementations in their respective languages (go/c++)
./src/harnesses/: rust-side FFI implementation and argument/return type wrapping/unwrapping to call natively compiled harness libraries
./build.rs: build script for the rust fuzzer; links both harnesses dynamically to the fuzzer
./src/main.rs: entry-point; composes all components together and runs the fuzz-loop
./src/cliargs.rs: argument parsing
./src/input.rs: URL input data type definitions
./src/output.rs: URL output / return type data definitions
./src/generator.rs: generate URLs as test inputs
./src/mutator.rs: mutate existing URLs from the corpus for new test inputs

Input#

The basic data structure used throughout the fuzzer as harness input is a URL. When I speak of “URL” here, I mean an absolute-URL-with-fragment. At this stage of the fuzzer, we represent such a URL simply as a string consisting only of printable ASCII characters:

src/input.rs
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
/// Models an absolute-URL-with-fragment (https://url.spec.whatwg.org/#url-writing).
pub type UrlInput = PrintableAscii;

/// Newtype around BytesInput that prints printable ASCII characters.
/// This requires verbose pass-through of all needed trait implementations.
#[derive(Clone, Hash, Serialize, Deserialize)]
pub struct PrintableAscii(BytesInput);

impl PrintableAscii {
    pub fn new(bytes: BytesInput) -> Self {
        let mut res = Self(bytes);
        res.force_ascii_printable();
        res
    }

    pub fn force_ascii_printable(&mut self) {
        let low: u8 = 0x20;
        let high: u8 = 0x7e;
        for byte in self.0.mutator_bytes_mut() {
            if !(low..=high).contains(byte) {
                *byte = low + (*byte % (high - low + 1));
            }
        }
    }
}

Output#

The harnesses define structured output types for diffing:

src/output.rs
68
69
70
71
72
73
74
75
76
77
#[derive(PartialEq)]
pub struct UrlOutput<'a> {
    pub scheme: &'a [u8],
    pub userinfo: &'a [u8],
    pub host: &'a [u8],
    pub port: &'a [u8],
    pub path: &'a [u8],
    pub query: &'a [u8],
    pub fragment: &'a [u8],
}

This rust-representation does not own the parsed byte buffers and instead wraps the returned values. This allows for simpler/faster conversion, without string parsing/validation as UTF8, and without unnecessarily copying memory.

Executor#

The executor is the glue between the compiled harnesses (shared libraries) that export C-FFI functions char* → size_t → UrlOutput_, and their respective rust wrappers &str → UrlOutput, which translate the data types from raw C FFI types to rust-types.

Here we use LibAFL’s builtin InProcessExecutor, which – additionally to performing the function call into the shared library – wraps signal handling and allows the harness function to actually crash without taking down the whole fuzzer program.

Note that here we have a simplified rust harness with a combined harness closure. Since we can call both harnesses in the same way as a shared library, we do not need a distinct DiffExecutor (which would allow executing different harnesses in different ways). This makes the resulting fuzzer very short in code, since the clear distinction of LibAFL concepts/components is not present and the harness closure itself combines executor, observers for the return value, and objective feedback (differential oracle) into one single unit:

src/main.rs
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
    // Differential harness; mixes multiple LibAFL concepts together
    // (executor, observer, feedback).
    let mut harness = |input: &UrlInput| {
        let url = input.to_string();

        let ret1 = chromium::parse_url(&url);
        let ret2 = go::parse_url(&url);

        if all_equal(&[ret1.as_url(), ret2.as_url()]) {
            ExitKind::Ok
        } else {
            // Differential triggers a crash.
            ExitKind::Crash
        }
    };

This approach scales well to three or – in general – an arbitrary number of targets. However it cannot represent scenarios that need differing or independent implementations for each of the mentioned components (e.g. different execution methods).

For both harnesses we observe the structured parsed return value of the harness function. Besides the return value, for the Chromium harness, we additionally observe coverage information from the LLVM SanCov hooks:

src/main.rs
104
105
106
    // Here we only use the chromium backend for coverage-feedback.
    let observer_map_chromium =
        unsafe { StdMapObserver::from_mut_slice("edges_chromium", edges_map_mut_slice()) };

In theory it would probably be better to get coverage-guidance from all harnesses into a combined coverage map (e.g. first half of the map written by harness 1, second half of the map written by harness 2). However when I tried to do that first, I didn’t know of a good way of extracting Go coverage information, and then when learning about golibafl I didn’t want to refactor the code again.

Therefore, our coverage guidance relies only on instrumenting one of the two harnesses (hey, at least better than nothing). The hope is that the implementation logic is sufficiently similar that higher coverage in one harness also drives up coverage in the other and hits interesting edge-cases there too – since the actual implemented functionality are similar.

Feedback#

The feedback for guiding the fuzzer is a MaxMapFeedback of the Chromium coverage map; i.e. it tries to maximize the coverage:

src/main.rs
110
111
    // Feedback to maximize coverage / coverage-guidance.
    let mut feedback = MaxMapFeedback::new(&observer_map_chromium);

Finally, the objective feedback (= crash oracle) is the actual differential / semantic test. The harness function returns a crash result when the parsed URLs differ in any way (this includes one parser failing while the other succeeds):

src/main.rs
113
114
115
    // Solution / crash-oracle is crash-feedback returned by the differential
    // harness.
    let mut objective = CrashFeedback::new();

Initial corpus seeding#

The initial corpus is seeded from two sources:

from a set of valid urls:

src/input.rs
11
12
13
14
15
16
17
/// Manually seed with potentially interesting URLs.
pub const SEED_URLS: &[&str] = &[
    // Basic URL.
    "https://example.com/",
    // URL with all possible fields set.
    "http://user:pass@example.com:8080/some/test/path?some=1&query=2&params=3#some-fragment",
];

from randomly generated printable ASCII strings (which uses LibAFL’s builtin component RandPrintablesGenerator):

src/main.rs
137
    let mut generator = UrlGenerator::new(RandPrintablesGenerator::new(0x80.try_into()?));

src/generator.rs
13
pub struct UrlGenerator(RandPrintablesGenerator);

src/generator.rs
21
22
23
24
25
26
27
28
29
// Pass-through implementation.
impl<S> Generator<UrlInput, S> for UrlGenerator
where
    S: HasRand,
{
    fn generate(&mut self, state: &mut S) -> Result<UrlInput, Error> {
        self.0.generate(state).map(UrlInput::new)
    }
}

Mutator#

Mutation is done by wrapping / delegating to LibAFL’s builtin HavocScheduledMutator with havoc_mutations, which performs a series of byte-level mutations on the underlying byte buffer. It also includes crossover-mutation (e.g. splicing in slices from other inputs in the corpus).

Since arbitrary byte-level mutations almost certainly lead to non-ASCII characters, and even bytes that are not valid UTF8, we fix this in a processing step by forcing the result back into the printable ASCII range.

src/main.rs
141
    let mutator = UrlMutator::new(HavocScheduledMutator::new(havoc_mutations()));

src/mutator.rs
16
pub struct UrlMutator(HavocScheduledMutator<HavocMutationsType>);

src/mutator.rs
38
39
40
41
42
43
44
45
46
    fn mutate(
        &mut self,
        state: &mut S,
        input: &mut UrlInput,
    ) -> Result<libafl::mutators::MutationResult, libafl::Error> {
        let res = self.0.mutate(state, input)?;
        input.force_ascii_printable();
        Ok(res)
    }

Remaining components#

For the remaining components I use standard implementations from LibAFL:

StdState
StdRand
InMemoryOnDiskCorpus for feedback (is_interesting / new coverage) and objective feedback (differentials/crashes)
QueueScheduler
StdFuzzer
SimpleMonitor and SimpleEventManager
StdMutationalStage

And the standard fuzz-loop:

src/main.rs
193
    fuzzer.fuzz_loop(&mut stages, &mut executor, &mut state, &mut mgr)?;

Results#

The differentials start pouring in immediately after starting the fuzzer. It runs at > 10k executions per second on my old laptop (i5-6300U; one virtual cpu core for the fuzzer, two in total for the VM; a Debian 13 qemu/kvm VM on a Debian 13 host).

$ ./fuzzer --core 1
Pinning fuzzer to CPU core 1
Using RNG seed: 0x6e88bd0ee231e978
Setting up harnesses
Setting up observers
Setting up feedbacks
Setting up state
Setting up scheduler
Setting up fuzzer
Setting up generator
Setting up mutator and mutational stage
Setting up monitor
Setting up executors
Loading testcases from previous corpus
Previous corpus was empty; generating initial testcases
[UserStats #0] run time: 0s, clients: 1, corpus: 0, objectives: 0, executions: 1, exec/sec: 0.000, edges_chromium: 810/37206 (2%)
[Testcase #0] run time: 0s, clients: 1, corpus: 1, objectives: 0, executions: 1, exec/sec: 0.000, edges_chromium: 810/37206 (2%)
[UserStats #0] run time: 0s, clients: 1, corpus: 1, objectives: 0, executions: 2, exec/sec: 0.000, edges_chromium: 871/37206 (2%)
[Testcase #0] run time: 0s, clients: 1, corpus: 2, objectives: 0, executions: 2, exec/sec: 0.000, edges_chromium: 871/37206 (2%)
[UserStats #0] run time: 0s, clients: 1, corpus: 2, objectives: 0, executions: 3, exec/sec: 0.000, edges_chromium: 926/37206 (2%)
[Testcase #0] run time: 0s, clients: 1, corpus: 3, objectives: 0, executions: 3, exec/sec: 0.000, edges_chromium: 926/37206 (2%)
[Objective #0] run time: 0s, clients: 1, corpus: 3, objectives: 1, executions: 4, exec/sec: 0.000, edges_chromium: 926/37206 (2%)
[Objective #0] run time: 0s, clients: 1, corpus: 3, objectives: 2, executions: 5, exec/sec: 0.000, edges_chromium: 926/37206 (2%)
[UserStats #0] run time: 0s, clients: 1, corpus: 3, objectives: 2, executions: 6, exec/sec: 0.000, edges_chromium: 941/37206 (2%)
[Testcase #0] run time: 0s, clients: 1, corpus: 4, objectives: 2, executions: 6, exec/sec: 0.000, edges_chromium: 941/37206 (2%)
[Objective #0] run time: 0s, clients: 1, corpus: 4, objectives: 3, executions: 7, exec/sec: 0.000, edges_chromium: 941/37206 (2%)
...
[Objective #0] run time: 20s, clients: 1, corpus: 177, objectives: 81951, executions: 205956, exec/sec: 10.31k, edges_chromium: 1389/37206 (3%)
[Objective #0] run time: 20s, clients: 1, corpus: 177, objectives: 81952, executions: 205959, exec/sec: 10.31k, edges_chromium: 1389/37206 (3%)
[Objective #0] run time: 20s, clients: 1, corpus: 177, objectives: 81953, executions: 205960, exec/sec: 10.31k, edges_chromium: 1389/37206 (3%)
[Objective #0] run time: 20s, clients: 1, corpus: 177, objectives: 81954, executions: 205961, exec/sec: 10.31k, edges_chromium: 1389/37206 (3%)
[Objective #0] run time: 20s, clients: 1, corpus: 177, objectives: 81955, executions: 205963, exec/sec: 10.31k, edges_chromium: 1389/37206 (3%)
[Objective #0] run time: 20s, clients: 1, corpus: 177, objectives: 81956, executions: 205964, exec/sec: 10.31k, edges_chromium: 1389/37206 (3%)
[Objective #0] run time: 20s, clients: 1, corpus: 177, objectives: 81957, executions: 205965, exec/sec: 10.31k, edges_chromium: 1389/37206 (3%)
[Objective #0] run time: 20s, clients: 1, corpus: 177, objectives: 81958, executions: 205966, exec/sec: 10.31k, edges_chromium: 1389/37206 (3%)
[Objective #0] run time: 20s, clients: 1, corpus: 177, objectives: 81959, executions: 205971, exec/sec: 10.31k, edges_chromium: 1389/37206 (3%)
[Objective #0] run time: 20s, clients: 1, corpus: 177, objectives: 81960, executions: 205972, exec/sec: 10.31k, edges_chromium: 1389/37206 (3%)

After 20 seconds we have 205972 executions, 81960 differentials, but only a very small corpus of 177 inputs that lead to new coverage. Obviously this is very bad and makes the fuzzer essentially unusable in this stage, since it cannot run for a long time without flooding the disk, likely with meaningless duplicates.

Note that since we haven’t done any post-processing at all, most of those are overly complicated, non-minimal inputs, and duplicates showing the same root-cause.

With that said, we can nonetheless do some manual (and AI-assisted) classification/triaging, input shrinking and root-causing of individual examples. A single input can be reproduced by specifying its ID, like ./fuzzer --reproduce-testcase-id 4c878f3c541bdd84. All crashing inputs can be reproduced by iterating, like so:

clear ; ls crashes/ | while read ID ; do echo $ID ; ./fuzzer --reproduce-testcase-id $ID ; echo ; done

The following list shows some filtered examples of differentials and a short comment on why they stood out.

Go fails, while Chromium does not:

$ ./fuzzer --reproduce-testcase-id 033576e0c45da307
input:    http://user:pass@ex0 menae-frafmenath?some=1test/path&query=2&paraMs=3#sommple.com:8ome/t
chromium: Some(UrlOutput { scheme: "http", userinfo: "user:pass", host: "ex0%20menae-frafmenath", port: "", path: "/", query: "some=1test/path&query=2&paraMs=3", fragment: "sommple.com:8ome/t" })
go:       None

A more minimal input that produces the same behavior:

$ ./fuzzer --reproduce-url 'http://user:pass@ex0 m-?'
input:    http://user:pass@ex0 m-?
chromium: Some(UrlOutput { scheme: "http", userinfo: "user:pass", host: "ex0%20m-", port: "", path: "/", query: "", fragment: "" })
go:       None

Chromium fails, while Go does not:

$ ./fuzzer --reproduce-testcase-id 0396299ba919a433
input:    https:/2APaa@2aY8Bo^Teeeeen
chromium: None
go:       Some(UrlOutput { scheme: "https", userinfo: "", host: "", port: "", path: "/2APaa@2aY8Bo%5ETeeeeen", query: "", fragment: "" })

A more minimal case:

$ ./fuzzer --reproduce-url 'https::some'
input:    https::some
chromium: None
go:       Some(UrlOutput { scheme: "https", userinfo: "", host: "", port: "", path: ":some", query: "", fragment: "" })

ASCII input can lead to non-UTF8 output. Surprising to me was that some printable-ASCII URL inputs (subset of UTF8) are parsed into byte outputs that are not even valid UTF8. Here is an example in the host field. Note that --reproduce-testcase-id behaves as follows: if the output URL part is valid UTF8 it is printed as a string, otherwise as a byte array. As far as I can see, all cases stem from the fact that the Go function I chose to call in the harness (there are also other API functions exposed) decodes percent-encoding in the parsed URL to a byte buffer (example shown here is from another run):
{{ /* clear ; ls crashes/ | while read ID ; do ./fuzzer –reproduce-testcase-id $ID ; echo ; done | grep -F ‘"[’ -B2 */ }}
```
$ ./fuzzer --reproduce-testcase-id 0c4d44fa2c898fee
input:    https://Z%aa/
chromium: None
go:       Some(UrlOutput { scheme: "https", userinfo: "", host: "[90, 170]", port: "", path: "/", query: "", fragment: "" })
```

Differing host:

$ ./fuzzer --reproduce-testcase-id 09fd68c4a707f486
input:    http:7s:/yAfNbms=3#s
chromium: Some(UrlOutput { scheme: "http", userinfo: "", host: "7s", port: "", path: "/yAfNbms=3", query: "", fragment: "s" })
go:       Some(UrlOutput { scheme: "http", userinfo: "", host: "", port: "", path: "7s:/yAfNbms=3", query: "", fragment: "s" })

A more minimal example:

$ ./fuzzer --reproduce-url https:7
input:    https:7
chromium: Some(UrlOutput { scheme: "https", userinfo: "", host: "0.0.0.7", port: "", path: "/", query: "", fragment: "" })
go:       Some(UrlOutput { scheme: "https", userinfo: "", host: "", port: "", path: "7", query: "", fragment: "" })

Host IP address normalization (Go treats it as an opaque string and doesn’t change it):

$ ./fuzzer --reproduce-url 'httP://0X??(~:pa~s@examaA /test4paMMMp:tYss@-Ctos:/2&'
input:    httP://0X??(~:pa~s@examaA /test4paMMMp:tYss@-Ctos:/2&
chromium: Some(UrlOutput { scheme: "http", userinfo: "", host: "0.0.0.0", port: "", path: "/", query: "?(~:pa~s@examaA%20/test4paMMMp:tYss@-Ctos:/2&", fragment: "" })
go:       Some(UrlOutput { scheme: "http", userinfo: "", host: "0X", port: "", path: "", query: "?(~:pa~s@examaA /test4paMMMp:tYss@-Ctos:/2&", fragment: "" })

$ ./fuzzer --reproduce-url 'http://1/test'
input:    http://1/test
chromium: Some(UrlOutput { scheme: "http", userinfo: "", host: "0.0.0.1", port: "", path: "/test", query: "", fragment: "" })
go:       Some(UrlOutput { scheme: "http", userinfo: "", host: "1", port: "", path: "/test", query: "", fragment: "" })

$ ./fuzzer --reproduce-url 'http://01/test'
input:    http://01/test
chromium: Some(UrlOutput { scheme: "http", userinfo: "", host: "0.0.0.1", port: "", path: "/test", query: "", fragment: "" })
go:       Some(UrlOutput { scheme: "http", userinfo: "", host: "01", port: "", path: "/test", query: "", fragment: "" })

Similarly, port number normalization:

$ ./fuzzer --reproduce-url http://test:080/test
input:    http://test:080/test
chromium: Some(UrlOutput { scheme: "http", userinfo: "", host: "test", port: "", path: "/test", query: "", fragment: "" })
go:       Some(UrlOutput { scheme: "http", userinfo: "", host: "test", port: "080", path: "/test", query: "", fragment: "" })

$ ./fuzzer --reproduce-url http://test:09999/test
input:    http://test:09999/test
chromium: Some(UrlOutput { scheme: "http", userinfo: "", host: "test", port: "9999", path: "/test", query: "", fragment: "" })
go:       Some(UrlOutput { scheme: "http", userinfo: "", host: "test", port: "09999", path: "/test", query: "", fragment: "" })

Dot-segment path normalization (the Go harness doesn’t normalize, the Chromium harness does; here output from another run):

$ ./fuzzer --reproduce-url 'zkaaa:/.?J#  01 A::aaii0'
input:    zkaaa:/.?J#  01 A::aaii0
chromium: Some(UrlOutput { scheme: "zkaaa", userinfo: "", host: "", port: "", path: "/", query: "J", fragment: "%20%2001%20A::aaii0" })
go:       Some(UrlOutput { scheme: "zkaaa", userinfo: "", host: "", port: "", path: "/.", query: "J", fragment: "%20%2001%20A::aaii0" })

$ ./fuzzer --reproduce-url 'http://test/.././test'
input:    http://test/.././test
chromium: Some(UrlOutput { scheme: "http", userinfo: "", host: "test", port: "", path: "/test", query: "", fragment: "" })
go:       Some(UrlOutput { scheme: "http", userinfo: "", host: "test", port: "", path: "/.././test", query: "", fragment: "" })

Similar to other issues, performing some of this classification work in a more structured and automated way is on the list of open tasks that could be part of the next post.

Tentative conclusions (so far)#

One could go into many more examples, but I don’t think that is useful. From the examples I’ve reviewed, I think one can draw the following conclusions so far:

Besides the different specifications, there are underlying systematic differences regarding choices of normalized URL representation (multiple representations can be valid, but still differ in exact byte values) leading to many meaningless differentials (maybe I did not use the correct APIs; this could also be my error). E.g., percent encoding canonicalization is handled differently, in different parts of the URL.

Even for something seemingly simple as a URL it is not so easy to wrap multiple harnesses into exposing the same semantics as an interface. For example, the Go standard library function url.Parse also parses non-absolute (relative) URLs, while the Chromium function to my knowledge parses absolute URLs; this alone leads to many expected / documented discrepancies.

Things I learned about#

I nonetheless learned many new things in more detail than before while building this; e.g. about:

Building fuzzers with LibAFL and implementing LibAFL components; using a semantic/differential oracle with structured output, not only crashing as objective feedback.
Wrapping implementations in harnesses (code + build options); compiling Chromium.
FFI and static/dynamic linking between multiple languages.
SanCov internals and custom hooks.

Room for improvement#

As mentioned in the introduction and throughout the post, the state of this project is very far from perfect. Here are some specific ideas I could think of regarding room for improvement and features which could still be implemented as possible next steps (maybe in a potential next part of this series):

Reduce the noise of meaningless differentials; e.g. by using the following specific strategies.
Consistent handling of absolute vs. relative URLs (e.g. simply making sure the generator and mutator always produce absolute URLs with a present scheme; this circumvents having to hook/change harness implementations in a complicated way).
Wrap implementations/harnesses to return a more unified/normalized structured output representation that abstracts away arbitrary, not meaningful choices/ differences in normalization (may be difficult to implement).
Structured generation and mutation (not simple byte-level operations), which may be able to hit specific interesting classes of differences easier. (Or not, since URLs are not really structurally nested/complicated and may be fully explored through byte mutations).
Token dictionary mutation.
Automated input classification/grouping: E.g. by having the differential comparison not only return equal/not-equal, but also which fields differ. Buckets:
- one parser fails, the other doesn’t: chromium-fails, go-fails
- if both succeed, sets (ordered vec) of differing field names
Automated input shrinking/minimization, to get a short minimal input that produces a given behavior/output (root-causing the behavior).
Related: Automated corpus minimization/deduplication.
Separate out the differential harness function and map the individual parts to the corresponding intended LibAFL components (executor/observer/differential feedback).
Fuzzer introspection and measuring/annotating statistics: e.g. how many input cases simply lead to wasted executions because all harnesses/parsers fail.
Generalized diff of more than two implementations.
Seed initial corpus with WHATWG-provided adversarial URLs: urltestdata.json.