Repository navigation
test: add memory checks and fuzz testing for REST and WebSocket parsers - #35
Conversation
|
@SebastianBoehler |
|
Thanks Bill, I reviewed the latest head ( I found one build issue to fix before merging: in Could you apply the sanitizer link flags to the library itself as well? Changing Please add a focused shared-library build check with fuzzers enabled so this configuration stays covered. The rest of the reviewed changes look good. |
|
@yluoc Thanks for the sanitizer link fix in The new macOS shared-library CI job is still failing. It stops while compiling IXWebSocket with Homebrew LLVM 18: Could you fix the macOS compiler/SDK/header configuration and rerun the job? The other seven checks pass. We need this new check to complete successfully before merging. |
Summary
The SDK parses REST responses and WebSocket messages it does not control, but the tests only covered inputs we wrote by hand. Nothing checked how the parsers handle malformed or unexpected data, and AddressSanitizer and UBSan never ran in CI.
This PR adds three layers of checks:
build.yml): builds and runs the offline suite with AddressSanitizer and UBSan on Ubuntu 22.04.test_package_consumeris excluded because it links an uninstrumented external project; the regular matrix still runs it.tests/fuzz/):fuzz_websocket_messagescovers the market, tick size, and user stream parsers. Accepted book updates must leave every level with a price in (0, 1), a non-negative size, and correct ordering.fuzz_rest_responsescovers the order book, open order, order response, cancellation, trade, and market parsers. Accepted order books must have valid levels and no duplicate prices.std::exception. A crash, sanitizer report, or broken invariant is a failure.POLYMARKET_CLIENT_BUILD_FUZZERS=ON(off by default, Clang only) builds libFuzzer targets. Normal test builds compile the same harnesses with a replay driver, so every CI configuration replays the seeds.fuzz.yml): runs each fuzzer for 1 minute on PRs, 15 minutes nightly, or a chosen duration from a manual run. Failing inputs are uploaded as an artifact.CONTRIBUTING.mddocuments how to run the fuzzers locally.Verification
Commands run and their actual results:
ctest -LE live -E '^test_package_consumer$'(gcc 13, Debug, ASan + UBSan): 63 of 63 passed, including both seed replays.ctest -L fuzz(Clang 18 fuzz build): 2 of 2 passed.fuzz_websocket_messages -max_total_time=180: about 1.44M inputs, no findings.fuzz_rest_responses -max_total_time=180: about 576K inputs, no findings.fuzz.ymlsteps run locally in a fresh build (fuzz_rest_responses, 60 s): seed replay passed, about 186K inputs, no findings.crash-*tofuzz-run/artifacts/, the path the upload step collects.clang-format --dry-run --Werrorandclang-tidy(21.1.6) ontests/fuzz/*.cpp: clean.Checks not run and the reason (or none):
scripts/quality.py: the localnpxresolves to a Windows npm and fails. CI will run them.Compatibility and evidence
Public API, binary compatibility, or release implications (or none):
None. No public headers or library behavior changed. The new CMake option is off by default, and the replay executables are built only when tests are enabled.
For protocol changes: not applicable.
For live checks: not applicable. All results come from local fixtures and fuzz runs; no live endpoints were used.