Skip to content

test(codec): ignore the zstd encoder tests on wasm32 rather than abort the suite - #389

Closed
EnRaiha wants to merge 1 commit into
NodeDB-Lab:mainfrom
EnRaiha:test/codec-gate-zstd-encoder-wasm
Closed

EnRaiha wants to merge 1 commit into
NodeDB-Lab:mainfrom
EnRaiha:test/codec-gate-zstd-encoder-wasm

Conversation

@EnRaiha

@EnRaiha EnRaiha commented Sep 27, 2026

Copy link
Copy Markdown
Contributor

What

Gate the eight zstd encoder tests in nodedb-codec/src/zstd_codec.rs with
#[cfg_attr(target_arch = "wasm32", ignore = "...")]. Test attributes and
comments only; no behaviour change — the wasm arm still returns the same error.

Why

nodedb-codec already splits the codec by target: C libzstd off wasm, ruzstd on
it (nodedb-codec/Cargo.toml:31-36), and compress_native's wasm arm is an
explicit stub returning CompressFailed (nodedb-codec/src/zstd_codec.rs:312-320);
no fallback sits under it, and pipeline propagates that error for
ColumnCodec::Zstd. The eight tests call encode/encode_with_level, so they panic —
and on wasm32-wasip1 a panic aborts the process: the suite dies at the first
one and reports nothing after it. That makes the crate's wasip1 suite unusable,
which is what a shared-crate wasm job for NodeDB-Lite needs.

That stub is a design choice, not a missing dependency: ruzstd 0.9 ships an
encoder (ruzstd::encoding — pub mod encoding in lib.rs, compress in
encoding/mod.rs, FrameCompressor defined in encoding/frame_compressor.rs and
re-exported from encoding/mod.rs) that this crate has not adopted, and the C
library would need a C-to-WASM toolchain. Adopting either one changes what the wasm
build does, so it is deliberately not part of this change. The stub's own comments
used to call it a fallback that encodes "a minimal Zstd frame" — it never did, and
they now describe the stub.

cfg_attr(ignore) keeps them compiled and listed by --list on that target, so
the gap shows up in the run. cfg(not(target_arch = "wasm32")) would delete them
from the target instead, and a --skip list in a consumer workflow would record
the gap as a number in someone else's file.

How to check it

# native: the eight still run, and nothing is ignored
cargo test -p nodedb-codec
# 263 passed; 0 failed; 0 ignored

# wasip1: the four aborts owned by other changes are skipped, to reach this one
RUSTFLAGS="" CARGO_TARGET_WASM32_WASIP1_RUNNER="wasmtime -W max-wasm-stack=33554432 --dir=." \
  cargo test --profile ci --target wasm32-wasip1 -p nodedb-codec -- \
  --skip delta::tests::rejects_overflowed_noncanonical_and_huge_input_before_allocation \
  --skip double_delta::tests::huge_count_and_noncanonical_tail_are_rejected_before_decode_work \
  --skip fastlanes::codec::tests::hostile_counts_and_block_shapes_fail_before_allocation_or_looping \
  --skip vector_quant::opq::tests::top1_recall_on_training_set
# 252 passed; 0 failed; 8 ignored; 4 filtered out

Red arm: with the ignores removed and the same four skips, the same command
aborts —

test zstd_codec::tests::better_ratio_than_lz4 ... Error: failed to run main module
exit 134

Each ignored test names its reason:

test zstd_codec::tests::empty_data ... ignored, nodedb-codec has no wasm encoder: encode() returns CompressFailed

cargo fmt --all -- --check exits 0; cargo clippy -p nodedb-codec --all-targets -- -D warnings exits 0 natively and for --target wasm32-wasip1.

Scope

  • The module's other four tests stay live on both targets: three are decode-side,
    and streaming_input_limit_precedes_buffer_growth drives the encoder's
    declared-limit check, which rejects before any encoder work.
  • The four --skip flags above name tests owned elsewhere: three abort through a
    32-bit overflow (delta::tests::rejects_overflowed_noncanonical_and_huge_input_before_allocation,
    double_delta::tests::huge_count_and_noncanonical_tail_are_rejected_before_decode_work,
    fastlanes::codec::tests::hostile_counts_and_block_shapes_fail_before_allocation_or_looping)
    and are fixed by an open change of their own;
    vector_quant::opq::tests::top1_recall_on_training_set is a recall threshold
    the target misses and stays a named skip in the consumer job, not here.
  • No CI workflow is added here; this is the prerequisite for the shared-crate
    wasm job that belongs in the NodeDB-Lite repository.

…t the suite

`nodedb-codec` gates its codec by target already: C libzstd off wasm, ruzstd on
it (`Cargo.toml:31-36`), and `compress_native`'s wasm arm is an explicit stub
returning `CompressFailed` (`src/zstd_codec.rs:312-320`). The eight tests that call
`encode`/`encode_with_level` therefore panic, and on
`wasm32-wasip1` a panic aborts the process: the binary dies at the first one and
reports nothing after it.

The stub is a design choice rather than a missing dependency: ruzstd 0.9 ships an
encoder (`ruzstd::encoding`) this crate has not adopted, and the C library would
need a C-to-WASM toolchain. Wiring up either one changes what the wasm build does
and is a change of its own; this commit only stops the absent encoder from killing
the suite. No fallback exists in this crate — `pipeline` propagates
`CompressFailed` for `ColumnCodec::Zstd`. The stub's own comments said it was a
fallback that encodes "a minimal Zstd frame", which it never did; they now describe
the stub.

They are gated with `#[cfg_attr(target_arch = "wasm32", ignore = "…")]` rather
than `#[cfg(not(target_arch = "wasm32"))]`. The attribute keeps them compiled and
listed by `--list` on that target, so the gap is visible in the run instead of
absent from it, and they keep running natively.

Red arm, with the four tests other changes own skipped:

    $ cargo test --profile ci --target wasm32-wasip1 -p nodedb-codec -- <4 skips>
    test zstd_codec::tests::better_ratio_than_lz4 ... Error: failed to run main module
    exit 134

Green arm, same command:

    test result: ok. 252 passed; 0 failed; 8 ignored; 0 measured; 4 filtered out

Each ignored test prints its reason:

    test zstd_codec::tests::empty_data ... ignored, nodedb-codec has no wasm encoder: encode() returns CompressFailed

Native is unchanged at 263 passed, 0 failed, 0 ignored. The module's other four
tests stay live on both targets: three are decode-side, and
`streaming_input_limit_precedes_buffer_growth` drives the encoder's declared-limit
check, which rejects before any encoder work.

Why an in-source ignore and not a `--skip` list in the consumer: the gap is
categorical — the target has no encoder at all, not a threshold that moves — and
a named skip in someone else's workflow records it as a number rather than as a
property of this crate.

fmt clean; clippy clean for this crate on native and on `wasm32-wasip1`.
Copilot AI lite review requested due to automatic review settings September 27, 2026 16:13

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@EnRaiha

EnRaiha commented Sep 28, 2026

Copy link
Copy Markdown
Contributor Author

Closing. This adds #[cfg_attr(target_arch = "wasm32", ignore = …)] to eight zstd encoder tests in nodedb-codec. NodeDB is a server, does not build for wasm32, and will not support wasm. The consumer that needed this is NodeDB Lite's shared-crates-wasip1 job. wasm gating belongs there, not here.

@EnRaiha EnRaiha closed this Sep 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants