Skip to content

1.2.0 wheel: compiled binary cannot resolve its dataflow worker ($bunfs ModuleNotFound); -j silently falls back to sequential #171

Description

@rahlk

Observation

Running the 1.2.0 PyPI wheel's compiled binary with --emit neo4j -j 8 on a 2,184-file project (superset-frontend) logs

[codeanalyzer-ts] WARN graph extraction workers failed (worker error: BuildMessage: ModuleNotFound resolving "/$bunfs/root/dataflow/worker.js" ...); continuing

and then completes on the sequential path — the graph lands (154,451 nodes, all six dataflow relationship types present), but -j had no effect: the worker pool never started.

src/dataflow/pool.ts:28-29 resolves the worker as new URL(compiled ? "./dataflow/worker.js" : …, import.meta.url) when import.meta.url contains $bunfs. In the compiled binary that URL does not resolve. package.json builds with bun build ./src/main.ts ./src/dataflow/worker.ts --compile --outfile dist/cants — two entrypoints into one executable — so the worker module is compiled in, but not at the path the pool computes.

Running the same analysis from source (bun run src/index.ts …) starts the workers normally.

Why it matters

  • Every consumer of the wheel (the Python SDK pins it) silently loses level-3 parallelism; on large repos that is the difference between minutes and the 10m42s / 28.6 GB scale numbers in the ledger.
  • It is a WARN, so a CI job would not notice.

Suggested fix

Resolve the compiled worker by the entrypoint name Bun assigns inside $bunfs (or embed the worker via Bun.embeddedFiles / a ?worker import and pass the resulting URL), and add a smoke test that runs the built binary with -j 2 on test/fixtures/dataflow-app and asserts the pool started (no workers failed warning).

Definition of done

  • dist/cants -i test/fixtures/dataflow-app -a 3 -j 2 emits no graph extraction workers failed warning.
  • The release workflow's wheel smoke test exercises -j 2, not only --help.

Found while emitting a reference graph for python-sdk leg 2.5.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions