Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@ The format is based on Keep a Changelog and this project adheres to Semantic Ver

## [Unreleased]

### Local paper MVP qualification (NRCL-107)

- Expose experimental Kalshi data/streaming, sports matching, price-rule, recording, simulation, and journal capabilities through the existing capability matrix. Public JSON fields and existing imports remain unchanged.
- Document supported venue boundaries and executable edge cases. Extend both synthetic venues' malformed-book, exact-depth, fee/cash, timestamp-boundary, and unsupported-venue regression coverage. Novig and international Polymarket remain unsupported by the local paper pipeline; this does not enable live trading or real-data capture.

### Added
- **Exchange Core:** Added exchange abstraction contracts and normalized models in `neural/exchanges/`.
- **Kalshi Adapter Routing:** Refactored `TradingClient` to route Kalshi behavior through an exchange adapter without breaking default usage.
Expand Down
61 changes: 61 additions & 0 deletions docs/architecture/local-paper-acceptance.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
---
title: Local Paper MVP Acceptance
description: Kalshi-first edge cases, supported venue boundaries, and release-candidate gates
---

## Goal and scope

NRCL-107 qualifies a local paper MVP: reproducible Kalshi experiments, explicit
Polymarket US synthetic support, and safe rejection of unsupported behavior.
NRCL-105 supplies the Vaticor consumer migration. This acceptance contract does
not resume deferred NRCL-101/102, authorize capture, or qualify hosted/live trading.

## Executable acceptance matrix

Paths below are repository-relative. Each row names behavior and the test that
asserts it; running a suite is evidence only for those supported behaviors.

| Boundary | Required positive and negative behavior | Executable evidence |
| --- | --- | --- |
| Kalshi decimal data | Preserve decimal strings and native identity; reject malformed prices, unexpected markets, redirects, and incomplete pagination | `tests/exchanges/test_kalshi_data.py`: `test_exact_decimal_market_and_serialization`, `test_invalid_price_fails_closed`, `test_pagination_never_silently_truncates`, `test_bad_identity_and_redirect` |
| HTTP failure handling | Bounded retries for 429/5xx/timeouts; no retry on auth failure or beyond allowed Retry-After | Same file: `test_bounded_retries`, `test_no_unauthorized_or_early_retry`, `test_network_retries_exhaust_and_malformed_json_closes` |
| Book continuity | Gaps, duplicate/out-of-order sequence and malformed deltas invalidate the book; fresh snapshot restores it | `tests/exchanges/test_kalshi_stream.py`: `test_exact_snapshot_delta_removal_and_recovery`, `test_sequence_and_shape_fail_closed`, `test_bad_delta_invalidates` |
| Stream recovery | Disconnect, timeout, cancellation, or gap clears book before teardown; reconnect budget enforced; recordings replay deterministically | Same file: `test_invalidate_before_socket_teardown`, `test_gap_reconnect_recording_and_replay`, `test_retry_budget_and_invalidation` |
| Authentication boundary | Exercise local WebSocket handshake; redirects cannot forward credentials | Same file: `test_local_wire_transport_and_auth`, `test_redirect_rejected_before_credentials_forwarded`. These do not establish venue-authenticated access. |
| Normalized venue data | Both synthetic venues reject empty/one-sided/crossed books, bad ordering, stale/future timestamps, sequence gaps, malformed depth, false provenance and incomplete sessions | `tests/test_recordings.py`: `test_quality_faults_fail_before_returning_a_report`, `test_source_age_boundary_to_microsecond` |
| Exact fill and budget boundaries | Both venues cancel orders below full executable depth; exact depth fills; fees included in reserved cash down to a decimal unit; exit fees cannot make cash negative | Same file: `test_fill_or_kill_at_exact_depth_boundary`, `test_reserved_cash_boundary_includes_fees`, `test_exit_fees_never_make_cash_negative` |
| Model limits | Reject off-lot quantity, invalid numeric assumptions and unsupported venues/outcomes; expire old intents; retain unrealized positions | `tests/test_paper_recording.py`: `test_reject_invalid_model_inputs`, `test_expiry_and_terminal_cancel`, `test_open_position_remains_unrealized`; `tests/test_recordings.py`: `test_unsupported_venue_cannot_create_strategy_or_recording`, `test_no_strategy_cannot_reuse_yes_sports_proposition` |
| Causality and reconnect | A source observation before/at the intent cannot fill it; reconnect cancels intent while preserving holdings | `tests/test_recordings.py`: `test_delayed_source_cannot_fill_before_or_at_intent_time`; `tests/test_paper_recording.py`: `test_reconnect_cancels_intent_and_preserves_holdings` |
| Determinism | Same supported logic yields expected decimal traces on both synthetic venues while identities remain distinct | `tests/test_recordings.py`: `test_same_strategy_logic_uses_full_decimal_depth_across_venues`; `tests/kernel/test_replay.py` |
| Durable execution | Duplicate submissions reuse identity; process death rolls back; concurrent workers publish once; corrupt input becomes terminal failure | `tests/test_paper_worker.py`: `test_snapshot_dedup_and_restart`, `test_process_death_after_simulation_rolls_back`, `test_concurrent_processes_publish_one_result`, `test_bad_recording_is_terminal_failure` |
| Historical reads | Preserve original journal/recording bytes, reject unknown/damaged or WAL archives, retain old model results without reinterpretation | `tests/test_paper_query.py`: `test_reopen_compare_and_source_bytes_across_synthetic_venues`, `test_reads_do_not_acquire_writer_lock_or_initialize_journal`, `test_historical_models_stay_opaque_and_read_only`, `test_damaged_journals_fail_without_repairing_them`, `test_wal_archive_rejected_without_creating_sidecars` |
| Fair comparison | Reject different recordings, quantities, assumptions and unfinished jobs; preserve tiny fee products | Same file: `test_incompatible_assumptions_and_unfinished_jobs_reject`, `test_derived_fees_preserve_product_scale` |
| Settlement matching | Compare identity and rule evidence; distinguish ordinary winner, forfeit/postponement, doubleheader and partial-game cases; unknown never implies equivalent | `tests/contracts/test_sports_matching.py`: `test_customer_comparison_cases`, `test_missing_identity_on_both_sides_is_unknown`, `test_missing_rule_hash_and_incomplete_review_cannot_claim_compatibility` |
| Public support discovery | Capability names resolve to importable modules without promoting experimental APIs or loading optional stacks during kernel import | `tests/kernel/test_capabilities.py` |

## Release-candidate gates

1. Run the full SDK suite and required Ruff, Black, and mypy checks on the candidate.
Run documentation/example validators and the documentation site's typecheck/build.
2. Build the wheel from the candidate; record commit and artifact digest. Install
in an isolated environment and exercise capabilities, replay, saved jobs and
read-only comparison outside the source checkout.
3. With Vaticor's reviewed NRCL-105 integration, run installed-wheel integration
tests and browser save/run/reopen/clone/compare journeys on both synthetic venues.
Include invalid imports, incompatible comparisons and restart/archive behavior.
4. Required hosted checks and exact-candidate review must pass. Attach evidence
and limitations to Plane and the PR. Merge, release publication and deployment
remain distinct approvals and outcomes.

## Compatibility and limits

The current patch adds capability descriptions and regression coverage; it does
not migrate journals, remove 0.4.x imports, or change simulation model identity.
Existing capability JSON fields remain unchanged; consumers must accept added
capability rows. Public paper APIs remain experimental.

Real venue wire fixtures, actual account authentication, fee schedules, permitted
retention/replay, real-game quality and live order behavior require separate
evidence. Novig remains a research candidate. See the
[support matrix](./stability#local-paper-venue-support) and
[access and cost plan](../research/2026-09-10-venue-access-and-cost-plan).
3 changes: 2 additions & 1 deletion docs/architecture/meta.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
"overview",
"contracts",
"sports-matching",
"stability"
"stability",
"local-paper-acceptance"
]
}
37 changes: 37 additions & 0 deletions docs/architecture/stability.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,10 @@ matrix.
| Kernel normalization | Stable | base wheel | Public `neural.kernel` contract |
| Kernel replay | Stable | base wheel | Deterministic output and digest |
| CLI diagnostics | Stable | base wheel | JSON envelope retained |
| Kalshi market data and book streaming | Experimental | base wheel | Read-only; stream authentication supplied by caller |
| Sports matching and price-rule specification | Experimental | base wheel | Explicit identity, settlement evidence, and unsupported combinations |
| Recording replay and paper simulation | Experimental | base wheel | Synthetic normalized sports inputs; next-book fill-or-kill |
| Durable paper jobs and read-only journal | Experimental | base wheel | Local persistence; supported journal versions only |
| Kalshi auth and collection | Experimental | `neural-sdk[trading]` | May change before 1.0 |
| Paper and venue adapters | Experimental | `neural-sdk[trading]` | Paper-first; no live authority |
| Strategy and backtesting | Experimental | `neural-sdk[analysis]` | May change before 1.0 |
Expand All @@ -50,6 +54,39 @@ matrix.
Existing imports remain available during the `0.4.x` compatibility window.
New code should use `neural.kernel` for stable contracts.

## Local paper venue support

The newer local paper APIs above are distinct from the older optional trading
adapters. Their presence in the base wheel does not promote them to stable APIs.

| Behavior | Kalshi | Polymarket US | Novig |
| --- | --- | --- | --- |
| Native market discovery and decimal books in the local paper data path | `neural.kalshi`; read-only | No native capture client in this path | Unsupported |
| Native book streaming/recovery | `neural.kalshi_stream`; caller supplies signer | Unsupported | Unsupported |
| Legacy `kalshi-book/1` replay | YES/NO; provenance unverified | Unsupported | Unsupported |
| Normalized `neural-book/1` replay | Synthetic, full-depth YES team-wins | Synthetic, full-depth YES team-wins | Unsupported |
| Price-rule paper simulation and local jobs | Supported within model limits | Supported on normalized fixtures within model limits | Unsupported |
| Authenticated venue or real-game acceptance | Separate permission-gated verification | Deferred real-data pilot | Access and adapter evidence unresolved |
| Live execution acceptance | Not established by this MVP | Not established by this MVP | Not established by this MVP |

`polymarket_us` means Polymarket US. International `polymarket`, `novig`, and
unknown venue identifiers are rejected by the price-rule and recording APIs.
Existing provider-specific modules elsewhere in the package are not proof that
the local paper pipeline supports their live capture or execution paths.

The simulator runs at most one buy/exit cycle. It uses the next eligible book,
fill-or-kill semantics, a 0.01 model lot, and caller-supplied per-contract fees
on both sides. Insufficient executable depth cancels the whole order; no partial
fill is booked. Fees are exact decimal assumptions, not implemented venue fee
schedules. No queue priority, slippage model beyond recorded depth, settlement,
or profitability claim is implied. A valid live stream sequence alone is not
proof of market freshness. Normalized replay enforces source age and ordering.

See [Local Paper MVP Acceptance](./local-paper-acceptance) for executable
acceptance cases and the release-candidate gates. Additional venue support must
pass those shared cases with explicit venue-specific assumptions before this
matrix is expanded. Novig research does not enable an adapter.

## Compatibility policy

- Stable symbols retain compatible behavior within a minor release line.
Expand Down
20 changes: 15 additions & 5 deletions examples/09_daily_nba_markets.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,15 @@ async def fetch_daily_nba_markets(
if not markets.empty:
preview_columns = [
column
for column in ("ticker", "title", "home_team", "away_team", "game_date", "yes_ask", "volume")
for column in (
"ticker",
"title",
"home_team",
"away_team",
"game_date",
"yes_ask",
"volume",
)
if column in markets.columns
]
if preview_columns:
Expand All @@ -64,11 +72,14 @@ async def fetch_daily_nba_markets(
return output_path



def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(description="Fetch and store a daily NBA market snapshot.")
parser.add_argument("--status", default="open", help="Kalshi market status filter, default: open")
parser.add_argument("--limit", type=int, default=200, help="Maximum markets to fetch, default: 200")
parser.add_argument(
"--status", default="open", help="Kalshi market status filter, default: open"
)
parser.add_argument(
"--limit", type=int, default=200, help="Maximum markets to fetch, default: 200"
)
parser.add_argument(
"--output-dir",
type=Path,
Expand All @@ -83,7 +94,6 @@ def parse_args() -> argparse.Namespace:
return parser.parse_args()



def main() -> None:
args = parse_args()
asyncio.run(
Expand Down
8 changes: 4 additions & 4 deletions examples/10_daily_nba_markets_polymarket.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,9 +81,7 @@ def fetch_daily_polymarket_markets(
output_path = output_dir / f"polymarket_{sport.lower()}_markets_{snapshot_date}.csv"
markets.to_csv(output_path, index=False)

print(
f"Wrote {len(markets)} Polymarket {sport.upper()} markets to {output_path.as_posix()}"
)
print(f"Wrote {len(markets)} Polymarket {sport.upper()} markets to {output_path.as_posix()}")
if not markets.empty:
preview_columns = [
column
Expand Down Expand Up @@ -122,7 +120,9 @@ def parse_args() -> argparse.Namespace:
default="open",
help="Optional status filter applied after fetch, default: open",
)
parser.add_argument("--limit", type=int, default=200, help="Maximum markets to fetch, default: 200")
parser.add_argument(
"--limit", type=int, default=200, help="Maximum markets to fetch, default: 200"
)
parser.add_argument(
"--output-dir",
type=Path,
Expand Down
4 changes: 1 addition & 3 deletions neural/contracts/registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,9 +105,7 @@ def _semantic_details(payload: dict[str, Any]) -> list[str]:

lineage = payload["lineageRefs"]
contract_payload = payload["payload"]
lineage_identities = [
(str(ref["schemaName"]), str(ref["objectId"])) for ref in lineage
]
lineage_identities = [(str(ref["schemaName"]), str(ref["objectId"])) for ref in lineage]
if len(lineage_identities) != len(set(lineage_identities)):
details.append("lineageRefs: duplicate schemaName and objectId")

Expand Down
5 changes: 3 additions & 2 deletions neural/data_collection/polymarket_us.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,12 @@ def _extract_game_context(raw: dict[str, Any]) -> dict[str, Any]:
"home_team": home_team,
"away_team": away_team,
"game_date": raw.get("gameStartTime") or raw.get("startDate") or raw.get("endDate"),
"market_type": raw.get("sportsMarketTypeV2") or raw.get("sportsMarketType") or raw.get("marketType"),
"market_type": raw.get("sportsMarketTypeV2")
or raw.get("sportsMarketType")
or raw.get("marketType"),
}



@dataclass(slots=True)
class PolymarketUSConfig:
sport: str | None = None
Expand Down
49 changes: 49 additions & 0 deletions neural/kernel/capabilities.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,55 @@ def as_dict(self) -> dict[str, str | None]:
extra="trading",
summary="Kalshi authentication and signed HTTP clients.",
),
Capability(
name="data.kalshi",
module="neural.kalshi",
status=CapabilityStatus.EXPERIMENTAL,
extra=None,
summary="Read-only Kalshi markets and decimal books; bounded pagination and retries.",
),
Capability(
name="stream.kalshi",
module="neural.kalshi_stream",
status=CapabilityStatus.EXPERIMENTAL,
extra=None,
summary="Single-market Kalshi book recovery and recording; caller supplies authentication.",
),
Capability(
name="sports.matching",
module="neural.sports",
status=CapabilityStatus.EXPERIMENTAL,
extra=None,
summary="Sports proposition and settlement comparison; unknown rules never imply equivalence.",
),
Capability(
name="strategy.price_rule",
module="neural.strategy",
status=CapabilityStatus.EXPERIMENTAL,
extra=None,
summary="Price-rule specifications for Kalshi and Polymarket US; validation is not execution.",
),
Capability(
name="recordings.paper",
module="neural.recordings",
status=CapabilityStatus.EXPERIMENTAL,
extra=None,
summary="Legacy Kalshi replay and synthetic Kalshi/Polymarket US YES sports recordings.",
),
Capability(
name="simulation.paper",
module="neural.paper",
status=CapabilityStatus.EXPERIMENTAL,
extra=None,
summary="Offline next-book fill-or-kill, one buy/exit cycle; supplied fees, no settlement.",
),
Capability(
name="experiments.paper",
module="neural.paper_worker",
status=CapabilityStatus.EXPERIMENTAL,
extra=None,
summary="Durable local paper jobs; read-only history/comparison through neural.paper_query.",
),
Capability(
name="data_collection",
module="neural.data_collection",
Expand Down
2 changes: 0 additions & 2 deletions neural/trading/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -386,5 +386,3 @@ def _run_coro_sync(coro: Any) -> Any:
"Cannot execute sync paper-trading call inside a running event loop. "
"Use await TradingClient.place_order_async(..., paper=True)."
)


Loading
Loading