From 4fef078066c5e657a477414e96a9ea84b89ce19a Mon Sep 17 00:00:00 2001 From: John Lambert Date: Tue, 15 Sep 2026 09:11:45 -0400 Subject: [PATCH 1/7] Add complete HermitCrab optimization round record (2026-08/09) Full docs-only record of the September 2026 HermitCrab optimization round: 22 ledger attempts (2 earlier shipped wins, 12 closed as measured-does-not-pay, 3 closed before building, 5 reopened this round), the probe design needed to rebuild the measurement instrumentation from scratch, the per-mechanism disposition (9 retained mechanisms with individual records, 4 rejected with individual records), and the branch/worktree inventory. The round's own conclusion, from the disposition doc: do not merge the branch as a package (896 lines across 28 files, not reviewable as one change, and the package speedup could not be attributed to its parts); isolate what's worth isolating separately, and refocus on PR #491 and grammar hygiene where the large returns are. Kept here as a durable "do not retry this" record, not as a merge candidate. Copied from branch perf/hc-optimization-archive onto current master, replacing the stale two-file version of this PR. Minimal fixups only: the ledger, probe-design, disposition, and counts docs were read in full to check for (a) references to branches deleted since the archive was cut and (b) private corpus data. Found and fixed: the disposition doc's several references to `perf/hc-edge-prefilter`, whose own branch and archive tag have both since been deleted; the mechanism itself is not lost, since it also lives in the whole-round archive (branch `origin/perf/hc-optimization-archive`) copied in by this same commit, so the references now name that ref instead of calling the mechanism unrecoverable. The branch-wide parity sentence in the counts doc is qualified to say what `WordAnalysisSignature` actually verifies on the FLEx exports (morph counts, since `Morpheme.Id` is empty), matching the caveat the disposition doc already recorded. The finished execution-plan checklist is dropped now that its outcome is captured in the disposition record instead. No private word forms or corpus data found; the docs already record only word-list indices, counts, and ratios, per the round's own scrubbing note. No findings, numbers, or conclusions were otherwise altered. Docs only, no code changes. Co-Authored-By: Claude Opus 5 Co-Authored-By: Claude Sonnet 5 --- docs/hermitcrab-optimization-ledger.md | 283 ++++++++++++++++++ docs/hermitcrab-perf-2026-09-counts.md | 93 ++++++ docs/hermitcrab-perf-2026-09-disposition.md | 135 +++++++++ docs/hermitcrab-probe-design.md | 238 +++++++++++++++ docs/optimizations/deferred-template-clone.md | 35 +++ docs/optimizations/edge-segment-prefilter.md | 64 ++++ .../feature-struct-frozen-tree-clone.md | 28 ++ .../feature-struct-frozen-tree-equality.md | 29 ++ .../feature-struct-visited-state-reuse.md | 28 ++ docs/optimizations/flat-register-arrays.md | 25 ++ .../shape-copy-single-pass-map.md | 28 ++ docs/optimizations/shape-sharing-on-clone.md | 44 +++ .../syntactic-feature-structure-sharing.md | 32 ++ .../alternatives-memo-and-hoist.md | 15 + .../compounding-split-prefilter.md | 11 + .../final-template-prune.md | 25 ++ .../two-pass-nondeterministic-traversal.md | 14 + 17 files changed, 1127 insertions(+) create mode 100644 docs/hermitcrab-optimization-ledger.md create mode 100644 docs/hermitcrab-perf-2026-09-counts.md create mode 100644 docs/hermitcrab-perf-2026-09-disposition.md create mode 100644 docs/hermitcrab-probe-design.md create mode 100644 docs/optimizations/deferred-template-clone.md create mode 100644 docs/optimizations/edge-segment-prefilter.md create mode 100644 docs/optimizations/feature-struct-frozen-tree-clone.md create mode 100644 docs/optimizations/feature-struct-frozen-tree-equality.md create mode 100644 docs/optimizations/feature-struct-visited-state-reuse.md create mode 100644 docs/optimizations/flat-register-arrays.md create mode 100644 docs/optimizations/shape-copy-single-pass-map.md create mode 100644 docs/optimizations/shape-sharing-on-clone.md create mode 100644 docs/optimizations/syntactic-feature-structure-sharing.md create mode 100644 docs/rejected-optimizations/alternatives-memo-and-hoist.md create mode 100644 docs/rejected-optimizations/compounding-split-prefilter.md create mode 100644 docs/rejected-optimizations/final-template-prune.md create mode 100644 docs/rejected-optimizations/two-pass-nondeterministic-traversal.md diff --git a/docs/hermitcrab-optimization-ledger.md b/docs/hermitcrab-optimization-ledger.md new file mode 100644 index 000000000..28443b33f --- /dev/null +++ b/docs/hermitcrab-optimization-ledger.md @@ -0,0 +1,283 @@ +# HermitCrab optimization ledger — tried, closed, do not retry + +**Purpose: stop this work being redone.** One row per optimization attempted, what was expected, +what actually happened, and the number that settled it. If you are about to try something on this +list, read its row first — several of these look irresistible on paper and three of them have +already been independently rediscovered and re-retracted. + +Deliberately in-repo and durable, not evicted into a PR accordion. A closed avenue is only closed +if the next person can find out cheaply that it is closed. + +**This is the only report.** The working docs it was distilled from +(`hermitcrab-packed-forest-research.md`, `hermitcrab-forest-memo-plan.md`, +`hermitcrab-forest-memo-ceiling.md`, `hermitcrab-synthesis-fold-probes.md`) are chronological +process records containing retracted intermediate claims, and are deliberately **not** merged. +They are preserved only as this record. **The evidence branches were deleted after the round +closed**, along with all instrumentation: research scaffolding is not worth maintaining in two +product assemblies, since the probes gate on `volatile bool` reads inside `Matcher.cs`, a hot inner +loop in `SIL.Machine` used by every consumer. + +**`docs/hermitcrab-probe-design.md` is the replacement.** It carries the probe architecture and its +two-assembly gate trap, the eight exclusive wall-time buckets and why `synForward` must be recorded +net of cascade and battery, the per-node decomposition, the census designs, all three key designs +with field-level audits, the classifier's direction trap, and the full A/B methodology with the +three measurement failures that forced each part of it — enough to rebuild any probe in a day. The +expensive part was never the code; it was learning what to measure and which measurements lie. + +**No instrumentation is merged, by design.** The probes gate on `volatile bool` reads inside +`Matcher.cs` — a hot inner loop in `SIL.Machine` used by every consumer — which is an acceptable +cost for a measurement run and not one to carry in product code indefinitely. Rebuilding a probe +from a branch is cheap; maintaining research scaffolding in two product assemblies is not. + +## Final disposition of the September follow-up + +The follow-up branch does deliver product improvements. Its authoritative summary is +`hermitcrab-perf-2026-09-counts.md`, and each retained optimization has one record under +`docs/optimizations/`. The retained set is: deferred template cloning, feature-structure traversal-state +reuse, frozen-flat-tree equality, frozen-flat-tree cloning, single-pass `Shape.CopyTo` mapping, +copy-on-write shape sharing, flat FST register arrays, the analysis edge-segment prefilter, and +copy-on-write syntactic feature structures. + +Four attractive experiments were deliberately excluded from the final product diff and are recorded under +`docs/rejected-optimizations/`: the negative two-pass nondeterministic traversal, the inert final-template +prune, the zero-value alternatives memo/hoist, and the low-ceiling compounding split prefilter. This final +disposition supersedes intermediate “ships” language for those experiments later in this historical ledger. + +The minimized branch passed all 603 runnable tests in the conformance-inclusive suite (one expected +Windows symlink-privilege skip). A fixed three-word sample had exact output parity on Sena, Amharic, +Indonesian, Mbugwe, and Aweti; only Amharic (1.76x) and Mbugwe (1.83x) cleared the recorded spread and timing +reliability rules. The other three ratios are retained as directional evidence only. + +## Earlier ledger round: net speed delivered none + +No optimization in the "closed" section shipped, and none of them made anything faster. The +fastest thing tried was 4% **slower**. The pre-existing wins in this codebase (rows 1–3) predate +this work. What this effort produced is the measurement, the closed family, the located target, +and this ledger. + +--- + +## Earlier shipped results + +| # | Optimization | Expected | Outcome | Number | +| --- | --- | --- | --- | --- | +| 1 | Memoize analysis morphological-rule cascade | Same state reached by many rule orders; cache it | Works, but the cascade was never the cost | 2,555 expansions vs a 2,546-state floor; 1.4 s of a 30.5 s word | +| 2 | Memoize affix-template battery (same key) | Battery was 93% of wall time | **The big win.** Reduced how *often* the battery runs | 38,840 runs -> 2,581; word 30.5 s -> **6.1 s** | + +### Retracted historical prototype + +Row 3 in the original ledger claimed that shape sharing through `CloneShareFrozenShape` had shipped. No +reachable production commit carried that implementation. Its historical prototype measured 4.5–7.6% fewer +allocated bytes and no wall-time change. The September follow-up independently built a different, retained +copy-on-write design through `CloneForEngine`; see `docs/optimizations/shape-sharing-on-clone.md`. + +## Closed — measured, does not pay + +| # | Optimization | Expected | Why it failed | Number | +| --- | --- | --- | --- | --- | +| 4 | **Narrow `AnalysisStateKey`** (drop strictly-shrinking rules' un-application counts) | `MaxApplicationCount` defaults to 1, so the count multiset is "which subset of rules was used" — 2^27 states. Dropping 19 of 27 rules gives 2^8 | Shape + syntactic FS + realizational FS *already* discriminate. The count component was implied by its neighbours. A worst-case bound collapsing is not the realised state count collapsing | Realised state collapse **Sena 1.12x** (state-weighted) vs a 1.3 gate; **4 of 7** heavy probes exactly 1.00, including the 2,556-state probe. The three that moved were 1.13x, 1.16x, and **1.38x**, and all returned zero parses: R tracks failure, not size. **Reopening condition:** a grammar with genuinely high R; the census is cheap and is the right first question to ask of any new grammar | +| 5 | **Fold-step sharing memo** (share deterministic synthesis fold steps by computed value) | Probe measured 3.22x and 8.10x shareable steps; ~50% wall ceiling on two reliable fixtures | A sound key must carry the **ordered remaining trail**. Two candidates then have to agree on their entire future to share a step, which almost never happens | **hits = 0** on the best fixture; **0.96x** (4.5% *slower* — pure key-construction cost); 1.06x on the second, inside 21.9% noise | +| 6 | **Order-insensitive synthesis-input dedupe** | 9,774x duplicate synthesis inputs | HermitCrab's morphological rules are **not order-invariant**: same rule multiset, different order, different output. Merging loses parses | 2 violations on Sena, 12 independently on Indonesian. Sound version: **15–40%** | +| 7 | **Trail-position indexing of synthesis rules** | 11,445,538 of 11,445,538 rejections are "this rule is not the pending trail rule" — 100% of the histogram | Each rejection is one array index plus one reference compare. Count is not cost | 11.4M x 29 ns = 0.33 s of 143 s = **0.2%** | +| 8 | Synthesis-side length bound (Gate A) | Reject candidates that cannot reach the surface length | At the comparison point the candidate is still the bare root; its affix trail applies later inside `_synthesisRule.Apply`. Rejected valid parses | Unit suite 64 -> 34 passing. Reverted | +| 9 | Analysis length ceiling (Gate B) | Same idea, analysis side | Sound and correct. No available corpus contains the pathology it prunes | Byte-identical output, **no measurable speedup** (~4–7% slower, within noise) | +| 10 | Lexical reachability gate (Phase 5) | Prune candidates from which no lexicon root is reachable | Both reference corpora have real compounding in their deepest stratum, which disqualifies the gate everywhere | Proven **no-op** on both corpora | +| 11 | Tandem lexical intersection (T2) | Kill doomed branches early via the lexicon | The oracle only sees lexical unreachability. The expensive words fail on checks that run *after* lexical lookup succeeds | Pooled **23.5%** dead steps vs a 30% gate; one failure word at exactly **0.0%** dead | +| 21 | **Memo-key caching on the frozen `Word`** | Keys are built ~28x per distinct state, which looked like a cheap win. | High count did not imply high cost; caching would add state and invalidation complexity around negligible work. | **CLOSED:** all `AnalysisStateKey` construction and hashing on Amharic measured **3 ms of 81,707 ms = 0.0%**. | +| 12 | Pool small short-lived collections | Fewer allocations, faster | `HashSet/Dictionary.Clear()` is O(capacity); Gen0 already beats pooling at this size | -15–17% bytes but **+8.6% wall**, +12% on the parallel battery. Reverted | + +## Closed before building — motivation refuted by measurement + +| # | Optimization | Expected | Why it died | +| --- | --- | --- | --- | +| 13 | Surface-length pruning inside the synthesis fold (P2) | "The only lever on Amharic's ~160 ms per synthesis run" | Amharic's forward synthesis is **14 ms**. The 160 ms was arithmetic on an unmeasured denominator — dividing 30 s by 186 synthesis inputs and assuming the time was there | +| 14 | Nogood lattice subsumption (P4) | Nogood hits dominate positive replays **434,628 to 25,102** — the engine mostly proves subtrees *empty*, and generalising "empty" over the feature lattice would turn nogoods from points into regions | **CLOSED on direct evidence** (it was previously closed on adjacent evidence, which is why it was reopened). Two heavy Sena probes produced 16,176 expansions -> 16,176 distinct keys -> **16,176 singleton buckets, 0 subsumable (0.0%)**, and 12,595 -> 12,595 -> **12,595 singletons, 0 subsumable**. Not one pair of nogood entries even shares a `(shape, stratum, counts, non-head)` bucket, so there is nothing for a subsumption ordering to act on. Breadth across all 33 conformance fixtures agrees: 7,493 empty expansions, 7,398 distinct keys, **0 subsumable (0.0%)** pooled, every non-empty bucket a singleton. Direction checked explicitly as `earlier.Subsumes(later)` (E subsumes Q, general over specific); the reverse would be unsound. Independently reconfirms row 4 from a different angle: the keys are maximally discriminating | +| 15 | Move constraints into the analysis key (Maxwell & Kaplan's category-splitting) | Their biggest measured win: make the chart prune what the constraint solver otherwise would | Everything synthesis rejects on is root- or realization-dependent, and the root is unknown until `LexicalLookup`. Also a Sena analysis state costs ~0.73 ms while a synthesis input costs at most ~0.12 ms — **states are dearer than synthesis inputs** | + +## Reopened this round, and their outcomes + +| # | Optimization | Expected / observation | Outcome | +| --- | --- | --- | --- | +| 22 | **Analysis-side rule prefilter** | 94.4% of Sena allomorph pattern matches failed, so a cheap necessary-condition check had a large direct-work ceiling. | **RETAINED after the earlier self-time cap was refuted by direct work counts.** The original ~10% ceiling counted matcher self-time but omitted traversal allocation, feature unification, and GC booked elsewhere. The implemented edge-segment prefilter removed **87% of FST traversals**, 54% of arc checks, and 85% of traversal instances on 45 Sena words, with exact parity on those words and all 464 conformance-fixture words. See `docs/optimizations/edge-segment-prefilter.md`. | +| 16 | **`ExpandAlternatives` dedupe** | 20.7% of Sena wall, the largest bucket nobody had instrumented. Apparent census: 395,026 alternatives -> **61** distinct (0.0%), 85% of duplicates same-analysis-word | **CLOSED on the sound census.** With the pending-trail content added to the key, 395,026 -> **351,414 distinct (89.0%)** -- only **11%** are duplicates at all, and all 43,612 of them are same-analysis-word. Sound ceiling = 20.7% x 0.110 = **2.28% of Sena wall**, against a gate of >=10% to build and <5% to close. **The fourth apparent redundancy to evaporate under a complete key** | +| 17 | **Per-node cost in the analysis cascade** | 84% of the Amharic cascade is not linguistic computation; clone is 46.7% and 37.4% was unattributed | **Partly settled, remainder still open.** (a) The remainder split **MISSES its <15% gate by ~2.5x** — still ~38% on Amharic `anCascade`. Memo, Trail and **Equality each measured <0.2%**, so `Word.ValueEquals` behind the cascade's `HashSet` is *not* the missing mass, contrary to the standing hypothesis. **Two caveats the first write-up missed.** (i) The ~38% figure comes from a *contended* run — three timing probes were live — and descheduled wall time lands preferentially in the remainder while deflating every self-time share, so **CPU starvation is itself an unbracketed candidate biasing in exactly the direction observed**. (ii) The claim that GC pause time is unmeasurable is **wrong at the measurement point**: only the library assemblies are netstandard2.0; the probe harness lives in `tests/SIL.Machine.Morphology.HermitCrab.Tests`, which targets **net10.0**, so `GC.GetTotalPauseDuration()` and `GC.GetGCMemoryInfo().PauseTimePercentage` are directly callable. Next probe, gated: re-run serial — remainder <20% means contention was the missing mass and 17(a) recloses on the cheaper-clone target; still >=30% then bracket with `GetTotalPauseDuration()` deltas — ΔPause >=50% of the remainder's absolute ms confirms GC (build target converges with clone/freeze at 46.7%), <=15% refutes it and the next split is enumerator/LINQ orchestration (the 2026-07 alloc audit already flagged `ExecuteCommands` enumerator boxing); a collection census over 24 Amharic words gives gen0 8,217 / gen1 2,729 / gen2 301, an unusually high gen1+gen2 ratio consistent with — not proof of — GC as the remainder. (b) **Clone-discard fraction = 66.2%** (created 2,505, discarded 1,659), a count and therefore contention-immune, against a >=70% gate: **misses by ~4 points, so the build is "make a clone cheaper", not "defer cloning"**. Scoped to `AnalysisAffixTemplateRule`'s eager per-template clone, the one clone-before-verification site; the other four analysis-side sites clone only after a successful match and were excluded rather than blended in | +| 18 | Corpus-scope memoization | `AnalysisScope` dies per word, but `AnalysisStateKey` is word-independent by construction, so entries are reusable across a corpus. Nogood hits dominate positive replays 10-17x, making cross-word nogood reuse the cheapest available form of reuse | **CLOSED, and worse than a no-op.** Indonesian, 121 words, **serial on a clean machine: 1.015x, 1.4% wall-clock reduction** against a >=20% build gate and <5% close gate. Costs **68 MB** added peak managed memory. Cross-word reuse is real but small: 39 memo, 73 nogood, 237 template-nogood hits. Token-vs-type: **121 tokens, 121 distinct**, so no type-level dedupe is available either. **A first, contended run of the same harness reported 2.385x / 58.1%** with a 210.9% spread on the shared arm against 13.7% on the baseline; two testhosts were live. The counts were byte-identical across both runs — only the timing was contaminated, and min-of-N on a high-spread arm picks the luckiest sample. Analysis-set equality held throughout, so the key genuinely is word-independent. **Sena, first 300 words, is the decisive case: the shared scope is 28-32% *slower* than per-word scopes** (two clean serial runs, no contention: 282.7s vs 372.2s and 308.9s vs 396.7s baseline-vs-shared; deterministic counters byte-identical across both runs), and costs **~29.3 GB** added peak managed memory against a 2 GB budget. Root cause: `AnalysisScope.MaxMemoEntries` (100,000) is a *per-scope*, not per-word, budget. Two heavy early words (#74, #76 of 300) alone fill the shared `Memo` table to the cap and `TemplateMemo` to 55,071; every word after that loses the free within-word memoization a fresh per-word scope would have given it for nothing, because `Store` silently no-ops past the cap (see its comment) with no eviction. Cross-word hits (shared minus baseline) are **negative** — memo -77,648, nogood -747,607 — i.e. sharing produced *fewer* total hits than 300 independent fresh-table runs would have on their own; the small `+820` template-hit / `+9` template-nogood gain does not come close to covering it. Amharic, capped at 28 words (word 29 hangs, excluded per standing guidance): **0.9% wall-clock reduction**, table nowhere near the cap (Memo 1,251, TemplateMemo 652 of 100,000), consistent with row 17's finding that Amharic's cost is dominated by clone/allocation per node rather than distinct-state count, so avoiding a handful of cross-word re-derivations barely moves wall time. Token-vs-type on all three corpora, at the runs' own sizes and on each corpus's full word list: **zero duplicate tokens** — whole-word caching is not available as a cheaper alternative anywhere here. No analysis-set divergence on any corpus, at any scope regime: the stop condition that would have put rows 1-3 in question did not fire. **A shared-but-capped memo table is not "the same optimization, smaller ceiling" — past the cap it actively takes away the free per-word memoization the product already has, and retains every live entry for the run's full duration instead of per-word, which is where the 29 GB comes from.** Any future attempt needs real eviction (LRU, generational, or per-word sub-budgets) before this is worth re-measuring, and even then the >=20% gate looks unreachable given Indonesian's ceiling under an uncapped-in-practice regime was already only 1.4% | +| 19 | Contexted constraints / abstract feature-only replay | Pack without merging states by replaying only shape-independent constraints. | **CLOSED.** Cost-weighted P1b die-point histograms put shape-free work at **6.0%** and **0.0%** on two heavy Sena probes and **0.4%** on 28 Amharic words; all fire the <30% close gate. All 33 conformance fixtures were measured for breadth: 24 close, 3 nominally fire the build gate, 3 are undecided, 2 have zero rejections, and 1 has zero words. The three build-gate fixtures are all sub-50 ms and cannot carry timing claims. One reliable fixture confirms that count and cost can diverge, but die-point rejection cost itself is only ~0.1–0.2% of wall on the real corpora. Abstract replay would optimize a phase that costs approximately nothing on the corpora treated as ground truth. Not built. | +| 20 | **Row 5's family on `GenerateWords`** (generation, ~100% synthesis share vs parsing's ~5%/~0.3%) | Row 20 was unmeasured; generation's share is ~100% so row 5's fold ratios would apply at face value if they transferred. `UseSynthesisFoldMemo` had never been wired into `GenerateWords` (only into the parse-then-synthesize path) — wired in under the same untraced/`MaxDegreeOfParallelism==1` restriction to make this measurable at all | Hits are no longer uniformly zero: 3 of 23 lookup-reaching fixtures clear the 5% gate (diacritic-segments 33.3%, suffixing-vowel-harmony 28.6%, suffixing-evidential-adjacency-chain 37.7%; a 4th sits just under it at 4.8%), reopening row 5 narrowly for generation. But the reopened case recloses immediately: the A/B on the best fixture shows the wall-clock "win" sitting inside its own noise floor, and the pooled hit rate is dominated by `deep-optional-affix-nesting` (925 of 1,613 lookups) at hits=0, reproducing row 5's exact 4.5%-slower key-construction cost on the identical fixture. Parity: 0 divergences, 1,236 generations, 32/33 fixtures (33rd throws identically both sides). Best case: 130 hits / 5 reps (37.7% hit rate), **7.8%** wall-clock reduction inside a **22.2%** off-arm spread — not a result. 19/23 lookup-reaching fixtures at exactly **0%**; pooled hit rate **2.67%** (43/1,613), skewed by the one large zero-hit fixture. | + +--- + +## The lesson that generalises + +Three independent measurements, three boundaries, same collapse: + +| measurement | key used | apparent | sound | +| --- | --- | --- | --- | +| synthesis-input dedupe (#6) | order-insensitive | 9,774x | 15–40% | +| fold-entry census | trail position only | 6,476x | not established | +| fold-step sharing (#5) | trail position only | 3.22x / 8.10x | **hits = 0** | + +**The redundancy in HermitCrab's synthesis is apparent, not real. The trail is what makes each +step distinct, and every measurement showing large shareable work is measuring a key that omits +it.** The thing that makes sharing visible is the thing that makes sharing wrong. + +That closes a *family*: packed parse forests **of the merge-by-key kind**, fold-step sharing and +synthesis-input dedupe all require distinct derivations to converge on a genuinely identical +state. In this engine they do not converge -- the same fact as the rules being non-order-invariant, +seen from the other side. This argument did not itself close row 19 because a contexted-constraint forest +packs without merging states; row 19 was later measured independently and closed by its die-point costs. + +## Where the time actually is + +Measured with eight exclusive buckets summing to wall. Amharic is a single run (`unaccounted` +0.1%). **Sena is spliced from two runs -- see the note in its row.** + +| grammar | breakdown | +| --- | --- | +| **Sena** heavy words | battery **51.4%**, cascade 18.4%, all synthesis ~5% -- *pre-`ExpandAlternatives` run (143,303 ms, unaccounted 20.1%)*; `ExpandAlternatives` **20.3%**, unaccounted 1.5% -- *later run (112,713 ms)*. **Two runs: not co-summable.** One re-run would put all Sena shares on one denominator | +| **Amharic** | analysis cascade **~95%** of analysis, which is 99.5% of wall; **all synthesis 0.3%** | + +**Every optimization in this ledger that worked reduced how many nodes are visited. None touched +what a node costs.** Amharic spends **~170 ms per analysis state**. (The *measured* state-count floor -- 2,555 +expansions against 2,546 states -- is a Sena heavy probe. For Amharic, near-floor behaviour is +**inferred** from R ~ 1, which compares the full key against the narrowed key, not against a +semantic floor.) + +### Inside one node (Amharic, 28 words, both probe layers live) + +| bucket | `anCascade` 81,707 ms | `anBattery` 7,368 ms | +| --- | --- | --- | +| clone / freeze / allocation | **46.7%** | 41.9% | +| **remainder (unattributed)** | **37.4%** | 15.4% | +| matcher traversal | 12.5% | **39.3%** | +| FeatureStruct algebra | 3.4% | 3.5% | +| memo-key construction | 0.0% | 0.0% | + +Censuses: 27,385 rule attempts — guards reject 19.5% (application-count 6.3%, unifiability 13.2%), +**80.5% reach matching**. Of 25,178 allomorph pattern matches, **1,938 succeed (7.7%)**, 23,240 +fail (**92.3%**). + +**The headline: only ~16% of the Amharic analysis cascade is *identified* as linguistic +computation** -- matching 12.5% plus feature unification 3.4%. Clone/freeze is **46.7%** and +**37.4% is unattributed**, which is unknown rather than known-non-linguistic. The defensible +statement is that at most a sixth of the cascade has been shown to decide linguistic questions; +of the rest, half is measured (clone) and half is still open. + +**Gate outcomes, written before the run.** "Any bucket >=50% becomes the sole build target" — clone +is 46.7%, does not fire. "Pattern matching dominates *and* >=80% of attempts fail -> build a +prefilter" — 92.3% do fail, but matching is 12.5%, so it does not dominate; does not fire. +**"Remainder >=30% means the decomposition is incomplete and that is the result" — 37.4%, FIRES.** + +So there is no declared build target yet. The next probe splits the 37.4%: `RuleCascade` / +enumerator orchestration, trail dictionary operations, LINQ allocation, memo lookups, GC. + +**A caution on the 46.7% before anyone acts on it.** `Word.Clone` being half the cascade does not +by itself mean cloning is waste — the cascade forks a `Word` per rule application precisely so it +can explore alternatives without mutating shared state. The deciding question is what fraction of +clones are *discarded unused*: cloned, rejected by a guard or a failed match, thrown away. Given +92.3% of pattern matches fail, that fraction has now been **measured at 66.2%** (row 17b), which misses the >=70% gate and points at the cheaper-clone build rather than the defer-clone one. If most +clones are discarded, the lever is **not cloning until a rule commits**. If most survive, the lever +is **making a clone cheaper** (copy-on-write shape, which the RUSTIFY branch already explored). +Different builds; measure first. **Gate: probe the clone-discard fraction; >=70% discarded means +the defer-clone build, below that the cheaper-clone build.** + +## Two durable code constraints + +Neither is expressible in code, and both bite silently. + +1. **`Word.ReplayOnto` does not splice `_mrulesUnapplied`.** It splices `_mruleApps` and + `_nonHeadApps` only. This is safe *solely* because `AnalysisStateKey` includes the per-rule + count multiset, which guarantees a memo hit has identical arrival and stored-arrival counts. + **Anyone narrowing that key breaks correctness with no test failure.** Fix, if the key ever + changes: store arrival counts on `MemoEntry` and compute `stored - storedArrival + query` — a + no-op under today's key, which is also how to test it. This is the one code change from this + work worth making on its own merits, as defensive hardening. +2. **HermitCrab's morphological rules are not order-invariant.** Same pending-rule multiset, + different application order, different synthesis output — 2 cases on Sena, 12 independently on + Indonesian. This bounds every packed-readout scheme. It is the assumption + Maxwell and Kaplan (CL 19(4):571–590) rely on and this engine does not satisfy. + +## Calibration record — predictions vs outcomes + +Written before the measurements, scored after. Kept because the misses are informative. + +| prediction | outcome | +| --- | --- | +| Sena key collapse 2–4x | **1.12x** — wrong, and it was the number that decided the project | +| Indonesian key collapse 1.0–1.3x | 1.17x — right | +| Amharic "most likely to disappoint" | best static classification of the three (31 of 36 rules) — wrong | +| Fold sharing ~50% wall ceiling | **0.96x realised** — wrong; the ceiling was measured with a key that would lose parses | +| Amharic synthesis-bound at ~160 ms/run | forward synthesis is **14 ms** — wrong; arithmetic on an unmeasured denominator | + +## Method rules earned here + +1. **Counting is not timing.** Three published findings in this work were retracted, all arithmetic + on an unmeasured denominator: "synthesis is the bottleneck" (candidate counts), "Amharic is + synthesis-bound" (30 s / 186 inputs), "fixtures only look synthesis-heavy because they are small" + (divided by `synForward`, which excludes the buckets the work runs in). +2. **A ratio measured with an incomplete key is not a ratio.** See the table above, three times. +3. **Warm-up dominates small fixtures.** A first A/B here showed 1.53x purely from JIT; the off-arm + alone varied 42.7% between its own first and second sample. Discard warm-up, take min of N, + interleave arms, and print the off-arm spread as a noise floor beside every speedup. +4. **Sub-50 ms fixtures cannot carry a timing claim.** One 10 ms fixture's share moved 1.3% -> 14.5% + between two runs of identical code. +5. **Three grammars minimum, reported unpooled.** Maxwell & Kaplan measured a 100x swing between two + variants of one grammar. A pooled average across fixtures of different size has had to be + retracted twice in this project. +6. **Min-of-N is only valid when the arm's spread is small relative to the effect.** Row 18's + first run read 2.385x with a 210.9% spread on one arm and 13.7% on the other; serial and clean + it is **1.015x**. Min-of-N picks each arm's luckiest sample independently, so a high-spread arm + wins spuriously. Print the spread beside every speedup and refuse a number whose spread exceeds + it. +7. **Never dispatch timing-sensitive probes concurrently.** Three at once put four testhosts on the + machine and invalidated the bucket percentages in rows 17 and 22 as well as row 18's headline. + Counts survive contention; shares and speedups do not. +8. **A shared bounded cache without eviction is not a smaller version of the optimization** — it + competes with itself for a fixed budget and can land strictly worse than no sharing. Row 18's + shared scope produced *negative* cross-word hits. +9. **Never aggregate the test logger's console output.** It prints every line twice, the second + copy space-prefixed; a `^\s*` grep doubles every total. This produced a 2x error in row 20's + first write-up. +10. **Search completeness is never traded for speed.** HermitCrab is the permanent fallback engine + behind the FST work. +7. **A bucket reporting exactly `0.00` is broken until proven otherwise.** `NodeCostProbe.Enabled` + was declared and read in `Matcher.cs` but set by nothing — two gates in two assemblies, one + wired. Both lower-layer buckets read `0.00ms`, which looks like "measured and negligible" + rather than "never ran", and their time silently inflated the remainder. Wire cross-assembly + probe gates from a single place. +8. **Always carry an explicit remainder column.** It caught the wall-split misattribution and the + dead probe gate. Four of the five retractions in this work would have been caught at the point + of measurement by a remainder that did not sum. + +--- + +## Addendum, September 2026: the round that followed, measured by counts + +Branch `perf/hc-defer-template-clone` (on upstream master, after the merged memoization PR 456) re-ran +this ledger's open questions with deterministic step counters instead of wall time. Its records live +with the code: `docs/hermitcrab-perf-2026-09-counts.md` and one file per change under +`docs/optimizations/`. Corrections to this ledger that fell out of it: + +- **Row 3 ("shape sharing shipped") was not true on master.** No reachable commit carried it; master's + `Word.Clone` deep-copied the Shape and every node feature structure. The follow-up independently implements + a different design: internal `CloneForEngine` copy-on-write with un-share at mutation sites, while public + `Word.Clone` remains a deep mutable clone. A shared shape is frozen, so a missed mutation site throws. +- **Row 22's ~10% cap for an analysis-rule prefilter was wrong by an order of magnitude.** It was + computed from matcher *self-time*. Counted directly, an edge-segment prefilter (patterns are anchored + at both ends and matched by unification, so the first/last literal segment must unify with the + shape's first/last segment) removes **87% of FST traversals** and 54% of arc checks on 45 Sena words, + with identical analyses on every Sena word and all 464 conformance-fixture words. +- **"memo-key construction 0.0%" (row 21) was wrong at the profile level.** Hashing each new word's + syntactic feature structure for `AnalysisStateKey` is ~15% of CPU on the heaviest Sena word; the key + itself is cheap but freezing a fresh clone of a nested feature structure is not. +- **Row 16's complete-key result still stands.** An experimental `ExpandAlternatives` memo/hoist did not + reduce clones and was removed. The final branch has no alternatives-expansion change relative to the + pinned production baseline; the intermediate 75% counter reduction is not a shipped result. +- **The ledger never cited John Maxwell's work**: PR 491 (final-template prune in analysis) and the + 2025 `investigate_linear_traverse` branch. Both were tested. The prune was inert on every local corpus + because each has a partial morpheme, and the traversal measured negative by counts (instances used 2.3x) + because the current traverser already pools instances. Both implementations and their toggles were removed; + their rejected-lead records remain. +- **Wall time on the measurement machine moved 2x between windows with identical binaries** (a session + transition in between). Every verdict in this ledger that rests on a wall-time delta under ~20% should + be read with that in mind; the round's method note is to count steps on the sequential path. diff --git a/docs/hermitcrab-perf-2026-09-counts.md b/docs/hermitcrab-perf-2026-09-counts.md new file mode 100644 index 000000000..16cc86354 --- /dev/null +++ b/docs/hermitcrab-perf-2026-09-counts.md @@ -0,0 +1,93 @@ +# HermitCrab optimization round, September 2026: final disposition + +This is the final record for the count-driven round that followed +`hermitcrab-optimization-ledger.md`. Temporary counters were used to choose targets and were then removed; +the final branch carries only product changes, focused tests, the parity/timing harness, and these records. + +## Measurement scope + +The exploration used sequential parsing, one warm-up followed by repeated samples, and canonical analysis +signature sets as the parity gate. Deterministic work counts came from 45 Sena words (five known-heavy probes +plus the first 40 list entries) and 464 words from 33 conformance fixtures. Timing was never run concurrently. +The final branch-wide sample compares production commit `d7347226` (harness-only baseline tip `4f677735`) +with candidate code tip `7370e2dd`. + +The probe counters are intentionally absent from the final diff. They put volatile gates in hot code across +two assemblies and would create permanent maintenance cost for a completed experiment. The reconstruction +design remains in `hermitcrab-probe-design.md`. + +## Retained optimizations + +Each retained mechanism has one record with its invariant, implementation, evidence, parity coverage, +portability, and limits: + +| Optimization | Principal evidence | Record | +| --- | --- | --- | +| Deferred template clone | 1.054x on the 45-word Sena battery; exact parity | `optimizations/deferred-template-clone.md` | +| Feature-structure traversal-state reuse | equality/freezing allocation left the hot profile; not isolated | `optimizations/feature-struct-visited-state-reuse.md` | +| Frozen flat-tree equality | removes visited sets from the proved tree case; not isolated | `optimizations/feature-struct-frozen-tree-equality.md` | +| Frozen flat-tree clone | removes the copies map from the proved tree case; not isolated | `optimizations/feature-struct-frozen-tree-clone.md` | +| Single-pass `Shape.CopyTo` mapping | removes a second enumeration/dictionary build; not isolated | `optimizations/shape-copy-single-pass-map.md` | +| Copy-on-write shape sharing | clone 36% → 9% of profile; peak working set 11 GB → 6 GB | `optimizations/shape-sharing-on-clone.md` | +| Flat FST register arrays | `CreateInstanceMDArray` 11.5% → 0.2% of profile | `optimizations/flat-register-arrays.md` | +| Analysis edge-segment prefilter | 87% fewer FST traversals and 54% fewer arc checks on 45 Sena words | `optimizations/edge-segment-prefilter.md` | +| Copy-on-write syntactic feature structures | removes repeated clone/freeze/hash work measured at ~15% of a heavy Sena profile | `optimizations/syntactic-feature-structure-sharing.md` | + +Only the deferred clone was timed alone. Profile movement and deterministic work removal justify the other +retained mechanisms; the five-language result below measures the branch as a package and must not be divided +among them. + +## Rejected or removed experiments + +| Experiment | Deciding evidence | Final state | +| --- | --- | --- | +| Two-pass nondeterministic traversal | 2.3x more traversal instances, only 1.6% fewer arc checks, 27% more clones | removed; `rejected-optimizations/two-pass-nondeterministic-traversal.md` | +| Final-template interleaving prune | zero prunes on Sena, Amharic, and Mbugwe because partial morphemes disable it | removed; `rejected-optimizations/final-template-prune.md` | +| `ExpandAlternatives` memo/hoist | word clones unchanged at 4,747,013; no retained difference from production ordering | removed; `rejected-optimizations/alternatives-memo-and-hoist.md` | +| Compounding split prefilter | only ~2% of clones and ~9% of FST traversals were in scope | not built; `rejected-optimizations/compounding-split-prefilter.md` | + +All associated production switches, alternate traversal code, and research counters were deleted. The two +retained internal toggles exist solely for correctness/performance A/B of the edge prefilter and syntactic +feature-structure sharing. + +## Five-language branch-wide sample + +Each row used three fixed word-list indices and two reverse-ordered process rounds (baseline/candidate, then +candidate/baseline). Every process parsed sequentially, discarded one warm-up, and recorded three repetitions, +for six measured samples per arm. Times are the sum of each word's minimum sample across both rounds. Baseline +and candidate used identical grammar names and word keys; `scripts/hc-bench-compare.ps1` compared error +outcomes and rejected missing keys or signature differences. All ten arm comparisons passed the legacy +`WordAnalysisSignature` check, which on these FLEx-exported grammars compares morph counts only (`Morpheme.Id` +is empty; see `hermitcrab-perf-2026-09-disposition.md`), and the saved outputs contained zero parse or load +errors. + +| Language | Word-list indices | Baseline ms | Candidate ms | Ratio | Interpretation | +| --- | --- | ---: | ---: | ---: | --- | +| Sena | 49, 52, 74 | 1,000.7 | 448.3 | 2.23x | directional: 0/3 probes clear both the 50 ms and spread rules | +| Amharic | 8, 9, 267 | 6,410.0 | 3,646.3 | **1.76x** | stable: 3/3 probes clear both rules | +| Indonesian | 0, 9, 44 | 99.0 | 56.0 | 1.77x | exploratory: 0/3 probes clear both rules | +| Mbugwe | 0, 65, 1272 | 8,979.4 | 4,901.6 | **1.83x** | stable: 3/3 probes clear both rules | +| Aweti | 2, 3, 27 | 134.2 | 54.4 | 2.47x | exploratory: 0/3 probes clear both rules | + +The ratios are deliberately unpooled and are not extrapolated to whole corpora. An initially selected heavy +Sena trio did not complete the baseline within five minutes, and an Aweti sample containing index 0 crossed +two minutes; both whole testhosts were terminated and replaced symmetrically before any comparison. This is +sampling evidence, not a claim about those excluded words. + +## Correctness gate + +The minimized code tip was merged into the conformance integration branch and run through the complete +HermitCrab conformance-inclusive project: + +- NUnit discovered and ran 604 non-explicit tests: 603 passed, zero failed. +- One symlink-security test was skipped because the Windows host lacks symlink privilege. +- Fixture manifests, DTD/schema checks, source/graph audits, 452 fixture rows, and 348 coverage items passed. +- The ordinary HermitCrab suite also passed with syntactic feature-structure sharing disabled. + +## What remains + +The edge prefilter leaves roughly 44% of remaining traversals successful, so another edge check has little +headroom. Compounding construction is too small on the measured heavy word. Alternatives and packed-readout +schemes remain bounded by the ordered pending trail. The next credible large change is a persistent/shared +trail representation that can avoid memo-replay and alternative-materialization clones without merging +linguistically distinct derivations; it is a data-structure redesign, not a safe incremental tweak. diff --git a/docs/hermitcrab-perf-2026-09-disposition.md b/docs/hermitcrab-perf-2026-09-disposition.md new file mode 100644 index 000000000..596c4e7cf --- /dev/null +++ b/docs/hermitcrab-perf-2026-09-disposition.md @@ -0,0 +1,135 @@ +# HermitCrab optimization round, September 2026: final disposition of the branch + +**Date:** 2026-09-03. **Branch:** `perf/hc-optimization-pr` @ `179708d2`, renamed `perf/hc-optimization-archive`. +**Decision:** do not merge the branch. Isolate one mechanism onto master as its own small PR; archive the rest with +its records. Re-focus on sillsdev/machine PR #491 and on grammar hygiene, which are where the large returns are. + +## Why the package is not merged + +- 896 source lines across 28 files; about a third inside `SIL.Machine.FeatureModel.FeatureStruct`, the rest in FST + traversal internals, `Word`, `Shape`, and the analysis rules. Not reviewable as one change. +- Only one mechanism (deferred template clone, 1.054x) was ever timed alone. The package figure (1.76x Amharic, + 1.83x Mbugwe) cannot be attributed to its parts (`hermitcrab-perf-2026-09-counts.md`). +- The parity gate that blessed the package compared morph counts, not analyses: `MorpherTests.WordAnalysisSignature` + joins `Morpheme.Id`, which comes from `` and is empty for every FLEx export used. On Sena, Mbugwe + and Amharic the signature therefore degenerates to a morph count per analysis, so every earlier "exact parity" + claim on those grammars verified counts, not analyses. Anything re-measured on them needs a + gloss + allomorph-index + syntactic-feature-structure signature instead. +- Meanwhile PR #491's mechanism, measured with the guard lifted (`AlwaysEnforceFinalTemplates = true`), is + 8–33x on Mbugwe and Sena with identical analysis sets, and 298x on the single heaviest Sena word; Amharic + barely moves because its per-word cost is not in the template interleaving. The branch is worth keeping only + where #491 cannot fire (Amharic-shaped grammars), and only in its most reviewable piece. + +## Disposition per mechanism + +| Mechanism | Lines / files | Evidence on the branch | Disposition | +| --- | --- | --- | --- | +| Analysis edge-segment prefilter | ~120 in 2 rule files + 9 in `Morpher` + tests | 87% fewer FST traversals on 45 Sena words (deterministic counts); one invariant; internal toggle | **Isolated onto master** as `perf/hc-edge-prefilter` (`be7f241d`), see below — **that standalone branch and its own archive tag were later deleted, but the mechanism is preserved in the whole-round archive branch `perf/hc-optimization-archive`** | +| Copy-on-write `Shape` sharing | `Word`, `Shape`, every clone-then-mutate site | clone 36% → 9% of profile; peak working set 11 GB → 6 GB | Archived. Aliasing rules are subtle; re-open only if memory becomes the constraint. Commit `dfd1be94`, `e18ca86d` | +| Copy-on-write syntactic `FeatureStruct` sharing | `Word`, `Morpher`, rules | ~15% of a heavy Sena profile; never isolated | Archived. `4a6a47ed`, `7f9fdef3`, `da35ad4e`, `f08fefa7` | +| `FeatureStruct` equality/clone/freeze fast paths, visited-set pooling | 312 lines in core FeatureModel | "not isolated" | Archived. `325f8419`, `424a3c23`, `b23a7c99`, `00ece2ba`, `2f4dabe0` | +| Flat FST register arrays | FST internals | `CreateInstanceMDArray` 11.5% → 0.2% of profile | Archived. `b1692a7a` | +| Deferred template clone | `AnalysisAffixTemplateRule` | 1.054x alone | Archived. `57703d6b`, `24d75c82` | +| `ExpandAlternatives` memo/hoist | `Morpher` | no retained difference | Already removed on the branch. `d8886b0c`, `f4195be1` | +| Two-pass nondeterministic traversal | FST | negative | Already removed. `c3366fcc` | +| Final-template prune (own version) | analysis rules | zero prunes under the guard | Already removed; superseded by #491 work. `7e6ec59d` | +| PerfBench / compare script / ParseCounters | tests, scripts | harness | Archived. Its signature is degenerate on FLEx exports; any revival must switch to a gloss/allomorph/feature-structure signature first | +| Research records (`docs/optimizations/*`, `docs/rejected-optimizations/*`, ledger, counts, probe design, this file, the #491 re-review) | docs | — | Kept on the archive branch as the record of what was tried and why | + +## The isolated prefilter + +**Status update:** `perf/hc-edge-prefilter` was deleted after this round closed, and its own archive tag was +later deleted too, but the mechanism is not lost: it remains on the whole-round archive +branch `perf/hc-optimization-archive`, which still carries +`Morpher.EdgePrefilterEnabled`, the edge constraints in `AnalysisAffixProcessRule`, and +`AnalysisEdgePrefilterTests`. The description below is kept as a standalone record of the isolated, reviewable +form, in case that smaller diff is preferred over pulling the mechanism out of the full archive again. + +Branch `perf/hc-edge-prefilter`, worktree `.worktrees/prefilter`, one commit `be7f241d` on `origin/master` +(`d7347226`). Content: `AnalysisAffixProcessAllomorphRuleSpec` captures the left/right top-level `Constraint` +feature structures at compile time; `AnalysisAffixProcessRule.Apply` rejects an allomorph before its matcher runs +when the shape's first/last Segment node provably cannot unify with them, using the matcher's own `IsUnifiable` +call and `UseDefaults`; optional edge nodes and non-Constraint edges always pass; tracing emits the same +`MorphologicalRuleNotUnapplied` the matcher would have. `Morpher.EdgePrefilterEnabled` (internal, default true) is +the same-binary A/B switch. The research counters were not carried over. `AnalysisEdgePrefilterTests` (final branch +version, no counter dependency) came with it. HermitCrab suite: 104/104. + +### Standalone measurement (same binary, toggle off = master behavior, toggle on; sequential, memo on) + +Signature: gloss + allomorph index + syntactic feature structure, compared as multisets. Counters: throwaway +increments at the two nondeterministic `Traverse` entries and at `CheckInputMatch` (not committed). + +| Grammar | Words | Wall ms off → on | FST traversals off → on | Arc checks off → on | Parity | +| --- | ---: | ---: | ---: | ---: | --- | +| Sena | 42 (49,52,74,0–39; one invalid line skipped) | 71,296 → 69,900 (−2%) | 8,760,382 → 1,270,122 (−85%) | 12,618,566 → 6,004,919 (−52%) | identical 42/42 | +| Mbugwe | 22 (0,65,1272,1–19) | 110,609 → 96,293 (−13%) | 28,054,872 → 11,419,808 (−59%) | 36,811,510 → 25,960,854 (−29%) | identical 22/22 | +| Amharic | 10 (8,9,267,5,10,12,15,20,25,30) | 111,617 → 103,403 (−7%) | 145,050 → 129,802 (−10%) | 16,705,704 → 15,777,072 (−6%) | identical 10/10 | + +**Reading.** The count reduction reproduces (Sena −85% traversals, matching the branch's −87%), and parity holds +under a real signature. But the traversals removed are the cheap ones: they would have died at their first arc +check, so wall time moves only 2–13%. The branch's package figure (1.76x Amharic, 1.83x Mbugwe) therefore came +mostly from the allocation, clone, and GC work, not from this mechanism. Standalone, the prefilter is correct, +small, and reviewable, but it is **not a large return**. Recommendation at the time: keep `perf/hc-edge-prefilter` +ready as an optional small PR (7–13% on Bantu grammars, where it stacks with #491's prune on the surviving +branches), and do not spend review budget on it ahead of #491. The standalone single-commit branch is deleted +(see status note above); a PR now starts from the mechanism preserved in the whole-round archive +(branch `perf/hc-optimization-archive`) rather than resuming +that branch. + +Raw logs: session scratchpad `pf-sena.log`, `pf-mbugwe.log`, `pf-amharic.log`. + +## Branch and worktree inventory for the round (2026-08-20 to 2026-09-03) + +Every branch below is local unless marked. Worktrees for archived branches were removed on 2026-09-03; the +branches remain. + +| Branch | Tip | What it is | State | +| --- | --- | --- | --- | +| `perf/hc-optimization-archive` (was `perf/hc-optimization-pr`) | `b58b1642`+ | the whole round, code + records | archived, unmerged; worktree `.worktrees/ledger` | +| `perf/hc-defer-template-clone` | `4a6a47ed` | working branch of the round | fully contained in the archive; worktree removed | +| `perf/hc-engine-alloc` | `f22f88a8` | allocation/two-pass side branch | fully contained in the archive; worktree removed | +| `perf/hc-conformance-check` | `0799a977` | archive merged into the conformance integration branch for the 603-test gate | verification only; worktree `.worktrees/hc-conf` can go | +| `perf/hc-edge-prefilter` | `be7f241d` | the one isolated mechanism on `origin/master` | optional small PR at the time; **branch and its own archive tag have both since been deleted, but the mechanism is preserved on branch `perf/hc-optimization-archive`** | +| `pr-491` | `96c97f76` | jtmaxwell3's PR head | worktree `.worktrees/pr491` holds uncommitted throwaway counters + `FinalTemplateUpperBound` harness | +| `docs/hc-optimization-ledger` (remote, PR #490) | `cc806613` | the ledger, now 23 rows | open docs PR | +| `feature/forest-memo`, `feature/synthesis-fold-probes`, `feature/synthesis-fold-sharing`, `feature/per-node-cost` | — | evidence branches of the ledger round | see `hermitcrab-optimization-ledger.md` | +| `investigate-memoization-rule-order` | `4f5aeeb4` | despite the name, conformance-suite commits | belongs to the conformance line, not this round | + +Records in this directory: `hermitcrab-optimization-ledger.md` (23 attempts), `hermitcrab-perf-2026-09-counts.md` +(count round), `hermitcrab-probe-design.md`, `optimizations/*` and `rejected-optimizations/*` (one record per +mechanism), and this file. The round's execution checklist has been folded into this record; it is not kept as +a separate plan document. + +Four further records of the round were deliberately left out of this set, because they concern another author's +open PR and a sibling project rather than this round's own mechanisms. They are described here so that nothing +depends on reading them, and they remain on the `perf/hc-optimization-archive` branch: + +- `pr491-final-template-prune-upper-bound.md` — the measurement record behind the 8–33x figures above: PR #491's + prune with its partiality guard lifted, per-grammar wall time, affix-rule applications and template entries, + plus the analysis-set parity check and the signature caveat quoted earlier. It also proves the guard + over-broad, since partial *lexical entries* can never rescue a pruned branch (only partial rules can, and only + at or below the template's stratum). +- `pr491-review-draft.md` — the review comment drafted from that record: the win with the guard lifted, a + duplicate-parse bug when enforcement is active (an all-slots-skipped clone differing from its input only by + `FinalTemplateState` survives the `ApplyTemplates` equality filter), the rules-only per-stratum guard, and + state transitions that run even while the prune is disabled and cost memo hits. +- `pangloss-final-template-prune-handoff.md` — the same prune handed to PanGloss (the Rust port) for + implementation there. +- `pangloss-hc-rust-speedups-handoff-2026-09-02.md` — speedup leads handed the other way, from PanGloss into + this round. Archived; its section 4 is superseded. + +Privacy: word forms and glosses from the private grammars appear only as word-list indices and rule ids in these +records. The raw A/B logs (which contain word forms) live only in the session scratchpad and are not to be +committed. The main checkout's `.git/info/exclude` now lists the local grammar backups, `Mbugwe.xml`, the Aweti +word list, and the rulestats HTML report so they cannot be staged by accident. + +## What to do next, in order + +1. Fix and merge PR #491 (duplicate parses, memo splitting while disabled, entry-counting guard). Draft comment in the + re-review record. +2. Classify the unclassified affixes in the FLEx projects (Mbugwe 1, Amharic 1, Sena 6); that turns #491's default + prune on. Add the finding to the grammar advisor. +3. ~~Open the prefilter PR from `perf/hc-edge-prefilter`~~ — that branch and its own archive tag are both + deleted; a PR now starts from the mechanism preserved on branch + `perf/hc-optimization-archive` rather than resuming the deleted branch. +4. Port the prune to PanGloss per the handoff (`HANDOFF-pangloss-final-template-prune.md`, session scratchpad). diff --git a/docs/hermitcrab-probe-design.md b/docs/hermitcrab-probe-design.md new file mode 100644 index 000000000..eb6275d85 --- /dev/null +++ b/docs/hermitcrab-probe-design.md @@ -0,0 +1,238 @@ +# HermitCrab probe design — enough to rebuild it + +Companion to `hermitcrab-optimization-ledger.md`. That document records *what was tried and why it +failed*. This one records *how it was measured*, in enough detail to reconstruct the instrumentation +and the keys from scratch. + +**Why this exists:** the seven probe branches that produced the ledger's numbers were deleted after +the round closed. Research scaffolding in two product assemblies is not worth maintaining — the +probes gate on `volatile bool` reads inside `Matcher.cs`, a hot inner loop in `SIL.Machine` used by +every consumer. So the code is gone and this is the design record. Anything here can be rebuilt in a +day; the expensive part was learning what to measure and which measurements lie. + +--- + +## 1. Instrumentation architecture + +Two probe classes, in two assemblies, with **two independent gates**. + +`SynthesisProbe` in `SIL.Machine.Morphology.HermitCrab`, and `NodeCostProbe` in `SIL.Machine`. Each +has its own `internal static volatile bool Enabled`, default `false`. **Both must be set.** Setting +only the HermitCrab one makes every lower-layer bucket read exactly `0.00ms`, which looks like +"measured and negligible" rather than "never ran" — this cost a round (ledger method rule: a bucket +reporting exactly zero is broken until proven otherwise). Wire both from one place in the harness. + +Everything is insertion-only: a `RecordDie(...)` or `Add*Ticks(...)` call placed immediately before +an existing `return` / `continue`, never a change to control flow. Where a site read +`else if (IsTracing) { trace }`, it becomes `else { if (IsTracing) { trace } Record(...) }` — +behaviour-preserving. Counters are `Interlocked`; timers are `Stopwatch.GetTimestamp()` deltas +accumulated with `Interlocked.Add`, with `Reset*` methods so a harness reads per-word deltas. + +### The wall-time split + +**Eight mutually exclusive top-level slices that sum to wall**, measured around +`Morpher.ParseWord`: + +`lookup` (brackets `LexicalLookup`, materialised with `.ToList()` so the bracket covers only the +lookup itself) · `synCascade` · `synBattery` · `synForward` · `synExpand` (brackets +`Word.ExpandAlternatives()` in `SynthesizeSequential`) · `anTotal` (brackets the whole +`_analysisRule.Apply(input)` call) · `unaccounted` (defined as the remainder, never tracked). + +`synForward` is recorded **net of** `synCascade` and `synBattery`: capture both counters before the +`_synthesisRule.Apply` call and subtract their deltas after. This matters enormously and caused a +25x published error — the work a synthesis fold memo would share runs *inside* the cascade and +battery brackets, so the share to divide by is `synCascade + synBattery + synForward`, not +`synForward` alone. Template slot rules compile to the same `SynthesisAffixProcessRule` classes via +`RuleBatch` (`SynthesisAffixTemplateRule` ctor), which is why they land in `synBattery`. + +Nested *inside* `anTotal`, as a breakdown rather than additional slices: `anCascade` (the +`_mrulesRule.Apply` call in `AnalysisStratumRule.ApplyMorphologicalRules`), `anBattery` +(`ApplyTemplateBattery`), `anPhono` (`_prulesRule.Apply`). + +**Always print the remainder as its own column.** It caught the misattributed split, the dead probe +gate, and would have caught four of five retractions at the point of measurement. + +### The per-node decomposition + +Inside `anCascade` and `anBattery`, sub-buckets: `match` · `feature` · `clone` · `key` · `memo` · +`trail` · `equality` · explicit remainder. + +`match` and `feature` come from the `SIL.Machine` layer. To avoid double counting, the HermitCrab +side reads `NodeCostProbe.MatcherTicks` / `FeatureTicks` **before and after** the cascade call and +records the delta — so the lower-layer buckets are slices *inside* the HermitCrab bucket, not +alongside it. Matcher self-time has nested work subtracted at the source (inside `Matcher.cs`); +feature work needs a re-entrancy depth guard or recursive `Unify` calls double-count. + +Measured outcome, so a rebuild can sanity-check itself: on Amharic `anCascade`, clone **46.7%**, +remainder **37.4%**, match 12.5%, feature 3.4%, key **0.0%**, and memo / trail / equality each +**<0.2%**. + +### Censuses + +Rule attempts, split by which guard rejected them (`guardSelector`, +`guardApplicationCount`, `guardUnifiability`, `reachedMatching`); allomorph pattern-match +attempts vs matches; clone created vs discarded vs kept. + +Die points, as an enum with a counter array **and a parallel tick array** — timing each individual +guard check, not the enclosing `Apply()`. Count and cost diverge: `RuleNotApplicableOrPatternMismatch` +was **100.0% of events** and about **0.2% of time**, because each rejection is one list index plus one +reference compare (~29 ns). Never report a die-point histogram by count alone. + +Die points, classified for the abstract-replay question — **shape-free**: synthesis-side +`MaxApplicationCount`, syntactic-feature unification, MPR features, realizational +subsumption/blocking, co-occurrence and obligatory features in `IsWordValid`. **Shape-dependent**: +allomorph pattern match / rule-not-applicable, allomorph environments, final `IsMatch`. +`LexicalLookupMiss` is neither — it precedes any synthesis step. + +--- + +## 2. The keys + +### `AnalysisStateKey` (pre-existing) + +`(Shape, Stratum, SyntacticFeatureStruct, RealizationalFeatureStruct, NonHeadCount, per-rule +un-application count multiset)`. Order-independent by design — no analysis-side rule reads trail +order, which is the redundancy it collapses. The multiset is XOR-combined, not `*31`-rolled, so it +hashes commutatively. + +**Freeze-on-read in the constructor is mandatory.** `AnalysisAffixTemplateRule.Apply` reassigns +`SyntacticFeatureStruct` to a fresh unfrozen clone *after* the owning `Word` is frozen, and that +setter has no `CheckFrozen()` guard. + +### The narrowed variant (row 4) + +Filter the count multiset to rules that are **not** provably shrinking. Two requirements learned the +hard way: build the retain-map **once per grammar**, not per key (key construction is on the hottest +path); and filter during hashing and equality rather than by materialising a filtered dictionary, so +the narrow key allocates no more than the full one. **Absent from the map means retain** — a rule the +classifier walk does not cover must be counted, never silently dropped. + +A `retainedCount` field is needed so equality can compare sizes without enumerating. + +### `SynthesisStateKey` (row 5) — and the trap + +Everything a synthesis step reads: shape+annotations, syntactic FS, realizational FS, MPR set, root +allomorph, disjunctive allomorph indices, applied-rule counts, `IsPartial`, +`IsLastAppliedRuleFinal`, stratum, **and the ordered remaining trail** — +`Word.MorphologicalRuleTrail` sliced to `[0, PendingTrailPosition]`. + +**The trail content is the whole game.** A key with `PendingTrailPosition` (an integer index) and no +trail content is fine for *measuring* and catastrophic for *skipping work*: two candidates at the +same index with different pending sequences compare equal, and merging them discards one +derivation's continuation. Every large apparent redundancy in this engine was measured with such a +key, and each collapsed when the content was added — 9,774x → 15–40%, 61-of-395,026 → 351,414-of-395,026, +3.22x/8.10x → `hits = 0`. + +**`Word.ValueEquals` is not a starting point.** It omits `_syntacticFS`, MPR features and the +disjunctive allomorph indices, all of which synthesis reads. Inheriting from it produces a silently +inflated ratio. + +Two more properties a real memo needs: stored results are **sets**, not values (realizational rules +are trail-exempt — `SynthesisRealizationalAffixProcessRule` has no `IsMorphologicalRuleApplicable` +gate — and several allomorphs of one rule can legitimately match one input before the disjunctive +break); and stored outputs embed the producing candidate's trail, so replay must **re-anchor**, the +mirror of `Word.ReplayOnto`. + +### `RuleLengthClassifier` (row 4's static half) + +Verdicts: `Shrinking` / `NonShrinking` / `Unknown`, where `Unknown` is treated identically to +`NonShrinking` by the key and kept distinct only so a census can tell "this grammar has zero +morphemes" from "this grammar has constructs we cannot analyse". + +An `AffixProcessAllomorph` shrinks iff **every** allomorph (a) inserts at least one *segment* +(`InsertSegments` counting only `HCFeatureSystem.Segment` nodes — a boundary-only insertion is +conservatively NonShrinking, since `InsertSegments.GenerateAnalysisLhs` omits boundaries from the +analysis pattern; `InsertSimpleContext` counts as 1), (b) has **every Lhs part captured** by a Rhs +`CopyFromInput` or `ModifyFromInput`, and (c) copies no part twice. + +The direction trap, which is easy to invert: on the analysis side rules are *un*-applied, so a rule +that inserts on synthesis **removes** on analysis, and a rule that truncates on synthesis +**untruncates** — an Lhs part with no Rhs copy makes the un-applied word **grow**. The authority is +`AnalysisMorphologicalTransform.GenerateShape`, which walks the Lhs parts and either copies the +captured span or calls `Untruncate` to regenerate the part from its pattern. Reduplication (a part +copied twice) and `CompoundingRule` are `Unknown`; conservatism only costs pruning opportunity. + +Measured census, for a rebuild to check against: Sena 19 of 27 rules shrinking, Indonesian 10 of 15, +Amharic 31 of 36. + +--- + +## 3. Harness design + +`[TestFixture]`, `[Explicit]`, environment-driven, writing only `TestContext` lines and no files — +this repo never commits real grammars, and derived corpus data must not leak into a committed path. +**Never print rule, stratum, morpheme or lexical-entry names, or word lists.** Counts, ratios and +timings only. Conformance fixture IDs are committed and safe. + +Env: `HC_MEMO_GRAMMAR`, `HC_MEMO_WORDS`, `HC_MEMO_MAX_WORDS`, `HC_PROBE_WORDS` (explicit +comma-separated list), `HC_PROBE_FIXTURES_ROOT`, `HC_MEMO_TIMEOUT_MS`. + +Breadth comes from `SIL.Machine.Morphology.HermitCrab.Conformance.Fixture.DiscoverAll(root)` over +`conformance/{languages,edge-cases}` — 33 grammars, each `grammar.xml` + `words.yaml` with +hand-derived expectations. Depth comes from the three real corpora. **Amharic: never exceed the first +28 words** — zero-based index 28 is pathological and burns hours. + +Run the memoized path: `new Morpher(traceManager, language, maxDegreeOfParallelism: 1)`. Tracing +disables the memo, so a traced run measures nothing. + +### A/B methodology, all of it earned by being wrong first + +Parity is the gate and speed is only reported. Acceptance is **analysis-set equality on canonical +morpheme-signature sets** (`MorpherTests.WordAnalysisSignature`, sorted) — never object or byte +equality, because a replayed `Word` is legitimately not field-for-field identical. + +Timing: **discard a warm-up rep per arm, then take min of N interleaved samples, and print the +off-arm spread beside every speedup.** Refuse any speedup smaller than its own spread. Three +failures forced each part of this — a first A/B read 1.53x that was entirely JIT; another read +2.385x with a 210.9% spread on one arm that was 1.015x serial; and adding one arm's warm-up while +discarding the other's manufactured a uniform ~2.4x across every fixture including ones with zero +shareable steps. + +**Never dispatch timing-sensitive probes concurrently.** Three at once put four testhosts on the +machine and invalidated three rows' bucket percentages. Check `tasklist | grep -c testhost` before +any timing run and report the count alongside the numbers. Counts survive contention; shares and +speedups do not. + +**Never aggregate the test logger's console output.** It prints every line twice, the second copy +space-prefixed, so a `^\s*` grep doubles every total — this published a 2x error. + +Sub-50 ms fixtures cannot carry a timing claim: one 10 ms fixture's share moved 1.3% → 14.5% +between two runs of identical code. Flag them (`reliable = wallMs >= 50`) rather than filtering them +out silently. + +### One concurrency hazard worth knowing + +A per-word timeout wrapper built on `Task` has no cooperative cancellation, so an abandoned word's +task keeps mutating engine state after the harness moves on. Harmless with a per-word +`AnalysisScope`; under a **shared** scope it collided with the next word on the same +`Memo`/`InProgress` collections and produced `IndexOutOfRangeException` inside +`MemoizedCombinationRuleCascade`. A shared-arm timeout must abort the run, not continue. Also note +`Task.Result` throws `AggregateException`, not the bare exception type. + +### Final reusable harness + +The retained `PerfBench` is narrower than the historical probes above. It accepts `HC_BENCH_GRAMMAR`, +`HC_BENCH_WORD_LIST`, `HC_BENCH_REPS`, `HC_BENCH_OUT`, and `HC_BENCH_LABEL`; it records timings, errors, +resource observations, memo diagnostics, and canonical result signatures. It does not retain the completed +round's hot-loop counters. + +Timeouts are now the caller's responsibility: start one `dotnet test` process for an arm and terminate the +whole process if its budget expires. Do not restore an in-process `Task` timeout—the abandoned parse continues +running, contaminates later samples, and can race shared engine state. `scripts/hc-bench-compare.ps1` compares +grammar and word keys symmetrically before comparing errors, signatures, timing, or spread. + +--- + +## 4. Two durable engine facts, restated here so they survive independently + +**`Word.ReplayOnto` does not splice `_mrulesUnapplied`.** It splices `_mruleApps` and `_nonHeadApps` +only. This is safe *solely* because `AnalysisStateKey` includes the count multiset, which guarantees +a memo hit has identical arrival and stored-arrival counts. **Narrowing that key breaks correctness +with no test failure.** Fix, if the key ever changes: store arrival counts on `MemoEntry` and compute +`stored - storedArrival + query` — a no-op under today's key, which is also how to test it. + +**`AnalysisScope.MaxMemoEntries` is a per-scope cap with no eviction.** Sharing one scope across a +corpus therefore makes words compete for a fixed budget: two heavy Sena words exhausted it at index +74 and 76, and every later word lost the within-word memoization a fresh scope would have given it. +Cross-word hits went *negative* and the corpus ran 28–32% slower. A shared bounded cache without +eviction is not a smaller version of the optimization. diff --git a/docs/optimizations/deferred-template-clone.md b/docs/optimizations/deferred-template-clone.md new file mode 100644 index 000000000..e5c6d5d7f --- /dev/null +++ b/docs/optimizations/deferred-template-clone.md @@ -0,0 +1,35 @@ +# Deferred per-template clone in `AnalysisAffixTemplateRule` + +**Status:** retained. **Portable:** the observation is; the saving is allocation, so the Rust port gains +little unless it also clones eagerly. + +## The observation + +`AnalysisAffixTemplateRule.Apply` made an internal engine clone and froze it once per template before trying any +slot, then ran the slot DFS on the clone. The ledger measured 66% of those clones discarded unused +(2,505 created, 1,659 discarded, Amharic) and closed the "defer" option because it missed a 70% gate by +four points, choosing "make a clone cheaper" instead. + +Reading the code shows the clone had exactly one load-bearing job. The input is already frozen at every +call site; slot rules never mutate their input (`ApplyRhs` clones `match.Input`); intermediate `Source` +pointers are overwritten by the stratum. The only path that emits the input object itself is the +all-slots-skipped terminal, and `Apply` then mutates every emitted word's `SyntacticFeatureStruct`, so +that one path needs a private copy. + +## The change + +Run the DFS on the frozen input directly (untraced path). At the two emission points that can carry the +original input through unmodified (`ApplySlots` terminal, `ParallelApplySlots` push), emit a frozen +clone when the word is reference-equal to the input. The traced path keeps the eager clone because the +trace manager stamps `CurrentTrace` onto the words it is handed. A naive "no clone anywhere" version was +tried first and failed the new tests immediately (`FeatureStruct.Add` threw on a frozen structure). + +## Measured and verified + +Alternating wall-time A/B against the pinned baseline, 45 Sena words, min of two interleaved rounds: +1.054x overall, heavy words 1.02x to 1.08x; parity clean on all words and both rounds. Smaller than the +ledger's framing suggested: the eager clone was not where the heavy words' time went. + +Focused template tests cover required/optional slots, traced execution, and both sequential and parallel +paths. The complete conformance-inclusive HermitCrab run at the final code tip passed 603 tests with one +expected host-privilege skip. diff --git a/docs/optimizations/edge-segment-prefilter.md b/docs/optimizations/edge-segment-prefilter.md new file mode 100644 index 000000000..c1a905480 --- /dev/null +++ b/docs/optimizations/edge-segment-prefilter.md @@ -0,0 +1,64 @@ +# Edge-segment prefilter for analysis affix rules + +**Status:** retained (`AnalysisAffixProcessRule`; internal same-binary toggle +`Morpher.EdgePrefilterEnabled`). **Portable:** yes; it is a property of the pattern semantics, +not of the C# engine. + +## The observation + +Every analysis affix pattern is compiled with `AnchoredToStart = true`, `AnchoredToEnd = true`, +`MatchingMethod = Unification`, and a filter that shows the matcher only Segment-type annotations. The +pattern is built from the allomorph's Lhs/Rhs: copied stem parts become groups with quantifiers, and each +inserted segment becomes a single `Constraint` node carrying that segment's feature structure +(`InsertSegments.GenerateAnalysisLhs`). A `Constraint` compiles to one mandatory arc with no epsilon +alternative (`Constraint.GenerateNfa`). + +So when the pattern's first top-level node is a `Constraint`, the traversal's first step must unify it +with the shape's first segment node, or the traversal fails; likewise the last `Constraint` and the last +segment node. The only way an annotation can be skipped is when it is marked `Optional`. + +On the heaviest Sena word, 3,564,736 pattern-match attempts produced 170,863 matches (95% failed), and +each failed attempt still ran a full nondeterministic traversal: 4,017,574 traversal instances, +9,111,459 arc checks. + +## The change + +At compile time, `AnalysisAffixProcessAllomorphRuleSpec` records `LeftEdgeConstraint` / +`RightEdgeConstraint`: the feature structure of the pattern's first / last child when that child is a +bare `Constraint` (null when it is a group, quantifier or alternation). At run time, before each +allomorph's `MultiplePatternRule.Apply`, `AnalysisAffixProcessRule.Apply` finds the shape's first and +last Segment-type nodes (skipping boundaries exactly as the matcher's filter does) and rejects the +allomorph if `edgeNode.FeatureStruct.IsUnifiable(edgeConstraint)` fails, using the same unification call +and `UseDefaults` setting the matcher itself uses. An `Optional` edge node disables the check on that +side. When tracing, the same `MorphologicalRuleNotUnapplied` event is emitted that the matcher's failure +would have produced, so traces are unchanged. + +## Measured (deterministic counts, sequential path, one parse per word, same binary) + +| 45 Sena words | prefilter off | prefilter on | +| --- | --- | --- | +| FST traversals | 12,376,416 | 1,617,662 | +| FST arc checks | 34,170,777 | 15,583,242 | +| traversal instances used | 15,642,175 | 2,429,643 | +| pattern-match attempts (counted before the check) | 11,979,916 | 11,979,916 | + +Parity: identical analysis signature sets on all 45 Sena words and on all 464 conformance-fixture words +(33 grammars). Fixture traversals fell 21% (31,278 to 24,741); their patterns fail less often. + +`AnalysisEdgePrefilterTests` covers left/right constraints, boundary skipping, optional nodes, and the +disabled path. The broader HermitCrab and conformance suites cover integration behavior; the final +conformance-inclusive run passed 603 tests with one expected host-privilege skip. + +## Why the ledger's estimate was wrong + +The ledger (row 22) capped a prefilter at ~10% using the matcher's *self-time* share. The traversal's cost +is mostly booked elsewhere (instance and register allocation, feature unification, GC), so self-time +understated it by a large factor. By counts, the prefilter removes 87% of traversals outright. + +## Limits + +After the change, ~44% of the remaining traversals succeed, so a second-segment check has little left to +reject. The rule loop still runs ~110 attempts per cascade state; depending on which safe edge constraints +exist and whether the first rejects, an attempt performs zero, one, or two prefilter unifications before any +remaining match. Indexing allomorphs by edge segment could avoid both checks and matching work, but its +headroom was not established and it was not attempted. diff --git a/docs/optimizations/feature-struct-frozen-tree-clone.md b/docs/optimizations/feature-struct-frozen-tree-clone.md new file mode 100644 index 000000000..642c1cde2 --- /dev/null +++ b/docs/optimizations/feature-struct-frozen-tree-clone.md @@ -0,0 +1,28 @@ +# Reuse or skip the feature-structure clone map + +**Status:** retained (`FeatureStruct.Clone`). +**Portable:** yes; the specific pool is .NET-oriented, while the tree proof is general. + +## Invariant + +A frozen structure marked as a flat tree has no aliases or cycles to preserve below its root. Its top-level +clone therefore cannot encounter the same value twice, so the old-to-new copies map has no semantic work to +do. Recursive cloning inside another graph must still use the caller's map even when the child itself looks +tree-shaped, because the parent graph may refer to that child elsewhere. + +## Implementation + +A top-level clone skips the map for a proven frozen flat tree. General clones borrow a per-thread map, clear +it in `finally`, and fall back to an independent map for reentrant calls. The internal recursive constructor +always honors the caller-provided map. + +## Evidence + +This removes dictionary creation and lookup from the common clone case. It travelled with the broader clone +allocation work and was not timed independently, so the record deliberately makes no isolated speed claim. + +## Correctness coverage and limits + +`FeatureStructCloneTests` covers alias preservation, cycles, reentrant cloning, and the distinction between a +top-level tree and a tree embedded in a shared graph. The complete conformance-inclusive run passed 603 tests +with one expected host-privilege skip. diff --git a/docs/optimizations/feature-struct-frozen-tree-equality.md b/docs/optimizations/feature-struct-frozen-tree-equality.md new file mode 100644 index 000000000..5378fb48e --- /dev/null +++ b/docs/optimizations/feature-struct-frozen-tree-equality.md @@ -0,0 +1,29 @@ +# Allocation-free equality for frozen flat feature structures + +**Status:** retained (`FeatureStruct.ValueEquals`). +**Portable:** yes; the proof depends on immutability and tree shape, not the CLR. + +## Invariant + +During `Freeze`, a feature structure is marked as a flat tree only when every reachable value is a +`SimpleFeatureValue`, no value instance repeats, and no nested `FeatureStruct` is present. Once frozen, that +fact cannot change. Comparing two such structures can never trigger the general algorithm's reentrancy or +cycle guard, so a feature-count/key/value comparison is equivalent to the general visited-pair walk. + +## Implementation + +`ValueEquals` uses the direct leaf comparison only when both operands are frozen and carry the tree proof. +Every other shape—including shared leaves, nested structures, and cycles—uses the general algorithm. A +test-only slow-path entry point lets randomized tests compare both implementations on the same inputs. + +## Evidence + +The optimization removes all visited-set work from the dominant frozen, flat case. It was observed in the +same profile that motivated traversal-state reuse, but was not isolated from the other feature-structure +changes; no standalone wall-time ratio is claimed. + +## Correctness coverage and limits + +`FeatureStructTreeFastPathTests` cross-checks randomized flat structures against the slow path and exercises +false comparisons and structures that must not qualify. `FeatureStructValueEqualsFreezeTests` covers shared +and cyclic graphs. The full conformance-inclusive run passed 603 tests with one expected host skip. diff --git a/docs/optimizations/feature-struct-visited-state-reuse.md b/docs/optimizations/feature-struct-visited-state-reuse.md new file mode 100644 index 000000000..d140a2728 --- /dev/null +++ b/docs/optimizations/feature-struct-visited-state-reuse.md @@ -0,0 +1,28 @@ +# Reuse feature-structure traversal state + +**Status:** retained (`FeatureStruct.ValueEquals` and `Freeze`). +**Portable:** the allocation-avoidance principle is portable; the `[ThreadStatic]` pool is .NET-specific. + +## Invariant + +Equality and freezing need visited sets only after a recursive value is reached. A top-level +operation owns its traversal state until it returns. A nested top-level call cannot borrow that same state, +so it falls back to fresh collections. + +## Implementation + +The recursive APIs accept nullable state and allocate it only when traversal requires it. Ordinary top-level +calls borrow pre-sized sets and dictionaries from a per-thread pool, clear them in `finally`, and release the +pool. The pool has an `InUse` guard so reentrant calls retain the old independent-state semantics. + +## Evidence + +The heaviest Sena profile put feature-structure equality/freezing at 13.7% of wall time, with `HashSet` +resizing a major allocation source. After this change, `ValueEqualsImpl` and those resizes disappeared from +the top of the profile. This mechanism was not timed in isolation, so no standalone speedup is claimed. + +## Correctness coverage and limits + +`FeatureStructPooledVisitedSetsTests` covers reuse, clearing after success and failure, and reentrancy. Cyclic +and shared graphs continue down the general path. The complete conformance-inclusive HermitCrab run at the +final code tip passed 603 tests with one expected host-privilege skip. diff --git a/docs/optimizations/flat-register-arrays.md b/docs/optimizations/flat-register-arrays.md new file mode 100644 index 000000000..359abb125 --- /dev/null +++ b/docs/optimizations/flat-register-arrays.md @@ -0,0 +1,25 @@ +# Flat register arrays in the FST traversal + +**Status:** retained (`SIL.Machine.FiniteState`: `Register[]` indexed by `RegisterArray.Idx(r, k)` +replaces `Register[,]` internally; the public `FstResult.Registers` stays two-dimensional and is +materialized once per accepting state). **Portable:** no; a .NET-specific allocation path. + +## The observation + +11.5% of the heaviest Sena word's CPU profile was inside `System.Array.CreateInstanceMDArray`, the slow +generic allocator for multi-dimensional arrays, called from `Matcher.AllMatches` (8.6%) and traversal +setup (2.3%): one `new Register[n, 2]` per match call and per traversal instance. + +## The change + +Mechanical: flat arrays of length `registerCount * 2`, a two-constant index helper, `Array.Copy` on the +flat struct array, and `RegistersEqualityComparer` comparing the flat layout. No behaviour change. + +## Measured + +`CreateInstanceMDArray` 11.5% → 0.2% of the profile. Counts unchanged by construction. Together with the +other allocation changes it contributed to the heaviest word's wall time; not measured in isolation. + +`RegistersEqualityComparerTests` covers equality and hashing of the flattened layout, while the existing FSA +and FST suites cover accepting results and register materialization. The complete conformance-inclusive run +passed 603 tests with one expected host-privilege skip. diff --git a/docs/optimizations/shape-copy-single-pass-map.md b/docs/optimizations/shape-copy-single-pass-map.md new file mode 100644 index 000000000..ed8c15828 --- /dev/null +++ b/docs/optimizations/shape-copy-single-pass-map.md @@ -0,0 +1,28 @@ +# Build `Shape.CopyTo`'s node map in the copy pass + +**Status:** retained (`Shape.CopyTo`). +**Portable:** yes, when a port has the same node-remapping requirement. + +## Invariant + +Annotations refer to nodes in the source shape and must be remapped to the corresponding nodes in the copy. +The source and destination nodes are already visited in matching order while cloning, so that loop can record +the mapping directly. No second `Zip`/`ToDictionary` traversal is required. An empty source range needs no +map; every nonempty range builds one because annotations are queried only after nodes have been copied. + +## Implementation + +`Shape.CopyTo` allocates the map on the first node of a nonempty range and fills it as each node is cloned. +Annotation ranges are then copied through that map exactly as before. + +## Evidence + +The change removes one enumeration and one LINQ-built dictionary from each annotated shape copy. It was not +timed independently from shape copy-on-write and feature-structure clone work, so no standalone ratio is +claimed. + +## Correctness coverage and limits + +`ShapeCopyToTests` covers empty ranges, leaf and spanning annotations, node/range remapping, and independence +of the copy. +The complete conformance-inclusive run passed 603 tests with one expected host-privilege skip. diff --git a/docs/optimizations/shape-sharing-on-clone.md b/docs/optimizations/shape-sharing-on-clone.md new file mode 100644 index 000000000..702de8f35 --- /dev/null +++ b/docs/optimizations/shape-sharing-on-clone.md @@ -0,0 +1,44 @@ +# Copy-on-write Shape sharing in `Word.Clone` + +**Status:** retained (`Word.CloneForEngine`, `EnsureOwnShape`, `ResetShape`, `Shape.CreateEmptyLike`). +**Portable:** partly. The *observation* (three of four clone sites never use the copied shape) is +engine-independent; the mechanism (shared reference to an immutable structure) is how a Rust port would +naturally write it already. + +## The observation + +A CPU profile of the heaviest Sena word (60 s sampled, memoized path) put `Word` cloning at 36% of wall, +of which `Shape.CopyTo` was 28%: every clone deep-copied every `ShapeNode`, every node's feature +structure (`SimpleFeatureValue.CloneImpl` 22%), every annotation, and built a node-mapping dictionary. +The callers: `Word.ReplayOnto` 15% (memo replay; never mutates the shape), `Word.ExpandAlternatives` 9% +(immediately *replaces* `_shape`), `AnalysisAffixProcessAllomorphRuleSpec.ApplyRhs` 5% (`GenerateShape` +immediately `Clear()`s the copy), non-head clones 2.5%, `LexicalLookup` (the `RootAllomorph` setter +replaces `_shape`). + +## The change + +- Internal engine clone: if the source shape is frozen, share the reference and mark `_shapeShared`; otherwise + deep-copy it. Public `Word.Clone()` always returns a deep, independently mutable shape. +- `EnsureOwnShape()`: clone once, on demand, before in-place mutation. Called in `AnalysisStratumRule` and + `SynthesisStratumRule` before phonological rules are applied to a fresh clone (rewrite specs insert + nodes and hold node references obtained by matching, so the un-share must precede the match). +- `ResetShape()`: replace with a new empty shape of the same configuration. Used by the four + `GenerateShape`/rebuild sites (analysis affix, analysis compounding subrule, synthesis affix, synthesis + compounding) instead of clone-then-clear. +- `ExpandAlternatives` marks the adopted shape as shared; the `RootAllomorph` setter and constructors mark + their fresh clones as owned. + +The safety net is that a shared shape is frozen: any mutation through a missed site throws +(`Shape.CheckFrozen`, frozen node feature structures; a missing guard on `Annotation.Range` was added). +During development the first full test run threw at exactly the two stratum sites, which is how they +were found; no other site surfaced across 122 HermitCrab tests, 464 fixture words and 45 Sena words. + +## Measured + +Profile after the change: `Word` clone 36% → 9% of wall, `Shape.CopyTo` 28.6% → 13% (the remainder is +the stratum's `EnsureOwnShape` and part copying inside `GenerateShape`), `ReplayOnto` 21.7% → 6.5%. Peak +working set on the heaviest word 11 GB → 6 GB. Wall-time attribution is unreliable on this machine (see +the round document); counts are unchanged by construction. + +`WordShapeSharingTests` covers public-clone independence, copy-on-write, reset paths, and missed-mutation +guards. The final conformance-inclusive run passed 603 tests with one expected host-privilege skip. diff --git a/docs/optimizations/syntactic-feature-structure-sharing.md b/docs/optimizations/syntactic-feature-structure-sharing.md new file mode 100644 index 000000000..40ff2af22 --- /dev/null +++ b/docs/optimizations/syntactic-feature-structure-sharing.md @@ -0,0 +1,32 @@ +# Copy-on-write syntactic feature structures in engine clones + +**Status:** retained (`Word.CloneForEngine`, `EnsureOwnSyntacticFeatureStruct`). +**Portable:** partly; immutable sharing is general, while the ownership plumbing is specific to this engine. + +## Invariant + +HermitCrab freezes a `Word` before it becomes an input to the analysis or synthesis graph. Internal clones may +therefore share that frozen word's syntactic feature structure until a rule needs to mutate it. The public +`Word.Clone()` contract remains a deep, mutable clone and never exposes copy-on-write sharing to callers. + +## Implementation + +Internal engine sites use `CloneForEngine()`. A per-`Morpher` setting is copied onto new words and controls +whether those clones share a frozen syntactic feature structure. The three in-place analysis mutation sites +call `EnsureOwnSyntacticFeatureStruct()` before modification; assignments of a fresh structure need no guard. +The opt-out is internal and captured per `Morpher`, preventing one parser or test from changing another +existing parser's behavior. + +## Evidence + +The heaviest Sena profile attributed roughly 15% of CPU to freezing and hashing a newly cloned syntactic +feature structure while constructing analysis keys. Sharing removes that repeated deep clone/freeze on the +engine path. The final five-language measurement is branch-wide rather than an isolated toggle result, so no +standalone speedup is assigned here. + +## Correctness coverage and limits + +`WordSyntacticFsSharingTests` covers copy-on-write, nested/shared structures, public-clone mutability, and +isolation between morphers. The broader HermitCrab suite exercises integration behavior and passed both with +sharing enabled and with `HC_SHARE_SYNTACTIC_FS=false`; the full conformance-inclusive run passed 603 tests +with one expected host skip. Sharing is intentionally unavailable for mutable source words. diff --git a/docs/rejected-optimizations/alternatives-memo-and-hoist.md b/docs/rejected-optimizations/alternatives-memo-and-hoist.md new file mode 100644 index 000000000..419cba7ef --- /dev/null +++ b/docs/rejected-optimizations/alternatives-memo-and-hoist.md @@ -0,0 +1,15 @@ +# `ExpandAlternatives` memoization and hoisting + +**Status:** rejected and removed from production. + +The experiment memoized `Source` recursion during `Word.ExpandAlternatives` and computed an analysis word's +expansion once outside the per-root-candidate loop. It was exact, but on the 45-word Sena battery it left word +clones unchanged at 4,747,013: deeper strata did not branch, so it saved only small temporary list work. + +A later change moved work back behind successful lexical lookup, but after the experimental memo/hoist was +removed the final product path is the same ordering as the pinned production baseline: each lexical candidate +expands its synthesis word. There is therefore no retained alternatives optimization in this branch, and the +earlier 75% “alternatives expanded” reduction must not be attributed to the final diff. + +The broader deduplication avenue remains closed by the complete-key census in the main ledger: once pending +trail content is included, only 11% of alternatives duplicate, for a 2.28% wall-time ceiling on Sena. diff --git a/docs/rejected-optimizations/compounding-split-prefilter.md b/docs/rejected-optimizations/compounding-split-prefilter.md new file mode 100644 index 000000000..067fbd07d --- /dev/null +++ b/docs/rejected-optimizations/compounding-split-prefilter.md @@ -0,0 +1,11 @@ +# Compounding split prefilter + +**Status:** rejected before implementation; temporary counters removed. + +The candidate was to query lexical reachability before materializing compounding splits. A census on the +heaviest Sena probe counted 35,176 splits, 5,524 root candidates, and 1,121 outputs. That work represented +about 2% of word clones and 9% of FST traversals—too small a ceiling for an additional indexing and correctness +surface, especially because real compounding prevents the simpler grammar-wide lexical gate. + +No production change was built. Reopen only if a different grammar shows compounding split construction as a +dominant measured bucket and supplies a sound, stratum-aware reachability predicate. diff --git a/docs/rejected-optimizations/final-template-prune.md b/docs/rejected-optimizations/final-template-prune.md new file mode 100644 index 000000000..a6f0002b5 --- /dev/null +++ b/docs/rejected-optimizations/final-template-prune.md @@ -0,0 +1,25 @@ +# Final-template interleaving prune + +**Status:** rejected and removed from production. **Portable:** the proof is portable where its guard holds. + +The experiment mirrored synthesis's rule that a non-template affix process normally cannot follow a final +template. Analysis could skip the inverse interleaving when every relevant template was final and the grammar +contained no partial morphemes. + +The available real grammars defeated the useful guard: Sena has 25 partial lexical entries and six partial +rules; Amharic and Mbugwe each have a partial rule. The prune fired zero times on all three. Synthetic tests +proved that the rule could fire safely, but that establishes correctness, not value. Carrying production +branches, state, toggles, and counters for a zero-effect optimization violated the final minimum-surface goal, +so all of them were removed. + +Reopen only for a corpus with the reported pathological final-template interleavings and no partial +morphemes, or with a proved narrower per-state partiality condition. Measure real prunes before shipping. + +## Addendum 2026-09-03 +Superseded in part. The same prune, measured with the partiality guard lifted, is 8-33x on Mbugwe and Sena with +identical analysis sets (298x on the single heaviest Sena word). The guard, not the mechanism, is what made it +inert: partial *lexical entries* can never rescue a pruned branch, because synthesis applies no template to a +partial root and then fails the candidate with `PartialParse`, so a rules-only, per-stratum guard is strictly +stronger and still sound. Reopen on that basis rather than on the corpus counts above. The full measurement +record is `docs/pr491-final-template-prune-upper-bound.md` on the `perf/hc-optimization-archive` branch, which +is outside this docs set. diff --git a/docs/rejected-optimizations/two-pass-nondeterministic-traversal.md b/docs/rejected-optimizations/two-pass-nondeterministic-traversal.md new file mode 100644 index 000000000..801af1c30 --- /dev/null +++ b/docs/rejected-optimizations/two-pass-nondeterministic-traversal.md @@ -0,0 +1,14 @@ +# Two-pass nondeterministic traversal + +**Status:** measured negative and removed from production. + +This port of the `investigate_linear_traverse` approach first merged traversal instances by state, position, +and variable bindings, then rebuilt captures. HermitCrab's existing traversal already pools instances, and +its patterns do not have the repeated-group shape needed to repay the extra bookkeeping. + +On the 45-word Sena battery the alternative used 2.3 times as many traversal instances, reduced arc checks by +only 1.6%, produced more duplicate matches, and increased word clones by 27%. Conformance fixtures pointed in +the same direction. Toggle hooks, alternate algorithms, and counters were removed; only this result remains. + +Reopen only with a corpus whose automata have substantial equivalent active-state convergence and a census +showing that capture reconstruction is cheaper than the current pooled traversal. From 5bbfa018bdbd5228c3f1aa73db951a4f01d167d3 Mon Sep 17 00:00:00 2001 From: John Lambert Date: Fri, 18 Sep 2026 21:00:17 -0400 Subject: [PATCH 2/7] docs: record the state+position traversal dedupe round PR #511 proposes skipping traversal instances whose (State, AnnotationIndex) was already pushed. The motivating pathology is real and reproduces on a 2-state fsa: Advance forks per Optional annotation, so instances are exponential in optional count and independent of the state count. The key is too coarse. A 20,000-case differential fuzz finds 112 cases where Match() changes, 108 from ignored variable bindings and 4 from shortened ranges on the deterministic path, with AllMatches().First() identical in all 20,000 as the control. A narrowed gate -- deterministic method, no capture groups -- measures 0 divergences with the full bound preserved. Two censuses close the extensions: environment matchers are 0.20%/1.18% of traversal instances on Amharic/Mbugwe, and the analysis rules' apparent 93.95% collapse is 99.7% alternate captures, because AnalysisAffixProcessRule and AnalysisCompoundingRule set AllSubmatches and enumerate morph boundaries on purpose. That makes a fourth row for the apparent-vs-sound table. Co-Authored-By: Claude Opus 5 --- docs/hermitcrab-optimization-ledger.md | 4 +- .../state-position-traversal-dedup.md | 95 +++++++++++++++++++ .../two-pass-nondeterministic-traversal.md | 4 + 3 files changed, 102 insertions(+), 1 deletion(-) create mode 100644 docs/rejected-optimizations/state-position-traversal-dedup.md diff --git a/docs/hermitcrab-optimization-ledger.md b/docs/hermitcrab-optimization-ledger.md index 28443b33f..e6c3acabe 100644 --- a/docs/hermitcrab-optimization-ledger.md +++ b/docs/hermitcrab-optimization-ledger.md @@ -85,6 +85,7 @@ copy-on-write design through `CloneForEngine`; see `docs/optimizations/shape-sha | 11 | Tandem lexical intersection (T2) | Kill doomed branches early via the lexicon | The oracle only sees lexical unreachability. The expensive words fail on checks that run *after* lexical lookup succeeds | Pooled **23.5%** dead steps vs a 30% gate; one failure word at exactly **0.0%** dead | | 21 | **Memo-key caching on the frozen `Word`** | Keys are built ~28x per distinct state, which looked like a cheap win. | High count did not imply high cost; caching would add state and invalidation complexity around negligible work. | **CLOSED:** all `AnalysisStateKey` construction and hashing on Amharic measured **3 ms of 81,707 ms = 0.0%**. | | 12 | Pool small short-lived collections | Fewer allocations, faster | `HashSet/Dictionary.Clear()` is O(capacity); Gen0 already beats pooling at this size | -15–17% bytes but **+8.6% wall**, +12% on the parallel battery. Reverted | +| 23 | **`(State, AnnotationIndex)` traversal dedupe** (PR #511) | `Advance` forks per `Optional` annotation, so instances are exponential in optional count and independent of `|States|`; keying on state+position bounds it at `N * |States|` | The key omits what filters the traversal. Variable bindings gate later arcs; registers carry the captures `AllSubmatches` consumers read | **112 of 20,000 differential-fuzz cases change `Match()`** (108 bindings, 4 shortened ranges) while `AllMatches().First()` is identical in all 20,000. Narrowed to the deterministic method with no capture groups: **0 divergences**, full bound preserved. Extension censuses: environment matchers are **0.20%/1.18%** of instances (Amharic/Mbugwe); the analysis rules' apparent **93.95%** collapse is **99.7% alternate captures**. See `rejected-optimizations/state-position-traversal-dedup.md` | ## Closed before building — motivation refuted by measurement @@ -109,13 +110,14 @@ copy-on-write design through `CloneForEngine`; see `docs/optimizations/shape-sha ## The lesson that generalises -Three independent measurements, three boundaries, same collapse: +Four independent measurements, four boundaries, same collapse: | measurement | key used | apparent | sound | | --- | --- | --- | --- | | synthesis-input dedupe (#6) | order-insensitive | 9,774x | 15–40% | | fold-entry census | trail position only | 6,476x | not established | | fold-step sharing (#5) | trail position only | 3.22x / 8.10x | **hits = 0** | +| FSA traversal dedupe (#23) | state + position | 93.95% collapsible | **0.28%** register-identical | **The redundancy in HermitCrab's synthesis is apparent, not real. The trail is what makes each step distinct, and every measurement showing large shareable work is measuring a key that omits diff --git a/docs/rejected-optimizations/state-position-traversal-dedup.md b/docs/rejected-optimizations/state-position-traversal-dedup.md new file mode 100644 index 000000000..40a9fb66a --- /dev/null +++ b/docs/rejected-optimizations/state-position-traversal-dedup.md @@ -0,0 +1,95 @@ +# `(State, AnnotationIndex)` traversal deduplication + +**Status:** rejected in its general form; a narrowed form is sound and was measured effective. + +The idea, proposed independently as PR #511: inside `Traverse`, skip any newly produced traversal +instance whose `(State, AnnotationIndex)` pair was already pushed, on the grounds that the surviving +instance differs only in its registers and "registers only record the match, they do not filter the +traversal." + +## The motivating pathology is real + +The cost driver is not the state count. `TraversalMethodBase.Advance` forks a copy of the instance for +every `Optional` annotation at the next offset, so instance count grows exponentially in the number of +optional annotations along the input and is independent of `|States|`. A synthetic microbenchmark on a +**2-state** FSA measured 2,097,150 instances popped at 20 optional annotations (~2^(k+1)); at k>=24 it +exceeds a 5,000,000 budget. The originally reported "200,000 traversals on a 4-state fsa" is reproduced +and exceeded on fewer states. With the skip in place, the same cells are bounded at `N * |States|` +(measured ratio 0.5-1.0x), turning non-terminating cells into instant ones. + +## Why the general form is unsound + +A differential fuzz — 20,000 random pattern/input pairs, fixed seed, identical generator compiled against +both arms, comparing success, range, every group range and every variable binding — found **112 cases +where `Match()` differs from production**, while `AllMatches().First()` was identical in all 20,000. That +control pins the divergence to exactly the path the skip changes. + +- **108 cases: variable bindings.** Instances carry live `VariableBindings`, which filter later arcs + through `CheckInputMatch`; the key omits them. `Matcher.Compile` determinizes only variable-free + patterns, so every alpha-variable rule runs the nondeterministic method where the key is blindest. + Minimal case: pattern `high=$v0+` anchored both ends over 5 annotations, production matches `[0,5)`, + the skip returns no match at all. +- **4 cases: capture registers, on the deterministic path.** All are alternation plus capture groups and + all shorten the match *range*, e.g. `(g0(back=back+)|g1(high=high+back=back+))` going from `[0,4)` to + `[0,1)`. So restricting to the deterministic method is not by itself a sufficient guard. + +Both committed suites stay green under the unnarrowed change (826/0/3 and 98/0/0), so the existing tests +cannot see any of this. + +## The narrowed form that is sound + +Gate the skip on the deterministic method **and** the absence of capture groups, and leave the +nondeterministic method untouched: + +```csharp +bool dedupeByState = !allMatches && Fst.GroupNames.Count() <= 1; +``` + +Measured: fuzz divergences **112 -> 0**; the group-free microbenchmark sweep cell-for-cell identical to +the unnarrowed change across all 18 cells, so the full bound is preserved; grouped patterns fall back to +production behaviour exactly; both suites unchanged. `GroupNames.Count()` should be hoisted out of +`Traverse` — it is O(1) via the `ICollection` fast path but is evaluated once per call. + +## Where it cannot be extended, and why + +Two censuses attributed every FSA traversal call to its constructing matcher and counted, per category, +how many pushes a `(State, AnnotationIndex)` key would have collapsed — and critically, whether the +collapsed instance's registers were identical to the surviving one's. + +*Environment matchers* (the subject of issue #515) read only `.Success` and `.VariableBindings` from their +matches, never a range or group, so they looked like the ideal target. They are 0.20% of traversal +instances on Amharic and 1.18% on Mbugwe. Removing every redundant environment push would be worth 0.12% +and 0.004% of total pushes respectively. Not built. + +*Analysis-side morphological rules* hold nearly all the volume and all the apparent opportunity: +2,801,478 of Amharic's 2,808,088 pushes, 93.95% collapsible; Mbugwe 14.44%. Labelling the remaining +construction sites showed the entire figure is exactly `AnalysisAffixProcessRule` and +`AnalysisCompoundingRule` — both built with `AllSubmatches = true` and calling `AllMatches` precisely to +enumerate every distinct morph-boundary placement. Register comparison at each would-be collapse: + +| matcher | identical registers | differing registers | +| --- | --- | --- | +| Amharic `analysis-affix` | 7,446 (0.29%) | 2,573,312 (99.71%) | +| Amharic `analysis-compound` | 16 (0.03%) | 50,920 (99.97%) | +| Mbugwe `analysis-affix` | 3,022 (5.49%) | 52,056 (94.51%) | +| Mbugwe `analysis-compound` | 1,886 (2.93%) | 62,396 (97.07%) | + +The genuinely free slice is 0.28% of collapses on Amharic and 4.11% on Mbugwe — at most 0.6% of total +traversal instances, reachable only by adding a per-instance register-equality guard. `RootAllomorphTrie`'s +raw FST collapses 0% despite ~62k pushes, which is correct rather than a missed opportunity: a prefix trie +is already maximally state-shared at construction. + +Adding `VariableBindings` to the key costs nothing on this evidence: `collapsedWithBindings == collapsed` +on every row of both grammars, with a distinct-signature counter confirming the signature function +discriminates. The information that differs lives in the registers, which is exactly what `AllSubmatches` +consumers read. + +## Relationship to `two-pass-nondeterministic-traversal.md` + +That experiment merged on state, position **and bindings** and then rebuilt captures, and was measured +negative largely on the rebuild cost. This one merges on state and position only and keeps the first +instance's captures, which is cheaper and unsound rather than expensive and sound. The two are not +alternatives to the same design; they are the two ways of getting the key wrong in opposite directions. + +**Reopen only** with a grammar where the identical-registers split looks materially different from the +four rows above, or for the narrowed group-free form, which needs no further evidence to be safe. diff --git a/docs/rejected-optimizations/two-pass-nondeterministic-traversal.md b/docs/rejected-optimizations/two-pass-nondeterministic-traversal.md index 801af1c30..344a6bd0e 100644 --- a/docs/rejected-optimizations/two-pass-nondeterministic-traversal.md +++ b/docs/rejected-optimizations/two-pass-nondeterministic-traversal.md @@ -12,3 +12,7 @@ the same direction. Toggle hooks, alternate algorithms, and counters were remove Reopen only with a corpus whose automata have substantial equivalent active-state convergence and a census showing that capture reconstruction is cheaper than the current pooled traversal. + +**See also** `state-position-traversal-dedup.md`, which drops bindings from the key instead of keeping them +and skips capture reconstruction entirely. It is cheaper than this experiment and unsound for the same +reason this one was expensive: the captures are load-bearing for `AllSubmatches` callers. From 7bbc06081724ae3faa5a563346389b8fb7e5cfa3 Mon Sep 17 00:00:00 2001 From: John Lambert Date: Wed, 21 Oct 2026 10:42:43 -0400 Subject: [PATCH 3/7] docs: research on the asymptotics of the analysis search Co-Authored-By: Codex --- docs/hermitcrab-optimization-ledger.md | 9 + .../hermitcrab-search-asymptotics-research.md | 368 ++++++++++++++++++ 2 files changed, 377 insertions(+) create mode 100644 docs/hermitcrab-search-asymptotics-research.md diff --git a/docs/hermitcrab-optimization-ledger.md b/docs/hermitcrab-optimization-ledger.md index e6c3acabe..ed8c839e5 100644 --- a/docs/hermitcrab-optimization-ledger.md +++ b/docs/hermitcrab-optimization-ledger.md @@ -29,6 +29,15 @@ expensive part was never the code; it was learning what to measure and which mea cost for a measurement run and not one to carry in product code indefinitely. Rebuilding a probe from a branch is cheap; maintaining research scaffolding in two product assemblies is not. +## Unmeasured search-asymptotics research + +[Analysis-search research](hermitcrab-search-asymptotics-research.md) integrates six independent reports, +checks their code claims, and separates new experiments from this ledger's closed families. It ranks +residual lexical viability, a grammar-compiled template battery, recoverable trace-class reduction, and +exact finite-state compilation for a verified subgrammar. It maps them to lexicon-driven pruning during +unapplication and making the affix-template battery cheaper without memoization. **No new measurement, +speedup, or numbered optimization row is claimed.** + ## Final disposition of the September follow-up The follow-up branch does deliver product improvements. Its authoritative summary is diff --git a/docs/hermitcrab-search-asymptotics-research.md b/docs/hermitcrab-search-asymptotics-research.md new file mode 100644 index 000000000..48951c95f --- /dev/null +++ b/docs/hermitcrab-search-asymptotics-research.md @@ -0,0 +1,368 @@ +# HermitCrab analysis search — asymptotic research + +**Disposition: count ceilings before building.** The first two candidates below fit the two open work +items without a search-sized memo. Partial-order reduction has a narrow correctness route and poor prior +coverage. Exact finite-state compilation is a larger, restricted-subgrammar project. None is a measured +optimization, and none establishes a polynomial parser for arbitrary FieldWorks grammars. + +This integrates the six supplied reports: **R1** partial-order reduction and rewriting; **R2** packed forests +and subset DP; **R3** finite-state methods; **R4** search and symbolic sets; **R5** enumeration and graphs; +**R6** abstract interpretation and morphology prior art. The reports are research inputs, not independent +measurements. The [ledger](hermitcrab-optimization-ledger.md), its +[September disposition](hermitcrab-perf-2026-09-disposition.md), and the +[probe design](hermitcrab-probe-design.md) remain the measurement record. + +## The problem in one page + +The motivating case is **a grammar with 16 extra affix processes in one Unordered stratum**. Analysis +starts at the surface, unapplies phonology, then interleaves morphological-rule unapplication with affix +templates. Every candidate can stop for lexical lookup; candidates from intermediate strata also enter +the lookup/synthesis pipeline. A found lexical entry supplies its root allomorphs, each candidate's +alternatives are expanded, and forward synthesis must both pass validity checks and regenerate the surface +form [C1, C4, C5]. + +There are three different sources of growth: + +- **Rule orders.** The combination cascade restarts its rule loop at zero after every successful + unapplication. With `k` enabled, single-use rules, there can be `k!` full orders and + `sum(j=0..k) k!/(k-j)!` ordered prefixes. Multiple application is enabled; ordinary affix/compound + unapplication checks each rule's limit rather than a cascade-wide used-rule mask. C# realizational + unapplication has no such count guard, so a finite count-vector bound needs an eligibility proof for + the rule kinds included [C1, C2]. Allomorph branches and compounding add further choices. The 16 added + processes are not necessarily the total `k`. +- **Optional sites.** Inverse deletion inserts optional nodes into one phonological shape; it does not + immediately allocate every subset shape. Later matching still has consume/skip choices. The Rust root + trie already merges equal active trie positions. General morphological matching also returns captures + and variable bindings, so its branches cannot be merged on position alone [C6, C7]. Optionality can + therefore multiply matcher work even while the shape representation stays compact. +- **Alternatives.** An analysis-state key deliberately omits trail order. The stratum merge keeps + alternatives, and expansion grafts subsequent changes onto their source histories. Forward synthesis + reads the pending rule in order. A shared shape or rule multiset is not a shared synthesis continuation + [C3–C5]. One final analysis does not imply that only one trail needs checking. + +All numbers here are **supplied by the common brief, not reproduced in this job**: + +| Aggregate | Reported value | +| --- | --- | +| Baseline parse time | approximately 15–60 ms per input in both implementations | +| Time after the 16 additions | 17–22 s for one expensive input in Rust | +| Lexical lookups / distinct shapes | 97,575 / 56, from trace mode | +| Distinct canonical states | 10,079; they differ in shape or per-rule counts | +| Template-battery runs / distinct input states | 43,641 / 4,988; approximately 2.9 s | +| Lookup + alternative expansion + synthesis | approximately 6.5 s | +| Alternatives synthesized / distinct final analyses | 367,278 / 1 | +| Compounding bound in the case | `MaxStemCount = 2` | +| Deletion-rule variants | 200–1,600 inputs reach a 1 s per-input timeout | + +These are ceilings to investigate, not interchangeable denominators. Tracing disables the ordinary +analysis merge and C# memoization [C3, C4]; the trace lookup count cannot establish an untraced duplicate +rate. Nor do 56 shapes establish 56 lookup keys across strata. The apparent battery repetition is at most +38,653 calls (88.6%) under the brief's identity, before checking that identity and the cost of avoiding it. +The 367,278-to-one ratio proves neither order invariance nor a synthesis-sharing opportunity. + +## Verified code anchors + +Line numbers below were read directly, not copied from the reports. **C#** means the requested main +checkout, commit `a4b29742b6274a01c7398ca3e800a19fa6d2c9aa`; paths are relative to +`src/SIL.Machine.Morphology.HermitCrab/` unless qualified. The six central files (`AnalysisStratumRule`, +`AnalysisStateKey`, `MemoizedCombinationRuleCascade`, `AnalysisAffixTemplateRule`, `Morpher`, `Word`) are +text-identical in this PR worktree's starting commit `5bbfa018bdbd5228c3f1aa73db951a4f01d167d3`, ignoring +line endings. **Rust** paths are relative to the supplied checkout's `rust/crates/`, with branch tip +`fa3bfb186852baaaf1ed28fcc0ba4602b2c19e96` at inspection. This identifies inspected files; it is not a +claim that either external checkout was clean. + +| Anchor / fact | C# file:line | Rust file:line | +| --- | --- | --- | +| **C1 — cascade and template interleaving** | `AnalysisStratumRule.cs:24–61,128–137,174–244`; base combination loop: `src/SIL.Machine/Rules/CombinationRuleCascade.cs:25–51` | `pg-rules/src/cascade.rs:219–265`; `pg-rules/src/stratum.rs:811–888`; strata/results: `pg-parse/src/morpher.rs:701–765` | +| **C2 — application/compound guards and feature effects** | `MorphologicalRules/AnalysisAffixProcessRule.cs:39–63`; `MorphologicalRules/AnalysisCompoundingRule.cs:39–77,129–133`; realizational exception: `MorphologicalRules/AnalysisRealizationalAffixProcessRule.cs:40–58` | `pg-rules/src/stratum.rs:707–724,792–805` | +| **C3 — existing sharing and actual keys** | `AnalysisStateKey.cs:31–36,60–85,94–104`; merge: `AnalysisStratumRule.cs:130–167`; inner memo: `MemoizedCombinationRuleCascade.cs:39–94`; battery memo: `AnalysisStratumRule.cs:196–218` | key fields: `pg-rules/src/analysis_state_key.rs:22–47`; key construction: `pg-rules/src/stratum.rs:664–694`; ordered output key: `pg-rules/src/word.rs:309–321,410–433`; stratum fold: `pg-rules/src/stratum.rs:1180–1239` | +| **C4 — lexical lookup, expansion, acceptance, fallback** | `Morpher.cs:167–201,346–367,417–445,634–665`; expansion: `Word.cs:445–486` | `pg-parse/src/morpher.rs:459–499,765–826,933–997`; expansion: `pg-rules/src/word.rs:487–548` | +| **C5 — ordered trails are observable** | `Word.cs:202–215,253–259,339–347,371–382,627–646` | `pg-rules/src/word.rs:318–319`; pending-rule test and consumption: `pg-rules/src/stratum.rs:1546–1571,1606–1609` | +| **C6 — optional nodes and root-trie sharing** | `PhonologicalRules/NarrowAnalysisRewriteRuleSpec.cs:40–59`; optional insertion argument: `src/SIL.Machine/Annotations/Shape.cs:160–174` | deletion sites/insertion: `pg-rules/src/rewrite.rs:2536–2581`; root trie consume/skip/dedup: `pg-parse/src/root_trie.rs:227–277` | +| **C7 — general matcher work carries captures** | `MorphologicalRules/AnalysisAffixProcessRule.cs:24–33`; capture-dependent reconstruction: `MorphologicalRules/AnalysisMorphologicalTransform.cs:61–100`; optional forks/registers/bindings: `src/SIL.Machine/FiniteState/TraversalMethodBase.cs:248–308` | template slot results and their semantic deduplication: `pg-rules/src/stratum.rs:1057–1075` | +| **C8 — template admission and slot semantics** | `AnalysisAffixTemplateRule.cs:18–73`; non-disjunctive union: `src/SIL.Machine/Rules/RuleBatch.cs:52–61`, configured at `AnalysisStratumRule.cs:27–30` | `pg-rules/src/stratum.rs:894–962,990–1075` | +| **C9 — proposer is followed by exact-engine confirmation** | exact confirmation target: `Morpher.cs:346–367` | current emitter: `pg-foma/src/emit.rs:2615–2675,2704–2745,2768–2808`; confirmation filters/reparse: `pg-foma-runtime/src/confirm.rs:498–517` | + +In particular, the C# key contains shape, stratum, syntactic and realizational features, non-head count, +and exact per-rule unapplication counts. It contains neither final-template state nor a separate morphology +history. Rust adds both, saturates counts at each rule's maximum, and clears its analysis final-template +state at the stratum fold (`pg-rules/src/stratum.rs:1196–1200`). The same key name does not imply the same +equivalence relation [C3]. Neither key is a synthesis key. This inspection verifies these fields and call +sites; it does not certify a new key as complete for every continuation. + +There is a second relevant implementation difference: Rust's common dispatch applies the count gate and +pushes a pending-rule entry for every morphological rule kind. C# realizational unapplication has no count +gate and records counts without adding a pending-trail entry [C2, C5]. These are verified bookkeeping +differences, not a demonstrated output divergence here. Any proof using bounded counts or trail identity +must address the C# semantics explicitly; copying Rust's control vector is insufficient. + +## Ranked candidates + +Rank is the order in which to spend research effort under the memory constraint, not a promised speedup. +The reference labels point to the bibliography below; **every literature citation is from memory and was +not checked against a publication or online**. + +| Rank / technique | Home field + reference | Mapping in C# / Rust | Exact-parity condition; check and fallback | Expected asymptotic change | Cheapest ceiling experiment | Difference from closed rows | +| --- | --- | --- | --- | --- | --- | --- | +| **1. Grammar-compiled residual viability**; includes lexical goal propagation, abstract interpretation, and residual lexical coaccessibility | Program analysis / demand-driven databases / automata; [L1, L2, L5] | Before recursive expansion in C1/C2, with summaries of templates C8, optional nodes C6, and lexical targets C4; Rust `combination_rec` and `apply_one_mrule` | **Static:** sound transfer summaries for all remaining operations and reachable strata/roots, including compounds. **Runtime:** prune only a proved-empty abstract continuation. Unsupported operation, budget exhaustion, guessing/overlay not covered: unknown, ordinary search | Fewer explored prefixes; a certified precedence restriction can change `k!` orders to `L` viable linear extensions. No universal worst-case improvement | Shadow checks at C1/C2: count provably dead branches and descendant rule/template/lookup work, unpooled; count summary size and query work | Rows 10/11 reopen only with residual transformations **and compounding** covered; root-reachable synthesis failures remain. This is neither row 8's bare-root length bound nor row 19's feature-only synthesis replay | +| **2. Compile the template battery into a shared-prefix control trie** | Grammar factoring / parsing / finite-state compilation; [L3, L5]. A specialization of the reports' compiled-control ideas, not a reported measurement | C8: replace repeated reverse slot walks inside C1's battery. Rust `run_template_batch`, `template_unapply_slots`, `apply_slot_batch` | **Static:** share only identical slot-batch operations and optionality, with per-template admission and terminal labels retained. **Runtime:** identical input to a shared prefix; preserve all branch outcomes and existing dedup semantics. Trace, unsupported effects, or control-size budget: original battery | For `t` templates sharing prefix work `W_p`: `t·W_p` becomes `W_p`, plus divergent branches and terminal bookkeeping. For a deterministic `p`-slot prefix, this is `t·p` to `p`. No reduction of unrelated rule permutations | Compile-only census of distinct reversed slot prefixes; at C8 count attempts/outcomes at common prefixes, admitted templates, skip-only terminals, clones, and maximum DFS depth. Separate per-battery savings from avoiding battery calls | No `(Word,state)` result cache: sharing is in the grammar's control trie. Does not repeat rows 1/2/18 or the alternatives memo/hoist. Deferred cloning and the final-template filter already have separate records | +| **3. Trace-class POR with complete ordering recovery**; critical-pair analysis supplies certificates | Concurrency / term rewriting; [L4] | C1 unordered recursion; class readout at C4/C5. Same boundaries in Rust `combination_rec` and the expansion/synthesis consumer | **Static:** strong independence with enabledness and branch/effect correspondence, not just equal final shapes. **Runtime:** stack-local sleep sets; retain every class's ordered histories and all stopping/template points. Uncertified pair is dependent; unhandled rule kind uses ordinary search | Ordered-prefix exploration becomes trace-class exploration; fully independent single-use, single-outcome rules give `2^k` classes and `O(k·2^k)` transition attempts. Recovery/synthesis can still require `k!` trails | Count certified pairs **where enabled**, classes, and prefix/template work avoided; separately count recovered trails and synthesis. Earlier eligibility was only 0.00–0.05% of attempts in the supplied research context | Recovers the orders discarded by the rejected commuting-order canonicalization. Skipping class members without a full synthesis/output-invariance proof repeats row 6; merging synthesis steps repeats row 5 | +| **4. Exact finite-state compilation of a verified subgrammar** | Computational morphology / transducers; [L5] | Compile C1/C2/C6/C8 and the lexicon/forward checks C4 into a labelled relation; Rust emitter C9 is a starting point for coverage inspection, not an exact replacement | **Static:** exact regular encoding of features, annotations, root/non-head identity, all ordered outputs, copying and rule limits; unsupported construct or compile-size overflow rejects eligibility. **Runtime:** budget checked before allocating input-product storage; ordinary parser on refusal | Control alone: `prod(i)(m_i+1)` states, at most `k` outgoing choices per state; for single use `2^k` states / `O(k·2^k)` arcs. Full relation and output enumeration may still be exponential/factorial | Coverage audit and compiled states/arcs first; then accepted labelled paths, confirmation work actually removed, distinct complete analyses, and bytes. Compare against 367,278 trails, not just one output | Shares a **grammar-compiled** relation without erasing order or adding a per-search chart. A permissive proposer that still reparses has not removed the cascade | + +## 1. Residual viability: prune only a proved-empty continuation + +Let `S` be a concrete HermitCrab internal `Word` plus cascade/stratum control. Let `Reach(S)` include +every possible stopping candidate after the remaining inverse operations, including lookup in intermediate +strata and every permitted compound split. Choose a finite abstract domain, an abstraction `alpha`, and +transfers satisfying `alpha(T(S)) <= T#(alpha(S))` for **sets** of results. Compute a conservative closure +of the remaining transfers. If that closure has empty intersection with an over-approximation of all +compatible lexical targets, no concrete continuation can find a root. Removing that branch therefore +removes no successful synthesis and no reported analysis. This follows by induction over each concrete +continuation; it needs no assumption that rule order commutes. + +The target set must include everything ordinary lookup could supply. Compound non-head lookup is already +inside unapplication [C2]; it cannot be ignored or treated as a single final root. Root guessing must +either have its own sound target abstraction or retain the original candidates for the fallback branch +[C4]. Rust supplied-root overlays also affect targets (`pg-parse/src/morpher.rs:459–499`). A finite list of +ordinary root spellings is insufficient in either situation. + +Start with a grammar-compiled coarse domain: mandatory/optional segment-length intervals, safe segment +classes, and only feature constraints whose transfer is proved. Inverse truncation can **grow** a shape: +`MorphologicalRules/AnalysisMorphologicalTransform.cs:54–80,108–132` reconstructs uncaptured input parts. +Optional nodes require both skip and consume possibilities [C6]. An unsupported transform maps to top, +never to failure. Staged +refinement is useful only if every stage remains an over-approximation; an inconclusive expensive stage +returns unknown. This is the safe part of R6's abstract interpretation, R4's magic sets, R3's lazy product, +and R5's extension-oracle proposals. They are one pruning family, not four independent wins. + +R4's precedence idea is a special case with a stricter obligation. To install `r before s`, prove that +**every successful complete path containing both occurrences** respects that order, including paths with +intervening enabling rules/templates. Failure of the adjacent two-step sequence `s;r` is insufficient. +Rules are optional: `s` without `r` must remain allowed. A runtime implementation can forbid a later `r` +after `s` only under that complete-path certificate; it cannot require `r` before admitting `s` altogether. +Begin with single-use rules, no certified edges for unknown constructs, and no inference from prefix/suffix +direction alone. A cycle says that the affected combination is impossible, not that individual rules +should be deleted. + +This must remain grammar-sized: use finite transfer tables and stack-local query work, not a table keyed +by every visited `S`. A lazy lexical product that needs an unbounded visited-state set fails the memory +constraint; use a statically bounded abstraction or return unknown. Count dead **subtrees and their work** +without double-counting descendants. The ledger's 23.5% pooled dead-step result and zero on one expensive +input are reasons to stop if the stronger oracle still mostly accepts. No lexical oracle can reject a +root-reachable path merely because later synthesis will fail. + +## 2. Make the template battery cheaper by factoring its control + +The code provides a useful, smaller target than a whole-search chart. C# builds a non-disjunctive battery +and non-disjunctive batches for individual slots. Each template walks slots from last to first; an optional +slot permits the unchanged input to continue, while a required slot stops that skip path [C8]. Templates +are therefore finite control programs over **set-valued** morphological transformations. + +Compile a trie of those reversed control programs. Share a prefix only when its slot operation has the +same ordered rule identities, application role, optionality, and relevant compiled settings. Keep terminal +template identities and their entry predicates. At each battery invocation, evaluate each template's +selector/required-feature predicate on the **original input**, as today; a shared edge is eligible only +for the admitted descendant templates. Preserve per-template finality policy in Rust. Do not move entry +predicates to a transformed intermediate input [C8]. + +For an admitted template, the original slot path has an identical path through the trie. Conversely, +every trie terminal has an admitted original template. At a shared edge the inputs and operations are +identical, so applying the operation once yields exactly the result set each original execution would +have produced. Induction over prefix length preserves each template's ordered rule trail and complete +internal `Word` effects. Branches split at differing slots; required slots retain no skip edge, optional +slots retain both routes, and all-skipped terminals remain. The union of terminal results is consequently +the original battery result set. This is control factoring, with no identification of two independently +arrived search states and no commutation assumption. + +Two implementation boundaries matter. First, retain the existing slot/battery deduplication and representative +semantics: C# uses `Word.ValueEquals`, which omits syntactic features; Rust explicitly generalizes syntactic +features on collisions [C5, C8]. A new global semantic-key merge is not justified by the trie proof. The +first eligible fragment should exclude differing duplicate-result representatives unless their equivalence +is certified. Second, keep template-specific effects at terminals; if an effect changes a subsequent slot +operation, it belongs in the prefix identity or prevents sharing. Tracing follows the ordinary battery, +because template-specific trace events observe separate executions. + +The new trie has at most one node per original slot-prefix position: `O(B)` grammar storage for `B` total +slots, plus rule lists and terminal membership. Evaluate it depth-first with only the current internal +`Word`, branch iterator, and admitted-template information on the stack. No completed `(trie node, Word)` +results are retained for later arrivals. Live traversal storage is bounded by grammar depth and the current +rule-result enumeration, not total states visited. Existing result accumulation is still required. If +implementation instead introduces a growing active-state chart, it has left this proposal. + +This can reduce the factor due to duplicated template control. It does **not** turn the 43,641 battery calls +into 4,988 calls; that would need different search structure or the ruled-out memo. No supplied report +measures common slot prefixes, so a compile-only prefix census is the first gate. This specialization +combines R2's structural sharing and the reports' finite-state compilation ideas at a grammar-sized +boundary. It is a derived proposal, not an experiment any report already ran. + +## 3. Partial-order reduction: preserve the trace class, then recover its orders + +R1 supplies the essential correction to canonicalization. A class contains paths related by swapping +adjacent independent **occurrences**, not by sorting a multiset and deleting its other orders. With repeated +applications, occurrence identity includes its count/position; nondeterministic allomorph outcomes must +also have a proved correspondence. Templates, compounding, and unsupported operations initially act as +dependent barriers. + +For every certified pair and reachable eligible state, require (1) either transition preserves the other's +enabledness and limits; (2) both two-step result sets have a branch-by-branch bijection; (3) matched endpoints +agree on everything subsequent unapplication, templates and lookup can read; and (4) occurrence effects can +be replayed in each recovered order with all trail, non-head and feature history preserved. Disjoint-looking +string edits, critical-pair joinability, or local confluence alone prove none of these for the complete +internal `Word`. C2 and C7 show why feature effects and captured reconstruction are part of the obligation. +Mark uncertain pairs dependent; a conservative restricted certificate is preferable to runtime speculation. + +Under these conditions, adjacent swaps neither lose a path nor invent an impossible one. Their transitive +closure partitions executions into trace classes. A sleep-set traversal can explore one representative +of each class, **including classes of every stopping prefix**; the dependency ordering of occurrences +retains its other linearizations. Template opportunities at intermediate prefixes must remain represented, +not just the terminal class. Lazy enumeration of each class's linear extensions, with exact occurrence +effects restored, reconstructs the original ordered synthesis inputs. Running ordinary synthesis on each +restores precisely the original analysis set. This is the completeness argument missing from merely +keeping the representative. + +Store the grammar's conservative pair matrix (`O(k^2)`) and current path/sleep/linearization stacks. The +occurrence bound `D` must come from actual grammar limits; a current class can use `O(D^2)` ordering data. +Replace that class's explicit histories or stream them to the existing consumer; do not add a search-wide +class registry or DPOR backtracking graph. If the certificate or stack budget cannot be satisfied, use +the ordinary traversal. A streaming consumer is an architectural change to C4, not a drop-in set key. + +For fully independent, single-use rules with one outcome per occurrence there is one class per subset, +hence `2^k` classes rather than factorially many ordered prefixes. Allomorph branching can add further +classes. **That bound is on analysis traversal, not end-to-end parsing.** If +forward synthesis or output identity distinguishes class members, all relevant orders still need exact +validation/readout. Skipping their synthesis needs a separate proof of identical complete acceptance and +reported analysis, for every root and allomorph. C5 and ledger rows 5/6 rule out assuming it globally. +R6's stronger output-invariance condition is right for **discarding** orders; R1's recovery construction +is right for avoiding repeated pre-synthesis work while preserving them. The earlier 0.00–0.05% eligibility +is a warning, not evidence that better certificates exist. + +## 4. Finite-state compilation: exact only for a certified fragment + +An exact compiler would encode a relation from surface input to complete accepted analysis labels. It +must represent the inverse/forward rule behavior, lexical alternatives, template paths, stratum control, +features, annotations and non-heads that affect validity, with every observable ordered identity retained. +Compile bounded rule-use counts as a control vector: there are `prod(i)(m_i+1)` vectors and at most `k` +choices out of one vector. The vector alone describes permission to try rules, not their semantic outputs. + +If each primitive's encoding is exact, relational union preserves all branches and composition preserves +all legal sequences. Intersecting with the input and lexical/validity constraints then preserves exactly +the accepting paths. This gives a structural induction proof for the compiled fragment. Output labels +must carry sufficient allomorph/morpheme order, root position, non-head provenance and feature information +to reconstruct the same reported analyses. If compilation is only an over-approximation, it may reject +proved-impossible branches as in candidate 1, but its surviving paths still require ordinary confirmation. + +The compiler must refuse constructs with no established regular encoding. In particular arbitrary copying +of an unbounded captured string is not an ordinary finite-state relation; finite alphabets alone do not +fix that. Bounded copying, finite feature domains and bounded compound/rule counts need explicit encodings, +not assumptions. Unknown annotation or constraint behavior also rejects exact eligibility. Compile-time +state/arc/byte limits bound grammar storage, with fallback if composition expands too far. + +For an accepted finite relation with `Q` states and `E` arcs and an input of length `n`, recognition can +use a grammar/input-bounded product, rather than materializing each complete trail. Epsilon behavior must +be handled exactly, and productive zero-input cycles must be excluded or proved bounded. An `O(nQ)` +reachability/coaccessibility structure is acceptable only with its byte bound checked before allocation; +otherwise use ordinary parsing. Recognition sharing does not bound distinct labelled outputs or the +cost of eliminating duplicate output identities. Both the compiled relation and its readout can still +be exponential; `2^k` is the **control-state** bound, not the number of analyses. + +R3 is right that a compact proposer plus exact reparse does not solve the confirmation cascade. Its +blanket `O(k^2)` description of the present proposer is too broad: the actual emitter has a subset-mask +path for eligible unordered underlying-token zones, a dedicated-level fallback, and a legacy every-rule +per-level path [C9]. The comment in `pg-foma/src/unordered.rs:44–61` describes the latter, not all current +execution. No claim about the proposer's universal completeness or exact parity is established here. + +## Where the six reports disagreed + +| Issue | Resolution after reading code and the ledger | +| --- | --- | +| **Is the cascade already subset DP?** R2/R4/R5 emphasize a new inner chart; R6 says that would repeat memoization | R2 is right about **Rust**: C1 uses recursive enumeration and C3 merges later. R6 is right about **C#**: the sequential inner cascade and battery already replay state-keyed results. A count vector is not a sufficient semantic key. A capped chart still grows with search and is outside the hard constraint | +| **What is in `AnalysisStateKey`?** The brief/R2 describe more than the older probe-design key | R2 matches Rust's morphology-history/finality fields. Applying that description to C# is wrong; C3 lists the actual different keys. Neither warrants synthesis merging | +| **Every rule is bounded by its application count**, assumed in the brief and count-vector proposals | Correct for Rust's shared gate and C# ordinary affix/compound rules; C# realizational unapplication is an exception and does not extend the pending trail [C2, C5]. The general bound is conditional, not established for every rule kind | +| **Can commuting orders be reduced?** R1 permits recoverable classes; R6 requires invariant output | Both conditions have a different scope. Recovery can preserve every path; discarding an order requires R6's stronger condition. This resolves the ledger's parse-signature objection, but does not establish a synthesis saving | +| **Packed forest/ZDD/product graph is the best lead** in R2/R4/R5, versus R6's memory objection | R5 is right that an exact product needs full synthesis state and ordered continuation/provenance; one final analysis supplies no small-state bound. R6 is right under this job's constraint. Generic product charts are not recommended. A ZDD encodes sets; ordered trails need position/occurrence encoding, whose size none of the reports bounds | +| **Optional deletion is already compact**, emphasized by R3/R6 | Correct for shape construction and Rust trie active states [C6]. It does not establish cheap general morphological matching: C7 needs captures/bindings, and ledger row 23 closes position-only deduplication | +| **The proposer is `O(k^2)`**, R3 | Verified only for the legacy branch. C9's executed mode dispatch and mask loop take precedence over the older descriptive comment | +| **Simple static precedence could remove impossible orders**, R4 | Conditional only: adjacent-pair failure does not rule out an intervening enabler, and an optional predecessor cannot become mandatory. Candidate 1 states the stronger complete-path certificate | +| **A 64-key root cache bounds walks by `min(U,64)`**, R5; R4 also proposes bounded reuse | The raw search boundary is reusable under fixed grammar/overlay [C4], but the bound is wrong when `U > 64`: uncached keys are searched on every revisit. A byte cap must also cover shapes and result lists. Under the no-memo instruction this is not recommended | + +## Considered and not recommended + +- **Count-vector charts, generic packed analysis forests, ZDD trail tables, analysis-DAG × synthesis-state + products:** search-growing tables; rows 1/4/5/6/16 already test the necessary convergence, and a fixed cap + with restart does not make these a different allowed technique. +- **Packed synthesis readout keyed only by trail position or current shape:** omits ordered continuation + and root-dependent state; rows 5/6/16 close it. `O(Q+T+A)` only helps after proving `Q` and provenance + readout small, which no report does. +- **State/position-only matcher deduplication or two-pass capture reconstruction:** row 23 and the + [two-pass record](rejected-optimizations/two-pass-nondeterministic-traversal.md) already settle these; + the safe group-free fragment is existing prior work, not a new research result. +- **Fixed-cap exact root-result cache:** useful only for raw lookup, still memoization, and neither + 56 shapes nor 64 entries bounds all `(stratum, shape)` result bytes or repeated misses. +- **Root-only lexical gates, edge prefilters, current-length bounds, feature-only abstract replay, and + nogood subsumption:** rows 8–11/14/19/22 already cover them; candidate 1 survives only with its stronger + residual relation and a new measured dead-subtree ceiling. +- **Commuting-order canonicalization without recovery, symmetry/orbit counts:** distinct rule/morpheme + identities and ordered trails remain observable; no output-fixing grammar automorphism was established. +- **Meet-in-the-middle:** large midpoint frontiers violate the memory constraint; unordered rule splits + do not automatically turn factorial work into two square-root searches. +- **A*, k-best/k-shortest paths, reverse-search enumeration alone:** all valid analyses are required; + rearranging or ranking trail enumeration does not eliminate failed trails without a sound extension oracle. +- **Generic Earley/CKY, scalar semirings, equality saturation/e-graphs:** no proved semantics-preserving + span/equivalence boundary for these transforms; adding full transformation state/provenance restores + the unresolved search size. +- **Bimachines, two-level morphology, compile-replace, or parser names alone:** constrained morphology + formalisms supply candidate 4's tools, not a correctness proof for arbitrary HermitCrab behavior. + +## Implications for the two work items + +**Lexicon-driven pruning during unapplication.** Start with candidate 1 in shadow mode at C1/C2. The +deliverable is a per-input census of proved-dead subtrees, saved descendant work, unknown cases and +abstraction cost. Include remaining morphology, template transitions, intermediate-stratum lexical goals, +optional sites and compound non-head roots. Disable pruning where guessing/overlays are uncovered. Close +it if the result merely reproduces row 10's no-op or row 11's weak root-only ceiling. Exact product +coaccessibility is a possible stronger oracle only after a grammar-sized representation exists; building +a whole search graph to ask whether that graph has a successful leaf is not an allowed shortcut. + +**Making the affix-template battery cheaper without memoization.** Start with candidate 2's static +prefix census and admission/slot work counts. Its target is repeated control **within one invocation**. +Candidate 1 can also avoid an invocation whose entire continuation is proved dead; candidate 3 might +avoid repeated prefix work if independence coverage is real. Neither the 43,641/4,988 ratio nor the +historical battery memo authorizes reinstating a cache. Keep deferred cloning, safe edge dispatch and +the final-template policy distinct from this research: their evidence and limits already have records. +In particular, the [final-template record](rejected-optimizations/final-template-prune.md) has an addendum +reopening a narrower guard; its old zero-prune result is not a new lead to rediscover here. + +For either item, compare complete analysis sets with ordered morpheme/allomorph identities, root position, +features and non-head provenance. Do not use the legacy count-degenerate signature called out in the +September disposition. Tests support a proof; a finite corpus cannot prove parity for every export. +Tracing and stateful selectors use the existing path unless their invocation/event semantics are preserved. +Configured alternative/work limits also use the existing path unless the same limit behavior is reproduced +[C1, C4]; a resource-limited optimized query returns unknown and falls back, never a partial exact result. + +## Evidence limits and references + +**Not verified:** the brief's timings/counters, the earlier POR eligibility percentage, exact applicability +of any proposed certificate/compiler to the motivating grammar, common template-prefix coverage, a small +packed product/ZDD bound, and any end-to-end speedup. No real-grammar benchmark or new optimization was +run. Historical measurements are attributed to the supplied brief or the existing records, not fresh +results. Code inspection verifies the listed control/data facts; it does not establish universal parity +of the current Rust port or the current FST proposer. All future experiments need the stronger output +signature and ordinary-parser fallback. + +All entries below are **[memory; bibliographic details and applicability not independently verified]**. +No network or publication lookup was available. They identify home fields, not proofs of these proposals. + +- **L1.** P. Cousot and R. Cousot (1977), “Abstract Interpretation: A Unified Lattice Model for Static + Analysis of Programs by Construction or Approximation of Fixpoints.” +- **L2.** C. Beeri and R. Ramakrishnan (1991), “On the Power of Magic.” +- **L3.** B. Billot and B. Lang (1989), “The Structure of Shared Forests in Ambiguous Parsing.” Structural + sharing background; the grammar-prefix specialization here does not adopt a runtime parse chart. +- **L4.** A. Mazurkiewicz (1987), “Trace Theory”; P. Godefroid (1996), *Partial-Order Methods for the + Verification of Concurrent Systems*; F. Baader and T. Nipkow (1998), *Term Rewriting and All That*. +- **L5.** R. Kaplan and M. Kay (1994), “Regular Models of Phonological Rule Systems”; M. Mohri (1997), + “Finite-State Transducers in Language and Speech Processing”; K. Beesley and L. Karttunen (2003), + *Finite State Morphology*. + +The reports also invoke Held–Karp sequencing DP, Tomita-style shared forests, Minato's ZDDs, graph +coaccessibility, and PC-KIMMO/AMPLE/XAmple. These are useful vocabulary for the considered designs; +none supplies the missing state-equivalence, memory, or arbitrary-HermitCrab encoding proof. From 3d824e171e0805a90634ea9221f4cfe54955c0fa Mon Sep 17 00:00:00 2001 From: John Lambert Date: Wed, 21 Oct 2026 20:49:25 -0400 Subject: [PATCH 4/7] Label the C# anchors in the asymptotics research as a pre-#491 snapshot The anchors were read at a4b29742, before #491 added final-template state to AnalysisStateKey, so the key comparison described a master that no longer exists. It is now dated, and says what #491 changed. Co-Authored-By: Claude Opus 5.5 --- .../hermitcrab-search-asymptotics-research.md | 20 ++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/docs/hermitcrab-search-asymptotics-research.md b/docs/hermitcrab-search-asymptotics-research.md index 48951c95f..6c1874377 100644 --- a/docs/hermitcrab-search-asymptotics-research.md +++ b/docs/hermitcrab-search-asymptotics-research.md @@ -67,9 +67,10 @@ checkout, commit `a4b29742b6274a01c7398ca3e800a19fa6d2c9aa`; paths are relative `src/SIL.Machine.Morphology.HermitCrab/` unless qualified. The six central files (`AnalysisStratumRule`, `AnalysisStateKey`, `MemoizedCombinationRuleCascade`, `AnalysisAffixTemplateRule`, `Morpher`, `Word`) are text-identical in this PR worktree's starting commit `5bbfa018bdbd5228c3f1aa73db951a4f01d167d3`, ignoring -line endings. **Rust** paths are relative to the supplied checkout's `rust/crates/`, with branch tip -`fa3bfb186852baaaf1ed28fcc0ba4602b2c19e96` at inspection. This identifies inspected files; it is not a -claim that either external checkout was clean. +line endings. The C# anchors are a snapshot at `a4b29742`, before #491 (`b240b7b5`) merged; they were not +re-read at current master. **Rust** paths are relative to the supplied checkout's `rust/crates/`, with +branch tip `fa3bfb186852baaaf1ed28fcc0ba4602b2c19e96` at inspection. This identifies inspected files; it is +not a claim that either external checkout was clean. | Anchor / fact | C# file:line | Rust file:line | | --- | --- | --- | @@ -83,12 +84,13 @@ claim that either external checkout was clean. | **C8 — template admission and slot semantics** | `AnalysisAffixTemplateRule.cs:18–73`; non-disjunctive union: `src/SIL.Machine/Rules/RuleBatch.cs:52–61`, configured at `AnalysisStratumRule.cs:27–30` | `pg-rules/src/stratum.rs:894–962,990–1075` | | **C9 — proposer is followed by exact-engine confirmation** | exact confirmation target: `Morpher.cs:346–367` | current emitter: `pg-foma/src/emit.rs:2615–2675,2704–2745,2768–2808`; confirmation filters/reparse: `pg-foma-runtime/src/confirm.rs:498–517` | -In particular, the C# key contains shape, stratum, syntactic and realizational features, non-head count, -and exact per-rule unapplication counts. It contains neither final-template state nor a separate morphology -history. Rust adds both, saturates counts at each rule's maximum, and clears its analysis final-template -state at the stratum fold (`pg-rules/src/stratum.rs:1196–1200`). The same key name does not imply the same -equivalence relation [C3]. Neither key is a synthesis key. This inspection verifies these fields and call -sites; it does not certify a new key as complete for every continuation. +In particular, at `a4b29742` the C# key contains shape, stratum, syntactic and realizational features, +non-head count, and exact per-rule unapplication counts. It contains neither final-template state nor a +separate morphology history; #491 later added final-template state to the key, so current master lacks only +the morphology history. Rust adds both, saturates counts at each rule's maximum, and clears its analysis +final-template state at the stratum fold (`pg-rules/src/stratum.rs:1196–1200`). The same key name does not +imply the same equivalence relation [C3]. Neither key is a synthesis key. This inspection verifies these +fields and call sites; it does not certify a new key as complete for every continuation. There is a second relevant implementation difference: Rust's common dispatch applies the count gate and pushes a pending-rule entry for every morphological rule kind. C# realizational unapplication has no count From cf7667bcec7182078b413c4a5569f1a5135f5c5e Mon Sep 17 00:00:00 2001 From: John Lambert Date: Wed, 21 Oct 2026 21:24:25 -0400 Subject: [PATCH 5/7] Say the retained optimizations live on the archive branch, not master "Status: retained" read as shipped, but none of the nine records' symbols exist on master; they are on perf/hc-optimization-archive. Each record and the ledger's disposition now say where they are retained. Co-Authored-By: Claude Opus 5.5 --- docs/hermitcrab-optimization-ledger.md | 12 ++++++------ docs/optimizations/deferred-template-clone.md | 4 ++-- docs/optimizations/edge-segment-prefilter.md | 6 +++--- .../feature-struct-frozen-tree-clone.md | 2 +- .../feature-struct-frozen-tree-equality.md | 2 +- .../feature-struct-visited-state-reuse.md | 5 +++-- docs/optimizations/flat-register-arrays.md | 7 ++++--- docs/optimizations/shape-copy-single-pass-map.md | 2 +- docs/optimizations/shape-sharing-on-clone.md | 8 ++++---- .../syntactic-feature-structure-sharing.md | 5 +++-- 10 files changed, 28 insertions(+), 25 deletions(-) diff --git a/docs/hermitcrab-optimization-ledger.md b/docs/hermitcrab-optimization-ledger.md index ed8c839e5..cc2b24312 100644 --- a/docs/hermitcrab-optimization-ledger.md +++ b/docs/hermitcrab-optimization-ledger.md @@ -40,12 +40,12 @@ speedup, or numbered optimization row is claimed.** ## Final disposition of the September follow-up -The follow-up branch does deliver product improvements. Its authoritative summary is -`hermitcrab-perf-2026-09-counts.md`, and each retained optimization has one record under -`docs/optimizations/`. The retained set is: deferred template cloning, feature-structure traversal-state -reuse, frozen-flat-tree equality, frozen-flat-tree cloning, single-pass `Shape.CopyTo` mapping, -copy-on-write shape sharing, flat FST register arrays, the analysis edge-segment prefilter, and -copy-on-write syntactic feature structures. +The follow-up branch does deliver product improvements, retained on `perf/hc-optimization-archive` and not +merged to master. Its authoritative summary is `hermitcrab-perf-2026-09-counts.md`, and each retained +optimization has one record under `docs/optimizations/`. The retained set is: deferred template cloning, +feature-structure traversal-state reuse, frozen-flat-tree equality, frozen-flat-tree cloning, single-pass +`Shape.CopyTo` mapping, copy-on-write shape sharing, flat FST register arrays, the analysis edge-segment +prefilter, and copy-on-write syntactic feature structures. Four attractive experiments were deliberately excluded from the final product diff and are recorded under `docs/rejected-optimizations/`: the negative two-pass nondeterministic traversal, the inert final-template diff --git a/docs/optimizations/deferred-template-clone.md b/docs/optimizations/deferred-template-clone.md index e5c6d5d7f..59155adf1 100644 --- a/docs/optimizations/deferred-template-clone.md +++ b/docs/optimizations/deferred-template-clone.md @@ -1,7 +1,7 @@ # Deferred per-template clone in `AnalysisAffixTemplateRule` -**Status:** retained. **Portable:** the observation is; the saving is allocation, so the Rust port gains -little unless it also clones eagerly. +**Status:** retained on `perf/hc-optimization-archive`, not merged. **Portable:** the observation is; the +saving is allocation, so the Rust port gains little unless it also clones eagerly. ## The observation diff --git a/docs/optimizations/edge-segment-prefilter.md b/docs/optimizations/edge-segment-prefilter.md index c1a905480..e6215d81b 100644 --- a/docs/optimizations/edge-segment-prefilter.md +++ b/docs/optimizations/edge-segment-prefilter.md @@ -1,8 +1,8 @@ # Edge-segment prefilter for analysis affix rules -**Status:** retained (`AnalysisAffixProcessRule`; internal same-binary toggle -`Morpher.EdgePrefilterEnabled`). **Portable:** yes; it is a property of the pattern semantics, -not of the C# engine. +**Status:** retained on `perf/hc-optimization-archive`, not merged (`AnalysisAffixProcessRule`; internal +same-binary toggle `Morpher.EdgePrefilterEnabled`). **Portable:** yes; it is a property of the pattern +semantics, not of the C# engine. ## The observation diff --git a/docs/optimizations/feature-struct-frozen-tree-clone.md b/docs/optimizations/feature-struct-frozen-tree-clone.md index 642c1cde2..3677c7750 100644 --- a/docs/optimizations/feature-struct-frozen-tree-clone.md +++ b/docs/optimizations/feature-struct-frozen-tree-clone.md @@ -1,6 +1,6 @@ # Reuse or skip the feature-structure clone map -**Status:** retained (`FeatureStruct.Clone`). +**Status:** retained on `perf/hc-optimization-archive`, not merged (`FeatureStruct.Clone`). **Portable:** yes; the specific pool is .NET-oriented, while the tree proof is general. ## Invariant diff --git a/docs/optimizations/feature-struct-frozen-tree-equality.md b/docs/optimizations/feature-struct-frozen-tree-equality.md index 5378fb48e..6f34aa04d 100644 --- a/docs/optimizations/feature-struct-frozen-tree-equality.md +++ b/docs/optimizations/feature-struct-frozen-tree-equality.md @@ -1,6 +1,6 @@ # Allocation-free equality for frozen flat feature structures -**Status:** retained (`FeatureStruct.ValueEquals`). +**Status:** retained on `perf/hc-optimization-archive`, not merged (`FeatureStruct.ValueEquals`). **Portable:** yes; the proof depends on immutability and tree shape, not the CLR. ## Invariant diff --git a/docs/optimizations/feature-struct-visited-state-reuse.md b/docs/optimizations/feature-struct-visited-state-reuse.md index d140a2728..f7c2b63ee 100644 --- a/docs/optimizations/feature-struct-visited-state-reuse.md +++ b/docs/optimizations/feature-struct-visited-state-reuse.md @@ -1,7 +1,8 @@ # Reuse feature-structure traversal state -**Status:** retained (`FeatureStruct.ValueEquals` and `Freeze`). -**Portable:** the allocation-avoidance principle is portable; the `[ThreadStatic]` pool is .NET-specific. +**Status:** retained on `perf/hc-optimization-archive`, not merged (`FeatureStruct.ValueEquals` and +`Freeze`). **Portable:** the allocation-avoidance principle is portable; the `[ThreadStatic]` pool is +.NET-specific. ## Invariant diff --git a/docs/optimizations/flat-register-arrays.md b/docs/optimizations/flat-register-arrays.md index 359abb125..553e7eb99 100644 --- a/docs/optimizations/flat-register-arrays.md +++ b/docs/optimizations/flat-register-arrays.md @@ -1,8 +1,9 @@ # Flat register arrays in the FST traversal -**Status:** retained (`SIL.Machine.FiniteState`: `Register[]` indexed by `RegisterArray.Idx(r, k)` -replaces `Register[,]` internally; the public `FstResult.Registers` stays two-dimensional and is -materialized once per accepting state). **Portable:** no; a .NET-specific allocation path. +**Status:** retained on `perf/hc-optimization-archive`, not merged (`SIL.Machine.FiniteState`: +`Register[]` indexed by `RegisterArray.Idx(r, k)` replaces `Register[,]` internally; the +public `FstResult.Registers` stays two-dimensional and is materialized once per accepting state). +**Portable:** no; a .NET-specific allocation path. ## The observation diff --git a/docs/optimizations/shape-copy-single-pass-map.md b/docs/optimizations/shape-copy-single-pass-map.md index ed8c15828..900515da2 100644 --- a/docs/optimizations/shape-copy-single-pass-map.md +++ b/docs/optimizations/shape-copy-single-pass-map.md @@ -1,6 +1,6 @@ # Build `Shape.CopyTo`'s node map in the copy pass -**Status:** retained (`Shape.CopyTo`). +**Status:** retained on `perf/hc-optimization-archive`, not merged (`Shape.CopyTo`). **Portable:** yes, when a port has the same node-remapping requirement. ## Invariant diff --git a/docs/optimizations/shape-sharing-on-clone.md b/docs/optimizations/shape-sharing-on-clone.md index 702de8f35..a5ffd563f 100644 --- a/docs/optimizations/shape-sharing-on-clone.md +++ b/docs/optimizations/shape-sharing-on-clone.md @@ -1,9 +1,9 @@ # Copy-on-write Shape sharing in `Word.Clone` -**Status:** retained (`Word.CloneForEngine`, `EnsureOwnShape`, `ResetShape`, `Shape.CreateEmptyLike`). -**Portable:** partly. The *observation* (three of four clone sites never use the copied shape) is -engine-independent; the mechanism (shared reference to an immutable structure) is how a Rust port would -naturally write it already. +**Status:** retained on `perf/hc-optimization-archive`, not merged (`Word.CloneForEngine`, `EnsureOwnShape`, +`ResetShape`, `Shape.CreateEmptyLike`). **Portable:** partly. The *observation* (three of four clone sites +never use the copied shape) is engine-independent; the mechanism (shared reference to an immutable +structure) is how a Rust port would naturally write it already. ## The observation diff --git a/docs/optimizations/syntactic-feature-structure-sharing.md b/docs/optimizations/syntactic-feature-structure-sharing.md index 40ff2af22..9f99d4570 100644 --- a/docs/optimizations/syntactic-feature-structure-sharing.md +++ b/docs/optimizations/syntactic-feature-structure-sharing.md @@ -1,7 +1,8 @@ # Copy-on-write syntactic feature structures in engine clones -**Status:** retained (`Word.CloneForEngine`, `EnsureOwnSyntacticFeatureStruct`). -**Portable:** partly; immutable sharing is general, while the ownership plumbing is specific to this engine. +**Status:** retained on `perf/hc-optimization-archive`, not merged (`Word.CloneForEngine`, +`EnsureOwnSyntacticFeatureStruct`). **Portable:** partly; immutable sharing is general, while the ownership +plumbing is specific to this engine. ## Invariant From bae594e3cd0aa3a59e275338f32afeb9aa1dfe23 Mon Sep 17 00:00:00 2001 From: John Lambert Date: Wed, 21 Oct 2026 21:26:11 -0400 Subject: [PATCH 6/7] Qualify the ledger's parity claims as WordAnalysisSignature matches The signature reduces to morph counts on Sena, Mbugwe and Amharic, as the disposition doc says, so "exact output parity" and "identical analyses" overstated what those checks verified. All three places now say what was compared and point to the caveat. Co-Authored-By: Claude Opus 5.5 --- docs/hermitcrab-optimization-ledger.md | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/docs/hermitcrab-optimization-ledger.md b/docs/hermitcrab-optimization-ledger.md index cc2b24312..4042f16ad 100644 --- a/docs/hermitcrab-optimization-ledger.md +++ b/docs/hermitcrab-optimization-ledger.md @@ -52,10 +52,12 @@ Four attractive experiments were deliberately excluded from the final product di prune, the zero-value alternatives memo/hoist, and the low-ceiling compounding split prefilter. This final disposition supersedes intermediate “ships” language for those experiments later in this historical ledger. -The minimized branch passed all 603 runnable tests in the conformance-inclusive suite (one expected -Windows symlink-privilege skip). A fixed three-word sample had exact output parity on Sena, Amharic, -Indonesian, Mbugwe, and Aweti; only Amharic (1.76x) and Mbugwe (1.83x) cleared the recorded spread and timing -reliability rules. The other three ratios are retained as directional evidence only. +The minimized branch passed all 603 runnable tests in the conformance-inclusive suite (one expected Windows +symlink-privilege skip). A fixed three-word sample matched on `WordAnalysisSignature` on Sena, Amharic, +Indonesian, Mbugwe, and Aweti; on Sena, Mbugwe and Amharic that signature compares morph counts, not +analyses (`hermitcrab-perf-2026-09-disposition.md`). Only Amharic (1.76x) and Mbugwe (1.83x) cleared the +recorded spread and timing reliability rules. The other three ratios are retained as directional evidence +only. ## Earlier ledger round: net speed delivered none @@ -108,7 +110,7 @@ copy-on-write design through `CloneForEngine`; see `docs/optimizations/shape-sha | # | Optimization | Expected / observation | Outcome | | --- | --- | --- | --- | -| 22 | **Analysis-side rule prefilter** | 94.4% of Sena allomorph pattern matches failed, so a cheap necessary-condition check had a large direct-work ceiling. | **RETAINED after the earlier self-time cap was refuted by direct work counts.** The original ~10% ceiling counted matcher self-time but omitted traversal allocation, feature unification, and GC booked elsewhere. The implemented edge-segment prefilter removed **87% of FST traversals**, 54% of arc checks, and 85% of traversal instances on 45 Sena words, with exact parity on those words and all 464 conformance-fixture words. See `docs/optimizations/edge-segment-prefilter.md`. | +| 22 | **Analysis-side rule prefilter** | 94.4% of Sena allomorph pattern matches failed, so a cheap necessary-condition check had a large direct-work ceiling. | **RETAINED after the earlier self-time cap was refuted by direct work counts.** The original ~10% ceiling counted matcher self-time but omitted traversal allocation, feature unification, and GC booked elsewhere. The implemented edge-segment prefilter removed **87% of FST traversals**, 54% of arc checks, and 85% of traversal instances on 45 Sena words, with `WordAnalysisSignature` parity on those words (a morph-count comparison on Sena; see `hermitcrab-perf-2026-09-disposition.md`) and all 464 conformance-fixture words. See `docs/optimizations/edge-segment-prefilter.md`. | | 16 | **`ExpandAlternatives` dedupe** | 20.7% of Sena wall, the largest bucket nobody had instrumented. Apparent census: 395,026 alternatives -> **61** distinct (0.0%), 85% of duplicates same-analysis-word | **CLOSED on the sound census.** With the pending-trail content added to the key, 395,026 -> **351,414 distinct (89.0%)** -- only **11%** are duplicates at all, and all 43,612 of them are same-analysis-word. Sound ceiling = 20.7% x 0.110 = **2.28% of Sena wall**, against a gate of >=10% to build and <5% to close. **The fourth apparent redundancy to evaporate under a complete key** | | 17 | **Per-node cost in the analysis cascade** | 84% of the Amharic cascade is not linguistic computation; clone is 46.7% and 37.4% was unattributed | **Partly settled, remainder still open.** (a) The remainder split **MISSES its <15% gate by ~2.5x** — still ~38% on Amharic `anCascade`. Memo, Trail and **Equality each measured <0.2%**, so `Word.ValueEquals` behind the cascade's `HashSet` is *not* the missing mass, contrary to the standing hypothesis. **Two caveats the first write-up missed.** (i) The ~38% figure comes from a *contended* run — three timing probes were live — and descheduled wall time lands preferentially in the remainder while deflating every self-time share, so **CPU starvation is itself an unbracketed candidate biasing in exactly the direction observed**. (ii) The claim that GC pause time is unmeasurable is **wrong at the measurement point**: only the library assemblies are netstandard2.0; the probe harness lives in `tests/SIL.Machine.Morphology.HermitCrab.Tests`, which targets **net10.0**, so `GC.GetTotalPauseDuration()` and `GC.GetGCMemoryInfo().PauseTimePercentage` are directly callable. Next probe, gated: re-run serial — remainder <20% means contention was the missing mass and 17(a) recloses on the cheaper-clone target; still >=30% then bracket with `GetTotalPauseDuration()` deltas — ΔPause >=50% of the remainder's absolute ms confirms GC (build target converges with clone/freeze at 46.7%), <=15% refutes it and the next split is enumerator/LINQ orchestration (the 2026-07 alloc audit already flagged `ExecuteCommands` enumerator boxing); a collection census over 24 Amharic words gives gen0 8,217 / gen1 2,729 / gen2 301, an unusually high gen1+gen2 ratio consistent with — not proof of — GC as the remainder. (b) **Clone-discard fraction = 66.2%** (created 2,505, discarded 1,659), a count and therefore contention-immune, against a >=70% gate: **misses by ~4 points, so the build is "make a clone cheaper", not "defer cloning"**. Scoped to `AnalysisAffixTemplateRule`'s eager per-template clone, the one clone-before-verification site; the other four analysis-side sites clone only after a successful match and were excluded rather than blended in | | 18 | Corpus-scope memoization | `AnalysisScope` dies per word, but `AnalysisStateKey` is word-independent by construction, so entries are reusable across a corpus. Nogood hits dominate positive replays 10-17x, making cross-word nogood reuse the cheapest available form of reuse | **CLOSED, and worse than a no-op.** Indonesian, 121 words, **serial on a clean machine: 1.015x, 1.4% wall-clock reduction** against a >=20% build gate and <5% close gate. Costs **68 MB** added peak managed memory. Cross-word reuse is real but small: 39 memo, 73 nogood, 237 template-nogood hits. Token-vs-type: **121 tokens, 121 distinct**, so no type-level dedupe is available either. **A first, contended run of the same harness reported 2.385x / 58.1%** with a 210.9% spread on the shared arm against 13.7% on the baseline; two testhosts were live. The counts were byte-identical across both runs — only the timing was contaminated, and min-of-N on a high-spread arm picks the luckiest sample. Analysis-set equality held throughout, so the key genuinely is word-independent. **Sena, first 300 words, is the decisive case: the shared scope is 28-32% *slower* than per-word scopes** (two clean serial runs, no contention: 282.7s vs 372.2s and 308.9s vs 396.7s baseline-vs-shared; deterministic counters byte-identical across both runs), and costs **~29.3 GB** added peak managed memory against a 2 GB budget. Root cause: `AnalysisScope.MaxMemoEntries` (100,000) is a *per-scope*, not per-word, budget. Two heavy early words (#74, #76 of 300) alone fill the shared `Memo` table to the cap and `TemplateMemo` to 55,071; every word after that loses the free within-word memoization a fresh per-word scope would have given it for nothing, because `Store` silently no-ops past the cap (see its comment) with no eviction. Cross-word hits (shared minus baseline) are **negative** — memo -77,648, nogood -747,607 — i.e. sharing produced *fewer* total hits than 300 independent fresh-table runs would have on their own; the small `+820` template-hit / `+9` template-nogood gain does not come close to covering it. Amharic, capped at 28 words (word 29 hangs, excluded per standing guidance): **0.9% wall-clock reduction**, table nowhere near the cap (Memo 1,251, TemplateMemo 652 of 100,000), consistent with row 17's finding that Amharic's cost is dominated by clone/allocation per node rather than distinct-state count, so avoiding a handful of cross-word re-derivations barely moves wall time. Token-vs-type on all three corpora, at the runs' own sizes and on each corpus's full word list: **zero duplicate tokens** — whole-word caching is not available as a cheaper alternative anywhere here. No analysis-set divergence on any corpus, at any scope regime: the stop condition that would have put rows 1-3 in question did not fire. **A shared-but-capped memo table is not "the same optimization, smaller ceiling" — past the cap it actively takes away the free per-word memoization the product already has, and retains every live entry for the run's full duration instead of per-word, which is where the 29 GB comes from.** Any future attempt needs real eviction (LRU, generational, or per-word sub-budgets) before this is worth re-measuring, and even then the >=20% gate looks unreachable given Indonesian's ceiling under an uncapped-in-practice regime was already only 1.4% | @@ -273,11 +275,12 @@ with the code: `docs/hermitcrab-perf-2026-09-counts.md` and one file per change `Word.Clone` deep-copied the Shape and every node feature structure. The follow-up independently implements a different design: internal `CloneForEngine` copy-on-write with un-share at mutation sites, while public `Word.Clone` remains a deep mutable clone. A shared shape is frozen, so a missed mutation site throws. -- **Row 22's ~10% cap for an analysis-rule prefilter was wrong by an order of magnitude.** It was - computed from matcher *self-time*. Counted directly, an edge-segment prefilter (patterns are anchored - at both ends and matched by unification, so the first/last literal segment must unify with the - shape's first/last segment) removes **87% of FST traversals** and 54% of arc checks on 45 Sena words, - with identical analyses on every Sena word and all 464 conformance-fixture words. +- **Row 22's ~10% cap for an analysis-rule prefilter was wrong by an order of magnitude.** It was computed + from matcher *self-time*. Counted directly, an edge-segment prefilter (patterns are anchored at both ends + and matched by unification, so the first/last literal segment must unify with the shape's first/last + segment) removes **87% of FST traversals** and 54% of arc checks on 45 Sena words, with identical analysis + signatures on every Sena word (morph counts only on Sena; see the disposition doc) and all 464 + conformance-fixture words. - **"memo-key construction 0.0%" (row 21) was wrong at the profile level.** Hashing each new word's syntactic feature structure for `AnalysisStateKey` is ~15% of CPU on the heaviest Sena word; the key itself is cheap but freezing a fresh clone of a nested feature structure is not. From 806793672781a112232ec6f3210798c1e080b354 Mon Sep 17 00:00:00 2001 From: John Lambert Date: Wed, 21 Oct 2026 21:40:44 -0400 Subject: [PATCH 7/7] docs: record the 2026-10-21 search-cost round (rows 24-30) Two port bugs found by diffing HC-Rust against the oracle --- ...hermitcrab-2026-10-21-search-cost-round.md | 377 ++++++++++++++++++ docs/hermitcrab-optimization-ledger.md | 25 +- 2 files changed, 400 insertions(+), 2 deletions(-) create mode 100644 docs/hermitcrab-2026-10-21-search-cost-round.md diff --git a/docs/hermitcrab-2026-10-21-search-cost-round.md b/docs/hermitcrab-2026-10-21-search-cost-round.md new file mode 100644 index 000000000..3522a8a74 --- /dev/null +++ b/docs/hermitcrab-2026-10-21-search-cost-round.md @@ -0,0 +1,377 @@ +# HermitCrab search cost, 2026-10-21 round + +**Disposition: three large reductions in search work stand up. Two of them were bugs in the Rust +port, not algorithmic gains.** HC-Rust ([sillsdev/PanGloss](https://github.com/sillsdev/PanGloss)) +is a port of `SIL.Machine.Morphology.HermitCrab` under test against `hc.dll`. On one private +grammar, HC-Rust did far more search than C# because two kinds of shape node had lost the segment +identity that C# gives them. Fixing both made HC-Rust match C# and removed most of that grammar's +cost. The one algorithmic change that survived, sharing trailing affix-template slot batches, was +built in HC-Rust and then ported to C# as [Machine PR #546](https://github.com/sillsdev/machine/pull/546). +The lead ranked first in the [search-asymptotics research](hermitcrab-search-asymptotics-research.md), +lexically-dead branch pruning, was measured first and rejected on its ceiling. + +This record follows the [optimization ledger](hermitcrab-optimization-ledger.md): one entry per +attempt, the number that settled it, and what differs from earlier attempts. Ledger rows 24-30 +summarise it. + +## Data and units + +**Grammars.** The public reference grammars are Sena, Amharic, Mbugwe, Aweti and Indonesian. The +round's motivating grammar was a **private featureless project (Austronesian, ~30k lexical entries, +3 strata)**. It was measured in three variants: *base*, *v1*, and a *deletion variant* that adds a +rule deleting two stop classes after nasals and has no compounding. Each variant was run on three +word lists: 200 nonwords, a 563-word dev list and a 7,206-word held list. Only aggregate numbers +from it appear here; no word forms, glosses or lexical entries. + +**Units.** The owner rule for this round is that gains are stated as **parser work**: "N% less work +to parse the words". +- **Parser work** is HC-Rust's per-word `work_steps`, summed over a word list. A work unit is one + reservation against HC-Rust's per-word work cap. Every analysis attempt reserves one, and so does + every inner matcher step, segmentation step, root-lookup step and confirmation step. It is + deterministic, so it survives machine contention, unlike wall time. +- **Steps** are analysis attempts (`steps <= work_steps`). They are a secondary figure. +- **Milliseconds** appear only as labelled secondary figures. +- **C# figures** are in C#'s own counters (slot batches, lookups), never in HC-Rust work units. They + are not converted. + +**Parity comparators.** HC-Rust used `rust/tools/parse_compare.py`, which compares complete +analyses per word and reports each row IDENTICAL or not. Entries 090 and 103 ran it with +`--multiset`; entry 095 records only `parse_compare.py`. PR #546 compared ordered morpheme-ID/surface signature lists, +ordinal-sorted, with multiplicity. Neither is the legacy morph-count `WordAnalysisSignature` that the +September disposition calls out. + +**Runs.** Unless noted, HC-Rust runs were `pangloss batch --threads 1` with `HC_STEP_STATS=1`. +Change A's full-list parity runs used `--threads 2` with the default step cap and no wall timeout. + +## What stands up + +| # | Change | Kind | Where | Effect, in parser work | Parses | +|---|---|---|---|---|---| +| A | Analysis-side affix literals keep `StrRep` identity on wide featureless tables | HC-Rust port bug. C# already exact | PanGloss divergence entry 090 (analysis-side `Lhs` follow-up) | Private project, 7 spot words: base **98.5% less** (2,365,602 -> 35,913); worst variant **99.8% less** (50,778,583 -> 83,677). Steps on the full lists fell 46-55x | IDENTICAL on all three private lists and on Amharic, Sena, Indonesian | +| B | A re-inserted segment-class node keeps its members on a featureless grammar | HC-Rust port bug. C# already exact | PanGloss entry 103, [`414ac6b5`](https://github.com/sillsdev/PanGloss/commit/414ac6b5), [record](https://github.com/sillsdev/PanGloss/blob/main/docs/divergences/103-class-deletion-reinsertion-identity.md) | Deletion variant, held list: **94.7% less** (303,966,311 -> 16,220,190). Its 16 words that hit a 1 s limit: **99.9% less** | IDENTICAL on every list. Byte-identical work wherever it cannot fire | +| C | Shared trailing slot trie in the affix-template battery | Algorithmic, no memo | PanGloss entry 095, [record](https://github.com/sillsdev/PanGloss/blob/main/docs/divergences/095-shared-template-slot-batches.md); C# [PR #546](https://github.com/sillsdev/machine/pull/546) | Sena **11.5% less** (all 6,146 words); Mbugwe **18.1% less** (1,364-word partial run); Aweti 0.35% less; zero elsewhere | IDENTICAL in HC-Rust; zero signature differences in C# samples | + +The census instrumentation that settled the rejected lead is +[`d1690c13`](https://github.com/sillsdev/PanGloss/commit/d1690c13). It is gated behind +`HC_BRANCH_CENSUS=1` and inert when unset: parser work and parses are identical to main on six +lists. + +## The lesson: diff the port on the oracle's own export before optimizing the algorithm + +The bugs were found by a comparison, not by a profile. HC-Rust normally loads a FieldWorks project +through its own snapshot reader. Here it was also run on the XML that C# itself exports for the same +project. On the private project's slowest word: + +| Engine and input | Lexical lookups | +|---|---| +| C#, its own export | 49 | +| HC-Rust, C#'s export | 18 | +| HC-Rust, its own snapshot | 1,655 | + +These counts were supplied by the round's brief; this write-up did not reproduce them. A ~90x gap +between two inputs to the same engine is a loading or compilation difference, not a search-strategy +difference. Following it led to both identity bugs: + +- **A (entry 090 follow-up).** HC-Rust's analysis-side affix literals carried identity only through + a single 64-bit id lane. That lane does not exist above 64 character definitions. Provisional + letters pushed the private project to 70, and the grammar has no phonological features, so + nothing else distinguished segments. An enclitic literal therefore matched *any* final segment. + Every word long enough was unapplied against it, and synthesis rejected each bogus candidate. C# + gives a featureless segment a `StrRep` `EqualTo` constraint (`CharacterDefinitionTable.cs:68` at +`a4b29742`, the same snapshot the research doc's anchors use), so + its literal matches only itself. The fix applies HC-Rust's chunked `StrRep` matcher on this path, + only for featureless grammars. Featured wide tables (Amharic) keep the feature-only + over-approximation, which C# shares, because a featured node has no `StrRep`. +- **B (entry 103).** `NarrowAnalysisRewriteRuleSpec.Unapply` re-inserts a deletion rule's LHS + constraint as an optional node. For a `SegmentNaturalClass` that node carries the union of its + members' structures, which on a featureless grammar is "`StrRep` in members", and + `RootAllomorphTrie` unifies against it. HC-Rust gave the node no identity, and its root trie also + ignored a query node's member set. Analysis and root lookup could therefore take any segment at + every re-insertion site. The fix sets the member set on featureless grammars only, and the trie + refuses an edge only when C#'s unification would fail. + +**Neither fix is a C# optimization.** Before the fixes HC-Rust did more work than C#; afterwards it +does what C# does. They are recorded here because they decided where this round's effort went, and +because they explain several figures in the research brief: +- **The deletion variant's 1 s timeouts.** The brief reported 200-1,600 inputs reaching a 1 s + per-input timeout, from an earlier build that had neither fix; that count was not re-measured. + What was measured is narrower: after fix A and before fix B, 16 held-list words still hit a 1 s + limit, and fix B cut their parser work by 99.9%. Both port bugs are implicated, and the two counts + are not comparable. +- **Where the time went.** In a two-word profile of the deletion variant before fix B, analysis was + 0.1-0.3% of parser work, root lookup about 10% and synthesis about 90%. Zero branches were + lexically dead. No analysis-side pruning could have helped. + +One residual is **not explained**. On the slowest word, HC-Rust on C#'s export made 18 lookups where +C# made 49. Parses were identical, but an efficiency nobody can argue for is not a result. It is +listed under open leads rather than claimed. + +**Method rule.** Before measuring an algorithmic lead on a port, run the port on the oracle's own +export of the same grammar and compare a cheap count, such as lookups per word. A large gap between +the port's two inputs is a port bug, and its fix can dwarf any algorithmic change. This round, it +did so twice out of three. + +## C1: prune analysis branches that cannot reach a root (research lead #1) — rejected on its ceiling + +**Expected.** A sound summary of the remaining transformations would prune analysis nodes from +which no lexical lookup can find a root. This is the research doc's rank-1 candidate, residual +viability. It differs from ledger rows 10 and 11 only if compounding and the remaining morphology +are modelled. + +**Measured.** A hindsight census charged each unapplication's steps and work to its input node. +Folded alternatives alias their canonical. A node is *live* when a lookup on it, on an alias or on +a descendant found a root. Dead cost is split three ways: +- **Dead leaves:** a failing rule battery plus one failing lookup. +- **Dead, may compound:** fewer non-heads than `MaxStemCount`, with a compounding rule in this or a + deeper stratum. +- **Dead internal, plain:** the residual. + +Only the plain bucket is what a sound, compound-aware summary could actually remove at a profit. A +dead leaf costs about as much to prove dead as to run. A may-compound node is row 10's +disqualifier. + +| List (words) | Parser work | Analysis / lookup / synthesis | Lexically-dead ceiling | Dead leaves | Dead, may compound | **Dead internal, plain** | Repeat-lookup work | +|---|---|---|---|---|---|---|---| +| Private, base, held (7,206) | 50,345,810 | 52.7 / 11.8 / 35.2% | 22.6% | 6.1% | 14.6% | **1.9%** | 2.4% | +| Private, v1, held (7,206) | 105,768,202 | 70.5 / 6.7 / 22.6% | 38.7% | 19.9% | 6.6% | **12.2%** | 1.8% | +| Private, deletion variant, held (7,206), before fix B | 303,966,311 | 8.5 / 35.7 / 55.8% | 16.0% | 12.7% | 0.0% | **3.4%** | 10.4% | +| Indonesian (70) | 1,906,012 | 85.8 / 2.1 / 12.0% | 41.0% | 15.3% | 23.8% | **1.9%** | 0.7% | +| Amharic (669 completed) | 39,395,506 | 97.2 / 1.4 / 1.4% | 66.1% | 20.3% | 45.8% | **0.0%** | 0.6% | +| Sena (5,134 completed at 30 s) | 271,894,826 | 89.9 / 3.0 / 7.1% | 55.5% | 11.9% | 39.0% | **4.7%** | 2.2% | +| Mbugwe, first 400 (394 completed) | 610,569,774 | 73.6 / 3.3 / 23.0% | 45.5% | 9.7% | 35.8% | **0.0%** | 3.3% | +| Aweti (183 of 206 completed at 30 s) | 488,089,456 | 93.6 / 1.0 / 5.3% | 74.9% | 35.1% | 0.0% | **39.8%** | 1.0% | + +The census builds include fix A but predate change C. Template-heavy shares (Sena, and possibly +Amharic and Mbugwe) are therefore against an older, larger denominator. For scale, Sena was 271.9M +work units on 5,134 completed words then, and is 240.7M on all 6,146 now. + +**Verdict: not built.** +- **Plain bucket.** It is 0-5% of parser work on six of eight lists. The exceptions are v1 at 12.2% + and Aweti at 39.8%, the latter on 183 completed words. +- **Most dead work is unprunable.** It sits in leaves, or in nodes that could still split off a + compound non-head. Amharic, Sena and Mbugwe put 36-46% of parser work there. +- **Any sound summary has a large scope.** It must model reduplication, infixes, truncation, + deletion re-insertion and compounding before it is allowed to prune anything. + +**What is different from prior attempts.** +- **Row 11 ceiling.** Row 11 found 23.5% pooled dead steps. Dead *steps* here are 43-94% of steps, + so dead branches are common. But they are cheap: the ceiling falls once it is weighted by work + and the unprunable buckets are removed. +- **Row 11's other failure mode persists:** the root is reachable but synthesis fails. On the + deletion variant before fix B, 86% of synthesis work went to canonicals whose lookups hit but + never matched the surface. On Mbugwe the share is 98%. No lexical oracle can prune either. + +**C# transfer.** C#'s `Morpher`, `AnalysisStratumRule` and `AnalysisRewriteRule` run the same +search, so the same buckets should apply. No Machine PR unless the shadow experiment under open +leads shows a sound summary deciding a meaningful share of the plain bucket. + +### Per-parse distinct-shape lookup — rejected + +The brief's "97,575 lookups over 56 shapes" came from trace mode, where analysis merging is off. The +research doc warned that a trace-mode count cannot establish an untraced duplicate rate, and the +untraced census confirms it: +- **Calls per key.** The private project makes 1.13-1.49 lookup calls per distinct + (stratum, shape). +- **Ceiling.** Exact repeat-lookup work is 0.5-3.3% of parser work on every list, or 10.4% on the + deletion variant before fix B. +- **Policy.** Avoiding it would be a per-parse memo, which this work excludes. + +C#'s `LexicalLookup` repeats lookups the same way, so there is no Machine PR. Dead-candidate lookups +are 30-75% of lookup work, but lookup is only 1-12% of parser work, so they were not pursued either. + +## C2: share trailing slot batches inside one battery call (research lead #2) — kept, ported to C# + +**Expected.** The battery walks each template's slots last-first. Templates whose trailing slots +use the same rules, in the same order, with the same optionality, run the same slot batch on the +same word. Factoring the templates into a trie runs that batch once. This is the research doc's +rank-2 candidate. It is control factoring, not a cache: nothing is keyed by a visited word, and +nothing outlives one battery call. + +**Battery share of all parser work (HC-Rust, before C):** + +| Grammar | Battery share | +|---|---| +| Mbugwe | 61% (86 step-capped words are about 2/3 of the total) | +| Private project, v1, held | 45% | +| Sena | 23% | +| Amharic | 21% | +| Private project, base | 8% | +| Private project, loaded from C#'s export | 9% | +| Private project, deletion variant | 6.5% | +| Indonesian | 0 (no templates) | + +**Static census.** This counts rule applications on the battery's own input, before and after +sharing: +- **Shared tails:** Sena 111 -> 45, Mbugwe 397 -> 196, Aweti 143 -> 100. +- **No sharing possible:** Amharic 50 -> 50, and the private project 30 -> 30. Each has one template. + +**Dynamic ceiling, Sena.** A trie-exact key (same trailing slots and same full word state) predicts +448,573 repeated slot batches, which is 30.8M work units or 11.3% of all Sena work. The built trie +avoided exactly 448,573. + +### HC-Rust (PanGloss entry 095) + +Run with `--threads 1 --step-cap 2000000` (Aweti 500000) and no wall timeout. + +| Grammar | Words compared | Parses | Parser work | Steps | Fire count | +|---|---|---|---|---|---| +| Sena | 6,146 | IDENTICAL | **11.5% less** | 31.6% fewer | 171,319 shared, 448,573 avoided (= predicted ceiling) | +| Mbugwe | 1,364 (partial run) | IDENTICAL | **18.1% less** | 19.3% fewer | - | +| Aweti | 170 | IDENTICAL | 0.35% less | 0.16% fewer | 1,939 shared, 2,159 avoided | +| Private project (4 grammar variants: base, v1, deletion, C#'s export; x 3 lists) | all | IDENTICAL | 0 | 0 | 0 | +| Amharic, Indonesian | all | IDENTICAL | 0 | 0 | 0 | + +**Tests.** +- **Differential test.** Compares the shared walk with each template's own walk for all words up to + 5 segments. It fails when optionality is dropped from the trie key. +- **Fire-count test.** Fails with sharing disabled. +- **Conformance.** `-Scope all` passed 2043/2043. Fixtures at the PR #480 head showed the same + failures as the baseline. + +### C# (Machine PR #546) + +`AnalysisAffixTemplateRule` has the same shape, so the trie ported directly to sequential +`ApplySlots`. It is restricted to untraced, sequential walks with no alternative limit. Tracing, +`ParallelApplySlots`, finite alternative limits, custom compiled templates, changed slot +optionality, and grammars without shared tails all keep the original battery. +`AnalysisScope.TemplateMemo` (row 2) still wraps the battery unchanged: row 2 reduces how often a +battery runs, and #546 reduces repeats inside one run, so the two compose. + +The figures below are C# slot batches on deterministic 200-row samples, single-threaded and warmed, +with mode order alternated per input: + +| Input | Completed pairs | Slot batches before -> after | Fewer batches | Parse time, secondary (ms under concurrent load) | +|---|---|---|---|---| +| Synthetic shared-tail fixture (10 words x 100) | 1,000 | 2,000 -> 1,000 | 50.0% | 1.27 s -> 1.05 s | +| Sena (200 sampled rows) | 172 | 47,072 -> 32,287 | 31.4% | 20.07 s -> 16.67 s | +| Mbugwe (200 sampled rows; 61 pairs capped at 2 s in either mode, excluded) | 139 | 72,467 -> 58,486 | 19.3% | 58.43 s -> 51.33 s | + +- **Parity in the samples.** Every completed pair had identical signature lists, with zero + differences and zero errors. +- **Differential test.** `SharedWalkMatchesPerTemplateWalkInOrder` compares ordered `Word` equality + and complete `AnalysisStateKey` sequences across four grammar arrangements and 2,904 input/state + combinations. +- **Fire test.** `UntracedBatteryRunsSharedSlotBatchOnce` fails with sharing disabled (`Expected: 1; + But was: 2`). +- **Unit tests.** HermitCrab passed 126/126. +- **Conformance.** The PR #480-head fixtures were run on the branch assembly and on the unchanged + base `aab237b2`. With memoization on, both were 58 passed / 2 failed. With memoization off, the + branch was 58/2 and the base 59/1. Both assemblies fail traced-rule attribution in + `boundary-leftover-in-narrowing-lhs-strrep`. `deep-optional-affix-nesting` crossed its 15 s budget + in three runs; it has no shared slot tails and takes the original walk. + +These two engines measure different things, so do not compare the columns. C#'s "31.4% fewer slot +batches" on Sena and HC-Rust's "31.6% fewer steps" are not the same unit, and neither is HC-Rust's +11.5% less parser work. + +**Conformance pins.** The C# 60/60 self-check below is `hc.dll` at `72dbd4f9`, the PR #480 head. The +58/60 above is PR #546's base `aab237b2`. They are different assemblies, and the two counts are not +in conflict. + +**What is different from prior attempts.** +- **Row 2.** The battery memo reduces battery *invocations*. C reduces repeated slot batches + *within one invocation*, with no `(Word, state)` table. +- **Row 18.** C is not a cross-word cache. +- **The research doc's prediction holds.** It said this would not turn 43,641 battery calls into + 4,988. It does not; it shrinks each call. + +**Rejected alongside.** A rule-level memo per battery call was no bigger than slot-level sharing on +Sena (33.7M vs 32.9M work units of repeats), and it is a memo. + +## Conformance state at the round's close + +The run used the PR #480 head, Machine `72dbd4f9`. + +- **C# self-check.** `hc-conformance --include-pathological` passed **60/60 fixtures**: 609 entries + in the `words.yaml` files, none skipped. Upstream fixtures grew from 44 to 60 since the previous + pin; nothing was removed. +- **HC-Rust.** One fixture failed: `edge-cases/clitic-rule-designation`, 5 words. Machine PR #529 + added `MorphologicalRule@isCliticRule`, which HC-Rust has not yet ported. It is a parity gap, not a + divergence, and the port is under way. Every other new or modified fixture passes in both engines. + No fixture passes in HC-Rust and fails in C#. + +## How today's measurements bear on the six research reports + +The integrated ranking is in the [research doc](hermitcrab-search-asymptotics-research.md). This +section records only what changed. + +| Lead (report) | Status after this round | +|---|---| +| **1. Residual viability**: abstract interpretation with staged refinement (R6), lazy residual-lexicon product (R3), magic sets / goal propagation (R4), extension oracle (R5) | **Ceiling measured, rejected for now** (C1). The plain dead-internal bucket is 0-5% on six of eight lists. Reopens only through the Aweti/v1 shadow experiment below | +| **2. Template battery as a shared control trie** (derived from R2/R3) | **Built, kept, ported** (C; Machine PR #546) | +| **3. Trace-class POR with recovery**: Mazurkiewicz traces (R1), sleep/source-set DPOR (R1), critical-pair certificates (R1); verified POR (R6) | Not pursued. No new measurement. The earlier 0.00-0.05% eligibility still stands as the warning | +| **4. Exact finite-state compilation of a verified subgrammar** (R3, R6, R2 #3) | Not pursued as an exact engine. A narrower **FST-derived precheck** is the best remaining lead for single-template grammars (open leads) | +| Fixed-cap root-result cache (R5 #2; R4 aside) | **Rejected**: repeat-lookup work is 0.5-3.3% of parser work untraced, and the cache is a memo. The 97,575/56 premise was a trace-mode artefact | +| Count-vector chart DP with packed provenance (R2 #1); packed trail forest for synthesis (R2 #2); `AnalysisStateKey` graph with ZDD trails (R4 #1); analysis-DAG x synthesis-state product (R5 #1) | Not pursued. They are search-growing tables, a closed family. The finding that 86-98% of synthesis work is root-reachable failure is consistent with the reports' own warning that one output analysis does not bound product states | +| Proven rule-precedence DAG (R4 #2) | Not pursued. It needs complete-path certificates (research doc section 1) | +| Symmetry reduction (R1 #4) | Not pursued. The reports themselves rated it a probable no-fit | +| Meet-in-the-middle (R3, R4), A*, k-best and k-shortest paths, reverse search (R4, R5), Earley/CKY, scalar semirings (R2), e-graphs (R4), bimachines, two-level morphology, compile-replace (R3) | Still not recommended. Unchanged | + +**Prediction scored.** The research doc said the trace-mode lookup ratio could not establish an +untraced duplicate rate. That was right: 0.5-3.3%. + +**The finding the reports could not have made.** Where it was measured, much of the private +project's cost was two port bugs: 98.5-99.8% less work on the spot words, and 94.7% less on the +deletion variant's held list. No ranked lead addressed that, because each one assumed the port's +search matched the oracle's. It is not the whole story: with fix A in place, v1's held list still +costs 105.8M work units, 45% of it in a one-template battery that C cannot touch (open lead 1). + +## Open leads, not pursued + +Each lead is tagged **[C# analogue]** where the same question applies to `hc.dll`, and +**[HC-Rust]** where it concerns the port only. + +**Work reductions, with a ceiling to measure first:** +1. **FST-derived precheck before slot-rule traversal, for single-template grammars** [C# analogue]. + C cannot help a grammar with one template, and on the private project's v1 variant the battery + is still 45% of parser work. A necessary condition derived from each slot rule's compiled matcher + could reject a rule before traversal begins. This differs from row 22's edge-segment prefilter in + being judged in work units, since a traversal charges per arc, and in being derived from the + whole matcher rather than its first and last segments. Its ceiling, the work spent in + zero-match traversals, has not been measured. +2. **Shadow summary experiment on Aweti and v1** [C# analogue]. These are the only lists whose + plain dead-internal bucket is material (39.8% and 12.2%). Run a sound summary in shadow mode and + count the plain-bucket work it would actually decide, along with the summary's own cost. Close + lead 1 if the decided share is small. +3. **Mbugwe's step-capped words** [C# analogue]. 86 of 1,645 words hit the 2M step cap and account + for about 2/3 of Mbugwe's work. They have not been examined individually. +4. **Root-reachable synthesis failure** [C# analogue]. 86% (deletion variant, before fix B) and 98% + (Mbugwe) of synthesis work goes to candidates whose lookup hits but whose synthesis never matches + the surface. No lexical oracle reaches this. It is the largest unaddressed bucket on Mbugwe. + +**C# port follow-ons to #546** [C#]: + +5. **`ParallelApplySlots`** needs its own design for its layered breadth-first queue. +6. **Share compiled slot rules across templates**, in addition to slot batches. +7. **Measure in a deterministic work counter.** The C# gain is measured in slot batches on samples, + not in a work counter on full lists. + +**Time-only costs (no work units charged, so outside the owner metric):** +8. **`ana_allomorph_matches`** deep-clones segments with `segs.to_vec()` and rebuilds + `format!("p{i}")` part names on every allomorph call [HC-Rust]. About 5.1M segment-vector builds + are avoidable per private-v1 held run: 5.48M slot-rule attempts against 365K slot batches. + Hoisting `segs_of` or borrowing segments in `Transduce::new` removes them. +9. **`root_trie::search_segs_ident`** checks its active set with `Vec::contains`, which is quadratic + [HC-Rust]. A bitset would be a pure time win. + +**Identity over-approximations left in HC-Rust** [HC-Rust]. Synthesis re-validates all of them, so +none can change a parse, but each over-explores relative to C# on some grammar: + +10. An `InsertContext` (natural-class insertion) in analysis-side `Lhs` still carries no identity + above 64 character definitions (entry 090's remainder). +11. Narrowing leftovers from `ana_narrow_general` keep no class identity (entry 103's remainder). No + measured grammar exercises a class LHS there. +12. `union_narrowing_node` checks only `char_def`, so it ignores an LHS class node's member set. +13. RHS class nodes in synthesis (`syn_narrow`, `syn_epenthesis`) lack identity on featureless + grammars. They were left alone because they affect rendering. + +**Unexplained observations:** +14. **The lookup gap** [HC-Rust vs C#]. HC-Rust on C#'s export made 18 lookups on the slowest word + against C#'s 49. Parses were identical; the cause was not investigated. +15. **Sena SKIPPED rows** [HC-Rust]. A census run with a 30 s wall limit reported 1,012 SKIPPED + rows, while no-limit A/B runs completed all 6,146 words in about 50 s. Not investigated. + +## Data handling + +Private-project word lists, per-word logs and exports stay on the measuring machine. This record +carries only aggregate counts. Conformance fixtures are synthetic. diff --git a/docs/hermitcrab-optimization-ledger.md b/docs/hermitcrab-optimization-ledger.md index 4042f16ad..4582360a2 100644 --- a/docs/hermitcrab-optimization-ledger.md +++ b/docs/hermitcrab-optimization-ledger.md @@ -35,8 +35,8 @@ from a branch is cheap; maintaining research scaffolding in two product assembli checks their code claims, and separates new experiments from this ledger's closed families. It ranks residual lexical viability, a grammar-compiled template battery, recoverable trace-class reduction, and exact finite-state compilation for a verified subgrammar. It maps them to lexicon-driven pruning during -unapplication and making the affix-template battery cheaper without memoization. **No new measurement, -speedup, or numbered optimization row is claimed.** +unapplication and making the affix-template battery cheaper without memoization. The research doc itself claims no measurement. Its first two candidates were measured on +2026-10-21 (rows 27 and 26): see [the search-cost round](hermitcrab-2026-10-21-search-cost-round.md). ## Final disposition of the September follow-up @@ -117,6 +117,27 @@ copy-on-write design through `CloneForEngine`; see `docs/optimizations/shape-sha | 19 | Contexted constraints / abstract feature-only replay | Pack without merging states by replaying only shape-independent constraints. | **CLOSED.** Cost-weighted P1b die-point histograms put shape-free work at **6.0%** and **0.0%** on two heavy Sena probes and **0.4%** on 28 Amharic words; all fire the <30% close gate. All 33 conformance fixtures were measured for breadth: 24 close, 3 nominally fire the build gate, 3 are undecided, 2 have zero rejections, and 1 has zero words. The three build-gate fixtures are all sub-50 ms and cannot carry timing claims. One reliable fixture confirms that count and cost can diverge, but die-point rejection cost itself is only ~0.1–0.2% of wall on the real corpora. Abstract replay would optimize a phase that costs approximately nothing on the corpora treated as ground truth. Not built. | | 20 | **Row 5's family on `GenerateWords`** (generation, ~100% synthesis share vs parsing's ~5%/~0.3%) | Row 20 was unmeasured; generation's share is ~100% so row 5's fold ratios would apply at face value if they transferred. `UseSynthesisFoldMemo` had never been wired into `GenerateWords` (only into the parse-then-synthesize path) — wired in under the same untraced/`MaxDegreeOfParallelism==1` restriction to make this measurable at all | Hits are no longer uniformly zero: 3 of 23 lookup-reaching fixtures clear the 5% gate (diacritic-segments 33.3%, suffixing-vowel-harmony 28.6%, suffixing-evidential-adjacency-chain 37.7%; a 4th sits just under it at 4.8%), reopening row 5 narrowly for generation. But the reopened case recloses immediately: the A/B on the best fixture shows the wall-clock "win" sitting inside its own noise floor, and the pooled hit rate is dominated by `deep-optional-affix-nesting` (925 of 1,613 lookups) at hits=0, reproducing row 5's exact 4.5%-slower key-construction cost on the identical fixture. Parity: 0 divergences, 1,236 generations, 32/33 fixtures (33rd throws identically both sides). Best case: 130 hits / 5 reps (37.7% hit rate), **7.8%** wall-clock reduction inside a **22.2%** off-arm spread — not a result. 19/23 lookup-reaching fixtures at exactly **0%**; pooled hit rate **2.67%** (43/1,613), skewed by the one large zero-hit fixture. | +## 2026-10-21 search-cost round + +Full record: [hermitcrab-2026-10-21-search-cost-round.md](hermitcrab-2026-10-21-search-cost-round.md). +The unit is HC-Rust parser work (summed per-word `work_steps`, deterministic), so "N% less work to +parse the words". Steps are secondary, and ms are labelled. C# figures are in C#'s own counters. +Rows 24 and 25 are **HC-Rust port bugs: C# was already exact, and no C# change follows from them.** + +| # | Optimization | Expected | Outcome | Number | +| --- | --- | --- | --- | --- | +| 24 | **Analysis-side affix literals keep `StrRep` identity on wide featureless tables** (HC-Rust only; PanGloss divergence 090 follow-up) | Running HC-Rust on C#'s own export of a private featureless project made 18 lookups on its slowest word, against 1,655 on HC-Rust's own snapshot (C#: 49). That points at a loading difference, not search | **KEPT in HC-Rust; no C# change.** Above 64 character definitions HC-Rust lost the literal's identity, so an enclitic matched any final segment. C# gives featureless segments a `StrRep` `EqualTo` constraint (`CharacterDefinitionTable.cs:68`) | 7 spot words: **98.5% less work** (2,365,602 -> 35,913); worst variant **99.8% less** (50,778,583 -> 83,677). Steps on full lists fell 46-55x. Parses IDENTICAL on three private lists and on Amharic, Sena and Indonesian | +| 25 | **Re-inserted segment-class node keeps its members on featureless grammars** (HC-Rust only; PanGloss entry 103, `414ac6b5`) | A deletion-rule variant of the same project hit 1 s timeouts. A profile showed analysis at 0.1-0.3% of work, synthesis at ~90%, and zero lexically dead branches | **KEPT in HC-Rust; no C# change.** HC-Rust's re-inserted node matched any segment, and its root trie ignored the member set. C# unifies against the class's `StrRep` union | Held list (7,206 words): **94.7% less work** (303,966,311 -> 16,220,190). The 16 words at the 1 s limit: **99.9% less**. IDENTICAL everywhere; byte-identical work where it cannot fire | +| 26 | **Shared trailing slot trie in the template battery** (research candidate 2; PanGloss 095; Machine **PR #546**) | Templates with identical trailing slots (rules, order, optionality) run the same slot batch on the same word. Sena's trie-exact ceiling is 448,573 batches, 11.3% of work | **KEPT and ported.** Control factoring within one battery call, with no cache. In C# it covers sequential, untraced walks without an alternative limit. It composes with row 2 | HC-Rust: Sena **11.5% less work** (31.6% fewer steps, 448,573 batches avoided = ceiling); Mbugwe 18.1% less (1,364-word partial); Aweti 0.35%; 0 on one-template grammars. C# batches: Sena sample 47,072 -> 32,287 (31.4%); Mbugwe sample 72,467 -> 58,486 (19.3%). C# tests: a 2,904-combination differential, 126/126 HermitCrab tests, conformance 58/60 equal to the unchanged base `aab237b2` with memo on | +| 27 | **Prune analysis branches that cannot reach a root** (research candidate 1; census `d1690c13`) | The hindsight lexically-dead ceiling is 16-75% of parser work | **CLOSED on the prunable share.** Most dead work is leaves, which cost as much to prove dead as to run, or nodes that may still compound (row 10's disqualifier, 36-46% on Amharic, Sena and Mbugwe). Root-reachable synthesis failure is untouched: 86-98% of synthesis work where measured | The plain dead-internal bucket, the only one a sound summary could remove at a profit, is **0-5% on 6 of 8 lists**. The exceptions are v1 at 12.2% and Aweti at 39.8%. **Reopening condition:** a shadow summary on Aweti/v1 deciding a meaningful share of that bucket | +| 28 | **Per-parse distinct-shape lookup dedupe / fixed-cap root-result cache** (research reports R4, R5) | "97,575 lookups over 56 shapes" | **CLOSED.** That figure was trace mode, which disables merging. It is a memo regardless | Untraced, 1.13-1.49 calls per distinct (stratum, shape). Repeat-lookup work is **0.5-3.3%** of parser work (10.4% only on row 25's pre-fix bug). Dead-candidate lookups are 30-75% of lookup work, but lookup is 1-12% of parser work | +| 29 | **Rule-level memo per battery call** | Repeated slot-rule applications inside one battery | **CLOSED.** It is no bigger than row 26 and it is a memo | Sena repeats: 33.7M work units (rule-level) vs 32.9M (slot-level) | +| 30 | **Hoist `segs_of` / borrow segments in `Transduce::new`; bitset for the root-trie active set** (HC-Rust) | Fewer allocations and a less quadratic trie walk | **NOT BUILT.** Time-only, with 0 work units by construction, so outside the owner metric | ~5.1M avoidable segment-vector builds per private-v1 held run (5.48M slot-rule attempts vs 365K slot batches). `Vec::contains` on the trie active set is quadratic | + +**Lesson of the round:** two of its three large reductions (rows 24 and 25) were port bugs, found +by running the port on the oracle's own grammar export and comparing lookup counts. Do that before +optimizing the algorithm. + --- ## The lesson that generalises